Outpatient Health Care Usability Profile Brief: Charles Drum
Charles E. Drum, JD, PhD
The Americans with Disabilities Act (ADA) was passed in 1990 with the belief that it would make lots of different places, including doctor’s offices, accessible for people with disabilities. Under the ADA, state and local governments need to make sure that existing buildings are accessible and that people with disabilities can receive programs and services. Private places such as doctor’s offices and hospitals must be accessible and barriers must be removed in existing facilities. The ADA helped with this by creating accessibility guidelines for buildings and facilities. These guidelines have more information to help make sure places are accessible. The guidelines have over 500 accessibility requirements.
Unfortunately, a lot of medical clinics are still inaccessible to people with disabilities. When health clinics are inaccessible, people with disabilities don’t get the kind of medical care that they need. While there are a few tools that measure the accessibility of clinics, most people haven’t put a lot of testing into them. People shorten the over 500 accessibility items without thinking very much about which ones are the most important and they don’t check to see if a couple of people use it, are they getting the same results. Sometimes they even call them “accessibility surveys” even though they shorten a lot of the accessibility requirements. The purpose of this project was to create a tool that clinic staff and others could use to identify and remove access barriers and improve the usability of health care clinics.
Research Objective/Research Question or Training Goals
Our goal was to develop a checklist called the Outpatient Health Care Usability Profile (OHCUP) that measures the usability of health clinics and is reliable, valid and easy to use. Usability means that people with disabilities can make use of a clinic to receive health care even if the clinic does not meet all the technical requirements of the accessibility guidelines. We wanted to measure the usability of clinics from the parking lot to the exam room and in between.
We used the following steps to create a tool that is valid. Validity means that it measures what it’s supposed to measure (the usability of health clinics). We formed three focus groups of people with mobility, sensory, or cognitive disabilities and asked them to review each of the over 500 accessibility requirements and tell us which were the most important by type of disability. Three focus groups were formed consisting of: ten people with cognitive disabilities, ten people with mobility disabilities and ten people with sensory disabilities. These groups reviewed each of the over 500 accessibility requirements and prioritized the access requirements considering their disability. This process significantly reduced the number of accessibility requirements. We then sent the items to a group of ADA experts and asked them to look at each item and tell us if the item was essential to the usability of a clinic. Content Validity Ratios were calculated for each item (CVR= (ne-(N2)/ (N2)) that further reduced the number of items by 21, to about 160 in the OHCUP. Some apply to people with mobility disabilities, some to people with sensory disabilities, and some to people with cognitive disabilities. We changed the wording of some of the items to make them easier to understand than the accessibility guidelines.
- P1. the required number of designated parking spaces displaying the symbol shown to the right is provided.
- A1. the route of travel does not require the use of stairs.
- RR13. The faucet is operable with a closed fist.
- EE4. Where emergency alarms are provided, they have flashing lights.
To pilot test the OHCUP, we used the following steps to create a tool that is reliable. Reliable means that two or more raters would agree on scoring more often than not and not because of chance. First, we had two people go to ten different health clinics and try out the OHCUP. A range of clinics were chosen based on size and geography in the Portland Metro area. We compared their scores on each item at each clinic to see how often they agreed using Cohen’s Kappa coefficient. Cohen’s Kappa coefficient is a measure of inter-rater reliability that takes into account agreement occurring by chance. K score =1 if raters are in complete agreement and it is not because of chance. The goal is to get as close to 1 as possible for each item. By convention, a Kappa >.70 is considered acceptable inter-rater reliability. The OHCUP Kappa for Round 1 was .61.
Although the two raters agreed most of the time, we changed a number of the items and sent the raters to another ten clinics to re-test the OHCUP. The Kappa for this second round of testing was .77. We also calculated the Alternative Statistic (Agreement Coefficient) which was .90. Kappa results in chance agreement probabilities ranging from .5 to 1. Several of our items where the raters agreed, the Kappa calculations indicated that the probability of that agreement being by chance was 100%. This is not very logical. The alternative chance-corrected statistic, developed by Kilem Gwet, addresses this problem. With this statistic, the calculated probability of chance agreement ranges from 0 to .5, which is much more consistent with reality.
The goal for this statistic is 1. The raters agreed even more often, but we still changed a few more items and sent the raters out to another nine clinics just to test those items.
On the third round of testing, items with low Kappas were re-tested in nine clinics and resulted in a Kappa of .89 and an agreement coefficient of .97. We also tracked the amount of time that it took to test the usability of each clinic with the OHCUP.
The OHCUP is a valid and reliable tool for measuring the usability of health clinics. It takes about 30 minutes or so to measure clinics but the items are pretty easy to understand. In the pilot test, most of the clinics received a “usable” grade on about 80% of the OHCUP items.
This is one of the few tools to evaluate the level of usability of outpatient health clinics based on ADA guidelines. Making clinics more usable should help people with disabilities get needed medical services.
The OHCUP should be a useful tool for disability advocates to take to clinics and use. It comes with clear directions and tools on how to measure usability. Patients with disabilities and disability organizations should be able to go to health providers with hard information on where improvements can be made in the areas that the OHCUP measures.
The OHCUP is a helpful tool for clinic directors and managers as well. It can be self-administered in about 30 minutes and is a concise tool that clinic staff and others will be able to use to identify and remove barriers and improve access to health care services. States might consider using the OHCUP in their licensing programs, as a short-hand way of making clinics more accessible.
The OHCUP is not a substitute for the full accessibility requirements of the ADA. It is also important to remember that the OHCUP does not measure the attitudes of health care providers. Sometimes attitudes can be worse than physical barriers to health care services.
It is very important to do more research on the OHCUP to see what kind of scores clinics receive in different parts of the country.