Developing a Function-Free Measure of Health Status
The introductory presentation outlined the problems with using functional questions to assess health in persons with disabilities. Dr. Hall’s review of existing measures showed varying degrees of functional bias in other measures. Dr. Horner-Johnson’s study indicated that the BRFSS questions on health status measure physical and mental health, while Dr. Andresen’s study showed that these domains of health are generally similar for persons with and without disabilities. Dr. Nosek’s study shows the need to examine gender differences in a measure of health status.
The Expert Panel on Health Measurement has guided the research team through a number of steps in developing a new health status measure. The Institutional Review Board of OHSU provided oversight for all data collection methods with participants.
Definitions and Conceptual Model
Through panel discussion and review of previous literature, the panel developed the following definitions:
Health: A balanced (harmonious, unified, dynamic, complete) state of physical, mental and social well being and not merely the absence of disease.
Function: Physical and mental activities whose performance can be directly affected by an underlying impairment. For the purposes of this study, this definition is intended to directly identify those biases that we want to remove from the measurement of health.
Disability: An umbrella term defined by limitation in any of multiple dimensions related to underlying physical or mental impairments. Dimensions could encompass social roles, specific activities or functions.
Environment: Natural, built and social (e.g., economic, political) context in which we live our lives.
We also adopted a conceptual model of health that included four domains of health, with the possibility of subcomponents within these domains. The conceptual domains were:
Physical health—e.g., sick/well, pain, energy/fatigue, resilience/robustness, stamina/endurance.
Mental health—e.g., distress, mood, self-efficacy, emotional regulation, concentration, confidence.
Social Health—e.g., meaningful relationships, civic/political/social engagement, intimacy, isolation, abuse/neglect, oppression/discrimination.
Living your Values—e.g., spirituality, meaning in life, life satisfaction, employment/vocation, recreation.
Recognizing how important the environment can be in determining health and the experience of disability, we wanted to add a scale on environment that would not be regarded as a measure of health, but would be included to understand how to interpret the findings on health (i.e. potential covariate).
Environment—e.g., safety/security, built environment, transportation, air/water/climate, social policy/attitudes.
The following figure shows the steps we have taken in developing the measure.
- A work-group examined each item from each of the 85 measures that met criteria and agreed on assignment into domains. Identical or highly similar items were eliminated.
- The Expert Panel members rated each item for importance to health. Ratings across members were assessed by CVR process for agreement. The number of items was reduced from 648 to 243.
- Content of items was rewritten into a standard format. Based on review of the literature and Expert Panel discussion, we agreed to phrase questions as referring to the previous 4 weeks, to make sure that each item only asked about one element, and adopted a 7-option rating scale (Never or Almost Never, Not often, Occasionally, Sometimes, Often, Very Often, Always or Almost Always).
- The Expert Panel was now asked to review each item in terms of potential functional bias for persons with physical, communication, mental, or vision/hearing problems. If a panelist objected to an item, they were asked to rewrite the item if possible. Panelists were also asked to add any new items they thought of.
- A smaller working group probed more deeply into how people think about health and what goes into rating their health. Interviewers administered a set of sample items with a group of 10 adults with different disabilities. Using a cognitive appraisal profile, they queried to see that people understood the items, how they understood the items, and what contexts they used to rate their health. This information provided a richness for understanding the complexity of rating, but did not change the number of items or their format
- The remaining items were sent to a Validity Panel. This was a panel of people knowledgeable about specific disabilities from two perspectives—national disability organizations for specific disabilities, and RRTC's working in the areas of physical, mental, vision, hearing and intellectual disabilities. These 10 representatives went through the same process, reviewing each item and tagging those items they thought might be biased or adding areas they thought were overlooked.
- The remaining items were sent to two centers that specialize in cultural competency and disabilities. Their staff provided feedback on questions that might be confusing, offensive or have different meaning for people from culturally and linguistically diverse backgrounds.
- Items identified as problems by the Expert Panel, the Validity Panel, or the cultural competence experts were reworded or eliminated. The remaining 121 health items and 25 environment items were tested with a pre-pilot group to check for understandability of the items.
- We decided to retain all items for the pilot test.
Text of Figure 1: Item Development
- Subgroup Item Pool
- Collate items from relevant HRQOL measures
- Group items into domains
- Combine/revise similar items
- Expert Panel Assess Health (3 batches)
- Assess importance
- Ignore wording
- Ignore redundancy
- Ignore functional bias
- Subgroup Format
- Eliminate redundancy
- Standardize format
- Expert Panel Function
- Functional framework
- Work on wording
- Validity Panel Function
- Functional framework
- Item bias
- Item confusion
- Things overlooked
- Pre-pilot testing 25 (+25 non-disabled)
- Gender, age, education distribution
- Cognitive testing 10+10 Revise Items
- Pilot Test
- 240 healthy (60 mobility; 60 mental health; 60 sensory (30 vision/30 hearing); 60 general population)
- Factor analyses of conceptual model
- Index of discrimination
- Field Test
- 700 adults with disabilities from across the country
- Two-week test – retest – 100
- Intervention sensitivity pre-post – 50
Pilot Test of the Function Free Health Measure
Sample: To maximize our opportunities to detect bias based on the items and not real differences in health, we screened all potential participants on a chronic health conditions checklist and accepted only those with few chronic health conditions. Through various means, we recruited convenience samples of 60 persons with spinal cord injury, 60 people in treatment for mental health concerns, 30 people with significant vision impairment, 30 people with significant hearing impairment. We worked to keep groups approximately equivalent for age (<45; >45 years), education level (HS, no college/some college or graduate), and gender (except for SCI where we accepted more males).
Methods: Participants accepted into the study were mailed packets with the following instruments: the 121 items of the new measure, the BRFSS HRQOL items, the SF-36 items, demographic information and a return, stamped envelope. When the materials were received, a $20 gift card was mailed to them.
Data were entered and rechecked for reliability of data entry.
There is good support for the conceptual model. Factor analyses of items within domains and across the total sample show first factors with high eigenvalues and high internal reliability coefficients (for Mental Health and for Values/Beliefs). There is some support for considering additional scales within domains
Next steps will examine differences between the general population sample and each of the other samples on specific items. We anticipate that some differences will be evident for some items that reflect real differences in health. We will also examine for gender differences.
Scores on the new measure domains/individual items will be compared with the BRFSS items and the SF-36 items.
Based on these findings we will select items for a shorter measure that will be field tested on a sample of 700 adults with disabilities from across the country that have not been screened for health. Additional measures will again include the SF-36, the BRFSS HRQOL, a listing of chronic health conditions, a listing of common secondary conditions, and a measure of functional limitations.