Health Status Tools Development Study
Principal Investigator: Gloria Krahn, PhD, MPH; Trevor Hall, PsyD, Willi Horner-Johnson, PhD; Expert Panel on Measurement
The purpose of this study is to develop and test a health status measurement tool to reliably assess the health status of individuals regardless of functional ability. The tool will also be tested to determine if it is sufficiently sensitive to detect change due to interventions. Several approaches will be used to develop and test the items used in the tool as well as validate its use with people living with disabilities.
Hypothesis 1: A measurement tool can be developed to reliably assess the health status of individuals regardless of functional ability.
Hypothesis 2: A measurement tool can be developed to reliably assess self-reported health status and that is sufficiently sensitive to detect change due to interventions
Background to the Hypotheses: In its 30-year history, Health Related Quality Of Life (HRQOL) and health status assessment has generated numerous measures and developed a knowledge base about health assessment. Many current measures of health assessment are rooted in the presumption that health and functional status (disability) are conceptually similar constructs. Measures like the SF-36 confounds functional ability with health and used the ability to discriminate between the scores of people with disabilities and the general population as proof of the validity of the measure. For disability and health researchers, it is critically important to disentangle this conceptual confounding. Without this distinction in assessment, researchers cannot use these measures to assess health disparities or to confirm effectiveness of health intervention efforts to reduce these disparities.
The first steps are to define health and function. Conceptualization of health status should lead to item selection and generation for its assessment. Data from R7 will help determine how well the HRQOL items of the Behavior Risk Factor Surveillance System function together and with what other elements they correlate highly. The next step in the process is scale construction. The expert panel will deliberate on whether health status should be regarded as unidimensional or multi-dimensional. While traditional scale construction using classical test theory focused largely on items in relation to a total score, other approaches to test construction also emphasize the item in relation to other items. Item Response Theory (IRT) represents a statistical approach to careful selection of items. For example, Rasch modeling is a type of IRT. It constructs a line of measurement, with the items placed hierarchically along a continuum based on the probabilistic linear interaction between the person’s “ability” and the item’s “difficulty” A final issue for health status assessment is that of a tool being sufficiently responsive to change that it documents effects of intervention efforts. A second tool may be needed that more responsively captures changes in health behaviors that may not have immediate self-perceived effect on health status.
Research Design: Expert panel process for conceptualization of measure, classical test theory and item response theory approaches for selection of test items and validation of the scale.
Sample: Independent samples of persons with disabilities for alpha and beta testing of the tool(s)
a) Cognitive testing of test items with 25 adults with disabilities (AWD) and 25 adults without disabilities of differing education levels
b) pilot testing of the tool with 240 adults, (60 mobility impairment, 60 sensory impairment, 60 mental health disability; 60 without impairment/disability) varying in age and education; all will also complete the SF-36.
c) Field testing with 500 adults with disabilities varying in education and age; 100 will complete a two-week test-retest reliability check; 60 will be enrolled in pre-post test from an intervention study.
Data Collection and Measurement: There are four stages to data collection: 1) item selection and/or generation; 2) cognitive appraisal testing of the items on health status; 3) initial testing of the items and scale construction for health status (alpha test); and 4) field testing of the health status tool and the health behaviors tool with several samples (beta test). The expert panel will provide oversight and consultation throughout this process.
Initial item selection for health status will be determined based on the expert review panel recommendations from R6 regarding definition of health status and its component element(s) (e.g., physical, mental, social and spiritual/living one’s values), and from the panel’s recommendations for desired measurement characteristics. These recommendations will serve as the framework for the health status scale construction. Findings from the analyses of R6 and R7 will influence the potential inclusion of items. Items will be examined for reading level, with the intent of keeping reading level to the 8th grade level or below. Once the initial set of questions has been drafted, it will undergo cognitive appraisal testing. Problem items will be reworded for increased clarity and precision as needed. Additional respondents will be recruited if a second round of cognitive appraisal testing is necessary. The initial draft of the health status tool along with the health behavior tool will be pilot-tested with 100 adults with disabilities to identify potential difficulties with administration, respondent burden, and to provide initial data on item performance. At least half of this sample will be recruited through Care Oregon and be selected to have an educational level of some college or less to ensure utility with a range of educational backgrounds. Field-testing of the tools will occur through a series of samples. First, a sample of 700 adults with disabilities will be recruited that is stratified by educational level and gender. This sample size should be adequate to allow for factor analyses of a beta-test measure with as many as 60-70 items. Demographic information will also be collected that includes age, type and severity of disability. These variables will be examined later for their correlation with the scales and the items, but will not be used for stratification purposes. Administration will be conducted as a paper and pencil test or online. All respondents will be asked to complete all items from the health status tool and the health behavior tool. Respondents will be compensated for their participation (budgeted at $30 per respondent). A second sample of 50 adults who are participating in a health promotion intervention program will be asked to complete the tools as pre and post-test measures of their participation. This sample size of 50 will be used because it is a reasonable cohort size for an intervention program. If effect sizes cannot be detected with this sample size, the tool(s) are not likely to be useful as measures of intervention effectiveness. A third sample of 100 individuals with disabilities will be recruited for test-retest reliability. These may be drawn from the larger general sample with a retest administration occurring 2 weeks after first administration to examine short-term stability of the measure(s).
Data Analyses: Classical test theory indicators will be used for initial screening of potential items, and item response theory will be used for refinement of the unidimensional scale(s). For the pilot test data, the draft items of the heath status tool will be subjected to item analysis using standard statistical procedures. An index of discrimination will be calculated as an item-total correlation, with items with a correlation of less than .40 being considered for elimination. An endorsement index will be calculated as the proportion of people responding “yes” to an item; those with scores of less than .20 (too hard) or .80 (too easy) will be considered for elimination. Test-retest reliability indices will be calculated across the two test administrations, and items with less than a .50 will be considered for elimination. These results will inform the selection of items for the beta-test version. Analyses will be performed in either the Statistical Package for Social Sciences (SPSS) or Statistical Analysis Software (SAS) program. Exploratory factor analyses will be conducted on the larger sample to determine the degree to which a unidimensional measure is feasible. Based on these findings, item response theory analyses will proceed with the full set or subsets of the health status items. Log-odds units (logits) will be calculated for items. Item calibration (expressed as logits) will define the hierarchical order of severity of “difficulty” of the item along the health status continuum. Logits of greater magnitude indicate increasing item severity. In addition, the unidimensionality of the scale or subscales will be evaluated through a goodness-of-fit test (Chi-square fit statistics) and by a formal test of the assumption of local independence. Successive Rasch analyses will be performed until a final set of items satisfy the model fit requirements. These analyses will be conducted in BIGSTEPS or an equivalent program. Analyses for the health behavior tool will be more limited and follow classical test theory procedures. Unidimensionality is not necessarily a desired feature of this tool. For the tool to demonstrate responsiveness to a range of interventions (e.g, physical activity, stress management, nutrition), a “profile” measure may be more appropriate that includes several components that are measured separately. Analyses for this tool will examine item-subtotal correlations, item-endorsement indices, test-retest correlations, and pre-test/post-test comparisons. Analyses will be conducted in SPSS or SAS.
Anticipated Findings: A beta-tested measure of health status assessment for adults with disabilities that has been constructed to be nonbiased with respect to functional abilities. Psychometric characteristics will have been determined for internal consistency, hierarchical linearity, test-retest reliability, responsivity to intervention, and correlation with the SF-36.
