Development of an 18-Item Measure of Symptom Burden in Patients With Glaucoma From the Collaborative Initial Glaucoma Treatment Study’s Symptom and Health Problem Checklist

Key Points

Question  Can a brief set of questions on symptoms experienced by patients with glaucoma be derived from a 43-item questionnaire used in a clinical trial?

Findings  In this measure development study of 607 patients with open-angle glaucoma, factor analysis was used to derive an 18-item questionnaire from the 43-item Symptoms and Health Problems Chart used in the Collaborative Initial Glaucoma Treatment Study. The new questionnaire’s 2 subscales capture local eye and visual function symptoms, demonstrate sound internal consistency, and are responsive to measures of glaucoma severity.

Meaning  A new questionnaire is presented that captures relevant symptoms experienced by patients receiving care for glaucoma.

Importance  Administration of a brief questionnaire to evaluate glaucoma symptoms would lend useful information for patient care.

Objectives  To develop a shortened glaucoma symptom measure based on the Collaborative Initial Glaucoma Treatment Study (CIGTS) Symptom and Health Problem Checklist (SHPC) and evaluate its psychometric properties.

Design, Setting, and Participants  This measure development study evaluated the factor structure of the 43-item SHPC that was obtained from CIGTS participants at baseline and every 6 months thereafter. These 607 participants were enrolled at 14 clinical centers in the United States and had newly diagnosed open-angle glaucoma. Their mean deviation (SD) from visual field testing was −5.5 (4.3) dB. Data were collected from October 1993 through April 1997.

Main Outcomes and Measures  The factor structure of the SHPC, confirmatory factor analysis of the resulting 18-item questionnaire (SHPC-18), the reliability of the SHPC-18, and associations of the 2 symptom subscales (Local Eye and Visual Function) of the SHPC-18 with visual field severity and 25-item National Eye Institute Visual Function Questionnaire (NEI-VFQ25) subscales.

Results  Among the 607 participants (mean [SD] age at enrollment, 57.5 [10.9] years), 334 (55.0%) were men and 273 (45.0%) were women; 231 (38.1%) were African American. Exploratory factor analysis and longitudinal growth modeling documented Local Eye and Visual Function symptom subscales. Cronbach α values for mean weighted internal consistency were 0.83 and 0.89 for the Local Eye and Visual Function subscales, respectively, and remained stable over time. Scores on each subscale significantly correlated with the NEI-VFQ25 total score (r = −0.41 and r = −0.59, respectively) and with all subscale scores (P < .01). Participants with more severe glaucoma had higher (worse) mean (SD) scores than those with mild glaucoma at baseline on the Local Eye (4.68 [6.62] vs 3.07 [5.60]) and Visual Function (8.44 [11.45] vs 4.42 [8.94]; P < .05) SHPC-18 subscales. Participants who underwent trabeculectomy reported a higher frequency of any Local Eye symptoms than those treated with topical medications (eg, at 12 months, 153 of 269 [56.9%] vs 11 of 276 [40.9%]; P < .001).

Conclusions and Relevance  These results suggest that the SHPC-18 is a reliable, responsive, and psychometrically sound measure of patient-reported, glaucoma-related symptoms. The measure is responsive to treatment and discriminates the severity of glaucoma. This shorter version of the original SHPC measure may be useful in clinical and research settings to better understand the influence of glaucoma symptoms on patients’ daily life.

Open-angle glaucoma affects more than 2 million people in the United States, with an estimated worldwide prevalence of more than 44 million.^1,2 The disease is characterized by progressive damage to the optic nerve with an associated loss of peripheral vision in the absence of therapy, which includes topical drops, laser therapies, and surgical interventions. Because of the chronicity of glaucoma, patients often need frequent clinic visits and use topical therapies daily; some patients require surgical treatments. Chronic diseases such as glaucoma often have a detrimental influence on a patient’s quality of life (QOL). Loss of vision, ocular discomfort, expensive medical and surgical therapies, and the time investment for care interact to increase the burden of glaucoma. However, the association of glaucoma with QOL is often not assessed in the clinic or the research setting. Several health-related QOL (HRQOL) measures have been used to assess individuals who have glaucoma, but many of these measures were originally developed to assess the HRQOL associated with other ocular conditions^3-6 or have not undergone a rigorous assessment of their psychometric properties.^7-9 Moreover, many of the measures available to assess glaucoma-specific HRQOL have focused solely on functional limitations from the disease but not on the burden of symptoms^8,10,11 or on the effect of topical and surgical interventions.¹²

The Collaborative Initial Glaucoma Treatment Study (CIGTS) was a multicenter, randomized clinical trial designed to compare medical and surgical therapy as the initial treatment of open-angle glaucoma. This study used a glaucoma-related symptom questionnaire, the 43-item Symptom and Health Problem Checklist (SHPC), administered before the initiation of treatment for glaucoma and then biannually during extended follow-up. The SHPC has demonstrated internal consistency, test-retest reliability, and criterion validity in patients with glaucoma.^9,13 However, the questionnaire’s 4 a priori–determined subscales have not been evaluated using psychometric methods. Moreover, the SHPC is long, limiting its utility in a clinical setting or its incorporation in research studies. We performed a psychometric analysis of the SHPC with the objective of developing a shorter measure that could be useful in clinical and research settings to assess symptom burden associated with glaucoma and its treatment. Our aim was to show that a shorter SHPC could adequately measure symptom experience and be responsive to a wide spectrum of disease severity and treatment modalities in patients with glaucoma.

Six hundred seven patients aged 25 to 75 years and newly diagnosed as having glaucoma at 14 clinical centers across the United States were enrolled in the CIGTS from October 1993 through April 1997. Eligible patients were required to have newly diagnosed primary open-angle, pseudoexfoliative, or pigmentary glaucoma in 1 or both eyes with no more than 2 weeks of glaucoma treatment. Participants were randomized to initial treatment with trabeculectomy (n = 300) or topical medications (n = 307). More details of the study’s population and protocol are found in prior publications.^9,14-16 Study approval was obtained from the institutional review boards of all CIGTS clinical and study operations centers for the CIGTS conduct and these data analyses. Written informed consent was obtained from all CIGTS participants.

The CIGTS HRQOL questionnaire is described by Janz et al.⁹ Its intent was to capture the following 3 HRQOL domains: impairment, functional status, and health perceptions. Among the measures used were the Visual Activities Questionnaire,¹⁷ the Center for Epidemiological Studies–Depression measure,¹⁸ the Sickness Impact Profile,¹⁹ and the SHPC. Between 2 baseline visits conducted at the clinical centers, CIGTS participants completed a baseline questionnaire administered by trained personnel by telephone. Participants answered questionnaires at 2 and 6 months after treatment initiation and every 6 months thereafter. The HRQOL questionnaire was administered by interviewers from a centralized interviewing center with a mean completion time of 48 minutes. Details regarding the method of administration have been described in prior publications.^9,16 Five years after the start of the study, the National Eye Institute (NEI) released the 25-item NEI Visual Functioning Questionnaire (NEI-VFQ25),⁵ which was added to the CIGTS HRQOL questionnaire in 1998.

The SHPC was the focus of the current analysis. A panel of ophthalmologists, with input from focus groups of persons with glaucoma, developed the list of 43 symptoms related to glaucoma and/or adverse effects of the treatments for glaucoma.⁹ Participants were asked whether they experienced each symptom in the past 7 days. If they reported experiencing the symptom, they were asked to what degree the symptom was attributable to glaucoma or its treatment (entirely due, partially due, or not at all due) and how bothersome it was on a 5-point scale ranging from a lot (5 points) to not at all (1 point). Empirically, the following 4 symptom subscales were identified: Visual Function (11 items; eg, difficulties going from light to dark, trouble seeing when stepping down), Local Eye (7 items; eg, eye irritation and burning, excessive tearing), Systemic (20 items; eg, nausea, rashes), and Psychological (5 items; eg, frequent mood swings, nervousness).

Visual acuity with best correction was tested at baseline and every 6 months using a modification of the Early Treatment of Diabetic Retinopathy Study protocol. The Humphrey 24-2 full-threshold visual field testing protocol was performed at baseline and every 6 months with a comprehensive ophthalmologic examination. The methods used to evaluate the visual field in the CIGTS have been described previously.^15,20 Participants were categorized into 1 of 3 groups with increasing visual field damage based on their mean deviation according to a method described by McKean-Cowdin et al.²¹ Although mean (SD) duration of follow-up was 7.2 (2.3) years, 54-month follow-up information was used in this study owing to availability of the NEI-VFQ25 and clinical information on most participants at that time.

The analysis consisted of exploratory factor analyses and a longitudinal growth model. Exploratory factor analyses were performed for each visit’s data to determine the number of factors (ie, subscales) at each visit and the specific items that loaded on (ie, were associated with) each factor. The number of factors was determined using parallel analysis^22-24 in which items that are relatively strongly associated with one another tend to produce factors with relatively high eigenvalues (a mathematical indicator of vector strength). Because some items may generate factors that are no more than statistical noise, the observed eigenvalues were compared with randomly generated threshold eigenvalues. Factor eigenvalues were calculated using Mplus software (version 5.1)²² with unweighted least squares estimation factor analysis and promax rotation to allow for factor covariation. In addition, all items were modeled as ordinal variables. Monte Carlo PCA for Parallel Analysis³⁰ was used to generate eigenvalue thresholds from a random data set. Factors with eigenvalues greater than these randomly generated thresholds were retained. After the number of factors was determined, an item was considered to load on a given factor if its associated factor loading was greater than 0.40 (per convention). Factor internal consistency was assessed by calculating Cronbach α; the conventional 0.70 cutoff for exploratory research was applied.²⁵

Longitudinal growth models were performed to evaluate whether the proposed factor structure met minimum goodness-of-fit thresholds and provided a better fit to the data than the most likely alternative model by using weighted least squares measurement.²² After establishment of a consistent factor structure across visits, factor loading in the longitudinal growth model was set to not vary across visits. Additional fit measures were also calculated using the root mean square error of approximation, with values of less than 0.05 indicating a good fit,²⁴ and the Comparative Fit Index, with values greater than 0.95 conventionally accepted as a good fit. We also used χ² difference tests to determine the most appropriate model.

The convergent validity of the instrument was assessed using visual field measures of glaucoma severity and visual functioning (NEI-VFQ25, available only at the 54-month visit). Comparisons were performed between the 2 treatment arms using χ² statistics. Subscale scores were compared among the 3 glaucoma severity subgroups using Kruskal-Wallis tests. Spearman correlations were calculated between our reduced version of the SHPC and the NEI-VFQ25 subscales.

Demographic, clinical, and SHPC data on CIGTS study participants over time are shown in Table 1. Two hundred seventy-three participants (45.0%) were women, and 334 (55.0%) were men. Two hundred thirty-one participants (38.1%) were African American, and 289 (47.6%) were older than 60 years. Mean (SD) age of the CIGTS population was 57.5 (10.9) years. Mean deviation (SD) from visual field testing was −5.5 (4.3) dB at baseline. All 607 CIGTS enrollees completed the SHPC and clinical measures at baseline; with follow-up, some censoring events (eg, losses to follow-up, deaths) occurred. Thus, by month 54, SHPC and visual field data were available for 456 CIGTS participants (75.1%). The distribution of age, sex, and race remained consistent during follow-up.

Unweighted least squares factor analyses with promax rotation generated factor eigenvalues for all 11 visits (eFigure in the Supplement). Across the 11 visits, parallel analysis eigenvalue thresholds for the symptom subscales varied slightly owing to changes in sample size (eg, 9.33 to 12.17 for the Visual Function subscale and 1.32 to 2.00 for the Local Eye subscale). For 9 of the 11 visits, only the first and second factors exceeded their eigenvalue threshold, suggesting the retention of 2 factors. In the remaining 2 visits, only the first factor exceeded its eigenvalue thresholds, whereas the second factor eigenvalues approached their thresholds. In no case did a third factor eigenvalue approach its parallel analysis eigenvalue threshold. Thus, the analysis showed that a 2-factor model best represented the items.

Items that reflected physical problems with the eye and surrounding areas tended to load together in a local eye factor (eg, eye pain and droopy eyelids). Items that assessed visual perception problems loaded on a visual function factor (eg, difficulties with near vision and visual distortion). These item loadings were consistent across visits, as shown in Table 2. Of the 198 factor loadings, only 11 deviated from the expected 2-factor structure, producing a 94% consistency rate. Stratifying by treatment allocation, race, sex, or age did not appreciably change the factor structure or item loadings.

The Local Eye (7 items) and Visual Function (11 items) symptom subscales showed good internal consistency that was sustained over time. Cronbach α values for the Local Eye subscale ranged from 0.77 to 0.86; for the Visual Function subscale, from 0.86 to 0.90. The mean weighted Cronbach α was 0.83 for the Local Eye subscale and 0.89 for the Visual Function subscale.

Based on the 2-factor model, 2 longitudinal growth models were fit to the 18 items across all 11 visits simultaneously to confirm the factor structure. In the 2-factor model, items were loaded onto their respective Local Eye or Visual Function subscale. In the 1-factor model, all items were loaded onto a single scale. The 2-factor (χ²_19 341 = 43 627) and 1-factor (χ²_19 359 = 55 320) models fit the data better than the null model (χ²_19 503 = 977 320). Additional fit indices (root mean square error of approximation and the Comparative Fit Index) revealed the better fit of the 2-factor model and the weighed least squares measurement χ² difference testing (Δχ²₁₈ = 1133; P < .01). We named this reduced (18-item) version of the SHPC the SHPC-18.

We examined associations between clinical measures and the subscale scores. Consistent with the findings of Janz et al⁹ with regard to the 43-item SHPC, the frequency of experiencing any symptom on the 2 SHPC-18 subscales increased from the baseline visit to the earlier posttreatment visits. Because the glaucoma status of CIGTS participants did not change from baseline to month 3, the observed increase in frequency of experiencing any symptoms was attributed to glaucoma treatments that were initiated after the baseline visits (topical medication or incisional surgery). The Figure shows this increase from baseline to month 3 to be more evident for local eye symptoms than for visual function symptoms. The reported symptom frequency declined after this initial increase and eventually plateaued. Participants who underwent surgery reported a significantly greater prevalence of having any local eye symptom. For example, at month 12, 153 of 269 surgery group participants (56.9%) reported having a local eye symptom vs 113 of 276 medical group participants (40.9%; P < .001). Specific local eye symptoms that were reported more frequently in the surgery group at month 12 included excessive tearing, the feeling that something was in the eye, eye pain, and eye redness. At most time intervals, Visual Function symptom subscale problems were comparable between surgery and medicine groups.

The 2 subscales of the SHPC-18 correlated with visual functioning as measured by the NEI-VFQ25 at 54 months (Table 3). All correlations were negative because of the 2 measures’ scale differences, such that a worse (higher) score on the SHPC-18 subscales was associated with a worse (lower) score on the NEI-VFQ25 subscales. Because of the NEI-VFQ25 focus on functioning, the SHPC-18 Visual Function symptoms subscale had consistently larger correlations with the NEI-VFQ25 subscales than the Local Eye symptoms subscale, with the exception of the NEI-VFQ25 Ocular Pain subscale.

Increased glaucoma severity based on the classification of McKean-Cowdin et al²¹ was associated with higher mean (SD) Visual Function subscale scores (Table 4) at baseline (group 1, 4.42 [8.94]; group 2, 7.91 [12.39]; and group 3, 8.44 [11.45]; P < .001) and at month 54 (group 1, 2.33 [5.58]; group 2, 4.18 [8.92]; and group 3, 7.90 [12.36]; P < .001). A similar trend was observed for the Local Eye subscale scores at baseline (group 1, 3.07 [5.60]; group 2, 4.62 [7.61]; and group 3, 4.68 [6.62]; P = .02) and at month 54 (group 1, 2.04 [4.61]; group 2, 3.33 [6.65]; and group 3, 4.88 [7.60]; P = .04).

Although the CIGTS 43-item SHPC was empirically divided into 4 symptom subscales, our study revealed a psychometric basis for 2 of those subscales, Local Eye and Visual Function, by using factor analysis techniques. The SHPC-18 showed good reliability, responsiveness, association with a measure of glaucoma severity and correlation with the NEI-VFQ25 subscales. With fewer items, this scale could be more readily used in the clinic setting because its 18 items should take less time to administer than the 43-item SHPC. We recommend that patients be asked whether they experienced the symptom in the past 7 days and, if so, how burdensome the symptom was. Symptom attribution can be difficult for patients to assess, and most CIGTS participants (71%-73%) attributed local eye and visual function symptoms to glaucoma. Scoring would range from 0 to 35 for the 7-item Local Eye subscale and from 0 to 55 for the 11-item Visual Function subscale. The SHPC-18 is less burdensome and conveys better psychometric properties for use in patients with glaucoma than the 43-item SHPC.

Our inferences are strengthened by the longitudinal design of the CIGTS, which allowed for assessment of patient-reported symptoms before and after the initiation of glaucoma treatment for more than 4 years. This follow-up revealed patterns of change in symptom reporting over time. The similarity of the distribution of glaucoma severity based on visual field damage in CIGTS participants relative to patients with glaucoma in other population-based studies supports the generalizability of the factor structure and observed associations.^21,26

Because the SHPC-18 captures local eye and visual function symptoms of glaucoma and its treatment, its use provides some advantages compared with other measures that have been used in the population with glaucoma to assess functional associations with vision loss. Many of these measures do not quantify the scope or burden of symptoms that patients with glaucoma experience. For example, the NEI-VFQ25 and the Glaucoma Quality of Life–15 questionnaires are psychometrically evaluated instruments that can detect the association of decreased vision with glaucoma.^4,5,8,27 These measures focus primarily on functional changes and do not capture the scope of symptoms covered in the SHPC-18. However, we did find consistent, statistically significant correlations among the NEI-VFQ25 and the SHPC-18 subscales.

Several other measures that assess nonvisual symptoms of glaucoma have limited applicability. The Comparison of Ophthalmic Medication for Tolerability measure¹² obtains information on other aspects of the glaucoma experience (eg, satisfaction) and is limited by its focus on topical therapy alone. This instrument has less applicability to the surgical or laser therapies frequently used to treat glaucoma. In light of the increasing treatment of nonrefractory glaucoma with modalities other than topical medications,²⁸ an instrument that captures patients’ treatment experience is essential. The 10-item Glaucoma Symptom Scale⁷ measures how bothersome symptoms associated with the topical treatment of glaucoma are, and its domain that captures 4 visual symptoms correlated with Advanced Glaucoma Intervention Study visual field scores but not with Esterman visual field scores.²⁹ In addition, the psychometric evaluation of the Glaucoma Symptom Scale was performed cross-sectionally, not longitudinally, as was the SHPC-18, in a population of patients with treated glaucoma, which does not allow for assessing its responsiveness to change in glaucoma severity over time.

Our study had several limitations. We did not evaluate the factor structure in another cohort to further substantiate the psychometric analysis, which would strengthen our conclusions regarding the psychometric properties of the SHPC-18. Some loss to follow-up occurred over time, but study participants who attended at 54 months were similar to those who were lost to follow-up, as documented in Table 1 and data previously presented by Janz et al.¹⁶ The limited number of CIGTS participants who experienced substantial glaucomatous progression prevented a person-based assessment of the association between the SHPC-18 measures of local eye or visual function symptoms and glaucomatous progression. Finally, the CIGTS questionnaires were developed and evaluated for patients with 3 types of open-angle glaucoma and thus may not fully capture the symptoms experienced by those with other types of glaucoma.

The SHPC-18 is a relatively brief, psychometrically robust measure that can be used to assess the presence of and burden associated with visual and nonvisual symptoms in patients receiving treatment for glaucoma. To extend its application, the SHPC-18 needs to be evaluated in clinical settings for its timing and methods (eg, electronic, self-administered) of administration and evaluated in other glaucoma subtypes and in patients with differing racial/ethnic distributions or levels of health literacy. The smallest change in an SHPC-18 score that a patient would identify as important remains to be determined. Continued development of methods to evaluate HRQOL in patients with glaucoma will benefit the patient and the clinician. Because of the influence of glaucoma on patients, the SHPC-18 can provide clinicians with additional information to guide treatment decisions.

Corresponding Author: David C. Musch, PhD, MPH, Kellogg Eye Center, 1000 Wall St, Ann Arbor, MI 48105 (dmusch@med.umich.edu).

Accepted for Publication: September 8, 2017.

Published Online: November 2, 2017. doi:10.1001/jamaophthalmol.2017.4574

Author Contributions: Dr Musch had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Musch, Tarver, Janz.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Tarver, Goren.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Goren.

Obtained funding: Musch, Tarver, Janz.

Administrative, technical, or material support: Musch, Tarver.

Study supervision: Musch, Tarver.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Musch reports membership in data and safety monitoring boards for clinical trials of glaucoma treatment sponsored by Glaukos Corporation, InnFocus, LLC, and Ivantis, Inc. No other disclosures were reported.

Funding/Support: This study relied on information collected during the conduct of the Collaborative Initial Glaucoma Treatment Study, which was supported by grant EY09148 from the National Institutes of Health.

Role of the Funder/Sponsor: The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: Leslie M. Niziol, MS, Department of Ophthalmology and Visual Sciences, Medical School, University of Michigan, Ann Arbor, provided statistical assistance in the revision of this manuscript and did not receive compensation for the work.