Frequency of anchor responses by baseline Ocular Surface Disease Index (Allergan, Inc, Irvine, California) category. CGI indicates clinician global impression; and SGA, subject global assessment.
Minimal clinically important differences (MCID) for the clinician global impression (CGI) and the subject global assessment (SGA) by baseline Ocular Surface Disease Index (OSDI) (Allergan, Inc, Irvine, California) category.
Miller KL, Walt JG, Mink DR, Satram-Hoang S, Wilson SE, Perry HD, Asbell PA, Pflugfelder SC. Minimal Clinically Important Difference for the Ocular Surface Disease Index. Arch Ophthalmol. 2010;128(1):94-101. doi:10.1001/archophthalmol.2009.356
PAUL P.LEEMDAuthor Affiliations: ICON Clinical Research, San Francisco (Dr Miller and Mr Mink), and Allergan, Inc, Irvine (Mr Walt and Dr Satram-Hoang), California; Cole Eye Institute at Cleveland Clinic, Cleveland, Ohio (Dr Wilson); Corneal Specialists and Cataract and Refractive Surgery, Ophthalmic Consultants of Long Island, Rockville Centre, Long Island (Dr Perry), and Department of Ophthalmology, The Mount Sinai Medical Center, New York (Dr Asbell), New York; and Alkek Eye Center, Baylor College of Medicine, Houston, Texas (Dr Pflugfelder).
Copyright 2010 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2010
To assess the minimal clinically important difference (MCID) for the Ocular Surface Disease Index (OSDI; Allergan Inc, Irvine, California, holds the copyright), a 12-item patient-reported outcome questionnaire designed to quantify ocular disability due to dry eye disease.
Study data were collected within the Restasis Review of Efficacy and Safety vs Tears in the Relief of Dry Eye (RESTORE), an observational registry. A clinician global impression (CGI) and a subject global assessment (SGA) served as anchors to estimate the MCID for the overall OSDI score (range, 0-100). The overall OSDI score defined the ocular surface as normal (0-12 points) or as having mild (13-22 points), moderate (23-32 points), or severe (33-100 points) disease. RESTORE patients were included if they completed the OSDI at the baseline visit and at a follow-up visit and had a global change rating (SGA or CGI).
Three hundred ten patients were included (82.3% white and 81.6% female [mean age, 57.8 years]). The CGI and SGA correlated with the OSDI score change for all OSDI categories except the normal category. The MCID ranged from 7.0 to 9.9 for all OSDI categories. The MCID ranged from 4.5 to 7.3 for mild or moderate disease and from 7.3 to 13.4 for severe disease.
Using observational data, we estimated the MCIDs for different baseline OSDI categories of dry eye disease. These results will assist clinicians and researchers when interpreting OSDI score changes.
Keratoconjunctivitis sicca, also known as dry eye disease, is a condition that results in dryness of the conjunctiva and cornea due to decreased function of tear glands or rapid evaporation of tears.1 The prevalence of dry eye disease is greater among women than among men and increases with age.2 The symptoms of dry eye disease include a dry gritty feeling in the eyes that is often accompanied by burning, redness, and the sensation of a foreign object in the eye. If left untreated, dry eye disease can damage the delicate tissues of the surface of the eye that may lead to impaired vision.3
To diagnose dry eye disease clinicians rely on patient reports of symptoms and clinical examinations.1,4 Although ophthalmologists may use several dry eye disease testing methods (such as tear breakup time, Schirmer test, and corneal and conjunctival staining), there is a lack of consensus on diagnostic criteria, classification of disease states, and aims and interpretation of specific diagnostic tests. The lack of agreement is partly because of the weak correlation between clinical diagnostics and subjective symptom severity.1,3 Therefore, patients' self-report of symptoms is typically a determining factor in dry eye disease diagnosis.1,3 Given the unique nature of dry eye disease symptoms, a patient-reported outcome (PRO) questionnaire is a useful and appropriate tool to identify dry eye disease and to assess changes in patients' symptoms.
The Ocular Surface Disease Index (OSDI; Allergan, Inc, Irvine, California, holds the copyright), a PRO questionnaire, was designed to provide rapid assessment of the range of ocular surface symptoms related to chronic dry eye disease, their severity, and their effect on the patient's ability to function.5 The OSDI includes the following 3 domains: ocular symptoms, vision-related function, and environmental triggers.6,7 The goals of the OSDI are to make the diagnosis of ocular surface disease easier, quicker, and more reliable and to provide evidence of differences in ocular disability due to dry eye disease. The OSDI allows clinicians to collect comprehensive subjective data in addition to a clinical history, and it can be used as a tool for measuring the effectiveness of a specific dry eye disease treatment.
The US Food and Drug Administration (FDA) suggests that PRO questionnaires (such as the OSDI) should be based on a clear conceptual framework and that there should be evidence supporting their psychometric properties.8 The content of the OSDI was generated through acceptable measures. Consistent with the FDA's guidance on the use of PRO instruments, the OSDI has been proven to be a reliable instrument for measuring the severity of dry eye disease.6 The FDA also recommends specification of the minimal clinically important difference (MCID) as a benchmark for interpreting the mean score differences between treatment arms in a clinical trial. The MCID is defined as “the smallest difference in score in that domain of interest which subjects perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management.”9(p408)
The MCID for an instrument can vary according to the characteristics of the population being studied. In studies of the psychometric properties of the OSDI, Schiffman et al6 found that the specificity and sensitivity improved when the instrument was used to discriminate more severe disease. The objective of this article is to expand on the evidence supporting the use of the instrument by assessing the MCID for the OSDI according to the severity of patients' symptoms.
Patients with dry eye disease were recruited from 75 sites for the Restasis Review of Efficacy and Safety vs Tears in the Relief of Dry Eye (RESTORE) registry, an open-label nonrandomized observational study created to evaluate the effectiveness of cyclosporine ophthalmic emulsion (0.05%) (Restasis; Allergen, Inc) vs eyedrops and other nonprescription treatments in patients with dry eye disease. Physicians invited consecutive patients whose tear production was presumed to be suppressed in association with dry eye disease to participate in the RESTORE study. To be eligible, patients had to have a diagnosis of dry eye disease, currently use artificial tears daily, have normal eyelid position and closure, and be likely to complete all required follow-up visits. Patients were ineligible for enrollment into the study if they had current or prior use of cyclosporine ophthalmic emulsion, were enrolled in an investigational drug or device study, had any current or previous topical ophthalmic or oral cyclosporine use within the past 3 years, or had a condition that in the physician's opinion may have put the patient at significant risk, confounded the study results, or interfered with the patient's participation in the study.
The RESTORE registry was designed to collect clinical and outcomes data without influencing the treatment or interventions prescribed by clinicians caring for patients with dry eye disease. As such, a RESTORE patient completed a baseline visit at the time he or she opted to enroll in the study, but follow-up visits were based on the patient's usual medical care (as determined by the clinician) and were not mandated by the study. The first patient in the RESTORE registry completed the OSDI in September 2004, and data collection is ongoing.
Demographic data, medication use, and clinical variables were collected at baseline, including clinical properties of the cornea, lid, conjunctiva, and tear film, as well as visual acuity test results. Safety, PROs (including the OSDI), and efficacy test results (ie, tear breakup time, the Schirmer test, and corneal and conjunctival staining) were assessed at baseline visits and at follow-up visits. The historical lack of agreement between any objective measure and the subjective assessments made a single clinical measure unattractive to use as anchors in the MCID computation. Instead, the physicians in this study were asked to use traditional clinical assessments (eg, diagnostic tear film assessments and clinical appearance) to provide the clinician global impression (CGI) of change in dry eye disease symptoms at each follow-up visit. At each follow-up visit, patients were asked to assess their overall change from baseline in the subject global assessment (SGA). The answers to these questions served as MCID anchors (Table 1). Patients were categorized according to their responses on the CGI and SGA, and these categories anchor the OSDI score to an expected behavior. The anchors were used to determine the MCID in the OSDI needed to detect a meaningful difference in a patient's dry eye disease.
Although patients completed the questionnaires on paper, the study sites entered and submitted electronic data via the Internet. Data submission was monitored continuously to ensure that data were current and that the data quality was within the specified parameters.
The OSDI is a 12-item self-administered PRO questionnaire to assess ocular surface symptoms.5,6 The questionnaire is of low burden to the patient (it takes approximately 5 minutes to complete) and has been used successfully by researchers and by clinicians. The OSDI has an overall score and 3 subscale scores (ocular symptoms [3 items], vision-related function [6 items], and environmental triggers [3 items]). Each OSDI item is scored on a Likert-type scale ranging from 0 to 4 points, where 0 indicates none of the time and 4 all of the time. The OSDI overall and subscale scores range from 0 to 100. Based on their OSDI scores, patients can be categorized as having a normal ocular surface (0-12 points) or as having mild (13-22 points), moderate (23-32 points), or severe (33-100 points) ocular surface disease. The scoring of the OSDI was performed according to the published guidelines.5 The OSDI has satisfactory internal consistency, test-retest reliability, validity, sensitivity, and specificity for use among patients with ocular surface disease.6
To be included in the analytic cohort, a patient had to have met the RESTORE enrollment criteria and completed a baseline OSDI assessment and a follow-up assessment on which at least 1 anchor item was answered (either the CGI completed by the physician or the SGA completed by the patient). For patients who had multiple follow-up visits, the analysis included the first follow-up assessment on which either anchor was completed. These anchors were used to group patients according to the expected OSDI score change.
To confirm the usefulness of the anchors, Pearson product moment correlations were computed between the OSDI score change and each anchor for each OSDI category at baseline (all OSDI categories, normal, mild, moderate, and severe). A small statistically significant positive correlation is considered sufficiently strong to allow for computation of an MCID.10
We hypothesized that because patients were receiving physician care for their condition, the number of responses in the worsening categories would be minimal for the CGI and SGA. Therefore, we used an anchor-based analytic approach previously described by Jaeschke et al9 and by Wiebe et al11 to account for the small sample sizes expected in the worsening groups. In this method, the OSDI scores are grouped (or folded together) based on anchor value intensity regardless of scale direction as follows. For the CGI, marked improvement was grouped with marked worsening, moderate improvement was grouped with moderate worsening, and minimal improvement was grouped with minimal worsening. For the SGA, much improved was grouped with much worse, and improved was grouped with worse. For items on the worsening side of the scale, the sign of the OSDI score is changed. That is, negative OSDI score changes are converted to positive OSDI scores (or vice versa) to group them with their counterparts on the improving side of the scale. Lines are fitted through these transformed data to generate regression coefficients. These regression coefficients are the slope of the regression line or the rate of change in the anchor as a function of the OSDI, and they represent our estimate of the MCID. This method prevented the few worsening responses from having an inordinate influence on the slope of the regression line. The result was more robust and stable MCID estimates that are applicable to the entire OSDI score range (as opposed to worsening or improvement scores alone). An MCID for each OSDI category was estimated using each clinical anchor (CGI and SGA).
In addition, 3 distribution-based estimates were applied to the OSDI scores among all patients and were used as an additional confirmation of the MCID. These methods provide descriptive statistics on the magnitude of change observed in the study in standard deviation units of the whole sample. The first is the standard deviation of the overall OSDI score for all patients divided by 2 to establish a 0.5-SD estimate. The second and third are the effect size and the standardized response mean for the OSDI scores. The effect size is the ratio of the difference between the mean baseline and follow-up OSDI scores to the standard deviation of the baseline score. The standardized response mean is the ratio of the observed change score to the standard deviation of the baseline score, reflecting the variability of the change scores.
As of March 2008, a total of 310 patients enrolled in the RESTORE registry were eligible to be included in the MCID analysis cohort. Table 2 gives the baseline characteristics among all patients and according to their OSDI category at the baseline assessment. Most patients in the study were female (253 [81.6%]) and of white race/ethnicity (255 [82.3%]). The mean (SD) age of patients at baseline was 57.8 (16.6) years, and the mean (SD) time since the dry eye diagnosis was 5.0 (5.9) years. The time between the baseline assessment and the first follow-up assessment ranged from 10 to 15 months for most patients. The largest OSDI category was the severe category (140 [45.2%]), and the smallest OSDI category was the normal category (18 [5.8%]).
Table 3 gives the percentages of patients having diseases and conditions that are thought to contribute to dry eye disease symptoms among all patients and according to their OSDI category at the baseline assessment. The most commonly reported comorbidity was cataracts (25.5% among all OSDI categories), followed by rheumatoid arthritis (15.2% among all OSDI categories). Cataracts and rheumatoid arthritis were more common in the mild category (37.0% and 25.9%, respectively) than in the other OSDI categories. The percentages of patients who were smokers were fairly consistent across OSDI categories, ranging from 4.1% in the moderate category to 8.6% in the severe category. The percentage of patients who had undergone laser in situ keratomileusis was highest in the severe category (22.1%).
Table 4 gives the correlations of the OSDI score changes with the CGI and SGA anchors for each OSDI category. The OSDI score changes for patients in the normal category did not correlate with the responses to either anchor, as expected. However, both anchors showed statistically significant positive correlations with the OSDI score changes for the other OSDI categories (>0.35 for all). Therefore, the CGI and SGA were considered appropriate anchors for measuring the OSDI score changes in the mild, moderate, and severe OSDI categories.
Figure 1 shows the response frequencies for the CGI and SGA. For each instrument, the response options are grouped according to the degree of change (eg, moderate worsening is grouped with moderate improvement). The responses from patients who improved are stacked above the responses from patients who worsened in the same category. The most extreme responses are positioned farthest from the unchanged group (at the left). As expected for all severity categories and for both anchors, more patients had improved symptoms than worsening symptoms at the follow-up assessment. In general, patients in the mild or moderate category had small improvement, whereas patients in the severe category had larger improvement.
Table 5 gives the mean OSDI score change for each OSDI category (excluding the normal category) and for each response to the CGI and SGA. For the most part, the larger the magnitude of change reported on the CGI or SGA, the greater is the OSDI score change from baseline. For all OSDI categories, the largest OSDI score change was reported in the group whose physicians reported the most change on the CGI (ie, marked change) and in the group who self-reported the most change on the SGA (ie, much changed). The trend holds in the pooled grouping of all OSDI categories, including the normal category. The mean OSDI score change for the unchanged group approximates zero for both anchors and for all severity categories except the CGI mild category. The mild group seems to report OSDI score changes at the follow-up visit (mean [SD], 13.1 [20.2]) that go undetected by the clinician. Patients whose physicians noted minimal change on the CGI reported the smallest OSDI score change (mean [SD], 0.4 [12.1]). The largest mean (SD) OSDI score changes were reported by patients who noted much changed on the SGA (−31.8 [12.4]) and whose physicians noted marked change on the CGI (−31.3 [11.7]).
While the means for each anchor response (listed in Table 5) give an indication of the OSDI score change necessary to detect a clinical difference, the MCIDs (shown in Figure 2) summarize the information across anchor responses. For each OSDI category, Figure 2 shows the MCID point estimate (95% confidence interval) for both anchors. There is good agreement between the MCIDs computed with the CGI and SGA except in patients who reported severe OSDI scores at baseline. This also contributes to the differences found in the pooled OSDI categories. Overall, clinicians and patients consider an OSDI score change of 4.5 to 7.3 to be meaningful for mild or moderate symptoms. A greater OSDI score change of 7.3 to 13.4 is required before patients with severe symptoms consider the change to be meaningful.
The results of the distribution-based analyses among all patients (N = 310) are consistent with the findings of the anchor-based approach. The 0.5-SD approach yields an MCID of 9.8. The effect size of 0.51 and the standardized response mean of 0.57 confirm our anchor-based estimate of the MCID for all patients.
The unique nature of ocular surface disease requires that characterization of the condition rely not only on clinical objective measures but also on direct reports of symptoms by patients. Clinicians may use various tools in practice to solicit the information they need to adequately diagnose, rate, or monitor the progression of dry eye disease symptoms. However, for investigators to use any subjective measures (such as the OSDI) to interpret intervention effects and to make claims about a product, the European Medicines Evaluation Agency12 and the FDA8 have suggested that the burden is on the sponsor company to determine the change that is required to be clinically meaningful.
This study estimated the MCID for the OSDI within a population receiving physician care for their dry eye disease. Data collection for this study occurred within the context of an observational registry and not within a standardized environment of a randomized clinical trial. This naturalistic study design results in data that mimic findings in clinical practice. Furthermore, the population studied herein is representative of previous research in dry eye disease (ie, mostly white women older than 50 years).
For the analyses, patients were classified according to self-reported OSDI category at baseline (normal, mild, moderate, and severe) to account for anticipated MCID variations across patients with different dry eye disease severity at baseline. Patients in the 4 groups were similar in terms of demographics, time since dry eye disease diagnosis, and period between visits. Patients with severe dry eye disease had more comorbid conditions than patients with less severe dry eye disease, but a greater percentage of patients with mild dry eye disease had rheumatoid arthritis and a history of cataracts than patients in the other OSDI categories.
The resulting linear regression analysis of data indicates that the MCID for the OSDI ranged from 7.0 to 9.9 for all OSDI categories. The distribution-based approach supported these estimates for all OSDI categories. In patients with mild or moderate symptoms, the MCID for the OSDI is 4.5 to 7.3. In patients with severe symptoms, the MCID for the OSDI is 7.3 to 13.4.
The MCID differences suggest that, in using the OSDI to evaluate treatment response, clinicians and researchers will need to consider the characteristics of a population before establishing the definition of responders. In other words, the response to treatment among only patients with severe dry eye disease would need to be set higher than the response to treatment among the entire population with dry eye disease, which would be higher than the response to treatment among patients with moderate dry eye disease.
We chose to evaluate the MCID using an anchor-based approach that relies on reports from both the patient and the clinician. To reduce patient burden, the SGA had fewer response options than the CGI. Therefore, scores for the clinician and patient do not fully overlap and cannot reasonably be averaged into a single global rating that considers the patient and clinician ratings. However, by examining the MCID using each anchor individually, we detect trends that are seen in clinical practice.
For example, it has been noted that the objective improvement reported by the clinician is often greater than the symptomatic improvement and that there can be a substantial disconnect between signs and symptoms in dry eye disease. The difference may be due to multiple factors, including clinician bias. Because the clinician presumably relies on clinical tests to assess whether the patient has improved since baseline, the clinician may detect improvement in ocular appearance before the patient notices a decrease in symptoms. Conversely, the patient may detect effects on nerve sensitivity before objective signs are present. A single-anchor analysis may not detect this subtle difference.
Given the nature of dry eye disease care, it is likely that RESTORE patients were using cyclosporine ophthalmic emulsion or nonprescription medication. Therefore, the expected data set would mostly include patients who reported clinical improvement on both anchors and on the OSDI. Given the small sample sizes, the correlation coefficients for only the worsening patients would likely be unstable and would not allow for a reliable estimate of a unidirectional MCID. Rather than excluding the responses of patients who worsened, we used methods that would account for the small sample sizes without discarding the data entirely. To maintain data from the worsening groups, we used a technique that allowed analysis of the entire anchor range by sorting the patient responses according to the magnitude of change without accounting for whether a patient reported worsening or improving symptoms. This technique assumes that the OSDI score change needed to detect a clinical improvement is the same change needed to detect a clinical deterioration. To some degree, the interpretability of the OSDI is limited because it more heavily emphasizes the responses of those who improve clinically than those who worsen clinically. Nevertheless, the value in evaluating the MCID is in the context of potential use to demonstrate a treatment effect; in this case, the change to be detected is reduced dry eye disease symptoms and improved OSDI scores. The OSDI was able to detect not only large clinical improvements but also small (or minimal) clinical improvements.
The anchor-based approach used to estimate the MCID for the OSDI assumes that there is some association or relationship between clinician and patient ratings of change responses and OSDI score change.13 The MCID for the OSDI can be applied to patients with mild, moderate, and severe baseline OSDI scores; for these patients, the OSDI score change correlated well with the CGI and SGA. However, for patients with normal baseline OSDI scores, the OSDI score change did not correlate well with either anchor. At enrollment, all patients (including those in the normal category) had a diagnosis of dry eye disease and were using artificial tears daily. We postulate that if a patient reports minimal symptoms or none on the OSDI and the instrument classifies the patient as being in the normal category, then perhaps dry eye disease alone does not explain the patient's symptoms at baseline or his or her reason for the baseline clinical visit. The physician diagnosis may be based on a clinical result that is unapparent to the patient and is not captured on the subjective assessment. Under these conditions, an OSDI score change may not correspond to a global change whether rated by clinician or patient. In addition, the small sample size of the normal group likely contributed to the lack of agreement.
In conclusion, this study provides an estimate of the MCID for the OSDI. This estimate allows the instrument to be used with confidence as an end point in clinical trials designed to evaluate an intervention to treat patients with dry eye disease.
Correspondence: Kimberly L. Miller, PhD, ICON Clinical Research, 188 Embarcadero, Ste 200, San Francisco, CA 94105 (Kimberly.Miller@iconplc.com).
Submitted for Publication: August 22, 2008; final revision received June 26, 2009; accepted June 30, 2009.
Author Contributions: Dr Miller had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Financial Disclosure: Dr Miller and Mr Mink are employed by ICON Clinical Research, which receives research funding from Allergan, Inc. Mr Walt is and Dr Satram-Hoang was employed by Allergan, Inc, the study sponsor. Drs Wilson, Perry, Asbell, and Pflugfelder are speakers for Allergan, Inc. Drs Asbell and Pflugfelder have received research funding from Allergan, Inc. Drs Perry, Asbell, and Pflugfelder have received research funding from competitors of Allergan, Inc. Drs Wilson, Perry, and Pflugfelder are consultants for Allergan, Inc. Dr Pflugfelder is a consultant for competitors of Allergan, Inc.
Funding/Support: This study was supported by Allergan, Inc.
Role of the Sponsor: Allergan, Inc had a role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; and in the preparation, review, and approval of the manuscript.
Additional Information: Allergan, Inc holds the copyright for the Ocular Surface Disease Index.
Additional Contributions: Carolyn A. Roberts, BA, assisted in the preparation of the manuscript.