Sakane Y, Yamaguchi M, Yokoi N, Uchino M, Dogru M, Oishi T, Ohashi Y, Ohashi Y. Development and Validation of the Dry Eye–Related Quality-of-Life Score Questionnaire. JAMA Ophthalmol. 2013;131(10):1331-1338. doi:10.1001/jamaophthalmol.2013.4503
A validated questionnaire that assesses symptoms and the effect of dry eye disease (DED) on the patient’s daily life is needed to evaluate the condition.
To develop and validate the Dry Eye–Related Quality-of-Life Score (DEQS) questionnaire in Japan.
Design, Setting, and Participants
Development and evaluation of the DEQS questionnaire were conducted in 4 phases: (1) item generation, (2) pilot study to assess content and face validity, (3) preliminary psychometric validation study to examine factor structure and reduce the number of items, and (4) final psychometric validation with 224 participants (203 with DED and 21 serving as controls). Participants completed the DEQS questionnaire, the Short Form-8, and the National Eye Institute Visual Function Questionnaire-25. To evaluate reproducibility and responsiveness, individuals with DED completed the DEQS questionnaire twice.
Main Outcomes and Measures
Internal consistency, reproducibility, discriminant validity, concurrent validity, and responsiveness.
Items were generated from a literature review, and 35 items were selected for the draft version. In the pilot study, 11 items were deleted; furthermore, 9 items were deleted based on the results of item analysis and factor analysis in the preliminary validation study. The final 15-item DEQS questionnaire consisted of an Overall Summary scale and 2 multi-item subscales: Impact on Daily Life and Bothersome Ocular Symptoms. The psychometric analysis revealed that the DEQS questionnaire has good internal consistency, test-retest reliability, discriminant validity, and responsiveness to change. The DEQS questionnaire correlated well with the mental component of the Short Form-8 and showed strong correlations with 4 subscales (Ocular Pain, Near Vision, Distance Vision, and Mental Health) of the National Eye Institute Visual Function Questionnaire-25.
Conclusions and Relevance
The DEQS questionnaire is valid and reliable for evaluating the multifaceted effect of DED on the patient’s daily life, including mental health, and it can be used easily in routine clinical practice.
Dry eye disease (DED) is a multifactorial disease of the tears and ocular surface that results in symptoms of discomfort, visual disturbance, and tear film instability, as well as potential damage to the ocular surface.1 The report of the National Eye Institute/Industry Workshop on Clinical Trials in Dry Eyes2 in 1995 emphasized the requirement of the presence of symptoms in the definition of DED, but this requirement was absent from the Japanese definition until recently.3 In 2006, the Japanese Dry Eye Society modified the definition and diagnostic criteria of DED.4 Because the new definition and diagnostic criteria incorporated symptoms, reliable assessment of symptoms has become necessary in Japan.
The diverse symptoms and visual disturbance of DED have a considerable effect on the daily and social lives of patients. Miljanović et al5 reported that, compared with patients without DED, those with DED were affected more negatively while performing everyday activities such as reading, computer use, professional work, driving, and watching television. Patients with DED seek medical treatment to improve the irritative ocular symptoms that affect their quality of life (QOL). The goals of treating DED are improving the patient’s ocular comfort and QOL and restoring the ocular surface and tear film to the normal homeostatic state.6
Many studies7- 9 have shown a lack of correlation between symptoms and clinical findings, such as Schirmer testing and fluorescein staining. This discrepancy can be explained by the natural variability of the disease process, the “subjective” nature of symptoms, and the variability in pain thresholds and cognitive responses to questions about physical sensations in the eyes.10
Regarding diagnosis and evaluating the therapeutic effect, clinicians must consider the patient’s reported symptoms as well as the clinical signs. Therefore, a validated questionnaire that assesses symptoms, vision-related function, and the effect of DED on QOL is needed.11,12
The currently validated DED-specific questionnaires, the Ocular Surface Disease Index (OSDI) and the Impact of Dry Eye on Everyday Life (IDEEL), can be used for measuring DED severity and patient-reported outcomes in clinical trials. The OSDI contains 12 questions and can assess the frequency of DED symptoms and their effect on vision-related functioning.13 Although the OSDI is widely used for diagnosis and evaluation of symptom severity, it does not fully cover the effect of DED on patients’ everyday lives, such as psychological and social aspects. The IDEEL, which is a 57-item questionnaire, covers all relevant domains of DED.14 However, the IDEEL requires approximately 30 minutes for completion, and it is not easy to use in routine clinical practice. The purpose of this study was to develop a new questionnaire that can be used easily in routine clinical practice and to evaluate the multifaceted effect of DED on the patient’s daily life.
A series of studies was conducted at 3 facilities: Ehime University School of Medicine, Kyoto Prefectural University of Medicine, and Ryogoku Eye Clinic. The study participants were recruited at the time of routine outpatient visits at each institution. The studies were conducted in compliance with the tenets of the Declaration of Helsinki, and written informed consent was obtained from all participants. The ethics committee at each hospital approved this research.
The diagnostic criteria used in the studies conformed to those defined by the Japanese Dry Eye Society in 2006 (Table 1).4 The research group included 6 dry eye expert clinicians (Y.S., M.Y., N.Y., M.U., M.D., and Yuichi O.), a statistician (Yasuo O.), and an outcomes research scientist (T.O.). They provided expertise and participated extensively in the decisions throughout the process of developing and finalizing the questionnaire.
First, we created an item pool. Items were generated from a literature review of existing ocular surface disease–specific questionnaires.13- 15 The symptoms that patients with DED often reported in our daily practice were also added to the item pool. Subsequently, we selected items based on clinical importance, relevance to DED, and importance from a patient’s perspective.
A pilot study was conducted to evaluate the content and face validity of the questionnaire. The eligible individuals were aged 20 years or older and had a confirmed diagnosis of DED. First, the participants completed the first draft of the Dry Eye–Related Quality-of-Life Score (DEQS) questionnaire. The participants were subsequently interviewed and the following points were addressed: the participants’ general impressions, the comprehensiveness of the questionnaire, the clarity of the instructions and the items/response choices, the readability of the format and layout, and the participants’ interpretations of and opinions regarding each question. They also were asked whether all major aspects relevant to their DED-related QOL were included in the questionnaire. The time required to complete the questionnaire was recorded.
A preliminary validation study was performed to examine factor structure and to reduce the number of items. The eligible individuals were aged 20 years or older, were literate in Japanese, and demonstrated an adequate cognitive state to complete the questionnaire. People with DED had to have received a diagnosis of either definite or probable DED according to the Japanese DED criteria.4 They also had to have an abnormality in tear function and symptoms associated with DED. Individuals serving as controls were recruited from among those who did not have significant ocular disease and abnormality in tear function. Participants who had an uncontrolled disease or disability that affected their activities of daily living (including ocular allergy) or who had undergone ocular surgery within the previous 6 months were excluded.
All participants completed the second draft of the DEQS questionnaire, which first assesses the frequency of symptoms and disability and then assesses the degree. We adopted 2-step scales because answering questions regarding the frequency of symptoms appears to be easier than responding directly to the degree scale. The frequency was scored on a 5-point Likert scale ranging from 0 to 4, where 0 indicates that the respondent does not have the symptom and 4 indicates the highest frequency. The degree was scored on a 4-point Likert scale ranging from 1 to 4, with a larger number indicating a greater burden. When the score of the frequency scale is 0, the score of the degree scale is also 0. We used the degree score because we believed that it represents the patient’s burden and is more appropriate for evaluating the effect of DED on daily life. The summary score was calculated with the following formula: summary score = ([sum of the degree scores for all questions answered] × 25)/(total number of questions answered). The summary score ranges from 0 to 100, with a higher score representing greater disability. Subscale scores were computed similarly by using only the items from each subscale. To determine the factor structure and reduce the number of items, statistical consideration based on item analysis and factor analysis, as well as the clinical relevance of items, was discussed.
A validation study was conducted to verify the psychometric validity and reliability of the final version of the DEQS questionnaire. We evaluated the item analysis, factor analysis, reliability, validity, and responsiveness.
The eligible individuals were aged 20 years or older and were literate in Japanese. Participants with DED had to have received the diagnosis as either definite or probable DED by the Japanese DED criteria.4 They also had to have an abnormality in tear function, a tear film breakup time (BUT) of 5 seconds or less, and symptoms due to DED at the time of enrollment. Participants with DED consisted of 2 groups: a retest group and a punctal plug group. Those in the retest group were recruited from patients who had no change in treatment at the time of enrollment. Participants in the punctal plug group were recruited from patients who underwent punctal plug occlusion at the time of enrollment. Controls had to have a BUT of 6 seconds or more and a fluorescein staining score of 2 points or less (range, 0-9 points) in both eyes at the time of enrollment. In the control group, elderly women were predominately recruited to ensure that the demographics were as close as possible to those of the patients with DED. Exclusion criteria were the same as those described for the preliminary validation study. All participants were able to answer the questionnaire without assistance.
Participants completed 3 questionnaires in the following order: the final 15-item version of the DEQS questionnaire, a Japanese version of the Short Form-8 Health Survey (SF-8),16 and a Japanese version of the 25-item National Eye Institute Visual Function Questionnaire (VFQ-25). The SF-8 is a measure of general health status, and the VFQ-25 was found to be reliable and valid across several common eye diseases.17,18 To evaluate the test-retest reliability, individuals in the retest group were asked to complete the DEQS questionnaire at home 2 weeks after the first test and to return it by mail.
Conjunctival and corneal fluorescein staining and BUT clinical tests were performed to assess the correlation between the DEQS questionnaire and clinical findings. To evaluate fluorescein staining of the ocular surface, the eye was divided into 3 equal compartments representing the nasal conjunctiva, cornea, and temporal conjunctiva. The maximum staining score for each area was 3 points, and the maximum staining score for the overall surface was 9 points. Regarding Schirmer 1 testing (without anesthesia), we included any available data measured within 6 months before the time of enrollment.
Item analysis was conducted by using whole data to examine the percentage of missing values and the item distribution. Items with high levels of missing data (>20%) were considered for deletion.
Factor analysis was used to determine the factor structure. Factors with an eigenvalue greater than 1 were retained.19 Under the designated number of factors, factor analysis was carried out with the maximum-likelihood solution and promax rotation. The threshold level for factor loading was 0.40.
Internal consistency, reliability, and reproducibility (test-retest reliability) were examined. Internal consistency was measured using Cronbach α, in which a score of 0.7 or higher is acceptable.20 Test-retest reliability was evaluated by examining the intraclass correlation coefficients between the first and second entry (2 weeks later). Patients who reported stability in their DED symptoms in the previous 2 weeks were included the analysis. An intraclass correlation coefficient of 0.70 or more was considered acceptable for test-retest reliability.21
Discriminant validity was assessed by a known group comparison between the DED group and the control group. Concurrent validity was computed by correlating (Pearson coefficients) DEQS questionnaire scores with the other health measures, namely, the SF-8 and VFQ-25. Concurrent validity was supported if similar domains or subscales in the SF-8 and VFQ-25 were substantially correlated with the DEQS questionnaire.22 Additionally, correlations with clinical measures, such as the fluorescein staining score, BUT, and Schirmer score, were examined.
The scores before and after the punctal plug insertion treatment were compared by using the 2-tailed paired t test to evaluate responsiveness. The significance level was set at .05.
We created an item pool of 45 items and selected 35 items for the first draft of the questionnaire (the first draft DEQS questionnaire). Items were collected from existing ocular surface disease–specific questionnaires and DED experts’ opinions.
A pilot study was conducted with 20 participants with DED. The mean time to complete the first DEQS draft was 9 minutes 19 seconds. Items that were not specific to DED, overlapped with other items, and had a confusing representation were removed, thus reducing the number of items from 35 to 24. Furthermore, the layout of the questionnaire was improved based on the participant’s opinions.
This study included 142 participants: 112 patients with DED and 30 controls. Based on the results of the item analysis and factor analysis, 9 items were removed from the second draft, 24-item DEQS questionnaire: 2 items that showed no significant difference from the control group, 4 items related to environmental triggers of DED (windy setting, use of air conditioning, driving, and contact lens wear) for which more than 60% of the patients answered “not applicable,” and 3 items that were determined to be removable by factor analysis.
The demographics of the validation study participants are presented in Table 2. The validation study included 203 patients with DED and 21 controls. The mean ages for the DED and control groups were 63.9 and 71.0 years, respectively, which were significantly different (P = .002). Most of the population was female. The following scores were obtained from the clinical DED tests for the DED participants: mean fluorescein staining score, 2.7 (range, 0-9); BUT, 1.94 seconds (range, 0-5); and Schirmer testing, 5.3 mm (range, 0-29). In the DED group, 63 patients (31.0%) had comorbid Sjögren syndrome.
The mean score of all items was significantly higher in the DED group than in the control group (Table 3). The missing data rate of each item was less than 4.8%.
Factor analysis revealed the presence of 2 subscales (Table 3), interpreted as Bothersome Ocular Symptoms (6 items) and Impact on Daily Life (9 items).
The results of internal consistency and test-retest reliability are reported in Table 4. Regarding internal consistency, Cronbach α values of Impact on Daily Life, Bothersome Ocular Symptoms, and the summary score were 0.93, 0.83, and 0.93, respectively, and all exceeded the recommended level of 0.7. Test-retest reliability was evaluated with 116 DED participants with a mean (SD) period between the test and retest of 14.2 (1.5) days (range, 8-21 days). The intraclass correlation coefficients, which represent reproducibility, ranged from 0.81 to 0.93, and all surpassed the acceptable level of 0.7.
The discriminant validity of the DEQS questionnaire is reported in Table 5. The summary score and each subscale score were significantly higher in the DED group than in the control group (P < .001). In addition, both SF-8 components and all the VFQ-25 subscales except Social Function and Color Vision showed significant differences between the DED and control groups (Table 5). The mental component of the SF-8 and 4 subscales of the VFQ-25 (Ocular Pain, Distance Vision, Mental Health, and Dependency) showed high statistical significance (P < .001).
The correlations between the DEQS questionnaire and the SF-8 are reported in Table 6. The summary score and each subscale score of the DEQS questionnaire significantly correlated with each component summary of the SF-8 (physical and mental component summaries). The correlation coefficients between the mental component summary of the SF-8 and each DEQS score ranged from –0.43 to –0.52. All DEQS questionnaire scores significantly correlated with all subscale scores and the total score of the VFQ-25. The correlation coefficients of the DEQS questionnaire and the VFQ-25 were in the range of −0.20 to −0.77 (Table 6). Among these subscales, 4 subscales and the total score of the VFQ-25 showed a high (r > |0.6|) correlation with the DEQS questionnaire scores: Near Vision and Distance Vision correlated with Impact on Daily Life and Mental Health, and the total score correlated with the summary score and Impact on Daily Life, and Ocular Pain correlated with all DEQS questionnaire scores.
Correlations between the clinical measures (including fluorescein staining score, BUT, and Schirmer score) and the summary score and each subscale score of the DEQS ranged from −0.19 to 0.12 (Table 6).
Responsiveness was evaluated in 10 patients with DED who underwent punctal plug insertion. Both clinical signs and all DEQS questionnaire scores (summary and 2 subscale scores) showed significant improvement after treatment (Table 7).
The bothersome symptoms and visual disturbance associated with DED were reported to have a negative effect on the patient’s QOL.5,11,12,23,24 Several studies have also shown that DED symptoms and their effect on QOL were poorly associated with clinical findings.7- 9,25 Thus, to evaluate the condition of DED and therapeutic effects, a validated questionnaire that assesses symptoms and the effect of DED on QOL is as important as clinical findings. The DEQS questionnaire was developed to evaluate the symptoms and effect of DED on the patient’s daily life and can be used easily in routine clinical practice. We developed the DEQS questionnaire by referring to the Food and Drug Administration guidelines for patient-reported outcome measures26 to use the DEQS questionnaire also for patient-reported outcome in clinical studies.
The DEQS questionnaire consists of 15 items and 2 subscales: Impact on Daily Life and Bothersome Ocular Symptoms. In the present study, the DEQS questionnaire showed both high internal consistency and good to excellent test-retest reliability. The DEQS questionnaire scores also correlated well with other health status measures, such as the VFQ-25 and the SF-8. The VFQ-25 is a non–disease-specific instrument designed to evaluate QOL and to assess the treatment of ocular disease.17,18 The SF-8 is a shortened version of the SF-36,16 which is used as a comprehensive QOL measurement aimed at the concept of general health regardless of the type of disease or condition. The VFQ-25 and the SF-36 have been used in many studies11,23- 25,27,28 to evaluate the effect of DED on the patient’s QOL. Each component score of the SF-8 correlated with the DEQS questionnaire, especially, the mental component summary. The DEQS questionnaire also correlated with each subscale score of the VFQ-25. As expected, Ocular Pain, which is only a subscale of the VFQ-25 and is apparently related to the typical symptoms of DED, showed strong correlations with the summary score and each subscale score of the DEQS questionnaire. In addition, a subscale of the DEQS questionnaire that represents effect on daily life had strong correlations with 3 subscales: Near Vision, Distance Vision, and Mental Health. This suggested that the Impact on Daily Life subscale evaluated vision-related QOL and mental health efficiently. The above results verified the concurrent validity of the DEQS questionnaire, and it was shown that the DEQS questionnaire can evaluate the effect of DED on the patient’s QOL, including mental health.
The discriminant validity of the DEQS questionnaire was verified from the finding that all DEQS questionnaire scores were significantly higher in the DED group than in the control group. In addition, both component scores of the SF-8 and all of the VFQ-25 subscale scores except Social Function and Color Vision showed significant differences between the DED and control groups. In particular, the mental health component of the SF-8 and 4 subscales of the VFQ-25 (Ocular Pain, Distance Vision, Mental Health, and Dependency) showed high statistical significance. Low VFQ-25 scores in patients with DED have been reported in many studies.23- 25,27,28 Additionally, several recent studies have shown that patients with DED had impaired mental health,28- 30 which might be explained by low subscale scores related to the mental health of these patients in both the SF-8 and VFQ-25 scales.
In this study, we recruited as many elderly women to the control group as possible to match the DED participants because most patients with DED are elderly women.10 Therefore, the mean age of the DED group was significantly lower than that of the control group. It was anticipated that this age difference might affect the validity of the discrimination of the DED and control groups because the physical function–related QOL might be impaired with age. However, the discriminant validity was verified sufficiently, suggesting that this age difference had little effect.
Correlations between the clinical measures and the DEQS questionnaire scores were relatively low. Similar disagreements between clinical findings and symptoms of DED have been reported.7- 9 We also examined whether the effect of treatment can be evaluated by using the DEQS questionnaire. All DEQS questionnaire scores and clinical findings significantly improved after punctal plug insertion, suggesting that the DEQS questionnaire is appropriate for evaluating changes in the DED condition and the therapeutic effect.
In the DEQS questionnaire, 2 scales are used to assess the frequency and degree of symptoms and disability. We decided to use the degree score, which represents the patient’s burden, because we believed that it was more appropriate for assessing the effect of DED on daily life. Although the frequency score was not used, the frequency score, as well as the degree score of all items, showed significant differences between the DED and control groups. Whether the frequency score can be applied to improve the DEQS questionnaire further is an issue that should be investigated in future studies. The DEQS questionnaire was validated only in Japan. Therefore, translations and cross-cultural adaptation will be necessary for using it in other countries.
In conclusion, the DEQS questionnaire has good reliability, validity, specificity, and responsiveness and is a useful instrument for assessing the severity of DED symptoms and their effect on QOL. A combination of clinical findings and the DEQS questionnaire may assist in routine clinical practice associated with DED.
Submitted for Publication: January 18, 2013; final revision received March 22, 2013; accepted March 25, 2013.
Corresponding Author: Yuri Sakane, MD, Department of Ophthalmology, Ehime University School of Medicine, Shitsukawa, Toon, Ehime 791-0295, Japan.
Published Online: August 15, 2013. doi:10.1001/jamaophthalmol.2013.4503.
Author Contributions: Dr Sakane had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Sakane, Yamaguchi, Dogru, Oishi, Yasuo Ohashi, Yuichi Ohashi.
Acquisition of data: Sakane, Yamaguchi, Yokoi, Uchino.
Analysis and interpretation of data: Sakane, Yamaguchi, Dogru, Oishi, Yasuo Ohashi.
Drafting of the manuscript: Sakane, Yokoi, Yuichi Ohashi.
Critical revision of the manuscript for important intellectual content: Sakane, Yamaguchi, Uchino, Dogru, Oishi, Yasuo Ohashi.
Statistical analysis: Sakane, Oishi, Yasuo Ohashi.
Administrative, technical, and material support: Uchino, Dogru, Oishi.
Study supervision: Dogru, Oishi, Yasuo Ohashi, Yuichi Ohashi.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by Santen Pharmaceutical Co, Ltd.
Role of the Sponsor: Santen Pharmaceutical Co, Ltd had a role in the following areas: generating the development concepts; design of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript.
Additional Contributions: Santen Pharmaceutical Co, Ltd and the Japanese Dry Eye Society hold the copyright for the DEQS questionnaire.