Research Questions and Outcomes Prioritized by Patients With Dry Eye | External Eye Disease | JAMA Ophthalmology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Steps and Flow of Survey Participants, Research Questions, and Outcomes in This Study
Steps and Flow of Survey Participants, Research Questions, and Outcomes in This Study

NEI indicates National Eye Institute.

Figure 2.  Research Questions, Ratings of Importance, and Ranking in Round 1 of the Delphi Survey
Research Questions, Ratings of Importance, and Ranking in Round 1 of the Delphi Survey

IQR indicates interquartile range.

Figure 3.  Outcomes, Ratings of Importance, and Ranking in Round 2 of the Delphi Survey
Outcomes, Ratings of Importance, and Ranking in Round 2 of the Delphi Survey

IQR indicates interquartile range.

aThe 18 popular outcomes in existing research are ocular burning or stinging; ocular discomfort; ocular dryness; visual acuity; patient’s overall assessment of ocular surface symptoms; ocular foreign body sensation; tear film stability; ocular adverse events; artificial tear use; tear production or volume; corneal staining; nonocular adverse events; conjunctival hyperemia; ocular surface staining; conjunctival staining; conjunctival impression cytology; dryness of the mouth; and salivary flow.

bThe 10 unpopular outcomes in existing research are ocular pain; influence of dry eye disease on patient’s daily life; vision-related quality of life; patient’s acceptability or satisfaction with treatment; ocular gritty or sandy sensation; overall assessment of treatment effectiveness assessed by patients; ocular tiredness or fatigue; photosensitivity or photophobia; intolerance to air drafts; and treatment cost.

Figure 4.  Preferred Outcome Measurement Periods for All 28 Outcomes Rated in Round 2 of the Delphi Survey
Preferred Outcome Measurement Periods for All 28 Outcomes Rated in Round 2 of the Delphi Survey

aFor the 18 popular outcomes in existing research (ocular burning or stinging; ocular discomfort; ocular dryness; visual acuity; patient’s overall assessment of ocular surface symptoms; ocular foreign body sensation; tear film stability; ocular adverse events; artificial tear use; tear production or volume; corneal staining; nonocular adverse events; conjunctival hyperemia; ocular surface staining; conjunctival staining; conjunctival impression cytology; dryness of the mouth; and salivary flow), we obtained preferred measurement periods during Delphi round 1 (n = 622 patients). For each outcome, we allowed patients to indicate multiple measurement periods.

bFor the 10 unpopular outcomes in existing research (ocular pain; influence of dry eye disease on patient’s daily life; vision-related quality of life; patient’s acceptability or satisfaction with treatment; ocular gritty or sandy sensation; overall assessment of treatment effectiveness assessed by patients; ocular tiredness or fatigue; photosensitivity or photophobia; intolerance to air drafts; and treatment cost), we obtained preferred measurement periods during Delphi round 2 (n = 420 patients). For each outcome, we allowed patients to indicate multiple measurement periods.

Table.  Demographic and Clinical Characteristics of Survey Participants (ie, Patients With Dry Eye) Stratified by Delphi Round
Demographic and Clinical Characteristics of Survey Participants (ie, Patients With Dry Eye) Stratified by Delphi Round
1.
Stapleton  F, Alves  M, Bunya  VY,  et al.  TFOS DEWS II Epidemiology Report.  Ocul Surf. 2017;15(3):334-365. doi:10.1016/j.jtos.2017.05.003PubMedGoogle ScholarCrossref
2.
Sun  MJ, Rubin  GS, Akpek  EK, Ramulu  PY.  Impact of glaucoma and dry eye on text-based searching.  Transl Vis Sci Technol. 2017;6(3):24. doi:10.1167/tvst.6.3.24PubMedGoogle ScholarCrossref
3.
Uchino  M, Schaumberg  DA.  Dry eye disease: impact of quality of life and vision.  Curr Ophthalmol Rep. 2013;1(2):51-57. doi:10.1007/s40135-013-0009-1PubMedGoogle ScholarCrossref
4.
Gayton  JL.  Etiology, prevalence, and treatment of dry eye disease.  Clin Ophthalmol. 2009;3:405-412. doi:10.2147/OPTH.S5555PubMedGoogle ScholarCrossref
5.
Mathews  PM, Ramulu  PY, Swenor  BS, Utine  CA, Rubin  GS, Akpek  EK.  Functional impairment of reading in patients with dry eye.  Br J Ophthalmol. 2017;101(4):481-486. doi:10.1136/bjophthalmol-2015-308237PubMedGoogle ScholarCrossref
6.
van Landingham  SW, West  SK, Akpek  EK, Muñoz  B, Ramulu  PY.  Impact of dry eye on reading in a population-based sample of the elderly: the Salisbury Eye Evaluation.  Br J Ophthalmol. 2014;98(5):639-644. doi:10.1136/bjophthalmol-2013-303518PubMedGoogle ScholarCrossref
7.
Yu  T, Li  T, Lee  KJ, Friedman  DS, Dickersin  K, Puhan  MA.  Setting priorities for comparative effectiveness research on management of primary angle closure: a survey of Asia-Pacific clinicians.  J Glaucoma. 2015;24(5):348-355. doi:10.1097/IJG.0b013e31829e5616PubMedGoogle ScholarCrossref
8.
Schaumberg  DA, Nichols  JJ, Papas  EB, Tong  L, Uchino  M, Nichols  KK.  The international workshop on meibomian gland dysfunction: report of the subcommittee on the epidemiology of, and associated risk factors for, MGD.  Invest Ophthalmol Vis Sci. 2011;52(4):1994-2005. doi:10.1167/iovs.10-6997ePubMedGoogle ScholarCrossref
9.
Behrens  A, Doyle  JJ, Stern  L,  et al; Dysfunctional tear syndrome study group.  Dysfunctional tear syndrome: a Delphi approach to treatment recommendations.  Cornea. 2006;25(8):900-907. doi:10.1097/01.ico.0000214802.40313.faPubMedGoogle ScholarCrossref
10.
Craig  JP, Nelson  JD, Azar  DT,  et al.  TFOS DEWS II Report Executive Summary.  Ocul Surf. 2017;15(4):802-812. doi:10.1016/j.jtos.2017.08.003PubMedGoogle ScholarCrossref
11.
Saldanha  IJ, Dickersin  K, Hutfless  ST, Akpek  EK.  Gaps in current knowledge and priorities for future research in dry eye.  Cornea. 2017;36(12):1584-1591. doi:10.1097/ICO.0000000000001350PubMedGoogle ScholarCrossref
12.
Dean  S, Mathers  JM, Calvert  M,  et al.  “The patient is speaking”: discovering the patient voice in ophthalmology.  Br J Ophthalmol. 2017;101(6):700-708. doi:10.1136/bjophthalmol-2016-309955PubMedGoogle ScholarCrossref
13.
Friedman  NJ.  Impact of dry eye disease and treatment on quality of life.  Curr Opin Ophthalmol. 2010;21(4):310-316.PubMedGoogle Scholar
14.
Members of the Dry Eye WorkShop Research Subcommittee.  Research in dry eye: report of the Research Subcommittee of the International Dry Eye WorkShop (2007).  Ocul Surf. 2007;5(2):179-193. doi:10.1016/S1542-0124(12)70086-1PubMedGoogle ScholarCrossref
15.
Zarin  DA, Tse  T, Williams  RJ, Califf  RM, Ide  NC.  The ClinicalTrials.gov results database--update and key issues.  N Engl J Med. 2011;364(9):852-860. doi:10.1056/NEJMsa1012065PubMedGoogle ScholarCrossref
16.
Saldanha  IJ, Dickersin  K, Wang  X, Li  T.  Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability.  PLoS One. 2014;9(10):e109400. doi:10.1371/journal.pone.0109400PubMedGoogle ScholarCrossref
17.
The Dry Eye Zone. About. https://www.dryeyezone.com/about/. Accessed May 9, 2018.
18.
Alkhaffaf  B, Bruce  I, Glenny  A-M, Williamson  P, Blazeby  J, Morris  R.  Are the priorities of patients & researchers aligned in the reporting of outcomes in gastric cancer surgery trials?  Eur J Surg Oncol. 2017;43(11):2225. doi:10.1016/j.ejso.2017.10.143Google ScholarCrossref
19.
Le  JT, Viswanathan  S, Tarver  ME, Eydelman  M, Li  T.  Assessment of the incorporation of patient-centric outcomes in studies of minimally invasive glaucoma surgical devices.  JAMA Ophthalmol. 2016;134(9):1054-1056. doi:10.1001/jamaophthalmol.2016.2101PubMedGoogle ScholarCrossref
20.
Chalmers  I, Glasziou  P.  Avoidable waste in the production and reporting of research evidence.  Lancet. 2009;374(9683):86-89. doi:10.1016/S0140-6736(09)60329-9PubMedGoogle ScholarCrossref
21.
Glasziou  P, Altman  DG, Bossuyt  P,  et al.  Reducing waste from incomplete or unusable reports of biomedical research.  Lancet. 2014;383(9913):267-276. doi:10.1016/S0140-6736(13)62228-XPubMedGoogle ScholarCrossref
22.
Saldanha  IJ, Li  T, Yang  C, Owczarzak  J, Williamson  PR, Dickersin  K.  Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV/AIDS.  J Clin Epidemiol. 2017;84:85-94. doi:10.1016/j.jclinepi.2017.02.005PubMedGoogle ScholarCrossref
23.
Novack  GD, Asbell  P, Barabino  S,  et al.  TFOS DEWS II Clinical Trial Design Report.  Ocul Surf. 2017;15(3):629-649. doi:10.1016/j.jtos.2017.05.009PubMedGoogle ScholarCrossref
24.
Food and Drug Administration. Center for Drug Evaluation and Research. Approval package for application number 21-023 (trade name: Restasis). https://www.accessdata.fda.gov/drugsatfda_docs/nda/2003/21-023_Restasis_Approv.PDF. Accessed May 9, 2018.
25.
Food and Drug Administration. Center for Drug Evaluation and Research. Approval package for application number NDA 208073 (trade name: Xiidra). https://www.accessdata.fda.gov/drugsatfda_docs/appletter/2016/208073Orig1s000ltr.pdf. Accessed May 9, 2018.
26.
Sullivan  BD, Crews  LA, Messmer  EM,  et al.  Correlations between commonly used objective signs and symptoms for the diagnosis of dry eye disease: clinical implications.  Acta Ophthalmol. 2014;92(2):161-166. doi:10.1111/aos.12012PubMedGoogle ScholarCrossref
27.
O’Brien  PD, Collum  LM.  Dry eye: diagnosis and current treatment strategies.  Curr Allergy Asthma Rep. 2004;4(4):314-319. doi:10.1007/s11882-004-0077-2PubMedGoogle ScholarCrossref
28.
Lemp  MA.  Report of the national eye institute/industry workshop on clinical trials in dry eyes.  CLAO J. 1995;21(4):221-232.PubMedGoogle Scholar
29.
Williamson  PR, Altman  DG, Blazeby  JM,  et al.  Developing core outcome sets for clinical trials: issues to consider.  Trials. 2012;13:132. doi:10.1186/1745-6215-13-132PubMedGoogle ScholarCrossref
30.
Kirkham  JJ, Clarke  M, Williamson  PR.  A methodological approach for assessing the uptake of core outcome sets using ClinicalTrials.gov: findings from a review of randomised controlled trials of rheumatoid arthritis.  BMJ. 2017;357:j2262. doi:10.1136/bmj.j2262PubMedGoogle ScholarCrossref
31.
Williamson  PR, Altman  DG, Bagley  H,  et al.  The COMET Handbook: version 1.0.  Trials. 2017;18(suppl 3):280. doi:10.1186/s13063-017-1978-4PubMedGoogle ScholarCrossref
32.
Wolffsohn  JS, Arita  R, Chalmers  R,  et al.  TFOS DEWS II Diagnostic Methodology report.  Ocul Surf. 2017;15(3):539-574. doi:10.1016/j.jtos.2017.05.001PubMedGoogle ScholarCrossref
33.
Saldanha  IJ, Lindsley  K, Do  DV,  et al.  Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases.  JAMA Ophthalmol. 2017;135(9):933-940. doi:10.1001/jamaophthalmol.2017.2583PubMedGoogle ScholarCrossref
34.
Hashemi  H, Khabazkhoob  M, Kheirkhah  A,  et al.  Prevalence of dry eye syndrome in an adult population.  Clin Exp Ophthalmol. 2014;42(3):242-248. doi:10.1111/ceo.12183PubMedGoogle ScholarCrossref
35.
Hua  R, Yao  K, Hu  Y, Chen  L.  Discrepancy between subjectively reported symptoms and objectively measured clinical findings in dry eye: a population based analysis.  BMJ Open. 2014;4(8):e005296. doi:10.1136/bmjopen-2014-005296PubMedGoogle ScholarCrossref
36.
Uchino  M, Yokoi  N, Uchino  Y,  et al.  Prevalence of dry eye disease and its risk factors in visual display terminal users: the Osaka study.  Am J Ophthalmol. 2013;156(4):759-766. doi:10.1016/j.ajo.2013.05.040PubMedGoogle ScholarCrossref
Original Investigation
October 2018

Research Questions and Outcomes Prioritized by Patients With Dry Eye

Author Affiliations
  • 1Center for Evidence Synthesis in Health, Brown University School of Public Health, Providence, Rhode Island
  • 2Dry Eye Company LLC, Poulsbo, Washington
  • 3Consumers United for Evidence-Based Healthcare, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
  • 4Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
  • 5Reviews Editor, JAMA Ophthalmology
  • 6Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, Maryland
JAMA Ophthalmol. 2018;136(10):1170-1179. doi:10.1001/jamaophthalmol.2018.3352
Key Points

Question  What are patient priorities for future dry eye research?

Findings  A 6-step process (identifying research questions from a prior survey of clinicians and identifying outcomes from existing research, followed by a 2-round online Delphi survey of 420 patients with dry eye) was used to identify 8 research questions and 26 outcomes important to patients with dry eye. The top 3 questions pertained to effectiveness of patient education, environmental modifications, and topical anti-inflammatory drops, and the top 3 outcomes included ocular burning or stinging, ocular discomfort, and ocular pain.

Meaning  Results of this study may refine future research assessing dry eye treatments.

Abstract

Importance  Dry eye is a common ocular surface condition with significant influence on patient quality of life and societal economic burden. There is an urgent need to prioritize new research for dry eye.

Objective  To identify and rank research questions and outcomes important to patients with dry eye.

Design, Setting, and Participants  This study was conducted using the following 6 steps: (1) identifying research questions from a previous survey of clinicians who treat patients with dry eye; (2) identifying outcomes from existing research (systematic reviews and their cited clinical trials in the Cochrane Eyes and Vision US Satellite database of eyes and vision reviews, and National Eye Institute–funded clinical trials registered on ClinicalTrials.gov) as of June 13, 2017; (3) identifying a sample of patients with dry eye from the email subscribers to the online newsletter KeratoScoop; (4) and (5) conducting a 2-round Delphi survey of those patients online in November and December 2017, respectively; and (6) designating and ranking questions and outcomes as important.

Main Outcomes and Measures  Importance assigned to research questions and outcomes for dry eye. A research question or outcome ranked by at least 75% of patients as 6 or higher on a scale of 0 to 10 was considered important.

Results  Among the 420 patients from 15 countries who completed both rounds of the Delphi survey, most were 60 years of age or older (233 [56%]), female (348 [83%]), white (393 [94%]), and of non-Hispanic ethnicity (398 [95%]). Among the 12 questions that clinicians had previously prioritized, patients rated 8 as important. The top 3 questions pertained to effectiveness of patient education, environmental modifications, and topical anti-inflammatory eye drops for dry eye. Among the 109 outcomes identified in existing research on dry eye, patients rated 26 as important. Ten of these 26 were unpopular in existing research, with fewer than 10% of 158 studies reporting these outcomes. Of the 10 most important outcomes, 9 were associated with symptoms or quality of life. The 3 outcomes rated most important by patients were ocular burning or stinging, ocular discomfort, and ocular pain.

Conclusions and Relevance  This study identified research questions and outcomes important to patients with dry eye. A considerable gap was noted between outcomes in existing research on dry eye and outcomes patients consider important. Future research on dry eye should consider addressing the important research questions and outcomes identified herein, taking into account the patient perspective.

Introduction

Dry eye is a multifactorial ocular surface condition that occurs when tear film homeostasis is disturbed. One of the most frequent ocular conditions (prevalence 5% to 50% globally1), dry eye is more common among women and with increasing age.1 Dry eye negatively impacts quality of life1-4 and functional capacity, such as reading ability.5,6 Another major impact is economic: $3.8 billion per year is spent on managing this condition in the United States ($783 per patient).7 When productivity loss, physician visits, and other costs are considered, societal and per patient expenditures on dry eye approximate $55.4 billion per year and $11 302 per year, respectively.7

Current dry eye treatment algorithms are mostly based on expert opinion, rather than on reliable evidence of improvement in specific outcomes.8-10 This field urgently needs prioritization of new research so that effective treatments are brought to bear on patient care. We previously surveyed clinicians managing dry eye to identify their most important unanswered clinical questions.11 Most questions pertained to topical and other treatments already being used clinically.11 Dry eye research prioritization efforts must also incorporate patient perspectives,12 in part because assessments of both the disease process and treatment outcomes include the use of patient reports. In addition, patient voices must also inform outcome prioritization efforts.3,12-14 Our objective in the present study was to identify and rank research questions and outcomes important to patients with dry eye.

Methods

We followed the 6-step approach described below (eAppendix 1 in the Supplement). The Johns Hopkins Bloomberg School of Public Health Institutional Review Board (Baltimore, Maryland) approved this study. Written patient informed consent was obtained as part of the survey.

Step 1: Selecting Research Questions for Patient Prioritization

We previously identified 24 research questions (hereinafter referred to as questions) important to clinicians treating patients with dry eye.11 We selected the 12 questions rated as most important by the clinicians to be prioritized by patients in the present study.

Step 2: Selecting Outcomes for Patient Prioritization

A completely specified outcome has 5 elements: domain, specific measurement, specific metric, method of aggregation, and time points.15,16 In the present study, we focused on the domain (eg, visual acuity). For instances in which a study used a single instrument (eg, Ocular Surface Disease Index) to aggregate and present multiple domains, we considered the domain representing the aggregated outcome (eg, patient overall assessment of ocular surface symptoms). For instances in which a study separately considered the aggregated outcome and its individual domains (eg, ocular itching), we determined that the study had included both the domain representing the aggregated outcome and the individual domains. In other words, we deconstructed aggregated outcomes only when the study investigators did.

We identified all outcomes in existing research assessing interventions for dry eye reported as of June 13, 2017. We defined existing research as systematic reviews (hereinafter called reviews), published clinical trials, and National Eye Institute–funded trials registered on ClinicalTrials.gov (hereinafter called registered trials).

Outcomes in Reviews

We identified reviews assessing intervention effectiveness for dry eye through the Cochrane Eyes and Vision US Satellite database of eyes and vision reviews. This database, which supports our work with the American Academy of Ophthalmology’s Preferred Practice Patterns, includes both Cochrane and non-Cochrane reviews. We extracted each outcome named in each review’s Methods or Results sections.

Outcomes in Published Trials

We examined each published randomized trial that each review included. For each trial, we examined the journal article cited in the review, and when multiple journal articles were cited, we examined the first article cited. We extracted each outcome named in the Methods or Results section of each trial.

Outcomes in Registered Trials

We identified each outcome examined in National Eye Institute–funded trials registered on ClinicalTrials.gov. To be more inclusive of outcomes for rating by patients, we used a low cutoff to define popularity of an outcome. We defined an outcome as popular in existing research if 10% or more of studies (ie, reviews, published trials, and registered trials) examined it; otherwise, the outcome was defined as unpopular.

Step 3: Identifying Survey Participants (Patients With Dry Eye)

We surveyed subscribers to KeratoScoop, a weekly online newsletter created by one of us (R.P.), who is the sole proprietor and president of the Dry Eye Company LLC, which does not accept any commercial funding. The Dry Eye Company LLC includes the Dry Eye Zone, an online information portal for dry eye.17 Individuals typically subscribe to KeratoScoop after visiting Dry Eye Zone’s blog posts, webpages, or Facebook groups. While subscription to KeratoScoop is free and open to anyone, most subscribers are patients with dry eye. Given the absence of an established sampling theory for Delphi surveys, we did not conduct a priori sample size or power calculations.

Step 4: Conducting Delphi Round 1

In November 2017, we sent a website link for our survey (designed using Qualtrics survey software) by email to all current subscribers to KeratoScoop, an online newsletter generated by one of us (R.P.). We asked subscribers whether they or someone for whom they are a caregiver (eg, family member) were currently experiencing or had previously experienced dry eye, restricting participation in the present study to those who answered affirmatively. We asked caregivers to complete the survey on behalf of the person for whom they were providing care. For simplicity, we refer to all survey respondents hereinafter as patients. We followed the initial email with reminder emails 2 and 3 weeks later. We accepted responses up to 4 weeks after the first invitation.

Round 1 of the survey included 5 groups of items: (1) demonstration of understanding the purpose of the survey; (2) patient characteristics and email address; (3) ratings of importance of the questions prioritized by clinicians (eAppendix 2 in the Supplement); (4) ratings of importance of popular outcomes (eAppendix 3 in the Supplement); and (5) consideration of any of the unpopular outcomes as important. For outcomes, we also asked for preferred periods for measurement after starting treatment in a trial in the event the patient was to participate in a trial testing a new treatment of dry eye. The options included less than 3 months, 3 to 6 months, 7 to 12 months, more than 12 months, all periods, or no opinion (eAppendix 3 in the Supplement).

To facilitate patient comprehension and the provision of informed ratings, we accompanied all technical terms and concepts with lay language clarification. A long-term patient with dry eye (R.P.) helped develop the lay language.

Step 5: Conducting Delphi Round 2

For round 2 of the survey, we compiled the ratings of each outcome using a histogram (with the median) and an anonymized list of any patient comments regarding that outcome from round 1. In round 2, patients re-rated each outcome by taking into account their own response and those of their peers from round 1. Patients also provided ratings and preferred measurement periods for the 10 outcomes that were unpopular in existing research but most often preferred in round 1. In December 2017, one of us (I.J.S.) emailed to all patients who had completed round 1 a website link for round 2, with reminder emails sent 2 and 3 weeks later. We did not compensate patients for their participation.

Step 6: Designating and Ranking Important Research Questions and Outcomes

We analyzed the median, interquartile range (IQR), and range for each question and outcome, classifying as important all questions or outcomes that at least 75% of patients rated 6 or higher on a scale of 0 to 10 and classifying as moderately important all questions or outcomes that at least 75% of patients rated 5 or higher. To rank the questions or outcomes, we sorted them in decreasing order of the median, and when the median was tied, in decreasing order of the 25th percentile.

Results
Steps 1 and 2: Selecting Research Questions for Patient Prioritization and Selecting Outcomes for Patient Prioritization

For step 1, we selected the 12 highest-rated questions by the clinicians11 for rating by the patients. For step 2, we identified 20 systematic reviews, published between 2009 and 2017, inclusive, that examined 63 unique outcomes (median, 7.0 outcomes per review; IQR, 4.5-10.5). The reviews included 134 published trials (median, 6.5 trials per review; IQR, 3.0-13.5). The 134 trials presented in the included reviews, published between 1984 and 2015, inclusive, examined 96 unique outcomes (median, 6.0 outcomes per trial; IQR, 4.0-19.0).

We identified 4 registered trials, which examined 35 unique outcomes (median, 7.5 outcomes per trial; IQR, 3.8-11.2). Together, the published and registered trials examined 1.7 times as many unique outcomes as examined in the reviews (105 vs 63).

Across all 158 studies denoted as existing research for dry eye (ie, 20 reviews, 134 published trials, and 4 trials registered on ClinicalTrials.gov; eAppendix 4 in the Supplement), we identified 109 unique outcomes (eAppendix 5 in the Supplement). We organized the outcomes into the following 6 mutually exclusive categories: 35 symptoms, 28 signs or clinical testing, 29 laboratory measurements, 4 safety outcomes, 7 quality-of-life–related outcomes, and 6 other outcomes (eAppendix 5 in the Supplement). Among the 109 unique outcomes, we categorized 18 as popular and 91 as unpopular in existing research. The 18 popular outcomes included 6 of 35 symptoms, 7 of 28 signs or clinical testing, 2 of 29 laboratory measurements, 2 of 4 safety outcomes, and 1 of 6 other outcomes (eAppendix 6 in the Supplement). No popular outcomes were categorized as associated with quality of life.

Steps 3 to 5: Identifying Survey Participants and Conducting Delphi Rounds 1 and 2

We sent round 1 of the Delphi survey to 13 761 persons who were email subscribers to KeratoScoop (Figure 1). The email was opened by 4211 subscribers (31%), of which 741 subscribers (18%) clicked through to the survey website for further information and began round 1. Among those who accessed the website through the email link, 622 patients (84%) and 420 patients (57%) completed rounds 1 and 2, respectively. The 420 persons who completed round 2 resided in 15 countries and included 414 patients (99%) and 6 caregivers (1%).

The self-reported characteristics of patients were similar between those who completed rounds 1 and 2 (Table). Among the 420 patients completing round 2, most were 60 years of age or older (233 [56%]), female (348 [83%]), white individuals (393 [94%]), non-Hispanic ethnicity (398 [95%]), and currently residing in the United States (358 [85%]) or Canada (32 [8%]). More than two-thirds of the patients (290 [69%]) had been living with dry eye for 6 years or longer. Blepharitis was the most common underlying diagnosis (178 [43%]). In addition, 158 patients (38%) had received no underlying diagnosis (Table).

Step 6: Designating and Ranking Important Research Questions and Outcomes
Rating of Questions

Patients rated 8 of 12 questions as important and 4 of 12 as moderately important (Figure 2). Among the 8 questions rated as important, 4 (50%) addressed topical treatments, 1 (13%) an environmental intervention, 1 (13%) an educational intervention, 1 (13%) a systemic treatment, and 1 (13%) a general treatment. The 3 most important questions pertained to effectiveness of patient education, environmental modifications, and topical anti-inflammatory eye drops.

Ranking of Questions by Underlying Diagnosis

Most rankings did not appear to differ by patient subgroups defined by underlying diagnosis (Figure 2). For example, the question on patient education was top ranked by all subgroups, except by patients with rheumatoid arthritis, who ranked it second. When questions were specific to certain conditions (eg, autologous serum for Sjögren syndrome), they were top ranked by patients with those conditions.

Rating of Outcomes

The medians, IQRs, and ranges of ratings of outcomes in round 2 of the survey were very similar to those in round 1 (eAppendix 7 in the Supplement). In round 2, patients rated 28 outcomes (ie, 18 popular outcomes plus 10 unpopular outcomes selected in round 1). Figure 3 shows ratings for these 28 outcomes in round 2 in decreasing order of the 25th percentile. We classified 26 of 28 outcomes as important and 2 of 28 as not important. The 10 most important outcomes included 6 symptoms, 3 outcomes associated with quality of life, and 1 sign or clinical testing outcome. Among these 10 outcomes, 4 were unpopular in existing research. The 3 most important outcomes were ocular burning or stinging, ocular discomfort, and ocular pain.

Both outcomes classified as not important pertained to salivary function (dryness of the mouth and salivary flow), which is generally compromised in patients with Sjögren syndrome. Compared with patients without Sjögren syndrome, those with it assigned statistically significantly higher ratings for dryness of the mouth (median, 10.0; IQR, 3.8-11.2 vs median, 6.0; IQR, 3.0-8.0; P < .001) and for salivary flow (median, 9.0; IQR 7.0-10.0 vs median, 5.0; IQR, 2.0-8.0; P < .001).

Ranking of Outcomes by Underlying Diagnosis

Rank ordering of outcomes was consistent across diagnostic subgroups (Figure 3). Of note, salivary function outcomes (dryness of the mouth and salivary flow) classified by the overall group as not important were also ranked lowest by patients with Sjögren syndrome. Although patients with Sjögren syndrome assigned a high rating to these salivary function outcomes (see preceding paragraph), they rated outcomes pertaining to the eye higher.

Period Preferences for Outcome Measurements

For 12 of 28 outcomes (43%), and notably for 7 of 11 symptom outcomes (64%), 75% or more of all patients preferred measurement within 3 months of starting treatment in a trial (Figure 4). For all 28 outcomes, 50% or more of patients preferred measurement within 6 months. For some outcomes (eg, all 4 outcomes associated with quality of life), patients preferred measurement during all periods. For 9 of 10 outcomes rated lowest (eg, conjunctival staining and conjunctival impression cytology), fewer than 50% wanted measurement beyond 6 months after starting treatment (Figure 4).

Discussion

Using a 2-round Delphi survey of 420 patients with dry eye, we identified 8 important research questions and 26 important outcomes for dry eye. Patients rated effectiveness of patient education, environmental modifications, and topical anti-inflammatory eye drops as most important. The 10 highest-rated outcomes included 6 symptoms, 3 outcomes associated with quality of life, and 1 sign or clinical testing outcome. Among these top 10 outcomes, we determined 4 to be unpopular in existing research on dry eye. Given their preeminence to both treating clinicians and patients, the important questions and outcomes we identified should be considered in trials and reviews.

Our results add to a growing body of work showing that researchers often do not report outcomes important to patients, the most directly affected stakeholders.18-22 Among the 26 outcomes that patients deemed important in the present study, 10 were unpopular in existing research, suggesting that patients and researchers do not agree on what outcomes matter most or that patient relevance is not being adequately considered as a factor in the choice of outcomes for research or both. For example, the US Food and Drug Administration approved 2 drugs for dry eye based on evidence of improvement in 1 symptom and 1 sign in trials.23-25 However, there is only a limited correlation between symptoms and signs in dry eye.26-28 Such limited correlation and the evolution of newer clinical measurements might have, at least in part, accounted for the discrepancy in outcomes considered important by patients and researchers. Most funding agencies now encourage, and some require, inclusion of patient-important outcomes in trials. The findings of our study, by identifying patient-important outcomes for dry eye, could help trialists satisfy this requirement.

While diverging perspectives between researchers and patients are not unique to dry eye, they make a compelling and urgent case for developing a core outcome set for this burdensome and expensive condition. Core outcome sets are agreed on minimum sets of outcomes that should be reported by all trials in a given disease area.29 They promote consistency across trials, thereby facilitating evidence synthesis and evidence-based health care.30 By identifying 109 outcomes examined in existing research and prioritizing the 26 outcomes among them most important to patients, we have completed 2 early steps of core outcome set development.31 Multi-stakeholder consensus development efforts are now needed to narrow the list of 26 important outcomes into a core outcome set. Such efforts should also consider outcomes discussed in recent reports of the Tear Film and Ocular Surface Society.23,32

We also identified likely differences in perspectives among researchers studying dry eye, that is, discrepancies among those conducting trials, and discrepancies between those conducting trials and those conducting reviews. Indeed, there were 1.7 times as many unique outcomes across the trials than across the reviews. This finding is consistent with what we reported previously for 4 other prevalent eye conditions.33 In the present study, this multiplicity in outcomes likely accounted for the large proportion of unpopular outcomes among all outcomes in existing research (91 of 109 [83%]).

Strengths

First, to our knowledge, this is the only study to systematically engage patients with dry eye in determining priorities for research questions and outcomes. Second, the patients had received a range of underlying diagnoses. This enabled us to meaningfully examine whether the assigned ratings differed by underlying diagnosis. Third, we had a relatively large sample size, with 420 patients completing both Delphi rounds. Fourth, patients were predominantly older women, a population known to disproportionately experience dry eye.10 Fifth, most patients had lived with dry eye for a long time (69% for more than 6 years). This allowed more experience-informed ratings of the importance of questions and outcomes than that which might have been feasible with a sample of patients who had more recently received this diagnosis. Sixth, we used the Delphi method to conduct online surveys of geographically dispersed patients. The anonymity of responses likely promoted honest ratings that were unaffected by dominant voices, a common challenge during in-person group deliberations. A particular benefit of engaging patients exclusively was that their voices were not readily influenced by clinicians or other experts. Seventh, although the survey contained technical terms, we added detailed clarifying lay language. A long-term patient with dry eye (R.P.) helped develop this clarifying language to ensure survey accessibility. Finally, we also obtained from patients their preferences regarding when during trials for dry eye they would like each important outcome to be measured.

Limitations

First, it is possible that we missed some outcomes in existing randomized trials addressing dry eye because we searched only for trials included in systematic reviews and in the ClinicalTrials.gov registry. We did not conduct a comprehensive search of all dry eye trials. Second, perhaps because of the regionality of the KeratoScoop newsletter, only small proportions of patients were of Asian race (2%) or Hispanic ethnicity (4%). Because dry eye is also common in these populations,10,34-36 future research should examine whether our results apply to them. Third, although 31% of recipients of KeratoScoop opened our invitation email, only 18% of those who opened it clicked through to the website and began round 1 of the survey. However, of the 741 individuals who began round 1 of the survey, 84% and 68% completed rounds 1 and 2, respectively. Some factors might have contributed to survey noncompletion: (1) although we included detailed clarifying language for all technical terms, the survey’s complexity might have discouraged some patients, and (2) dry eye symptoms themselves might have interfered with survey completion. Finally, we identified all survey patients through the KeratoScoop newsletter. We do not know whether the priorities of patients with dry eye who do not subscribe to KeratoScoop are similar or different.

Conclusions

Through a 2-round Delphi survey of 420 patients with dry eye, we identified 8 research questions and 26 outcomes important to patients. Those conducting research and developing core outcome sets for dry eye should consider these priorities, which have also been informed by clinicians and existing research addressing dry eye.

Back to top
Article Information

Accepted for Publication: June 15, 2018.

Corresponding Author: Ian J. Saldanha, MBBS, MPH, PhD, Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice, Brown University School of Public Health, 121 S Main St, Box G-S121-8, Providence, RI 02912 (ian_saldanha@brown.edu).

Published Online: August 16, 2018. doi:10.1001/jamaophthalmol.2018.3352

Author Contributions: Dr Saldanha had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Saldanha, Dickersin, Akpek.

Acquisition, analysis, or interpretation of data: Saldanha, Petris, Han, Akpek.

Drafting of the manuscript: Saldanha.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Saldanha, Akpek.

Obtained funding: Dickersin.

Administrative, technical, or material support: Saldanha, Han, Akpek.

Supervision: Saldanha, Akpek.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Ms Petris reported being the sole proprietor and president of the Dry Eye Company LLC, a for-profit corporation that oversees the family of entities Dry Eye Zone (an information portal), Dry Eye Shop (an online store), Dry Eye Talk (a place for discussion among subscribers), Dry Eye Digest (Ms Petris’s blog), and KeratoScoop (Ms Petris’s newsletter). The Dry Eye Company does not accept any commercial funding; its profits are generated solely through product sales on the Dry Eye Shop. No other disclosures were reported.

Funding/Support: This work was funded by a grant (EY020140) from the National Eye Institute and a grant (U01 EY020522) from Cochrane Eyes and Vision, both to Dr Dickersin.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr Dickersin is the Reviews Editor of JAMA Ophthalmology, but she was not involved in any of the decisions regarding review of the manuscript or its acceptance.

Additional Contributions: We thank the patients and caregivers who participated in this study by responding to 1 or both rounds of the Delphi survey. Sueko Ng, MHS (Cochrane Eyes and Vision), Jimmy Le, MA (Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health), and Kai Sheng, BA (University of California, Berkeley) assisted with obtaining information about outcomes from existing research. All 3 individuals were compensated for their time.

References
1.
Stapleton  F, Alves  M, Bunya  VY,  et al.  TFOS DEWS II Epidemiology Report.  Ocul Surf. 2017;15(3):334-365. doi:10.1016/j.jtos.2017.05.003PubMedGoogle ScholarCrossref
2.
Sun  MJ, Rubin  GS, Akpek  EK, Ramulu  PY.  Impact of glaucoma and dry eye on text-based searching.  Transl Vis Sci Technol. 2017;6(3):24. doi:10.1167/tvst.6.3.24PubMedGoogle ScholarCrossref
3.
Uchino  M, Schaumberg  DA.  Dry eye disease: impact of quality of life and vision.  Curr Ophthalmol Rep. 2013;1(2):51-57. doi:10.1007/s40135-013-0009-1PubMedGoogle ScholarCrossref
4.
Gayton  JL.  Etiology, prevalence, and treatment of dry eye disease.  Clin Ophthalmol. 2009;3:405-412. doi:10.2147/OPTH.S5555PubMedGoogle ScholarCrossref
5.
Mathews  PM, Ramulu  PY, Swenor  BS, Utine  CA, Rubin  GS, Akpek  EK.  Functional impairment of reading in patients with dry eye.  Br J Ophthalmol. 2017;101(4):481-486. doi:10.1136/bjophthalmol-2015-308237PubMedGoogle ScholarCrossref
6.
van Landingham  SW, West  SK, Akpek  EK, Muñoz  B, Ramulu  PY.  Impact of dry eye on reading in a population-based sample of the elderly: the Salisbury Eye Evaluation.  Br J Ophthalmol. 2014;98(5):639-644. doi:10.1136/bjophthalmol-2013-303518PubMedGoogle ScholarCrossref
7.
Yu  T, Li  T, Lee  KJ, Friedman  DS, Dickersin  K, Puhan  MA.  Setting priorities for comparative effectiveness research on management of primary angle closure: a survey of Asia-Pacific clinicians.  J Glaucoma. 2015;24(5):348-355. doi:10.1097/IJG.0b013e31829e5616PubMedGoogle ScholarCrossref
8.
Schaumberg  DA, Nichols  JJ, Papas  EB, Tong  L, Uchino  M, Nichols  KK.  The international workshop on meibomian gland dysfunction: report of the subcommittee on the epidemiology of, and associated risk factors for, MGD.  Invest Ophthalmol Vis Sci. 2011;52(4):1994-2005. doi:10.1167/iovs.10-6997ePubMedGoogle ScholarCrossref
9.
Behrens  A, Doyle  JJ, Stern  L,  et al; Dysfunctional tear syndrome study group.  Dysfunctional tear syndrome: a Delphi approach to treatment recommendations.  Cornea. 2006;25(8):900-907. doi:10.1097/01.ico.0000214802.40313.faPubMedGoogle ScholarCrossref
10.
Craig  JP, Nelson  JD, Azar  DT,  et al.  TFOS DEWS II Report Executive Summary.  Ocul Surf. 2017;15(4):802-812. doi:10.1016/j.jtos.2017.08.003PubMedGoogle ScholarCrossref
11.
Saldanha  IJ, Dickersin  K, Hutfless  ST, Akpek  EK.  Gaps in current knowledge and priorities for future research in dry eye.  Cornea. 2017;36(12):1584-1591. doi:10.1097/ICO.0000000000001350PubMedGoogle ScholarCrossref
12.
Dean  S, Mathers  JM, Calvert  M,  et al.  “The patient is speaking”: discovering the patient voice in ophthalmology.  Br J Ophthalmol. 2017;101(6):700-708. doi:10.1136/bjophthalmol-2016-309955PubMedGoogle ScholarCrossref
13.
Friedman  NJ.  Impact of dry eye disease and treatment on quality of life.  Curr Opin Ophthalmol. 2010;21(4):310-316.PubMedGoogle Scholar
14.
Members of the Dry Eye WorkShop Research Subcommittee.  Research in dry eye: report of the Research Subcommittee of the International Dry Eye WorkShop (2007).  Ocul Surf. 2007;5(2):179-193. doi:10.1016/S1542-0124(12)70086-1PubMedGoogle ScholarCrossref
15.
Zarin  DA, Tse  T, Williams  RJ, Califf  RM, Ide  NC.  The ClinicalTrials.gov results database--update and key issues.  N Engl J Med. 2011;364(9):852-860. doi:10.1056/NEJMsa1012065PubMedGoogle ScholarCrossref
16.
Saldanha  IJ, Dickersin  K, Wang  X, Li  T.  Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability.  PLoS One. 2014;9(10):e109400. doi:10.1371/journal.pone.0109400PubMedGoogle ScholarCrossref
17.
The Dry Eye Zone. About. https://www.dryeyezone.com/about/. Accessed May 9, 2018.
18.
Alkhaffaf  B, Bruce  I, Glenny  A-M, Williamson  P, Blazeby  J, Morris  R.  Are the priorities of patients & researchers aligned in the reporting of outcomes in gastric cancer surgery trials?  Eur J Surg Oncol. 2017;43(11):2225. doi:10.1016/j.ejso.2017.10.143Google ScholarCrossref
19.
Le  JT, Viswanathan  S, Tarver  ME, Eydelman  M, Li  T.  Assessment of the incorporation of patient-centric outcomes in studies of minimally invasive glaucoma surgical devices.  JAMA Ophthalmol. 2016;134(9):1054-1056. doi:10.1001/jamaophthalmol.2016.2101PubMedGoogle ScholarCrossref
20.
Chalmers  I, Glasziou  P.  Avoidable waste in the production and reporting of research evidence.  Lancet. 2009;374(9683):86-89. doi:10.1016/S0140-6736(09)60329-9PubMedGoogle ScholarCrossref
21.
Glasziou  P, Altman  DG, Bossuyt  P,  et al.  Reducing waste from incomplete or unusable reports of biomedical research.  Lancet. 2014;383(9913):267-276. doi:10.1016/S0140-6736(13)62228-XPubMedGoogle ScholarCrossref
22.
Saldanha  IJ, Li  T, Yang  C, Owczarzak  J, Williamson  PR, Dickersin  K.  Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV/AIDS.  J Clin Epidemiol. 2017;84:85-94. doi:10.1016/j.jclinepi.2017.02.005PubMedGoogle ScholarCrossref
23.
Novack  GD, Asbell  P, Barabino  S,  et al.  TFOS DEWS II Clinical Trial Design Report.  Ocul Surf. 2017;15(3):629-649. doi:10.1016/j.jtos.2017.05.009PubMedGoogle ScholarCrossref
24.
Food and Drug Administration. Center for Drug Evaluation and Research. Approval package for application number 21-023 (trade name: Restasis). https://www.accessdata.fda.gov/drugsatfda_docs/nda/2003/21-023_Restasis_Approv.PDF. Accessed May 9, 2018.
25.
Food and Drug Administration. Center for Drug Evaluation and Research. Approval package for application number NDA 208073 (trade name: Xiidra). https://www.accessdata.fda.gov/drugsatfda_docs/appletter/2016/208073Orig1s000ltr.pdf. Accessed May 9, 2018.
26.
Sullivan  BD, Crews  LA, Messmer  EM,  et al.  Correlations between commonly used objective signs and symptoms for the diagnosis of dry eye disease: clinical implications.  Acta Ophthalmol. 2014;92(2):161-166. doi:10.1111/aos.12012PubMedGoogle ScholarCrossref
27.
O’Brien  PD, Collum  LM.  Dry eye: diagnosis and current treatment strategies.  Curr Allergy Asthma Rep. 2004;4(4):314-319. doi:10.1007/s11882-004-0077-2PubMedGoogle ScholarCrossref
28.
Lemp  MA.  Report of the national eye institute/industry workshop on clinical trials in dry eyes.  CLAO J. 1995;21(4):221-232.PubMedGoogle Scholar
29.
Williamson  PR, Altman  DG, Blazeby  JM,  et al.  Developing core outcome sets for clinical trials: issues to consider.  Trials. 2012;13:132. doi:10.1186/1745-6215-13-132PubMedGoogle ScholarCrossref
30.
Kirkham  JJ, Clarke  M, Williamson  PR.  A methodological approach for assessing the uptake of core outcome sets using ClinicalTrials.gov: findings from a review of randomised controlled trials of rheumatoid arthritis.  BMJ. 2017;357:j2262. doi:10.1136/bmj.j2262PubMedGoogle ScholarCrossref
31.
Williamson  PR, Altman  DG, Bagley  H,  et al.  The COMET Handbook: version 1.0.  Trials. 2017;18(suppl 3):280. doi:10.1186/s13063-017-1978-4PubMedGoogle ScholarCrossref
32.
Wolffsohn  JS, Arita  R, Chalmers  R,  et al.  TFOS DEWS II Diagnostic Methodology report.  Ocul Surf. 2017;15(3):539-574. doi:10.1016/j.jtos.2017.05.001PubMedGoogle ScholarCrossref
33.
Saldanha  IJ, Lindsley  K, Do  DV,  et al.  Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases.  JAMA Ophthalmol. 2017;135(9):933-940. doi:10.1001/jamaophthalmol.2017.2583PubMedGoogle ScholarCrossref
34.
Hashemi  H, Khabazkhoob  M, Kheirkhah  A,  et al.  Prevalence of dry eye syndrome in an adult population.  Clin Exp Ophthalmol. 2014;42(3):242-248. doi:10.1111/ceo.12183PubMedGoogle ScholarCrossref
35.
Hua  R, Yao  K, Hu  Y, Chen  L.  Discrepancy between subjectively reported symptoms and objectively measured clinical findings in dry eye: a population based analysis.  BMJ Open. 2014;4(8):e005296. doi:10.1136/bmjopen-2014-005296PubMedGoogle ScholarCrossref
36.
Uchino  M, Yokoi  N, Uchino  Y,  et al.  Prevalence of dry eye disease and its risk factors in visual display terminal users: the Osaka study.  Am J Ophthalmol. 2013;156(4):759-766. doi:10.1016/j.ajo.2013.05.040PubMedGoogle ScholarCrossref
×