A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment | Neonatology | JAMA Ophthalmology | JAMA Network
[Skip to Content]
[Skip to Content Landing]
Figure 1.  Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Laser
Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Laser

The infant was born at 25 weeks 3 days gestational age and developed treatment-requiring disease. The automated retinopathy of prematurity severity scores are 5.91 (A), 8.01 (B), and 6.99 (C).

Figure 2.  Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Intravitreal Bevacizumab
Serial Wide-Angle Retinal Images and Corresponding Clinical Diagnosis From Deep Learning in an Infant Treated With Intravitreal Bevacizumab

The infant was born at 24 weeks 5 days gestational age and developed treatment-requiring disease. The automated retinopathy of prematurity severity scores are 6.21 (A), 8.94 (B), and 6.09 (C).

Figure 3.  Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Progression and Regression in a Cohort of Infants Who Developed Treatment-Requiring Disease
Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Progression and Regression in a Cohort of Infants Who Developed Treatment-Requiring Disease

Mean scores are displayed relative to time of treatment (which is time 0). Number of clinical examinations (n) evaluated at each time is displayed. Error bars indicate 95% CIs.

Figure 4.  Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants Who Were Treated With Laser Photocoagulation vs Antivascular Endothelial Growth Factor (Anti-VEGF) Agents
Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants Who Were Treated With Laser Photocoagulation vs Antivascular Endothelial Growth Factor (Anti-VEGF) Agents

Mean scores are displayed 4 weeks prior to, before, and after treatment. Error bars indicate 95% CIs.

Figure 5.  Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants With Treatment-Requiring Disease Who Received Laser Photocoagulation and Who Required Retreatment With Laser vs Those Who Required No Retreatment
Quantitative Retinopathy of Prematurity (ROP) Vascular Severity Score Distribution in a Cohort of Infants With Treatment-Requiring Disease Who Received Laser Photocoagulation and Who Required Retreatment With Laser vs Those Who Required No Retreatment

Mean scores are displayed 4 weeks before and after treatment, and the number of clinical examinations is shown at each time. At treatment and 1, 2, and 3 weeks after treatment, retreated eyes had significantly higher ROP vascular severity scores than eyes that never required retreatment. Error bars indicate 95% CIs.

aP < .05.

1.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Arch Ophthalmol. 1988;106(4):471-479. doi:10.1001/archopht.1988.01060130517027PubMedGoogle ScholarCrossref
2.
Gilbert  C.  Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control.  Early Hum Dev. 2008;84(2):77-82. doi:10.1016/j.earlhumdev.2007.11.009PubMedGoogle ScholarCrossref
3.
The Committee for the Classification of Retinopathy of Prematurity.  An international classification of retinopathy of prematurity.  Arch Ophthalmol. 1984;102(8):1130-1134. doi:10.1001/archopht.1984.01040030908011PubMedGoogle ScholarCrossref
4.
International Committee for the Classification of Retinopathy of Prematurity.  The international classification of retinopathy of prematurity revisited.  Arch Ophthalmol. 2005;123(7):991-999. doi:10.1001/archopht.123.7.991PubMedGoogle ScholarCrossref
5.
Early Treatment For Retinopathy Of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial.  Arch Ophthalmol. 2003;121(12):1684-1694. doi:10.1001/archopht.121.12.1684PubMedGoogle ScholarCrossref
6.
Mintz-Hittner  HA, Kennedy  KA, Chuang  AZ; BEAT-ROP Cooperative Group.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity.  N Engl J Med. 2011;364(7):603-615. doi:10.1056/NEJMoa1007374PubMedGoogle ScholarCrossref
7.
Fierson  WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists.  Screening examination of premature infants for retinopathy of prematurity.  Pediatrics. 2018;142(6):e20183061. doi:10.1542/peds.2018-3061PubMedGoogle ScholarCrossref
8.
Reynolds  JD, Dobson  V, Quinn  GE,  et al; CRYO-ROP and LIGHT-ROP Cooperative Study Groups.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies.  Arch Ophthalmol. 2002;120(11):1470-1476. doi:10.1001/archopht.120.11.1470PubMedGoogle ScholarCrossref
9.
Mintz-Hittner  HA.  Retinopathy of prematurity: intravitreal injections of bevacizumab: timing, technique, and outcomes.  J AAPOS. 2016;20(6):478-480. doi:10.1016/j.jaapos.2016.10.002PubMedGoogle ScholarCrossref
10.
Darwish  D, Chee  R-I, Patel  SN,  et al.  Anti-Vascular endothelial growth factor and the evolving management paradigm for retinopathy of prematurity.  Asia Pac J Ophthalmol (Phila). 2018;7(3):136-144. doi:10.22608/APO.201850PubMedGoogle Scholar
11.
Fleck  BW, Williams  C, Juszczak  E,  et al; BOOST II Retinal Image Digital Analysis (RIDA) Group.  An international comparison of retinopathy of prematurity grading performance within the Benefits of Oxygen Saturation Targeting II trials.  Eye (Lond). 2018;32(1):74-80. doi:10.1038/eye.2017.150PubMedGoogle ScholarCrossref
12.
Campbell  JP, Kalpathy-Cramer  J, Erdogmus  D,  et al; Imaging and Informatics in Retinopathy of Prematurity Research Consortium.  Plus disease in retinopathy of prematurity: a continuous spectrum of vascular abnormality as a basis of diagnostic variability.  Ophthalmology. 2016;123(11):2338-2344. doi:10.1016/j.ophtha.2016.07.026PubMedGoogle ScholarCrossref
13.
Kalpathy-Cramer  J, Campbell  JP, Erdogmus  D,  et al; Imaging and Informatics in Retinopathy of Prematurity Research Consortium.  Plus disease in retinopathy of prematurity: improving diagnosis by ranking disease severity and using quantitative image analysis.  Ophthalmology. 2016;123(11):2345-2351. doi:10.1016/j.ophtha.2016.07.020PubMedGoogle ScholarCrossref
14.
Campbell  JP, Ataer-Cansizoglu  E, Bolon-Canedo  V,  et al; Imaging and Informatics in ROP (i-ROP) Research Consortium.  Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis.  JAMA Ophthalmol. 2016;134(6):651-657. doi:10.1001/jamaophthalmol.2016.0611PubMedGoogle ScholarCrossref
15.
Chiang  MF, Thyparampil  PJ, Rabinowitz  D.  Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity.  Arch Ophthalmol. 2010;128(9):1153-1159. doi:10.1001/archophthalmol.2010.199PubMedGoogle ScholarCrossref
16.
Chiang  MF, Jiang  L, Gelman  R, Du  YE, Flynn  JT.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.  Arch Ophthalmol. 2007;125(7):875-880. doi:10.1001/archopht.125.7.875PubMedGoogle ScholarCrossref
17.
Campbell  JP, Ryan  MC, Lore  E,  et al; Imaging & Informatics in Retinopathy of Prematurity Research Consortium.  Diagnostic discrepancies in retinopathy of prematurity classification.  Ophthalmology. 2016;123(8):1795-1801. doi:10.1016/j.ophtha.2016.04.035PubMedGoogle ScholarCrossref
18.
Quinn  GE, Ells  A, Capone  A  Jr,  et al; e-ROP (Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity) Cooperative Group.  Analysis of discrepancy between diagnostic clinical examination findings and corresponding evaluation of digital images in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity study.  JAMA Ophthalmol. 2016;134(11):1263-1270. doi:10.1001/jamaophthalmol.2016.3502PubMedGoogle ScholarCrossref
19.
Scott  KE, Kim  DY, Wang  L,  et al.  Telemedical diagnosis of retinopathy of prematurity intraphysician agreement between ophthalmoscopic examination and image-based interpretation.  Ophthalmology. 2008;115(7):1222-1228. doi:10.1016/j.ophtha.2007.09.006PubMedGoogle ScholarCrossref
20.
Mintz-Hittner  HA, Geloneck  MM, Chuang  AZ.  Clinical management of recurrent retinopathy of prematurity after intravitreal bevacizumab monotherapy.  Ophthalmology. 2016;123(9):1845-1855. doi:10.1016/j.ophtha.2016.04.028PubMedGoogle ScholarCrossref
21.
Mueller  B, Salchow  DJ, Waffenschmidt  E,  et al.  Treatment of type I ROP with intravitreal bevacizumab or laser photocoagulation according to retinal zone.  Br J Ophthalmol. 2017;101(3):365-370. doi:10.1136/bjophthalmol-2016-308375PubMedGoogle Scholar
22.
Wittenberg  LA, Jonsson  NJ, Chan  RVP, Chiang  MF.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity.  J Pediatr Ophthalmol Strabismus. 2012;49(1):11-19. doi:10.3928/01913913-20110222-01PubMedGoogle ScholarCrossref
23.
Wallace  DK.  Computer-assisted quantification of vascular tortuosity in retinopathy of prematurity (an American Ophthalmological Society thesis).  Trans Am Ophthalmol Soc. 2007;105:594-615.PubMedGoogle Scholar
24.
Ataer-Cansizoglu  E, Bolon-Canedo  V, Campbell  JP,  et al; i-ROP Research Consortium.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: performance of the “i-ROP” system and image features associated with expert diagnosis.  Transl Vis Sci Technol. 2015;4(6):5. doi:10.1167/tvst.4.6.5PubMedGoogle ScholarCrossref
25.
Brown  JM, Campbell  JP, Beers  A,  et al; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.  Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.  JAMA Ophthalmol. 2018;136(7):803-810. doi:10.1001/jamaophthalmol.2018.1934PubMedGoogle ScholarCrossref
26.
Taylor  S, Brown  JM, Gupta  K,  et al; Imaging and Informatics in Retinopathy of Prematurity Consortium.  Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning  [published online July 3, 2019].  JAMA Ophthalmol. doi:10.1001/jamaophthalmol.2019.2433Google Scholar
27.
World Medical Association.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects.  JAMA. 2013;310(20):2191-2194. doi:10.1001/jama.2013.281053PubMedGoogle ScholarCrossref
28.
Ryan  MC, Ostmo  S, Jonas  K,  et al.  Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology.  AMIA Annu Symp Proc. 2014;2014:1902-1910.PubMedGoogle Scholar
29.
Redd  TK, Campbell  JP, Brown  JM,  et al; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.  Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity.  [published online November 23, 2018].  Br J Ophthalmol. 2018;bjophthalmol-2018-313156. doi:10.1136/bjophthalmol-2018-313156PubMedGoogle Scholar
30.
Gupta  MP, Chan  RVP, Anzures  R, Ostmo  S, Jonas  K, Chiang  MF; Imaging & Informatics in ROP Research Consortium.  Practice patterns in retinopathy of prematurity treatment for disease milder than recommended by guidelines.  Am J Ophthalmol. 2016;163:1-10. doi:10.1016/j.ajo.2015.12.005PubMedGoogle ScholarCrossref
31.
Walz  JM, Bemme  S, Reichl  S,  et al; Retina.net ROP-Register-Studiengruppe.  Treated cases of retinopathy of prematurity in Germany: 5-year data from the Retina.net ROP registry  [in German].  Ophthalmologe. 2018;115(6):476-488. doi:10.1007/s00347-018-0701-5PubMedGoogle ScholarCrossref
32.
VanderVeen  DK, Melia  M, Yang  MB, Hutchinson  AK, Wilson  LB, Lambert  SR.  Anti-vascular endothelial growth factor therapy for primary treatment of type 1 retinopathy of prematurity: a report by the American Academy of Ophthalmology.  Ophthalmology. 2017;124(5):619-633. doi:10.1016/j.ophtha.2016.12.025PubMedGoogle ScholarCrossref
33.
Sankar  MJ, Sankar  J, Chandra  P.  Anti-vascular endothelial growth factor (VEGF) drugs for treatment of retinopathy of prematurity.  Cochrane Database Syst Rev. 2018;1(1):CD009734. doi:10.1002/14651858.CD009734.pub3PubMedGoogle Scholar
34.
Hewing  NJ, Kaufman  DR, Chan  RVP, Chiang  MF.  Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts.  JAMA Ophthalmol. 2013;131(8):1026-1032. doi:10.1001/jamaophthalmol.2013.135PubMedGoogle ScholarCrossref
35.
Brown  JM, Campbell  JP, Beers  A,  et al. Fully automated disease severity assessment and treatment monitoring in retinopathy of prematurity using deep learning. In: Zhang  J, Chen  P-H, eds.  Proceedings of SPIE 10579, Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Bellingham, WA: SPIE; 2018, doi:10.1117/12.2295942.
36.
Hartnett  ME.  Pathophysiology and mechanisms of severe retinopathy of prematurity.  Ophthalmology. 2015;122(1):200-210. doi:10.1016/j.ophtha.2014.07.050PubMedGoogle ScholarCrossref
37.
Hajrasouliha  AR, Garcia-Gonzales  JM, Shapiro  MJ, Yoon  H, Blair  MP.  Reactivation of retinopathy of prematurity three years after treatment with bevacizumab.  Ophthalmic Surg Lasers Imaging Retina. 2017;48(3):255-259. doi:10.3928/23258160-20170301-10PubMedGoogle ScholarCrossref
38.
Snyder  LL, Garcia-Gonzalez  JM, Shapiro  MJ, Blair  MP.  Very late reactivation of retinopathy of prematurity after monotherapy with intravitreal bevacizumab.  Ophthalmic Surg Lasers Imaging Retina. 2016;47(3):280-283. doi:10.3928/23258160-20160229-12PubMedGoogle ScholarCrossref
39.
Kang  KB, Orlin  A, Lee  TC, Chiang  MF, Chan  RVP.  The use of digital imaging in the identification of skip areas after laser treatment for retinopathy of prematurity and its implications for education and patient care.  Retina. 2013;33(10):2162-2169. doi:10.1097/IAE.0b013e31828e6969PubMedGoogle ScholarCrossref
40.
Ting  DSW, Wu  W-C, Toth  C.  Deep learning for retinopathy of prematurity screening.  [published online November 23, 2018].  Br J Ophthalmol. 2018;bjophthalmol-2018-313290. doi:10.1136/bjophthalmol-2018-313290PubMedGoogle Scholar
41.
Ting  DSW, Pasquale  LR, Peng  L,  et al.  Artificial intelligence and deep learning in ophthalmology.  Br J Ophthalmol. 2019;103(2):167-175. doi:10.1136/bjophthalmol-2018-313173PubMedGoogle ScholarCrossref
Original Investigation
July 3, 2019

A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment

Author Affiliations
  • 1Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
  • 2Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
  • 3Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
  • 4Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
  • 5Massachusetts General Hospital & Brigham and Women’s Hospital Center for Clinical Data Science, Boston
  • 6Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
  • 7Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland
JAMA Ophthalmol. 2019;137(9):1029-1036. doi:10.1001/jamaophthalmol.2019.2442
Key Points

Question  Can a quantitative measurement of retinopathy of prematurity severity be used over time to monitor disease regression after treatment?

Findings  In this cohort study of at-risk infants using data collected for the Imaging and Informatics in Retinopathy of Prematurity study, the quantitative retinopathy of prematurity vascular severity score developed using an automated deep learning–based plus disease classifier consistently reflected clinical disease posttreatment regression in 46 included eyes with laser or bevacizumab treatment.

Meaning  Tracking quantitative measurements of retinopathy of prematurity severity may be an effective method of following disease regression and identifying patients at risk for recurrence after retinopathy of prematurity treatment.

Abstract

Importance  Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP).

Objectives  To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment.

Design, Setting, and Participants  This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included.

Main Outcomes and Measures  The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction.

Results  Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001).

Conclusions and Relevance  This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.

Introduction

Retinopathy of prematurity (ROP) remains a growing cause of visual impairment worldwide although most cases of blindness can be prevented with appropriate screening and timely treatment.1,2 Two international classification of ROP committees have established modern classification schema,3,4 and several National Institutes of Health–funded trials1,5,6 have defined the standard of care for ROP management. Owing to differences in primary prevention, screening guidelines vary by region,7,8 but treatment guidelines are consistent worldwide. More recently, treatment with vascular endothelial growth factor antagonists (anti-VEGF agents) has increased.6,9,10 However, there is wide variability in application of current treatment guidelines. An ROP diagnosis is subjective and qualitative, with high rates of interobserver diagnostic variability even in clinical trials.8,11-19 In particular, plus disease is the most important clinical feature defining treatment-requiring ROP and has been shown to be a highly variable diagnosis among examiners.8,11-14,16,17 Differences in clinical outcomes and in rates of disease recurrence may be partly caused by differences in ROP severity at the time of treatment.

Furthermore, although the natural history of ROP progression has been well described by the Cryotherapy for ROP (CRYO-ROP) and Early Treatment for ROP (ETROP) trials,1,5 to date, there is no standard nomenclature for classifying treated ROP or for describing disease regression patterns. There are no evidence-based guidelines for disease retreatment in patients with persistent, recurrent, or reactivated ROP. The lack of those standards and guidelines has become increasingly problematic as ROP treatment with anti-VEGF agents has become more common with the high rate of recurrence and the importance of long-term monitoring after anti-VEGF treatment.6,9 Finally, there appear to be significant differences in disease regression patterns after treatment with laser photocoagulation vs treatment with anti-VEGF agents.6,20,21

Quiz Ref IDA quantitative approach to ROP classification and management would provide an important tool for studying posttreatment changes. Computer-based image analysis has been used to quantitatively evaluate plus disease in ROP.13,14,22-25 Members of our group have previously developed a fully automated algorithm for plus disease diagnosis on a 3-level scale (normal, preplus, plus) and have shown that it can classify disease with comparable or better accuracy than clinical experts.25 In a parallel study, our group has described a quantitative vascular severity scale on a continuous scale from 1 (normal) to 9 (severe) and has shown its utility for monitoring disease progression.26 In the present study, we assessed whether the quantitative ROP vascular severity scale could be used to track disease regression after treatment and to evaluate differences between treatments using laser photocoagulation vs an anti-VEGF agent.

Methods

This study was performed using data from the multicenter Imaging and Informatics in ROP (i-ROP) cohort. The i-ROP consortium comprises 9 tertiary referral centers that screen high volumes of at-risk infants for ROP. The present study was approved by the Institutional Review Board at the coordinating center (Oregon Health & Science University) and by each of the 9 participating institutions (Columbia University, University of Illinois at Chicago, William Beaumont Hospital, Children’s Hospital Los Angeles, Cedars-Sinai Medical Center, University of Miami, Asociacion para Evitar la Ceguera en Mexico). All institutions abided by the tenets of the Declaration of Helsinki,27 and written informed consent was obtained from the parents of all infants enrolled.

Data Set

Deidentified images from clinical examinations performed between July 2011 and December 2016 were assessed. All images were obtained using a commercially available camera (RetCam; Natus Medical Incorporated). Each study eye examination was classified (no ROP, mild ROP, type 2 ROP or preplus, or type 1 ROP) using a reference standard diagnosis that harmonized findings from clinical examination and image-based examination by multiple experts using previously described methods.28

Image Analysis

The i-ROP deep learning system was used to classify the probability of plus disease on a 3-level scale (normal, preplus, plus) for each image in the data set. An automated ROP vascular severity score was then assigned to each image, from 1 (normal retinal vasculature) to 9 (worst retinal vasculature, eg, severe plus disease) using methods previously published29 based on the probabilities of each disease category: (1 × probability of normal) + (5 × probability of preplus disease) + (9 × probability of plus disease).

All patients with treatment-requiring ROP in the i-ROP cohort study were eligible for inclusion in the present study. Infants were excluded if they did not have at least an imaging examination in the 4 weeks prior to treatment, at the time of treatment, and within 4 weeks after treatment.

Statistical Analysis

For all infants who developed treatment-requiring ROP, the vascular severity scores were analyzed in weekly cohorts in the 4 weeks prior to and following treatment, adjusted for time of treatment. The distribution of scores at each point before and after treatment was compared against the scores at the time of treatment using the Wilcoxon rank sum test.

Two separate subgroup analyses were performed. First, the primary analysis was conducted in 2 groups: eyes that were treated only with laser and eyes that were treated only with an anti-VEGF agent. Treatment with laser or anti-VEGF in the i-ROP cohort study was performed solely at investigator discretion. Second, eyes that required retreatment within 6 weeks were analyzed, and multivariable logistic regression was performed to identify the factors that were associated with the need for subsequent treatment (ROP vascular severity score at time of initial treatment, rate of disease progression, gestational age, birth weight, and postmenstrual age [PMA] at the time of initial treatment). Statistical analyses were performed using MatLab, version 2014A (MathWorks). Descriptive statistics, the t test, the Kruskal-Wallis test, the Wilcoxon rank sum test with Bonferroni correction, and 2-way analysis of variance (ANOVA) were used to assess an association between the ROP vascular severity score and progression. An α level of .05 was used with P < .05 denoting statistical significance.

Results
Description of Study Cohort

In total, 5255 eye examinations from 871 infants were assessed for eligibility in this study, and 91 eyes received treatment (5.4%). Of those 91 eyes, 46 met study eligibility criteria based on the availability of images before treatment, at the time of treatment, and after treatment. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) PMA at time of treatment was 37.4 (3.8) weeks. No patients included in the present study later required vitrectomy.

Figure 1 and Figure 2 present representative images from eyes that required treatment. Among the patients, 38 eyes (25 patients) were initially treated with laser, and 8 eyes (5 patients) were initially treated with bevacizumab. Of 46 eyes, 45 (98%) were diagnosed as having type 1 ROP by the examining clinician. One eye was diagnosed as having type 2 ROP but treated outside of published guidelines because the infant was undergoing general anesthesia for treatment of the fellow eye.30

ROP Vascular Severity Score Before and After Treatment

Quiz Ref IDFigure 3 shows the mean (95% CI) ROP vascular severity scores for each point before and after treatment of 46 eyes. The mean (SD) ROP vascular severity score at the time of treatment was 7.43 (1.89). The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment, peaked at treatment, and decreased for at least 2 weeks after treatment (all 3 P < .001 assessed using unpaired 2-tailed t tests with Bonferroni correction).

Figure 4 shows the same data points by treatment modality (anti-VEGF vs laser). Patients receiving anti-VEGF therapy had higher mean (SD) ROP severity scores (8.79 [0.39]) compared with laser-treated eyes (7.15 [1.96]) at the time of treatment (P = .02). The mean change in ROP vascular severity score in the first week after treatment was higher in the bevacizumab-treated group, decreasing at a mean (SD) rate of −3.28 (1.51) vs −1.91 (1.74) in the laser-treated group (P = .04).

Characteristics of Retreated Eyes

Of 8 eyes, 3 (38%) initially treated with bevacizumab were retreated with bevacizumab because of poor initial response and were deemed treatment failures with mean (SD) time to retreatment of 9 (7) days. No delayed recurrence (after initial disease regression) was seen within 6 weeks for the bevacizumab group. Of 38 eyes, 14 (37%) initially treated with laser required retreatment within 6 weeks. The mean (SD) time to retreatment in this group was 2.5 (2.7) weeks. The PMA at time of treatment did not vary between eyes that required retreatment (median [interquartile range] PMA, 37.0 [1.7] weeks; mean [SD] PMA, 38.9 [5.8] weeks) and those who were treated a single time (median [interquartile range] PMA, 37.0 [3.3] weeks; mean [SD] PMA, 36.4 [2.2] weeks; P = .47, Wilcoxon rank sum test).

Figure 5 displays the mean (95% CI) ROP severity scores for patients who required retreatment with laser within 6 weeks compared with those who did not. Multivariable logistic regression showed that the ROP vascular severity score at time of initial treatment (odds ratio [OR], 1.2; 95% CI, 1.04-1.41; P = .01), the birth weight (OR, 1.01, 95% CI, 1.00-1.01; P < .001), and the ROP vascular severity score progression rate after initial screening (OR, 2.84; 95% CI, 1.87-4.32; P < .001) were associated with requiring retreatment. Gestational age (OR, 1.11; 95% CI, 0.72-1.73; P = .63) and PMA at time of treatment (OR, 1.11; 95% CI, 0.96-1.30; P = .16) were not statistically significant. Eyes that eventually required retreatment had a mean (SD) ROP vascular severity score at the initial treatment of 8.47 (1.08) compared with a score of 6.48 (2.08) in eyes that did not eventually require retreatment (P < .001). The retreated eyes had significantly higher mean (SD) disease severity from treatment through 3 weeks (week 0, 8.47 [1.08], P < .01; week 1, 7.15 [2.43], P < .01; week 2, 6.16 [1.87], P = .02; week 3, 7.05 [2.22], P = .03) after initial treatment compared with mean (SD) for single-treated eyes (week 0, 6.49 [2.08]; week 1, 3.92 [2.49]; week 2, 3.23 [1.91]; week 3, 2.94 [2.72]). Based on the regression analysis results, each point increase in the ROP vascular severity score at the time of initial treatment increased the odds for retreatment 1.2 times. Each point per week increase in the ROP vascular severity score rate of progression at initial screening increased the odds for retreatment 2.8 times.

Discussion

Quiz Ref IDIn the present study, we showed the feasibility of using a quantitative ROP severity score derived from a deep learning system to monitor disease regression in a cohort of infants with treatment-requiring disease. Our key study findings were as follows: (1) ROP quantitative vascular severity score trends over time were associated with clinically observed disease progression (before treatment) and regression (after treatment). (2) Infants treated with bevacizumab had a higher severity score at the time of treatment and more rapid disease regression than infants treated with laser photocoagulation. (3) Among patients treated with laser, those who eventually required retreatment had a higher ROP vascular severity score at the time of initial treatment and a more aggressive pace of disease than patients who did not eventually require retreatment.

The first key finding was that an ROP vascular severity score derived from a deep learning classifier mirrored the clinically observed disease progression that led to the decision to treat and the clinically observed disease regression after treatment. The natural history of ROP progression has been well described by the CRYO-ROP and ETROP studies.1,5 However, the terms recurrence, retreatment, and reactivation are often used interchangeably or inconsistently in the literature.10,20 Moreover, with 2 different types of treatments, it is becoming more apparent that regression patterns, disease recurrence risk, and long-term management needs are different between laser-treated and anti-VEGF–treated eyes.10,20,21,31-33 Many existing studies are confounded by differences in disease severity between infants treated with laser vs an anti-VEGF agent, the latter of which tends to be used for more posterior and aggressive disease.31-33 In addition, differences in lengths of follow-up time after treatment of anti-VEGF–treated eyes can recur much later, and variable published recurrence rates after laser treatment create difficulty interpreting the literature.10,20,31-33 The findings in the present study suggest that this quantitative ROP vascular severity score could be used to identify typical and atypical disease regression patterns and to potentially guide retreatment decision for treatment failure (lack of initial response to treatment) or recurrence (reprogression of disease after initial disease regression following treatment). Further validation in larger cohorts and in prospective studies will provide additional information about the clinical utility of this quantitative score.

Systematic differences in the level of severity at which treatment is initiated have been well reported by clinicians.11,12,14-18,28 These differences may be attributable to the subjectivity of plus disease or zone I diagnosis or to bias in the clinical factors relevant for the decision to treat.34 That these biases existed in all previous ROP clinical trials, and that, to date, there are no evidence-based objective criteria for identifying treatment-requiring ROP, leads to real-world differences among physicians regarding level of disease severity at treatment.11 Prospective evaluation of this automated ROP vascular severity score may be able to address these challenges in the future.

To our knowledge, although there is growing literature on disease retreatment and recurrence, little published work has systematically examined the typical pace of disease regression following ROP treatment.6,10,20,21,31-33,35,36 The second key finding of the present study was that the ROP vascular severity score showed promise as a tool to investigate the differences between therapeutic laser-induced and anti-VEGF treatment–induced regression. In our data, eyes treated with bevacizumab had more aggressive progression of disease before treatment and had higher ROP vascular severity scores at the time of treatment. Of course, this may be partially explained by the tendency of clinicians to select bevacizumab, rather than laser photocoagulation, as treatment for more aggressive disease, such as aggressive posterior ROP or zone I ROP.31-33 Furthermore, we showed more rapid disease regression after treatment with an anti-VEGF agent than after laser treatment (Figure 4). This has important implications for retreatment due to primary treatment failure as well as for monitoring disease recurrence after anti-VEGF treatment.

Early retreatment in anti-VEGF–treated eyes is often caused by treatment failure. The reasons this occurs are unknown but may be because of egress of medication out of the eye following intravitreal injection or differences in the VEGF concentration in eyes with more severe disease. Indeed, 3 of 8 eyes (38%) in this study were treated a second time with bevacizumab in the week following primary treatment. Delayed retreatment with an anti-VEGF agent is typically because of disease recurrence, that is, after initial disease regression, with subsequent recurrent neovascularization or development of plus disease.6,10,20 This late disease recurrence has been reported as long as 100 weeks’ PMA, and clinically significant sequelae of anti-VEGF treatment have been reported as long as 3 years after treatment.37,38 Following the BEAT-ROP (Bevacizumab Eliminates the Angiogenic Threat for Retinopathy of Prematurity) study that reported lower rates of retreatment in eyes treated with anti-VEGF agents through 54 weeks, most subsequent studies have found higher retreatment rates with anti-VEGF agents.10,20 These studies are potentially difficult to compare owing to differences in underlying disease severity (eg, some studies may have had more severe disease, on average, than BEAT-ROP) because early reports have highlighted higher retreatment rates in eyes with more aggressive and posterior disease.21,32,33 The higher severity score in eyes treated with bevacizumab, presumably associated with more aggressive posterior disease, leads to the question of whether earlier treatment of aggressive-posterior ROP (at a lower severity score) might lead to improved outcomes. This has not been tested. However, because a parallel study by some in our group has shown that eyes that eventually progress to treatment-requiring disease might be identified earlier using this quantitative ROP severity score, it may be possible to design a study to answer this question in the future.26,35

The third key finding is that among eyes treated with laser, those that required retreatment had a higher ROP vascular severity score and more aggressive pace of disease at the initial treatment. This was true even after adjusting for birth weight, gestational age, PMA, and rate of initial disease progression. There are 3 potential explanations for this finding: (1) those infants had more aggressive disease at baseline, which was more likely to recur; (2) those patients were observed more conservatively until progressing to a more severe level of disease before treatment was initiated, either intentionally or because of diagnostic differences among examiners;11,12 or (3) there was variability in treatment efficacy among different physicians.39 Although the present study was not powered to detect differences in retreatment rates among physicians, the retreatments were distributed among study sites, and all of the i-ROP centers relied on experienced ROP clinicians. We note that the laser retreatment rate in the present cohort (14 of 38 [37%]) was higher than rates typically reported in the literature. Because of the inclusion criteria that patients in this study required posttreatment imaging, there may be a selection bias toward more severe disease in this subcohort of i-ROP patients. Further prospective analysis using the ROP severity score may identify the optimal severity for intervention to minimize disease recurrence and treatment failure.

Limitations

Quiz Ref IDThere are a number of study limitations. First, the effect of excluding patients for lack of available imaging is unclear, but there are several potential reasons imaging may not have been obtained. For example, there was no requirement for capturing images at every examination although this was the standard practice of most clinicians in the study. In addition, images were generally obtained following the ophthalmoscopic examination, and therefore in cases of clinical instability, photography may have been deferred. Infants with mild ROP originally may have been examined less frequently than weekly. Moreover, outborn infants who were referred in for treatment without prereferral images were excluded. Except as noted above with regard to retreatment, we do not feel that excluding those infants introduced any systematic bias into the study, but analysis using larger prospective data sets will be valuable. Second, in terms of comparing treatment modalities, because there was no randomization to treatments and the number of patients treated with the anti-VEGF agent was small, we would caution generalizing these conclusions on regression pattern differences between treatments until larger data sets can be analyzed. Third, this study was not designed to analyze retreatments after 6 weeks, and thus we are unable to evaluate this technology for monitoring disease recurrence following anti-VEGF treatment. This will be an important area for future analysis. Fourth, this deep learning technology is currently available only for research studies; thus, the real-world effect of these findings is currently limited. Further work must be done to demonstrate the association of image quality, field of view, and camera system with ROP vascular severity score. Fifth, it will be necessary to perform similar analysis in cohorts outside of North America and among infants treated with anti-VEGF agents other than bevacizumab to assess the generalizability of the present findings. Such studies will be particularly important in low- and middle-income countries, where ROP is epidemic and oxygen regulation produces a more aggressive disease phenotype.

Conclusions

The present results suggested that an automated quantitative ROP vascular severity score obtained using a deep learning classifier may be used to monitor disease regression after treatment. This technology exhibits promise for monitoring typical and atypical disease regression patterns, identifying disease recurrence, and providing an objective metric to guide treatment and posttreatment decision making in the future.40 These findings may be generalizable to other ophthalmologic and medical diseases that are currently managed on the basis of subjective clinical characteristics.41

Back to top
Article Information

Accepted for Publication: April 14, 2019.

Corresponding Author: Michael F. Chiang, MD, Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, 3375 SW Terwilliger Blvd, Portland, OR 97239 (chiangm@ohsu.edu).

Published Online: July 3, 2019. doi:10.1001/jamaophthalmol.2019.2442

Author Contributions: Drs Gupta and Campbell contributed equally to this work and are considered co–first authors. Dr Chiang had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Campbell, Taylor, Erdogmus, Ioannidis, Kalpathy-Cramer, Kim, Chiang.

Acquisition, analysis, or interpretation of data: Gupta, Campbell, Taylor, Brown, Ostmo, Chan, Dy, Kalpathy-Cramer, Kim, Chiang.

Drafting of the manuscript: Gupta, Campbell, Brown.

Critical revision of the manuscript for important intellectual content: Campbell, Taylor, Ostmo, Chan, Dy, Erdogmus, Ioannidis, Kalpathy-Cramer, Kim, Chiang.

Statistical analysis: Gupta, Campbell, Chiang.

Obtained funding: Erdogmus, Ioannidis, Kalpathy-Cramer, Chiang.

Administrative, technical, or material support: Gupta, Campbell, Brown, Ostmo, Kalpathy-Cramer, Chiang.

Supervision: Campbell, Chan, Dy, Erdogmus, Kalpathy-Cramer, Chiang.

Conflict of Interest Disclosures: Dr Campbell reported receiving grants from Genentech and personal fees from Allergan outside the submitted work. Dr Chan reported receiving personal fees from Alcon, Allergan, Visunex Medical Systems, Beyeonics, and Genentech outside the submitted work. Dr Dy reported receiving grants from the National Science Foundation. Dr Ioannidis reported receiving grants from National Science Foundation and from the National Institutes of Health during the conduct of the study. Dr Kalpathy-Cramer reported receiving grants from the National Eye Institute and the National Science Foundation during the conduct of the study and receiving personal fees from INFOTECHSoft outside the submitted work. Dr Chiang reported receiving grants from the National Institutes of Health, the National Science Foundation, and Genentech; receiving nonfinancial support from Clarity Medical Systems; receiving personal fees from Novartis during the conduct of the study; and being an equity owner in Initeleretina outside the submitted work. Drs Campbell and Brown, Ms Ostmo, and Drs Chan, Dy, Erdogmus, Ioannidis, Kalpathy-Cramer, and Chiang have a patent application pending on the described technology.

Funding/Support: This project was supported by grants R01EY19474, K12EY027720, and P30EY10572 from the National Institutes of Health; by grants SCH-1622679, SCH-1622542, and SCH-1622536 from the National Science Foundation; and by Research to Prevent Blindness in the form of unrestricted departmental funding to the Casey Institute and a Career Development Award (Dr Campbell).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: The members of the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Consortium follow. Oregon Health & Science University: Michael F. Chiang, MD, Susan Ostmo, MS, Sang Jin Kim, MD, PhD, Kemal Sonmez, PhD, and J. Peter Campbell, MD, MPH. University of Illinois at Chicago: R. V. Paul Chan, MD and Karyn Jonas, RN. Columbia University: Jason Horowitz, MD, Osode Coki, RN, Cheryl-Ann Eccles, RN, and Leora Sarna, RN. Weill Cornell Medical College: Anton Orlin, MD. Bascom Palmer Eye Institute: Audina Berrocal, MD, and Catherin Negron, BA. William Beaumont Hospital: Kimberly Denser, MD, Kristi Cumming, RN, Tammy Osentoski, RN, Tammy Check, RN, and Mary Zajechowski, RN. Children’s Hospital Los Angeles: Thomas Lee, MD, Evan Kruger, BA, and Kathryn McGovern, MPH. Cedars Sinai Hospital: Charles Simmons, MD, Raghu Murthy, MD, and Sharon Galvis, NNP. LA Biomedical Research Institute: Jerome Rotter, MD, Ida Chen, PhD, Xiaohui Li, MD, Kent Taylor, PhD, and Kaye Roll, RN. Massachusetts General Hospital: Jayashree Kalpathy-Cramer, PhD. Northeastern University: Deniz Erdogmus, PhD, and Stratis Ioannidis, PhD. Asociacion para Evitar la Ceguera en Mexico: Maria Ana Martinez-Castellanos, MD, Samantha Salinas-Longoria, MD, Rafael Romero, MD, Andrea Arriola, MD, Francisco Olguin-Manriquez, MD, Miroslava Meraz-Gutierrez, MD, Carlos M. Dulanto-Reinoso, MD, and Cristina Montero-Mendoza, MD.

Meeting Presentation: This paper was presented at the Annual Meeting of the Association for Research in Vision and Ophthalmology; May 1, 2018; Honolulu, Hawaii.

References
1.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Arch Ophthalmol. 1988;106(4):471-479. doi:10.1001/archopht.1988.01060130517027PubMedGoogle ScholarCrossref
2.
Gilbert  C.  Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control.  Early Hum Dev. 2008;84(2):77-82. doi:10.1016/j.earlhumdev.2007.11.009PubMedGoogle ScholarCrossref
3.
The Committee for the Classification of Retinopathy of Prematurity.  An international classification of retinopathy of prematurity.  Arch Ophthalmol. 1984;102(8):1130-1134. doi:10.1001/archopht.1984.01040030908011PubMedGoogle ScholarCrossref
4.
International Committee for the Classification of Retinopathy of Prematurity.  The international classification of retinopathy of prematurity revisited.  Arch Ophthalmol. 2005;123(7):991-999. doi:10.1001/archopht.123.7.991PubMedGoogle ScholarCrossref
5.
Early Treatment For Retinopathy Of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial.  Arch Ophthalmol. 2003;121(12):1684-1694. doi:10.1001/archopht.121.12.1684PubMedGoogle ScholarCrossref
6.
Mintz-Hittner  HA, Kennedy  KA, Chuang  AZ; BEAT-ROP Cooperative Group.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity.  N Engl J Med. 2011;364(7):603-615. doi:10.1056/NEJMoa1007374PubMedGoogle ScholarCrossref
7.
Fierson  WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists.  Screening examination of premature infants for retinopathy of prematurity.  Pediatrics. 2018;142(6):e20183061. doi:10.1542/peds.2018-3061PubMedGoogle ScholarCrossref
8.
Reynolds  JD, Dobson  V, Quinn  GE,  et al; CRYO-ROP and LIGHT-ROP Cooperative Study Groups.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies.  Arch Ophthalmol. 2002;120(11):1470-1476. doi:10.1001/archopht.120.11.1470PubMedGoogle ScholarCrossref
9.
Mintz-Hittner  HA.  Retinopathy of prematurity: intravitreal injections of bevacizumab: timing, technique, and outcomes.  J AAPOS. 2016;20(6):478-480. doi:10.1016/j.jaapos.2016.10.002PubMedGoogle ScholarCrossref
10.
Darwish  D, Chee  R-I, Patel  SN,  et al.  Anti-Vascular endothelial growth factor and the evolving management paradigm for retinopathy of prematurity.  Asia Pac J Ophthalmol (Phila). 2018;7(3):136-144. doi:10.22608/APO.201850PubMedGoogle Scholar
11.
Fleck  BW, Williams  C, Juszczak  E,  et al; BOOST II Retinal Image Digital Analysis (RIDA) Group.  An international comparison of retinopathy of prematurity grading performance within the Benefits of Oxygen Saturation Targeting II trials.  Eye (Lond). 2018;32(1):74-80. doi:10.1038/eye.2017.150PubMedGoogle ScholarCrossref
12.
Campbell  JP, Kalpathy-Cramer  J, Erdogmus  D,  et al; Imaging and Informatics in Retinopathy of Prematurity Research Consortium.  Plus disease in retinopathy of prematurity: a continuous spectrum of vascular abnormality as a basis of diagnostic variability.  Ophthalmology. 2016;123(11):2338-2344. doi:10.1016/j.ophtha.2016.07.026PubMedGoogle ScholarCrossref
13.
Kalpathy-Cramer  J, Campbell  JP, Erdogmus  D,  et al; Imaging and Informatics in Retinopathy of Prematurity Research Consortium.  Plus disease in retinopathy of prematurity: improving diagnosis by ranking disease severity and using quantitative image analysis.  Ophthalmology. 2016;123(11):2345-2351. doi:10.1016/j.ophtha.2016.07.020PubMedGoogle ScholarCrossref
14.
Campbell  JP, Ataer-Cansizoglu  E, Bolon-Canedo  V,  et al; Imaging and Informatics in ROP (i-ROP) Research Consortium.  Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis.  JAMA Ophthalmol. 2016;134(6):651-657. doi:10.1001/jamaophthalmol.2016.0611PubMedGoogle ScholarCrossref
15.
Chiang  MF, Thyparampil  PJ, Rabinowitz  D.  Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity.  Arch Ophthalmol. 2010;128(9):1153-1159. doi:10.1001/archophthalmol.2010.199PubMedGoogle ScholarCrossref
16.
Chiang  MF, Jiang  L, Gelman  R, Du  YE, Flynn  JT.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.  Arch Ophthalmol. 2007;125(7):875-880. doi:10.1001/archopht.125.7.875PubMedGoogle ScholarCrossref
17.
Campbell  JP, Ryan  MC, Lore  E,  et al; Imaging & Informatics in Retinopathy of Prematurity Research Consortium.  Diagnostic discrepancies in retinopathy of prematurity classification.  Ophthalmology. 2016;123(8):1795-1801. doi:10.1016/j.ophtha.2016.04.035PubMedGoogle ScholarCrossref
18.
Quinn  GE, Ells  A, Capone  A  Jr,  et al; e-ROP (Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity) Cooperative Group.  Analysis of discrepancy between diagnostic clinical examination findings and corresponding evaluation of digital images in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity study.  JAMA Ophthalmol. 2016;134(11):1263-1270. doi:10.1001/jamaophthalmol.2016.3502PubMedGoogle ScholarCrossref
19.
Scott  KE, Kim  DY, Wang  L,  et al.  Telemedical diagnosis of retinopathy of prematurity intraphysician agreement between ophthalmoscopic examination and image-based interpretation.  Ophthalmology. 2008;115(7):1222-1228. doi:10.1016/j.ophtha.2007.09.006PubMedGoogle ScholarCrossref
20.
Mintz-Hittner  HA, Geloneck  MM, Chuang  AZ.  Clinical management of recurrent retinopathy of prematurity after intravitreal bevacizumab monotherapy.  Ophthalmology. 2016;123(9):1845-1855. doi:10.1016/j.ophtha.2016.04.028PubMedGoogle ScholarCrossref
21.
Mueller  B, Salchow  DJ, Waffenschmidt  E,  et al.  Treatment of type I ROP with intravitreal bevacizumab or laser photocoagulation according to retinal zone.  Br J Ophthalmol. 2017;101(3):365-370. doi:10.1136/bjophthalmol-2016-308375PubMedGoogle Scholar
22.
Wittenberg  LA, Jonsson  NJ, Chan  RVP, Chiang  MF.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity.  J Pediatr Ophthalmol Strabismus. 2012;49(1):11-19. doi:10.3928/01913913-20110222-01PubMedGoogle ScholarCrossref
23.
Wallace  DK.  Computer-assisted quantification of vascular tortuosity in retinopathy of prematurity (an American Ophthalmological Society thesis).  Trans Am Ophthalmol Soc. 2007;105:594-615.PubMedGoogle Scholar
24.
Ataer-Cansizoglu  E, Bolon-Canedo  V, Campbell  JP,  et al; i-ROP Research Consortium.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: performance of the “i-ROP” system and image features associated with expert diagnosis.  Transl Vis Sci Technol. 2015;4(6):5. doi:10.1167/tvst.4.6.5PubMedGoogle ScholarCrossref
25.
Brown  JM, Campbell  JP, Beers  A,  et al; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.  Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.  JAMA Ophthalmol. 2018;136(7):803-810. doi:10.1001/jamaophthalmol.2018.1934PubMedGoogle ScholarCrossref
26.
Taylor  S, Brown  JM, Gupta  K,  et al; Imaging and Informatics in Retinopathy of Prematurity Consortium.  Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning  [published online July 3, 2019].  JAMA Ophthalmol. doi:10.1001/jamaophthalmol.2019.2433Google Scholar
27.
World Medical Association.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects.  JAMA. 2013;310(20):2191-2194. doi:10.1001/jama.2013.281053PubMedGoogle ScholarCrossref
28.
Ryan  MC, Ostmo  S, Jonas  K,  et al.  Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology.  AMIA Annu Symp Proc. 2014;2014:1902-1910.PubMedGoogle Scholar
29.
Redd  TK, Campbell  JP, Brown  JM,  et al; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.  Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity.  [published online November 23, 2018].  Br J Ophthalmol. 2018;bjophthalmol-2018-313156. doi:10.1136/bjophthalmol-2018-313156PubMedGoogle Scholar
30.
Gupta  MP, Chan  RVP, Anzures  R, Ostmo  S, Jonas  K, Chiang  MF; Imaging & Informatics in ROP Research Consortium.  Practice patterns in retinopathy of prematurity treatment for disease milder than recommended by guidelines.  Am J Ophthalmol. 2016;163:1-10. doi:10.1016/j.ajo.2015.12.005PubMedGoogle ScholarCrossref
31.
Walz  JM, Bemme  S, Reichl  S,  et al; Retina.net ROP-Register-Studiengruppe.  Treated cases of retinopathy of prematurity in Germany: 5-year data from the Retina.net ROP registry  [in German].  Ophthalmologe. 2018;115(6):476-488. doi:10.1007/s00347-018-0701-5PubMedGoogle ScholarCrossref
32.
VanderVeen  DK, Melia  M, Yang  MB, Hutchinson  AK, Wilson  LB, Lambert  SR.  Anti-vascular endothelial growth factor therapy for primary treatment of type 1 retinopathy of prematurity: a report by the American Academy of Ophthalmology.  Ophthalmology. 2017;124(5):619-633. doi:10.1016/j.ophtha.2016.12.025PubMedGoogle ScholarCrossref
33.
Sankar  MJ, Sankar  J, Chandra  P.  Anti-vascular endothelial growth factor (VEGF) drugs for treatment of retinopathy of prematurity.  Cochrane Database Syst Rev. 2018;1(1):CD009734. doi:10.1002/14651858.CD009734.pub3PubMedGoogle Scholar
34.
Hewing  NJ, Kaufman  DR, Chan  RVP, Chiang  MF.  Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts.  JAMA Ophthalmol. 2013;131(8):1026-1032. doi:10.1001/jamaophthalmol.2013.135PubMedGoogle ScholarCrossref
35.
Brown  JM, Campbell  JP, Beers  A,  et al. Fully automated disease severity assessment and treatment monitoring in retinopathy of prematurity using deep learning. In: Zhang  J, Chen  P-H, eds.  Proceedings of SPIE 10579, Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Bellingham, WA: SPIE; 2018, doi:10.1117/12.2295942.
36.
Hartnett  ME.  Pathophysiology and mechanisms of severe retinopathy of prematurity.  Ophthalmology. 2015;122(1):200-210. doi:10.1016/j.ophtha.2014.07.050PubMedGoogle ScholarCrossref
37.
Hajrasouliha  AR, Garcia-Gonzales  JM, Shapiro  MJ, Yoon  H, Blair  MP.  Reactivation of retinopathy of prematurity three years after treatment with bevacizumab.  Ophthalmic Surg Lasers Imaging Retina. 2017;48(3):255-259. doi:10.3928/23258160-20170301-10PubMedGoogle ScholarCrossref
38.
Snyder  LL, Garcia-Gonzalez  JM, Shapiro  MJ, Blair  MP.  Very late reactivation of retinopathy of prematurity after monotherapy with intravitreal bevacizumab.  Ophthalmic Surg Lasers Imaging Retina. 2016;47(3):280-283. doi:10.3928/23258160-20160229-12PubMedGoogle ScholarCrossref
39.
Kang  KB, Orlin  A, Lee  TC, Chiang  MF, Chan  RVP.  The use of digital imaging in the identification of skip areas after laser treatment for retinopathy of prematurity and its implications for education and patient care.  Retina. 2013;33(10):2162-2169. doi:10.1097/IAE.0b013e31828e6969PubMedGoogle ScholarCrossref
40.
Ting  DSW, Wu  W-C, Toth  C.  Deep learning for retinopathy of prematurity screening.  [published online November 23, 2018].  Br J Ophthalmol. 2018;bjophthalmol-2018-313290. doi:10.1136/bjophthalmol-2018-313290PubMedGoogle Scholar
41.
Ting  DSW, Pasquale  LR, Peng  L,  et al.  Artificial intelligence and deep learning in ophthalmology.  Br J Ophthalmol. 2019;103(2):167-175. doi:10.1136/bjophthalmol-2018-313173PubMedGoogle ScholarCrossref
×