[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.159.129.152. Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Download PDF
Figure 1.
Receiver Operating Characteristic Curves From the Prediction Models Using Various Combinations of Predictors for Referral-Warranted Retinopathy of Prematurity
Receiver Operating Characteristic Curves From the Prediction Models Using Various Combinations of Predictors for Referral-Warranted Retinopathy of Prematurity

The receiver operating characteristic curves from referral-warranted retinopathy of prematurity prediction models using various combinations of predictors including demographic predictors (birth weight [BW], gestational age [GA], sex, and race/ethnicity), ocular predictors (preplus disease, retinopathy of prematurity stage, and retinal hemorrhage), respiratory status, and postnatal weight gain. AUC indicates area under receiver operating characteristic curve.

Figure 2.
Predicted Probability of Referral-Warranted (RW) Retinopathy of Prematurity (ROP) From the Final Multivariate Prediction Model
Predicted Probability of Referral-Warranted (RW) Retinopathy of Prematurity (ROP) From the Final Multivariate Prediction Model

The predicted probability of RW-ROP for each infant with RW-ROP (orange dots) and for infants without RW-ROP (blue dots) was shown. Using a cut point of 0.05 (dashed line), the sensitivity is 96.0% and specificity is 52.7% for predicting RW-ROP.

Table 1.  
Multivariate Analysis for Predictors of RW-ROP
Multivariate Analysis for Predictors of RW-ROP
Table 2.  
Sensitivity and Specificity for Prediction of RW-ROP Using Various Cut Points of the Predicted Probability Calculated From Final Multivariate Prediction Model
Sensitivity and Specificity for Prediction of RW-ROP Using Various Cut Points of the Predicted Probability Calculated From Final Multivariate Prediction Model
1.
Gilbert  C.  Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84(2):77-82.
PubMedArticle
2.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Arch Ophthalmol. 1988;106(4):471-479.
PubMedArticle
3.
Good  WV, Hardy  RJ; Early Treatment for Retinopathy of Prematurity Cooperative Group.  The multicenter study of Early Treatment for Retinopathy of Prematurity (ETROP). Ophthalmology. 2001;108(6):1013-1014.
PubMedArticle
4.
Mintz-Hittner  HA, Kennedy  KA, Chuang  AZ; BEAT-ROP Cooperative Group.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity. N Engl J Med. 2011;364(7):603-615.
PubMedArticle
5.
Early Treatment for Retinopathy of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the Early Treatment for Retinopathy of Prematurity Randomized Trial. Arch Ophthalmol. 2003;121(12):1684-1694.
PubMedArticle
6.
Ells  AL, Holmes  JM, Astle  WF,  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology. 2003;110(11):2113-2117.
PubMedArticle
7.
Gilbert  C, Fielder  A, Gordillo  L,  et al; International NO-ROP Group.  Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs. Pediatrics. 2005;115(5):e518-e525.
PubMedArticle
8.
Section on Ophthalmology American Academy of Pediatrics; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus.  Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2006;117(2):572-576.
PubMedArticle
9.
Lee  SK, Normand  C, McMillan  D, Ohlsson  A, Vincer  M, Lyons  C; Canadian Neonatal Network.  Evidence for changing guidelines for routine screening for retinopathy of prematurity. Arch Pediatr Adolesc Med. 2001;155(3):387-395.
PubMedArticle
10.
Haines  L, Fielder  AR, Scrivener  R, Wilkinson  AR; Royal College of Paediatrics and Child Health, the Royal College of Ophthalmologists and British Association of Perinatal Medicine.  Retinopathy of prematurity in the UK I: the organisation of services for screening and treatment. Eye (Lond). 2002;16(1):33-38.
PubMedArticle
11.
Binenbaum  G, Ying  GS, Quinn  GE,  et al; Premature Infants in Need of Transfusion Study Group.  A clinical prediction model to stratify retinopathy of prematurity risk using postnatal weight gain. Pediatrics. 2011;127(3):e607-e614.
PubMedArticle
12.
Binenbaum  G, Ying  GS, Quinn  GE,  et al.  The CHOP postnatal weight gain, birth weight, and gestational age retinopathy of prematurity risk model. Arch Ophthalmol. 2012;130(12):1560-1565.
PubMedArticle
13.
Hellström  A, Smith  LEH, Dammann  O.  Retinopathy of prematurity. Lancet. 2013;382(9902):1445-1457.
PubMedArticle
14.
Mittal  M, Dhanireddy  R, Higgins  RD.  Candida sepsis and association with retinopathy of prematurity. Pediatrics. 1998;101(4, pt 1):654-657.
PubMedArticle
15.
Manzoni  P, Maestri  A, Leonessa  M, Mostert  M, Farina  D, Gomirato  G.  Fungal and bacterial sepsis and threshold ROP in preterm very low birth weight neonates. J Perinatol. 2006;26(1):23-30.
PubMedArticle
16.
Chen  ML, Guo  L, Smith  LE, Dammann  CE, Dammann  O.  High or low oxygen saturation and severe retinopathy of prematurity: a meta-analysis. Pediatrics. 2010;125(6):e1483-e1492.
PubMedArticle
17.
Schaffer  DB, Palmer  EA, Plotsky  DF,  et al; The Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Prognostic factors in the natural course of retinopathy of prematurity. Ophthalmology. 1993;100(2):230-237.
PubMedArticle
18.
Hardy  RJ, Palmer  EA, Dobson  V,  et al; Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Risk analysis of prethreshold retinopathy of prematurity. Arch Ophthalmol. 2003;121(12):1697-1701.
PubMedArticle
19.
Graham E Quinn; e-ROP Cooperative Group.  Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity: study design. Ophthalmic Epidemiol. 2014;21(4):256-267.
PubMedArticle
20.
Quinn  GE, Ying  GS, Daniel  E,  et al; e-ROP Cooperative Group.  Validity of a telemedicine system for the evaluation of acute-phase ROP. JAMA Ophthalmol. 2014;132(10):1178-1184.
PubMedArticle
21.
International Committee for the Classification of Retinopathy of Prematurity.  The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol. 2005;123(7):991-999.
PubMedArticle
22.
Senterre  T, Rigo  J.  Reduction in postnatal cumulative nutritional deficit and improvement of growth in extremely preterm infants. Acta Paediatr. 2012;101(2):e64-e70.
PubMedArticle
23.
Fenton  TR, Nasser  R, Eliasziw  M, Kim  JH, Bilan  D, Sauve  R.  Validating the weight gain of preterm infants between the reference growth curve of the fetus and the term infant. BMC Pediatr. 2013;13:92.
PubMedArticle
24.
Good  WV, Hardy  RJ, Dobson  V,  et al; Early Treatment for Retinopathy of Prematurity Cooperative Group.  The incidence and course of retinopathy of prematurity: findings from the early treatment for retinopathy of prematurity study. Pediatrics. 2005;116(1):15-23.
PubMedArticle
25.
Hellström  A, Engström  E, Hård  AL,  et al.  Postnatal serum insulin-like growth factor I deficiency is associated with retinopathy of prematurity and other complications of premature birth. Pediatrics. 2003;112(5):1016-1020.
PubMedArticle
26.
Hellstrom  A, Perruzzi  C, Ju  M,  et al.  Low IGF-I suppresses VEGF-survival signaling in retinal endothelial cells: direct correlation with clinical retinopathy of prematurity. Proc Natl Acad Sci U S A. 2001;98(10):5804-5808.
PubMedArticle
27.
Yang  MB, Rao  S, Copenhagen  DR, Lang  RA.  Length of day during early gestation as a predictor of risk for severe retinopathy of prematurity. Ophthalmology. 2013;120(12):2706-2713.
PubMedArticle
28.
Saunders  RA, Donahue  ML, Christmann  LM,  et al; The Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Racial variation in retinopathy of prematurity. Arch Ophthalmol. 1997;115(5):604-608.
PubMedArticle
29.
Darlow  BA, Hutchinson  JL, Henderson-Smart  DJ, Donoghue  DA, Simpson  JM, Evans  NJ; Australian and New Zealand Neonatal Network.  Prenatal risk factors for severe retinopathy of prematurity among very preterm infants of the Australian and New Zealand Neonatal Network. Pediatrics. 2005;115(4):990-996.
PubMedArticle
30.
Walsh  MC, Morris  BH, Wrage  LA,  et al; National Institutes of Child Health and Human Development Neonatal Research Network.  Extremely low birthweight neonates with protracted ventilation: mortality and 18-month neurodevelopmental outcomes. J Pediatr. 2005;146(6):798-804.
PubMedArticle
31.
Carlo  WA, Finer  NN, Walsh  MC,  et al; SUPPORT Study Group of the Eunice Kennedy Shriver NICHD Neonatal Research Network.  Target ranges of oxygen saturation in extremely preterm infants. N Engl J Med. 2010;362(21):1959-1969.
PubMedArticle
32.
Stenson  B, Brocklehurst  P, Tarnow-Mordi  W; UK BOOST II trial; Australian BOOST II trial; New Zealand BOOST II trial.  Increased 36-week survival with high oxygen saturation target in extremely preterm infants. N Engl J Med. 2011;364(17):1680-1682.
PubMedArticle
33.
Sood  BG, Madan  A, Saha  S,  et al; NICHD neonatal research network.  Perinatal systemic inflammatory response syndrome and retinopathy of prematurity. Pediatr Res. 2010;67:394-400.
PubMedArticle
34.
Choi  BC.  Slopes of a receiver operating characteristic curve and likelihood ratios for a diagnostic test. Am J Epidemiol. 1998;148(11):1127-1132.
PubMedArticle
35.
Reynolds  JD, Dobson  V, Quinn  GE,  et al; CRYO-ROP and LIGHT-ROP Cooperative Study Groups.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol. 2002;120(11):1470-1476.
PubMedArticle
36.
Haines  L, Fielder  AR, Baker  H, Wilkinson  AR.  UK population based study of severe retinopathy of prematurity: screening, treatment, and outcome. Arch Dis Child Fetal Neonatal Ed. 2005;90(3):F240-F244.
PubMedArticle
Original Investigation
March 2015

Predictors for the Development of Referral-Warranted Retinopathy of Prematurity in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study

Author Affiliations
  • 1Department of Ophthalmology, Scheie Eye Institute, University of Pennsylvania, Perelman School of Medicine, Philadelphia
  • 2Division of Pediatric Ophthalmology, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
  • 3Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
  • 4Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
JAMA Ophthalmol. 2015;133(3):304-311. doi:10.1001/jamaophthalmol.2014.5185
Abstract

Importance  Detection of treatment-requiring retinopathy of prematurity (ROP) involves serial eye examinations. An ROP prediction model using predictive factors could identify high-risk infants and reduce required eye examinations.

Objective  To determine predictive factors for the development of referral-warranted (RW) ROP.

Design, Setting, and Participants  This multicenter observational cohort study included secondary analysis of data from the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity Study. Infants included in the study had a birth weight (BW) of less than 1251 g.

Exposures  Serial ROP examinations of premature infants who had 2 or more ROP examinations.

Main Outcomes and Measures  Incidence of RW-ROP (defined as the presence of plus disease, zone I ROP, or ROP stage 3 or greater in either eye) and associations with predictive factors.

Results  Among 979 infants without RW-ROP at first study-related eye examination (median postmenstrual age, 33 weeks; range, 29-40 weeks) who underwent at least 2 eye examinations, 149 (15.2%) developed RW-ROP. In a multivariate model, significant predictors for RW-ROP were male sex (odds ratio [OR], 1.80; 95% CI, 1.13-2.86 vs female), nonblack race (OR, 2.76; 95% CI, 1.50-5.08 for white vs black race and OR, 4.81; 95% CI, 2.19-10.6 for other vs black race), low BW (OR, 5.16; 95% CI, 1.12-7.20 for ≤500 g vs >1100 g), younger gestational age (OR, 9.79; 95% CI, 3.49-27.5 for ≤24 weeks vs ≥28 weeks), number of quadrants with preplus disease (OR, 7.12; 95% CI, 2.53-20.1 for 1-2 quadrants and OR, 18.4; 95% CI, 4.28-79.4 for 3-4 quadrants vs no preplus disease), stage 2 ROP (OR, 4.13; 95% CI, 2.13-8.00 vs no ROP), the presence of retinal hemorrhage (OR, 4.36; 95% CI, 1.57-12.1 vs absence), the need for respiratory support (OR, 4.99; 95% CI, 1.89-13.2 for the need for controlled mechanical ventilator; OR, 11.0; 95% CI, 2.26-53.8 for the need for high-frequency oscillatory ventilation vs no respiratory support), and slow weight gain (OR, 2.44; 95% CI, 1.22-4.89 for weight gain ≤12 g/d vs >18 g/d). These characteristics predicted the development of RW-ROP significantly better than BW and gestational age (area under receiver operating characteristic curve, 0.88 vs 0.78; P < .001).

Conclusions and Relevance  When controlling for very low BW and prematurity, the presence of preplus disease, stage 2 ROP, retinal hemorrhage, and the need for ventilation at time of first study-related eye examination were strong independent predictors for RW-ROP. These predictors may help identify infants in need of timely eye examinations.

Introduction

Retinopathy of prematurity (ROP) is a leading cause of treatable blindness in children worldwide.1 Retinopathy of prematurity can often be effectively treated with laser retinal ablative surgery or other treatments when diagnosed early.24 However, detection of ROP involves subjecting many infants to uncomfortable, resource-intensive serial diagnostic eye examinations by ophthalmologists, while fewer than 10% of the examined infants require treatment.5 An option for decreasing the number of eye examinations is to develop a screening strategy to identify infants with high-risk demographic or ocular characteristics who need ROP examinations by an ophthalmologist to consider treatment. To address this issue, Ells et al6 introduced the concept of referral-warranted (RW) ROP for the detection of potentially severe ROP features observed on retinal images. These features included plus disease, zone I ROP, or ROP stage 3 or greater, and they are also the key components of current indications for treatment of ROP (type 1 ROP) or for increased surveillance (type 2 ROP) as subsequently defined by the results of the Early Treatment for ROP randomized trial.5

Currently, guidelines for identification of infants who need ROP evaluations are mainly based on birth weight (BW) and gestational age (GA), with varying cutoffs.7,8 However, these guidelines have low specificity.9,10 Identification of additional predictive factors for RW-ROP may help identify a subset of highest-risk infants for more frequent retinal imaging to identify the need for eye examinations, while the remaining lower-risk infants could be screened at lower frequency. Previous research has identified several postnatal risk factors for severe ROP including slow postnatal weight gain,1113 neonatal infection,14,15 and high oxygen supplementation.16 One report from a large randomized clinical trial conducted in the late 1980s examined both the demographic (BW, GA, and race/ethnicity) and ROP characteristics, and it found that race/ethnicity and ocular characteristics of ROP (ie, stage, zone, and plus disease) independently predicted an unfavorable ROP outcome.17,18 In the current study, we hypothesized that the ocular findings from the first study-related eye examination, along with demographic characteristics and postnatal factors, may help identify high-risk infants who will subsequently develop RW-ROP. To test this hypothesis, we assessed nonocular and ocular characteristics for predicting the development of RW-ROP in a subset of nearly 1300 premature infants enrolled from 13 participating clinical centers for the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Methods

The e-ROP Study is a multicenter observational cohort study, supported by the National Eye Institute of the National Institutes of Health, to evaluate the validity, reliability, feasibility, safety, and relative cost-effectiveness of a digital imaging system to identify infants with RW-ROP. The details of the design, methods, and primary outcome results for the e-ROP Study have been published.19,20 Major features of the diagnostic examination and the candidate predictors for RW-ROP are described here.

Study Participants

Infants eligible for enrollment into the e-ROP Study had BW less than 1251 g and were likely to survive 28 days and remain in neonatal intensive care units for serial ROP examinations. Written informed consent was obtained from the parents/guardians of eligible infants. The study protocol and informed consent process was approved by the institutional review boards of the participating clinical centers.

Diagnostic Eye Examination

Enrolled infants underwent serial diagnostic examinations in both eyes by e-ROP Study–certified ophthalmologists. The first study-related eye examination started at 4 weeks after birth or at 32 weeks’ postmenstrual age (PMA), whichever came later for babies with GA of 28 weeks or greater. For babies with GA of less than 28 weeks, the protocol at the individual clinical center determined the timing of the first eye examination. Earlier examination prior to the first e-ROP Study–related eye examination could have been conducted by nonstudy ophthalmologists, but their data were not included in this study. Follow-up diagnostic examinations were conducted at least every other week, unless medically contraindicated,19 until the ophthalmologist noted 1 of the following: mature retinal vessels, immature zone III on 2 occasions at least 7 days apart, ROP regressed or regressing on 2 visits at least 7 days apart, treatment for severe ROP, or the infant reached 40 weeks PMA with no ROP or only stage 1 or 2 ROP.

The results of ROP (stage, zone, and preplus/plus disease) from eye examinations were summarized using the 2005 International Classification of ROP.21 Using the findings of the diagnostic examination, RW-ROP was defined as the presence of plus disease, zone I ROP, or ROP stage 3 or greater in either eye. As previously noted, this definition of RW-ROP is equivalent to the Early Treatment for ROP Trial type 1 ROP and type 2 ROP.5,21

Predictive Factors

Demographic characteristics were collected from the infant’s parent/guardian at enrollment including BW, GA, sex, race, ethnicity, and singleton or multiple births.

Data on feeding and respiratory status of an infant in the past 12 hours and the most recent weight prior to the first study-related eye examination were collected by the study coordinator. Based on the BW and weight at the first study-related eye examination, the weight gain rate was calculated using 2 approaches. First, we calculated the weight gain rate (grams/day) as the weight gain between birth and the first study-related eye examination divided by days of life at first study-related eye examination. This weight gain rate has been found to be predictive of the risk for ROP.11,12 Second, we calculated the relative weight gain rate (grams/kilograms/day) as the weight gain rate (grams/day)/(average of weight on examination date and birth weight). This relative weight gain rate has been used in the neonatal intensive care unit for monitoring growth and, conversely, the nutritional deficit in preterm infants.22,23

At the first study-related eye examination performed by a study-certified ophthalmologist at a study clinical center, ocular findings were recorded including characteristics of posterior pole vessels; quadrants with plus/preplus disease; most dominant vascular feature; presence, stage, and zone of ROP; and retinal hemorrhage.

Statistical Analysis

We first analyzed the predictors of RW-ROP using univariate analysis followed by a multivariate logistic regression model that included predictors with P < .10 in the univariate analyses and their 2-way interaction terms. The multivariate model went through backward selection of predictors, and the final multivariate model retained the predictors with P < .05. Ocular predictors at the first study-related eye examination were determined based on the worse eye. Development of RW-ROP was defined as the presence of RW-ROP in either eye at any study-related follow-up eye examination. The odds ratio (OR) with 95% CIs associated with each predictor were calculated from logistic regression models. The predictions of RW-ROP were evaluated using the area under receiver operating characteristic curve (AUC). Sensitivity and specificity corresponding to various cut points of predicted probability of RW-ROP from final multivariate model were also calculated. All the statistical analyses were performed in SAS version 9.4 (SAS Institute Inc).

Results

Among 1284 infants enrolled into the e-ROP Study, 979 infants (76.2%) had at least 2 study-related eye examinations and did not have RW-ROP at the first study-related eye examination; thus, they were included in this risk-prediction analysis (eFigure in the Supplement). The first study-related eye examination on these 979 eligible infants was conducted at a median PMA of 33 weeks, and the last study-related eye examination at a median PMA of 38 weeks, with a median of 3 eye examinations conducted at the e-ROP Study clinical centers. The mean BW was 860 g (range, 330-1250 g) and the mean GA was 27 weeks (range, 23-34 weeks). A total of 149 infants (15.2%) developed RW-ROP in 1 eye (27 infants) or both eyes (122 infants). The median time from first study-related eye examination to the development of RW-ROP was 20 days (range, 2-75 days). Approximately 61% of infants (91 of 149) with RW-ROP were treated.

In univariate analyses (eTables 1-3 in the Supplement), demographic predictors (eTable 1 in the Supplement) found to be associated with RW-ROP were low BW (P < .001), low GA (P < .001), male sex (P = .03), and nonblack race (P = .02). The ocular findings from the first study-related eye examination (eTable 2 in the Supplement) found to be associated with RW-ROP were the presence of preplus disease (P < .001), larger number of quadrants with preplus disease (P < .001), zone I incomplete vascularization (P = .04), stage 1 or 2 ROP (P < .001), zone II ROP (P < .001), and the presence of retinal hemorrhage (P < .001). The need for respiratory support (P < .001) or the absence of enteral feedings (P = .004) proceeding the first study-related eye examination was associated with a high risk for RW-ROP. A slower weight gain (≤12 g/d) by the time of the first study-related eye examination was associated with increased risk for RW-ROP (OR, 1.96; 95% CI, 1.20-3.22) when compared with the infants with weight gain of more than 18 g/d. However, relative weight gain (grams/kilograms/day) was not associated with the risk for RW-ROP (P = .67; eTable 3 in the Supplement).

In the multivariate analysis (Table 1), demographic and clinical factors at the first study-related eye examination that were independently associated with an increased risk for RW-ROP included male sex (OR, 1.80; 95% CI, 1.13-2.86; P = .01), nonblack race (OR, 2.76; 95% CI, 1.50-5.08 for white vs black race and OR, 4.81; 95% CI, 2.19-10.6 for other vs black race; P < .001), low BW (OR, 5.16; 95% CI, 1.12-7.20 for ≤500 g vs >1100 g; P = .049), younger GA (OR, 9.79; 95% CI, 3.49-27.5 for ≤24 weeks vs ≥28 weeks; P < .001), more quadrants with preplus disease (OR, 7.12; 95% CI, 2.53-20.1 for 1-2 quadrants and OR, 18.4; 95% CI, 4.28-79.4 for 3-4 quadrants vs no preplus disease; P < .001), stage 2 ROP (OR, 4.13; 95% CI, 2.13-8.00 vs no ROP; P < .001), the presence of retinal hemorrhage (OR, 4.36; 95% CI, 1.57-12.1 vs absence; P = .005), the need for controlled mechanical ventilator or high-frequency oscillatory ventilation (OR, 4.99; 95% CI, 1.89-13.2 for the need for controlled mechanical ventilator; OR, 11.0; 95% CI, 2.26-53.8 for the need for high-frequency oscillatory ventilation vs no respiratory support; P < .001), and weight gain of 12 g/d or less (OR, 2.44; 95% CI, 1.22-4.89 for weight gain ≤12 g/d vs >18 g/d; P = .001).

The results for predictions of RW-ROP using significant predictors are presented in eTable 4 in the Supplement and Figure 1. The combination of BW and GA predicted the RW-ROP with AUC of 0.78 (95% CI, 0.75-0.82). When the sex and race/ethnicity of the infant were also included, the AUC improved to 0.81 (P = .02 for comparing with the AUC from BW and GA). When the ocular findings from the first study-related eye examination were also added to the prediction model with demographic characteristics (ie, BW, GA, sex, and race/ethnicity), the prediction was again improved with AUC of 0.85 (P < .001). Finally, when respiratory support status and weight gain were added in the final multivariate model, the AUC was 0.88 (95% CI. 0.85-0.91), significantly better than the prediction by demographic characteristics (P < .001) and also better than the prediction by demographics and ocular findings (P = .005).

The predicted risk for RW-ROP from the final multivariate model is shown in Figure 2 for each infant with RW-ROP and without RW-ROP. The sensitivity and specificity for predicting RW-ROP at various cut points of predicted probability are in Table 2. Of note, the sensitivity was 96.0% (95% CI, 91.5-98.1%) and specificity was 52.7% (95% CI, 49.2-56.0) when the predicted probability of 0.05 or greater was considered as high risk (Table 2).

Discussion

We evaluated the demographic, ocular, and medical status factors for predicting the development of RW-ROP based on data from infants with BW less than 1251 g enrolled into a large multicenter clinical study conducted in 13 clinical centers in North America. Our results confirmed that BW, GA, nonblack race (white or other), and slow weight gain were associated with an increased risk for developing RW-ROP. Furthermore, the study identified that the need for mechanical ventilation and several ocular findings (preplus disease, stage 2 ROP, and retinal hemorrhage) at the first study-related eye examination are important predictors for RW-ROP. The combination of these factors predicted RW-ROP significantly better than BW and GA combined (AUC, 0.88 vs 0.78; P < .001). Although these study findings may not be generalizable to other populations, the results suggest that ROP examination guidelines, currently based on BW and GA alone, can be modified to include other factors to improve detection of higher-risk infants.

It is important to note that this study determined the predictive factors for RW-ROP and did not attempt to predict only eyes with treatment-requiring ROP. Referral-warranted ROP includes the retinal findings of type 1 ROP and type 2 ROP as defined by the Early Treatment for ROP Trial.5,21 The rationale for using RW-ROP as a prediction outcome was driven by our planned use of a prediction model to identify a subset of high-risk infants who require more frequent evaluations. If validated by an independent study, our prediction model could be useful for risk stratification for infants at risk for RW-ROP.

Low BW and low GA are major predictors of ROP.13,17,18,24 This study confirmed their independent association with ROP. However, these 2 factors only moderately predicted RW-ROP with AUC of 0.78. As shown in other studies,13,24 GA tends to be more associated with ROP than BW (AUC of 0.77 for GA and 0.74 for BW). In this study, when controlled for GA, BW remained significant only for the smallest infants (<500 g).

We confirmed that lower postnatal weight gain was independently associated with an increased risk for RW-ROP.1113 The presumed mechanism for this association was that a low rate of weight gain is a proxy for low insulinlike growth factor I levels,25 which can result in a lower rate of retinal vessel growth.26 Low weight gain could also be a proxy for insufficient nutrition, severe lung disease, or other morbidities. We found that the relative weight gain rate (ie, grams/kilograms/day) was not associated with the risk for RW-ROP; this could be owing to the poor growth in this cohort of premature babies with significant comorbidities.

Our study found that black infants had a significantly lower risk for RW-ROP than those of white and other races/ethnicities. This result is consistent with findings from large natural history cohort studies of infants in the Cryotherapy for Retinopathy of Prematurity Study,17,18 the Early Treatment for Retinopathy of Prematurity Study,24 and a few other large observational clinical studies.27,28

Multiple-birth status has been reported to be associated with the increased risk for severe ROP.17,18,24,27 However, this study did not find multiple birth to be associated with the risk for RW-ROP. This lack of association was also found in a very large cohort of infants (N = 2105) in New Zealand.29

Our study was unique in that it combined both nonocular and ocular characteristics as candidate predictive factors. This study found that the presence of preplus disease, stage 2 ROP, and retinal hemorrhage at the first study-related eye examination were independently associated with an increased risk for subsequent development of RW-ROP. Approximately 5% of eyes were noted to have preplus disease at the first eye examination. These infants were 14 times more likely to develop RW-ROP. The number of quadrants with preplus disease was associated with RW-ROP in a dose-response manner, possibly owing to the progression of preplus to plus disease in most eyes (32 of 48, 66.7%). However, the absence of preplus disease at the first eye examination did not preclude the development of RW-ROP, as we found that 12.6% of infants who did not have preplus disease in either eye at the first study-related eye examination developed severe RW-ROP. Retinal hemorrhage at the first eye examination was uncommon (3.7% eyes), yet it was associated with an increased risk for developing RW-ROP (OR, 4.36; P < .001). Although we have not evaluated the location, type, and extent of retinal hemorrhage in this analysis, this association may suggest that retinal hemorrhage is an important sign of abnormal vasoproliferation. Incorporating ocular findings (preplus disease, stage 2 ROP, and retinal hemorrhage) from the first study-related eye examination improved the prediction of RW-ROP beyond the prediction based on demographic information alone (AUC, 0.85 vs 0.81; P = .001).

We found that the need for mechanical ventilation at the first study-related eye examination was an important predictor for RW-ROP. In this cohort, the first ROP examination occurred at a median PMA of 33 weeks. Mechanical ventilator use beyond 28 days of age is also a risk factor for severe lung disease and poor neurodevelopmental outcome.30 Severe chronic lung disease and its association with high oxygen intake is associated with an increased prevalence of ROP.31,32 Prolonged use of the mechanical ventilator is also associated with a higher risk for additional morbidities, such as late-onset infection and poor growth, which also contribute to the increased risk for RW-ROP.33

This prediction model effectively predicted the subsequent development of RW-ROP in 1 or both eyes of an infant at a median of 20 days (interquartile range, 7-28 days) prior to its occurrence. Including all of the predictive factors, AUC was 0.88 (95% CI, 0.85-0.91), which is very good.34 When the predicted probability of RW-ROP of 0.05 was used as a cut point, the model had sensitivity of 96% and specificity of 53%, implying that a large number of eye examinations (approximately 45%) could be avoided while still detecting most infants with RW-ROP.

The strengths of this study included the large number of infants enrolled from multiple clinical centers; the standard ROP examinations conducted by study-certified ophthalmologists; and the comprehensive evaluation of predictors that included both demographic and clinical characteristics and the ocular findings from the first study-related eye examination. However, the study was limited in that some first study-related eye examinations were not conducted at the same time when the first ROP eye examination is routinely performed because eligible infants may have been transferred in to an e-ROP Study clinical center after an eye examination. Also, the study excluded some babies who only underwent 1 study-related eye examination or who already had RW-ROP in their first study-related eye examination. These exclusions may limit the generalizability of our study findings. The inclusion of some infants (27.3%) who only completed 2 study-related eye examinations could potentially introduce bias. However, analysis among the subset of infants who completed at least 3 study-related eye examinations with their first eye examination no later than 33 weeks and the last eye examination no earlier than 37 weeks showed similar results. The current report may also be limited by possible misclassification of RW-ROP due to the inherent variability in the results of binocular indirect ophthalmoscopy examinations performed by ophthalmologists.35

Conclusions

Currently, the detection of treatment-requiring ROP is based on serial retinal examinations performed by skilled ophthalmologists, with only about 5% to 10% of infants needing treatment.8,36 Using a model like the one presented here could help identify lower-risk infants who are less likely to develop RW-ROP and require less-frequent imaging or eye examinations.19 The model also could identify infants who are at the highest risk and thus require more intensive imaging and examination schedules. If this tiered approach to ROP screening, based on an infant’s risk for developing RW-ROP, is validated, it would reduce the burden of ROP clinical examinations without compromising the timely identification of infants requiring treatment for acute ROP.

Back to top
Article Information

Corresponding Author: Gui-Shuang Ying, PhD, Department of Ophthalmology, Scheie Eye Institute, University of Pennsylvania, Perelman School of Medicine, 3535 Market St, Ste 700, Philadelphia, PA 19104 (gsying@mail.med.upenn.edu).

Submitted for Publication: September 5, 2014; final revision received October 15, 2014; accepted October 21, 2014.

Published Online: December 18, 2014. doi:10.1001/jamaophthalmol.2014.5185.

Author Contributions: Dr Ying had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Ying, Quinn.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Ying, Quinn, Baumritter.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Ying, Quinn, Wade.

Obtained funding: Quinn.

Administrative, technical, or material support: Quinn, Baumritter, Daniel.

Study supervision: Ying, Daniel.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Ying reported receiving personal fees from Janssen Research and Development LLC outside this work. No other disclosures were reported.

Funding/Support: This study was funded by a cooperative agreement grant (U10 EY017014) from the National Eye Institute of the National Institutes of Health, Department of Health and Human Services. Drs Ying, Quinn, Wade, Repka, and Daniel and Ms Baumritter report receiving grants from the National Eye Institute during the conduct of this study. Dr Repka reports funding from the American Academy of Ophthalmology outside this work.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: Members of the e-ROP Cooperative Group include the following: Office of Study Chair: The Children’s Hospital of Philadelphia: Graham E. Quinn, MD, MSCE (principal investigator [PI]); Kelly C. Wade, MD, PhD, MSCE; Agnieshka Baumritter, MS; Trang B. Duros; and Lisa Erbring. Baltimore, Maryland: Johns Hopkins University: Michael X. Repka, MD (PI); Jennifer A. Shepard, CRNP, David Emmert, BA; and C. Mark Herring. Boston, Massachusetts: Boston Children’s Hospital: Deborah VanderVeen, MD (PI); Suzanne Johnston, MD; Carolyn Wu, MD; Jason Mantagos, MD; Danielle Ledoux, MD; Tamar Winter, RN, BSN, IBCLC; Frank Weng; and Theresa Mansfield, RN. Columbus, Ohio: Nationwide Children’s Hospital and Ohio State University Hospital: Don L. Bremer, MD (PI); Mary Lou McGregor, MD; Catherine Olson Jordan, MD; David L. Rogers, MD; Rae R. Fellows, MEd, CCRC; Suzanne Brandt, RNC, BSN; and Brenda Mann, RNC, BSN. Durham, North Carolina: Duke University: David Wallace, MD (PI); Sharon Freedman, MD; Sarah K. Jones; Du Tran-Viet; and Rhonda “Michelle” Young. Louisville, Kentucky: University of Louisville: Charles C. Barr, MD (PI); Rahul Bhola, MD; Craig Douglas, MD; Peggy Fishman, MD; Michelle Bottorff; Brandi Hubbuch, RN, MSN, NNP-BC; and Rachel Keith, PhD. Minneapolis, Minnesota: University of Minnesota: Erick D. Bothun, MD (PI); Inge DeBecker, MD; Jill Anderson, MD; Ann Marie Holleschau, BA, CCRP; Nichole E. Miller, MA, RN, NNP; and Darla N. Nyquist, MA, RN, NNP. Oklahoma City, Oklahoma: University of Oklahoma: R. Michael Siatkowski, MD (PI); Lucas Trigler, MD; Marilyn Escobedo, MD; Karen Corff, MS, ARNP, NNP-BC; Michelle Huynh, MS, ARNP; and Kelli Satnes, MS, ARNP, NNP-BC. Philadelphia, Pennsylvania: Children’s Hospital of Philadelphia: Monte D. Mills, MD (PI); Will Anninger, MD; Gil Binenbaum, MD, MSCE; Graham Quinn, MD, MSCE; Karen A. Karp, BSN; and Denise Pearson, COMT. San Antonio, Texas: University of Texas: Alice Gong, MD (PI); John Stokes, MD; Clio Armitage Harper, MD; Laurie Weaver; Carmen McHenry, BSN; Kathryn Conner; Rosalind Heemer; and Elnora Cokley, RNC. Salt Lake City, Utah: University of Utah: Robert Hoffman, MD (PI); David Dries, MD; Katie Jo Farnsworth; Deborah Harrison, MS; Bonnie Carlstrom; and Cyrie Ann Frye, CRA, OCT-C. Nashville, Tennessee: Vanderbilt University: David Morrison, MD (PI); Sean Donahue, MD; Nancy Benegas, MD; Sandy Owings, COA, CCRP; Sandra Phillips, COT, CRI; and Scott Ruark. Calgary, Alberta, Canada: Hospital of the Foothills Medical Center: Anna Ells, MD, FRCS (PI); Patrick Mitchell, MD; April Ingram; and Rosie Sorbie, RN.

Data Coordinating Center: University of Pennsylvania School of Medicine: Gui-shuang Ying, PhD (PI); Maureen Maguire, PhD; Mary Brightwell-Arnold, BA, SCP; Maxwell Pistilli, MS; Kathleen McWilliams, CCRP; Sandra Harris; and Claressa Whearry.

Image Reading Center: University of Pennsylvania School of Medicine: Ebenezer Daniel, MBBS, MS, MPH (PI); E. Revell Martin, BA; Candace R. Parker Ostroff; Krista Sepielli; and Eli Smith.

Expert Readers: Antonio Capone, MD (The Vision Research Foundation, Royal Oak, Michigan); G. Baker Hubbard, MD (Emory University School of Medicine, Atlanta, Georgia); and Anna Ells, MD, FRCS (University of Calgary Medical Center, Calgary, Alberta, Canada).

Image Data Management Center: Inoveon Corp: Peter Lloyd Hildebrand, MD (PI); Kerry Davis; G. Carl Gibson; and Regina Hansen.

Cost-Effectiveness Component: Alex R. Kemper, MD, MPH, MS (PI), and Lisa Prosser, PhD.

Data Management and Oversight Committee: David C. Musch, PhD, MPH (chair); Stephen P. Christiansen, MD; Ditte J. Hess, CRA; Steven M. Kymes, PhD; SriniVas R. Sadda, MD; and Ryan Spaulding, PhD.

National Eye Institute: Eleanor B. Schron, PhD, RN.

References
1.
Gilbert  C.  Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84(2):77-82.
PubMedArticle
2.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Arch Ophthalmol. 1988;106(4):471-479.
PubMedArticle
3.
Good  WV, Hardy  RJ; Early Treatment for Retinopathy of Prematurity Cooperative Group.  The multicenter study of Early Treatment for Retinopathy of Prematurity (ETROP). Ophthalmology. 2001;108(6):1013-1014.
PubMedArticle
4.
Mintz-Hittner  HA, Kennedy  KA, Chuang  AZ; BEAT-ROP Cooperative Group.  Efficacy of intravitreal bevacizumab for stage 3+ retinopathy of prematurity. N Engl J Med. 2011;364(7):603-615.
PubMedArticle
5.
Early Treatment for Retinopathy of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the Early Treatment for Retinopathy of Prematurity Randomized Trial. Arch Ophthalmol. 2003;121(12):1684-1694.
PubMedArticle
6.
Ells  AL, Holmes  JM, Astle  WF,  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology. 2003;110(11):2113-2117.
PubMedArticle
7.
Gilbert  C, Fielder  A, Gordillo  L,  et al; International NO-ROP Group.  Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs. Pediatrics. 2005;115(5):e518-e525.
PubMedArticle
8.
Section on Ophthalmology American Academy of Pediatrics; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus.  Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2006;117(2):572-576.
PubMedArticle
9.
Lee  SK, Normand  C, McMillan  D, Ohlsson  A, Vincer  M, Lyons  C; Canadian Neonatal Network.  Evidence for changing guidelines for routine screening for retinopathy of prematurity. Arch Pediatr Adolesc Med. 2001;155(3):387-395.
PubMedArticle
10.
Haines  L, Fielder  AR, Scrivener  R, Wilkinson  AR; Royal College of Paediatrics and Child Health, the Royal College of Ophthalmologists and British Association of Perinatal Medicine.  Retinopathy of prematurity in the UK I: the organisation of services for screening and treatment. Eye (Lond). 2002;16(1):33-38.
PubMedArticle
11.
Binenbaum  G, Ying  GS, Quinn  GE,  et al; Premature Infants in Need of Transfusion Study Group.  A clinical prediction model to stratify retinopathy of prematurity risk using postnatal weight gain. Pediatrics. 2011;127(3):e607-e614.
PubMedArticle
12.
Binenbaum  G, Ying  GS, Quinn  GE,  et al.  The CHOP postnatal weight gain, birth weight, and gestational age retinopathy of prematurity risk model. Arch Ophthalmol. 2012;130(12):1560-1565.
PubMedArticle
13.
Hellström  A, Smith  LEH, Dammann  O.  Retinopathy of prematurity. Lancet. 2013;382(9902):1445-1457.
PubMedArticle
14.
Mittal  M, Dhanireddy  R, Higgins  RD.  Candida sepsis and association with retinopathy of prematurity. Pediatrics. 1998;101(4, pt 1):654-657.
PubMedArticle
15.
Manzoni  P, Maestri  A, Leonessa  M, Mostert  M, Farina  D, Gomirato  G.  Fungal and bacterial sepsis and threshold ROP in preterm very low birth weight neonates. J Perinatol. 2006;26(1):23-30.
PubMedArticle
16.
Chen  ML, Guo  L, Smith  LE, Dammann  CE, Dammann  O.  High or low oxygen saturation and severe retinopathy of prematurity: a meta-analysis. Pediatrics. 2010;125(6):e1483-e1492.
PubMedArticle
17.
Schaffer  DB, Palmer  EA, Plotsky  DF,  et al; The Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Prognostic factors in the natural course of retinopathy of prematurity. Ophthalmology. 1993;100(2):230-237.
PubMedArticle
18.
Hardy  RJ, Palmer  EA, Dobson  V,  et al; Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Risk analysis of prethreshold retinopathy of prematurity. Arch Ophthalmol. 2003;121(12):1697-1701.
PubMedArticle
19.
Graham E Quinn; e-ROP Cooperative Group.  Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity: study design. Ophthalmic Epidemiol. 2014;21(4):256-267.
PubMedArticle
20.
Quinn  GE, Ying  GS, Daniel  E,  et al; e-ROP Cooperative Group.  Validity of a telemedicine system for the evaluation of acute-phase ROP. JAMA Ophthalmol. 2014;132(10):1178-1184.
PubMedArticle
21.
International Committee for the Classification of Retinopathy of Prematurity.  The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol. 2005;123(7):991-999.
PubMedArticle
22.
Senterre  T, Rigo  J.  Reduction in postnatal cumulative nutritional deficit and improvement of growth in extremely preterm infants. Acta Paediatr. 2012;101(2):e64-e70.
PubMedArticle
23.
Fenton  TR, Nasser  R, Eliasziw  M, Kim  JH, Bilan  D, Sauve  R.  Validating the weight gain of preterm infants between the reference growth curve of the fetus and the term infant. BMC Pediatr. 2013;13:92.
PubMedArticle
24.
Good  WV, Hardy  RJ, Dobson  V,  et al; Early Treatment for Retinopathy of Prematurity Cooperative Group.  The incidence and course of retinopathy of prematurity: findings from the early treatment for retinopathy of prematurity study. Pediatrics. 2005;116(1):15-23.
PubMedArticle
25.
Hellström  A, Engström  E, Hård  AL,  et al.  Postnatal serum insulin-like growth factor I deficiency is associated with retinopathy of prematurity and other complications of premature birth. Pediatrics. 2003;112(5):1016-1020.
PubMedArticle
26.
Hellstrom  A, Perruzzi  C, Ju  M,  et al.  Low IGF-I suppresses VEGF-survival signaling in retinal endothelial cells: direct correlation with clinical retinopathy of prematurity. Proc Natl Acad Sci U S A. 2001;98(10):5804-5808.
PubMedArticle
27.
Yang  MB, Rao  S, Copenhagen  DR, Lang  RA.  Length of day during early gestation as a predictor of risk for severe retinopathy of prematurity. Ophthalmology. 2013;120(12):2706-2713.
PubMedArticle
28.
Saunders  RA, Donahue  ML, Christmann  LM,  et al; The Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Racial variation in retinopathy of prematurity. Arch Ophthalmol. 1997;115(5):604-608.
PubMedArticle
29.
Darlow  BA, Hutchinson  JL, Henderson-Smart  DJ, Donoghue  DA, Simpson  JM, Evans  NJ; Australian and New Zealand Neonatal Network.  Prenatal risk factors for severe retinopathy of prematurity among very preterm infants of the Australian and New Zealand Neonatal Network. Pediatrics. 2005;115(4):990-996.
PubMedArticle
30.
Walsh  MC, Morris  BH, Wrage  LA,  et al; National Institutes of Child Health and Human Development Neonatal Research Network.  Extremely low birthweight neonates with protracted ventilation: mortality and 18-month neurodevelopmental outcomes. J Pediatr. 2005;146(6):798-804.
PubMedArticle
31.
Carlo  WA, Finer  NN, Walsh  MC,  et al; SUPPORT Study Group of the Eunice Kennedy Shriver NICHD Neonatal Research Network.  Target ranges of oxygen saturation in extremely preterm infants. N Engl J Med. 2010;362(21):1959-1969.
PubMedArticle
32.
Stenson  B, Brocklehurst  P, Tarnow-Mordi  W; UK BOOST II trial; Australian BOOST II trial; New Zealand BOOST II trial.  Increased 36-week survival with high oxygen saturation target in extremely preterm infants. N Engl J Med. 2011;364(17):1680-1682.
PubMedArticle
33.
Sood  BG, Madan  A, Saha  S,  et al; NICHD neonatal research network.  Perinatal systemic inflammatory response syndrome and retinopathy of prematurity. Pediatr Res. 2010;67:394-400.
PubMedArticle
34.
Choi  BC.  Slopes of a receiver operating characteristic curve and likelihood ratios for a diagnostic test. Am J Epidemiol. 1998;148(11):1127-1132.
PubMedArticle
35.
Reynolds  JD, Dobson  V, Quinn  GE,  et al; CRYO-ROP and LIGHT-ROP Cooperative Study Groups.  Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol. 2002;120(11):1470-1476.
PubMedArticle
36.
Haines  L, Fielder  AR, Baker  H, Wilkinson  AR.  UK population based study of severe retinopathy of prematurity: screening, treatment, and outcome. Arch Dis Child Fetal Neonatal Ed. 2005;90(3):F240-F244.
PubMedArticle
×