Key PointsQuestion
Do postnatal weight gain–based algorithms have the potential to identify infants with type 1 retinopathy of prematurity (ROP) or severe ROP?
Findings
This systematic review and meta-analysis that included 61 studies (>37 000 infants) found that weight gain–based algorithms have adequate sensitivity, ranging from 0.89 to 1.00, and negative likelihood ratios (<0.2). However, specificity and positive likelihood ratios were inadequate.
Meaning
This study suggests that weight gain–based algorithms have adequate sensitivity and negative likelihood ratios and provide reasonable certainty that type 1 ROP or severe ROP is unlikely to develop (ie, the algorithm is useful for ruling out the disease).
Importance
The currently recommended method for screening for retinopathy of prematurity (ROP) is binocular indirect ophthalmoscopy, which requires frequent eye examinations entailing a heavy clinical workload. Weight gain–based algorithms have the potential to minimize the need for binocular indirect ophthalmoscopy and have been evaluated in different setups with variable results to predict type 1 or severe ROP.
Objective
To synthesize evidence regarding the ability of postnatal weight gain–based algorithms to predict type 1 or severe ROP.
Data Sources
PubMed, MEDLINE, Embase, and the Cochrane Library databases were searched to identify studies published between January 2000 and August 2021.
Study Selection
Prospective and retrospective studies evaluating the ability of these algorithms to predict type 1 or severe ROP were included.
Data Extraction and Synthesis
Two reviewers independently extracted data. This meta-analysis was performed according to the Cochrane guidelines and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines.
Main Outcomes and Measures
Ability of algorithms to predict type 1 or sever ROP was measured using statistical indices (pooled sensitivity, specificity, and summary area under the receiver operating characteristic curves, as well as pooled negative likelihood ratios and positive likelihood ratios and diagnostic odds ratios).
Results
A total of 61 studies (>37 000 infants) were included in the meta-analysis. The pooled estimates for sensitivity and specificity, respectively, were 0.89 (95% CI, 0.85-0.92) and 0.57 (95% CI, 0.51-0.63) for WINROP (Weight, IGF-1 [insulinlike growth factor 1], Neonatal, ROP), 1.00 (95% CI, 0.88-1.00) and 0.60 (95% CI, 0.15-0.93) for G-ROP (Postnatal Growth and ROP), 0.95 (95% CI, 0.71-0.99) and 0.52 (95% CI, 0.36-0.68) for CHOP ROP (Children’s Hospital of Philadelphia ROP), 0.99 (95% CI, 0.73-1.00) and 0.49 (95% CI, 0.03-0.74) for ROPScore, 0.98 (95% CI, 0.94-0.99) and 0.35 (95% CI, 0.22-0.51) for CO-ROP (Colorado ROP). The original PINT (Premature Infants in Need of Transfusion) ROP study reported a sensitivity of 0.98 (95% CI, 0.91-0.99) and a specificity of 0.36 (95% CI, 0.30-0.42). The pooled negative likelihood ratios were 0.19 (95% CI, 0.13-0.27) for WINROP, 0.0 (95% CI, 0.00-0.32) for G-ROP, 0.10 (95% CI, 0.02-0.53) for CHOP ROP, 0.03 (95% CI, 0.00-0.77) for ROPScore, and 0.07 (95% CI, 0.03-0.16) for CO-ROP. The pooled positive likelihood ratios were 2.1 (95% CI, 1.8-2.4) for WINROP, 2.5 (95% CI, 0.7-9.1) for G-ROP, 2.0 (95% CI, 1.5-2.6) for CHOP ROP, 1.9 (95% CI, 1.1-3.3) for ROPScore, and 1.5 (95% CI, 1.2-1.9) for CO-ROP.
Conclusions and Relevance
This study suggests that weight gain–based algorithms have adequate sensitivity and negative likelihood ratios to provide reasonable certainty in ruling out type 1 ROP or severe ROP. Given the implications of missing even a single case of severe ROP, algorithms with very high sensitivity (close to 100%) and low negative likelihood ratios (close to zero) need to be chosen to safely reduce the number of unnecessary examinations in infants at lower risk of severe ROP.
Retinopathy of prematurity (ROP) is a disease of pathologic neovascularization affecting preterm infants. In early postnatal life, hyperoxia leads to suppression of vascular growth factors (phase 1). Subsequently, as retinal hypoxia sets in, there is an upsurge of vascular growth factors leading to unregulated vasoproliferation (phase 2).1 ROP either regresses spontaneously or continues to advance and can progress to cause retinal detachment and blindness if not detected and treated early.2 Infants with lower gestation and lower birth weight have a higher risk of developing ROP. Currently, at-risk infants are screened using repeated eye examinations (binocular indirect ophthalmoscopy [BIO]) starting at approximately 30 to 32 weeks’ postmenstrual age and continuing until the retinal vasculature is fully mature (approximately 40 weeks’ postmenstrual age).3 However, only fewer than 10% of screened infants need treatment for ROP.2
Animal experiments have demonstrated the importance of nutrition and insulinlike growth factor 1 (IGF-1) in the retinal vascular development. Insufficient activation of endothelial growth factor by IGF-1 can alter the development of the retinal vasculature.4 In the clinical setting, low postnatal weight gain is considered as a surrogate marker for slower-than-expected increases in serum IGF-1 levels.5 Based on this hypothesis, risk prediction models such as WINROP (Weight, IGF-1, Neonatal Retinopathy of Prematurity),6 G-ROP (Postnatal Growth and Retinopathy of Prematurity),7 PINT (Premature Infants in Need of Transfusion) ROP,8 CHOP (Children's Hospital of Philadelphia) ROP,9 ROPScore,10 and CO-ROP (Colorado Retinopathy of Prematurity),11 have been evaluated to see whether they can predict the development of significant ROP. These models have the potential advantage of reducing the number of BIO examinations. However, to our knowledge, these models have not been implemented in clinical practice because of their limited generalizability.3
The rationale for this systematic review was to assess whether postnatal weight gain–based algorithms have the potential to predict the development of type 1 or severe ROP. This review gains further importance given the current COVID-19 pandemic, which has led to the curtailment of health services, including ROP screening programs, due to the limited availability of ophthalmologists and mobile screening teams.12 The aim of this systematic review was to synthesize evidence by pooling the diagnostic accuracy indices for postnatal weight gain–based algorithms for predicting type 1 or severe ROP in preterm infants.
This meta-analysis was performed according to the Cochrane guidelines13 and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines.14 The study protocol was registered in PROSPERO (CRD42020172874).15
A systematic search of the PubMed, MEDLINE, Embase, and Cochrane Library databases was performed to identify studies published between January 2000 and August 2021. PubMed was searched using the standard terminology (eMethods in the Supplement). Similar terminology was used while searching other databases. We also searched the Cochrane Library, ClinicalTrials.gov, grey literature (on OpenGrey, Google Scholar, and MedNar). No language restrictions were applied. The reference lists of all publications were searched manually for additional studies.
Two reviewers (S.A. and S.D.) independently collected data from the included studies. Study data were further verified by one of the reviewers (S.R.) who had not collected the study data earlier. A total of 61 studies were included for the final meta-analysis. The following data were collected from each study: year(s) the study was conducted, country, gestational age at birth, birth weight, weekly weight gain, total sample size, follow-up rates, tool used to diagnose ROP (indirect ophthalmoscopy or wide-field digital retinal imaging), prospective or retrospective design, high-income or low- to middle-income country (World Bank list of economies),16 and diagnostic indices (true positives, false positives, true negatives, and false negatives). The process of study selection is shown in the study flow diagram (eFigure 1 in the Supplement).
Criteria for Considering Studies for This Review
The outcomes examined were the sensitivity and specificity of postnatal weight gain–based algorithms to predict type 1 or severe ROP.2 Studies that used type 1 ROP or severe ROP as the target disease were included.
Both prospective and retrospective studies that met the following criteria were included: (1) retinopathy screening for preterm infants and (2) the ability of weight gain–based screening algorithms to predict type 1 or severe ROP. The following standard definitions were accepted for type 1 ROP2: zone I (any stage ROP with plus disease), zone I (stage 3 with or without plus disease), or zone II (stage 2 or 3 ROP with plus disease).
Severe ROP was defined as any ROP in zone I, stage 2 ROP in zone II with plus disease, or any stage 3 ROP. The studies included described the diagnostic ability of weight gain–based algorithms to predict type 1 or severe ROP in comparison with the findings by ophthalmologists either by BIO or by wide-field digital retinal imaging, which were considered the reference standards.3 Two reviewers (S.A. and S.D.) independently decided on the eligibility of studies for inclusion in the systematic review. Differences in opinion were resolved by discussion among all reviewers.
The quality assessment of included studies consisted of the following 4 domains according to the revised version of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2): patient selection, index test, reference standard, and flow and timing.17 In this review, index test refers to the “alarm” in any of the postnatal weight gain–based algorithms for type 1 or severe ROP. The included studies were assessed for risk of bias in each domain and for applicability concerns in the first 3 domains.
Meta-analysis was conducted using STATA, version 16 (StataCorp)18 by one of us (S.R.) in the presence of 2 other authors (S.A. and S.D.). We used the bivariate mixed-effects regression model to derive the pooled sensitivity, specificity, positive likelihood ratio (PLR), and negative likelihood ratio (NLR) and the diagnostic odds ratio (DOR) and respective 95% CIs. A summary receiver operating characteristic (ROC) curve was generated to display the results of individual studies. The following cutoffs were used for interpretation of the area under the ROC values: low (0.5-0.7), moderate (0.71-0.9), or high (>0.9) accuracy.19 The DOR is the ratio of the odds of positivity in the disease relative to the odds of positivity in those without the disease. The value of the DOR ranges from 0 to infinity, with higher values indicating better discriminatory test performance. A likelihood ratio of approximately 1 means the test result neither rules in nor rules out the condition. A PLR above 1 indicates increased evidence of disease; the higher from 1, the more chance of disease. An NLR below 0.1 is very strong evidence to rule out a disease.20
There is uneven practice21 with regard to oxygen saturation targeting, blood transfusion thresholds, and higher incidence of fetal growth restriction in low- and middle-income countries; these factors could have an impact on the performance of these algorithms. Hence, sensitivity analyses were conducted separately for high-income and low- and middle-income countries when possible.
The electronic database search yielded 779 titles and abstracts and 3 additional studies by exploring additional sources. After removal of duplicates, 160 articles were screened based on the title and abstract, and 85 articles were selected. The full text was read for eligibility, of which 61 were included in the final analysis. The 61 studies (>37 000 infants) included WINROP (n = 36),6,22-55 G-ROP (n = 9),7,42,56-61 PINT ROP (n = 1),8 CHOP ROP (n = 6),9,30,55,62-64 ROPScore (n = 5),10,30,55,65,66 and CO-ROP (n = 4).11,67-69 Studies that used a different cutoff score for ROPScore were not included in the analysis.64,70
The general characteristics of the studies included in the systematic review are reported in eTables 1 to 6 in the Supplement. The methodological quality and applicability of the included studies were assessed according to the QUADAS-2 guidelines.17 All 61 studies evaluated one of the weight gain–based algorithms, and there was minimal risk of bias. However, the reference standards used in the studies were either BIO or wide-field digital retinal imaging.
The pooled estimates from 36 studies (n = 11 500) for the sensitivity of the WINROP algorithm to predict type 1 or severe ROP was 0.89 (95% CI, 0.85-0.92), and the pooled estimate of the specificity was 0.57 (95% CI, 0.51-0.63) (Figure 1; Table). The summary area under the ROC curve was 0.82 (95% CI, 0.78-0.85) (Figure 2). The pooled PLR was 2.1 (95% CI, 1.8-2.4), and the pooled NLR was 0.19 (95% CI, 0.13-0.27) (eFigure 2 in the Supplement). The summary PLR and NLR for index test fell in the right lower quadrant of the likelihood matrix (Figure 3A). The pooled DOR of the WINROP algorithm was 11 (95% CI, 7-17).
WINROP (High-Income vs Low- to Middle-Income Countries)
The sensitivity analyses of the WINROP algorithm assessed its performance in high-income and low- to middle-income countries. For high-income countries, the pooled estimate from 24 studies (n = 8543) for the sensitivity of the WINROP algorithm was 0.91 (95% CI, 0.85-0.95), and the pooled estimate for the specificity was 0.60 (95% CI, 0.53-0.66) (Table). The summary area under the ROC curve was 0.82 (95% CI, 0.78-0.85). The PLR was 2.3 (95% CI, 1.9-2.7), and the NLR was 0.15 (95% CI, 0.08-0.26).
For low- to middle-income countries, the pooled estimate from 12 studies (n = 2957) for the sensitivity of the WINROP algorithm was 0.85 (95% CI, 0.78-0.90), and the pooled estimate for the specificity was 0.51 (95% CI, 0.39-0.64) (Table). The summary area under the ROC curve was 0.81 (95% CI, 0.78-0.84). The PLR was 1.7 (95% CI, 1.3-2.3), and the NLR was 0.29 (95% CI, 0.19-0.45).
The pooled estimates from 9 studies (n = 14 120) reported a sensitivity of 1.00 (95% CI, 0.88-1.00) and a specificity (7 studies) of 0.60 (95% CI, 0.15-0.93) for G-ROP to predict type 1 or severe ROP (Figure 4; Table). The summary area under the ROC curve was 0.99 (95% CI, 0.98-1.00). The PLR was 2.5 (95% CI, 0.7-9.1), and the NLR was 0.00 (95% CI, 0.00-0.32) (eFigure 3 in the Supplement). The pooled DOR of the G-ROP algorithm to predict type 1 or severe ROP was 3523 (95% CI, 4-3 155 457). The summary PLR and NLR for index test fell in the left lower quadrant of the likelihood matrix (Figure 3B).
The original PINT ROP study (n = 334) reported a sensitivity of 0.98 (95% CI, 0.91-0.99) and a specificity of 0.36 (95% CI, 0.30-0.42) for PINT ROP to predict type 1 or severe ROP (Table). The PLR was 1.55 (95% CI, 1.41-1.69), and the NLR was 0.04 (95% CI, 0.01-0.29). The DOR was 38.75 (95% CI, 9.00-58.00).
The pooled estimates from 6 studies (n = 2135) showed a sensitivity of 0.95 (95% CI, 0.71-0.99) and a specificity of 0.52 (95% CI, 0.36-0.68) for the CHOP ROP algorithm to predict type 1 or severe ROP (Table). The summary area under the ROC curve was 0.75 (95% CI 0.71-0.79). The PLR was 2.0 (95% CI, 1.5-2.6), and the NLR was 0.10 (95% CI, 0.02-0.53). The pooled DOR of the CHOP ROP algorithm was 20 (95% CI, 4-99).
The pooled estimates from 5 studies (n = 1625) showed a sensitivity of 0.99 (95% CI, 0.73-1.00) and a specificity of 0.49 (95% CI, 0.03-0.74) for ROPScore to predict type 1 or severe ROP (Table). The summary area under the ROC curve was 0.88 (95% CI, 0.84-0.90). The PLR was 1.9 (95% CI, 1.1-3.3), and the NLR was 0.03 (95% CI, 0.00-0.77). The pooled DOR of the ROPScore algorithm was 69 (95% CI, 2-2228).
The pooled estimates from 4 studies (n = 8082) showed a sensitivity of 0.98 (95% CI, 0.94-0.99) and a specificity of 0.35 (95% CI, 0.22-0.51) for CO-ROP to predict type 1 or severe ROP (Table). The summary area under the ROC curve was 0.95 (95% CI, 0.93-0.97). The PLR was 1.5 (95% CI, 1.2-1.9), and the NLR was 0.07 (95% CI, 0.03- 0.16). The pooled DOR of the CO-ROP algorithm was 22 (95% CI, 9-58).
The incidence of blindness after ROP is increasing globally. Early identification and treatment of ROP are of paramount importance to prevent irreversible blindness. Current screening protocols that require frequent retinal examinations place an enormous workload on the health care system. The current COVID-19 pandemic has further strained the health care system owing to staff shortages, bed shortages, and limited availability of transport. Also, there is a risk of COVID-19 transmission to the preterm infants undergoing screening for ROP due to close contact with screening staff. The Royal Ophthalmologist College has issued a statement in this regard to overcome the challenges by rationalizing the ROP screening criteria by using evidence-based weight gain–based algorithms.12
Weight gain–based screening models, such as WINROP, G-ROP, PINT ROP, CHOP ROP, ROPScore, and CO-ROP,6-11 have been evaluated in various studies to identify infants who should be referred for retinal examination. However, the 2018 policy statement from the American Academy of Pediatrics stated that the uses of such weight gain–based algorithms alone are not justified based on the current literature.3 Our systematic review increases the evidence base in this area by including more than 37 000 preterm infants from 61 studies in diverse settings across the world.
The general interpretation of the results of diagnostic accuracy studies are as follows: PLRs of more than 10 or NLRs less than 0.1 generate large and often conclusive changes in the posttest probability, PLRs from more than 5 to 10 or NLRs from 0.1 to 0.2 generate moderate shifts in posttest probability, PLRs from more than 2 to 5 or NLRs from more than 0.2 to 0.5 generate small changes in posttest probability, and PLRs from more than 1 to 2 or NLRs from more than 0.5 to 1 alter posttest probability to a very small degree.71,72 High sensitivity corresponds to a high negative predictive value and a low NLR and is the ideal property of a “rule-out” test.73,74 Because all of the algorithms that we evaluated had an NLR of less than 0.2 and a high sensitivity of 0.89 to 1.00, they are useful to rule out type 1 ROP if the algorithm had “no alarm.”
Given that the sensitivity of G-ROP were better than WINROP and the sample size was larger, G-ROP may be more suitable as a rule-out test compared with WINROP. However, G-ROP is not widely evaluated outside high-income countries. High specificity corresponds to a high positive predictive value and a high PLR and is the ideal property of a “rule-in” test.73,74 Because all of the algorithms had a very low specificity and a very low PLR of 1.5 to 2.5, they may not be suitable for ruling in the disease.
Although the overall sensitivity (0.89) and NLR (0.19) for WINROP were reasonably adequate, they were significantly lower than the validation studies that were performed in the early 2010. The sensitivity was 1.00 in a Swedish cohort6 of 353 infants and a Boston cohort22 of 318 infants. A multicenter study in the US and Canada showed that the sensitivity of the WINROP algorithm was 0.986.24 The sensitivity values were lower in studies conducted in low- to middle-income countries. The publication by Lundgren et al35 in 2018 showed an associaton between the decrease in sensitivity and the change in oxygen target ranges that occurred in 2010. This could have been the explanation for the low sensitivity in low- to middle-income countries, where there is uneven practice with regard to oxygen targeting in preterm infants.44-55
The G-ROP model was originally developed from a retrospective cohort of 7483 infants from 29 North American centers that showed a sensitivity of 1.00 to identify type 1 ROP.7 This finding was further validated in a prospective cohort of 3981 infants in North America with a similar sensitivity of 1.00.57 When validated outside the North American cohort, the sensitivity was 1.00 in a small cohort of infants from Japan,56 the UK, and Egypt60; however, it decreased to approximately 0.91 in a Turkish and Portuguese cohort.42,58
Summary PLRs and NLRs for index tests fell in the right lower quadrant of the likelihood matrix for WINROP and the left lower quadrant for G-ROP. This finding suggests that diagnostic indices are better for G-ROP compared with WINROP. However, the strength of WINROP is that it has been tested extensively in nearly 36 studies across the world.
There are a significant number of infants in low- and middle-income countries who develop type 1 ROP and who are more mature (>28 weeks) and heavier (>1050 g).75 Hence, validating the G-ROP algorithm in these groups of infants will give additional information about the performance of the weight gain component of the G-ROP algorithm (weight gain over 3 growth periods: 10-19 days, 20-29 days, and 30-39 days).
The CHOP ROP model was an updated version of the PINT ROP model and had a sensitivity of 1.00 to identify type 1 ROP in the original cohort and a sensitivity of 0.985 to identify type 1 ROP in a multicenter cohort.9,62 Sensitivities of 0.667 and 0.549 in the CHOP ROP model were reported from 2 different Indian cohorts.55,63 The ROPScore had additional risk factors, such as oxygen use in mechanical ventilation and blood transfusion.10 In the original cohort, it had a sensitivity of 0.9583; however, when it was tested using different cutoff scores, the sensitivity was 0.50 to 1.00 in different cohorts. Proportional weight gain calculated at 6 weeks of postnatal life could potentially miss some of the infants who would develop aggressive posterior ROP within this time period.
This review summarizes the evidence for weight gain–based algorithms and their performance in predicting type 1 or severe ROP. This review has highlighted how the performance of these weight gain–based models has varied in their predictive values depending on the situation in which it was tested. Hesitancy in implementing these algorithms into clinical practice is justifiable because the focus has been on not missing a single child who may need treatment.
Postnatal weight gain–based models are not ideal for infants who show a pattern of nonphysiological weight gain (eg, hydrocephalus, sepsis, and patent ductus arteriosus). Future studies should endeavor to incorporate other risk factors, such as oxygen use and sepsis, into the algorithm to improve their diagnostic performance. Another area of research would be to validate these algorithms against a combined reference standard rather than BIO or wide-field digital retinal imaging individually.
Strengths and Limitations
This study has some strengths. To our knowledge, this is the first systematic review including a meta-analysis to evaluate the diagnostic accuracy of postnatal weight gain–based algorithms with a sample size of more than 37 000 infants. The study sample included infants from both high-income and low- to middle-income countries, and additional analyses were performed in these subgroups. Furthermore, a standardized tool (ie, QUADAS-2) was used for the quality assessment. In addition, this systematic review and meta-analysis followed the recent PRISMA-DTA guidelines for transparent reporting.
This study also has some limitations. The weight gain–based algorithms rely strongly on accurate measurement of weight, which is very challenging in a neonatal intensive care setting and could potentially affect the performance of the algorithms. Apart from the index test, there were multiple heterogeneities present across the studies. These heterogeneities were partly explained by factors such as geographical background, workflow, and the choice of a reference standard, which in itself is known to have interobserver variability.
This systematic review and meta-analysis suggests that weight gain–based algorithms have adequate sensitivity and negative likelihood ratios to provide reasonable certainty in ruling out type 1 or severe ROP. Given the implications of missing even a single case of severe ROP, algorithms with very high sensitivity (close to 100%) and low negative likelihood ratio (close to zero) need to be chosen. These algorithms have the potential to reduce the number of unnecessary examinations for infants at lower risk of severe ROP. Future studies should endeavor to incorporate additional clinical parameters (eg, oxygen use and sepsis), which could potentially improve the diagnostic indices of these algorithms.
Accepted for Publication: September 29, 2021.
Published: November 23, 2021. doi:10.1001/jamanetworkopen.2021.35879
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Athikarisamy S et al. JAMA Network Open.
Corresponding Author: Sam Athikarisamy, MD, Neonatal Directorate, Perth Children’s Hospital, 15 Hospital Ave, Nedlands, Western Australia (sam.athikarisamy@health.wa.gov.au).
Author Contributions: Drs Athikarisamy and Desai had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Athikarisamy, Desai, Rao, Simmer, Lam.
Acquisition, analysis, or interpretation of data: Athikarisamy, Desai, Patole, Rao.
Drafting of the manuscript: Athikarisamy, Desai, Rao.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Athikarisamy, Desai, Rao.
Supervision: Patole, Rao, Simmer.
Conflict of Interest Disclosures: None reported.
2.Early Treatment For Retinopathy Of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial.
Arch Ophthalmol. 2003;121(12):1684-1694. doi:
10.1001/archopht.121.12.1684
PubMedGoogle ScholarCrossref 3.Fierson
WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists. Screening examination of premature infants for retinopathy of prematurity.
Pediatrics. 2018;142(6):e20183061. doi:
10.1542/peds.2018-3061
PubMedGoogle Scholar 4.Hellström
A, Engström
E, Hård
AL,
et al. Postnatal serum insulin-like growth factor I deficiency is associated with retinopathy of prematurity and other complications of premature birth.
Pediatrics. 2003;112(5):1016-1020. doi:
10.1542/peds.112.5.1016
PubMedGoogle ScholarCrossref 7.Binenbaum
G, Bell
EF, Donohue
P,
et al; G-ROP Study Group. Development of modified screening criteria for retinopathy of prematurity: primary results from the Postnatal Growth and Retinopathy of Prematurity Study.
JAMA Ophthalmol. 2018;136(9):1034-1040. doi:
10.1001/jamaophthalmol.2018.2753
PubMedGoogle ScholarCrossref 8.Binenbaum
G, Ying
GS, Quinn
GE,
et al; Premature Infants in Need of Transfusion Study Group. A clinical prediction model to stratify retinopathy of prematurity risk using postnatal weight gain.
Pediatrics. 2011;127(3):e607-e614. doi:
10.1542/peds.2010-2240
PubMedGoogle ScholarCrossref 13.Deeks
JJ, Wisniewski
S, Davenport
C. Chapter 4: Guide to the contents of a Cochrane Diagnostic Test Accuracy Protocol. In: Deeks
JJ, Bossuyt
PM, Gatsonis
C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy: Version 1.0.0. The Cochrane Collaboration; 2013.
15.Athikarisamy SE, Desai S, Rao S, Patole S. National Institute for Health Research: PROSPERO: International prospective register of systematic reviews. Weight gain based algorithm WINROP to predict type 1 or severe retinopathy of prematurity (ROP) in preterm infants: a systematic review and meta-analysis. Accessed June 30, 2021.
https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=172874 20.Šimundić
AM. Measures of diagnostic accuracy: basic definitions.
EJIFCC. 2009;19(4):203-211.
PubMed 23.Flückiger
S, Bucher
HU, Hellström
A, Lövqist
C, Sturm
V, Arri
SJ. The early postnatal weight gain as a predictor of retinopathy of prematurity. Article in German.
Klin Monbl Augenheilkd. 2011;228(4):306-310. doi:
10.1055/s-0031-1273217
PubMedCrossref 24.Wu
C, Löfqvist
C, Smith
LE, VanderVeen
DK, Hellström
A; WINROP Consortium. Importance of early postnatal weight gain for normal retinal angiogenesis in very preterm infants: a multicenter study analyzing weight velocity deviations for the prediction of retinopathy of prematurity.
Arch Ophthalmol. 2012;130(8):992-999. doi:
10.1001/archophthalmol.2012.243
PubMedGoogle ScholarCrossref 26.Lundgren
P, Stoltz Sjöström
E, Domellöf
M,
et al. WINROP identifies severe retinopathy of prematurity at an early stage in a nation-based cohort of extremely preterm infants.
PLoS One. 2013;8(9):e73256. doi:
10.1371/journal.pone.0073256PubMedGoogle Scholar 32.Jagła
M, Peterko
A, Olesińska
K, Szymońska
I, Kwinta
P. Prediction of severe retinopathy of prematurity using the WINROP algorithm in a cohort from Malopolska: a retrospective, single-center study.
Dev Period Med. 2017;21(4):336-343.
PubMedGoogle Scholar 33.Timkovic
J, Pokryvkova
M, Janurova
K, Barinova
D, Polackova
R, Masek
P. Evaluation of the WinROP system for identifying retinopathy of prematurity in Czech preterm infants.
Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub. 2017;161(1):111-116. doi:
10.5507/bp.2016.061
PubMedGoogle ScholarCrossref 34.Ali
E, Al-Shafouri
N, Hussain
A, Baier
RJ. Assessment of WINROP algorithm as screening tool for preterm infants in Manitoba to detect retinopathy of prematurity.
Paediatr Child Health. 2017;22(4):203-206. doi:
10.1093/pch/pxx053
PubMedGoogle ScholarCrossref 35.Lundgren
P, Hård
AL, Wilde
Å, Löfqvist
C, Smith
LEH, Hellström
A. Implementing higher oxygen saturation targets reduced the impact of poor weight gain as a predictor for retinopathy of prematurity.
Acta Paediatr. 2018;107(5):767-773. doi:
10.1111/apa.14049
PubMedGoogle ScholarCrossref 36.Chaves-Samaniego
MJ, Gómez Cabrera
C, Chaves-Samaniego
MC,
et al. Multicenter validation study of the WINROP algorithm as a method for detecting retinopathy of prematurity.
J Matern Fetal Neonatal Med. 2020;33(8):1302-1306. doi:
10.1080/14767058.2018.1517325PubMedGoogle Scholar 37.Ueda
K, Miki
A, Nakai
S, Yanagisawa
S, Nomura
K, Nakamura
M. Prediction of severe retinopathy of prematurity using the weight gain, insulin-like growth factor 1, and neonatal retinopathy of prematurity algorithm in a Japanese population of preterm infants.
Jpn J Ophthalmol. 2020;64(2):223-227. doi:
10.1007/s10384-019-00709-z
PubMedGoogle ScholarCrossref 39.Wirth
M, Naud
A, Caputo
G, Hascoët
JM. Algorithm for predicting threshold retinopathy of prematurity is insufficient and fundus examinations are still needed before 31 weeks.
Acta Paediatr. 2019;108(6):1049-1054. doi:
10.1111/apa.14654
PubMedGoogle ScholarCrossref 40.Desai
S, Athikarisamy
SE, Lundgren
P, Simmer
K, Lam
GC. Validation of WINROP (online prediction model) to identify severe retinopathy of prematurity (ROP) in an Australian preterm population: a retrospective study.
Eye (Lond). 2021;35(5):1334-1339. doi:
10.1038/s41433-020-1094-7
PubMedGoogle ScholarCrossref 42.Almeida
AC, Sandina
T, Azevedo
R,
et al. Retrospective comparison between growth and retinopathy of prematurity model versus WINROP model.
Can J Ophthalmol. 2021;S0008-4182(21)00079-X. Published online March 23, 2021. doi:
10.1016/j.jcjo.2021.02.030PubMedGoogle Scholar 44.Hård
AL, Löfqvist
C, Fortes Filho
JB, Procianoy
RS, Smith
L, Hellström
A. Predicting proliferative retinopathy in a Brazilian population of preterm infants with the screening algorithm WINROP.
Arch Ophthalmol. 2010;128(11):1432-1436. doi:
10.1001/archophthalmol.2010.255
PubMedGoogle ScholarCrossref 46.Sun
H, Kang
W, Cheng
X,
et al. The use of the WINROP screening algorithm for the prediction of retinopathy of prematurity in a Chinese population.
Neonatology. 2013;104(2):127-132. doi:
10.1159/000351297
PubMedGoogle ScholarCrossref 48.Bautista
GC, Ventura
RH. Local validation of WINROP, an online screening tool for retinopathy of prematurity.
Philipp J Ophthalmol. 2018;42(1):15-18.
Google Scholar 50.Sanghi
G, Narang
A, Narula
S, Dogra
MR. WINROP algorithm for prediction of sight threatening retinopathy of prematurity: initial experience in Indian preterm infants.
Indian J Ophthalmol. 2018;66(1):110-113. doi:
10.4103/ijo.IJO_486_17
PubMedGoogle ScholarCrossref 54.Sute
SS, Jain
S, Chawla
D, Narang
S. Use of an online screening algorithm—Weight, Insulin-derived growth factor 1, Neonatal Retinopathy of Prematurity (WINROP) for predicting retinopathy of prematurity in Indian preterm babies.
Indian J Ophthalmol. 2021;69(5):1214-1218. doi:
10.4103/ijo.IJO_1521_20
PubMedGoogle Scholar 55.Thomas
D, Madathil
S, Thukral
A,
et al. Diagnostic accuracy of WINROP, CHOP-ROP and ROPScore in detecting type 1 retinopathy of prematurity.
Indian Pediatr. 2021;58(10):915-921.
PubMedGoogle Scholar 57.Binenbaum
G, Tomlinson
LA, de Alba Campomanes
AG,
et al; Postnatal Growth and Retinopathy of Prematurity (G-ROP) Study Group. Validation of the Postnatal Growth and Retinopathy of Prematurity screening criteria.
JAMA Ophthalmol. 2020;138(1):31-37. doi:
10.1001/jamaophthalmol.2019.4517
PubMedGoogle ScholarCrossref 61.Caruggi
S, Scaramuzzi
M, Calevo
MG,
et al. Validation of the Postnatal Growth and Retinopathy of Prematurity screening criteria: a retrospective Italian analysis.
Eur J Ophthalmol. Published online April 22, 2021. doi:
10.1177/11206721211011362PubMedGoogle Scholar 62.Binenbaum
G, Ying
GS, Tomlinson
LA; Postnatal Growth and Retinopathy of Prematurity (G-ROP) Study Group. Validation of the Children’s Hospital of Philadelphia Retinopathy of Prematurity (CHOP ROP) model.
JAMA Ophthalmol. 2017;135(8):871-877. doi:
10.1001/jamaophthalmol.2017.2295
PubMedGoogle ScholarCrossref 65.Figueiredo
R, Sarmento
TM, Garrido
J, Marques
MI, Almeida
T, Carrasquinho
S. Applicability of the ROPScore as a predictive algorithm for early detection of retinopathy of prematurity.
Oftalmologia. 2020;44(2). doi:
10.48560/rspo.18704Google Scholar 66.Gulkas
S, Ozcan
Y, Gumus H. Validity and reliability of ROPScore scoring method to predict the severity of retinopathy of prematurity in premature infants.
Authorea. Preprint posted online March 7, 2021. doi:
10.22541/au.161515047.77230490/v1 70.Lucio
KCDV, Bentlin
MR, Augusto
ACL,
et al. The ROPScore as a Screening Algorithm for Predicting Retinopathy of Prematurity in a Brazilian Population.
Clinics (Sao Paulo). 2018;73:e377. doi:
10.6061/clinics/2018/e377PubMedGoogle Scholar 71.Jaeschke
R, Guyatt
GH, Sackett
DL; Evidence-Based Medicine Working Group. Users’ guides to the medical literature, III: how to use an article about a diagnostic test: B, what are the results and will they help me in caring for my patients?
JAMA. 1994;271(9):703-707. doi:
10.1001/jama.1994.03510330081039
PubMedGoogle ScholarCrossref 73.Rubinstein
ML, Kraft
CS, Parrott
JS. Determining qualitative effect size ratings using a likelihood ratio scatter matrix in diagnostic test accuracy systematic reviews.
Diagnosis (Berl). 2018;5(4):205-214. doi:
10.1515/dx-2018-0061
PubMedGoogle ScholarCrossref 74.Florkowski
CM. Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests.
Clin Biochem Rev. 2008;29(suppl 1):S83-S87.
PubMed 75.Gilbert
C, Fielder
A, Gordillo
L,
et al; International NO-ROP Group. Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs.
Pediatrics. 2005;115(5):e518-e525. doi:
10.1542/peds.2004-1180
PubMedGoogle ScholarCrossref