Validation of a New Retinopathy of Prematurity Screening Method Monitoring Longitudinal Postnatal Weight and Insulinlike Growth Factor I

Objective  To validate in a prospective study the surveillance algorithm WINROP for detecting infants at risk for proliferative retinopathy of prematurity (ROP).

Methods  Fifty preterm infants with a mean gestational age of 26 weeks were included. In the first step of WINROP, weekly measures of body weight and serum insulinlike growth factor I (IGF-I) level from birth until postmenstrual age 36 weeks are entered and compared with expected development. If any of the variables show a negative deviation to a certain degree, an alarm is given. In the second step, gestational age, birth weight, and IGF binding protein 3 level are entered.

Results  The WINROP algorithm identified all children (100% sensitivity) who were diagnosed with proliferative ROP 1.1 to 21.6 weeks later. No infants with no alarm or with alarm at low risk developed proliferative ROP. Alarm at high risk before postmenstrual age 32 weeks was given for 22 of 50 infants (44%); 9 of these infants developed proliferative ROP (54% specificity), of whom 8 were treated.

Conclusion  The WINROP algorithm may be a useful tool for modification of ROP screening.

Retinopathy of prematurity (ROP) in most cases regresses spontaneously. However, in some infants, ROP will progress to retinal detachment and blindness. Treatment (laser photocoagulation) ablates the retina. To avoid unnecessary destructive treatment, it is used only when the risk of progression to blindness is substantial. Currently, all infants with low gestational age (GA) and birth weight (BW) are deemed at risk and are screened with repeated eye examinations to detect late-stage ROP requiring treatment.^1-4 Although screening guidelines differ between countries, present screening criteria do not take into account postnatal clinical factors that are likely to influence the risk of ROP,^5-7 and only about 10% of those screened are found to require treatment.^8,9 Improved clinical criteria to predict which patients will ultimately develop severe ROP and who would therefore benefit from treatment would be of great value. Patients at risk could be closely monitored for treatment, and unnecessary examinations could be avoided.

We have developed a monitoring algorithm¹⁰ based on changes in the development of postnatal factors to improve early prediction of ROP. The algorithm WINROP (weight, insulinlike growth factor [IGF], neonatal) is a 2-stage monitoring algorithm. In the first stage, body weight and IGF-I level are monitored at each consecutive postnatal week with the purpose of detecting a slowdown in either variable as soon as possible. The method of statistical surveillance was used to develop the multivariate monitoring system. Online monitoring systems have been suggested for detecting poor intrauterine growth¹¹ and the incidence of influenza epidemics.^12,13 As we are interested in determining whether a slowdown (in IGF-I level or weight) has occurred since the birth of the child, the algorithm uses all of the observations since the start (or rather all of the differences between the observed and expected values for weight and IGF-I level for each consecutive week). If evidence of a slowdown is detected, an alarm is called. A follow-up test is then made based on IGF-I level, IGF-I binding protein 3 (IGFBP-3) level, weight, BW, and GA at birth.

In the initial study,¹⁰ the WINROP algorithm was tested on the same infants based on which measurements had been developed (ie, an in-sample evaluation). The result of that evaluation was promising and showed that the monitoring method detected 100% of infants in this cohort who required treatment for ROP according to the Early Treatment for Retinopathy of Prematurity criteria¹⁴ with a warning signal at least 4 weeks before onset of stage 3 ROP and 5 weeks before requiring treatment. A large percentage (84%) of those infants who did not develop ROP requiring treatment were also identified in 2 stages.

The aim of this study was to validate the Web-based WINROP system in a new group of very and extremely preterm infants.

Preterm infants born with a GA of less than 31 weeks and treated at the neonatal intensive care unit in Lund, Sweden, between January 16, 2005, and August 23, 2007, were recruited to this study. All of the pregnancies were dated by ultrasonography at 17 to 18 gestational weeks. The infants were defined as small for GA if the deviation in BW was more than 2 SDs below the GA-related mean of the population.¹⁵ Sixty-four eligible infants were identified during this period and 50 infants were included in the validation of WINROP. Of the 14 infants who were not included, 8 died during the study period; 4 had parents who chose not to participate; 1 had grade 3 intraventricular hemorrhage causing ventricular dilatation, hydrocephalus, and excessive nonphysiological weight gain; and 1 did not have enough measuring points and was excluded. Clinical characteristics of the study population are shown in Table 1. All of the infants were inborn and admitted to the neonatal intensive care unit. Enteral feeding with increasing amounts of breast milk was introduced early (2-6 hours after birth). Thereafter, individual fortification of the breast milk was performed. The Regional Committee for Research Ethics, Lund, approved the study, and the parents of the infants gave informed consent.

Venous blood samples (0.5 mL) were taken weekly and the serum was stored at −80°C until assayed. Samples were taken when blood was drawn for other purposes from a patent line during the first postnatal weeks. Subsequent weekly blood samples were obtained by puncture of a peripheral vein. All of the samples from an individual infant were analyzed in the same assay. Concentrations of IGF-I and IGFBP-3 were analyzed using an IGFBP-blocked radioimmunoassay and a specific radioimmunoassay (Mediagnost GmbH, Tübingen, Germany). The IGF-I samples were diluted 1:50 and the IGFBP-3 samples were diluted 1:300. The intra-assay coefficients of variation for IGF-I were 18%, 11%, and 7% at concentrations of 9, 33, and 179 μg/L, respectively. The intra-assay coefficients of variation for IGFBP-3 were 10%, 7%, and 6% at concentrations of 716, 1750, and 3929 μg/L, respectively. All of the samples were analyzed within the same assay. The methods have been described in detail previously.¹⁶

Standardized weight measurements according to clinical routines were performed in all of the infants weekly on the same day as sampling IGF-I and IGFBP-3 from the day of birth until discharge and then again at term age (40 gestational weeks). The infants were weighed outside the incubator on a digital scale with a scale accuracy of 0.001 kg.

Retinopathy of prematurity was classified according to the International Classification of Retinopathy of Prematurity¹⁷ and subdivided into stage 1 (demarcation line), stage 2 (ridge), stage 3 (ridge with extraretinal fibrovascular proliferations), stage 4 (subtotal retinal detachment), and stage 5 (total retinal detachment). In all of the gestational weeks, each child was classified according to the most advanced ROP stage observed. No ROP or stage 1 ROP was labeled ROP0/1, stage 2 ROP was labeled ROP2, and proliferative stage 3 ROP was labeled ROP3 if untreated or ROP3T if treated. The infants were examined according to a routine protocol. Screening was initiated when the baby reached a postmenstrual age (PMA) of 32 weeks and was performed once or twice per week depending on the severity of the disease until the retina was fully vascularized. The pupils were dilated with cyclopentolate hydrochloride, 0.25%, tropicamide, 0.25%, and neosynephrine hydrochloride, 1.25%, 1 hour before examination. The examination was performed by indirect ophthalmoscopy with a 25-diopter lens using indentation and a lid speculum by a trained pediatric ophthalmologist (K.H.) who had no knowledge of the IGF-I level or weight status. Care was taken to minimize pain and stress during the examinations.

Retinopathy of prematurity in stages 0 to 4 (International Classification of Retinopathy of Prematurity)¹⁷ and treatment (by laser ablation) were noted. No infant had plus disease in zone 1 or stage 2 ROP with plus disease in zone 2; thus, only patients with stage 3 ROP were treated.

The WINROP system was evaluated using sensitivity (probability that an alarm is called given that the child is at risk), specificity (probability that an alarm is not called given that the child is not at risk), positive predictive value (probability that the child is at risk given that an alarm is called), and advance alarm time (measured as the average number of weeks from an alarm by WINROP before needing treatment and before classification of stage 3 ROP). In online surveillance, it is important to use measures that reflect the timeliness of the alarms,¹⁸ and here we used the alarm time. Three outcomes are possible with the WINROP monitoring system. An infant being followed up can be classified as the following: (1) at no risk for developing proliferative ROP; (2) at low risk for developing sight-threatening ROP; or (3) at high risk for developing sight-threatening ROP. Because the WINROP system can give 2 types of alarms (low risk and high risk), sensitivity, specificity, and positive predictive value were calculated for each of these steps.

When the prototype version of the WINROP system for the Web was developed, the starting point was algorithms in Excel (Microsoft Corp, Redmond, Washington) with examples of 3 typical scenarios. The algorithm was translated into code in the programming language Ruby (http://www.ruby-lang.org); after it was verified that it behaved the same way as the Excel version, a Web application was built on top of the algorithm using the Web application framework Ruby on Rails (http://www.rubyonrails.org). Ruby and Ruby on Rails were chosen for flexibility reasons. Another reason for the choice of this platform was that in case of extreme performance problems in the future of the WINROP system, there is a clear path of combining Ruby with Java using JRuby and soon of combining it with.NET (Microsoft Corp) using IronRuby to solve these eventual problems if they arise.

In this study, 41 infants did not develop proliferative ROP (stage 3); for these infants, the WINROP system correctly gave no alarm for 13 children (32%) (Table 2). Notably, 5 of these children were born before GA 27 weeks (GA range, 23 weeks 3 days to 26 weeks 4 days). These children had a total of 59 eye examinations. Fifteen children (30%) received an alarm at low risk. Four of these children were born before GA 27 weeks (GA range, 25 weeks 5 days to 26 weeks 4 days). These children had a total of 66 eye examinations. In this low-risk group, 11 infants received their alarm before PMA 32 weeks. The remaining 4 infants received their alarm at PMA 33 or 34 weeks. No infants in the low-risk group developed proliferative ROP. Twenty-two infants (44%) had an alarm at high risk no later than PMA 32 weeks. In this study, all of the 9 high-risk infants were born between GA 23 weeks 0 days and 26 weeks 6 days. These infants had a total of 174 eye examinations. In this high-risk group, 9 of the 22 infants (41%) developed stage 3 ROP, of whom 8 (36%) had laser ablation for their proliferative retinopathy. Despite continuous monitoring after the initial alarm, there were no changes in alarm levels for any child. The results of the continuous monitoring and follow-up test are summarized in the Figure.

The sensitivity for the WINROP system in the first step, detecting stage 3 ROP, was 100% (9 of 9 infants) and the specificity was 32% (13 of 41 infants). In step 2, detecting sight-threatening ROP, the sensitivity was 100% (9 of 9 infants) and the specificity was 54% (15 of 28 infants). The sensitivity, specificity, and positive predictive value are summarized in Table 3.

Twenty-two children had a high-risk alarm, including all of the children with stage 3 ROP (n = 9). Eight of these infants were treated with laser therapy. The WINROP system gave an alarm before signs of proliferative ROP could be seen by the ophthalmologist, with an average advance warning of 9 weeks (range, 1.1-21.6 weeks). On average, there was an advance warning time of 11 weeks (range, 2.6-23.6 weeks) before treatment. All of the infants classified as being at high risk for developing proliferative ROP received an alarm no later than PMA 32 weeks (Table 4).

The main purpose of this study was to validate the Web-based algorithm WINROP based on serial measurements of weight and serum IGF-I level in 50 preterm children. The WINROP system gave no alarm for 13 of 50 infants (26%), and no child with ROP needing treatment by Early Treatment for Retinopathy of Prematurity criteria was within this group.

In the remaining 37 infants, an alarm was given and further evaluation of IGF-I level, IGFBP-3 level, BW, and GA at the time of the alarm was undertaken. Fifteen of these infants were considered to be at low risk for ROP needing treatment, and no child within this group developed proliferative ROP or required treatment. Thus, the algorithm correctly excluded 28 of 41 (67%) of those who did not need treatment (13 with no alarms and 15 with low risk at the second step of testing).

Among the remaining 22 of 50 children (44%) with alarm at high risk, 9 had proliferative ROP; 8 of these 9 infants were treated. The treated children were identified at a mean of 10 weeks before requiring laser therapy by Early Treatment for Retinopathy of Prematurity criteria.¹⁴

In this study, all of the high-risk infants were born between GA 23 weeks 0 days and 26 weeks 6 days. However, 5 children born within that range (before week 27) were considered at no risk (born between GA 23 weeks 3 days and 26 weeks 4 days) and 4 children were considered at low risk (born between GA 25 weeks 5 days and 26 weeks 4 days). Thus, of the infants born extremely preterm, 9 were considered at high risk, 9 were actually correctly cleared by the WINROP system, and all were classified before PMA 32 weeks.

Within current diagnostic screening programs, all examinations aim to identify infants with proliferative sight-threatening retinopathy. Even when the examinations are performed by experienced ophthalmologists, they are often stressful for the infant.^19-21 Approximately 90% to 95%^8,9 of those examined will never require treatment and thus do not benefit from the current screening.

We previously described a new algorithm (WINROP) that successfully detected early indications of an individual infant developing ROP requiring treatment.¹⁰ The WINROP algorithm was also shown to indicate those infants at minimal risk, thus requiring fewer or no eye examinations. The first step in the WINROP system is based on online surveillance of the development of a postnatal growth factor and weight. The WINROP algorithm calculates an alarm statistic that indicates when a child's values deviate from expected development. Because the deviations accumulate over time, a prolonged period of measurements below the expected values gives an increased indication for a slowdown in development. In the calculations performed by the WINROP system, the intercept a is individual, allowing child 1 to have a different intercept than that of child 2. In short, this means WINROP takes into account that each infant has his or her own starting level, which influences the development of both weight and IGF-I over time. When there is enough evidence, an alarm is called and a follow-up test is made to distinguish between a high-risk alarm and a low-risk alarm. At this point, the values for IGF-I level, IGFBP-3 level, weight, BW, and GA are reassessed to confirm whether the child is at high risk for developing ROP requiring treatment. Having several reassessment points makes the calculations in step 2 more robust. It should be emphasized that infants with a nonphysiologic weight gain, eg, children with hydrocephalus, cannot be monitored with this system.

Furthermore, it is important to clarify that the calculations in WINROP are dependent on a correct age determination of the child—if for any reason a child's data are put into the WINROP system with an age younger than that which the child actually is, the lower weight and IGF-I level for that child could be considered less deviating from what would be safe development for that child and therefore could delay the alarm week or at worst give no alarm.

There are generally 2 types of errors possible using a diagnostic tool, namely not detecting those in need of treatment and wrongly indicating those not at risk. We wanted an algorithm that detected all of those at risk (ie, sensitivity was more important than specificity). If the WINROP system is to be used, we must be confident that it identifies 100% of infants at risk for developing sight-threatening ROP. Therefore, in this study the algorithm has been validated on a new study population. This study population was chosen from a different area of Sweden with somewhat different nutrition and oxygen regimens. In addition, it included a more preterm population than that from which the algorithm was developed (26 weeks vs 28 weeks, respectively) and thus had a higher prevalence of infants needing treatment for sight-threatening ROP (16% vs 8%, respectively).

Currently, ROP screening is based on BW and GA of the infant. Screening examinations continue until the retina is fully vascularized or until any ROP regresses spontaneously or after treatment. Treated patients are examined repeatedly for many years. Every infant who fulfills inclusion criteria for screening is followed up with repeated ophthalmologic examinations. In total, there were 299 ophthalmologic screening examinations in this group of children, with 125 examinations performed in infants defined as being at no risk or at low risk by the WINROP system. Undoubtedly, numerous examinations are performed to find those few infants who might benefit from treatment. The examinations are time-consuming, costly, stressful, and, most importantly, often unnecessary.

For all assays, reproducibility is affected by factors such as the quality of sample collection and the use of proper internal and external standards. Interlaboratory variability is often due to differences in 1 or more of these characteristics. Samples drawn for IGF-I and IGFBP-3 measurements should be processed as soon as possible or kept frozen at −70°C or colder for longer storage. Because IGF-I circulates associated with IGFBPs, it is essential that a high-quality assay removes binding protein interference. Similarly, it is essential that the antibodies used in the assays have both high specificity and high affinity for IGF-I and IGFBP-3 (for review, see the article by Clemmons²²). It is important for each laboratory to determine whether its assays give measured IGF-I and IGFBP-3 levels similar to those in the Mediagnost assays or whether a conversion factor is required.

Several studies have shown a close relationship between low postnatal serum IGF-I levels and the development of ROP.^10,23,24 Our finding that longitudinal serum IGF-I levels in addition to weight development correctly identify children at risk for severe ROP suggests that IGF-I supplementation might be a potential preventive treatment for ROP.

To summarize, the WINROP system was able to show that 13 of the 50 infants (26%) had safe development of IGF-I and weight and would therefore require no eye examination. Of the infants who received an alarm, after further evaluation, 15 of the 50 infants (30%) were considered at low risk for needing treatment and would therefore require fewer eye examinations. Thus, of the original 50 extremely preterm infants, only 22 (44%) were considered at high risk for needing treatment and would be screened according to the routine protocol of eye examinations. The next step is to apply this promising validated screening tool to new and larger populations of preterm infants at risk for developing ROP requiring treatment.

Correspondence: Chatarina Löfqvist, PhD, Section of Pediatric Ophthalmology, The Queen Silvia Children's Hospital, Sahlgrenska Academy at University of Gothenburg, S-416 85 Göteborg, Sweden (chatarina.lofqvist@vgregion.se).

Submitted for Publication: August 26, 2008; final revision received November 7, 2008; accepted November 10, 2008.

Financial Disclosure: Trademark protection has been sought for the name WINROP. The rights to WINROP are owned jointly by Premacure AB and Dr Andersson. Drs Löfqvist and Hellström own shares in Premacure Holding AB, which is the 100% owner of Premacure AB.

Funding/Support: This study was supported by grants 2005-6739 and 14940 from the Swedish Medical Research Council, government grants ALFGB2770, ALFGBG-7211, and ALFGBG 7214, and grants from the Lund University Hospital and Lund Medical Faculty, the Physiographic Society in Lund, the Linnéa and Josef Carlsson Foundation, the Nils W. Svenningsen Foundation for Preterm Infants, the Anders Otto Svärd and Ulrika Eklund Foundation, De blindas Vänner, and the Göteborg Medical Society.

Additional Contributions: Research nurses Ann-Cathrin Berg and Eva Hammarstrand and the staff at the neonatal unit and ophthalmology department in Lund provided assistance and examination of patients, medical laboratory technologists Lisbeth Larsson and Monika Eriksson provided the growth factor analysis, software developer Niclas Nilsson at Activa provided the beta version of WINROP, and Anna-Lena Hård, MD, PhD, provided helpful suggestions during manuscript preparation.