eFigure. Flowchart for Enrollment, Diagnostic Examinations, Imaging, and Image Grading
Quinn GE, Ying G, Daniel E, Hildebrand PL, Ells A, Baumritter A, Kemper AR, Schron EB, Wade K, for the e-ROP Cooperative Group. Validity of a Telemedicine System for the Evaluation of Acute-Phase Retinopathy of Prematurity. JAMA Ophthalmol. 2014;132(10):1178-1184. doi:10.1001/jamaophthalmol.2014.1604
The present strategy to identify infants needing treatment for retinopathy of prematurity (ROP) requires repeated examinations of at-risk infants by physicians. However, less than 10% ultimately require treatment. Retinal imaging by nonphysicians with remote image interpretation by nonphysicians may provide a more efficient strategy.
To evaluate the validity of a telemedicine system to identify infants who have sufficiently severe ROP to require evaluation by an ophthalmologist.
Design, Setting, and Participants
An observational study of premature infants starting at 32 weeks’ postmenstrual age was conducted. This study involved 1257 infants with birth weight less than 1251 g in neonatal intensive care units in 13 North American centers enrolled from May 25, 2011, through October 31, 2013.
Infants underwent regularly scheduled diagnostic examinations by an ophthalmologist and digital imaging by nonphysician staff using a wide-field digital camera. Ophthalmologists documented findings consistent with referral-warranted (RW) ROP (ie, zone I ROP, stage 3 ROP or worse, or plus disease). A standard 6-image set per eye was sent to a central server and graded by 2 trained, masked, nonphysician readers. A reading supervisor adjudicated disagreements.
Main Outcomes and Measures
The validity of grading retinal image sets was based on the sensitivity and specificity for detecting RW-ROP compared with the criterion standard diagnostic examination.
A total of 1257 infants (mean birth weight, 864 g; mean gestational age, 27 weeks) underwent a median of 3 sessions of examinations and imaging. Diagnostic examination identified characteristics of RW-ROP in 18.2% of eyes (19.4% of infants). Remote grading of images of an eye at a single session had sensitivity of 81.9% (95% CI, 77.4-85.6) and specificity of 90.1% (95% CI, 87.9-91.8). When both eyes were considered for the presence of RW-ROP, as would routinely be done in a screening, the sensitivity was 90.0% (95% CI, 85.4-93.5), with specificity of 87.0% (95% CI, 84.0-89.5), negative predictive value of 97.3%, and positive predictive value of 62.5% at the observed RW-ROP rate of 19.4%.
Conclusions and Relevance
When compared with the criterion standard diagnostic examination, these results provide strong support for the validity of remote evaluation by trained nonphysician readers of digital retinal images taken by trained nonphysician imagers from infants at risk for RW-ROP.
Retinopathy of prematurity (ROP) is a leading cause of avoidable blindness in children worldwide1,2 and an increasing problem in underserved areas of the United States and Canada.3,4 In most settings, ophthalmologists travel regularly to neonatal units to perform serial eye examinations of infants at risk; however, less than 10% of infants examined require treatment.5 The interpretation of the ROP findings varies across examiners.6 A potential solution is to develop a telemedicine system using digital retinal imaging to detect sight-threatening disease. The solution must deal with key screening principles. Such evaluations differ fundamentally from clinical examinations by physicians and must provide compelling evidence for a high likelihood of altering the natural history of the disease in a significant proportion of individuals evaluated.7- 9
In recent years, studies have evaluated the validity of retinal imaging to detect moderate to severe ROP.10- 21 However, the sensitivities varied widely (33%11 to 100%10,12,14,18- 21), as did the number of infants included (range, 10-122210,18) and the primary outcome (plus disease10 to suspect treatment requiring18). Images for these studies were largely obtained by an ophthalmologist and were also graded by ophthalmologists. Given the current status of retinal imaging in ROP, the American Academy of Ophthalmology prioritized the need for “understanding …the place of digital wide-angle photography in the evaluation of at-risk infants.”22
Quiz Ref IDElls et al12 introduced the term referral-warranted ROP (RW-ROP) in 2003 for use in telemedicine to describe eyes with ROP that had high-risk characteristics defined as plus disease, ROP in zone I, or stage 3 ROP or greater. Eyes with RW-ROP require careful ophthalmoscopic examination and many require treatment.
This large multicenter, National Eye Institute–funded clinical study evaluated the validity of an ROP telemedicine system to detect eyes with RW-ROP.23 We compared remote evaluations of digital images to the findings of a criterion-standard indirect ophthalmoscopic examination performed by experienced ophthalmologists.
Infants with birth weight (BW) less than 1251 g meeting current ROP screening guidelines in 12 US centers and 1 Canadian center were included in the study. Exclusion criteria were postmenstrual age greater than 39 weeks at first opportunity for imaging unless transferred in for ROP treatment, admission to a neonatal intensive care unit (NICU) with regressing or treated ROP, significant media opacity precluding visualization of the retina, or major ocular or systemic congenital abnormality. The protocol and informed consent processes were approved by the institutional review boards of the participating study centers. Written informed consent was obtained for all participants.
Infants underwent serial ROP imaging in both eyes using the RetCam Shuttle (Clarity Medical Systems), in addition to a standard diagnostic examination by study-certified ophthalmologists experienced in diagnosing ROP. The diagnostic examination results were classified as having clinical findings consistent with RW-ROP: zone I ROP, stage 3 ROP, or plus disease. The imagers were masked to the results of the examination and the physicians were masked to images and subsequent grading. The timing of diagnostic examinations was determined by local clinical center criteria for usual clinical care and imaging sessions began at 32 weeks’ postmenstrual age. To prevent bias in terms of adverse events, the order of imaging and examinations alternated.
Imaging was conducted by 25 nonphysician study-certified personnel including NICU nurses (44%), neonatal nurse practitioners (24%), ophthalmic photographers (8%), an ocular coherence tomography technician (4%), an ophthalmic medical technologist (4%), and individuals with nonclinical backgrounds (16%). All participated in classroom and hands-on instruction in taking retinal images in infants and selecting and uploading images to the Inoveon ROP Data Center server in Oklahoma City, Oklahoma. Imagers were certified after submission of quality prestudy image sets and passing a knowledge assessment test.
Images were obtained for each eye using the video mode with a 130° wide-field imaging system. The imager selected still frames for a standard 6-image set consisting of the pupil and 5 retinal fields, with optic disc central, nasal, temporal, superior, and inferior.
Demographic data and the medical status of the infant before, during, and after each session were collected. Surveillance for ocular and systemic complications and other adverse events was conducted.
The paired diagnostic examinations and imaging sessions continued as clinically indicated until the examining ophthalmologist noted any of the following: mature retinal vessels, immature zone III on 2 occasions at least 7 days apart, ROP regressed or regressing on 2 occasions at least 7 days apart, treatment for severe ROP, or the infant reached 40 weeks’ postmenstrual age with no ROP or only stage 1 or 2 ROP.
Image sets were uploaded as unmodified uncompressed files in .png format from the RetCam Shuttle to the Inoveon ROP Data Center server and used for remote grading by nonphysician trained readers and by expert readers (ophthalmologists with ROP expertise). All readers participated in joint didactic and image grading training sessions, underwent a certification process, and practiced with training image sets. All readers, including 3 trained readers and 3 expert readers, used the same software to access the images, used standardized workstations to grade images, and recorded findings on the same web-based data collection form. Each image set was graded independently by 2 trained readers, with discrepancies adjudicated by a reading supervisor. All readers were masked to the results of diagnostic examinations, previous gradings for either eye of the infant, and demographic data.
All readers determined the quality of each of the 5 retinal images in a set as good, acceptable, poor, or missing. They also determined, by quadrant, whether the posterior pole vessels were normal or sufficiently abnormal to be plus disease and determined zone of vascularization or the zone in which morphologic features consistent with ROP were present. Retinopathy of prematurity was determined by the presence of a demarcation line (stage 1), a ridge (stage 2), extraretinal neovascularization (stage 3), or retinal detachment (stage 4).
Trained readers graded all images from 242 infants who developed RW-ROP based on the diagnostic examination. Approximately 80% of infants were not expected to develop RW-ROP; therefore, a random sample of 613 infants (60.5%) who never developed RW-ROP was selected a priori. All image sets from this selected subsample of infants (n = 242 + 613 = 855) were graded by the trained readers. A random sample of 200 of these 855 infants was also graded by expert readers (eFigure in the Supplement).
The sample size was determined by the need to have half width of the 95% CI of sensitivity within 5%. Assuming sensitivity between 80% and 95%, approximately 250 infants with RW-ROP were needed for the study.
At the same session, we compared the RW-ROP finding (positive, negative, or indeterminate) from evaluation of an image set to findings of the diagnostic examination consistent with RW-ROP (presence, absence, or indeterminate). Sensitivity and specificity of image grading of detecting RW-ROP were calculated by using the results of the diagnostic examination as the criterion standard. Eyes with indeterminate status in the diagnostic examination were excluded from the sensitivity/specificity analysis. When the image set did not provide sufficient information for determining RW-ROP status, the eye was scored as RW-ROP positive in sensitivity/specificity analysis because the primary aim of this study was to determine whether referral for a diagnostic examination was warranted.
We included only 1 session of digital image/diagnostic examinations from each eye in the primary analysis (single-session-per-eye analysis). For sensitivity calculation, the session when the diagnostic examination first identified RW-ROP was used, while a random session was chosen for each eye that did not develop RW-ROP for the specificity calculation.
Sensitivity was calculated as the proportion of RW-ROP–positive image gradings when examination indicated RW-ROP presence, and specificity was calculated as the proportion of RW-ROP–negative image gradings when examination indicated RW-ROP absence. The 95% CIs for sensitivity and specificity were calculated, with intereye correlation adjusted by generalized estimating equations24 using the sandwich robust estimate of variance.25 The prespecified subgroup analyses of sensitivity/specificity were performed by birth weight, gestational age (GA), and image quality.
Secondary analyses of sensitivity/specificity were conducted at the infant level by comparing the presence or absence of RW-ROP on examination vs RW-ROP positive/negative status from image grading at 1 selected session (single-session-per-infant analysis) and any sessions (any-session-per-infant analysis). The sensitivity and specificity for detecting the infants who underwent ROP treatment were also calculated based on the last session before treatment (last-session-per-infant analysis).
For per-infant analysis, the negative predictive value (NPV) and the positive predictive value (PPV) were calculated based on their corresponding sensitivity, specificity, and the observed rate of RW-ROP. All the statistical analyses were performed using SAS version 9.3 (SAS Institute Inc).
A total of 1284 infants with BW less than 1251 g were enrolled from May 25, 2011, through October 31, 2013 (eFigure in the Supplement). Twenty seven (2.1%) were discharged prior to a diagnostic examination. Table 1 provides the characteristics of 1257 study infants who completed at least 1 diagnostic examination. The mean BW was 864 g and mean GA of 27 weeks. Most infants had BW less than or equal to 1000 g including 34.1% of infants with BW of 750 g or less and 36.2% with BW from 751 to 1000 g. About half had GA of less than 27 weeks (51.9%) and few infants had GA of 31 weeks or greater (5.9%). More than half were non-Hispanic white, 29.3% black, and 49.2% female; 63.0% were born at the enrolling clinical center and 29.8% were multiple births, of which 84.3% were twin births.
Among 1257 infants who had eye examinations, the mean (SD) number of examinations per infant was 3.4 (2.1) (median, 3; range, 1-12) and 40% had 4 or more examinations. The median interval between examinations was 9 days (range, 1-54 days), with 11% of examinations within 1 week and 88% within 2 weeks.
At least 1 imaging session was conducted in 1241 infants. The mean (SD) number of imaging sessions per infant was 3.2 (2.0) (median, 3; range, 1-12) and 36% had 4 or more imaging sessions.
Among the 5520 image sets selected for trained reader grading, a total of 27 600 possible individual retinal images were evaluated for availability and image quality. Among these images, 3% were missing, 91% were adequate quality, and 6% were poor quality.
Retinopathy of prematurity was noted on examination in 801 infants (63.7%) (Table 1). Clinical findings consistent with RW-ROP were noted in 1 or both eyes of 244 infants (19.4%) and were bilateral in 87.8% of infants. The presence of RW-ROP could not be determined for only 2.4% of infants. Clinical findings consistent with RW-ROP were detected in 458 eyes (18.2%), most frequently owing to stage 3 or worse alone (48.5%) or in combination with zone I ROP and/or plus disease (33.6%; Table 2). All 3 components (plus disease, zone I ROP, and stage 3 ROP) were rarely present (3.3%).
Among the 5520 pairs of diagnostic examinations and image gradings, their RW-ROP status was in agreement in 78.6% (Table 3). In the 813 pairs (14.7%) with RW-ROP findings on diagnostic examination, image grading by trained readers detected 1 or more of the components of RW-ROP in 77.7% of image sets, while in 19.8%, the image grading did not detect RW-ROP and 2.5% were indeterminate. In the 4648 pairs without RW-ROP on diagnostic examination, trained readers agreed in 3703 image gradings (79.7%), while in 854 pairs (18.4%), trained readers detected findings consistent with RW-ROP. In 91 pairs, RW-ROP status was indeterminate.
When image sets for a single session were graded by the trained readers and compared with the results of diagnostic examinations (1709 image sets in which the first session when RW-ROP findings were diagnosed were used and a random session for infants without RW-ROP), the sensitivity for detection of RW-ROP was 81.9% (95% CI, 77.4-85.6), with a specificity of 90.1% (95% CI, 87.9-91.8) (Table 4).
As shown in Table 4, when both eyes of the infant were considered, as would routinely be done in a clinical setting, the sensitivity for RW-ROP was 90.0% (95% CI, 85.4-93.5), with specificity of 87.0% (95% CI, 84.0-89.5), NPV of 97.3%, and PPV of 62.5% at the RW-ROP prevalence rate of the study (19.4%).
When RW-ROP results from any session of image grading and diagnostic examination were paired for the infant, the sensitivity increased to 97.1% (95% CI, 94.0-98.6). Specificity for this comparison was 75.9% (95% CI, 72.2-79.1), NPV was 99.1%, and PPV was 49.2%.
We also calculated the sensitivity and specificity of this telemedicine system to detect RW-ROP in infants who underwent treatment in 1 or both eyes. When the last session before treatment was analyzed, sensitivity was 98.2% (95% CI, 94.4-99.4), with specificity of 80.2% (95% CI, 77.0-83.0), NPV of 99.6%, and PPV of 44.3% at a 13.8% treatment-requiring ROP rate. Only 3 of 162 infants treated by clinical center ophthalmologists did not have RW-ROP detected on the last image grading before treatment. On diagnostic examination, 1 infant had zone I stage 3 disease in both eyes and another 2 infants had plus disease in both eyes.
Expert and trained readers independently graded a random sample of 1312 image sets from 100 infants with RW-ROP and 100 without RW-ROP. Using all 1312, expert readers had a lower sensitivity of 85.9% (95% CI, 80.8-89.8) compared with 91.4% (95% CI, 86.1-94.8) for trained readers and a lower specificity of 56.5% (95% CI, 51.9-61.0) vs 73.3% (95% CI, 67.6-78.3) for detecting an eye with RW-ROP.
Quiz Ref IDThe timing of the detection of RW-ROP was compared between the image grading and diagnostic examination. In 87.5% of the cases (391 of 447), RW-ROP was detected on image grading before or at the same examination that documented RW-ROP findings. Referral-warrantedROP was detected on image grading an average (SD) of 15 (11) days earlier than the examination in 44.7% of cases (200 of 447). Image grading did not detect RW-ROP at any image grading in 7.2% of cases (32 of 447). In 5.4% of cases (24 of 447), RW-ROP was detected an average (SD) of 15 (13) days after the examination documented RW-ROP. The results were very similar when analyzed by infant (data not shown).
In the single-session-per-eye analysis, sensitivity of RW-ROP detection decreased with increasing BW (Table 5). This pattern was not observed for GA.
The quality of the images submitted was important. When all 5 retinal images were judged to be of good or acceptable image quality by trained readers, the sensitivity was 84.7% compared with 68.0% when 4 or fewer images were of good or acceptable quality. Specificity for this comparison was similar.
The e-ROP Study results provide strong support for the validity of using a telemedicine system consisting of trained nonphysician imagers and readers to detect RW-ROP in infants at risk. When image-set grading of an eye was compared with the results of its paired diagnostic examination, the sensitivity for detection of RW-ROP was 81.9%, with a specificity of 90.1%. When both eyes of an infant were considered, the sensitivity increased to 90.0%, with specificity of 87.0%, NPV of 97.3%, and PPV of 62.5% at the observed RW-ROP rate of 19.4%. Importantly, among infants treated for ROP, the sensitivity of image grading increased to 98.2%.
Quiz Ref IDAmong the strengths of the e-ROP Study is the successful use of nonphysician imagers. The imagers were trained to capture standard image sets for grading by nonphysician trained readers using a standard grading protocol. The diagnostic examinations were performed by ophthalmologists with extensive ROP experience. Furthermore, we elected to have the results of nonphysician trained readers as the primary outcome measure because, as in other retinal disorders,26,27 we confirmed in this study that dedicated and trained readers would perform at least as well as ophthalmologist experts and their availability was likely to be higher and sustained. Standardized imaging and grading protocols improve the generalizability of this ROP telemedicine system.
Quiz Ref IDThis study was limited by the difficulty of comparing image grading results with a criterion standard with known inherent variability.28,29 Among eyes judged at treatment level by the first examiner in the Cryotherapy for ROP Trial, there was disagreement on a second confirming examination in 12% of the cases.6 Other studies have highlighted the variability in diagnosing plus disease and disease zone among ophthalmologist examiners.30- 33 This variability across examiners may well help explain some of the false-positive and false-negative cases documented on image grading in the current study. In addition, the e-ROP Study limited enrollment to infants at high risk (ie, BW < 1251 g) and may not be generalizable to all infants eligible for ROP examinations. Furthermore, the e-ROP Study was not designed to assess the time to reporting results to the clinical center or to provide rapid feedback requesting additional images when image quality was poor. Sensitivity improved substantially with better image quality so efforts to improve image quality are warranted.
One of the most important factors to consider in evaluating telemedicine systems for ROP is how many infants with serious ROP might be missed. In this study with a RW-ROP rate of 19.4%, we found the NPV was high (97.3%). Despite this high NPV, if implementing such a system for clinical care, there will be a small number of infants in whom image grading does not detect RW-ROP and who are at risk for progression to blindness. What safeguards need to be in place for such infants? One might consider a more frequent imaging schedule than used in this study that mimicked current clinical care or also consider adding other algorithms, such as early weight patterns in predicting ROP risk,34,35 to determine those infants at highest risk.
Adopting a telemedicine system approach in ROP is influenced by factors other than validity including the availability of ROP specialists, the number of examinations required per week, and the prevalence of RW-ROP in a NICU. Licensing and liability issues must be dealt with, as well as establishing a consistent and reliable reading center. Procedures to address poor image quality are needed. Parental and NICU staff acceptance of a screening program that replaces a physician diagnostic examination with nonphysician imaging needs further assessment.
The results of this study provide important information about the validity of the e-ROP telemedicine system in managing ROP as we move forward to address the broader use of digital retinal imaging in NICUs in the United States and other regions in the world.
Corresponding Author: Graham E. Quinn, MD, MSCE, Division of Ophthalmology, The Children’s Hospital of Philadelphia, Wood Center, 1st Floor, Philadelphia, PA 19104 (firstname.lastname@example.org).
Submitted for Publication: February 27, 2014; final revision received April 8, 2014; accepted April 9, 2014.
Published Online: June 26, 2014. doi:10.1001/jamaophthalmol.2014.1604.
Author Contributions: Drs Quinn and Ying had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Quinn, Ying, Hildebrand, Ells, Baumritter, Kemper, Wade.
Acquisition, analysis, or interpretation of data: Quinn, Ying, Daniel, Hildebrand, Ells, Baumritter, Schron, Wade.
Drafting of the manuscript: Quinn.
Critical revision of the manuscript for important intellectual content: Ying, Daniel, Hildebrand, Ells, Baumritter, Kemper, Schron, Wade.
Statistical analysis: Quinn, Ying, Kemper, Wade.
Obtained funding: Quinn.
Administrative, technical, or material support: Quinn, Ying, Daniel, Hildebrand, Ells, Baumritter, Schron, Wade.
Study supervision: Quinn, Ying, Daniel, Hildebrand, Baumritter.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Hildebrand reported receiving support from Inoveon Corp and has a patent, Digital Disease Management System, with royalties paid to Inoveon Corp. Dr Ells has received personal fees from the scientific advisory board of Clarity Medical Systems. No other disclosures were reported.
Funding/Support: This study was funded by a cooperative agreement grant (U10 EY017014) from the National Eye Institute of the National Institutes of Health, Department of Health and Human Services. Dr Daniel has received grants from the National Institutes of Health and Dr Hildebrand has received grants from the National Eye Institute; this funding was not related to the current research.
Role of the Sponsor: The study was conducted under a cooperative agreement from the National Eye Institute and, as such, Dr Schron, the National Eye Institute project officer for this study, had input in the collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication, but not the design of the study.
Group Information: The e-ROP Study Group investigators include the following: Office of Study Chair: The Children’s Hospital of Philadelphia: Graham E. Quinn, MD, MSCE (principal investigator [PI]), Kelly Wade, MD, PhD, MSCE, Agnieshka Baumritter, MS, Trang B. Duros, Lisa Erbring; Johns Hopkins University: Michael X. Repka, MD (PI), Jennifer A. Shepard, CRNP, Pamela Donohue, ScD, David Emmert, BA, C. Mark Herring; Boston Children’s Hospital: Deborah VanderVeen, MD, Suzanne Johnston, MD, Carolyn Wu, MD, Jason Mantagos, MD, Danielle Ledoux, MD, Tamar Winter, RN, BSN, IBCLC, Frank Weng, Theresa Mansfield; Nationwide Children’s Hospital and Ohio State University Hospital: Don L. Bremer, MD (PI), Richard Golden, MD, Mary Lou McGregor, MD, Catherine Olson Jordan, MD, David L. Rogers, MD, Rae R. Fellows, MEd, CCRC, Suzanne Brandt, RNC, BSN, Brenda Mann, RNC, BSN; Duke University: David Wallace, MD (PI), Sharon Freedman, MD, Sarah K. Jones, Du Tran-Viet, Rhonda “Michelle” Young; University of Louisville: Charles C. Barr, MD (PI), Rahul Bhola, MD, Craig Douglas, MD, Peggy Fishman, MD, Michelle Bottorff, Brandi Hubbuch, RN, MSN, NNP-BC, Rachel Keith, PhD; University of Minnesota: Erick D. Bothun, MD (PI), Inge DeBecker, MD, Jill Anderson, MD, Ann Marie Holleschau, BA, CCRP, Nichole E. Miller, MA, RN, NNP, Darla N. Nyquist, MA, RN, NNP; University of Oklahoma: R. Michael Siatkowski, MD (PI), Lucas Trigler, MD, Marilyn Escobedo, MD, Karen Corff, MS, ARNP, NNP-BC, Michelle Huynh-Blunt, MS, ARNP, Kelli Satnes, MS, ARNP, NNP-BC; Children’s Hospital of Philadelphia: Monte D. Mills, MD, Will Anninger, MD, Gil Binenbaum, MD, MSCE, Graham E. Quinn, MD, MSCE, Karen A. Karp, BSN, Denise Pearson, COMT; University of Texas Health Science Center at San Antonio: Alice Gong, MD (PI), John Stokes, MD, Clio Armitage Harper, MD, Laurie Weaver, Carmen McHenry, BSN, Kathryn Conner, Rosalind Heemer, Elnora Cokley, RNC, Robin Tragus, MSN, RN, CCRC; University of Utah: Robert Hoffman, MD (PI), David Dries, MD, Katie Jo Farnsworth, Deborah Harrison, MS, Bonnie Carlstrom, Cyrie Ann Frye, CRA, OCT-C; Vanderbilt University: David Morrison, MD (PI), Sean Donahue, MD, Nancy Benegas, MD, Sandy Owings, COA, CCRP, Sandra Phillips, COT, CRI, Scott Ruark; Hospital of the Foothills Medical Center: Anna Ells, MD, FRCS (PI), Patrick Mitchell, MD, April Ingram, Rosie Sorbie, RN. Data Coordinating Center: University of Pennsylvania Perelman School of Medicine: Gui-shuang Ying, PhD (PI), Maureen Maguire, PhD, Mary Brightwell-Arnold, BA, SCP, Max Pistilli, MS, Kathleen McWilliams, CCRP, Sandra Harris, Claressa Whearry; Image Reading Center: University of Pennsylvania Perelman School of Medicine: Ebenezer Daniel, MBBS, MS, MPH (PI), E. Revell Martin, BA, Candace R. Parker Ostroff, Krista Sepielli, Eli Smith. Expert Readers: Antonio Capone, MD, The Vision Research Foundation; G. Baker Hubbard, MD, Emory University School of Medicine; Anna Ells, MD, FRCS, University of Calgary Medical Center. Image Data Management Center: University of Oklahoma Health Sciences Center/Inoveon Corp: P. Lloyd Hildebrand, MD (PI), Kerry Davis, G. Carl Gibson, Regina Hansen. Cost-Effectiveness Component: Alex R. Kemper, MD, MPH, MS (PI), Lisa Prosser, PhD. Data Management and Oversight Committee (DMOC): David C. Musch, PhD, MPH (chair), Stephen P. Christiansen, MD, Ditte J. Hess, CRA, Steven M. Kymes, PhD, SriniVas R. Sadda, MD, Ryan Spaulding, PhD. National Eye Institute: Eleanor B. Schron, PhD, RN.
Additional Contributions: We would like to acknowledge gratefully the contributions of Clare Gilbert, FRCOphth, MD, MSc (London School of Hygiene and Tropical Medicine); Alistair Fielder, FRCOphth (professor emeritus at City University London); and Judith Alexander, BA (former director of the Fundus Photograph Reading Center, Scheie Eye Institute, University of Pennsylvania Perelman School of Medicine), for their contributions to the early development of the e-ROP Study. They did not receive compensation from a funding sponsor for their contributions.