[Skip to Content]
[Skip to Content Landing]
Figure.
Mosaic Photograph Delineating the Individual Color Fundus Photographs Using 5 Fields
Mosaic Photograph Delineating the Individual Color Fundus Photographs Using 5 Fields

Each mosaic photograph was automatically generated from the multiple individual color fundus photographs using i2k Align Retina software (DualAlign) in the quadratic layout in full blend. The 5 fields are outlined in white.

Table 1.  
Time to ROP Diagnosis per ROP Expert Using Individual Color Fundus Photographs vs 1 Mosaic Photograph
Time to ROP Diagnosis per ROP Expert Using Individual Color Fundus Photographs vs 1 Mosaic Photograph
Table 2.  
Accuracy of ROP Diagnosis by 9 ROP Experts Using Individual Color Fundus Photographs vs 1 Mosaic Photograph
Accuracy of ROP Diagnosis by 9 ROP Experts Using Individual Color Fundus Photographs vs 1 Mosaic Photograph
Table 3.  
Unweighted Mean κ Statistics for Intergrader Agreement of ROP Diagnosis Among 9 ROP Experts Using Individual Color Fundus Images vs 1 Mosaic Photograph
Unweighted Mean κ Statistics for Intergrader Agreement of ROP Diagnosis Among 9 ROP Experts Using Individual Color Fundus Images vs 1 Mosaic Photograph
1.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Pediatrics. 1988;81(5):697-706.PubMedGoogle Scholar
2.
Early Treatment For Retinopathy Of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial.  Arch Ophthalmol. 2003;121(12):1684-1694.PubMedGoogle ScholarCrossref
3.
Fierson  WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists.  Screening examination of premature infants for retinopathy of prematurity.  Pediatrics. 2013;131(1):189-195.PubMedGoogle ScholarCrossref
4.
Fierson  WM, Capone  A  Jr; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association of Certified Orthoptists.  Telemedicine for evaluation of retinopathy of prematurity.  Pediatrics. 2015;135(1):e238-e254.PubMedGoogle ScholarCrossref
5.
Richter  GM, Williams  SL, Starren  J, Flynn  JT, Chiang  MF.  Telemedicine for retinopathy of prematurity diagnosis: evaluation and challenges.  Surv Ophthalmol. 2009;54(6):671-685.PubMedGoogle ScholarCrossref
6.
Ells  AL, Holmes  JM, Astle  WF,  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study.  Ophthalmology. 2003;110(11):2113-2117.PubMedGoogle ScholarCrossref
7.
Quinn  GE, Ying  GS, Daniel  E,  et al; e-ROP Cooperative Group.  Validity of a telemedicine system for the evaluation of acute-phase retinopathy of prematurity.  JAMA Ophthalmol. 2014;132(10):1178-1184.PubMedGoogle ScholarCrossref
8.
 Diabetic retinopathy study: report number 6: design, methods, and baseline results; report number 7: a modification of the Airlie House classification of diabetic retinopathy.   Invest Ophthalmol Vis Sci. 1981;21(1, pt 2):1-226.PubMedGoogle Scholar
9.
Patel  M, Kiss  S.  Ultra–wide-field fluorescein angiography in retinal disease.  Curr Opin Ophthalmol. 2014;25(3):213-220.PubMedGoogle ScholarCrossref
10.
Tsui  I, Franco-Cardenas  V, Hubschman  JP, Schwartz  SD.  Pediatric retinal conditions imaged by ultra wide field fluorescein angiography.  Ophthalmic Surg Lasers Imaging Retina. 2013;44(1):59-67.PubMedGoogle ScholarCrossref
11.
Kang  KB, Wessel  MM, Tong  J, D’Amico  DJ, Chan  RV.  Ultra-widefield imaging for the management of pediatric retinal diseases.  J Pediatr Ophthalmol Strabismus. 2013;50(5):282-288.PubMedGoogle ScholarCrossref
12.
Patel  CK, Fung  TH, Muqit  MM,  et al.  Non-contact ultra-widefield imaging of retinopathy of prematurity using the Optos dual wavelength scanning laser ophthalmoscope.  Eye (Lond). 2013;27(5):589-596.PubMedGoogle ScholarCrossref
13.
Chen  J, Ausayakhun  S, Ausayakhun  S,  et al.  Comparison of autophotomontage software programs in eyes with CMV retinitis.  Invest Ophthalmol Vis Sci. 2011;52(13):9339-9344.PubMedGoogle ScholarCrossref
14.
Patel  SN, Klufas  MA, Ryan  MC,  et al.  Color fundus photography versus fluorescein angiography in identification of the macular center and zone in retinopathy of prematurity.  Am J Ophthalmol. 2015;159(5):950-957.e2.PubMedGoogle ScholarCrossref
15.
Ryan  MC, Ostmo  S, Jonas  K,  et al.  Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology.  AMIA Annu Symp Proc. 2014;2014:1902-1910.PubMedGoogle Scholar
16.
Paulson  EK, Harris  JP, Jaffe  TA, Haugan  PA, Nelson  RC.  Acute appendicitis: added diagnostic value of coronal reformations from isotropic voxels at multi-detector row CT.  Radiology. 2005;235(3):879-885.PubMedGoogle ScholarCrossref
17.
Chiang  MF, Jiang  L, Gelman  R, Du  YE, Flynn  JT.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.  Arch Ophthalmol. 2007;125(7):875-880.PubMedGoogle ScholarCrossref
18.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Arch Ophthalmol. 1988;106(4):471-479.PubMedGoogle ScholarCrossref
19.
International Committee for the Classification of Retinopathy of Prematurity.  The international classification of retinopathy of prematurity revisited.  Arch Ophthalmol. 2005;123(7):991-999.PubMedGoogle ScholarCrossref
20.
Rao  R, Jonsson  NJ, Ventura  C,  et al.  Plus disease in retinopathy of prematurity: diagnostic impact of field of view.  Retina. 2012;32(6):1148-1155.PubMedGoogle ScholarCrossref
21.
Hewing  NJ, Kaufman  DR, Chan  RV, Chiang  MF.  Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts.  JAMA Ophthalmol. 2013;131(8):1026-1032.PubMedGoogle ScholarCrossref
22.
Ataer-Cansizoglu  E, Bolon-Canedo  V, Campbell  JP,  et al; i-ROP Research Consortium.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: performance of the “i-ROP” system and image features associated with expert diagnosis.  Transl Vis Sci Technol. 2015;4(6):5.PubMedGoogle ScholarCrossref
23.
Myung  JS, Paul Chan  RV, Espiritu  MJ,  et al.  Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: implications for training.  J AAPOS. 2011;15(6):573-578.PubMedGoogle ScholarCrossref
24.
Paul Chan  RV, Williams  SL, Yonekawa  Y, Weissgold  DJ, Lee  TC, Chiang  MF.  Accuracy of retinopathy of prematurity diagnosis by retinal fellows.  Retina. 2010;30(6):958-965.PubMedGoogle ScholarCrossref
25.
Chiang  MF, Thyparampil  PJ, Rabinowitz  D.  Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity.  Arch Ophthalmol. 2010;128(9):1153-1159.PubMedGoogle ScholarCrossref
26.
Rivero  ME, Bartsch  DU, Otto  T, Freeman  WR.  Automated scanning laser ophthalmoscope image montages of retinal diseases.  Ophthalmology. 1999;106(12):2296-2300.PubMedGoogle ScholarCrossref
27.
Silva  PS, Cavallerano  JD, Sun  JK, Noble  J, Aiello  LM, Aiello  LP.  Nonmydriatic ultrawide field retinal imaging compared with dilated standard 7-field 35-mm photography and retinal specialist examination for evaluation of diabetic retinopathy.  Am J Ophthalmol. 2012;154(3):549-559.e2.PubMedGoogle ScholarCrossref
28.
Soliman  AZ, Silva  PS, Aiello  LP, Sun  JK.  Ultra–wide field retinal imaging in detection, classification, and management of diabetic retinopathy.  Semin Ophthalmol. 2012;27(5-6):221-227.PubMedGoogle ScholarCrossref
29.
Wessel  MM, Aaker  GD, Parlitsis  G, Cho  M, D’Amico  DJ, Kiss  S.  Ultra–wide-field angiography improves the detection and classification of diabetic retinopathy.  Retina. 2012;32(4):785-791.PubMedGoogle ScholarCrossref
30.
Li  HK, Esquivel  A, Hubbard  LD, Florez-Arango  JF, Danis  RP, Krupinski  EA.  Mosaics versus early treatment diabetic retinopathy seven standard fields for evaluation of diabetic retinopathy severity.  Retina. 2011;31(8):1553-1563.PubMedGoogle ScholarCrossref
31.
Chiang  MF, Wang  L, Busuioc  M,  et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality.  Arch Ophthalmol. 2007;125(11):1531-1538.PubMedGoogle ScholarCrossref
32.
Chiang  MF, Melia  M, Buffenn  AN,  et al.  Detection of clinically significant retinopathy of prematurity using wide-angle digital retinal photography: a report by the American Academy of Ophthalmology.  Ophthalmology. 2012;119(6):1272-1280.PubMedGoogle ScholarCrossref
Original Investigation
Journal Club
November 2016

Influence of Computer-Generated Mosaic Photographs on Retinopathy of Prematurity Diagnosis and Management

Journal Club PowerPoint Slide Download
Author Affiliations
  • 1Department of Ophthalmology, Weill Cornell Medical College, New York, New York
  • 2Stein Eye Institute, University of California, Los Angeles
  • 3University at Buffalo, State University of New York, Buffalo
  • 4Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago
  • 5Casey Eye Institute, Oregon Health & Science University, Portland
  • 6Bascom Palmer Eye Institute, University of Miami, Miami, Florida
  • 7Associated Retinal Consultants, Oakland University, Royal Oak, Michigan
  • 8Asociación para Evitar la Ceguera en México, Mexico City, Mexico
  • 9Long Island Vitreoretinal Consultants, Long Island, New York
  • 10Department of Ophthalmology, Chang Gung Memorial Hospital, Taoyuan, Taiwan
  • 11Department of Ophthalmology, Chang Gung University, College of Medicine, Taoyuan, Taiwan
  • 12Center for Global Health, University of Illinois at Chicago, Chicago
JAMA Ophthalmol. 2016;134(11):1283-1289. doi:10.1001/jamaophthalmol.2016.3625
Key Points

Question  How do computer-generated mosaic photographs affect the diagnosis and management of retinopathy of prematurity (ROP)?

Findings  In a cohort study of 9 ROP experts examining 40 sets of images, use of mosaic photographs compared with multiple individual photographs was associated with improved diagnosis of stage 2 disease or worse, plus disease, and treatment-requiring ROP. The ROP experts also improved their intergrader agreement for the diagnosis of plus disease or not, stage 3 disease or worse or not, and type 2 ROP or not.

Meaning  Mosaic photography may improve the accuracy and reliability of image-based diagnosis of clinically significant ROP.

Abstract

Importance  Telemedicine is becoming an increasingly important component of clinical care for retinopathy of prematurity (ROP), but little information exists regarding the role of mosaic photography for ROP telemedicine diagnosis.

Objective  To examine the potential effect of computer-generated mosaic photographs on the diagnosis and management of ROP.

Design, Setting, and Participants  In this prospective cohort study performed from July 12, 2011, through September 21, 2015, images were acquired from ROP screening at 8 academic institutions, and ROP experts interpreted 40 sets (20 sets with individual fundus photographs with ≥3 fields and 20 computer-generated mosaic photographs) of wide-angle retinal images from infants with ROP. All experts independently reviewed the 40 sets and provided a diagnosis and management plan for each set presented.

Main Outcomes and Measures  The primary outcome measure was the sensitivity and specificity of the ROP diagnosis by experts that was calculated using a consensus reference standard diagnosis, determined from the diagnosis of fundus photographs by 3 experienced readers in combination with the clinical diagnosis based on ophthalmoscopic examination. Mean unweighted κ statistics were used to analyze the mean intergrader agreement among experts for diagnosis of zone, stage, plus disease, and category.

Results  Nine ROP experts (4 women and 5 men) who have been practicing ophthalmology for a mean of 10.8 years (range, 3-24 years) consented to participate. Diagnosis by the mosaic photographs compared with diagnosis by multiple individual photographs resulted in improvements in sensitivity for diagnosis of stage 2 disease or worse (95.9% vs 88.9%; difference, 7.0; 95% CI, 3.5 to 10.5; P = .02), plus disease (85.7% vs 63.5%; difference, 22.2; 95% CI, 7.6 to 36.9; P = .02), and treatment-requiring ROP (84.4% vs 68.5%; difference, 15.9; 95% CI, 0.8 to 31.7; P = .047). With use of the κ statistic, mosaic photographs, compared with multiple individual photographs, resulted in improvements in intergrader agreement for diagnosis of plus disease or not (0.54 vs 0.40; mean κ difference, 0.14; 95% CI, 0.07 to 0.21; P = .004), stage 3 disease or worse or not (0.60 vs 0.52; mean κ difference, 0.06; 95% CI, −0.06 to 0.18; P = .04), and type 2 ROP or not (0.58 vs 0.51; mean κ difference, 0.07; 95% CI, 0.03 to 0.11; P = .04). After viewing the mosaic photographs, experts altered their choice of management in 42 of 180 responses (23.3%; 95% CI, 17.1%-29.5%).

Conclusions and Relevance  Compared with multiple individual photographs, computer-generated mosaic photographs were associated with improved accuracy of image-based diagnosis for certain categories (eg, plus disease, stage 2 disease or worse, and treatment-requiring ROP) of ROP by experts. It is unclear, however, whether these findings are generalizable, and the results of this study may not be relevant to mosaic grading of other retinal vascular conditions.

Introduction

Retinopathy of prematurity (ROP) is a vasoproliferative disease that affects the retinas of premature infants. Although disease management standards have been established by evidence-based trials, such as Cryotherapy for Retinopathy of Prematurity (CRYO-ROP)1 and Early Treatment for Retinopathy of Prematurity (ET-ROP),2 the availability of ophthalmologists to provide examinations by binocular indirect ophthalmoscopy remains an important barrier to ensuring appropriate ROP care. There has been a growing movement to elucidate the role and usefulness of digital imaging in ROP diagnosis and management, and retinal evaluation through telemedicine interpretation of digital images is becoming an increasingly important component of clinical care for ROP.3-7

Retinal imaging used to evaluate ROP was initially described more than 50 years ago and was limited to a 30° field of view using a retinal camera (NM-200D; Nidek Inc). Imaging beyond the 30° field of view was popularized by the Early Treatment Diabetic Retinopathy Study Research Group, and its method of overlapping 7 standard stereoscopic fields in aggregate provided a 75° field of view.8 Currently, contact wide-field imaging systems, such as the RetCam (Clarity Medical Systems), can provide larger fields of view and have been adopted by the pediatric retina community to evaluate the eyes of infants at risk for ROP in the neonatal intensive care unit. Recently, nonmydriatric ultra–wide-field cameras with a 200° field of view have been used to evaluate pediatric retinal diseases,9-11 including ROP.12 Although the contact systems used for ROP screening are not yet able to acquire images with this 200° field of view, mosaic photographs of multiple individual images captured by the RetCam or a similar device may provide a larger field of view for evaluation compared with individual images captured by these camera systems.

To our knowledge, no study has assessed the role of computer-generated mosaic photography for ROP management. The purpose of this study was to determine the potential influence of mosaic photographs on the diagnosis and management of ROP.

Methods
Study Participants

In this prospective cohort study performed from July 12, 2011, through September 21, 2015, images were acquired from ROP screening at 8 academic institutions, and ROP experts interpreted wide-angle retinal images from infants with ROP. All eligible participants were defined as board-certified practicing ophthalmologists who met at least 1 of the following criteria: having been a principal investigator or certified investigator for the CRYO-ROP or ET-ROP study, having published at least 2 peer-reviewed ROP articles, and/or being a fellowship-trained pediatric ophthalmologist or retina specialist who has regularly performed ROP care at their institution for at least 3 years. These participants are further referred to as experts in this study. Approval from the institutional review boards of University of Illinois at Chicago and Weill Cornell Medical College was obtained to perform this study. Written informed consent was obtained from all study participants before participation, and waiver of consent was obtained for the use of deidentified retinal images.

Image Acquisition and Generation of Mosaic Images

Wide-angle fundus photographs of the posterior retina were captured from infants using the RetCam 3 at 8 participating academic centers (Oregon Health & Science University, Weill Cornell Medical College, Bascom Palmer Eye Institute of the University of Miami, Children's Hospital Los Angeles, William Beaumont Hospital, Columbia University Medical Center, Cedars Sinai Medical Center, and Asociación para Evitar la Ceguera en México). The mosaic photographs were created from multiple individual photographs using i2k Align Retina software, version 2.1.6 (DualAlign), which has previously been reported to create high-quality mosaic photographs (Figure).13

Study Design

Study experts were directed to a secure website developed by the authors. Demographic data were collected from each expert, including fellowship training (pediatric ophthalmology, medical retina, or surgical retina), years since completion of fellowship that the expert had been practicing ophthalmology, and comfort with reading fundus photographs and mosaics of fundus photographs in ROP (not comfortable, somewhat comfortable, or comfortable).

After the initial survey, fundus photographs were displayed as sets of 5 individual retinal images of each eye (superior, temporal, posterior, nasal, and inferior). For each image set, demographic information from the time of imaging was provided (birth weight, gestational age, and postmenstrual age). Sets of images with multiple individual photographs of 20 eyes were displayed in sequential order, and then the same 20 eyes were presented but with mosaic photographs created from the individual retinal images (eFigure in the Supplement). Within each image set, experts were asked to choose the zone (I, II, II-posterior, or III), stage (1, 2, 3, 4, or 5), plus (no, pre-plus, or plus), category (mild, type 2 ROP, pre-plus, or treatment-requiring ROP), management (observation, laser only, anti–vascular endothelial growth factor [VEGF] only, laser with anti-VEGF, or surgery), presence of aggressive posterior ROP (yes or no), image quality of the retinal images provided (adequate, somewhat adequate, or not adequate), and confidence (confident, somewhat confident, or not confident) in determining the clinical diagnosis based on retinal images provided. For each image set, experts were timed on how long it took to answer all questions.

Statistical Analysis

All data were analyzed using STATA/SE statistical software, version 12.0 (StataCorp). A Wilcoxon signed rank test was performed to determine whether there were differences in image quality or confidence in diagnosis using multiple individual photographs vs mosaic photographs.

For each image set, a reference standard diagnosis (RSD) was established as previously described.14,15 Briefly, the RSD was determined by combining the clinical diagnosis as determined by indirect ophthalmoscopy with the image-based diagnosis from multiple experienced readers.15 With use of the RSD, the performance of individual experts was evaluated for each modality (multiple individual photographs and mosaic photographs) using the receiver operating characteristic curve method. These results were then averaged to determine the sensitivity and specificity of each modality for detecting stage 1 disease or worse, stage 2 disease or worse, stage 3 disease or worse, zone I disease, disease in zone I or zone II, pre-plus disease or worse, plus disease or worse, mild ROP or worse, type 2 ROP or worse, and treatment-requiring ROP or not. The nonparametric sign test was performed to determine whether the mean difference in sensitivity and specificity between paired multiple individual photographs and mosaic photographs was significantly different from zero.16

To evaluate intergrader reliability, each International Classification of Retinopathy of Prematurity (ICROP) criterion was dichotomized, forming a 2-level categorization: stage 3 disease or worse or not, zone I disease or not, plus disease or not, type 2 ROP or worse or not, and treatment-requiring ROP or not. The unweighted κ statistic was then calculated to measure chance-adjusted agreement for each head-to-head pairing of readers. These results were averaged to determine the mean unweighted κ for each reader in each category. A widely accepted scale was used to interpret the results: 0 to 0.20 indicated slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, almost perfect agreement.17

Results
Study Experts

Nine ROP experts (4 women and 5 men) (based on the study definition) consented to participate. All the experts were retina specialists who reported that they were comfortable with reading fundus and mosaic photographs of ROP. The experts have been practicing ophthalmology for a mean of 10.8 years (range, 3-24 years). Each expert evaluated 40 image sets (20 sets of multiple individual photographs and 20 mosaic photographs) from 20 eyes, totaling 360 readings. Table 1 gives the time to diagnosis for each expert using multiple individual photographs and mosaic photographs. Three of 9 were faster using mosaic photographs (mean differences, 0.41; 95% CI, 0-0.80; 0.15; 95% CI, 0-0.30; and 0.17; 95% CI, 0.02-0.32). One of 9 was faster using multiple individual photographs (mean difference, −0.92; 95% CI, −1.79 to −0.05).

Accuracy of ROP Diagnosis by Experts

Table 2 gives the mean sensitivity and specificity for the diagnosis of stage, zone, plus disease, and category for the study experts. Compared with the diagnosis based on multiple individual photographs, ROP diagnosis based on mosaic photographs resulted in mean improvements in the sensitivity for diagnosing stage 2 disease or worse (7%; 95% CI, 3.5%-10.5%; P = .02), plus disease or not (22.2%; 95% CI, 7.6%-36.9%; P = .02), and treatment-requiring ROP (15.9%; 95% CI, 0.8%-31.7%; P = .05).

Intergrader Agreement of ROP Diagnosis

Table 3 lists the intergrader agreement using an unweighted κ value for each expert compared with all other experts. There were mean improvements in intergrader agreement using mosaic photographs for plus disease or not (0.14; 95% CI, 0.07-0.21; P = .004), stage 3 disease or worse or not (0.08; 95% CI, 0.01-0.14; P = .04), and type 2 ROP or worse or not (0.07; 95% CI, 0.03-0.11; P = .04).

Confidence in Diagnosis and Changes in Management

For the 180 multiple individual photograph readings, confidence in diagnosis was scored as confident in 102 responses (56.7%; 95% CI, 50.4%-64.7%). For the 180 mosaic photograph readings, confidence in diagnosis was scored as confident in 91 responses (50.6%; 95% CI, 44.1%-58.0%). Overall, there was no difference in experts’ confidence in diagnosis between the multiple individual photographs and mosaic photographs (mean difference, 0.05; 95% CI, −0.01 to 0.09; P = .27).

For the 180 multiple individual photograph readings, image quality was scored as adequate in 94 responses (52.2%; 95% CI, 45.0%-59.5%). For the 180 mosaic photograph readings, image quality was scored as adequate in 93 responses (51.7%; 95% CI, 44.4%-59.0%). Overall, there was no difference in image quality scoring between the multiple individual photographs and mosaic photographs (mean difference, 0.01; 95% CI, −0.02 to 0.03; P = .78).

After viewing mosaic photographs, experts altered their choice of management in 42 of 180 responses (23.3%; 95% CI, 17.1%-29.5%). A total of 12 of 42 responses (28.6%; 95% CI, 14.9%-43.2%) changed from observation to laser treatment. Ten of 42 responses (23.8%; 95% CI, 10.9%-36.7%) changed from observation to anti-VEGF therapy. Five of 42 responses (11.9%; 95% CI, 2.1%-21.7%) changed from laser treatment or anti-VEGF therapy to observation.

After experts viewed mosaic photographs for each case, 169 of 180 responses (93.9%; 95% CI, 91.0%-97.4%) indicated that mosaic photographs provided additional clinically useful information for the management of the patient.

Discussion

This study examined the potential influence of computer-generated mosaic photographs on the diagnosis and management of ROP. We found that use of a single mosaic photograph, compared with multiple individual photographs, may improve diagnosis of clinically significant ROP. In addition, mosaic photographs, compared with multiple individual photographs, may improve intergrader agreement for diagnosis of clinically significant ROP. Mosaic photographs of individual fundus images may have the greatest effect on the diagnosis of plus disease with improvements in the sensitivity of diagnosing plus disease and in intergrader agreement for the diagnosis of plus disease (Table 2 and Table 3).

On the basis of ICROP, plus disease is defined as posterior venous dilation and arteriolar tortuosity that meets or exceeds the degree of abnormality represented in a reference photograph.18,19 Given that this definition focuses on abnormalities within posterior pole vessels, the potential influence of the mosaic photograph on plus disease diagnosis suggests that experts may be incorporating information provided by evaluation of the peripheral retina that is beyond the ICROP definition of plus disease. Indeed, previous work has documented the diagnostic effect of field of view and has demonstrated that interexpert agreement in plus disease diagnosis is higher using wide-angle images compared with narrow-angle images.20 Furthermore, in a qualitative study that examined the diagnostic reasoning process of experts for plus disease, Hewing et al21 noted that some ROP experts specifically cited peripheral retinal vascular features as being useful for plus disease diagnosis. In addition, in a quantitative study of plus disease, Ataer-Cansizoglu et al22 determined that computer-based image analysis systems were most accurate with larger images that incorporated the peripheral retina. Collectively, these findings suggest that peripheral vessels may contain information that clinicians use diagnostically even though plus disease has historically been defined based on the posterior pole vessels. Although mosaic photography does not always present an increased field of view compared with the aggregate of multiple individual images, the potential influence of a mosaic photograph on the diagnosis of plus disease may be attributable to improved visualization of vascular changes by providing a continuous view.

Mosaic photography also affected the diagnosis of stage of ROP with improvements in the sensitivity for diagnosing stage 2 or worse and in the intergrader agreement for the diagnosis of stage 3 or worse or not. Similar to its influence on plus disease, the potential influence of mosaic photographs on the diagnosis of stage may be attributable to improved visualization of the peripheral retina and the ridge by providing a continuous view of the ridge from the posterior pole. This continuous view may allow examiners to appreciate other vascular features in relation to the location of the ridge and thus may facilitate visualization of the central vasculature as it approaches the ridge in the periphery.

The collective improvements in diagnosis of stage and plus disease are likely responsible for the improvements in the diagnosis of treatment-requiring ROP by mosaic photographs. These findings may be important for ROP training because previous studies23,24 have documented inconsistencies in diagnosing clinically significant disease among ophthalmologists in training. Notably, a difference in the sensitivity of zone I diagnosis with the mosaic photographs was not identified (mean difference, −3.9%; 95% CI, −23.4% to 15.6%; P = .34). Zone I of the retina is defined as a circle, the radius of which extends from the optic disc center to twice the distance from the optic disc center to the macular center. On the basis of this definition, it is possible that mosaic photographs could provide a better contiguous view in diagnosing zone disease; however, the minimal effect may suggest that experts are relying on additional information besides the location of the macular center.25

The imaging technology available for pediatric retinal disorders has been predominately through contact imaging systems, such as RetCam, which can provide a 130° field of view. Recently, there has been interest in incorporating noncontact ultra–wide-field imaging systems (Panoramic200A, Optos), which provide up to a 200° field of view.12 However, these imaging systems may be limited for ROP management because of difficulties with integration into the neonatal intensive care unit workflow.

Creating mosaic images through computer-generated mosaic photography, such as those used in this study, allows high-resolution, wide-field viewing of the fundus that can be used to diagnose and document various diseases of the retina and classify changes over time.26 Furthermore, a single mosaic image has the potential advantage of easier image modifications in the form of contrast, color, or tone enhancements to improve visualization of the retina. Currently, ultra–wide-field imaging techniques and mosaic images of individual fundus photographs have a common role in the diagnosis and management of other conditions, including diabetic retinopathy.27-29 Previous studies27,30 have noted that mosaic images and ultra–wide-field images are generally comparable with standard 7-field photographs for classifying the severity of diabetic retinopathy.

The modern technology of fundus photography, coupled with the ability to send digital images electronically to remote locations, has facilitated the development of telemedicine systems for ROP surveillance.4 Current telemedicine systems for ROP have acquired images using wide-angle cameras and then presented multifield individual overlapping images on a web-based platform.5,7,31,32 Incorporating automatically generated mosaic images into telemedicine platforms may be useful for clinicians. Indeed, when experts were asked about the clinical utility of the mosaic images, 169 of 180 responses (93.9%; 95% CI, 91.0%-97.4%) indicated that mosaic images provided additional clinically useful information for the treatment of the patient. Furthermore, mosaic photography also offers the practical advantages of more economical file storage and faster transmission. The mosaic images used had a mean file size of 2.4 MB, which was 37% the size of the composite individual fundus photographs. The reduced burden from smaller image files could be significant for telemedicine systems, particularly in developing and middle-income countries where reliable high-speed internet access may not be readily available.

This study has potential limitations. First, peripheral artifacts in individual images may result in mosaic photographs with marred junctions and chromic aberrations. Sharp edges at junctions can also be distracting, potentially interfering with viewing some abnormalities.30 However, a strength of this study is that the individual images were acquired from multiple institutions using different image capturers, and no difference was found in image quality scoring between the multiple individual photographs and mosaic photographs (mean difference, 0.01; 95% CI, −0.02 to 0.03; P = .78). Second, there were inconsistent findings among the experts regarding the time to diagnosis using mosaic photographs compared with multiple individual photographs. Study participants were not informed that they would be timed while completing the study; therefore, some may have completed the study during several sessions. To account for these differences, the study was designed such that each case was a separate webpage and the time to diagnosis was determined as the differences between the first and last click on each webpage. Third, there may be factors other than the mosaic images themselves that contributed to the differences identified. For example, the set of mosaic photographs were presented after the set of multiple individual images such that diagnostic accuracy could have improved merely because experts may have remembered their diagnosis using multiple individual images when assessing the mosaic photographs. Furthermore, it is possible that experts may be biased to be influenced by the mosaic images. However, we chose this study design to simulate a common real-world telemedicine scenario in which experts would likely review individual images before viewing a mosaic photograph. With this study design, after reviewing the mosaic photograph, the experts altered their choice of management in 42 of 180 responses (23.3%; 95% CI, 17.1%-29.5%) and changed their category of diagnosis (mild, type 2, pre-plus, or treatment-requiring ROP) in 52 of 180 responses (28.9%; 95% CI, 22.3%-35.5%). Fourth, in 4 of 20 cases, the mosaic software did not incorporate all available images when creating the mosaic photograph because of image duplication or image quality. Therefore, only individual fundus images used to create the mosaic photograph were presented in the study. Fifth, the RSD was determined using individual fundus images and not mosaic photographs. However, we integrated the clinical diagnosis determined from indirect ophthalmoscopy into the consensus RSD to account for findings noted on clinical examination. Sixth, although a web-based platform was created for the study to standardize the way images were presented to the ROP experts, it is possible that each ROP expert approached the image readings differently. Seventh, the generalizability of these findings is unknown, and the results of this study may not necessarily be relevant to other retinal vascular conditions. The differences identified may also be attributed to bias by the ROP experts for the mosaic itself. Furthermore, the generalizability of the findings from these 9 ophthalmologists to a broader range of ophthalmologists is unknown.

Conclusions

This study contributes to the current body of ROP knowledge by revealing that compared with multiple individual fundus photographs, computer-generated mosaic photographs may improve the accuracy of image-based diagnosis for certain categories (eg, plus disease, stage 2 or worse, and treatment-requiring ROP) of ROP by ROP experts.

Back to top
Article Information

Corresponding Author: R. V. Paul Chan, MD, Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, 1855 W Taylor St, Chicago, IL 60612 (rvpchan@uic.edu).

Accepted for Publication: August 14, 2016.

Published Online: September 29, 2016. doi:10.1001/jamaophthalmol.2016.3625

Author Contributions: Dr Chan had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Patel, Klufas, Chiang, Chan.

Acquisition, analysis, or interpretation of data: Patel, Klufas, Douglas, Jonas, Ostmo, Berrocal, Capone, Martinez-Castellanos, Chau, Drenser, Ferrone, Orlin, Tsui, Wu, Gupta, Chan.

Drafting of the manuscript: Patel, Douglas, Chiang, Klufas, Chan.

Critical revision of the manuscript for important intellectual content: Patel, Klufas, Jonas, Berrocal, Martinez-Castellanos, Chau, Drenser, Ferrone, Orlin, Tsui, Wu, Gupta, Chiang, Chan.

Statistical analysis: Patel, Klufas, Gupta, Chan.

Administrative, technical, or material support: Patel, Jonas, Ostmo, Berrocal, Drenser, Orlin, Wu, Chiang, Chan.

Study supervision: Patel, Klufas, Jonas, Ostmo, Chiang, Chan.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Chau reported having a patent pending from work involved from grant K12 EY021475 from the National Eye Institute regarding a retinal imaging system that was not used in this work. Dr Chiang reported being an unpaid member of the Scientific Advisory Board for Clarity Medical Systems. Dr Chan reported being a member of the Scientific Advisory Board for Visunex Medical Systems. No other disclosures were reported.

Funding/Support: This study was supported by grants R01 EY019474 (Mss Jonas and Ostmo and Drs Chiang and Chan), P30EY010572 (Ms Ostmo and Dr Chiang), and NEI K12 EY021475 (Dr Chau) from the National Institutes of Health; The iNsight Foundation (Ms Jonas and Dr Chan); and unrestricted departmental funding from Research to Prevent Blindness Inc (Drs Patel, Chiang, and Chan and Mss Jonas and Ostmo).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

Group Information: Members of the i-ROP Research Consortium: Oregon Health & Science University, Portland: Michael F. Chiang, MD, Susan Ostmo, MS, Kemal Sonmez, PhD, and J. Peter Campbell, MD, MPH. University of Illinois at Chicago: R. V. Paul Chan, MD, and Karyn Jonas, RN. Columbia University, New York, New York: Jason Horowitz, MD, Osode Coki, RN, Cheryl-Ann Eccles, RN, and Leora Sarna, RN. Bascom Palmer Eye Institute, Miami, Florida: Audina Berrocal, MD, and Catherin Negron, BA. William Beaumont Hospital, Royal Oak, Michigan: Kimberly Denser, MD, Kristi Cumming, RN, Tammy Osentoski, RN, and Tammy Check, RN. Children’s Hospital Los Angeles, California: Thomas Lee, MD, Evan Kruger, BA, and Kathryn McGovern, MPH. Cedars Sinai Hospital, Los Angeles, California: Charles Simmons, MD, Raghu Murthy, MD, and Sharon Galvis, NNP. Los Angeles Biomedical Research Institute, Los Angeles, California: Jerome Rotter, MD, Ida Chen, PhD, Xiaohui Li, MD, and Kaye Roll, RN. Massachusetts General Hospital, Boston: Jayashree Kalpathy-Cramer, PhD. Northeastern University, Boston, Massachusetts: Deniz Erdogmus, PhD. Asociación para Evitar la Ceguera en México, Mexico City: Maria Ana Martinez-Castellanos, MD, Samantha Salinas-Longoria, MD, Rafael Romero, MD, and Andrea Arriola, MD.

References
1.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Pediatrics. 1988;81(5):697-706.PubMedGoogle Scholar
2.
Early Treatment For Retinopathy Of Prematurity Cooperative Group.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial.  Arch Ophthalmol. 2003;121(12):1684-1694.PubMedGoogle ScholarCrossref
3.
Fierson  WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists.  Screening examination of premature infants for retinopathy of prematurity.  Pediatrics. 2013;131(1):189-195.PubMedGoogle ScholarCrossref
4.
Fierson  WM, Capone  A  Jr; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association of Certified Orthoptists.  Telemedicine for evaluation of retinopathy of prematurity.  Pediatrics. 2015;135(1):e238-e254.PubMedGoogle ScholarCrossref
5.
Richter  GM, Williams  SL, Starren  J, Flynn  JT, Chiang  MF.  Telemedicine for retinopathy of prematurity diagnosis: evaluation and challenges.  Surv Ophthalmol. 2009;54(6):671-685.PubMedGoogle ScholarCrossref
6.
Ells  AL, Holmes  JM, Astle  WF,  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study.  Ophthalmology. 2003;110(11):2113-2117.PubMedGoogle ScholarCrossref
7.
Quinn  GE, Ying  GS, Daniel  E,  et al; e-ROP Cooperative Group.  Validity of a telemedicine system for the evaluation of acute-phase retinopathy of prematurity.  JAMA Ophthalmol. 2014;132(10):1178-1184.PubMedGoogle ScholarCrossref
8.
 Diabetic retinopathy study: report number 6: design, methods, and baseline results; report number 7: a modification of the Airlie House classification of diabetic retinopathy.   Invest Ophthalmol Vis Sci. 1981;21(1, pt 2):1-226.PubMedGoogle Scholar
9.
Patel  M, Kiss  S.  Ultra–wide-field fluorescein angiography in retinal disease.  Curr Opin Ophthalmol. 2014;25(3):213-220.PubMedGoogle ScholarCrossref
10.
Tsui  I, Franco-Cardenas  V, Hubschman  JP, Schwartz  SD.  Pediatric retinal conditions imaged by ultra wide field fluorescein angiography.  Ophthalmic Surg Lasers Imaging Retina. 2013;44(1):59-67.PubMedGoogle ScholarCrossref
11.
Kang  KB, Wessel  MM, Tong  J, D’Amico  DJ, Chan  RV.  Ultra-widefield imaging for the management of pediatric retinal diseases.  J Pediatr Ophthalmol Strabismus. 2013;50(5):282-288.PubMedGoogle ScholarCrossref
12.
Patel  CK, Fung  TH, Muqit  MM,  et al.  Non-contact ultra-widefield imaging of retinopathy of prematurity using the Optos dual wavelength scanning laser ophthalmoscope.  Eye (Lond). 2013;27(5):589-596.PubMedGoogle ScholarCrossref
13.
Chen  J, Ausayakhun  S, Ausayakhun  S,  et al.  Comparison of autophotomontage software programs in eyes with CMV retinitis.  Invest Ophthalmol Vis Sci. 2011;52(13):9339-9344.PubMedGoogle ScholarCrossref
14.
Patel  SN, Klufas  MA, Ryan  MC,  et al.  Color fundus photography versus fluorescein angiography in identification of the macular center and zone in retinopathy of prematurity.  Am J Ophthalmol. 2015;159(5):950-957.e2.PubMedGoogle ScholarCrossref
15.
Ryan  MC, Ostmo  S, Jonas  K,  et al.  Development and evaluation of reference standards for image-based telemedicine diagnosis and clinical research studies in ophthalmology.  AMIA Annu Symp Proc. 2014;2014:1902-1910.PubMedGoogle Scholar
16.
Paulson  EK, Harris  JP, Jaffe  TA, Haugan  PA, Nelson  RC.  Acute appendicitis: added diagnostic value of coronal reformations from isotropic voxels at multi-detector row CT.  Radiology. 2005;235(3):879-885.PubMedGoogle ScholarCrossref
17.
Chiang  MF, Jiang  L, Gelman  R, Du  YE, Flynn  JT.  Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.  Arch Ophthalmol. 2007;125(7):875-880.PubMedGoogle ScholarCrossref
18.
Cryotherapy for Retinopathy of Prematurity Cooperative Group.  Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results.  Arch Ophthalmol. 1988;106(4):471-479.PubMedGoogle ScholarCrossref
19.
International Committee for the Classification of Retinopathy of Prematurity.  The international classification of retinopathy of prematurity revisited.  Arch Ophthalmol. 2005;123(7):991-999.PubMedGoogle ScholarCrossref
20.
Rao  R, Jonsson  NJ, Ventura  C,  et al.  Plus disease in retinopathy of prematurity: diagnostic impact of field of view.  Retina. 2012;32(6):1148-1155.PubMedGoogle ScholarCrossref
21.
Hewing  NJ, Kaufman  DR, Chan  RV, Chiang  MF.  Plus disease in retinopathy of prematurity: qualitative analysis of diagnostic process by experts.  JAMA Ophthalmol. 2013;131(8):1026-1032.PubMedGoogle ScholarCrossref
22.
Ataer-Cansizoglu  E, Bolon-Canedo  V, Campbell  JP,  et al; i-ROP Research Consortium.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity: performance of the “i-ROP” system and image features associated with expert diagnosis.  Transl Vis Sci Technol. 2015;4(6):5.PubMedGoogle ScholarCrossref
23.
Myung  JS, Paul Chan  RV, Espiritu  MJ,  et al.  Accuracy of retinopathy of prematurity image-based diagnosis by pediatric ophthalmology fellows: implications for training.  J AAPOS. 2011;15(6):573-578.PubMedGoogle ScholarCrossref
24.
Paul Chan  RV, Williams  SL, Yonekawa  Y, Weissgold  DJ, Lee  TC, Chiang  MF.  Accuracy of retinopathy of prematurity diagnosis by retinal fellows.  Retina. 2010;30(6):958-965.PubMedGoogle ScholarCrossref
25.
Chiang  MF, Thyparampil  PJ, Rabinowitz  D.  Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity.  Arch Ophthalmol. 2010;128(9):1153-1159.PubMedGoogle ScholarCrossref
26.
Rivero  ME, Bartsch  DU, Otto  T, Freeman  WR.  Automated scanning laser ophthalmoscope image montages of retinal diseases.  Ophthalmology. 1999;106(12):2296-2300.PubMedGoogle ScholarCrossref
27.
Silva  PS, Cavallerano  JD, Sun  JK, Noble  J, Aiello  LM, Aiello  LP.  Nonmydriatic ultrawide field retinal imaging compared with dilated standard 7-field 35-mm photography and retinal specialist examination for evaluation of diabetic retinopathy.  Am J Ophthalmol. 2012;154(3):549-559.e2.PubMedGoogle ScholarCrossref
28.
Soliman  AZ, Silva  PS, Aiello  LP, Sun  JK.  Ultra–wide field retinal imaging in detection, classification, and management of diabetic retinopathy.  Semin Ophthalmol. 2012;27(5-6):221-227.PubMedGoogle ScholarCrossref
29.
Wessel  MM, Aaker  GD, Parlitsis  G, Cho  M, D’Amico  DJ, Kiss  S.  Ultra–wide-field angiography improves the detection and classification of diabetic retinopathy.  Retina. 2012;32(4):785-791.PubMedGoogle ScholarCrossref
30.
Li  HK, Esquivel  A, Hubbard  LD, Florez-Arango  JF, Danis  RP, Krupinski  EA.  Mosaics versus early treatment diabetic retinopathy seven standard fields for evaluation of diabetic retinopathy severity.  Retina. 2011;31(8):1553-1563.PubMedGoogle ScholarCrossref
31.
Chiang  MF, Wang  L, Busuioc  M,  et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality.  Arch Ophthalmol. 2007;125(11):1531-1538.PubMedGoogle ScholarCrossref
32.
Chiang  MF, Melia  M, Buffenn  AN,  et al.  Detection of clinically significant retinopathy of prematurity using wide-angle digital retinal photography: a report by the American Academy of Ophthalmology.  Ophthalmology. 2012;119(6):1272-1280.PubMedGoogle ScholarCrossref
×