Is plus disease in retinopathy of prematurity more reliably and accurately diagnosed by eye-based assessment of overall retinal appearance or by quadrant-based assessment combining grades of individual quadrants?
In this multicenter cohort study including 141 preterm infants, graders diagnosed 197 eyes by reviewing the entire wide-angle retinal images (eye-based diagnosis; plus vs not plus) and subsequently by reviewing each of the cropped quadrant images (plus vs not plus), which was combined into a quadrant-based diagnosis. Graders had lower intragrader and intergrader agreement and agreement with reference standard diagnosis using quadrant-based diagnosis than eye-based diagnosis.
Eye-based diagnosis may have advantages over quadrant-based diagnosis regarding reliability and accuracy.
Presence of plus disease in retinopathy of prematurity is the most critical element in identifying treatment-requiring disease. However, there is significant variability in plus disease diagnosis. In particular, plus disease has been defined as 2 or more quadrants of vascular abnormality, and it is not clear whether it is more reliably and accurately diagnosed by eye-based assessment of overall retinal appearance or by quadrant-based assessment combining grades of 4 individual quadrants.
To compare eye-based vs quadrant-based diagnosis of plus disease and to provide insight for ophthalmologists about the diagnostic process.
Design, Setting, and Participants
In this multicenter cohort study, we developed a database of 197 wide-angle retinal images from 141 preterm infants from neonatal intensive care units at 9 academic institutions (enrolled from July 2011 to December 2016). Each image was assigned a reference standard diagnosis based on consensus image-based and clinical diagnosis. Data analysis was performed from February 2017 to September 2017.
Six graders independently diagnosed each of the 4 quadrants (cropped images) of the 197 eyes (quadrant-based diagnosis) as well as the entire image (eye-based diagnosis). Images were displayed individually, in random order. Quadrant-based diagnosis of plus disease was made when 2 or more quadrants were diagnosed as indicating plus disease by combining grades of individual quadrants post hoc.
Main Outcomes and Measures
Intragrader and intergrader reliability (absolute agreement and κ statistic) and accuracy compared with the reference standard diagnosis.
Of the 141 included preterm infants, 65 (46.1%) were female and 116 (82.3%) white, and the mean (SD) gestational age was 27.0 (2.6) weeks. There was variable agreement between eye-based and quadrant-based diagnosis among the 6 graders (Cohen κ range, 0.32-0.75). Four graders showed underdiagnosis of plus disease with quadrant-based diagnosis compared with eye-based diagnosis (by McNemar test). Intergrader agreement of quadrant-based diagnosis was lower than that of eye-based diagnosis (Fleiss κ, 0.75 [95% CI, 0.71-0.78] vs 0.55 [95% CI, 0.51-0.59]). The accuracy of eye-based diagnosis compared with the reference standard diagnosis was substantial to near-perfect, whereas that of quadrant-based plus disease diagnosis was only moderate to substantial for each grader.
Conclusions and Relevance
Graders had lower reliability and accuracy using quadrant-based diagnosis combining grades of individual quadrants than with eye-based diagnosis, suggesting that eye-based diagnosis has advantages over quadrant-based diagnosis. This has implications for more precise definitions of plus disease regarding the criterion of 2 or more quadrants, clinical care, computer-based image analysis, and education for all ophthalmologists who manage retinopathy of prematurity.
Kim SJ, Campbell JP, Kalpathy-Cramer J, et al. Accuracy and Reliability of Eye-Based vs Quadrant-Based Diagnosis of Plus Disease in Retinopathy of Prematurity. JAMA Ophthalmol. 2018;136(6):648–655. doi:10.1001/jamaophthalmol.2018.1195
Customize your JAMA Network experience by selecting one or more topics from the list below.
Create a personal account or sign in to: