Sensitivity and specificity of plus disease diagnosis based on quantitative parameters of the standard published photograph. A, Arterial tortuosity index (ATI). B, Venous diameter (VD). Solid curves display sensitivity and dashed curves display specificity of plus disease diagnosis as a function of the quantitative parameter value, compared with the reference standard of majority vote among 22 retinopathy of prematurity experts. Vertical lines denote the parameter values from the standard published photograph and corresponding sensitivity and specificity based on expert opinions.
Representative images demonstrating the spectrum of quantitative vascular characteristics in the study images. A, Arterial tortuosity index (ATI) of all analyzed arteries. Image I had a mean ATI of 1.09 and was classified as not plus disease by expert reference standard. Image II had a mean ATI of 1.15 and is the standard photograph. Image III had a mean ATI of 1.15 and was classified as not plus disease. Image IV had a mean ATI of 1.48 and was classified as plus disease. B, Venous diameter (VD) of all analyzed veins. Image I had a mean VD of 57 μm and was classified as not plus disease by expert reference standard. Image II had a mean VD of 67 μm and is the standard photograph. Image III had a mean VD of 79 μm and was classified as not plus disease. Image IV had a mean VD of 82 μm and was classified as plus disease.
Gelman SK, Gelman R, Callahan AB, Martinez-Perez ME, Casper DS, Flynn JT, Chiang MF. Plus Disease in Retinopathy of Prematurity: Quantitative Analysis of Standard Published Photograph. Arch Ophthalmol. 2010;128(9):1217-1220. doi:10.1001/archophthalmol.2010.186
Copyright 2010 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2010
Plus disease is defined as abnormality of the posterior retinal vessels in which the arterial tortuosity and venous dilation meet or exceed those of a standard photograph selected by expert consensus in the 1980s.1,2 This method has limitations, and studies have suggested that interexpert agreement in plus disease diagnosis is variable.3 Magnification of the standard photograph is larger than that of indirect ophthalmoscopy, and peripheral vessels are not visible in the narrow field of view. It is also unclear which vessels clinicians should focus on while evaluating tortuosity and dilation. This study seeks to quantify vascular characteristics of the standard photograph compared with expert interpretations of plus disease. Such data could help assess clinical applicability of the standard photograph and contribute to development of objective disease definitions based on quantitative principles.
Thirty-four wide-angle retinal images were interpreted by 22 retinopathy of prematurity experts for presence of plus disease.3 For each wide-angle image, a reference standard diagnosis (plus disease or not plus disease) was defined using majority vote among experts.
Vessels from wide-angle images and from the standard photograph1 were analyzed by a computer-based system to calculate the mean arterial tortuosity index (ATI) (length of vessel segment divided by length of straight line connecting vessel ends) and the venous diameter (VD) (area divided by length of vessel) among all vessels in each image as described previously.4,5 Because ATI is a ratio, it is independent of magnification and resolution. However, comparison of VD values among different images required adjustments for magnification differences. This was done by normalizing all optic disc diameters to 1015 μm based on prior literature.6
Sensitivity and specificity of the computer-based system for detecting plus disease were plotted as a function of cutoff values of ATI and VD that were used to separate plus disease from not plus disease (Figure 1) based on this reference standard. For example, the number of missed cases of plus disease (ie, false-negatives) would be expected to increase as the cutoff value of ATI (or VD) increases; therefore, sensitivity decreases as ATI (or VD) increases.
The mean ATI of the standard photograph was 1.15, which was significantly lower than the ATI in wide-angle images that were diagnosed as plus disease (mean, 1.26; P = .003) and not plus disease (mean, 1.19; P = .001). The mean VD of the standard photograph was 66.88 μm, which was significantly lower than both plus disease in wide-angle images (mean, 81.63 μm; P = .002) and not plus disease in wide-angle images (mean, 78.95 μm; P < .001).
If the standard photograph parameter values were used as the cutoff for plus disease, the mean ATI of 1.15 would result in 85% sensitivity and 38% specificity, while the mean VD of 66.88 μm would result in 85% sensitivity and 14% specificity (Figure 1).
Anecdotally, we feel that some experts regard the standard photograph as showing plus disease appearing more severe than their personal cutoffs for plus disease. This is inconsistent with findings from our study. To investigate, we compared the standard photograph with representative examples from 34 wide-angle images. This confirmed that the ATI in the standard photograph did appear low and suggested that after adjusting for magnification, the VD appeared visually similar to that seen in several wide-angle images diagnosed as not plus disease by experts (Figure 2). Therefore, we feel a possible explanation is that magnification and field of view in the standard photograph may cause difficulty for ophthalmologists.
A study limitation is that it was often difficult to identify precise disc margins during image normalization (although a consensus was reached among us), and infants may have true differences in disc size. Also, tortuosity as defined in this study was a gross measure, and future research involving other metrics for calculating tortuosity may be warranted.
These findings suggest that using the standard photograph as the cutoff definition for plus disease results in high sensitivity and low specificity compared with expert diagnoses from an independent set of wide-angle images. In other words, arterial tortuosity and venous dilation in the standard photograph may be less severe than what experts are considering to be plus disease. This raises questions about whether the current published photograph requires modification to represent true expert opinion or whether strategies are required to recalibrate ophthalmologists to the standard photograph.
Correspondence: Dr Chiang, Columbia University College of Physicians and Surgeons, 635 W 165th St, Box 92, New York, NY 10032 (firstname.lastname@example.org).
Financial Disclosure: None reported.
Funding/Support: This work was supported by a Career Development Award from Research to Prevent Blindness and by grant EY13972 from the National Eye Institute, National Institutes of Health (Dr Chiang).
Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.
Additional Information: Dr Chiang is an unpaid member of the scientific advisory board for Clarity Medical Systems, Pleasanton, California.
Additional Contributions: Earl Palmer, MD, provided a digital copy of the standard photograph.