Only hard exudates within 1 disc diameter of the fovea (middle circle) were considered. In this case, 4 sectors (starred) are affected by hard exudates.
Shown is the detection of clinically significant macular edema (CSME) confirmed by dilated biomicroscopy in combination with optical coherence tomography (left) and by stereoscopic fundus photography (right). HEs indicates hard exudates.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Litvin TV, Bresnick GH, Cuadros JA, Selvin S, Kanai K, Ozawa GY. A Revised Approach for the Detection of Sight-Threatening Diabetic Macular Edema. JAMA Ophthalmol. 2017;135(1):62–68. doi:10.1001/jamaophthalmol.2016.4772
Does a sectors approach for the detection of sight-threatening diabetic macular edema perform well and offer any advantages when tested against a criterion standard stereoscopic fundus photography approach?
In this cross-sectional study, a sectors approach showed accuracy in the detection of clinically significant macular edema and offered up to an 8.6% increase in specificity compared with the existing methods of detection.
A sectors approach may be considered for implementation in the detection of clinically significant macular edema by diabetic retinopathy screening programs and may be well suited for integration in algorithms of automated diabetic retinopathy detection when the use of optical coherence tomography is not feasible.
Diabetic macular edema is one of the leading causes of vision loss among working-age adults in the United States. Telemedicine screening programs and epidemiological studies rely on monoscopic fundus photography for the detection of clinically significant macular edema (CSME). Improving the accuracy of detecting CSME from monoscopic images could be valuable while recognizing the limitations of such detection in an era of optical coherence tomography detection of diabetic macular edema.
To evaluate the screening test accuracy of radially arranged sectors affected by hard exudates in the detection of CSME.
Design, Setting, and Participants
This investigation was a cross-sectional study of CSME grading in monoscopic images using a sectors approach. The Early Treatment Diabetic Retinopathy Study criteria were used to confirm the presence of CSME by the following 2 methods: stereoscopic fundus photography (method 1) and dilated biomicroscopy in combination with optical coherence tomography (method 2). Participants were recruited at a university-based practice between June 14, 2014, and December 28, 2015.
Main Outcomes and Measures
Area under the receiver operating characteristic curve, sensitivity, specificity, positive predictive value, and negative predictive value.
A total of 207 eyes from an ethnically/racially diverse group of 207 patients (mean [SD] age, 53.6 [10.8] years; 58.9% [122 of 207] female) were included in the analysis. Twelve eyes (5.8%) were diagnosed as having CSME based on method 1. The intermethod and intergrader agreement for CSME diagnosis and sector count was substantial (κ range, 0.66 [95% CI, 0.47-0.85] to 0.75 [95% CI, 0.53-0.97]; P < .001 for all). Area under the receiver operating characteristic curve was 93.2% (95% CI, 84.2%-100%) when evaluating a sectors approach against method 1 as a reference test and offered up to an 8.6% (95% CI, 3.0%-14.3%) increase in specificity compared with the existing methods of detection. The positive predictive value was 33.3% (95% CI, 25.6%-45.5%), and the negative predictive value was 98.1% (95% CI, 96.9%-100%). The results were similar when comparing a sectors approach with method 2 as a reference test.
Conclusions and Relevance
A sectors approach shows good screening test characteristics for the detection of CSME. Its implementation in the existing telemedicine programs would require minimal resources. This approach will have the greatest effect in a setting where implementation of optical coherence tomography, a more objective and sensitive way to detect retinal thickening, is not feasible. The proposed method also may be easily incorporated in the automated diabetic retinopathy detection algorithms.
Diabetic retinopathy remains the leading cause of vision impairment in working-age adults in the United States.1 Diabetic macular edema (DME) is a primary origin of vision loss in patients with diabetes, affecting an estimated 2.7% of adults with diabetes.1-3 The risk of vision loss associated with DME can be significantly reduced and the chance of vision gain increased with appropriate treatment when detected in time.2,4
Clinically significant macular edema (CSME) was originally defined by the Early Treatment Diabetic Retinopathy Study (ETDRS) in terms of retinal thickening as detected by stereo retinal photography or clinical examination with stereo biomicroscopy.4 In the absence of stereo photography or stereo biomicroscopy, the National Health and Nutrition Examination Survey and other epidemiological studies, as well as various teleophthalmology programs, rely on surrogate markers for the detection of CSME. Most, if not all, such programs rely on the detection of hard exudates (HEs), bright yellow lipoprotein deposits in the central macula, on 2-dimensional digital fundus photographs as a surrogate for CSME detection.5-9 However, there is a lack of agreement on the best parameters for grading the presence and severity of CSME with respect to the location and the extent of HEs in the macula.10,11 In addition, the simple presence of HEs in the macular region has been shown to have only modest specificity and, in some cases, poor sensitivity.10-13 There is a need to improve the accuracy of CSME detection using new screening approaches that are validated against the accepted standards for determining the presence of CSME.
In our group’s previous study,14 an exploratory analysis showed that the proportion of individuals with more advanced cases of CSME increased as the number of sectors affected by HEs increased. The main objective of the present study was to evaluate the accuracy of radially arranged sectors affected by HEs in the detection of any CSME by comparing this screening method with the following 2 methods: (1) stereoscopic fundus photography, a criterion standard used in research,15-19 and (2) dilated biomicroscopy in combination with optical coherence tomography (OCT), a criterion standard used in clinical practice.20-22 The need for inclusion of OCT information for the detection of CSME has been recognized in a number of publications because it increases objectivity and diagnostic accuracy.11,13,22-24 Inclusion of the 2 criterion standard diagnostic modalities allowed us to evaluate the internal validity of the study by quantifying the intermethod agreement for CSME diagnosis. In addition, we could determine how robust our estimates of the screening test characteristics are by comparing the estimates calculated with the 2 different diagnostic modalities.
Two hundred twenty-five consecutive patients with diabetes seen for an eye examination at Eastmont Wellness Center, Alameda Health System, Oakland, California, were enrolled in this study. All nonpregnant adult patients with a diagnosis of type 1 or 2 diabetes and without a history of glaucoma or a nondiabetic macular pathologic condition were eligible for enrollment. The study protocol was approved by the University of California, Berkeley, and the Alameda Health System institutional review boards. All participants signed a written informed consent form. Enrollment occurred between June 14, 2014, and December 28, 2015. Macular OCT scans of each eye were obtained after pharmacological pupillary dilation using a retina map protocol on spectral-domain OCT (iVue; Optovue, Incorporated). Subsequently, three 45° photographs of the ocular fundi were obtained using a digital camera (CR-DGI; Canon, Inc) following the EyePACS diabetic retinopathy screening (DRS) protocol.25 The primary field included the optic nerve and the macula. The second field was centered on the optic nerve. The third field included the optic nerve at the far nasal side of the image. In addition, nonsimultaneous stereoscopic 45° digital photographs of the macula were obtained using a modified technique first described by Allen26 in 1964 and detailed by Tyler.27 The agreement for the diagnosis of CSME was shown to be good between the standard 30° stereoscopic film photographs and the 45° stereoscopic digital photographs.16 All of the images were uploaded to a secure location in the EyePACS database. Patients’ medical records were accessed to obtain clinical data measured within 3 months of imaging.
The level of retinopathy for each eye was established by evaluating digital fundus images by 2 experienced EyePACS-certified graders (T.V.L. and K.K.) using the EyePACS grading system28 following the International Clinical Diabetic Retinopathy Severity Scale.29 The graders were unaware of the findings of each other.
A clinical diagnosis of CSME was established at the time of the examination by a clinician (K.K. or G.Y.O.) performing dilated noncontact stereoscopic fundus biomicroscopy and after reviewing macular OCT scans. Clinicians were not required to use any specific cutoff value for the central subfield thickness for the diagnosis of CSME, although it is understood that retinal thickening of less than 300 µm is difficult to detect reliably during stereoscopic biomicroscopy.30 Rather, evaluation of OCT scans was done in the context of the clinical examination with access to the iVue OCT normative retinal thickness data and examination of b-scans. Clinicians integrated information obtained from the dilated biomicroscopy and OCT scans to arrive at a clinical decision, as would be done in practice. Stereoscopic photographs of the macula were evaluated by the 2 graders, unaware of the grading results of each other and to the results of a clinical diagnosis of CSME. One grader (K.K.) was also involved in the clinical evaluation of approximately half of the enrolled participants; however, the grading of the stereoscopic photographs occurred several months after the clinical examination to reduce the possibility of recall. Stereoscopic photographs were graded on a 27-in color-calibrated monitor with 1920 × 1080–pixel resolution. Stereoscopic pairs were presented on the monitor using a custom program (Matlab; Mathworks, Inc) and viewed through a stereoscope (Screen-Vu; Berezin Stereo Photography Products). A diagnosis of CSME was established based on the ETDRS criteria.4 Cases with intergrader disagreement were adjudicated by a third expert grader (J.A.C.).
One of the monoscopic images captured during the nonsimultaneous stereophotography, equivalent to the standard field 3 EyePACS,25 was used to determine the number of sectors affected by HEs. The best-quality image was selected. A custom Matlab program was used to place the 8-sector grid centered on the fovea to ensure the consistency in the orientation of the grid placement across the participants and the graders (Figure 1). The number of sectors within a circle with the radius equal to 1 disc diameter (DD) that were affected by HEs was established by the 2 graders, unaware of each other’s grading. This grading was done during sessions separate from the grading for the presence of CSME to minimize recall. Any discrepancies in grading were adjudicated by the third grader. The presence of HEs within 1 DD from the fovea, which is equivalent to 1 sector, is used by some DRS programs to define CSME.28 All images were also graded by the 2 graders, unaware of each other’s grading, for the presence of HEs within 500 µm, a criterion used by large epidemiological studies (eg, the National Health and Nutrition Examination Survey) and other DRS programs for the detection of CSME.9,13
A single eye per patient was used to eliminate the need for correlation adjustment when both eyes of the same patient were included in the analysis. Exclusion based on the prespecified criteria was done on a per-eye basis. Fifty-five of the original 449 eyes (1 patient was monocular) were excluded for the following reasons: presence of any macular pathologic condition other than DME (n = 31), media opacity resulting in ungradable fundus images (n = 17), noncompletion of the study procedures (n = 6), and missing images (n = 1). Of the remaining eyes, those with a CSME diagnosis based on either of the diagnostic modalities were eligible for inclusion. In cases in which both eyes or neither of the eyes were diagnosed as having CSME, the right eye or a single remaining eye was selected for the analysis.
All of the analyses were performed using a computer program (R; https://cran.r-project.org). Intergrader agreement was evaluated using unweighted and weighted Cohen κ and percentage agreement value. A weight of 1 was assigned to exact agreement, and a weight of 0.75 was assigned for agreement within 1 step. A zero value was assigned for disagreement of more than 1 step. A 2-sided t test was used for comparison of continuous data. The Mann-Whitney test was used to compare continuous data that were not normally distributed. The Fisher exact test was used to assess categorical data. The significance level was set at P = .05. Area under the receiver operating characteristic curve and indicators of diagnostic accuracy were calculated using custom software packages (pROC and Epi).31,32 The McNemar test was used to compare sensitivity and specificity values of the diagnostic tests using another custom software package (DTComPair).33
Two hundred seven eyes were included in the final analysis. The demographic and clinical variables of the sample are summarized in Table 1. The CSME and non-CSME groups were balanced with respect to the demographic and clinical characteristics. Nine cases (4.3%) of CSME were identified during the OCT-assisted dilated fundus examination, and 12 cases (5.8%) were identified during the grading of the stereoscopic photographs. Intergrader agreement regarding the diagnosis of CSME based on stereoscopic photographs of the macula was substantial according to the classification scale proposed by Landis and Koch34 (κ = 0.66 [95% CI, 0.47-0.85], P < .001), and percentage agreement was 97.7%. The intermethod agreement for CSME diagnosis was substantial (κ = 0.75 [95% CI, 0.53-0.97], P < .001), and percentage exact agreement was 97.6%. Intergrader agreement in establishing the number of sectors was substantial for the unweighted κ (κ = 0.67 [95% CI, 0.53-0.81], P < .001): percentage exact agreement was 90.8%, and percentage within 1 sector agreement was 98.1%. Intergrader agreement for the detection of HEs within 500 µm from the center of the macula was substantial (κ = 0.75 [95% CI, 0.61-0.89], P < .001), and percentage agreement was 94.7%. Among 207 participants, 52.2% (n = 108) had no diabetic retinopathy, 15.9% (n = 33) had mild nonproliferative diabetic retinopathy (NPDR), 26.1% (n = 54) had moderate NPDR, 4.3% (n = 9) had severe NPDR, and 1.4% (n = 3) had proliferative diabetic retinopathy.
When evaluating the screening test accuracy of a sectors approach for the detection of CSME diagnosed during the dilated fundus examination and with evaluation of stereoscopic photographs, areas under the receiver operating characteristic curve were 97.6% (95% CI, 95.1%-100.0%) and 93.2% (95% CI, 84.2%-100.0%), respectively (Figure 2A). The sensitivity, specificity, positive predictive value, and negative predictive value for the 2 optimal thresholds of 1 and 3 sectors are summarized in Table 2. The optimal thresholds were selected by evaluating the local maxima on the corresponding area under the receiver operating characteristic curve and selecting those thresholds that resulted in the best sensitivity or specificity values, while keeping the value of the other parameter in this pair at or above 75%.
The 2 commonly used approaches for the detection of CSME in monoscopic images are (1) presence of HEs within 500 µm from the center of the macula and (2) presence of HEs within 1 DD from the center of the macula (equivalent to only using a cutoff of 1 sector).10,12,13,28 The performance of these 2 methods for the detection of CSME is summarized in Figure 2B and C and in Table 2. To compare a sectors approach with the 2 commonly used approaches for the detection of CSME, referenced above, we used the best sensitivity and specificity values achieved using a sectors approach. When using stereoscopic photographs as a reference test for CSME diagnosis, there was no difference in sensitivity values of a sectors approach and that of HEs within 500 µm. The difference in the corresponding specificity values was 5.1% (95% CI, 0.2%-10.3%) (P = .008). When using OCT-assisted clinical examination as a reference test, the difference in sensitivity values of a sectors approach and that of HEs within 500 µm was 11.1% (95% CI, −24.2% to 48.2%) (P = .32). The difference in the corresponding specificity values was 6.1% (95% CI, 0.9%-11.5%) (P = .001).
When comparing a sectors approach with that of HEs within 1 DD, the sensitivities will be the same because a cutoff of 1 sector is equivalent to the criterion of HEs within 1 DD. The difference in specificity values when stereoscopic photographs were used as a reference test was 8.2% (95% CI, 2.8%-13.8%) (P < .001). When OCT-assisted clinical examination was used as a reference test, the difference in specificity values was 8.6% (95% CI, 3.0%-14.3%) (P < .001).
A multivariable logistic regression analysis was performed to evaluate an association between the diagnosis of CSME confirmed by stereoscopic photographs using a sectors approach and that of HEs within 500 µm. Both were significant in a bivariate analysis. The sectors approach (odds ratio, 2.4; 95% CI, 1.3-4.9; P = .01) as well as HEs within 500 μm (odds ratio, 12.7; 95% CI, 1.3-311.7; P = .05) retained a statistically significant association with CSME in the multivariable model. An increase of 1 sector was associated with an odds ratio of 2.4 (95% CI, 1.3-4.9) for being diagnosed as having CSME, regardless of whether the HEs were located within or outside of 500 µm from the center of the macula.
We evaluated the performance of a revised screening approach for the detection of CSME using the number of sectors affected by HEs within the central 1 DD of the macula. Sectors showed good screening test characteristics in the detection of CSME compared with the criterion standard used in clinical practice (dilated biomicroscopy in combination with OCT) and the standard test used in research. Compared with the existing methods of CSME detection in monoscopic fundus photographs, a sectors approach offers increased specificity. The difference in sensitivity was not statistically significant. Good agreement in the screening test characteristics derived based on the 2 diagnostic methods suggests that the results can be generalized to a range of clinical settings. While standardized grading of stereoscopic photographs by multiple graders provides an accepted, rigorous, and repeatable evaluation of the presence of CSME, it is not generally performed in a clinical setting. More frequently, a decision to initiate treatment for sight-threatening DME is made, at least in part, based on dilated fundus biomicroscopy and the central subfield thickness on OCT.13
The proposed screening method allows for easy threshold optimization to aid in decision making with respect to the timing and type of service (general eye vs vitreoretinal subspecialty) to be specified in the referral. For example, an already implemented DRS program that uses digital fundus photography would maximize the detection of CSME by using 1 sector as a cutoff value and would refer patients meeting this criterion but not meeting the criterion of 3 sectors to a general eye care service, which could easily manage up to 12% of patients who were incorrectly identified as having CSME. Similarly, such a program would use a cutoff value of 3 sectors to refer patients directly to a retina specialist within an appropriately short period. Such triage will assure a low overreferral of approximately 4% and prioritize patients with more severe cases of DME, as shown in our group’s previous study.14 This grading approach will keep false-positive results and false-negative results to a desired minimum while requiring no additional resources for DRS programs.
Our results suggest that, regardless of the diagnostic method used, this screening test will miss up to 8% of CSME cases (100% minus the sensitivity of 1 sector) (Table 2), an acceptable false-negative result in most screening situations. In addition, this screening approach will result in unnecessary referrals of approximately 4% (100% minus the specificity of 3 sectors), an acceptable false-positive result. The strength of a sectors screening approach for CSME detection likely lies in its standardized measurement of both the extent and the proximity of retinal thickening in the macula captured by the CSME definition. Specifically, according to ETDRS data,4,19,35 the smaller area of retinal thickening closer to the fovea carries an increased risk for vision loss, as does the larger area of retinal thickening located farther away from the fovea, with center-involving edema carrying somewhat greater risk. To parallel that, it would require a larger patch of HEs to occupy 3 sectors if the patch is located farther away from the fovea than if it was located closer to the fovea. If HEs are assumed to approximate the location and the extent of retinal thickening, then a sectors approach gives greater weight to the HEs, and therefore edema, closer to the fovea.
A strength of this study is inclusion of 2 diagnostic methods for the diagnosis of CSME and substantial agreement between the 2 approaches. In addition, although this intermethod agreement is not perfect, the results of the screening test accuracy analysis calculated separately for each of the 2 diagnostic methods are similar, offering further support to the validity of the proposed screening method.
A limitation of this detection method lies in the fact that it does not include OCT measurements of the retinal thickness in the macula. Future studies stand to benefit from incorporating OCT information in the detection model. We focused our efforts on developing and evaluating a low-cost method that is easy to incorporate in the existing DRS programs without the additional expense.
A potential limitation of this study is the few CSME cases, which contributed to somewhat wider 95% CIs around the sensitivity estimates than desired, although they were still within reasonable range. However, the 95% CIs around the specificity estimates are desirably narrow. We considered it important to recruit consecutive patients with diabetes seen at an optometric clinic (rather than artificially recruiting a sample of patients enriched with CSME) to evaluate our screening approach in a real-life situation that favors generalization of the findings to a wider population.36
In summary, a sectors approach showed good screening test characteristics and offers increased specificity in the detection of CSME compared with the 2 methods now in use. The sensitivity of this method is high but not statistically different from the existing approaches. While this method of sight-threatening DME detection lacks the diagnostic accuracy offered by OCT technology, it showed good capacity to discriminate CSME cases when tested against the OCT-assisted method of detection. In addition, a sectors approach does not require DRS sites to invest resources in acquisition of additional equipment. The proposed method may also be easily incorporated in the automated diabetic retinopathy detection algorithms.
Accepted for Publication: October 8, 2016.
Corresponding Author: Taras V. Litvin, OD, PhD, School of Optometry, University of California, Berkeley, 200 Minor Hall, Berkeley, CA 94720 (firstname.lastname@example.org).
Published Online: December 8, 2016. doi:10.1001/jamaophthalmol.2016.4772
Author Contributions: Dr Litvin had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Litvin, Bresnick, Selvin, Ozawa.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Litvin, Selvin, Kanai.
Critical revision of the manuscript for important intellectual content: Litvin, Bresnick, Cuadros, Selvin, Ozawa.
Statistical analysis: Litvin, Selvin.
Obtained funding: Litvin, Bresnick, Ozawa.
Administrative, technical, or material support: Litvin, Bresnick, Cuadros, Kanai, Ozawa.
Study supervision: Litvin, Ozawa.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Bresnick and Cuadros reported having a personal financial interest in EyePACS LLC. No other disclosures were reported.
Funding/Support: This work was supported by grant K12EY017269 from the National Institutes of Health (Dr Litvin).
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: Olivia Israel, BS (University of California, Berkeley), and Angel Barajas, BS (University of California, Berkeley), helped with patient recruitment and data collection. Stacy Meuer, BS (University of Wisconsin–Madison), assisted with standardizing retinal image acquisition and grading. None received compensation.
Create a personal account or sign in to: