Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes

Question  How does a deep learning system (DLS) using artificial intelligence compare with professional human graders in identifying diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes?

Findings  In the primary validation dataset (71 896 images; 14 880 patients), the DLS had a sensitivity of 90.5% and specificity of 91.6% for detecting referable diabetic retinopathy; 100% sensitivity and 91.1% specificity for vision-threatening diabetic retinopathy; 96.4% sensitivity and 87.2% specificity for possible glaucoma; and 93.2% sensitivity and 88.7% specificity for age-related macular degeneration, compared with professional graders.

Meaning  The DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.

Importance  A deep learning system (DLS) is a machine learning technology with potential for screening diabetic retinopathy and related eye diseases.

Objective  To evaluate the performance of a DLS in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, possible glaucoma, and age-related macular degeneration (AMD) in community and clinic-based multiethnic populations with diabetes.

Design, Setting, and Participants  Diagnostic performance of a DLS for diabetic retinopathy and related eye diseases was evaluated using 494 661 retinal images. A DLS was trained for detecting diabetic retinopathy (using 76 370 images), possible glaucoma (125 189 images), and AMD (72 610 images), and performance of DLS was evaluated for detecting diabetic retinopathy (using 112 648 images), possible glaucoma (71 896 images), and AMD (35 948 images). Training of the DLS was completed in May 2016, and validation of the DLS was completed in May 2017 for detection of referable diabetic retinopathy (moderate nonproliferative diabetic retinopathy or worse) and vision-threatening diabetic retinopathy (severe nonproliferative diabetic retinopathy or worse) using a primary validation data set in the Singapore National Diabetic Retinopathy Screening Program and 10 multiethnic cohorts with diabetes.

Exposures  Use of a deep learning system.

Main Outcomes and Measures  Area under the receiver operating characteristic curve (AUC) and sensitivity and specificity of the DLS with professional graders (retinal specialists, general ophthalmologists, trained graders, or optometrists) as the reference standard.

Results  In the primary validation dataset (n = 14 880 patients; 71 896 images; mean [SD] age, 60.2 [2.2] years; 54.6% men), the prevalence of referable diabetic retinopathy was 3.0%; vision-threatening diabetic retinopathy, 0.6%; possible glaucoma, 0.1%; and AMD, 2.5%. The AUC of the DLS for referable diabetic retinopathy was 0.936 (95% CI, 0.925-0.943), sensitivity was 90.5% (95% CI, 87.3%-93.0%), and specificity was 91.6% (95% CI, 91.0%-92.2%). For vision-threatening diabetic retinopathy, AUC was 0.958 (95% CI, 0.956-0.961), sensitivity was 100% (95% CI, 94.1%-100.0%), and specificity was 91.1% (95% CI, 90.7%-91.4%). For possible glaucoma, AUC was 0.942 (95% CI, 0.929-0.954), sensitivity was 96.4% (95% CI, 81.7%-99.9%), and specificity was 87.2% (95% CI, 86.8%-87.5%). For AMD, AUC was 0.931 (95% CI, 0.928-0.935), sensitivity was 93.2% (95% CI, 91.1%-99.8%), and specificity was 88.7% (95% CI, 88.3%-89.0%). For referable diabetic retinopathy in the 10 additional datasets, AUC range was 0.889 to 0.983 (n = 40 752 images).

Conclusions and Relevance  In this evaluation of retinal images from multiethnic cohorts of patients with diabetes, the DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases. Further research is necessary to evaluate the applicability of the DLS in health care settings and the utility of the DLS to improve vision outcomes.

By 2040, it is projected that approximately 600 million people will have diabetes, with one-third expected to have diabetic retinopathy.^1-3 Screening for diabetic retinopathy, coupled with timely referral and treatment, is a universally accepted strategy for blindness prevention.² However, programs for screening diabetic retinopathy are challenged by issues related to implementation, availability of human assessors, and long-term financial sustainability.^2,4-7

A deep learning system (DLS) uses artificial intelligence and representation learning methods to process large data and extract meaningful patterns.^8,9 A few DLSs have recently shown high sensitivity and specificity (>90%) in detecting referable diabetic retinopathy from retinal photographs, primarily using high-quality images from publicly available databases from homogenous populations of white individuals.^10-12 The performance of a DLS in screening for diabetic retinopathy should ideally be evaluated in clinical or population settings in which retinal images from patients of different races and ethnicities (and therefore with varying fundi pigmentation) have varying qualities (eg, due to poor pupil dilation, media opacity, poor contrast or focus).^13,14 Furthermore, in screening programs for diabetic retinopathy, the detection of incidental but related vision-threatening eye diseases, such as glaucoma and age-related macular degeneration (AMD), should be incorporated because missing such cases is clinically unacceptable.¹⁵

The primary aim of this study was to train and validate a DLS to detect referable diabetic retinopathy, vision-threatening diabetic retinopathy, and related eye diseases (referable possible glaucoma and referable AMD) by evaluating retinal images obtained primarily from patients with diabetes in an ongoing community-based national diabetic retinopathy screening program in Singapore, with further external validation on referable diabetic retinopathy in 10 additional multiethnic datasets from different countries with diverse community- and clinic-based populations with diabetes. The secondary aim was to determine how the DLS could fit in 2 potential models of diabetic retinopathy screening—a fully automated model for communities with no existing screening programs and a semiautomated model in which referable cases from the DLS undergo a secondary assessment by human graders.

This study was approved by the centralized institutional review board (IRB) of SingHealth, Singapore (protocol SHF/FG648S/2015) and conducted in accordance with the Declaration of Helsinki. Information on race/ethnicity was collected to evaluate the consistency of DLS diagnostic performance across races/ethnicities. Patients’ informed consent was exempted by the IRB because of the retrospective nature of study using fully anonymized retinal images.

The DLS for referable diabetic retinopathy was developed and trained using retinal images of patients with diabetes who participated in the ongoing Singapore National Diabetic Retinopathy Screening Program (SIDRP) between 2010 and 2013 (SIDRP 2010-2013; Table 1 and Table 2). The SIDRP was established from 2010, progressively covered all 18 primary care clinics across Singapore, and screened half of the diabetes population by 2015.¹⁶ SIDRP uses digital retinal photography, a tele-ophthalmology platform, and assessment of diabetic retinopathy by a team of trained professional graders. For each patient, 2 retinal photographs (optic disc and fovea) were taken of each eye. All trained graders received 3 to 6 months of training before certification and underwent annual reaccreditation. Specifically for this study, in the training set (SIDRP 2010-2013), each retinal image was analyzed by 2 trained senior certified nonmedical professional graders (>5 years’ experience)¹⁷; if there were discordant findings between the nonmedical professional graders, arbitration was performed by a retinal specialist (PhD-trained with >5 years’ experience in conducting diabetic retinopathy assessment) to generate final grading.

For referable possible glaucoma and AMD, the DLS was trained using images from SIDRP 2010-2013 and several additional population- and clinic-based studies of patients with glaucoma and AMD (Table 1; eTable 1 in the Supplement).^{17-20,22,23,25-29}

The DLS consisted of a convolutional neural network to implicitly recognize characteristics of referable diabetic retinopathy, possible glaucoma, and AMD from the appearance in retinal images. Training of the DLS entailed exposure of multiple examples of retinal images (with and without each of the 3 conditions) to the neural networks, allowing the networks to gradually adapt their weight parameters to model and differentiate between conditions. Once the training was complete, the DLS could be used to classify unseen images. Technical details are shown in eFigure 1 in the Supplement.

Details of validation datasets are described in Table 1. For diabetic retinopathy, the primary validation dataset was the same SIDRP among patients seen between 2014 and 2015 (SIDRP 2014-2015). The primary analysis was to determine if the DLS was equivalent or better than 2 trained senior nonmedical professional graders (>5 years’ experience) currently employed in the SIDRP in detecting referable diabetic retinopathy and vision-threatening diabetic retinopathy, with reference to a retinal specialist (>5 years’ experience in diabetic retinopathy grading).

The DLS was then externally validated using 10 additional multiethnic cohorts of participants with diabetes from different settings (community, population-based, and clinic-based). A range of retinal cameras were used, and assessment of diabetic retinopathy was facilitated by retinal specialists, general ophthalmologists, trained nonmedical professional graders, or optometrists across the cohorts (Table 1). All retinal images were captured with JPEG compression format (resolutions 5-7 megapixels, except for images of eyes in the Hispanic cohort [<1 megapixel]).

Guangdong: 5 nonmedical United Kingdom–certified professional graders (>2 years’ experience), supervised by 1 retinal specialist (>10 years’ experience). Singapore Malay Eye Study, Singapore Indian Eye Study, and Singapore Chinese Eye Study: 1 certified professional senior grader (>7 years’ experience), supervised by 2 senior retinal specialists from Australia (>15 years’ experience). Beijing Eye Study: 4 Chinese board-certified ophthalmologists (>5 years’ experience), supervised by 2 retinal specialists (>20 years’ experience). African American Eye Study: 2 retinal specialists (>5 years’ experience). Royal Victorian Eye and Ear Hospital: 4 professional senior graders (>7 years’ experience). Mexican study: 2 retinal specialists (>5 years’ experience). Chinese University of Hong Kong: 3 retinal specialists (>6 years’ experience). The University of Hong Kong: 6 optometrists (>4 years’ experience). Singapore National Eye Center Glaucoma Study: 3 glaucoma specialists (>5 years’ experience). Singapore National Eye Center AMD Phenotyping Study: 10 retinal specialists (>5 years’ experience).

Diabetic retinopathy levels from all retinal images were defined using the International Classification Diabetic Retinopathy Scale.³⁰ Referable diabetic retinopathy was defined as a diabetic retinopathy severity level of moderate nonproliferative diabetic retinopathy or worse, diabetic macular edema, and/or ungradable image. Vision-threatening diabetic retinopathy was defined as severe nonproliferative diabetic retinopathy and proliferative diabetic retinopathy. Diabetic macular edema was assessed as present if hard exudates were detected at the posterior pole of the retinal images. If more than one-third of the photograph was obscured, it was considered ungradable and the individual was considered referable. Referable possible glaucoma was defined as a ratio of vertical cup to disc diameter of 0.8 or greater, focal thinning or notching of the neuroretinal rim, optic disc hemorrhages, or localized retinal nerve fiber layer defects—features sometimes referred to as glaucoma suspects. Referable AMD was defined as the presence of intermediate AMD (numerous medium-sized drusen, 1 large drusen ≥125 μm in greatest linear diameter, noncentral geographical atrophy, and/or advanced AMD [central geographical atrophy or neovascular AMD]) according to the Age-Related Eye Disease Study grading system.³¹

For the primary validation dataset (SIDRP 2014-2015), the reference standard was grading by a retinal specialist (>5 years’ experience in conducting diabetic retinopathy assessment) who was masked to the grading of the trained nonmedical professional graders. For all other retinal images from the 10 external validation datasets, reference standards were based on individual studies’ assessment of diabetic retinopathy, which was based on retinal specialists, general ophthalmologists, trained nonmedical professional graders, or optometrists (Table 1). The DLS performance for identifying referable diabetic retinopathy in the 10 external validation datasets was compared against these reference standards. For the analysis on referable possible glaucoma and referable AMD, the reference standard was the retinal specialist (Table 1).

Initially the area under the curve (AUC) of the receiver operating characteristic (ROC) curve of DLS was calculated on the training dataset of the SIDRP 2010-2013 across a range of classification thresholds, and one was selected that achieved a predetermined optimal sensitivity of 90% for detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, referable possible glaucoma, and referable AMD. For diabetic retinopathy screening, international guidelines recommended a minimum sensitivity of 60% (Australia) to 80% (United Kingdom).^32,33 In Singapore, the DLS sensitivity was preset at 90% based on the trained professional graders’ past performances and criteria set by the Ministry of Health, Singapore. The hypothesis determined was that the DLS was at least comparable to the professional graders’ performance.

Primary analysis was to evaluate the performance of the DLS in the setting of the ongoing SIDRP 2014-2015 (the primary validation set) by determining whether the DLS was equivalent or superior to professional graders in the screening program. Thus, the AUC, sensitivity, and specificity of the DLS vs the professional graders in detecting referable diabetic retinopathy and vision-threatening diabetic retinopathy was computed to the reference standard (retinal specialist) at individual-eye levels.

Next, the following subsidiary analyses were performed: (1) the analyses were repeated excluding patients who appeared in both the SIDRP 2010-2013 training set and the primary validation set of SIDRP 2014-2015 (n = 6291 seen more than once in SIDRP), with the patient treated as having referable diabetic retinopathy if either eye had referable diabetic retinopathy; (2) performance of the DLS was evaluated using higher-quality images with no media opacity (eg, cataracts) as noted by professional graders; (3) AUC subgroups were computed stratified by age, sex, and glycemic control; and (4) the analysis was repeated by calculating the AUC, sensitivity, and specificity of the DLS and the proportion of concordant and discordant eyes on the 10 external validation datasets, compared with the reference standards in these studies (retinal specialists, general ophthalmologists, trained graders, or optometrists; Table 1).

The DLS performance was then evaluated in detection of referable possible glaucoma and referable AMD, with reference to a retinal specialist, using the primary validation dataset (SIDRP 2014-2015).

For a secondary aim, an examination of how the DLS could fit in 2 potential diabetic retinopathy screening models was performed: a fully-automated model for communities with no existing screening programs, vs a semiautomated model in which referable cases from the DLS have a secondary assessment by human graders—a method currently used in some communities and countries (eg, United States, United Kingdom, and Singapore) (eFigure 2 in the Supplement).^32-35 For this analysis, in the fully-automated model, eyes were considered referable if any one of the 3 conditions (referable diabetic retinopathy, referable possible glaucoma, or referable AMD) were present. In the semiautomated model, eyes classified as referable by the DLS would undergo a secondary assessment by trained professional graders to reclassify eyes if necessary. For semiautomated models, evaluation was made of the proportion of images requiring secondary assessment when presetting the DLS sensitivity threshold at 90%, 95%, and 99% in detection of referable status.

Cluster-bootstrap, biased-corrected, asymptotic 2-sided 95% CIs adjusted for clustering by patients were calculated and presented for proportions (sensitivity, specificity) and AUC, respectively. In a few exceptional cases with estimate of sensitivity at the boundary of 100%, the exact Clopper-Pearson method was used instead to obtain CI estimates.³⁶

All hypotheses tested were 2-sided, and a P value of less than .05 was considered statistically significant. No adjustment for multiple comparisons was made because the study was restricted to a small number of planned comparisons. All analyses were performed using Stata version 14 (StataCorp).

From a total of 494 661 retinal images, the DLS was trained for detection of referable diabetic retinopathy (using 76 370 images), referable possible glaucoma (using 125 189 images), and referable AMD (using 72 610 images); performance of the DLS was evaluated using 112 648 images for detection of referable diabetic retinopathy, 71 896 images for referable possible glaucoma, and 35 948 images for referable AMD. All images were assembled between January 2016 and March 2017 (Table 1), the DLS training was completed in May 2016, and validation was completed in May 2017. Among 76 370 images in the training dataset, 11.7% demonstrated any diabetic retinopathy, 5.3% referable diabetic retinopathy, and 1.5% vision-threatening diabetic retinopathy. In the primary validation dataset, estimates were 8.0% for having any diabetic retinopathy, 3.0% for referable diabetic retinopathy, and 0.6% for vision-threatening diabetic retinopathy (n = 71 896 images). In the 10 external validation datasets, estimates were 35.3% for any diabetic retinopathy, 15.4% for referable diabetic retinopathy, and 3.4% for vision-threatening diabetic retinopathy (n = 40 752 images; Table 2). For possible glaucoma, 2630 images (1907 eyes) were considered referable; for AMD, 2900 images (1017 eyes) were considered referable (eTable 1 in the Supplement).

The overall patients demographics, diabetes history, and systemic risk factors of the training and validation datasets are listed in Table 3 (SIDRP 2010-2013 and SIDRP 2014-2015, primary validation set) and eTable 2 in the Supplement (10 external validation datasets for referable diabetic retinopathy and training datasets for referable possible glaucoma and referable AMD).

The diagnostic performance of the DLS as compared with trained professional graders, both with reference to the retinal specialist standard using this primary validation dataset, is shown in Table 4. The AUC of the DLS was 0.936 for referable diabetic retinopathy and 0.958 for vision-threatening diabetic retinopathy (Figure 1). Sensitivity of the DLS in detecting referable diabetic retinopathy was comparable with that of trained graders (90.5% vs 91.1%; P = .68), although the graders had higher specificity (91.6% vs 99.3%; P < .001) (Table 4; Figure 1). For vision-threatening diabetic retinopathy, the DLS had higher sensitivity compared with trained graders (100% vs 88.5%; P < .001), but lower specificity (91.1% vs 99.6%; P < .001). Among eyes with referable diabetic retinopathy, the sensitivity of diabetic macular edema was 92.1% for the DLS and 98.2% for professional graders.

Five subsidiary analyses were performed. First, the DLS showed similar diagnostic performance in 8589 unique patients of SIDRP 2014-2015 (with no overlap with training set) as in the primary analysis (eTable 3 in the Supplement). Second, in a subset of 97.4% eyes (n = 35 055) with excellent retinal image quality (no media opacity), the AUC of the DLS for referable diabetic retinopathy increased to 0.949 (95% CI, 0.940-0.957); for vision-threatening diabetic retinopathy, it increased to 0.970 (0.968-0.973). Third, the DLS showed comparable performance in different subgroups of patients stratified by age, sex, and glycemic control (Figure 2).

Fourth, the DLS showed clinically acceptable performance (sensitivity ≥90%) for referable diabetic retinopathy with respect to multiethnic populations of different communities, clinics, and settings (Table 5). Among the 10 external validation datasets, the AUC of referable diabetic retinopathy ranged from 0.889 to 0.983. The DLS showed clinically acceptable AUCs of greater than 0.90 for different cameras (eg, FundusVue, Canon, Topcon, and Carl Zeiss). Most datasets (except for Singapore Chinese, Malay, and Indian patients) had more than 80% concordance between the DLS and trained professional graders, with sensitivity of more than 91% in the eyes classified as referable by retinal specialists, general ophthalmologists, trained graders, or optometrists (Table 5).

Fifth, for referable possible glaucoma, the AUC of the DLS was 0.942 (95% CI, 0.929-0.954), sensitivity was 96.4% (95% CI, 81.7%-99.9%), and specificity was 87.2% (86.8%-87.5%); for referable AMD, the AUC was 0.931 (95% CI, 0.928-0.935), sensitivity was 93.2% (95% CI, 91.1%-99.8%) and specificity was 88.7% (95% CI, 88.3%-89.0%) (Figure 3).

For the secondary aim, we evaluated the performance of the DLS in 2 diabetic retinopathy screening models (eFigure 2 in the Supplement): the fully- automated model had sensitivity of 93.0% (95% CI, 91.5%-94.3%) and specificity of 77.5% (95% CI, 77.0%-77.9%) to detect overall referable cases (referable diabetic retinopathy, possible glaucoma, or AMD), while the semiautomated model (DLS followed by graders) had sensitivity of 91.3% (95% CI, 89.7%-92.8%) and specificity of 99.5% (95% CI, 99.5%-99.6%) to detect overall referable status. The performance of different semiautomated models with a preset sensitivity threshold of 90%, 95%, and 99% are shown in eTable 4 in the Supplement.

In this evaluation of nearly half a million of images from multiethnic community, population-based and clinical datasets, the DLS had high sensitivity and specificity for identifying referable diabetic retinopathy and vision-threatening diabetic retinopathy, as well as for identifying related eye diseases, including referable possible glaucoma and referable age-related macular degeneration. The performance of the DLS was comparable and clinically acceptable to the current model based on assessment of retinal images by trained professional graders and showed consistency in 10 external validation datasets of multiple ethnicities and settings, using diverse reference standards in assessment of diabetic retinopathy by professional graders, optometrists, or retinal specialists. This study also examined how the DLS could be deployed in 2 common diabetic retinopathy screening models: a “fully-automated” screening model that showed clinically acceptable performance to detect all 3 conditions, useful in communities without any existing diabetic retinopathy screening programs; and a “semi-automated” model in which diabetic retinopathy screening programs using trained professional graders already exist, and the DLS could be incorporated.

There have been previous studies of automated software for diabetic retinopathy screening^14,37,38; most recent ones used a DLS.^10-12 Gulshan et al¹⁰ reported a DLS with high sensitivity and specificity (>90%) and an AUC of 0.99 for referable diabetic retinopathy using approximately 10 000 images retrieved from 2 publicly available databases (EyePAC-1 and Messidor-2). Similarly, Gargeya and Leng¹¹ showed optimal DLS diagnostic performance in detecting any diabetic retinopathy using 2 other public databases (Messidor-2 and E-Ophtha). To facilitate translation, it is important to develop and test the DLS in clinical scenarios using diverse retinal images of varying quality from different camera types and in representative diabetic retinopathy screening populations.¹³ The current study therefore substantially added to other current studies.

First, the DLS was trained to also detect other related eye diseases including referable possible glaucoma and referable AMD in addition to diabetic retinopathy. Second, the training and validation data sets were substantially larger (nearly 500 000 images) and included images from patients of diverse racial and ethnic groups (ie, darker fundus pigmentation in African American and Indian individuals to lighter fundus in white individuals). The DLS showed consistent diagnostic performance across images of varying quality and different camera types, and across patients with varying systemic glycemic control level.

Third, primary validation of the DLS was conducted in an ongoing diabetic retinopathy screening program in which there were poorer quality images, including ungradable ones. This results in somewhat lower performance of the DLS (AUC, 0.936) than the system by Gulshan et al that used higher-quality images.¹⁰ Fourth, this study also had fewer cases of severe disease (eg, vision-threatening diabetic retinopathy, referable possible glaucoma, and referable AMD), but this is more representative of populations for routine diabetic retinopathy screening.³⁹

To ensure no degradation in health outcomes, a threshold was set to ensure false-negative rates were no worse than human assessment by trained professional graders. Although the results suggest that professional nonmedical graders may outperform the DLS (with high specificity of 99% for referable diabetic retinopathy and vision-threatening diabetic retinopathy), given the very low marginal cost of the DLS, the low prevalence rate of the conditions in the target screening population (<5%), and equality in health outcomes, the DLS could be used with a semiautomated model in which first-line screening with the DLS is followed by human assessment for patients who test positive. This will allow increasing screening episodes with lower cost and no degradation in health outcomes.

This study has several limitations. First, the training set was not developed entirely based on the retinal specialists’ grading for all images. Although the reference standard in the primary validation dataset used grading by a retinal specialist, reference standards for the external datasets were based on varying assessment by retinal specialists, general ophthalmologists, trained graders, or optometrists. The performance of the DLS may potentially be further improved if all images in the training and validation data sets had criterion standard references evaluated by the retinal specialists. Nevertheless, the diagnostic performance of the DLS remained clinically acceptable and highly reproducible in both the primary validation data set and in the 10 external datasets in which the reference standards vary depending on whether the images were evaluated by retinal specialists (African American, Mexican, Hong Kong Chinese), general ophthalmologists (Beijing Chinese), optometrists (Hong Kong Chinese) or professional nonmedical graders (the remaining datasets) from the different countries (Table 5).

Second, the DLS uses multiple levels of representation to analyze each retinal image without showing the actual diabetic retinopathy lesions (eg, microaneurysms, retinal hemorrhages). These data points can possibly be the shape or contour of the optic disc or tortuosity or caliber of the retinal vessels. Such black-box issues may have an effect on physicians’ acceptance for clinical use.¹³

Third, identification of diabetic macular edema from fundus photographs may not identify all cases appropriately without clinical examination and optical coherence tomography.

In this evaluation of retinal images from multiethnic cohorts of patients with diabetes, the DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases. Further research is necessary to evaluate the applicability of the DLS in health care settings and the utility of the DLS to improve vision outcomes.

Corresponding Author: Tien Yin Wong, MD, PhD, Singapore National Eye Center, 11 Third Hospital Ave, Singapore 168751 (wong.tien.yin@snec.com.sg).

Accepted for Publication: October 31, 2017.

Author Contributions: Drs D. S. Ting and T. Y. Wong had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs He, Cheng, G. C. M. Cheung, Tin, Hsu, Lee, and T. Y. Wong contributed equally.

Concept and design: D. S. Ting, C. Y-L. Cheung, Lim, G. S. W. Tan, Garcia-Franco, Finkelstein, Lamoureux, Sivaprasad, Varma, Jonas, He, Cheng, Hsu, M. Lee, T. Y. Wong.

Acquisition, analysis, or interpretation of data: D. S. Ting, C. Y-L. Cheung, Lim, G. S. W. Tan, Nguyen, Gan, Hamzah, Yeo, S. Y. Lee, E. Y. M. Wong, Sabanayagam, Baskaran, Ibrahim, N. C. Tan, Finkelstein, Lamoureux, I. Y. Wong, Bressler, Varma, Jonas, He, C. Y-L. Cheung, Tin, T. Y. Wong.

Drafting of the manuscript: D. S. Ting, Gan, Hamzah, Lamoureux, T. Y. Wong.

Critical revision of the manuscript for important intellectual content: D. S. Ting, C. Y. Cheung, Lim, G. S. W. Tan, Nguyen, Garcia-Franco, Yeo, S. Y. Lee, E. Y. M. Wong, Sabanayagam, Baskaran, Ibrahim, N. C. Tan, Finkelstein, I. Y. Wong, Bressler, Sivaprasad, Varma, Jonas, He, Cheng, C. Y. Cheung, Tin, Hsu, M. L. Lee, T. Y. Wong.

Statistical analysis: D. S. Ting, Lim, Nguyen, Gan, Sabanayagam, Ibrahim, Lamoureux, I. Y. Wong, Tin, T. Y. Wong.

Obtained funding: D. S. Ting, C. Y. Cheung, Tin, T. Y. Wong.

Administrative, technical, or material support: D. S. Ting, C. Y. Cheung, Lim, G. S. W. Tan, Hamzah, Garcia-Franco, Baskaran, N. C. Tan, Sivaprasad, Jonas, Cheng, Tin, T. Y. Wong.

Supervision: D. S. Ting, C. Y. Cheung, G. S. W. Tan, Yeo, S. Lee, Lamoureux, Varma, He, C. Y. Cheung, Hsu, M. L. Lee, T. Y. Wong.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs D. S. Ting, Lim, M. L. Lee, Hsu, and T. Y. Wong reported that they are coinventors on a patent for the deep learning system used in this study; potential conflicts of interests are managed according to institutional policies of the Singapore Health System (SingHealth) and the National University of Singapore (NUS). Dr Bressler reported holding a patent on a system and method for automated detection of age-related macular degeneration and other retinal abnormalities, unrelated to the deep learning system in this paper. Dr He reported holding a patent on an automated image analysis system and retinal camera for retinal diseases, unrelated to the deep learning system in this article. No other disclosures were reported.

Funding/Support: This project received funding from National Medical Research Council (NMRC), Ministry of Health (MOH), Singapore National Health Innovation Center (NHIC), Innovation to Develop Grant (NHIC-I2D-1409022); SingHealth Foundation Research Grant (SHF/FG648S/2015), and the Tanoto Foundation; unrestricted donations to the Retina Division, Johns Hopkins University School of Medicine. For the Singapore Epidemiology of Eye Diseases (SEED) study, we received funding from NMRC, MOH (grants 0796/2003, IRG07nov013, IRG09nov014, STaR/0003/2008 and STaR/2013; CG/SERI/2010) and Biomedical Research Council (grants 08/1/35/19/550 and 09/1/35/19/616). The Singapore Diabetic Retinopathy Program received funding from the MOH, Singapore (grants AIC/RPDD/SIDRP/SERI/FY2013/0018 and AIC/HPD/FY2016/0912).

Role of the Funders/Sponsors: The NMRC (Singapore), MOH (Singapore), NHIC (Singapore), and Tanoto Foundation, and Johns Hopkins School of Medicine had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.