Can artificial intelligence systems trained on normal data recognize anomalies in retinal images?
In this cross-sectional study of 88 692 high-resolution retinal images of 44 346 individuals with varying severity of diabetic retinopathy, novel anomaly detectors were developed and tested to a surrogate problem using EyePACS data, wherein detectors were trained only on normal retinas (nonreferable diabetic retinopathy) and subsequently tasked to detect abnormality (referable diabetic retinopathy). The detectors had a relatively high area under the receiver operating characteristic curve.
This surrogate example suggests anomaly detectors, not trained with diseased retina, can detect diabetic retinopathy; these detectors might play a role by training on normal retina to detect retinal anomalies or novel retinal disease presentations.
Anomaly detectors could be pursued for retinal diagnoses based on artificial intelligence systems that may not have access to training examples for all retinal diseases in all phenotypic presentations. Possible applications could include screening of population for any retinal disease rather than a specific disease such as diabetic retinopathy, detection of novel retinal diseases or novel presentations of common retinal diseases, and detection of rare diseases with little or no data available for training.
To study the application of anomaly detection to retinal diseases.
Design, Setting, and Participants
High-resolution retinal images from the publicly available EyePACS data set with fundus images with a corresponding label ranging from 0 to 4 for representing different severities of diabetic retinopathy. Sixteen variants of anomaly detectors were designed. For evaluation, a surrogate problem was constructed, using diabetic retinopathy images, in which only retinas with nonreferable diabetic retinopathy, ie, no diabetic macular edema, and no diabetic retinopathy or mild to moderate nonproliferative diabetic retinopathy were used for training an artificial intelligence system, but both nonreferable and referable diabetic retinopathy (including diabetic macular edema or proliferative diabetic retinopathy) were used to test the system for detecting retinal disease.
Main Outcomes and Measures
Anomaly detectors were evaluated by commonly accepted performance metrics, including area under the receiver operating characteristic curve, F1 score, and accuracy.
A total of 88 692 high-resolution retinal images of 44 346 individuals with varying severity of diabetic retinopathy were analyzed. The best performing across all anomaly detectors had an area under the receiver operating characteristic of 0.808 (95% CI, 0.789-0.827) and was obtained using an embedding method that involved a self-supervised network.
Conclusions and Relevance
This study suggests when abnormal (diseased) data, ie, referable diabetic retinopathy in this study, were not available for training of retinal diagnostic systems wherein only nonreferable diabetic retinopathy was used for training, anomaly detection techniques were useful in identifying images with and without referable diabetic retinopathy. This suggests that anomaly detectors may be used to detect retinal diseases in more generalized settings and potentially could play a role in screening of populations for retinal diseases or identifying novel diseases and phenotyping or detecting unusual presentations of common retinal diseases.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Burlina P, Paul W, Liu TYA, Bressler NM. Detecting Anomalies in Retinal Diseases Using Generative, Discriminative, and Self-supervised Deep Learning. JAMA Ophthalmol. Published online December 30, 2021. doi:10.1001/jamaophthalmol.2021.5557
Coronavirus Resource Center
Customize your JAMA Network experience by selecting one or more topics from the list below.