In the near future, automated systems using deep learning may be used in screening and computer-aided diagnosis in clinical ophthalmology. Recently, researchers have reported subtle adversarial attacks, such as artificial noise created to attack deep learning models, resulting in critical misclassifications.1 Previous dermatology literature has demonstrated that visually imperceptible noise can change a diagnosis from benign to malignant. Because images could be intentionally programmed to force the model to make a mistake, safety concerns are raised for clinical practice. However, while such adversarial attacks have been reported, to our knowledge, they have not been investigated extensively within ophthalmology. This study aims to verify that adversarial attacks can confuse deep learning systems based on imaging domains, such as fundus photography (FP), ultrawide-field FP (UWF), and optical coherence tomography (OCT).
This study was based on a publicly accessible retinal image database including 35 126 FP images from the Kaggle EyePacs,2 8217 UWF images from the TOP project,3 and 38 208 OCT images from a study by Kermany et al.4 To build binary classifier models to detect diabetic retinopathy or diabetic macular edema, images with other pathologic lesions and ungradable images were excluded from this study. The researchers used open web-based and deidentified data, and this study was determined to be exempt from ethical review according to the Korea National Institute for Bioethics Policy. All procedures were performed in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments.
The downloaded InceptionV3 deep learning model (Google) was pretrained on the ImageNet database, and the weights of the pretrained networks were fine-tuned for each imaging domain. The input image size was set to a pixel resolution of 224 × 224 by tuning the input tensor. One-tenth of the data set was randomly selected as a validation set. The fast gradient sign method (FGSM) for each imaging domain was used to generate noise for adversarial attack using InceptionV3.5 This is the most popular method of generating adversarial attacks on deep learning using the gradients of the loss function. Google Colab Pro (Google), a cloud service for deep learning research, was used for the experiment. The TensorFlow tutorial page was used to generate FGSM images. All codes are available at https://www.tensorflow.org/tutorials/generative/adversarial_fgsm.
The FGSM generated FP, UWF, and OCT perturbation images, fooling the deep learning model with very small intensities that would be undetectable by humans (Figure). Images with adversarial attacks can lead the model to misclassify both normal and pathologic input images. Adversarial examples generated using InceptionV3 led the same model to critical misclassification, with accuracy decreasing to 13.4% in FP, 5.0% in UWF, and 8.2% in OCT (Table). The adversarial attacks derived from InceptionV3 were transferable to other conventional deep learning methods, including MobileNetV2 and ResNet50, although there were smaller losses of accuracy with these models.
Recently, techniques for adversarial attacks and defenses of deep learning systems have been developed in the artificial intelligence community. Deep learning can produce expert-level diagnoses for diabetic retinopathy; therefore, relatively cheap and fast techniques may replace expensive experts soon. The conventional deep learning models for FP, UWF, and OCT are extremely vulnerable to adversarial attacks, which could be undetectable to humans. These attacks were partially transferable to other deep learning architectures. Because the perturbations were generated using InceptionV3, they were less effective on the MobileNetV2 and ResNet50 models in this study. Our study implies that deep learning may not be the ultimate solution to medical decision-making. If medical decisions are performed automatically by deep learning, adversarial attacks can be used for fraudulent purposes.1 Malicious actors could disrupt medical billing and reimbursement systems used by hospitals and insurance companies. Defensive techniques, including training deep learning with adversarial examples, denoising filters, and generative adversarial networks, might be effective at decreasing the effect of adversarial attacks.6 Our results suggest that the designers and approving agencies of medical deep learning systems should be careful to guard against adversarial attacks in clinical ophthalmology.
Accepted for Publication: June 30, 2020.
Corresponding Author: Tae Keun Yoo, MD, Aerospace Medical Center, Department of Ophthalmology, Republic of Korea Air Force, 635 Danjae-ro, Sangdang-gu, Cheongju, South Korea (email@example.com).
Published Online: October 1, 2020. doi:10.1001/jamaophthalmol.2020.3442
Author Contributions: Dr Yoo had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: All authors.
Administrative, technical, or material support: All authors.
Study supervision: Yoo.
Conflict of Interest Disclosures: None reported.
LA. Reproduction study using public data of: development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. PLoS One
. 2019;14(6):e0217541. doi:10.1371/journal.pone.0217541PubMedGoogle Scholar
et al. Accuracy of ultrawide-field fundus ophthalmoscopy-assisted deep learning for detecting treatment-naïve proliferative diabetic retinopathy. Int Ophthalmol
. 2019;39(10):2153-2159. doi:10.1007/s10792-019-01074-zPubMedGoogle ScholarCrossref
C. Explaining and harnessing adversarial examples. Cornell University Library. December 20, 2014. Accessed April 8, 2020. https://arxiv.org/abs/1412.6572