Evaluation of a Remote Diagnosis Imaging Model vs Dilated Eye Examination in Referable Macular Degeneration

Key Points

Question  Can nonmydriatic imaging equipment, located at the point of service, help in the identification of referable macular degeneration?

Finding  In this study of 159 patients in facilities with high rates of diabetic retinopathy and age-related macular degeneration, a retinal imaging device appeared to be equivalent to a standard examination in detecting referable macular degeneration within a setting with relatively high disease prevalence.

Meaning  A remote diagnosis imaging model may facilitate the identification of patients with referable retinal disease; this capability may assist in delivering timely treatment and warrant future research.

Importance  In improving clinical outcomes, developing a sustainable, transformative care delivery model is important for accessible, efficient, low-cost, high-quality community-based imaging and diagnosis of retinal diseases.

Objective  To test the feasibility and accuracy of the remote diagnosis imaging model as a clinical screening tool to facilitate the identification of referable macular degeneration.

Design, Setting, and Participants  A nonrandomized study of 159 patients was conducted in sites with a relatively high disease prevalence (Duke University Health System endocrinology clinic and 2 Duke University Health System assisted living centers in North Carolina). All patients underwent remote diagnosis imaging, defined as color fundus photography (CFP) and optical coherence tomography (OCT) of nondilated pupils, acquired by nonexpert imagers using a retinal imaging device located at the point of service. The criterion standard examination was defined as a traditional dilated eye examination performed by retinal specialists. Deidentified remote diagnosis images were graded for interpretability and presence of referable macular degeneration, defined as any condition requiring a retinal specialist attention. Data analysis was performed from November 20, 2015, to February 10, 2019.

Main Outcomes and Measures  Primary outcome was feasibility of the remote retinal imaging. Secondary outcomes were operational characteristics and diagnostic and referral accuracy.

Results  Of the 159 patients included in the study, the mean (SD) age of enrolled participants was 65 (17) years, with a female to male ratio of 1.3 to 1. Most patients were white (111 [69.8%]), 44 were black patients (27.7%), approximately 1% were Asian patients and Hispanic patients, and 2 patients declined to disclose their race/ethnicity. Thirty-five eyes (22.0%) were determined to require referral to the retinal specialist by criterion standard examination. Remote diagnosis image interpretability was better when OCT was used compared with CFP (241 [96.4%] vs 164 [65.6%]). Remote diagnosis had high diagnostic accuracy in identifying referable macular degeneration: OCT and CFP both had 94% sensitivity (95% CI, 84%-98%), and OCT had specificity higher than for CFP (93% [95% CI, 87%-96% ] vs 63% [95% CI, 53%-71%]). Substantial agreement was found between the criterion standard and OCT (κ = 0.83; 95% CI, 0.76-0.91; P < .001) and between the criterion standard and CFP (κ = 0.76; 95% CI, 0.64-0.87; P < .001). The nonvalidated patient satisfaction survey revealed that 122 participants (76.7%; mean score, 4.16; 95% CI, 3.98-4.35) preferred remote imaging over the standard care examination.

Conclusions and Relevance  Remote diagnosis imaging and a standard examination by a retinal specialist appeared equivalent in identifying referable macular degeneration in patients with high disease prevalence; these results may assist in delivering timely treatment and seem to warrant future research into additional metrics.

Diabetic retinopathy and age-related macular degeneration (AMD) are leading causes of vision loss in the United States, affecting more than 20 million people, of whom approximately 500 000 are considered legally blind.^1,2 Although effective treatments supported by large clinical trials exist, many patients lose their vision because of late diagnosis and delayed treatment.^3-8 Screening eye examinations are becoming increasingly mandated in the United States, owing to the Affordable Care Act.⁸ Traditional screening for eye disease typically requires examinations by eye care specialists through dilated pupils. However, fewer than 50% of patients at risk receive such screening. Teleophthalmic examination is associated with high effectiveness and is an alternative to live examination, yet several barriers, especially the cost and lack of universally accepted protocol, prevent this method from being widely accepted in the United States.^9-11

We distinguish the traditional teleophthalmic method from the remote diagnosis model in 4 key ways. The traditional method takes color fundus photography (CFP) of dilated pupils and is performed by expert photographers with expensive equipment in a dedicated bricks-and-mortar imaging center or a mobile van. Remote diagnosis is the use of CFP and optical coherence tomography (OCT) on nondilated pupils and is performed by nonexpert imagers using less expensive equipment permanently located at the point of service (eg, primary care clinic). This approach potentially offers patient convenience and a high rate of patient capture. Color fundus photography is the current standard for teleophthalmic screening and monitoring of retinal disease. However, CFP, especially through nondilated pupils, has high rates of uninterpretable images (approximately 20% to 30%).^12-16 In contrast, OCT images typically have low rates of ungradable images.¹⁷ In the clinic, OCT is the most specific for the diagnosis of vision-threatening diabetic macular edema and other vision-threatening macular lesions. Each of the 2 technologies has its own shortcomings, but combining them has the potential of improving clinical and cost-effectiveness.

The purpose of this study was to test the feasibility and diagnostic accuracy of a retinal imaging system of a combined CFP and OCT located at the point of service in an environment with relatively high prevalence of diabetic retinopathy or AMD.

This prospective, nonrandomized study enrolled 159 patients (318 eyes) in North Carolina communities with relatively high disease rate: Duke University Health System endocrinology clinic with a high prevalence of diabetic retinopathy, and 2 Duke University Health System assisted living centers with a high prevalence of AMD. All eligible patients in these sites were invited to participate by designated site nurse practitioners. Study participants had to be able to sit for an imaging device for image acquisition and to be capable of giving their own consent. Pregnant or nursing women were excluded from the study. This study received institutional review broad approval from Duke University Health System and complied with the Health Insurance Probability and Accountability Act of 1996. Written informed consent was obtained from all participants.

Remote diagnosis imaging of nondilated pupils was done at the point of service by trained but nonexpert imagers using US Food and Drug Administration–approved noncontact, portable retinal imaging device (iFusion; Optovue) that incorporated CFP and OCT. On a separate day, within 90 days of remote diagnosis imaging, a criterion standard examination, defined as a dilated eye examination, was performed by retinal specialists at Duke University Health System, with an ancillary diagnostic testing (OCT and CFP) done by expert photographers under the Duke standard care examination. The study design is illustrated in Figure 1. Categories of the prespecified outcomes are described in the Box.

Box Section Ref ID

The categories included macular degeneration referable to a retinal specialist, which was the presence of macular lesions known to require urgent intervention or frequent follow-up by a retinal specialist. Incidental findings were retinal or other ocular lesions requiring a nonurgent referral to a general ophthalmologist or optometrist. The remaining categories included no pathological condition and uninterpretable images. Each eye served as its own control and underwent remote screening and a criterion standard examination.

Two of us (M.H. and P.N.) initially graded deidentified images and made a case presentation to a panel of medical retina faculty. The final grading of deidentified images was done by consensus. Color fundus photography and OCT images were considered uninterpretable if no clear identification of macula was available (poor positioning) or extremely poor exposure owing to media opacity. The CFP images were assessed for diabetic retinopathy severity using the standard ETDRS scale and for AMD using the Age-Related Eye Disease Study 9-Step Severity Scale.^18,19 The OCT images were graded according to the presence of the following: significant vitreal debris, cystic intraretinal fluid, subretinal fluid, hyperreflective foci, pigment epithelial detachment, drusen, retinal pigment epithelium or retinal atrophy, macular holes, epiretinal membrane, and any other abnormal macular findings.

According to present retinal findings and prespecified outcome categories (Box), the possible outcomes were macular degeneration referable to a retinal specialist, incidental findings, no pathological condition, and uninterpretable images. Any ungradable images were grouped with referable disease, as these eyes would have failed image screening. For assessment of accuracy of remote image grading, the reference standard was the criterion standard clinical examination findings from the same patient. The sensitivity, specificity, and positive and negative predictive values of each outcome were calculated against the reference standard of the dilated criterion standard examination.

Medical students or nonophthalmologic health care professionals (eg, nurses, nurses’ aides, and technicians) were identified as imagers for this study. Before collecting images, each imager read the instruction manual for the imaging device. Next, imagers received practical instructions in the use of the device to help them develop their proficiency with acquiring both CFP and OCT images. After the initial training, imagers were observed and coached during their first imaging sessions with at least 5 study participants. Periodic review of remote images was conducted to direct the retraining of imagers in the event that poor image quality related to imaging technique was suspected.

Study participants were surveyed about their experience with imaging in the remote imaging clinic. Survey topics included previous experiences with eye care, perceptions about the study imaging and other common office procedures, and preferences regarding remote diagnosis imaging to screen for eye disease over complete eye examination. The nonvalidated questionnaire used a 5-point scale, with 1 indicating preference against remote diagnosis; 3, no preference; and 5, preference for remote diagnosis. The questionnaire is reproduced in the eAppendix in the Supplement.

For sample size estimation, we conceptualized the study as a noninferiority study. We assumed that the true sensitivity in the research clinics was 80%, that the true sensitivity in the general population clinics was 70%, and that the noninferiority margin was specified to be 5% (ie, confidently identify differences of 5% as statistically significant). In this case, a sample size of 282 was required. Recruiting patients was fairly straightforward; the risk was minimal, and the recruited individuals received a free eye examination.

Statistical analysis was conducted with SPSS statistics software, version 25 (IBM), from November 20, 2015, to February 10, 2019 (second resubmission). Cohen κ test was used to assess for an agreement between different imaging modalities and the reference standard. Following the system of Landis and Koch,²⁰ we defined κ values as 0 to 0.20 indicating slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and more than 0.81, almost perfect agreement. Statistically significant result was considered if P ≤ .05, which was calculated automatically by SPSS software and is presented for κ agreement only.

We created a MATLAB (The MathWorks, Inc) function to facilitate the comparison between the number of patients with interpretable images and referable macular degeneration and the criterion standard. The procedure was used to automatically process the medical results by counting, and the results were used for operational characteristics calculations.

Of the 159 patients included in the study, 34 (21.4%) were lost to follow-up (1 died, 31 did not complete the criterion standard clinical examination in the designated time frame, and 2 lacked remote images). The mean (SD) age of enrolled participants was 65 (17) years, with a female to male ratio of 1.3 to 1. Most patients were white (111 [69.8%]), 44 were black patients (27.7%), approximately 1% were Asian patients and Hispanic patients, and 2 patients declined to disclose their race/ethnicity. Thirty-five eyes (22.0%) were determined to require a referral for vision-threatening pathological condition according to the examination by the retinal specialist.

The deidentified images were graded according to predetermined criteria. In total, 241 (96.4%) of the OCT images were deemed to be interpretable, but only 164 (65.6%) of the CFP images. The main reasons for poor interpretability of the CFP were low-contrast images (70 [81.4%]) owing to anterior segment opacity and failure to capture the macula within the image (16 [18.6%]). In the OCT group, 5 images (55.6%) were noninterpretable because of low contrast and 4 (44.4%) had failure to capture the fovea within the scan. In the CFP subgroup, of the uninterpretable images, 31.4% (27 of 86 eyes) had a referable pathological condition; in the OCT subgroup, 11.1% (1 of 9 eyes).

Outcome interpretations for CFP, OCT, or combined CFP and OCT were categorized into 4 diagnostic groups: macular degeneration referable to a retina specialist, incidental findings, no pathological condition, and uninterpretable images. These imaging results were compared with the criterion standard results. Remote diagnosis imaging had a high success rate in identifying macular degeneration referable to a retina specialist (60 eyes for OCT, 34 eyes for CFP, and 61 eyes for CFP and OCT combined) (Figure 2 and Figure 3).

Sensitivity and specificity were used to assess for statistical measures of remote diagnosis performance (Figure 4). Results of remote diagnosis and criterion standard grading were processed in MATLAB. For the purpose of analysis, uninterpretable images were classified as requiring referral.

Unprocessed data are presented in Figure 2 and Figure 3 and were used to calculate operational characteristics (Figure 4). Sensitivity was very high at 94% (95% CI, 84%-98%) for both modalities, whereas specificity was higher with OCT (93%; 95% CI, 87%-96%) than with CFP (63%; 95% CI, 53%-71%). The CFP technique did less well because of the high rate of uninterpretable images. In addition, CFP had a lower positive predictive value compared with OCT (56% vs 87%), whereas with combined OCT and CFP, the positive predictive value was improved. Both technologies had high negative predictive value (95% and 97%; Figure 4). Substantial agreement was observed in referable vs nonreferable macular degeneration between the criterion standard and OCT (κ = 0.83; 95% CI, 0.76-0.91; P < .001) and between the criterion standard and CFP (κ = 0.76; 95% CI, 0.64-0.87; P < .001).

To assess the feasibility of the remote diagnosis model, in addition to measuring the overall rate of interpretable and uninterpretable images, we assessed patient acceptance of the remote retinal imaging experience. Results from the survey (scale: 1-5, with 1 favoring criterion standard and 5 favoring remote imaging) completed after participants underwent remote imaging revealed that most patients preferred remote imaging (122 [76.7%]; mean score, 4.16; 95% CI, 3.98-4.35) over the criterion standard care examination (15 [9.5%]), whereas the rest of the participants expressed no preference (22 [13.8%], selecting 3).

We found that 26 screened patients (10.4%) were identified with nonproliferative diabetic retinopathy level 43 or worse, with or without diabetic macular edema; 37 (14.8%) had intermediate or higher-level AMD, including exudative (wet) AMD (eTable in the Supplement). All of these pathologic conditions required further intervention by a retinal specialist.

The clinical effectiveness and cost-effectiveness associated with screening for diabetic retinopathy are well established. The American Diabetes Association guidelines recommend patients with diabetes to begin diabetic eye screening within 5 years of diagnosis for those with type 1 and at the time of diagnosis, along with yearly screenings, for those with type 2.²¹ Fewer than 50% of patients with diabetes in the United States follow these guidelines, and even fewer (10% to 20%) among underserved populations.^22-24

The cost-effectiveness associated with screening for AMD still remains unclear and warrants further investigation. Ophthalmic screening for AMD in adults is highly associated with reductions in the number of patients with blindness, but the overall cost-effectiveness has not been demonstrated.^25,26 Several studies suggest that, if carried out simultaneously with diabetic retinopathy screening, ophthalmic screening, besides being associated with clinical effectiveness, might also be economical.²⁷

Over the past decade, teleophthalmic method has gained recognition worldwide, especially in developing countries and underserved populations, for its effective eye care outcomes.^9,10 However, the required logistical complexity, lack of protocol consensus, and required photographer skill and grader expertise increase this modality’s expense, thereby limiting its widespread adoption in the United States.

Advances in retinal imaging devices and computer image analysis have the potential to improve screening for retinal disease. New imaging devices with simplified interfaces can be used in nonophthalmic clinics and operated by staff with limited training. These features reduce the expense of novel imaging devices and allow for their wider use. We distinguish the teleophthalmic method from the less expensive remote diagnosis model. In remote diagnosis, images are acquired by nonexpert imagers, through nondilated pupils using imaging devices located at the point of service.

This study evaluated the feasibility of remote retinal imaging in communities with relatively high disease prevalence. The primary end point was feasibility. Consistent with previously published data, the rate of ungradable images in the current study was higher for the CFP group than the OCT group. As reported by Silva et al,¹⁶ increasing age was associated with substantial increase in ungradable images (mean [SD] age, 65 [17] years). This finding might be associated with higher incidence of cataracts or smaller pupils or some other reasons and may warrant further investigation. Findings from a nonvalidated satisfaction survey suggested that most patients preferred this examination modality over an in-clinic retinal examination. To assess for referral accuracy, this study’s secondary end point was to identify macular degeneration referable to a retinal specialist. As demonstrated, remote imaging had success in identifying this pathologic condition. In addition, combining OCT and CFP was associated with improvements in operational outcomes.

These results suggest that a similar grading process could be a feasible and efficient approach to identifying participants at risk. Important questions such as evaluation of completed referrals, cost-effectiveness of quality of care and prevention of visual impairment by allowing for early diagnosis, cost savings from reduced travel, cost of implementing the program, and long-term visual outcomes should be part of future research.

With the growing aging population and prevalence of diabetes, the prevalence and the burden of AMD and diabetic retinopathy likely will continue to increase. Thus, recognizing patients at risk is important, along with offering timely treatment to improve visual outcomes and reduce cost. Because most of the current screening activity presumably occurs during in-clinic eye examinations, clinician effort and clinic time spent assessing patients with or without disease who do not require treatment might be better spent on other activities. With improved referral accuracy, more efficient use of ophthalmic clinical care and resources might be achieved.

This study has limitations. It was performed in relatively homogeneous populations (patients with diabetes and older adults) with high disease prevalence. High disease prevalence likely was associated with an increase in sensitivity, thus more objective information would be obtained in the community-based setting with lower disease prevalence. In addition, a high number of uninterpretable images accounted for referable macular degeneration that altered specificity and positive predictive value. These limitations, along with visual function correlation and assessment of cost-effectiveness, should be addressed in future studies.

We evaluated the feasibility of remote retinal imaging as a diagnostic tool for identifying referable macular degeneration. Good operational characteristics found in this study suggest the feasibility of using this model to screen and refer patients to a retinal specialist. We also confirmed that the combination of CFP and OCT in this setting was noninferior in macular degeneration screening compared with either OCT or CFP alone. Evaluation of additional metrics needs further attention.

Accepted for Publication: March 13, 2019.

Corresponding Author: Majda Hadziahmetovic, MD, Department of Ophthalmology, Duke University Medical Center, 2351 Erwin Rd, Durham, NC 27710 (majda.hadziahmetovic@duke.edu).

Published Online: May 16, 2019. doi:10.1001/jamaophthalmol.2019.1203

Author Contributions: Drs Hadziahmetovic and Cousins had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: Hadziahmetovic, Nicholas, Jindal, Mettu.

Drafting of the manuscript: Hadziahmetovic, Jindal, Cousins.

Critical revision of the manuscript for important intellectual content: Nicholas, Mettu, Cousins, Hadziahmetovic.

Statistical analysis: Hadziahmetovic, Jindal.

Obtained funding: Cousins.

Administrative, technical, or material support: Jindal, Mettu, Cousins.

Supervision: Mettu, Cousins.

Conflict of Interest Disclosures: Dr Nicholas reported grants from The Duke Endowment during the conduct of the study and a patent to Farsiu S, Chiu SJ, Toth CA, Izatt JA, Li XT, Nicholas PT. Segmentation and identification of layered structures in images; Duke University, assignee. US patent 20110182517. Published July 28, 2011. Dr Mettu reported grants from The Duke Endowment during the conduct of the study and personal fees from NotalVision outside of the submitted work. Dr. Cousins reported grants from The Duke Endowment during the conduct of the study and personal fees from NotalVision outside of the submitted work; and consulting fees from Stealth, PanOptica, Merck Pharmaceuticals, Clearside Biomedical, outside of the submitted work. No other disclosures were reported.

Funding/Support: This study was supported by grant TDE 64656-SP from the Duke Endowment.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge the contributions of the Duke Endocrinology Department as well as Croasdaile Village and Forest at Duke assisted living centers, along with all of their physicians, nurses, and staff who helped with the study. We also thank the patients who were willing to participate in the study. We thank Optovue (Fremont, CA) for providing the iFusion units used for this study. Special acknowledgment to Malina Sexton, BS, Duke Eye Center, for all her help with data collection; she received compensation for her contribution.