Implementation and Evaluation of a Large-Scale Teleretinal Diabetic Retinopathy Screening Program in the Los Angeles County Department of Health Services

Key Points

Question  What is the effect of a primary care–based teleretinal diabetic retinopathy (DR) screening program on rates of screening for DR and wait times for screening in a large safety net health care system?

Findings  In this quasi-experimental, pretest-posttest evaluation of a teleretinal DR screening program in the Los Angeles County Department of Health Services, the need for more than 14 000 visits to specialty care professionals was eliminated, annual rates of screening for DR increased by 16.3%, and wait times for screening were reduced by 89.2%.

Meaning  With standardization and oversight, primary care–based teleretinal DR screening programs have the potential to maximize access and efficiency in the safety net, where the need for such programs is most critical.

Importance  Diabetic retinopathy (DR) is the leading cause of blindness in adults of working age in the United States. In the Los Angeles County safety net, a nonvertically integrated system serving underinsured and uninsured patients, the prevalence of DR is approximately 50%, and owing to limited specialty care resources, the average wait times for screening for DR have been 8 months or more.

Objective  To determine whether a primary care–based teleretinal DR screening (TDRS) program reduces wait times for screening and improves timeliness of needed care in the Los Angeles County safety net.

Design, Setting, and Participants  Quasi-experimental, pretest-posttest evaluation of exposure to primary care–based TDRS at 5 of 15 Los Angeles County Department of Health Services safety net clinics from September 1, 2013, to December 31, 2015, with a subgroup analysis of random samples of 600 patients before and after the intervention (1200 total).

Exposure  Primary care clinic–based teleretinal screening for DR.

Main Outcomes and Measures  Annual rates of screening for DR before and after implementation of the TDRS program across the 5 clinics, time to screening for DR in a random sample of patients from these clinics, and a description of the larger framework of program implementation.

Results  Among the 21 222 patients who underwent the screening (12 790 female, 8084 male, and 348 other gender or not specified; mean [SD] age, 57.4 [9.6] years), the median time to screening for DR decreased from 158 days (interquartile range, 68-324 days) before the intervention to 17 days (interquartile range, 8-50 days) after initiation of the program (P < .001). Overall annual screening rates for DR increased from 5942 of 14 633 patients (40.6%) before implementation to 7470 of 13 133 patients (56.9%) after initiation of the program at all 15 targeted clinics (odds ratio, 1.9; 95% CI, 1.3-2.9; P = .002). Of the 21 222 patients who were screened, 14 595 (68.8%) did not require referral to an eye care professional, 4160 (19.6%) were referred for treatment or monitoring of DR, and 2461 (11.6%) were referred for other ophthalmologic conditions.

Conclusions and Relevance  A digital TDRS program was successfully implemented for the largest publicly operated county safety net population in the United States, resulting in the elimination of the need for more than 14 000 visits to specialty care professionals, a 16.3% increase in annual rates of screening for DR, and an 89.2% reduction in wait times for screening. Teleretinal DR screening programs have the potential to maximize access and efficiency in the safety net, where the need for such programs is most critical.

Diabetic retinopathy (DR) affects more than 5.3 million Americans and is the leading cause of blindness among adults of working age.¹ Among Latinos in Los Angeles, California—the ethnic majority in the Los Angeles County safety net—the prevalence of DR among those with diabetes is approximately 50%.² Although early detection and treatment can prevent blindness from DR,³ many persons with diabetes fail to receive appropriate screening examinations and/or sight-saving treatments.^4,5 On average, only 60% of patients with diabetes in the United States receive recommended annual eye examinations; in safety net populations, these rates have been shown to be less than 25%.^6,7

The Los Angeles County Department of Health Services (LAC DHS) is the largest publicly operated county safety net health care system in the United States, serving more than 800 000 patients annually. The LAC DHS safety net is a nonvertically integrated system serving underinsured and uninsured patients, charged with providing high-quality primary and specialty care despite substantial financial and social barriers.⁸ Timely access to specialty services in this underresourced, high-need setting has been an ongoing challenge, especially for eye care, with more than 200 primary care clinics referring patients to 6 optometry and 4 ophthalmology clinics. The increasing prevalence of diabetes during the past 2 decades⁹ has worsened this issue. Historically, wait times for retinal examinations for patients with newly diagnosed diabetes within the LAC DHS have been 8 months or more.

To address this need, we implemented a county-wide, primary care–based teleretinal DR screening (TDRS) program starting in September 2013. Primary care–based TDRS has been proven to be accurate (sensitivity, >80%; specificity, >90%)¹⁰ compared with the criterion standard for DR screening, 7 standard-field 35-mm Early Treatment of Diabetic Retinopathy Study protocol fundus photographs, and the clinical care standard of direct ophthalmoscopic examination by an eye care professional.^11-13 Teleretinal DR screening is well suited to solve the problems of the safety net because it increases access by screening through primary care rather than specialty care, improves efficiency by moving patients with normal retinal photographs out of the queue for appointments with specialty care professionals, and reduces wait times for those with treatable disease. The Veterans Health Administration and the Indian Health Service, among others, have implemented successful TDRS programs,^14-17 but primary care–based TDRS has never been implemented to scale in as large a US safety net system as the LAC DHS.

We describe the successful implementation of a primary care–based TDRS program in the LAC DHS. During implementation, we conducted a quasi-experimental pretest-posttest evaluation to determine whether TDRS decreased wait times for screening for DR and improved screening rates. Our paradigm for implementation is a model for other urban safety net populations where the need for such programs is arguably the greatest.

The LAC DHS had a total of 571 964 unique primary care visits in 2014 and 2015, caring for 64 826 persons with diabetes. Historically, the LAC DHS has had a fragmented specialty care delivery system, with 4 hospitals (with different academic university affiliations), 2 multiservice ambulatory care centers, 6 comprehensive health centers, and multiple primary care clinics operating in urban and rural geographic clusters, making systemwide clinical collaboration and standardization of processes challenging. In addition, the LAC DHS receives referrals for specialty care from more than 200 community partner clinic sites throughout LAC. This scenario creates a formidable environment for the provision of eye care services: a high volume of patients and a limited amount of eye care professionals spread throughout a large geographical area, ranging from urban to rural, in diverse practice settings with varied on-site access to specialists.

To improve access to DR screening, we implemented a TDRS program throughout 15 of the largest LAC DHS–operated primary care clinics. Our first clinic began screening in September 2013, followed by rolling expansion to all LAC DHS comprehensive health center primary care clinics, medical center primary care clinics, and multispecialty ambulatory care center primary care clinics by March 2015.

As of December 2015, a total of 58 certified medical assistants and licensed vocational nurses were trained and certified as fundus photographers. We trained existing certified medical assistants to use the cameras in primary care settings and to upload these digital images via our web-based screening software (EyePACS software; EyePACS LLC). We created a retinal photography clinic for which patients are scheduled in advance by their primary care professional or a care manager. This method best uses the photographers’ time because they often perform other services, and it also allowed us to give patients advance notice of dilation. However, capacity for walk-in appointments was also built in for same-day screening.

Photographers were trained to obtain fundus images according to a 3 standard-field protocol, with images centering on the fovea, optic disc, and temporal to the macula, as well as 1 external eye image.¹⁸ We use a single-drop dilation protocol with tropicamide ophthalmic solution, 1% (approximately 10 minutes to dilation) for all patients unless acceptable images can be obtained without dilation or a contraindication exists. In addition, we mandate that photographers upload a minimum number of cases monthly to maintain certification, thereby maximizing the quality of the images. Image quality is also graded by our readers, and photographers who do not meet adequacy requirements in any 3-month period undergo retraining.

Ten primary certified image readers, who are optometrists currently employed by the LAC DHS, read the screening photographs as part of the DHS-wide teleretinal reading center using a standardized, validated protocol including grade of DR, recommended timing for follow-up, and feedback on the quality of the images. Quality assurance overreads are performed on 10% of cases by 3 ophthalmologists (L.P.D.), while supervisory and adjudicating reads are also performed on cases flagged by the primary reader owing to questions regarding pathologic findings. Readers adhere to protocols governing timing and location of referrals to eye care specialists based on the severity of eye disease and urgency of need for treatment. These protocols were informed by nationally developed preferred practice guidelines for the care of patients with diabetic eye disease¹⁹ and were endorsed by a group of ophthalmology and primary care representatives from across LAC DHS institutions.

Results of the teleretinal screenings and follow-up recommendations are electronically transmitted to primary care professionals. Referrals for abnormal results are submitted via eConsult, a web-based referral system for specialty care, which allows for submission of screening results and subsequent scheduling of follow-up across the LAC DHS. Based on the results of screening, patients may be triaged into either optometry clinics for early levels of DR or ophthalmology clinics for more severe DR. For further description of the clinical pathway and program implementation strategies, please see the eFigure and eTable in the Supplement.

Patients were included in the overall description of the TDRS program implementation if they received a diagnosis of diabetes and received primary care–based teleretinal screening from September 1, 2013, to December 31, 2015. Eligibility for TDRS required that patients have no acute vision loss or major eye symptoms, that they be able to sit up and remain still for retinal photography, and that they not be actively cared for by any eye care professional within or outside the LAC DHS during the last 12 months.

We conducted a retrospective, repeated cross-sectional pretest-posttest evaluation using historical controls with exposure to primary care–based TDRS at the clinic level to evaluate the effect of TDRS on the proportion of patients with diabetes receiving retinal screening and the wait time for screening.

We evaluated the effect of the TDRS program in a subset of 5 DHS-operated clinics of the 15 total in which the program was implemented. To evaluate rates of screening for DR, we determined aggregate proportions of patients with diabetes screened across all 5 clinics in the preintervention and postintervention periods using encounter data. To evaluate time to screening, we measured time to screening among a random sample of 120 patients from each of the 5 clinics in both the preintervention (n = 600) and postintervention (n = 600) periods. The 5 clinics were selected for this evaluation based on patient volume, adequate staff for photography training, and appropriate space to place a fundus camera in the primary care clinic. The clinics were of similar size, patient population, and equal scope of care. Patients were included in the evaluation if they were 18 years of age or older, received a diagnosis of diabetes by a medical professional, and were active patients of the clinic (defined as having had ≥2 routine visits with a primary care professional at one of the designated study clinics in the past 12 months). Patients were excluded if they received regular care by an eye care clinician during the past 12 months. Approval, including a waiver of patient consent, was obtained from the institutional review boards of University of California, Los Angeles (UCLA), University of Southern California, and the Los Angeles Biomedical Research Institute at Harbor-UCLA.

Demographic and clinical characteristics were abstracted from the medical record, including dates of referral for retinal screening and the date the screening was performed. A 12-month period from December 1, 2011, to November 30, 2012, was used for the preintervention period. Identical information was collected during the postintervention phase using a 12-month period after the TDRS program was fully implemented at these 5 practices, from September 1, 2014, to August 31, 2015.

Encounter and claims data were used to estimate the proportion of patients with diabetes seen in each primary care clinic and the number of patients who were screened for DR in the 12-month preintervention and postintervention periods. Aggregate rates of screening were calculated using the same methods in the preintervention and postintervention periods.

In the preintervention and postintervention periods, the number of patients with diabetes receiving ophthalmology and optometry care was determined by obtaining information on patients referred to the LAC DHS eye clinics via record review and linkage of identifiers with medical records from the eye clinics. The wait time from referral to screening and the 95% CIs around this time estimate for preintervention and postintervention groups were calculated.

Summary statistics, including mean (SD) values, median values, interquartile ranges, and frequency distributions, were generated to characterize the study population. A generalized estimating equation logistic regression model was used to analyze the correlated repeated measures of clinics and to provide a population mean comparison for the preintervention and postintervention annual rates of screening. A nonparametric Wilcoxon rank-sum test was used to compare median time to screening between the preintervention and postintervention groups. All tests were 2-sided, and P < .05 was considered statistically significant.

With 120 individuals from each of 5 primary care clinics for both preintervention and postintervention populations, our study was powered to detect a 15% change in wait times to screening for DR. For our evaluation, if a patient chosen randomly to be part of the postintervention group was included in the preintervention group, he or she was excluded from the postintervention sample.

As of December 31, 2015, a total of 21 222 patients underwent screening for DR. The characteristics and disease severity of our screened population are reported in Table 1. Of the 21 222 screened, 14 595 (68.8%) did not need a referral to an eye care professional because they had normal screening photographs or only mild nonproliferative DR (NPDR); both conditions require only repeating fundus photographs annually. Of the 6627 patients (31.2%) who did need referral, 4160 (19.6%) were referred for DR (moderate NPDR, severe NPDR, and proliferative DR, as well as clinically significant macular edema), while 2461 (11.6%) were referred for other conditions, including ungradable photographs owing to the presence of cataracts and/or other eye problems (1040 [4.9%]).

The characteristics of participants in the pretest and posttest samples are found in Table 2. Overall annual screening rates for DR improved from 5942 of 14 633 patients (40.6%) before implementation of TDRS (fiscal year 2011-2012) to 7470 of 13 133 patients (56.9%) after the intervention (fiscal year 2014-2015) (odds ratio, 1.9; 95% CI, 1.3-2.9; P = .002) (Table 3). The Figure depicts a comparison of unadjusted screening rates over time at each of the 5 clinics before and after initiation of the TDRS program. The median time to screening for DR decreased from 158 days (interquartile range, 68-324 days) before the intervention to 17 days (interquartile range, 8-50 days) after the intervention (P < .001) (Table 4).

This study demonstrates a successful, sustainable implementation of a TDRS program in the largest publicly operated county safety net public health system in the United States. By embedding our program in primary care clinics and using certified medical assistants to take the fundus photographs and electronically transmit them to a reading center staffed by current LAC DHS optometrists, we were able to substantially improve both the detection of DR and the use of specialty eye care resources, resulting in the elimination of the need for more than 14 000 visits to specialty care professionals, a 16.3% increase in annual rates of screening, and an 89.2% reduction in wait times for screening.

Our results confirm findings from a smaller pilot study of 6 federally qualified health centers in the LAC DHS health care system showing that 1697 of 2732 patients (62.1%) screened had normal examination results and could be removed from the queue for ophthalmology appointments.²⁰ Of the 1035 patients (37.9%) who were referred for specialty eye care, 507 (18.6%) were referred for DR, 260 (9.5%) of whom needed possible treatment and were flagged for expedited referral. Clearly, TDRS has the potential to eliminate the need for a separate visit to an ophthalmologist for patients with minimal disease and to expedite treatment for those needing urgent ophthalmic attention. Although TDRS programs have been successfully implemented in other care settings, the LAC DHS is an example of a heterogeneous, nonvertically integrated system that is well suited to benefit from TDRS owing to its limited specialty resources and large burden of disease.

Numerous analyses have detailed the large cost benefit of programs aimed at increasing DR screening.^21,22 There is also a significant body of literature examining the cost-effectiveness of TDRS compared with conventional retinal examination in rural, urban, community, academic, and international settings.^23-25 With strategic planning and workflow implementation, TDRS is a potential cost-efficient alternative that can improve convenience and access to retinal screening across different practice settings, including that of county facilities.²⁶

A key to the sustainability of our program is the integration of DR screening into primary care practices, treating it as a diagnostic test to establish a need for referral to specialty eye care. By eliminating the need for a separate visit to a specialist, we are able to increase the number of patients screened for DR without increasing demand on specialty care, which is critical in a system in which more than 3000 people are currently waiting for eye care appointments. Most patients do not need to see an eye care professional, and removing them from the queue decreases the backlog of patients waiting for eye care appointments and allows for better use of patients’ limited resources (eg, transportation and time off work). When the LAC DHS TDRS program is fully operational, we will provide capacity for screening all patients with diabetes in the DHS primary care clinics (approximately 65 000 patients per year).

Implementation of telemedicine initiatives can be challenging in any health care system, and the barriers may be greater in underresourced safety net or public health systems. First, among non–eye care specialists, there is skepticism that TDRS can be as accurate and effective as an in-person examination with an eye care professional, despite a large body of supportive literature.^10-12 In addition to strategic planning and standardization, education of all critical stakeholders and support of hospital, clinic, and health system leadership are essential. A second barrier comes from within the specialty care community: eye care professionals fear that TDRS programs will decrease referrals for in-clinic visits. However, eye care professionals need to understand that improved rates of screening for DR and triage actually result in increased detection of patients with significant disease and therefore increased referrals of patients needing higher-level care.^27,28 In the safety net, this allows us to better match a health care professional’s skill set with the care he or she is providing, which is necessary to improve access and minimize cost. Lastly, critical to the implementation of this TDRS program has been addressing barriers to telemedicine, creating a framework for the future provision of teleophthalmologic care beyond screening for DR.

Our analysis is subject to some limitations affecting generalizability. First, the effect of our intervention may be diminished because postintervention data were collected during the program ramp-up period, while workflows were still being adjusted. Second, during the program rollout, the LAC DHS implemented a new electronic medical record system, which may have resulted in underreporting of rates of screening for DR in the period after implementation owing to improper coding. Third, implementation of our program was concurrent with California’s expansion of Medicaid and changes in government policies around the Affordable Care Act. The enrollment of our patient population into new health care plans caused continual shifts in clinical patient panels as we created patient-centered medical homes and strove to meet new mandates. Wait times for screening and clinic visits may have been lengthened as our patient population more frequently enrolled (and de-enrolled) in insurance programs. However, as transient patient populations are typical of the US safety net and seen in both preintervention and postintervention populations, the effects of this are likely minimal. Fourth, we cannot account for patients who received eye care outside of the LAC DHS, which may affect our rates of screening before and after the intervention. However, rates of outside care should be similar before and after the intervention and, therefore, should not affect changes in screening rates. Fifth, given the high cost and difficulty of finding well-matched controls within the geographically and demographically heterogeneous primary care practices of the LAC health care system, controls were historical, with the selected clinics acting as their own controls. Although it is possible that factors in addition to our intervention affected rates of screening for DR, the magnitude and uniformity of the change we observed suggests that our findings are unlikely to be attributable to secular trends over time. Lastly, although our TDRS program substantially reduced wait times to screening and improved rates of screening, more information is needed to demonstrate that patients who need treatment are actually receiving this care in an expedited fashion.

We showed that TDRS can be executed on a large scale in a heterogeneous, nonvertically integrated health care environment and can result in substantial improvements in both efficiency and quality of care. The safety net is ideal for telehealth interventions owing to limited resources and high disease burden; these interventions allow for health care professionals to work at the top of their skill set, which in turn increases access to care. We believe that the US safety net would be wise to invest in telehealth programs such as this one to address critical needs regarding access to care.

Accepted for Publication: December 12, 2017.

Corresponding Author: Lauren P. Daskivich, MD, MSHS, Ophthalmology and Eye Health Programs, Department of Health Services, Los Angeles County, 313 N Figueroa St, Ste 904C, Los Angeles, CA 90012 (lpdaskivich@dhs.lacounty.gov).

Published Online: March 27, 2017. doi:10.1001/jamainternmed.2017.0204

Author Contributions: Dr Daskivich had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Daskivich, Mangione.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Daskivich, Vasquez, Tseng, Mangione.

Critical revision of the manuscript for important intellectual content: Daskivich, Martinez, Tseng, Mangione.

Statistical analysis: Tseng, Mangione.

Obtained funding: Daskivich.

Administrative, technical, or material support: Daskivich, Vasquez, Martinez, Mangione.

Study supervision: Daskivich.

Conflict of Interest Disclosures: None reported.

Funding/Support: The authors received support from the Robert Wood Johnson Foundation Clinical Scholars Program, grant P30AG021684 from the National Institutes of Health/National Institute on Aging to the UCLA Resource Center for Minority Aging Research Center for Health Improvement of Minority Elderly, and grant UL1TR000124 from the National Institutes of Health/National Center for Advancing Translational Sciences to the UCLA Clinical and Translational Science. Dr Mangione’s work is also supported by the Barbara A. Levey and Gerald S. Levey Endowed Chair in Medicine.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr Mangione is a member of the US Preventive Services Task Force (USPSTF); however, this article does not represent the views and policies of the USPSTF.

Additional Contributions: Richard Baker, MD, Charles R. Drew University of Medicine and Science, contributed to the initial design of this project. He was not compensated for his contribution. We also thank the Los Angeles County Department of Health Services leadership, primary care staff, and eye care professionals who made this project possible.