Screening for Presence or Absence of Diabetic Retinopathy: A Meta-analysis

In 2000, 171 million people were estimated to have diabetes, and more than 5 million became blind owing to diabetic retinopathy (DR). This number is expected to double before 2030.^1-3 Diabetic retinopathy is progressive change in vascular permeability and a proliferation of fragile, new retinal blood vessels.^4-6 The duration of diabetes correlates with both DR and its severity.^7,8 Even with optimal glucose control, nearly all diabetic persons will eventually develop DR.

Studies have demonstrated that early diagnosis and laser treatment of DR can prevent severe vision loss.^9-11 Because DR has few symptoms until vision loss develops,¹² regular DR screening is critical.^3,13 Two recognized gold standards for DR screening are ophthalmological examination by a trained health professional (eg, ophthalmologist) using pupillary dilation (mydriasis) and stereoscopic 7-field fundus photography by a trained photographer with image interpretation by an experienced grader.⁴ Both methods require the specialist equipment and expertise of hospitals and specialist clinics.

Studies from Australia, the United States, and Spain show that only 50% to 60% of diabetic individuals are screened according to national guidelines.^14-16 Diabetic persons who live in rural and/or remote parts of high-income countries and those who live in low- and middle-income countries have poorer access to specialist ophthalmological services and lower rates of screening for DR.^13,17-19 Some populations have both high rates of diabetes and poor access to screening services. For example, 37% of indigenous adults in Australia have diabetes, 13% of whom have already progressed to diabetic vision loss.²⁰ Yet, in this population, only 20% of diabetic persons have had an eye examination in the past year.²¹

Outreach clinics by visiting specialists have been shown to improve access to specialist services, including DR screening, for hard-to-reach populations.^22,23 However, population-based DR screening is far more accessible if simple community-based screening facilities are routinely available. The need to bring the specialist to the patient in remote locations can be reduced for many types of health care, including screening for DR, using telemedicine technologies.²⁴

A nonspecialist, community-based, outreach model for DR screening has emerged in which: (1) Photography is used instead of direct examination by a specialist clinician, and the image is then sent to a central specialist center. (2) The camera is operated by a photographer, technician, or health worker. (3) The pupil is generally not dilated (nonmydriatic), unlike direct ophthalmoscopic examination and other photographic methods, which require time-consuming and temporarily disabling pupil dilation. (4) The image is interpreted at a different place and time by an ophthalmologist or other trained reader.

This model has been applied in rural and/or remote areas of Australia²⁵ and Canada,²⁶ and for American Indian and Alaskan Native populations.^27,28 For example, in the Kimberley region of northwestern Australia, local Aboriginal health care workers take retinal photographs that are subsequently interpreted by an ophthalmologist situated in Perth, 2000 km away.²⁹ The outreach model has also been used in mobile vans in the United Kingdom.³⁰

Outreach DR screening enables better coverage of at-risk populations, especially populations that are hard to reach.²⁶ However, this carries the potential disadvantages of reductions in screening sensitivity (which would result in emergent cases of DR being missed) and specificity (creating undue referrals to costly specialist eye services). Our meta-analysis aimed to estimate the effect on screening accuracy of variations in mydriasis and in the medical qualifications of photographers, the 2 key characteristics of outreach screening methods that have the greatest potential effect on screening coverage.

We searched MEDLINE, EMBASE, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, Rural and Remote Health Database (RURAL), and Indigenous Australia from their inception to June 2009. The search strategy comprised keywords for DR, DR-specific screening terms, and generic screening terms (Table 1). The reference lists of clinical guidelines and literature reviews (identified by searching these databases or already known by us) were searched for further relevant citations.

We included studies that aimed to evaluate the accuracy of a DR screening method among patients with diabetes or DR in which photography- or examination-based DR screening was compared with either 7-field mydriatic photography or dilated fundal examination by an ophthalmologist or equivalent specialist. Studies of automated analysis techniques and technologies were excluded because these are not currently standard practice. We included studies that measured sensitivity and specificity (widely accepted measures of a test's diagnostic accuracy³¹) to detect any DR, sight-threatening DR, or macular edema.

Two authors independently screened citations and full-text articles against inclusion criteria. Disagreements were resolved by consensus between these 2 authors or by adjudication from an ophthalmologist (H.R.T.).

Database items were developed in consultation with an experienced ophthalmologist (H.R.T.). Data extraction was performed by 2 authors and piloted using 10 included studies. Approximately 10% of data were audited, and appropriate corrections were made to the data and protocols.

We extracted data regarding country, region, and setting (specialist clinic, hospital, primary health care clinic, or outreach), number of patients and/or eyes in the sample, patient characteristics (percentage of men, age, and number of years with diabetes), prevalence of any DR in the sample, and the reference standard (7-field mydriatic photography or dilated fundal examination). Screening methods were categorized by screening instrument (film, Polaroid, or digital camera, direct examination, or scanning laser ophthalmoscope), mydriatic status (mydriasis, no mydriasis, mixed or not reported), number of photographic fields, medical qualifications of photographers (photographer or technician, nurse, general practitioner, or ophthalmologist), and qualifications of interpreter (general practitioner, ophthalmologist, optician, or grader).

For meta-analysis, screening methods were collapsed into 6 categories:

• Nonmydriatic camera, nonspecialist photographer: nonmydriatic photography performed by a person with no specialist medical or eye qualifications (nurse, photographer, or technician).

• Mydriatic camera, nonspecialist photographer: mydriatic photography performed by a person with no specialist medical or eye qualifications.

• Nonmydriatic camera, specialist photographer: nonmydriatic photography performed by a trained health professional (general practitioner or diabetologist) or person with specialist eye qualifications (ophthalmologist, retinal specialist, or optometrist).

• Mydriatic camera, specialist photographer: mydriatic photography performed by a trained health professional (general practitioner or diabetologist) or person with specialist eye qualifications (ophthalmologist, retinal specialist, or optometrist).

• Direct examination: direct eye examination methods.

• Other: for example, photographer not reported, mydriatic status mixed or not reported, or a combination of examination and photography.

Study characteristics were tabulated. Meta-analysis using Stata version 10.1 statistical software (StataCorp LP, College Station, Texas) determined sensitivity and specificity, adjusting for other characteristics of the studies and subjects. Forest plots portrayed sensitivity and specificity results of each included study. Owing to variations in the methods and reporting among studies, the unit of analysis was taken to be the number of patients rather than number of eyes. The quoted sensitivities and specificities in each article were then applied to the number of patients, producing a standard 2 × 2 table for each test within each article. Specificities of 100% were modified to enable analyses by reducing the number of true negatives by 1. We computed the 95% confidence intervals (CIs) in the forest plots using the Wilson score interval method.³²

To account for multiple tests within each study and heterogeneity of sensitivity and specificity across studies, a 3-level random-intercepts logistic regression model³³ using 20 numerical quadrature points was used, producing pooled estimates of sensitivity and specificity and 95% CIs (procedure xtmelogit in Stata version 10.1). This model accommodates heterogeneity in the sensitivity and specificity (on the logit scale) between studies and also between tests within the same study. Information concerning repeated measurement of subjects within the same study was not available; therefore, the logistic models assume that a different set of subjects was assessed for each test within each study.

Estimates of odds ratios (ORs) comparing sensitivity and specificity across subgroups were obtained by including the relevant covariate in the regression models. The small number of studies and tests within studies necessitated that all models assumed that the level of heterogeneity across studies and tests within studies were constant across subgroups. When the number of studies was very small and computational difficulties were encountered, the test-within-study variance component was omitted, and modeling proceeded using the between-study variance component only.

A flowchart of the search and selection of titles and abstracts is shown in Figure 1. Three hundred sixty-three titles and abstracts were identified; 188 potentially relevant full-text articles were evaluated; and 43 studies measured DR screening sensitivity and specificity. However, 22 different measures of DR were reported, using various permutations and combinations of DR classification levels. Only the category of “any DR” had sufficient consistency in definition, number of studies (n = 20), and number of sensitivity and specificity assessments (n = 40) for meta-analysis. The most common reason for exclusion of full-text articles was lack of comparison of screening method(s) with an accepted reference standard (n = 59).

Of the 20 included studies, 6 were conducted in the United Kingdom,^34-39 5 in the United States,^27,40-43 3 in Spain,^44-46 and 1 each in Australia,⁴⁷ Canada,²⁶ Germany,⁴⁸ Hong Kong,⁴⁹ India,⁵⁰ and the Netherlands⁵¹ (Table 2). Study settings were commonly diabetes clinics, primary health care facilities, and hospitals (4 studies each). The mean sample size was 207 patients (range, 14-773 patients). The mean age of study participants was 58.0 years (range, 48.0-68.8 years), and the mean percentage of male participants was 55.2% (range, 31.0%-98.4%). The mean duration of diabetes was 10.9 years (range, 3.7-17.7 years), and the mean prevalence of any DR was 46.0% (range, 9.0%-83.8%). Eleven studies contributed 1 assessment of a screening method; 4 studies contributed 2 screening assessments; 2 studies contributed 3 assessments; and 3 separate studies contributed 4, 5, and 6 assessments, respectively (Table 3).

The screening methods used were the use of a digital camera (17 assessments), the use of a film camera (8 assessments), direct examination (8 assessments), the use of a Polaroid camera (3 assessments), the use of various combinations of camera types or camera plus examination (3 assessments), and the use of a scanning laser ophthalmoscope (1 assessment) (Table 3). Pharmacological mydriasis was used in 23 assessments, no mydriasis was used in 12 assessments, a combination of mydriasis and no mydriasis was used in 4 assessments, and mydriatic status was not reported in 1 assessment. Of the methods involving imaging, single-field photography was most common (15 assessments), followed by 3-field photography (8 assessments) and 2-field photography (4 assessments).

Images were most frequently taken by a photographer or technician (12 assessments), general practitioner or physician (8 assessments), or nurse (3 assessments) and were most frequently interpreted by an ophthalmologist (16 assessments), trainee ophthalmologist (4 assessments), retinal specialist (4 assessments), or optometrist (4 assessments). Reference standards were dilated fundal examination (23 assessments), 7-field mydriatic photography (16 assessments), or both (1 assessment).

For statistical meta-analysis, 5 assessments were made using a nonmydriatic camera with a nonspecialist photographer; 8, mydriatic camera, nonspecialist photographer; 4, nonmydriatic camera, specialist photographer, 3, mydriatic camera, specialist photographer; 8, direct examination; and 12, the “other” category (eg, photographer not reported, mydriatic status mixed or not reported, or a combination of examination and photography).

Individual study estimates of sensitivity and specificity for detecting any DR are contained in the forest plots in Figures 2 and 3, respectively. The point estimates and 95% CIs are displayed, with screening assessments grouped by mydriatic status. For 2 studies,^39,49 the lower limit of the 95% CI for sensitivity was less than 25% (Figure 2). Pooled estimates for sensitivity and specificity by mydriatic group and overall are also displayed in Figures 2 and 3. The pooled estimates for all included studies were 82.5% (95% CI, 75.6%-87.9%) and 88.4% (95% CI, 84.5%-91.4%) for sensitivity and specificity, respectively. The pooled sensitivity and specificity estimates for the 4 assessments that used a mix of nonmydriatic and mydriatic (ie, mydriasis as required) screening methods^26,39,44 were 68.7% (95% CI, 43.4%-86.3%) and 89.0% (95% CI, 77.3%-95.1%), respectively.

Pooled estimates for various subgroups from the logistic regression models are presented in Table 4. All pooled estimates of sensitivity and specificity were greater than 80% except for the mydriatic status mixed or not reported category (sensitivity, 74.8%) and the direct examination category (sensitivity, 73.5% in photographer analysis and 73.9% in combination analysis). Variations in mydriatic status alone did not significantly influence sensitivity (OR, 0.89; 95% CI, 0.56-1.41; P = .61) or specificity (OR, 0.94; 95% CI, 0.57-1.54; P = .80) to detect any DR.

Variations in photographer medical qualification alone did not significantly influence sensitivity (P = .75) to detect any DR. The odds of a positive test result for diagnosing any DR when it was evident with the reference standard for photographers with specialist medical or eye qualifications were 1.25 times that of photographers without specialist medical or eye qualifications (95% CI, 0.31-5.12). For mydriatic camera methods, the OR was 1.61 (95% CI, 0.36-7.30; P = .54), and for nonmydriatic camera methods, the OR was 1.31 (95% CI, 0.30-5.66; P = .72).

The specificity of detection of any DR, however, was significantly higher for screening methods that use a photographer with specialist medical or eye qualifications (P = .001). When photographs were taken by photographers with specialist medical or eye qualifications, the odds of a negative screening test when DR was not evident with the reference standard were 3.86 times that when photographs were taken by photographers without specialist medical or eye qualifications (95% CI, 1.78-8.37). This was largely owing to the effect of specialists vs nonspecialists in photographs taken without mydriasis (OR, 5.65; 95% CI, 2.24-14.25; P < .001). For tests using mydriasis, although specificity was again greater for photographers with specialist medical or eye qualifications, this difference was not statistically significant (OR, 2.42; 95% CI, 0.86-6.84; P = .09). One study⁴⁶ yielded 6 of the 7 assessments in the “specialist” category.

In our meta-analysis of 40 assessments of 20 studies from 9 countries, sensitivity to detect any DR was not influenced by variations in mydriatic status or photographer medical qualifications, either alone or in combination. Therefore, retinal photography by a photographer with no specialist medical or eye qualifications (ie, a health worker or nurse), without using pupil-dilating eye drops (the outreach model), appears unlikely to miss cases of DR that screening methods using mydriasis or a photographer with specialist medical or eye qualifications would detect.

Improving screening coverage can prevent blindness due to DR and reduce overall health care costs; mathematical modeling demonstrates that increasing screening coverage from 30% to 80% in Australia would reduce the overall health care cost of DR and its consequences from approximately $193 million to $178 million per year.⁵² The relative benefit is likely much greater in rural and remote areas where specialist resources are scarce and diabetes prevalence is proportionally higher.

Most guidelines recommend annual screening of patients with diabetes.^13,53 Javitt et al⁵⁴ modeled the relationship between screening sensitivity and person-years of sight saved for annual screening and found that sensitivity values greater than 60% do not substantially add to person-years of sight saved or reduce screening costs. Javitt et al hypothesized that the diminishing additional benefit of sensitivity values greater than 60% was owing to the frequency of screening and the likelihood that DR cases missed on one visit will be detected on the next. In our meta-analysis, the pooled estimates for sensitivity to detect any DR for all image-based screening methods, regardless of mydriatic status and photographer medical qualifications, were greater than 80%.

We found that screening involving photographers without specialist medical or eye qualifications yielded significantly lower specificity (ie, a greater false positive rate) than screening involving photographers with specialist medical or eye qualifications, particularly when mydriasis was not used. This finding should be interpreted in the context that 1 study⁴⁶ yielded 6 of the 7 assessments in the specialist category. The greater number of false positives found when photographers without specialist medical or eye qualifications were used may be owing to imaging difficulties or poorer image quality in their photographs, especially when the pupil is not dilated. The result may be a referral to an eye specialist for further examination. In addition to inconveniencing patients, unnecessary referrals place a burden on limited ophthalmological infrastructure and personnel. Although false positives may be less harmful than missing DR and the opportunity to prevent vision loss, they may offset some of the economic advantages of outreach screening methods.

Only 4 assessments from 3 studies followed a mixed strategy of using mydriasis when required to obtain a better image. This is potentially a pragmatic, “best of both worlds” approach in which the time delay and discomfort of mydriasis is mostly avoided without compromising sensitivity and specificity. The pooled sensitivity estimate for these studies (68.7%) was lower than for other categories, although it still exceeded the threshold reported in Javitt et al,⁵⁴ and the specificity for mixed mydriasis (88.8%) was comparable with that of other screening methods. Further research is required to gain a more robust picture of the accuracy of this approach.

Although we considered studies that examined accuracy of detection of all degrees of retinopathy, we were only able to analyze studies of screening to detect any DR. There was too much variation in other categories of DR and too few assessments in any 1 category. Future studies of DR screening accuracy should use a consistent DR classification scheme, such as that of the International Clinical Diabetic Retinopathy Disease Severity Scale.⁵⁵ This scale was designed to enable clinically important DR grades to be identified by less experienced screeners, therefore improving communication between the various health professions involved in DR screening.⁴ It also specifies the equivalence of the most frequently used DR classification scale (ie, the Early Treatment of Diabetic Retinopathy Severity Scale) to the International Clinical Diabetic Retinopathy Disease Severity Scale.

Furthermore, 12 assessments from 9 studies had to be classified as “other” for analysis purposes. In 7 assessments, this was because details of the photographer could not be ascertained. Better reporting of such details of DR screening methods, combined with a consistent approach to DR classification, would enable a greater range of measures in DR screening studies to be meaningfully compared and analyzed.

This meta-analysis found that outreach DR screening is an effective alternative to screening that uses mydriasis or a medical specialist photographer. The outreach model has enormous potential to increase the screening coverage of at-risk populations in areas of the world where specialist ophthalmic resources are limited and to be a cost-effective way of preventing blindness among diabetics. Outreach DR screening could therefore contribute substantially to the Vision 2020 objective of setting up systems to prevent DR-related vision loss in high-risk countries, “taking into account the country's resources, social expectations and health-care infrastructure.”¹³

Correspondence: Hugh R. Taylor, MD, Melbourne School of Population Health, University of Melbourne, Level 5, 207 Bouverie St, Carlton, Melbourne, Victoria 3053, Australia (h.taylor@unimelb.edu.au).

Submitted for Publication: March 4, 2010; final revision received September 12, 2010; accepted September 15, 2010.

Published Online: December 13, 2010. doi:10.1001/archophthalmol.2010.319

Financial Disclosure: None reported.

Funding/Support: This study was supported by funding from the Harold Mitchell Foundation, the Ian Potter Foundation, and the Cybec Foundation; and a National Health and Medical Research Council Career Development Award (Prof Gruen).

Role of the Sponsor: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.

Additional Contributions: We thank Anne Parkhill, MBIT, DipLib, from Aptly Information Design for developing the search strategy, Jason Wasiak, MPH, from Monash University, Alfred Hospital, for his contribution to study selection, and Ornella Clavisi, MPH, BSc(Hons) from Monash University and Alex Hewitt, MD, PhD, from the Royal Victorian Eye and Ear Hospital, and Nathalie Colomer, MD, from Pontificia Universidad Católica de Chile for their contribution to the data extraction protocol design. Anne Parkhill received financial compensation for her contribution to this manuscript.