Both images were graded by an expert grader at Moorfields Eye Hospital Reading Centre.
eTable. Cross-tabulation of VCDR Grading by Imaging Modality
eFigure 1. Peek Retina 3-Dimensional Printed Prototype Adapter Fitted on a Samsung SIII Smartphone
eFigure 2. Flowchart of Images Included for Comparison
eFigure 3. Histogram Showing Distribution of VCDR in Reference and Peek Images With Mean (97.5th and 99.5th Percentiles) of Reference Images Indicated
Customize your JAMA Network experience by selecting one or more topics from the list below.
Bastawrous A, Giardini ME, Bolster NM, et al. Clinical Validation of a Smartphone-Based Adapter for Optic Disc Imaging in Kenya . JAMA Ophthalmol. 2016;134(2):151–158. doi:10.1001/jamaophthalmol.2015.4625
Copyright 2016 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Visualization and interpretation of the optic nerve and retina are essential parts of most physical examinations.
To design and validate a smartphone-based retinal adapter enabling image capture and remote grading of the retina.
Design, Setting, and Participants
This validation study compared the grading of optic nerves from smartphone images with those of a digital retinal camera. Both image sets were independently graded at Moorfields Eye Hospital Reading Centre. Nested within the 6-year follow-up (January 7, 2013, to March 12, 2014) of the Nakuru Eye Disease Cohort in Kenya, 1460 adults (2920 eyes) 55 years and older were recruited consecutively from the study. A subset of 100 optic disc images from both methods were further used to validate a grading app for the optic nerves. Data analysis was performed April 7 to April 12, 2015.
Main Outcomes and Measures
Vertical cup-disc ratio for each test was compared in terms of agreement (Bland-Altman and weighted κ) and test-retest variability.
A total of 2152 optic nerve images were available from both methods (also 371 from the reference camera but not the smartphone, 170 from the smartphone but not the reference camera, and 227 from neither the reference camera nor the smartphone). Bland-Altman analysis revealed a mean difference of 0.02 (95% CI, −0.21 to 0.17) and a weighted κ coefficient of 0.69 (excellent agreement). The grades of an experienced retinal photographer were compared with those of a lay photographer (no health care experience before the study), and no observable difference in image acquisition quality was found.
Conclusions and Relevance
Nonclinical photographers using the low-cost smartphone adapter were able to acquire optic nerve images at a standard that enabled independent remote grading of the images comparable to those acquired using a desktop retinal camera operated by an ophthalmic assistant. The potential for task shifting and the detection of avoidable causes of blindness in the most at-risk communities makes this an attractive public health intervention.
A total of 285 million people are visually impaired worldwide (Snellen acuity <6/18) of whom 39 million are blind (<3/60 better eye). Low-income countries carry approximately 90% of the burden of visual impairment, and 80% of this can be prevented or cured.1
There is a widening gap between the number of eye health care professionals worldwide and an increasing need as populations enlarge and age. Blinding eye disease is most prevalent in older people, and in many regions the population 60 years and older is increasing at twice the rate of the number of health care professionals.2,3
Diseases of the posterior segment are responsible for up to 37% of blindness in sub-Saharan Africa.4 However, diagnosis, monitoring, and treatment are challenging in resource-poor countries because of a lack of trained personnel and the prohibitive cost of imaging equipment.
Retinal imaging is frequently used in the diagnosis and monitoring of diseases, such as diabetic retinopathy, glaucoma and age-related macular degeneration, retinopathy of prematurity,5 and systemic diseases, such as hypertension,6 malaria,7 human immunodeficiency virus or AIDS,8 and syphilis.9
Ophthalmologists, physicians, and eye-care workers have used ophthalmoscopes of varying types for more than 150 years, with the first reported use by Dr William Cumming in 1846.10 The development of fundus cameras has made it possible to record and share images to collect evidence of disease presence, severity, and change.
The advent of digital imaging has made recording, processing, and sharing of images far quicker and cheaper than previous film-based methods.11 However, retinal cameras remain impractical in many low-income countries and in primary care settings throughout the world where early detection of eye disease is prohibited because of high cost, large size, low portability, infrastructure requirements (eg, electricity and road access), and difficulty of use.
Mobile telephone access has reached near-ubiquitous levels worldwide,12 with the highest worldwide increase in the rate of mobile telephone ownership in Africa. Telemedicine has in recent years begun to favor wireless platforms, with newer smartphone devices having high-powered computational functions, cameras, image processing, and communication capabilities.13 Mobile telephone cameras are promising when attached to imaging devices, such as microscopes14 and slitlamp biomicroscopes15; however, they remain impractical in many remote settings because of the size and expense of the equipment to which the smartphone is attached. The development of a handheld smartphone device used in clinical microscopy has proven successful.16
Retinal imaging is in principle similar to using a microscope; however, it is more complex because of the interaction between the camera optics with the optics and illumination of the eye.17 The goal of the smartphone-based adapter (Portable Eye Examination Kit [Peek Retina]) prototype was to demonstrate the feasibility of creating a portable mobile telephone retinal imaging system that is appropriate for field use in Kenya and similar contexts, characterized by portability, low cost, and ease of use by minimally trained personnel. Our primary aim was to validate such a smartphone adapter for optic nerve imaging in the context of a population-based study in Nakuru, Kenya.18
Feasibility of a smartphone adapter for optic nerve imaging to desktop retinal camera was evaluated in Kenya.
Differences in quality from image acquisition with a smartphone adapter by photographers not trained in health care compared with photographers trained in eye care were not identified.
Images from the smartphone adapter appeared comparable to images from a desktop camera when independently graded by experts.
These imaging systems may make such data collection more feasible in similar settings.
Participants included in the study were from the follow-up phase of a population-based cohort study on eye disease in Kenya (January 7, 2013, to March 12, 2014).18 One hundred clusters were selected at the baseline (January 26, 2007, to November 11, 2008), with a probability proportional to the size of the population.19 Households were selected within clusters using a modified compact segment sampling method.20 Each cluster was divided into segments so that each segment included approximately 50 people 50 years or older. An eligible individual was defined as someone 50 years or older living in the household for at least 3 months in the previous year at baseline and who was found and consented to follow-up assessment 6 years later (2013-2014).
The smartphone-based adapter was available for use in the final 75 of the 100 clusters revisited, and all available participants in those clusters were examined. All participants were examined with both the smartphone-based adapter and a desktop retinal camera (CentreVue+ Digital Retinal System, Haag-Streit), which acted as the reference standard.
The study adhered to the tenets of the Declaration of Helsinki and was approved by the ethics committees of the London School of Hygiene and Tropical Medicine and the African Medical and Research Foundation, Kenya. Approval was also granted by the Rift Valley provincial medical officer and the Nakuru District medical officer for health. Approval was sought from the administrative heads in each cluster, usually the village chief.
Informed consent was obtained from all participants. The objectives of the study and the examination process were explained in the local dialect to those eligible in the presence of a witness. All participants gave written (or thumbprint) consent.
Pharmacologic dilation in the pupils of all study participants was achieved using tropicamide, 1%, with phenylephrine, 2.5%, if needed. Dilation was not performed in participants deemed at risk for narrow angle closure (inability to visualize >180° of posterior pigmented trabecular meshwork on nonindentation gonioscopy at the slitlamp by the study ophthalmologist [A.B.]21).
Examination with the reference camera and the smartphone-based adapter was performed in a dimly lit room; however, conditions slightly varied among clusters. An ophthalmic assistant took retinal images with the reference camera, and 1 of 2 operators or photographers used the smartphone-based adapter; all users were masked to the alternative examination. The 2 examinations took place in different rooms as availability allowed (Figure 1).
An ophthalmic assistant digitally photographed the lens and fundus on all study participants with the reference camera, which is approved for national diabetic retinopathy screening in the United Kingdom (https://www.gov.uk/government/collections/diabetic-eye-screening-commission-and-provide). Two 45° fundus photographs were taken in each eye: one optic disc centered and the other macula centered. Images were then securely uploaded to the Moorfields Eye Hospital Reading Centre (MEHRC) for review and grading.
An experienced ophthalmic clinical officer or a lay technician with no health care background used a digital retinal camera (Samsung SIII GT-I9300; Samsung C&T Corp) and its native 8.0-megapixel camera with the smartphone-based adapter (Peek Retina) (eFigure 1 in the Supplement) to perform dilated retinal examinations on study participants. Images were recorded as video (approximate 3-10 seconds at 3-7 MB per eye) with single frames (<0.5 MB) used for disc analysis. Both examiners, henceforth termed photographers, received basic training in anatomy and the identification of retinal features (including optic nerve and optic cup) at the beginning of the study.
The smartphone-based adapter consists of a plastic clip that covers the telephone camera and flash (white LED) with a prism assembly. The prism deflects light from the flash to match the illumination path with the field of view of the camera to acquire images of the retina. The phone camera and clip are held in front and close to the eye, which allows the camera to capture images of the fundus.22 A video sweep of the optic disc was performed using the adapter on a smartphone with the native camera app on each eye and securely uploaded to the MEHRC for review and grading. A 1-hour training session on how to use the smartphone-based adapter was performed before the study commenced.
In a random subset of 100 optic nerve examinations performed with the smartphone-based adapter, bespoke software (Peek Grader, Peek Vision) (Figure 2) was used by 2 local study examiners (one nonophthalmologist experienced in retinal examination and one with no health care training, independent of the original photographers) to select still images of the optic disc from the video sweep and use on-screen calipers to measure the vertical cup-disc ratio (VCDR) with no training provided beyond that in the app instructions on caliper placement.
All images were initially examined on a large screen display for quality. For gradable images, 2 independent graders reviewed optic disc pairs. In case of grading difficulties, the adjudicator (T.P.) determined the image grade and verified a random sample of 10% of images for quality assurance and control. Graders regraded a random selection of 100 images after a minimum of 14 days to assess intragrader reliability. The adjudicator also graded 5% of randomly selected images to ensure quality control. Data were checked for consistency by a data monitor (N.S.). Optic disc images were graded as normal, suspicious, or abnormal. A disc was considered abnormal if there was neuroretinal rim thinning as defined by the ISNT rule (normal eyes have a characteristic configuration for disc rim thickness of inferior greater than or equal to superior greater than or equal to nasal greater than or equal to temporal),23 notching or disc hemorrhage was present, or the VCDR was 0.7 or greater. A suspicious disc was one for which adjudication was necessary to determine whether its appearance was abnormal.
All participants identified as having treatable disease in this study were offered appropriate care, including free surgery and transport to the Rift Valley General Provincial Hospital or St Mary’s Mission Hospital, Elementaita. A trained ophthalmic nurse or ophthalmic clinical officer discussed the diagnosis and provided counseling to the study participants. In addition, nonstudy attendees were examined and treated by the study team.
We used the Bland-Altman method24 to analyze agreement and repeatability between and within diagnostic tests and weighted κ scores to compare the VCDR measurements made on different image sets or on regrading.24,25 For κ weighted agreement of VCDR between observers and imaging methods, the following weights were applied: 1.0 for a 0.0 difference, 0.95 for a 0.05 difference, 0.90 for a 0.10 difference, 0.50 for a 0.15 difference, 0.20 for a 0.20 difference, and 0.00 for all differences greater than 0.20 as used in a previous analysis of disc agreement.25 We performed the following specific comparisons:
Reference desktop retinal camera image repeatability: subset of 100 optic disc images randomly selected for repeat grading by an MEHRC grader to assess intraobserver agreement.
Smartphone-based adapter repeatability: subset of 100 optic disc images randomly selected for repeat grading by an MEHRC grader to assess intraobserver agreement (the same individuals used for reference image intraobserver repeatability assessment).
Reference desktop retinal camera images by expert grader on large screen vs smartphone-based adapter images using the on-screen calipers in Peek Grader (Figure 2): the same 100 images as comparisons 1 and 2.
Smartphone-based adapter images by an MEHRC grader on the large screen vs smartphone-based adapter images by a field ophthalmologist or layperson using Peek Grader: the same 100 images as comparisons 1 and 2.
Reference desktop retinal camera images by an MEHRC grader vs smartphone-based adapter images by an MEHRC grader on a large screen: all 2152 image pairs analyzed together.
Reference desktop retinal camera images by an MEHRC grader vs smartphone-based adapter images by an MEHRC grader on a large screen: 2152 image pairs subdivided by whether the images were collected by an experienced photographer or a lay photographer.
Recruitment took place from January 7, 2013, to March 12, 2014. A total of 1460 individuals from 75 clusters participated. Their mean (SD) age was 68 (9) years (range, 55–99 years), and 700 (47.9%) were female. Participants underwent retinal examination using the smartphone-based adapter and the standard desktop retinal camera. A total of 2920 eyes were imaged, of which 2152 eyes (73.7%) had gradable images from both the smartphone-based adapter and the reference camera. In 170 eyes, a gradable image was obtainable with the smartphone-based adapter but not the reference camera, and, conversely, in 371 eyes, a gradable image was obtainable with the reference camera but not with the smartphone-based adapter. In 227 eyes a disc image was not possible from either modality (eFigure 2 in the Supplement).
The VCDR parameters derived from the analysis of the 2152 reference desktop retinal camera images from this population (eFigure 3 in the Supplement), using the definitions in the International Society for Geographical and Epidemiological Ophthalmology classification, were as follows: mean VCDR, 0.38; 97.5th percentile VCDR, 0.7; and 99.5th percentile, VCDR 0.9.
A set of images from 100 eyes were used to assess intraobserver repeatability. Bland-Altman analysis and κ scores found excellent intraobserver repeatability for the MEHRC graders for both the reference desktop retinal camera images (Table, comparison 1) and the smartphone-based adapter images (Table, comparison 2).
For the same 100 eyes, we compared the VCDR measured on the reference desktop retinal camera images by the MEHRC grader and the images of the same eye taken with the smartphone-based adapter with the VCDR graded on the telephone screen (Figure 2) by an ophthalmologist (Table, comparison 3a) or a layperson (Table, comparison 3b). Although the mean difference of the mean by the Bland-Altman method was less than 0.1, the weighted κ scores were relatively low. We performed a similar analysis with the smartphone-based adapter image graded by the MEHRC grader compared with the VCDR measured with the Peek Grader (Table, comparisons 4a and 4b). We again found a small difference in the mean difference but low κ scores.
We compared (Table, comparison 5) the VCDR measured by an expert grader (MEHRC) from the smartphone-based adapter and reference digital retinal camera images for 2152 eyes (eTable in the Supplement). The Bland-Altman analysis found a difference in the mean of −0.02 (95% CI, −0.21 to 0.17) (Figure 3).
Two members of the field team collected retinal images using the smartphone-based adapter. The first was a trained eye care worker experienced in the assessment of the retina (experienced photographer). The second had no prior health care or eye care experience but was proficient in the use of a smartphone (lay photographer). Bland-Altman analysis was performed comparing the reference images and smartphone-based adapter images, both graded at the MEHRC. For the 1239 eyes that had smartphone-based adapter images collected by the experienced retinal photographer, the difference in the mean was −0.02 (95% CI, −0.22 to 0.17) (Table, comparison 6a). For the 913 eyes that had smartphone-based adapter images collected by the lay photographer, the difference in the mean was also −0.02 (95% CI, −0.20 to 0.16) (Table, comparison 6b). There was no observable difference in image acquisition quality between the experienced retinal photographer and lay photographer.
The findings of this study are discussed within the context of optic disc imaging in a population-based study in Kenya. We compared the performance of 2 imaging modalities and different image-grading expertise. The results indicate that smartphone-based adapter images, when analyzed by an independent expert, have excellent agreement with images from a reference desktop retinal camera read by the same expert.
Intraobserver agreement within imaging modalities was also excellent for the reference camera and the smartphone-based adapter images. This finding indicates a high degree of confidence to be able to measure real change over time when a threshold for VCDR increase of 0.2 or greater is used.
Although the Bland-Altman limits of agreement were acceptable for all comparisons, the smartphone-based adapter, particularly when performed by a nonclinically trained user, was of only fair or slight agreement with the expertly graded reference image. The lower levels of agreement with the smartphone-based adapter may be accounted for by images being graded on a small screen with no user guidance given beyond basic instructions within the app to “measure the disc” and “measure the cup.”
Although stereoscopic disc images are the preference for optic nerve grading, monoscopic images, as used in this study, do not represent a disadvantage for grading glaucoma likelihood.26 The finding that nonclinically trained personnel can acquire images of the optic disc using a low-cost smartphone adapter that are of a standard that appears comparable to a desktop retinal camera operated by a dedicated ophthalmic technician or assistant suggests there is potential for use of such devices in mobile health and tele-ophthalmology.
In this study, we only assessed optic disc features; however, potential use in retinal diseases warrants further investigation, the findings of which would have implications for diabetic retinopathy screening programs. Previously described uses of smartphone-based cameras for diabetic retinopathy have been in a clinic setting when operated by a retinal specialist and found to provide good agreement with slitlamp biomicroscopy examination also performed by a retinal specialist.27,28 Further assessment of smartphone-based tools by nonspecialists in nonophthalmic settings is warranted.
A limitation of this study, typical of clinical research based on highly iterative technologies, is that, in relying on rapidly evolving platforms, the time to dissemination of results is long compared with the evolution of the technology itself. This limitation often results in the presentation of data from technology that have been superseded by subsequent prototypes or commercially available devices. In this field study, an early iteration of the smartphone-based adapter (internally identified as mark II) was used throughout. However, by the time of completing the analysis, a more advanced iteration of the smartphone-based adapter (mark VI) was available. An image acquired using mark VI is shown in Figure 4. When compared with Figure 2, which shows an image from mark II, a significant improvement is evident.
A further limitation is that no evaluation of optic discs from either imaging modality was performed without mydriasis. Previous investigations have found the limits of agreements between nonmydriatic optic disc grading to be outside clinically acceptable levels.29 We found it possible to acquire good optic nerve images in undilated pupils of 2.5- to 3.0-mm diameter.
The smartphone-based adapter prototypes, subsequent commercially available devices, and alternative portable retinal imaging systems could contribute to tackling avoidable blindness and in screening for diseases with eye manifestations, particularly in low-income countries and remote communities where mobile telephone infrastructure is ubiquitous but trained personnel are few. Existing telecommunications infrastructure can enable greater access to health care by permitting timely diagnosis using data sharing via the communication capabilities intrinsic to the telephone. With the development of automated retinal imaging systems,30 we could see real-time diagnostics by a technician rather than by the more scarcely available eye care personnel.
Coupling imaging with other smartphone-based diagnostic tests31 and geotagging enables database creation of examined individuals based on predetermined parameters as demonstrated by systems such as EpiCollect.32 Such systems make follow-up and epidemiologic data collection more feasible in resource-poor settings.
Smartphone penetration continues to increase with higher computing power, purpose-built software and hardware, greater connectivity, and lower handset costs. There is now an opportunity to reach the most underserved populations in a manner that was not possible just a decade ago.
Submitted for Publication: July 9, 2015; final revision received October 1, 2015; accepted October 3, 2015.
Corresponding Author: Andrew Bastawrous, MBChB, BSc (Hons), FHEA, MRCOphth, International Centre for Eye Health, Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom (email@example.com).
Published Online: November 25, 2015. doi:10.1001/jamaophthalmol.2015.4625.
Author Contributions: Dr Bastawrous had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Bastawrous, Shah, Kuper, Burton.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Bastawrous, Shah, Livingstone, Hu.
Critical revision of the manuscript for important intellectual content: Bastawrous, Giardini, Bolster, Peto, Livingstone, Weiss, Rono, Kuper, Burton.
Statistical analysis: Bastawrous, Peto, Weiss, Hu.
Obtained funding: Bastawrous, Burton.
Administrative, technical, or material support: Bastawrous, Giardini, Peto, Shah, Livingstone, Weiss, Rono.
Study supervision: Giardini, Peto, Kuper, Burton.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: The Nakuru Eye Disease Cohort Study was jointly funded by the Medical Research Council (MRC) and the Department for International Development (DFID) under the MRC/DFID Concordat agreement and Fight for Sight. Additional funding supporting the study (equipment and field staff) was provided by the International Glaucoma Association and the British Council for the Prevention of Blindness. Dr Burton is supported by the Wellcome Trust (reference 098481/Z/12/Z). Peek Vision research is funded under the Commonwealth Eye Health Consortium by the Queen Elizabeth Diamond Jubilee Trust.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
Additional Contributions: Kevin Wing, PhD, provided statistical analysis, and Peter Blows and Arrianne O’Shea provided statistical analysis. No compensation was provided.
Create a personal account or sign in to: