Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork

Importance  Visual acuity is the most frequently performed measure of visual function in clinical practice and most people worldwide living with visual impairment are living in low- and middle-income countries.

Objective  To design and validate a smartphone-based visual acuity test that is not dependent on familiarity with symbols or letters commonly used in the English language.

Design, Setting, and Participants  Validation study conducted from December 11, 2013, to March 4, 2014, comparing results from smartphone-based Peek Acuity to Snellen acuity (clinical normal) charts and the Early Treatment Diabetic Retinopathy Study (ETDRS) logMAR chart (reference standard). This study was nested within the 6-year follow-up of the Nakuru Eye Disease Cohort in central Kenya and included 300 adults aged 55 years and older recruited consecutively.

Main Outcomes and Measures  Outcome measures were monocular logMAR visual acuity scores for each test: ETDRS chart logMAR, Snellen acuity, and Peek Acuity. Peek Acuity was compared, in terms of test-retest variability and measurement time, with the Snellen acuity and ETDRS logMAR charts in participants’ homes and temporary clinic settings in rural Kenya in 2013 and 2014.

Results  The 95% CI limits for test-retest variability of smartphone acuity data were ±0.029 logMAR. The mean differences between the smartphone-based test and the ETDRS chart and the smartphone-based test and Snellen acuity data were 0.07 (95% CI, 0.05-0.09) and 0.08 (95% CI, 0.06-0.10) logMAR, respectively, indicating that smartphone-based test acuities agreed well with those of the ETDRS and Snellen charts. The agreement of Peek Acuity and the ETDRS chart was greater than the Snellen chart with the ETDRS chart (95% CI, 0.05-0.10; P = .08). The local Kenyan community health care workers readily accepted the Peek Acuity smartphone test; it required minimal training and took no longer than the Snellen test (77 seconds vs 82 seconds; 95% CI, 71-84 seconds vs 73-91 seconds, respectively; P = .13).

Conclusions and Relevance  The study demonstrated that the Peek Acuity smartphone test is capable of accurate and repeatable acuity measurements consistent with published data on the test-retest variability of acuities measured using 5-letter-per-line retroilluminated logMAR charts.

Visual acuity (VA) is the most frequently performed measure of visual function in clinical practice. Visual acuity measurements are used to establish the need for clinical investigation and quantify changes in central vision over time.

Four percent of those who attend general practice in the United Kingdom do so with an eye problem¹ and a formal measure of VA should be part of each of these consultations.² Globally, 285 million people have visual impairment, with 80% having diseases with known curative or preventive treatment. However, most live in low-income countries with minimal access to detection and subsequent treatment.³

The Snellen chart⁴ is the most common method for the measurement of VA in ophthalmic and general practice; however, it is limited by the nongeometric progression in letter sizing from line to line and the inconsistent number of letters per line.⁵ Different letters or optotypes (standardized symbols for testing vision) have varying legibility at the same size and secondary effects, such as crowding, are known to affect the ability of the patient to determine optotypes correctly and therefore could lead to measurement bias.

The limitations of the Snellen chart have largely been overcome with the development of logMAR acuity charts,⁶ which are now frequently used in clinical research, such as the Early Treatment Diabetic Retinopathy Study (ETDRS) charts. Despite this improvement, the Snellen chart remains the dominant method for acuity testing in clinical practice.⁷ This may be owing to several factors including familiarity, a well-recognized scoring system, smaller chart size, and the speed of performing the test relative to the ETDRS chart test.

Mobile telephone technology has evolved rapidly in recent years. In 2013, an estimated 280 million (20%) of the 1.4 billion mobile telephones sold were smartphones and this proportion is expected to increase, particularly in low-income settings,⁸ where fixed-line technology has been leapfrogged straight to mobile technology,⁹ providing the potential to access health care without the previously required infrastructure.¹⁰

The medical community is embracing mobile technologies with its potential in health care information delivery, real-time patient monitoring, research data collection, and mobile telemedicine for the provision of expertise to remote locations.¹⁰

We hypothesized that a logMAR-style smartphone-based vision test (Peek Acuity), with a fast-testing algorithm, would allow measurements to be made in a clinically acceptable time, with greater precision and reliability than is possible with Snellen charts. Visual acuity results can be displayed in familiar Snellen chart notation (imperial or metric) or logMAR.

The Peek Acuity test was developed and compared, in terms of test-retest variability (TRV) and measurement time, with the Snellen chart and the ETDRS-based tumbling E logMAR chart (reference standard) in controlled and uncontrolled (real-world) settings in rural Kenya.

Box Section Ref ID

This study, conducted from December 11, 2013, to March 4, 2014, was nested within the 6-year follow-up of the Nakuru Eye Disease Cohort in central Kenya, a population-based study that recruited 5000 individuals from 100 clusters in 2007 selected through probability proportionate to the size of the clusters, with individuals sampled within clusters through compact segment sampling.^11,12 Follow-up of the participants was undertaken in 2013 and 2014.¹² Three hundred consecutive participants from the final 21 survey clusters who were undergoing reference measures of VA as part of the cohort follow-up were invited to enroll into this additional study of alternative VA measures. A temporary mobile eye clinic was set up in the center of each cluster. All participants examined in the study were aged 55 years and older.

The study adhered to the tenets of the Declaration of Helsinki¹³ and was approved by the ethics committees of the London School of Hygiene and Tropical Medicine and the African Medical and Research Foundation, Kenya. Approval was sought from administrative heads in each cluster, usually the village chief, who were given a copy of the consent form to read and pass on to those in the village.

Informed consent was obtained from all participants. The objectives of the study and examination process were explained in the local dialect in the presence of a witness. All participants gave written (or thumbprint) consent to participate.

The Peek Acuity application was written in Android and, for the purposes of this study, was used on a Galaxy SIII GT-I9300 (Samsung C&T Corp) running Android 4.0. The application was directly installed onto the test devices. Screen brightness was set to 100% within the application and all other options detailed here are built in.

Quiz Ref IDPeek Acuity follows the standard ETDRS chart design with a 5 × 5 grid optotype letter E displayed in 1 of 4 orientations (90°, 180°, 270°,and 0°). The participant points in the direction they perceive the arms of the E to be pointing and the tester uses the touch screen to swipe accordingly, translating the gestures from the patient. The tester is masked to the presented optotype and is unaware whether the participant is providing the correct response. This method reduces verbal or nonverbal clues, which may bias the result. Single optotypes are shown to reduce confusion; however, a bounding box is used to simulate the crowding effect of a standard ETDRS chart using a crowding bar, with thickness equivalent to the limb of the optotype, and spacing between optotype and crowding bar equal to that of half the total optotype size. This contour interaction format matches that used by the reference standard ETDRS chart. A stair-casing algorithm is used to simulate clinical practice for time efficiency.

Quiz Ref IDPeek Acuity offers standardized alternatives to count fingers, hand movements, and light perception. For count fingers, the application randomly presents between 1 and 4 bars and a correct or incorrect response is recorded on screen. For hand movement, a solid black box, half the width of the screen, moves backward and forward across the screen. For perception of light, Peek Acuity switches on the telephone’s LED flashlight and the participant is asked to identify if and when they see the light come on and off, with the option to assess for perception of projection direction. Test completion is indicated by a sound and vibration alert.

Visual acuity results can be displayed in logMAR or metric or imperial Snellen units based on user preference. An additional option, SightSim, presents a live video feed with a gaussian blur equivalent to the outcome of the vision test (eFigure 1 in the Supplement), which is of value in sharing the information with those not familiar with acuity scoring.

Paired VA measures were made in both the participant’s home and in the central clinic on 2 consecutive days. For all tests, the presenting acuity was measured, with habitual correction if worn. On day 1, a health care worker with basic eye care training and a field worker without formal health care training visited participants in their homes. The participants were tested using (1) Peek Acuity (logMAR units) at 2 m and (2) a reduced 3-m tumbling E Snellen chart (Sussex Vision) inside or close to the participant’s home (eFigure 2 in the Supplement). The order of the test was determined randomly by coin toss. The detailed testing procedures are described in the eAppendix in the Supplement.

On day 2, the participants seen on day 1 were reassessed in the cluster’s central clinic. The same personnel retested the study participants using (1) Peek Acuity (logMAR units) at 2 m and (2) a reduced 3-m tumbling E Snellen chart to allow for measures of TRV. The order of the test was determined randomly by coin toss. The ETDRS VA was measured using a back-illuminated 4-m ETDRS chart (Precision Vision Inc) (eFigures 3 and 4 in the Supplement) by an ophthalmic clinical officer, which is the reference standard for this study. All testing (ETDRS, Snellen, and Peek Acuity) at the different cluster clinic sites was standardized: conducted indoors, the test area was screened with blackout curtains, and there were controlled ambient light levels within a range of 80 to 300 lux (ISO-TECH: ILM1332A light meter), in accordance with British standards for acuity assessment.¹⁴

In total, 8 comparisons of the various VA measures in the different settings were made (Figure 1).

For any pairwise comparison of methods, the TRV was estimated as 95% CI limits of agreement (mean paired difference between measures ±1.96 SD). Histograms of the distribution of the test-retest and between-test method variability data suggested that the data were consistent with a normal distribution. Scatterplots of the observed TRV plotted against the average of the difference between the test and retest measurements suggested that there were no systematic associations between TRV and the underlying bias relating to level of acuity. Therefore, the Bland and Altman¹⁵ methods were used for (1) bias (mean and 95% CI of the mean) between ETDRS (reference test) and both Snellen and Peek Acuity scores and (2) TRV for the paired Snellen acuity and Peek Acuity scores. Mean time scores between Snellen and Peek Acuity tests were compared using paired t tests. Acuity scores were converted into a logMAR for data analysis. In the Supplement, eTable 1 outlines the logMAR scores used including where acuity was too poor to measure with optoytypes.¹⁶

The Peek Acuity study took place between December 2013 and March 2014. Of the 300 participants selected, 293 enrolled (98%; 135 men and 158 women). In total, 272 people (91%; mean age, 65 years; range, 55-97) were examined and completed all 3 tests in the central clinic on day 2. Of these, 233 (86%) were available and had also taken both VA tests at home on day 1.

The median VA measured by the ETDRS chart for all eyes tested (all levels of vision including those unable to read the ETDRS chart) was 0.23 logMAR, with a range of −0.2 to 4.0 logMAR (Snellen equivalents: median, 20/32; range, 20/12.5 to no light perception).

The results of the 8 pairwise comparisons of the right eye VA described here are presented in Table 1 and Figure 2, with results for the left eye available in the eAppendix in the Supplement (no difference between the right and left eyes was found; eTable 2 in the Supplement). The comparisons of clinic-based Snellen and clinic-based Peek Acuity measures with the ETDRS chart under the standardized clinic conditions indicated that Snellen tests showed a high degree of correlation with the ETDRS chart but that this was higher still with Peek Acuity (95% CI, 0.05-0.10; P = .08). The mean difference between the Peek Acuity measure in the clinic and the ETDRS chart measure was 0.011 logMAR units (95% CI, −0.014 to 0.035) and 0.032 logMAR units (95% CI, 0.010 to 0.054) for the right and left eyes, respectively. This was equivalent to less than 3 letters on a line difference when taking the upper confidence limit of the mean difference. The correlation (scatter) plots and Bland-Altman difference plots for these comparisons in the right eye are shown in Figure 3A.

Comparing Peek Acuity tested at home with ETDRS testing in the clinic, the mean difference between the Peek Acuity score at home and the ETDRS score was 0.055 logMAR (95% CI, 0.023-0.088) and 0.072 logMAR (95% CI, 0.039-0.105) for the right and left eyes, respectively, which is equivalent to 5 letters or 1 line of difference (Table 1; Figure 3B).

The Peek Acuity TRV (comparison 7 as indicated in Figure 1) performed by the same examiner on day 1 at home and on day 2 in the clinic had a high correlation and a small difference of averages (Table 1; Figure 3C).¹⁷

Mean testing time for both eyes on 126 study participants in whom testing time was measured was 82 seconds (95% CI, 73-91 seconds) with Snellen and 77 seconds (95% CI, 71-84 seconds) with Peek Acuity, showing no difference (P = .13).

Peek Acuity used at home by a community health care worker was 85% sensitive and 98% specific (eTable 3 in the Supplement) at detecting eyes with severe visual impairment (deemed locally as the surgical cutoff point for operable cataract; Snellen equivalent of ≤6/60) when compared with the ETDRS testing in controlled conditions. In addition, there was excellent agreement across World Health Organization vision categories between ETDRS and Peek Acuity when used at home (eTable 4 in the Supplement).

No adverse events from performing any of the acuity tests were reported.

The ubiquity of smartphones among health care professionals¹⁸ and increasing penetration, particularly in low- and middle-income countries, provide potential for delivering high-quality, objective, repeatable, and acceptable vision testing throughout the world.

With most of the world’s blind people living in low-income countries, the need for tools to increase early detection and appropriate referral are vital if the prevalence of blindness and visual impairment is to be reduced. In high-income settings, where primary care consultations are time pressured and confidence in diagnosing ophthalmic problems is low,¹⁹ accessible tools to provide reliable measures to guide management are vital. The referral of patients with ophthalmic symptoms from primary care, such as in general practice or accident and emergency, to specialist care should include a measure of acuity that is reliable and accessible and further testing in these contexts is encouraged.

Quiz Ref IDIn this study, we aimed to develop and validate a smartphone-based VA test appropriate for use in challenging circumstances, such as rural Africa, as well as being reliable enough for use in routine clinical practice in well-established health care systems. Overall, Peek Acuity performed well and the testing time was no slower or less repeatable than with the Snellen test, while being comparable in accuracy to the ETDRS chart. For clinical and population screening use, the TRV of acuity should be consistent across the acuity range and measurable in terms of lines or letters of change; measurement error obscures true clinical change and reduces the statistical power of clinical trials using acuity as a primary outcome measure.²⁰ Peek Acuity testing proved to be repeatable and consistent. Our findings also indicated that the reduced Snellen chart is a repeatable and time-efficient VA test that still has application in clinical and field settings.

In our study, the TRV of the Snellen chart was higher than in comparable studies,^5,21 which may have been owing to tightly defined end points (no part scores were given for part completion of a line).

Although multiple applications for the testing of VA on smartphones are available, to our knowledge, most have not been tested for repeatability or reliability against a reference standard.²² This study found Peek Acuity to be comparable with ETDRS-style chart, with similar TRV to that previously reported for other tests.^23,24 Key attributes and benefits for Peek Acuity are outlined in Table 2.

Low vision in participants who have VA below the level that can be measured on a chart are subject to assessment of vision that lacks a standardized approach and is open to considerable variability. In standard practice, if no optotypes are visible at the reduced distance, counting fingers is performed, followed by hand movements and finally differentiating between perception of light and no perception of light. In practice, this crucial measure of vision that may differentiate poor and good prognosis for treatment is often overlooked owing to these nonstandardized measures. Peek Acuity offers a standardized approach to testing such low levels of vision, which could be also performed on a tablet but was not assessed formally in this study.

Quiz Ref IDThe study population comprised older-aged Kenyan adults, who may not be representative of other populations and age groups, limiting the generalizability. Other studies are ongoing to determine the suitability of this tool in different contexts across a range of different handsets and operating systems (this study only assessed the device on multiple handsets of the same telephone model and operating system), including a school-aged population. Reflection from smartphone screens owing to bright sunlight can be problematic, although antiglare screens have been shown to reduce this limitation on other platforms.²⁵ Smartphones are on the whole more expensive than a basic Snellen chart but less expensive than a retroilluminated logMAR or Snellen chart. With the increased availability of low-cost smartphones and tablets, many health care workers may already own a device suitable for downloading multiple applications.²⁶

Concerns exist about data sharing and misuse with mobile health platforms, which should be integrated with systems compliant with approved standards for data sharing.

Owing to the size, weight, and power requirements, it was not possible to perform the ETDRS chart test in participants’ homes and, therefore, TRV of the ETDRS test was not assessed as with the Snellen and Peek Acuity tests. Therefore, we were unable to assess ETDRS TRV in this environment.

Nonhealth care workers who received specific training in how to use Peek Acuity performed the testing; further investigation of Peek Acuity’s usability with only inbuilt instructions is required.

During the early development phase, Peek Acuity was performed at 3 m. However, in the study setting, it was often not possible to find an indoor space of 3 m to conveniently perform the test. In conditions where the ambient light measure on the telephone was greater than 1000 lux, measures of Peek Acuity did not correlate well with the reference standard. With a 4.8-inch screen, 720 × 1280 pixels, and a viewing distance of 2 m, it is possible to measure acuity of 1.0 logMAR and 1.3 logMAR (Snellen equivalent of 20/200 and 20/400, respectively) when the testing distance is reduced to 1 m. Therefore, the testing distance and software algorithm were changed to 2 m. Following this change, more than 90% of participants were tested indoors in their homes. The smartphone’s inbuilt ambient light detector (which was accessed in the Peek Acuity application to give a mean lux reading per VA test) provides a warning that test conditions are not suitable if more than 1000 lux is detected.

The more widespread testing of VA in low- and middle-income countries is likely to lead to greater awareness of treatable eye disease with an increased uptake of preventive and curative treatments. In nonophthalmic departments, an easily accessible, easy-to-use, accurate, and reliable vision test could lead to increased assessment of vision testing in routine practice.²⁷

Additional applications to assess visual function and imaging of the eye make smartphones an attractive option for delivering ophthalmic assessment.^28,29 In settings where ophthalmic instrumentation or ophthalmic-trained personnel are limited, the ability to reliably measure a change in vision or detect abnormal vision, automation of stair-casing, masking of presented information, and generation of a jargon-free result greatly improve efficacy in the hands of minimally trained personnel. The inherent connectivity and global positioning system features of the device may ultimately lead to more people receiving timely and appropriate treatment.

Corresponding Author: Andrew Bastawrous, MRCOphth, International Centre for Eye Health, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, England (andrew.bastawrous@lshtm.ac.uk).

Submitted for Publication: January 16, 2015; final revision received April 1, 2015; accepted April 9, 2015.

Published Online: May 28, 2015. doi:10.1001/jamaophthalmol.2015.1468.

Author Contributions: Dr Bastawrous had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Bastawrous, Livingstone, Jordan, Kuper, Burton.

Acquisition, analysis, or interpretation of data: Bastawrous, Rono, Weiss, Kuper, Burton.

Drafting of the manuscript: Bastawrous, Rono, Livingstone, Jordan, Burton.

Critical revision of the manuscript for important intellectual content: Bastawrous, Weiss, Kuper, Burton.

Statistical analysis: Bastawrous, Weiss.

Obtained funding: Bastawrous, Kuper, Burton.

Administrative, technical, or material support: Bastawrous, Rono, Livingstone, Jordan.

Study supervision: Bastawrous, Kuper, Burton.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Bastawrous and Livingstone and Mr Jordan have a patent pending on the Peek Vision retina hardware. No other disclosures were reported. Dr Bastawrous affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Funding/Support: The Nakuru Eye Disease Cohort Study was jointly funded by the Medical Research Council (MRC) and the Department for International Development (DFID) under the MRC/DFID Concordat agreement and Fight for Sight. Additional funding supporting the study (equipment and field staff) were provided by the International Glaucoma Association and the British Council for the Prevention of Blindness. The Technology for Eye Health project is funded by the Queen Elizabeth Diamond Jubilee Trust.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Correction: This article was corrected online August 13, 2015, for errors in the abstract and Figure 3.