eAppendix. Supplemental Methods
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Ismail HO, Moses AR, Tadrus M, Mohamed EA, Jones LS. Feasibility of Use of a Smart Speaker to Administer Snellen Visual Acuity Examinations in a Clinical Setting. JAMA Netw Open. 2020;3(8):e2013908. doi:10.1001/jamanetworkopen.2020.13908
Despite the growing interest in and widespread adoption of smart speakers in many industries, these devices have not been evaluated for use in clinical settings.1,2 In this study, we assessed the feasibility, reliability, precision, and accuracy of using the most ubiquitous smart speaker, Amazon Alexa,3 to perform Snellen visual acuity examinations autonomously in a clinical setting.
This diagnostic study was conducted at the Howard University (HU) Department of Ophthalmology from April 11, 2018, to May 18, 2018, and was approved by the HU institutional review board. Participants were recruited through convenience sampling at the HU College of Medicine and the HU Ophthalmology Clinic and included those who met the eligibility criteria (eAppendix in the Supplement). Participation was strictly voluntary, written informed consent was obtained, and no compensation was provided. This study followed the Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) reporting guidelines.
Each participant served as their control and underwent 2 versions of the Snellen visual acuity examination: the standard computerized examination administered by trained personnel (the control), and the examination administered by the smart speaker (the treatment). After the completion of both versions of the examination, all deidentified participant results were then sent to a cloud database for later review. All reported values are for both eyes (Table).
The concordance correlation coefficient (CCC) and a Bland-Altman plot were used to assess agreement between the tests administered by the smart speaker and by personnel. The significance threshold was set at P = .05, and 2-sided testing was used. Statistical analysis was performed using SPSS statistical software version 19 (IBM Corp) from June 2018 to May 2020.
A total of 74 participants were evaluated. Of those, only 68 participants (130 eyes) met the inclusion criteria and were evaluated in this study (median [interquartile range] age, 28 [24–36] years). Thirty-seven participants (54%) self-identified as male.
The mean (SEM) logarithm of the minimum angle of resolution (LogMAR) scores of both eyes together were 0.063 (0.014) for the treatment (smart speaker) and 0.049 (0.013) for the control (human) groups (P = .04). The mean (SEM) difference between the smart speaker and human LogMAR scores for both eyes together was 0.014 (0.001) (approximately 1 letter per examination) with a 95% CI of 0.001 to 0.029 and Cronbach α of 0.928. The reported precision and accuracy for both eyes (CCC, 0.8585; 95% CI, 0.7837 to 0.9088) were 0.8722 and 0.9843, respectively (Table). Bland-Altman analysis revealed a mean difference of −0.01 (95% CI, −0.13 to 0.10 ) LogMAR units between the human- and smart speaker–reported values for both eyes (Table).
To our knowledge, this study is the first to assess the feasibility of an autonomously administered visual acuity examination using a smart speaker device in a clinical setting. Although the differences in the reported data between Alexa and human values were statistically significant, in practice they were clinically insignificant and negligible at approximately 1 letter per examination. This study was limited by its small sample size. However, the results of this feasibility study are promising and suggest that the smart speaker performed comparably to the trained humans. With the growing adoption of telemedicine, smart speakers have the potential to be deployed in remote and low-resource settings for use by nonophthalmic personnel for screening and data collection.4,5 This could be explored in future studies.
Although some may question the utility of automating visual acuity examinations, this is an important first step toward the automation and validation of more complex screening examinations. Automation will become an increasingly necessary tool to augment and extend the reach of eye care practitioners as they adopt a more population-health management approach in the near future, especially in remote, low-resource, or other underserved settings.5,6
Accepted for Publication: June 8, 2020.
Published: August 19, 2020. doi:10.1001/jamanetworkopen.2020.13908
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Ismail HO et al. JAMA Network Open.
Corresponding Author: Leslie S. Jones, MD, Department of Ophthalmology, Howard University College of Medicine, 2041 Georgia Ave NW, Ste 2100, Washington, DC 20060 (L_s_Jones@howard.edu).
Author Contributions: Dr Jones had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Ismail, Moses, Tadrus, Mohamed.
Drafting of the manuscript: Ismail, Moses, Mohamed.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Ismail, Mohamed.
Administrative, technical, or material support: Ismail, Moses, Tadrus, Jones.
Supervision: Ismail, Tadrus, Jones.
Conflict of Interest Disclosures: Dr Jones reported receiving grants and personal fees from Allergan Inc outside the submitted work. No other disclosures were reported.
Additional Contributions: Bassam Mohmaud, Mahamud Osman, and Siem Sium (all students at the Northern Virginia Community College) assisted Mr Ismail with the technical development of the software used in this study. They were not compensated for their contributions.
Create a personal account or sign in to: