Feasibility of Use of a Smart Speaker to Administer Snellen Visual Acuity Examinations in a Clinical Setting | Ophthalmology | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Table.  Results From the Assessment of BCVA in the Smart Speaker and Human Administrations of the Snellen Visual Acuity Examination
Results From the Assessment of BCVA in the Smart Speaker and Human Administrations of the Snellen Visual Acuity Examination
1.
Sciuto  A, Saini  A, Forlizzi  J, Hong  JI. “Hey Alexa, what's up?”: a mixed-methods studies of in-home conversational agent usage. Published 2018. Accessed August 2019. https://dl.acm.org/doi/10.1145/3196709.3196772
2.
Laranjo  L, Dunn  AG, Tong  HL,  et al.  Conversational agents in healthcare: a systematic review.   J Am Med Inform Assoc. 2018;25(9):1248-1258. doi:10.1093/jamia/ocy072PubMedGoogle ScholarCrossref
3.
Kinsella  B. Amazon maintains smart speaker market share lead, apple rises slightly to 4.5%. Published 2018. Accessed December 2018. https://voicebot.ai/2018/09/12/amazon-maintains-smart-speaker-market-share-lead-apple-rises-slightly-to-4-5/
4.
Lynch  MG, Maa  AY.  The use of telemedicine to extend ophthalmology care.   JAMA Ophthalmol. 2016;134(5):543-544. doi:10.1001/jamaophthalmol.2016.0150PubMedGoogle ScholarCrossref
5.
Nelson  R.  Telemedicine and telehealth: the potential to improve rural access to care.   Am J Nurs. 2017;117(6):17-18. doi:10.1097/01.NAJ.0000520244.60138.1cPubMedGoogle ScholarCrossref
6.
Institute for Health Technology Transformation. Population health management: a roadmap for provider-based automation in a new era of healthcare. Published 2012. Accessed December 2018. https://www.exerciseismedicine.org/assets/page_documents/PHM%20Roadmap%20HL.pdf
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 1,388
    Citations 0
    Research Letter
    Health Informatics
    August 19, 2020

    Feasibility of Use of a Smart Speaker to Administer Snellen Visual Acuity Examinations in a Clinical Setting

    Author Affiliations
    • 1Department of Ophthalmology, Howard University College of Medicine, Washington, DC
    • 2Department of Cardiovascular Medicine, Mayo Clinic, Rochester, Minnesota
    JAMA Netw Open. 2020;3(8):e2013908. doi:10.1001/jamanetworkopen.2020.13908
    Introduction

    Despite the growing interest in and widespread adoption of smart speakers in many industries, these devices have not been evaluated for use in clinical settings.1,2 In this study, we assessed the feasibility, reliability, precision, and accuracy of using the most ubiquitous smart speaker, Amazon Alexa,3 to perform Snellen visual acuity examinations autonomously in a clinical setting.

    Methods

    This diagnostic study was conducted at the Howard University (HU) Department of Ophthalmology from April 11, 2018, to May 18, 2018, and was approved by the HU institutional review board. Participants were recruited through convenience sampling at the HU College of Medicine and the HU Ophthalmology Clinic and included those who met the eligibility criteria (eAppendix in the Supplement). Participation was strictly voluntary, written informed consent was obtained, and no compensation was provided. This study followed the Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) reporting guidelines.

    Each participant served as their control and underwent 2 versions of the Snellen visual acuity examination: the standard computerized examination administered by trained personnel (the control), and the examination administered by the smart speaker (the treatment). After the completion of both versions of the examination, all deidentified participant results were then sent to a cloud database for later review. All reported values are for both eyes (Table).

    The concordance correlation coefficient (CCC) and a Bland-Altman plot were used to assess agreement between the tests administered by the smart speaker and by personnel. The significance threshold was set at P = .05, and 2-sided testing was used. Statistical analysis was performed using SPSS statistical software version 19 (IBM Corp) from June 2018 to May 2020.

    Results

    A total of 74 participants were evaluated. Of those, only 68 participants (130 eyes) met the inclusion criteria and were evaluated in this study (median [interquartile range] age, 28 [24–36] years). Thirty-seven participants (54%) self-identified as male.

    The mean (SEM) logarithm of the minimum angle of resolution (LogMAR) scores of both eyes together were 0.063 (0.014) for the treatment (smart speaker) and 0.049 (0.013) for the control (human) groups (P = .04). The mean (SEM) difference between the smart speaker and human LogMAR scores for both eyes together was 0.014 (0.001) (approximately 1 letter per examination) with a 95% CI of 0.001 to 0.029 and Cronbach α of 0.928. The reported precision and accuracy for both eyes (CCC, 0.8585; 95% CI, 0.7837 to 0.9088) were 0.8722 and 0.9843, respectively (Table). Bland-Altman analysis revealed a mean difference of −0.01 (95% CI, −0.13 to 0.10 ) LogMAR units between the human- and smart speaker–reported values for both eyes (Table).

    Discussion

    To our knowledge, this study is the first to assess the feasibility of an autonomously administered visual acuity examination using a smart speaker device in a clinical setting. Although the differences in the reported data between Alexa and human values were statistically significant, in practice they were clinically insignificant and negligible at approximately 1 letter per examination. This study was limited by its small sample size. However, the results of this feasibility study are promising and suggest that the smart speaker performed comparably to the trained humans. With the growing adoption of telemedicine, smart speakers have the potential to be deployed in remote and low-resource settings for use by nonophthalmic personnel for screening and data collection.4,5 This could be explored in future studies.

    Although some may question the utility of automating visual acuity examinations, this is an important first step toward the automation and validation of more complex screening examinations. Automation will become an increasingly necessary tool to augment and extend the reach of eye care practitioners as they adopt a more population-health management approach in the near future, especially in remote, low-resource, or other underserved settings.5,6

    Back to top
    Article Information

    Accepted for Publication: June 8, 2020.

    Published: August 19, 2020. doi:10.1001/jamanetworkopen.2020.13908

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Ismail HO et al. JAMA Network Open.

    Corresponding Author: Leslie S. Jones, MD, Department of Ophthalmology, Howard University College of Medicine, 2041 Georgia Ave NW, Ste 2100, Washington, DC 20060 (L_s_Jones@howard.edu).

    Author Contributions: Dr Jones had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: All authors.

    Acquisition, analysis, or interpretation of data: Ismail, Moses, Tadrus, Mohamed.

    Drafting of the manuscript: Ismail, Moses, Mohamed.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Ismail, Mohamed.

    Administrative, technical, or material support: Ismail, Moses, Tadrus, Jones.

    Supervision: Ismail, Tadrus, Jones.

    Conflict of Interest Disclosures: Dr Jones reported receiving grants and personal fees from Allergan Inc outside the submitted work. No other disclosures were reported.

    Additional Contributions: Bassam Mohmaud, Mahamud Osman, and Siem Sium (all students at the Northern Virginia Community College) assisted Mr Ismail with the technical development of the software used in this study. They were not compensated for their contributions.

    References
    1.
    Sciuto  A, Saini  A, Forlizzi  J, Hong  JI. “Hey Alexa, what's up?”: a mixed-methods studies of in-home conversational agent usage. Published 2018. Accessed August 2019. https://dl.acm.org/doi/10.1145/3196709.3196772
    2.
    Laranjo  L, Dunn  AG, Tong  HL,  et al.  Conversational agents in healthcare: a systematic review.   J Am Med Inform Assoc. 2018;25(9):1248-1258. doi:10.1093/jamia/ocy072PubMedGoogle ScholarCrossref
    3.
    Kinsella  B. Amazon maintains smart speaker market share lead, apple rises slightly to 4.5%. Published 2018. Accessed December 2018. https://voicebot.ai/2018/09/12/amazon-maintains-smart-speaker-market-share-lead-apple-rises-slightly-to-4-5/
    4.
    Lynch  MG, Maa  AY.  The use of telemedicine to extend ophthalmology care.   JAMA Ophthalmol. 2016;134(5):543-544. doi:10.1001/jamaophthalmol.2016.0150PubMedGoogle ScholarCrossref
    5.
    Nelson  R.  Telemedicine and telehealth: the potential to improve rural access to care.   Am J Nurs. 2017;117(6):17-18. doi:10.1097/01.NAJ.0000520244.60138.1cPubMedGoogle ScholarCrossref
    6.
    Institute for Health Technology Transformation. Population health management: a roadmap for provider-based automation in a new era of healthcare. Published 2012. Accessed December 2018. https://www.exerciseismedicine.org/assets/page_documents/PHM%20Roadmap%20HL.pdf
    ×