[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.  Planned Level of Care Before and After Use of an Online Symptom Checker
Planned Level of Care Before and After Use of an Online Symptom Checker

The 158 083 patient interactions with the online triage tool do not represent a unique patient count; rather, if patients used the tool twice, they would be included twice in the data. When declaring the preencounter intent and postencounter intent planned level of care, there were 2 additional choices (self-treatment and watch and wait) for the postencounter intent planned level of care.

1.
Semigran  HL, Linder  JA, Gidengil  C, Mehrotra  A.  Evaluation of symptom checkers for self diagnosis and triage: audit study.  BMJ. 2015;351:h3480. doi:10.1136/bmj.h3480PubMedGoogle ScholarCrossref
2.
Fraser  H, Coiera  E, Wong  D.  Safety of patient-facing digital symptom checkers.  Lancet. 2018;392(10161):2263-2264. doi:10.1016/S0140-6736(18)32819-8PubMedGoogle ScholarCrossref
3.
Armstrong  S.  The apps attempting to transfer NHS 111 online.  BMJ. 2018;360:k156. doi:10.1136/bmj.k156PubMedGoogle ScholarCrossref
4.
Slack  WV.  A history of computerized medical interviews.  MD Comput. 1984;1(5):52-59, 68.PubMedGoogle Scholar
5.
Ashwood  JS, Mehrotra  A, Cowling  D, Uscher-Pines  L.  Direct-to-consumer telehealth may increase access to care but does not decrease spending.  Health Aff (Millwood). 2017;36(3):485-491. doi:10.1377/hlthaff.2016.1130PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Research Letter
    Health Informatics
    December 27, 2019

    Association of Use of Online Symptom Checkers With Patients’ Plans for Seeking Care

    Author Affiliations
    • 1Medical College of Wisconsin, Milwaukee
    JAMA Netw Open. 2019;2(12):e1918561. doi:10.1001/jamanetworkopen.2019.18561
    Introduction

    Online triage tools are increasingly being adopted in health care to aid patients in identifying the appropriate care level.1 However, there is a lack of empirical evidence on how patients use virtual triage and whether these tools influence care-seeking behavior.1-3 Using data from a free online triage tool, we describe the common symptoms queried by users and analyze whether the tool was associated with the level of care that patients intended to seek.

    Methods

    This cross-sectional study used data from patient encounters using Buoy Health, a freely available virtual triage chatbot. The tool is based on conversational medical interviewing.4 A user is prompted to type 1 or more concerns or symptoms and minimal demographic information (age and sex) and then respond to questions. At the end of the process, the chatbot provides up to 3 possible explanations with recommended levels of care. We classified patients’ chief concerns into categories based on organ system, concern, and location. The data include completed encounters in which preencounter and postencounter intention data were provided for 158 083 encounters, representing 12% of encounters from March 27, 2018, through December 18, 2018. For each encounter, users were prompted for their intended level of care at the beginning (preencounter intent) and at the end of the encounter (postencounter intent). The preencounter intent and postencounter intent levels of care were ranked by relative urgency, from watch and wait (level 0) to emergency department (level 7). The Medical College of Wisconsin institutional review board considered this study using a deidentified data set non–human participants research. We used the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline to guide our reporting.

    We used descriptive statistics to examine intended behavior by comparing patients’ preencounter and postencounter intent level of care. To assess the tool’s association with care seeking, we removed uncertain responses and examined whether their intended level of care increased or decreased. We used 2-sided χ2 tests to examine whether associations were statistically significant. A P value of .05 was considered statistically significant.

    Results

    The mean (SD) participant age was 40 (16.5) years and 122 929 of 158 083 encounters (78%) involved female patients. The 158 083 encounters resulted in 281 165 reported symptoms, with 204 958 symptoms having identifiable organ systems and 279 948 having identifiable concern types. The most common organ system concerns were related to reproductive (37 669 of 204 958 reported symptoms [18%]), general (34 120 of 204 958 reported symptoms [17%]), and gastrointestinal (30 994 of 204 958 reported symptoms [15%]) systems. The most common symptom types were pain (122 363 of 279 948 reported symptoms [44%]), abnormal functioning (62 949 of 279 948 reported symptoms [22%]), and discharge (21 622 of 279 948 reported symptoms [8%]). The most common preencounter intended level of care was primary care (73 546 encounters [47%]), followed by uncertain (53 649 encounters [34%]), urgent (14 928 encounters [9%]), and emergency (14 813 encounters [9%]). We found that the proportion of patients who were uncertain about the level of care decreased from 34% to 21% (difference, 13.4%; 95% CI, 13.1%-13.7%; P < .001). Among patients who chose preencounter and postencounter intended level of care (ie, did not select “uncertain”), we found that in 28 961 interactions (32%), patients reduced the urgency of their intended level of care; in 58 844 (65%), urgency remained the same; and in 3243 (4%), urgency increased (χ215 = 77936.6960; P < .001) (Figure).

    Discussion

    In this data set of more than 150 000 patient interactions with an online triage tool, the urgency of patients’ intended level of care decreased in more than one-quarter of the cases and increased in approximately 1 in 20 cases, with the remaining patients remaining at the same level. The study suggests that virtual triage tools are associated with patients’ intended behavior when seeking care based on triage questions. Reduced urgency of intended level of care for many interactions is different from many other direct-to-consumer telehealth solutions, which have reported the same or an increased demand for care services.5

    This preliminary research has several limitations. We only included patient interactions that provided postencounter intention data. We only have information within the context of patient-tool interactions and were not able to observe whether users had multiple interactions, whether behavior was affected, or the appropriateness of the triage results.

    Future research should link patients’ use of such online tools with health care records to understand how patients use online tools in tandem with clinicians to manage their health.

    Back to top
    Article Information

    Accepted for Publication: November 8, 2019.

    Published: December 27, 2019. doi:10.1001/jamanetworkopen.2019.18561

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Winn AN et al. JAMA Network Open.

    Corresponding Author: Aaron N. Winn, PhD, Medical College of Wisconsin, 8701 Watertown Plank Rd, Milwaukee, WI 53226 (awinn@mcw.edu).

    Author Contributions: Drs Winn and Crotty had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Winn, Somai, Crotty.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Winn, Somai, Crotty.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Winn, Somai, Fergestrom.

    Administrative, technical, or material support: Winn, Crotty.

    Supervision: Winn, Somai.

    Conflict of Interest Disclosures: Dr Crotty reported serving as an advisor for and holding stock options in Buoy Health during the conduct of the study; and serving as medical director of Inception Health LLC and having a spouse who owns stock in Apple Inc, Johnson & Johnson, GlaxoSmithKline, General Electric, Microsoft, and IBM outside the submitted work. No other disclosures were reported.

    Funding/Support: Dr Winn is supported by the National Center for Research Resources, the National Center for Advancing Translational Sciences, and the office of the Director, National Institutes of Health, through grant KL2TR001438.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We thank Megan Lappe for her assistance with the article. She was not compensated for her contribution.

    References
    1.
    Semigran  HL, Linder  JA, Gidengil  C, Mehrotra  A.  Evaluation of symptom checkers for self diagnosis and triage: audit study.  BMJ. 2015;351:h3480. doi:10.1136/bmj.h3480PubMedGoogle ScholarCrossref
    2.
    Fraser  H, Coiera  E, Wong  D.  Safety of patient-facing digital symptom checkers.  Lancet. 2018;392(10161):2263-2264. doi:10.1016/S0140-6736(18)32819-8PubMedGoogle ScholarCrossref
    3.
    Armstrong  S.  The apps attempting to transfer NHS 111 online.  BMJ. 2018;360:k156. doi:10.1136/bmj.k156PubMedGoogle ScholarCrossref
    4.
    Slack  WV.  A history of computerized medical interviews.  MD Comput. 1984;1(5):52-59, 68.PubMedGoogle Scholar
    5.
    Ashwood  JS, Mehrotra  A, Cowling  D, Uscher-Pines  L.  Direct-to-consumer telehealth may increase access to care but does not decrease spending.  Health Aff (Millwood). 2017;36(3):485-491. doi:10.1377/hlthaff.2016.1130PubMedGoogle ScholarCrossref
    ×