[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.  Network Graph of the Top 2% of Linked Searches Performed by the Top 5 Practitioner Groups
Network Graph of the Top 2% of Linked Searches Performed by the Top 5 Practitioner Groups

Edge colors were assigned to the single practitioner group with the highest frequency of searches for the pair of terms; however, the search may have been prevalent in other practitioner groups as well. For visual clarity, the network was restricted to the top 2% of the most frequently searched pairs of terms, which represent 0.6% of the 208 374 unique search terms. In addition, terms with fewer than 4 other terms connected to it were suppressed in this graph. The size of the search term text corresponds to the frequency with which that term was searched. APPs indicates advanced practice providers.

Table 1.  Characteristics of Searches Performed From April 1, 2018, to May 15, 2019
Characteristics of Searches Performed From April 1, 2018, to May 15, 2019
Table 2.  Users Conducting Searches per Practitioner Group
Users Conducting Searches per Practitioner Group
Table 3.  Overall Top 20 Most Frequently Searched Terms From April 1, 2018, to May 15, 2019a
Overall Top 20 Most Frequently Searched Terms From April 1, 2018, to May 15, 2019a
Table 4.  Top 10 Most Frequently Searched Terms, by Top-Searching Practitioner Groupsa
Top 10 Most Frequently Searched Terms, by Top-Searching Practitioner Groupsa
1.
Friedberg  MW, Chen  PG, Van Busum  KR,  et al.  Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy.  Rand Health Q. 2014;3(4):1.PubMedGoogle Scholar
2.
Kroth  PJ, Morioka-Douglas  N, Veres  S,  et al.  Association of electronic health record design and use factors with clinician stress and burnout.  JAMA Netw Open. 2019;2(8):e199609. doi:10.1001/jamanetworkopen.2019.9609PubMedGoogle Scholar
3.
Street  RL  Jr, Liu  L, Farber  NJ,  et al.  Keystrokes, mouse clicks, and gazing at the computer: how physician interaction with the EHR affects patient participation.  J Gen Intern Med. 2018;33(4):423-428. doi:10.1007/s11606-017-4228-2PubMedGoogle ScholarCrossref
4.
Arndt  BG, Beasley  JW, Watkinson  MD,  et al.  Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations.  Ann Fam Med. 2017;15(5):419-426. doi:10.1370/afm.2121PubMedGoogle ScholarCrossref
5.
Sinsky  C, Colligan  L, Li  L,  et al.  Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties.  Ann Intern Med. 2016;165(11):753-760. doi:10.7326/M16-0961PubMedGoogle ScholarCrossref
6.
Tai-Seale  M, Olson  CW, Li  J,  et al.  Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine.  Health Aff (Millwood). 2017;36(4):655-662. doi:10.1377/hlthaff.2016.0811PubMedGoogle ScholarCrossref
7.
Toll  E.  A piece of my mind: the cost of technology.  JAMA. 2012;307(23):2497-2498. doi:10.1001/jama.2012.4946PubMedGoogle ScholarCrossref
8.
Wachter  R.  The Digital Doctor: Hope, Hype and Harm at the Dawn of Medicine’s Computer Age. New York, NY: McGraw-Hill Education; 2015.
9.
McDonald  CJ, Callaghan  FM, Weissman  A, Goodwin  RM, Mundkur  M, Kuhn  T.  Use of internist’s free time by ambulatory care electronic medical record systems.  JAMA Intern Med. 2014;174(11):1860-1863. doi:10.1001/jamainternmed.2014.4506PubMedGoogle ScholarCrossref
10.
Downing  NL, Bates  DW, Longhurst  CA.  Physician burnout in the electronic health record era: are we ignoring the real cause?  Ann Intern Med. 2018;169(1):50-51. doi:10.7326/M18-0139PubMedGoogle ScholarCrossref
11.
Embi  PJ, Weir  C, Efthimiadis  EN, Thielke  SM, Hedeen  AN, Hammond  KW.  Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators.  J Am Med Inform Assoc. 2013;20(4):718-726. doi:10.1136/amiajnl-2012-000946PubMedGoogle ScholarCrossref
12.
Linder  JA, Schnipper  JL, Middleton  B.  Method of electronic health record documentation and quality of primary care.  J Am Med Inform Assoc. 2012;19(6):1019-1024. doi:10.1136/amiajnl-2011-000788PubMedGoogle ScholarCrossref
13.
Gidwani  R, Nguyen  C, Kofoed  A,  et al.  Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial.  Ann Fam Med. 2017;15(5):427-433. doi:10.1370/afm.2122PubMedGoogle ScholarCrossref
14.
Walker  K, Ben-Meir  M, Dunlop  W,  et al.  Impact of scribes on emergency medicine doctors’ productivity and patient throughput: multicentre randomised trial.  BMJ. 2019;364:l121. doi:10.1136/bmj.l121PubMedGoogle ScholarCrossref
15.
Cohen  GR, Friedman  CP, Ryan  AM, Richardson  CR, Adler-Milstein  J.  Variation in physicians’ electronic health record documentation and potential patient harm from that variation.  J Gen Intern Med. 2019;34(11):2355-2367; Epub ahead of print. doi:10.1007/s11606-019-05025-3PubMedGoogle ScholarCrossref
16.
Natarajan  K, Stein  D, Jain  S, Elhadad  N.  An analysis of clinical queries in an electronic health record search utility.  Int J Med Inform. 2010;79(7):515-522. doi:10.1016/j.ijmedinf.2010.03.004PubMedGoogle ScholarCrossref
17.
Yang  L, Mei  Q, Zheng  K, Hanauer  DA.  Query log analysis of an electronic health record search engine.  AMIA Annu Symp Proc. 2011;2011:915-924.PubMedGoogle Scholar
18.
Roman  LC, Ancker  JS, Johnson  SB, Senathirajah  Y.  Navigation in the electronic health record: a review of the safety and usability literature.  J Biomed Inform. 2017;67:69-79. doi:10.1016/j.jbi.2017.01.005PubMedGoogle ScholarCrossref
19.
Nolan  ME, Siwani  R, Helmi  H, Pickering  BW, Moreno-Franco  P, Herasevich  V.  Data use and navigation patterns among medical ICU clinicians during electronic chart review.  Appl Clin Inform. 2017;8(4):1117-1126. doi:10.4338/ACI-2017-06-RA-0110PubMedGoogle ScholarCrossref
20.
Sheehy  AM, Weissburg  DJ, Dean  SM.  The role of copy-and-paste in the hospital electronic health record.  JAMA Intern Med. 2014;174(8):1217-1218. doi:10.1001/jamainternmed.2014.2110PubMedGoogle ScholarCrossref
21.
Graber  ML, Siegal  D, Riah  H, Johnston  D, Kenyon  K.  Electronic health record-related events in medical malpractice claims.  J Patient Saf. 2019;15(2):77-85. doi:10.1097/PTS.0000000000000240PubMedGoogle ScholarCrossref
22.
Nuti  SV, Wayda  B, Ranasinghe  I,  et al.  The use of google trends in health care research: a systematic review.  PLoS One. 2014;9(10):e109583. doi:10.1371/journal.pone.0109583PubMedGoogle Scholar
23.
Biron  P, Metzger  MH, Pezet  C, Sebban  C, Barthuet  E, Durand  T.  An information retrieval system for computerized patient records in the context of a daily hospital practice: the example of the Léon Bérard Cancer Center (France).  Appl Clin Inform. 2014;5(1):191-205. doi:10.4338/ACI-2013-08-CR-0065PubMedGoogle ScholarCrossref
24.
Liang  LL.  Connected for Health: Using Electronic Health Records to Transform Care Delivery. Hoboken, NJ: Pfeiffer; 2010.
25.
Epic. Chart Search Setup and Support Guide. Verona, WI: Epic; 2018. https://userweb.epic.com. Accessed January 30, 2020.
26.
Observational Health Data Sciences and Informatics. Data standardization. https://ohdsi.org/data-standardization/. Published 2019. Accessed June 18, 2019.
27.
Fruchterman  TM, Reingold  EM.  Graph drawing by force-directed placement.  Softw Pract Exper. 1991;21(11):1129-1164. doi:10.1002/spe.4380211102Google ScholarCrossref
28.
Wisner  K, Lyndon  A, Chesla  CA.  The electronic health record’s impact on nurses’ cognitive work: an integrative review.  Int J Nurs Stud. 2019;94:74-84. doi:10.1016/j.ijnurstu.2019.03.003PubMedGoogle ScholarCrossref
29.
Zalis  M, Harris  M.  Advanced search of the electronic medical record: augmenting safety and efficiency in radiology.  J Am Coll Radiol. 2010;7(8):625-633. doi:10.1016/j.jacr.2010.03.011PubMedGoogle ScholarCrossref
30.
King  AJ, Cooper  GF, Hochheiser  H, Clermont  G, Hauskrecht  M, Visweswaran  S.  Using machine learning to predict the information seeking behavior of clinicians using an electronic medical record system.  AMIA Annu Symp Proc. 2018;2018:673-682.PubMedGoogle Scholar
31.
Longhurst  CA, Davis  T, Maneker  A,  et al; Arch Collaborative.  Local investment in training drives electronic health record user satisfaction.  Appl Clin Inform. 2019;10(2):331-335. doi:10.1055/s-0039-1688753PubMedGoogle ScholarCrossref
32.
Saleem  JJ, Flanagan  ME, Wilck  NR, Demetriades  J, Doebbeling  BN.  The next-generation electronic health record: perspectives of key leaders from the US Department of Veterans Affairs.  J Am Med Inform Assoc. 2013;20(e1):e175-e177. doi:10.1136/amiajnl-2013-001748PubMedGoogle ScholarCrossref
33.
Senathirajah  Y, Kaufman  D, Bakken  S.  User-composable electronic health record improves efficiency of clinician data viewing for patient case appraisal: a mixed-methods study.  EGEMS (Wash DC). 2016;4(1):1176. doi:10.13063/2327-9214.1176PubMedGoogle Scholar
34.
Institute of Medicine.  Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press; 2012.
35.
Cimino  JJ.  Putting the “why” in “EHR”: capturing and coding clinical cognition.  J Am Med Inform Assoc. 2019;26(11):1379-1384. doi:10.1093/jamia/ocz125PubMedGoogle ScholarCrossref
36.
Sutton  JM, Ash  SR, Al Makki  A, Kalakeche  R.  A daily hospital progress note that increases physician usability of the electronic health record by facilitating a problem-oriented approach to the patient and reducing physician clerical burden.  Perm J. 2019;23:23.PubMedGoogle Scholar
37.
Weiner  M.  Forced Inefficiencies of the electronic health record.  J Gen Intern Med. 2019;34(11):2299-2301; Epub ahead of print. doi:10.1007/s11606-019-05281-3PubMedGoogle ScholarCrossref
38.
Prager  JM, Liang  JJ, Devarakonda  MV.  SemanticFind: locating what you want in a patient record, not just what you ask for.  AMIA Jt Summits Transl Sci Proc. 2017;2017:249-258.PubMedGoogle Scholar
39.
Hanauer  DA, Wu  DT, Yang  L,  et al.  Development and empirical user-centered evaluation of semantically-based query recommendation for an electronic health record search engine.  J Biomed Inform. 2017;67:1-10. doi:10.1016/j.jbi.2017.01.013PubMedGoogle ScholarCrossref
40.
Hanauer  DA, Mei  Q, Law  J, Khanna  R, Zheng  K.  Supporting information retrieval from electronic health records: a report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE).  J Biomed Inform. 2015;55:290-300. doi:10.1016/j.jbi.2015.05.003PubMedGoogle ScholarCrossref
41.
Kannampallil  TG, Jones  LK, Patel  VL, Buchman  TG, Franklin  A.  Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings.  J Am Med Inform Assoc. 2014;21(e2):e249-e256. doi:10.1136/amiajnl-2013-002615PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Health Informatics
    March 6, 2020

    Assessment of Electronic Health Record Search Patterns and Practices by Practitioners in a Large Integrated Health Care System

    Author Affiliations
    • 1Division of Research, Kaiser Permanente Northern California, Oakland
    • 2Department of Medicine, University of California, San Francisco
    • 3The Permanente Medical Group, Oakland, California
    JAMA Netw Open. 2020;3(3):e200512. doi:10.1001/jamanetworkopen.2020.0512
    Key Points español 中文 (chinese)

    Question  What are the electronic health record search patterns and practices among electronic health record users?

    Findings  In this cross-sectional study of 12 313 047 patient electronic health record search activities, physicians and pharmacists conducted the most searches, with statin, colonoscopy, and pft being the most frequently searched terms. Search term networks revealed how users bridged together data across various domains relevant to a single clinical interaction.

    Meaning  Our results suggest that opportunities exist for improving the clinical use of the electronic health record by identifying potential areas to customize information retrieval based on diverse user roles and linked search approaches.

    Abstract

    Importance  The electronic health record (EHR) is a source of practitioner dissatisfaction in part because of challenges with information retrieval. To improve data accessibility, a better understanding of practitioners’ information needs within individual patient records is needed.

    Objective  To assess EHR users’ searches using data from a large integrated health care system.

    Design, Setting, and Participants  This retrospective cross-sectional analysis used EHR search data from Kaiser Permanente Northern California, an integrated health care delivery system with more than 4.4 million members. Users’ EHR search activity data were obtained from April 1, 2018, to May 15, 2019.

    Main Outcomes and Measures  Search term frequency was grouped by user and practitioner types. Network analyses were performed of co-occurring search terms within a single search episode, and centrality measures for search terms (degree and betweenness centrality) were calculated.

    Results  A total of 12 313 047 search activities (including 4 328 330 searches and 7 984 717 result views) conducted by 34 735 unique users within 977 160 unique patient EHRs were identified. In aggregate, users searched for 208 374 unique search terms and conducted a median of 4 searches (interquartile range, 1-28 searches). Of all 97 367 active EHR users, 34 735 (35.7%) conducted at least 1 search. However, of all 12 968 active EHR physician users, 9801 (75.6%) conducted at least 1 search, and of all 1908 active pharmacist users, 1402 (73.5%) conducted at least 1 search. The top 3 most commonly searched terms were statin (75 017 searches [1.7%]), colonoscopy (73 545 [1.7%]), and pft (54 990 [1.3%]). However, wide variation in top searches were noted across practitioner groups. Terms searched most often with another term in a single linked search episode included statin, lisinopril, colonoscopy, gabapentin, and aspirin.

    Conclusions and Relevance  Although physicians and pharmacists were the most active users of EHR searches, search volume and frequently searched terms varied considerably by and within user role. Further customization of the EHR interface may help leverage users’ search content and patterns to improve targeted information retrieval.

    Introduction

    The electronic health record (EHR) can be a major source of dissatisfaction for practitioners, contributing to burnout and stress.1,2 Practitioners and patients largely agree that practitioners spend too much time in front of the computer, often at the expense of engaging with the patient.3-8 Two major reasons for the excessive time spent in the EHR system include burdensome documentation demands and the inaccessibility of data.2,4,9 Although numerous studies10-15 have examined documentation burden and practices, the challenges of EHR information retrieval in clinical practice remain poorly understood.16-19 Studying information retrieval is critically important because variability in documentation practices, usability challenges, and propagation of erroneous information15,20 result in the need to navigate large volumes of low-value EHR data to locate clinically actionable patient information. Failure to identify relevant clinical data can lead to overuse of treatments or procedures, provision of low-quality care, and occurrence of medical errors.15,21

    An emerging mechanism used for patient information retrieval is the EHR search feature. Analyzing EHR user search patterns can reveal information needs by showing the types of information that may be difficult to locate through other means. In other applications, search data, such as Google Trends, are often used to study the types of information sought by populations.22 Widely available, comprehensive patient record search features are relatively new to the EHR, and their use has not been extensively studied.16,17,23 In addition to identifying information needs, examining searches by EHR user type can reveal the overall demand or penetration of the search functionality. Exploring terms frequently searched together can highlight patterns that can then be used to make data gathering in the EHR more intuitive and user-friendly. In this study, we analyzed more than 12 million EHR search activities from a large, highly integrated health care delivery system to assess search behaviors and identify opportunities to improve targeted information retrieval in clinical practice across a diverse set of users.

    Methods

    For this cross-sectional study, we evaluated EHR search data from Kaiser Permanente Northern California (KPNC), an integrated health care delivery system including 21 hospitals and serving 4.4 million members. KPNC began deploying a single comprehensive EHR system (Epic) in 2007. The EHR, known internally as KP HealthConnect, is used across all KPNC inpatient and outpatient facilities.24 KPNC determined that this study did not meet the definition of human subject research, and the study was therefore exempted from institutional review board approval. Patient and user identifications were blinded before analysis. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Starting in 2015, the Chart Search feature was made available in KP HealthConnect and allows users to search for specific terms within an individual patient’s EHR in structured and unstructured data (eg, clinical notes, laboratory data, medications, imaging, procedures, orders, allergies, immunizations, and problem list fields).25 Searches can match on the exact terms entered and synonyms generated through Systematized Nomenclature of Medicine ontologies. Results of the search are displayed in a list format with some contextual information when the search successfully finds a matching term or an indication that no results match the term. Users can click a positive search result, which brings them to the selected location of the term in the patient’s record.

    EHR Search Data

    We retrospectively evaluated KPNC EHR audit log data for search activities between April 1, 2018, and May 15, 2019. A search activity includes the search term itself or a view of a search result (click or hover). For each search activity, we retrieved the following information: search term; time stamp; blinded user and patient identifiers; search session identifier; and, when applicable, licensed practitioner type (eg, physician, nurse, pharmacist, or physician trainees) and specialty type (eg, internal medicine). For analysis, we further grouped licensed practitioners by discipline: physicians (including podiatrists and optometrists); nurses (registered nurses and licensed vocational nurses); nonphysician mental health practitioners (eg, social workers, therapists, and psychologists); pharmacists; rehabilitation therapists (eg, occupational therapists, physical therapists, and speech therapists); advanced practice providers (APPs; eg, nurse practitioners and physician assistants); medical assistants; and other. Note that not all EHR users are practitioners licensed to provide clinical care to patients (eg, administrative personnel and researchers).

    We used search term data in their recorded format to reflect how users were conducting searches but converted all terms to lowercase and removed nonalphanumeric characters and spaces. We produced summary statistics (eg, median and interquartile range [IQR]) for frequency of searches by user (using a denominator of all active EHR users defined as a user with any audit log activity data), practitioner group and time of day, and result views per search.

    To identify the most frequently searched clinical domains, we aggregated search terms using the Observational Health Data Sciences and Informatics (OHDSI) Standard Vocabulary synonym and domain tables.26 Because the same search term could match multiple OHDSI domains, we sequentially categorized terms within domains in the following order: condition, procedure, drug, observation, measurement or measured value, and other.

    Linked Search Networks

    To better understand search patterns, we examined terms that were searched together within a single search episode in the same patient record. We created search episodes by grouping searches conducted by the same user for the same patient in a single EHR session (based on unique session identification numbers). We used the phrase linked search terms to indicate a pair of search terms that occurred within the same search episode. We displayed these linked search terms using network graphs27 for the top-searching practitioner groups (physicians, pharmacists, nurses, nonphysician mental health, and APPs) and top-searching physician specialty (internal medicine). To identify highly interconnected terms (terms that appeared in multiple linked search term pairs across users), we calculated weighted degree and betweenness centrality metrics for each included term. Degree centrality describes the number of terms to which that term was linked during search episodes in our data set, whereas betweenness describes how often a node (search term) is in the shortest path between 2 other nodes.

    Statistical Analysis

    We used a χ2 test to examine the association between practitioner group and use of the search feature. A 2-sided P < .05 was considered to be statistically significant. All analyses were conducted in R, version 3.5.2 (The R Project for Statistical Computing), using packages including ggplot2, ggnet, and tidygraph.

    Results

    During the 13-month study period, 34 735 unique users performed 12 313 047 search activities, including 4 328 330 searches and 7 984 717 result views within 977 160 unique patient EHRs. In aggregate, users searched for 208 374 unique search terms. For 1 647 643 searches (38.1%), no results were viewed (click or hover), and the median number of results viewed per search was 1 (IQR, 0-3 result views).

    Table 1 displays the overall search characteristics. Users of the search feature conducted a median of 4 searches (IQR, 1-28 searches). The top 5% of search feature users searched more than 584 times during the study period, with a maximum of 51 784 searches by a single user. Grouped by time of day, most searches (2 178 074 [50.3%]) were conducted between noon and 6 pm. In eFigure 1 in the Supplement, we present search frequency by hour of the day for each practitioner group.

    Search Characteristics by Practitioner Groups

    The 5 practitioner groups conducting the most searches were physicians (2 042 804 [47.2%]), followed by pharmacists (689 852 [15.9%]) and nurses (239 379 [5.5%]). Of all 97 367 EHR users with any audit log activity data, 34 735 (35.7%) conducted at least 1 search. However, among 12 968 physicians, 9801 (75.6%) conducted at least 1 search, and among 1908 pharmacists, 1402 (73.5%) conducted at least 1 search (Table 2). The χ2 test demonstrated a significant association between clinician group and use of the search feature (χ24, 7357.6; n = 24 587; P < .001). Those who conducted a search were more likely to be a physician or pharmacist than those who did not conduct a search.

    Of those conducting at least 1 search, physicians conducted a median of 30 searches each (IQR, 5-179 searches), pharmacists conducted a median of 76 searches (IQR, 5-587 searches), and nurses conducted a median of 2 searches (IQR, 1-4 searches). Among searches by physicians, the specialties conducting the most searches were internal medicine (931 067 [45.6%]), family practice (226 472 [11.1%]), and psychiatry (130 317 [6.4%]).

    Most Frequently Searched Terms and Domains

    Table 3 displays the most frequently searched terms overall; statin (75 017 searches [1.7%]), colonoscopy (73 545 [1.7%]), and pft (54 990 [1.3%]) accounted for 4.7% of all searches. Overall, statin was searched by 2018 users (5.8% of users who conducted a search), colonoscopy was searched by 4130 (11.9%), and pft was searched by 2561 (7.4%). However, the most frequently searched terms differed substantially by practitioner groups, and no single term appeared in the top 10 most frequent searches of more than 2 practitioner groups (Table 4). For example, physicians most commonly searched for colonoscopy, pft, and ppd, whereas pharmacists most frequently searched medications, including statin, metformin, and lisinopril. When grouped by OHDSI domains, physicians most frequently searched for drugs and conditions (eg, sleep apnea and asthma), pharmacists most frequently searched for drugs, and nurses searched for observations (eg, A1c and mood) and conditions (eFigure 2 in the Supplement).

    Linked Searches Within Episodes

    The 4 328 330 searches in this study were conducted within 2 502 802 unique episodes (a single EHR session by a single user for the same patient). The Figure displays the network graph of linked search terms for the top 5 practitioner groups (physicians, pharmacists, nurses, mental health practitioners, and APPs). Dense search networks of linked search terms clustered around clinical topics, including cardiometabolic disease, mental health, routine diagnostic testing, respiratory disease, cognitive testing, and end-of-life care. For example, APP searches showed networks for procedures and diagnostic workups (eg, stress test, catheterization, and echo). Linked searches conducted by nonphysician mental health practitioners included pairs of terms such as social work and social services. Pharmacists commonly searched medications (eg, lisinopril, statin, and aspirin) in tandem with specific conditions (eg, heart attack, angina, and palpitation). Pharmacists also searched for changes in drug regimens using terms such as discontinue or decrease with medication names. eFigure 3 in the Supplement presents a focused network graph for internal medicine physicians (the specialty conducting the most searches).

    Search Terms by Centrality

    The most highly interconnected terms are listed in the eTable in the Supplement. Those terms with the highest degree centrality included statin, lisinopril, colonoscopy, gabapentin, and aspirin. Key terms bridging distinct clusters, based on betweenness centrality, were aspirin, statin, colonoscopy, alcohol, and gabapentin. For example, alcohol bridged search concepts related to screening practices, suicide, and social services.

    Discussion

    Within slightly more than 1 year of data from a large, integrated health care delivery system, more than 12 million EHR search activities were conducted by more than 34 000 users for nearly 1 million patients. We found that the EHR search feature was used by nearly three-quarters of physicians and pharmacists, but the median number of searches per search feature user was low. The high degree of variability in top search terms across practitioner groups underscores the wide diversity of information needs. Our evaluation of search networks also situated individual terms in their clinical context and revealed terms with high degrees of interconnectedness. Together, these results suggest that search activity data may be used to inform efforts to facilitate EHR information retrieval.

    Although most physicians and pharmacists used the search feature, only approximately one-third of all active EHR users conducted any search. Among those who had conducted a search, there was a median of only 4 searches per user for slightly more than a year. Search feature use was especially low among nurses, who represented the single largest group of practitioners. These findings may indicate that EHR design is better suited to nurses’ information needs; however, other literature does not support this hypothesis.28 Instead, we suspect that this finding reflects an opportunity for further penetration of the search feature among users and highly varied workflows among practitioners. For example, the EHR may be the primary source of contextual information for pharmacists who lack direct patient contact. To support practitioners, such as pharmacists, who frequently use the search feature to locate similar information, structured searches that can be customized could improve strategies often reused across patient records.29 User-specific customization could also be accomplished by tailoring EHR displays based on the user’s prior searches and information retrieval history.30

    We found that the most frequently searched terms were also common reasons for receiving health care (eg, statin, colonoscopy, and pft). Although the most frequently searched terms were used in less than 2% of searches, within practitioner groups, the top searches were less sparse. For example, nearly 8% of pharmacists’ searches were for statin, and the top 10 search terms by pharmacists composed nearly 20% of all their searches. Identification of highly searched terms provides an empirical method for determining concepts that would benefit from improved navigation pathways in the EHR. Variation in search patterns across practitioner groups suggests that there may be opportunities for further customization of the EHR interface by practitioner type. Customization is an important determinant of EHR satisfaction and has been recognized as a critical consideration in EHR redesign.31,32

    The contents of the searches in our study also underscore the challenges that practitioners face when attempting to develop a comprehensive clinical picture from siloed information within the EHR (ie, information fragmentation33,34). The network graphs showed linked searches across diverse domains (eg, medications, diagnoses, notes, and images) on a single clinical problem. Additional efforts to empirically link relevant items across domains may further improve EHR usability.35,36 For example, aspirin was searched with terms such as statin, catheterization, metformin, and pradaxa, which makes intuitive clinical sense and could be combined within a cardiometabolic disease and treatment pathway. In addition, a substantial portion of linked searches involved mental health concerns (eg, drugs, services, substance use, family history, and cognitive function), suggesting the need for a clustered presentation of mental health–related information.

    Although the search feature is a key tool for efficiently retrieving information from the mass of EHR data, we identified several uses of the search feature that seem to be forced inefficiencies37 of poor EHR design. For example, pharmacists’ searches for terms such as decrease and discontinue highlight limitations in their ability to pinpoint medication changes over time. A timeline display of medication orders, administrations, and refills could help pharmacists understand prescribing history more readily. Mental health practitioners commonly searched synonyms for social work (eg, lcsw, msw, or social services) together within a single search episode, suggesting specific challenges finding information on social services. The search feature may also need improved capabilities for retrieving synonyms and abbreviations, as well as related clinical information.38,39

    The present study contributes to the scant literature on EHR searching, which focuses primarily on homegrown solutions for searching within or across patient records.16,17,23,40 One of the only prior studies16 of searches within patient records included 2207 EHR searches by 436 unique users at a single institution and found that the most common searches were for laboratory or test results. In the present study, top searches across all practitioner groups were dominated by drugs, procedures, conditions, and clinical services, such as palliative care. This difference may be related to the size and scope of our study and the near-decade of EHR advancements since the publication of the prior study.16 A number of studies18,19,30,41 have explored practitioners’ information-seeking practices in general and EHR navigation, typically in small samples of practitioners and often using methods such as direct observation.

    The present study adds to the current information on EHR searches because of the breadth of the data, which provide a comprehensive overview of frequency and content of searches within patient records. However, we were unable to determine contextual information, such as why searches were conducted and whether users were satisfied with the results. For example, no results were viewed (click or hover) in more than one-third of searches, which might indicate that the search feature fails to meet users’ information needs. Alternatively, the finding may indicate that the search was successful because the specific term never appeared in the patient record or because it did but did not require further examination.

    Opportunity exists for future investigations to contextualize these findings using additional data sources and research methods. The EHR audit log data (including tasks performed before and after the search), patient-level data, and practitioner interviews and observations could help explain why a search was conducted and whether the search was successful. For example, practitioners could be interviewed or surveyed about barriers to locating information related to the top search terms identified in this study. Temporal data could be used to examine how search habits change over time as practitioners learn the tool or gain clinical expertise. There is also ample opportunity for deeper exploration of search patterns of specific practitioner subgroups.

    Limitations

    This study has limitations. First, we did not attempt to address spelling errors or abbreviations when aggregating search terms. The summary statistics therefore likely underestimate search term frequency. Second, the data represent searches within the Epic Chart Search feature and may not be generalizable to other EHR systems. Third, KPNC members receive most of their care within the KPNC system, and as a result, practitioners’ EHR search practices may not be generalizable. However, given the widespread use of Epic and the increasing presence of integrated health care systems in the United States, the findings may be increasingly relevant.

    Conclusions

    In this study, we identified potential opportunities to improve EHR navigation by examining the searches performed by EHR users within patient records. The findings suggest that key areas for improving EHR information retrieval include linking information related to key clinical areas as well as further customization of the EHR interface across user types. Our findings from these large-scale search data appear to offer an empirical approach to improving EHR design based directly on user information needs.

    Back to top
    Article Information

    Accepted for Publication: January 14, 2020.

    Published: March 6, 2020. doi:10.1001/jamanetworkopen.2020.0512

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Ruppel H et al. JAMA Network Open.

    Corresponding Author: Halley Ruppel, PhD, RN, Division of Research, Kaiser Permanente Northern California, 2000 Broadway Ave, Oakland, CA 94612 (halley.w.ruppel@kp.org).

    Author Contributions: Dr Ruppel had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Ruppel, Bhardwaj, Flagg, Liu.

    Acquisition, analysis, or interpretation of data: Ruppel, Bhardwaj, Manickam, Adler-Milstein, Ballesca, Liu.

    Drafting of the manuscript: Ruppel, Liu.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Ruppel, Bhardwaj, Manickam.

    Obtained funding: Liu.

    Administrative, technical, or material support: Bhardwaj, Flagg, Ballesca, Liu.

    Supervision: Ruppel, Adler-Milstein, Liu.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: This study was supported by funding from The Permanente Medical Group Delivery Science Fellowship Program (Dr Ruppel), grant K12HD052163 (Dr Ruppel) and grant R35GM128672 (Dr Liu) from the National Institutes of Health, and grant 8040 (Drs Adler-Milstein and Liu) from the Gordon and Betty Moore Foundation.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: Brian Hoberman, MD, Kaiser Permanente Northern California, provided electronic health record expertise and reviewed the manuscript. He was not compensated for his work.

    References
    1.
    Friedberg  MW, Chen  PG, Van Busum  KR,  et al.  Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy.  Rand Health Q. 2014;3(4):1.PubMedGoogle Scholar
    2.
    Kroth  PJ, Morioka-Douglas  N, Veres  S,  et al.  Association of electronic health record design and use factors with clinician stress and burnout.  JAMA Netw Open. 2019;2(8):e199609. doi:10.1001/jamanetworkopen.2019.9609PubMedGoogle Scholar
    3.
    Street  RL  Jr, Liu  L, Farber  NJ,  et al.  Keystrokes, mouse clicks, and gazing at the computer: how physician interaction with the EHR affects patient participation.  J Gen Intern Med. 2018;33(4):423-428. doi:10.1007/s11606-017-4228-2PubMedGoogle ScholarCrossref
    4.
    Arndt  BG, Beasley  JW, Watkinson  MD,  et al.  Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations.  Ann Fam Med. 2017;15(5):419-426. doi:10.1370/afm.2121PubMedGoogle ScholarCrossref
    5.
    Sinsky  C, Colligan  L, Li  L,  et al.  Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties.  Ann Intern Med. 2016;165(11):753-760. doi:10.7326/M16-0961PubMedGoogle ScholarCrossref
    6.
    Tai-Seale  M, Olson  CW, Li  J,  et al.  Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine.  Health Aff (Millwood). 2017;36(4):655-662. doi:10.1377/hlthaff.2016.0811PubMedGoogle ScholarCrossref
    7.
    Toll  E.  A piece of my mind: the cost of technology.  JAMA. 2012;307(23):2497-2498. doi:10.1001/jama.2012.4946PubMedGoogle ScholarCrossref
    8.
    Wachter  R.  The Digital Doctor: Hope, Hype and Harm at the Dawn of Medicine’s Computer Age. New York, NY: McGraw-Hill Education; 2015.
    9.
    McDonald  CJ, Callaghan  FM, Weissman  A, Goodwin  RM, Mundkur  M, Kuhn  T.  Use of internist’s free time by ambulatory care electronic medical record systems.  JAMA Intern Med. 2014;174(11):1860-1863. doi:10.1001/jamainternmed.2014.4506PubMedGoogle ScholarCrossref
    10.
    Downing  NL, Bates  DW, Longhurst  CA.  Physician burnout in the electronic health record era: are we ignoring the real cause?  Ann Intern Med. 2018;169(1):50-51. doi:10.7326/M18-0139PubMedGoogle ScholarCrossref
    11.
    Embi  PJ, Weir  C, Efthimiadis  EN, Thielke  SM, Hedeen  AN, Hammond  KW.  Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators.  J Am Med Inform Assoc. 2013;20(4):718-726. doi:10.1136/amiajnl-2012-000946PubMedGoogle ScholarCrossref
    12.
    Linder  JA, Schnipper  JL, Middleton  B.  Method of electronic health record documentation and quality of primary care.  J Am Med Inform Assoc. 2012;19(6):1019-1024. doi:10.1136/amiajnl-2011-000788PubMedGoogle ScholarCrossref
    13.
    Gidwani  R, Nguyen  C, Kofoed  A,  et al.  Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial.  Ann Fam Med. 2017;15(5):427-433. doi:10.1370/afm.2122PubMedGoogle ScholarCrossref
    14.
    Walker  K, Ben-Meir  M, Dunlop  W,  et al.  Impact of scribes on emergency medicine doctors’ productivity and patient throughput: multicentre randomised trial.  BMJ. 2019;364:l121. doi:10.1136/bmj.l121PubMedGoogle ScholarCrossref
    15.
    Cohen  GR, Friedman  CP, Ryan  AM, Richardson  CR, Adler-Milstein  J.  Variation in physicians’ electronic health record documentation and potential patient harm from that variation.  J Gen Intern Med. 2019;34(11):2355-2367; Epub ahead of print. doi:10.1007/s11606-019-05025-3PubMedGoogle ScholarCrossref
    16.
    Natarajan  K, Stein  D, Jain  S, Elhadad  N.  An analysis of clinical queries in an electronic health record search utility.  Int J Med Inform. 2010;79(7):515-522. doi:10.1016/j.ijmedinf.2010.03.004PubMedGoogle ScholarCrossref
    17.
    Yang  L, Mei  Q, Zheng  K, Hanauer  DA.  Query log analysis of an electronic health record search engine.  AMIA Annu Symp Proc. 2011;2011:915-924.PubMedGoogle Scholar
    18.
    Roman  LC, Ancker  JS, Johnson  SB, Senathirajah  Y.  Navigation in the electronic health record: a review of the safety and usability literature.  J Biomed Inform. 2017;67:69-79. doi:10.1016/j.jbi.2017.01.005PubMedGoogle ScholarCrossref
    19.
    Nolan  ME, Siwani  R, Helmi  H, Pickering  BW, Moreno-Franco  P, Herasevich  V.  Data use and navigation patterns among medical ICU clinicians during electronic chart review.  Appl Clin Inform. 2017;8(4):1117-1126. doi:10.4338/ACI-2017-06-RA-0110PubMedGoogle ScholarCrossref
    20.
    Sheehy  AM, Weissburg  DJ, Dean  SM.  The role of copy-and-paste in the hospital electronic health record.  JAMA Intern Med. 2014;174(8):1217-1218. doi:10.1001/jamainternmed.2014.2110PubMedGoogle ScholarCrossref
    21.
    Graber  ML, Siegal  D, Riah  H, Johnston  D, Kenyon  K.  Electronic health record-related events in medical malpractice claims.  J Patient Saf. 2019;15(2):77-85. doi:10.1097/PTS.0000000000000240PubMedGoogle ScholarCrossref
    22.
    Nuti  SV, Wayda  B, Ranasinghe  I,  et al.  The use of google trends in health care research: a systematic review.  PLoS One. 2014;9(10):e109583. doi:10.1371/journal.pone.0109583PubMedGoogle Scholar
    23.
    Biron  P, Metzger  MH, Pezet  C, Sebban  C, Barthuet  E, Durand  T.  An information retrieval system for computerized patient records in the context of a daily hospital practice: the example of the Léon Bérard Cancer Center (France).  Appl Clin Inform. 2014;5(1):191-205. doi:10.4338/ACI-2013-08-CR-0065PubMedGoogle ScholarCrossref
    24.
    Liang  LL.  Connected for Health: Using Electronic Health Records to Transform Care Delivery. Hoboken, NJ: Pfeiffer; 2010.
    25.
    Epic. Chart Search Setup and Support Guide. Verona, WI: Epic; 2018. https://userweb.epic.com. Accessed January 30, 2020.
    26.
    Observational Health Data Sciences and Informatics. Data standardization. https://ohdsi.org/data-standardization/. Published 2019. Accessed June 18, 2019.
    27.
    Fruchterman  TM, Reingold  EM.  Graph drawing by force-directed placement.  Softw Pract Exper. 1991;21(11):1129-1164. doi:10.1002/spe.4380211102Google ScholarCrossref
    28.
    Wisner  K, Lyndon  A, Chesla  CA.  The electronic health record’s impact on nurses’ cognitive work: an integrative review.  Int J Nurs Stud. 2019;94:74-84. doi:10.1016/j.ijnurstu.2019.03.003PubMedGoogle ScholarCrossref
    29.
    Zalis  M, Harris  M.  Advanced search of the electronic medical record: augmenting safety and efficiency in radiology.  J Am Coll Radiol. 2010;7(8):625-633. doi:10.1016/j.jacr.2010.03.011PubMedGoogle ScholarCrossref
    30.
    King  AJ, Cooper  GF, Hochheiser  H, Clermont  G, Hauskrecht  M, Visweswaran  S.  Using machine learning to predict the information seeking behavior of clinicians using an electronic medical record system.  AMIA Annu Symp Proc. 2018;2018:673-682.PubMedGoogle Scholar
    31.
    Longhurst  CA, Davis  T, Maneker  A,  et al; Arch Collaborative.  Local investment in training drives electronic health record user satisfaction.  Appl Clin Inform. 2019;10(2):331-335. doi:10.1055/s-0039-1688753PubMedGoogle ScholarCrossref
    32.
    Saleem  JJ, Flanagan  ME, Wilck  NR, Demetriades  J, Doebbeling  BN.  The next-generation electronic health record: perspectives of key leaders from the US Department of Veterans Affairs.  J Am Med Inform Assoc. 2013;20(e1):e175-e177. doi:10.1136/amiajnl-2013-001748PubMedGoogle ScholarCrossref
    33.
    Senathirajah  Y, Kaufman  D, Bakken  S.  User-composable electronic health record improves efficiency of clinician data viewing for patient case appraisal: a mixed-methods study.  EGEMS (Wash DC). 2016;4(1):1176. doi:10.13063/2327-9214.1176PubMedGoogle Scholar
    34.
    Institute of Medicine.  Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press; 2012.
    35.
    Cimino  JJ.  Putting the “why” in “EHR”: capturing and coding clinical cognition.  J Am Med Inform Assoc. 2019;26(11):1379-1384. doi:10.1093/jamia/ocz125PubMedGoogle ScholarCrossref
    36.
    Sutton  JM, Ash  SR, Al Makki  A, Kalakeche  R.  A daily hospital progress note that increases physician usability of the electronic health record by facilitating a problem-oriented approach to the patient and reducing physician clerical burden.  Perm J. 2019;23:23.PubMedGoogle Scholar
    37.
    Weiner  M.  Forced Inefficiencies of the electronic health record.  J Gen Intern Med. 2019;34(11):2299-2301; Epub ahead of print. doi:10.1007/s11606-019-05281-3PubMedGoogle ScholarCrossref
    38.
    Prager  JM, Liang  JJ, Devarakonda  MV.  SemanticFind: locating what you want in a patient record, not just what you ask for.  AMIA Jt Summits Transl Sci Proc. 2017;2017:249-258.PubMedGoogle Scholar
    39.
    Hanauer  DA, Wu  DT, Yang  L,  et al.  Development and empirical user-centered evaluation of semantically-based query recommendation for an electronic health record search engine.  J Biomed Inform. 2017;67:1-10. doi:10.1016/j.jbi.2017.01.013PubMedGoogle ScholarCrossref
    40.
    Hanauer  DA, Mei  Q, Law  J, Khanna  R, Zheng  K.  Supporting information retrieval from electronic health records: a report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE).  J Biomed Inform. 2015;55:290-300. doi:10.1016/j.jbi.2015.05.003PubMedGoogle ScholarCrossref
    41.
    Kannampallil  TG, Jones  LK, Patel  VL, Buchman  TG, Franklin  A.  Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings.  J Am Med Inform Assoc. 2014;21(e2):e249-e256. doi:10.1136/amiajnl-2013-002615PubMedGoogle ScholarCrossref
    ×