[Skip to Navigation]
Sign In
Figure 1.  Comparing Electronic Health Record Use Between US and Non-US Clinicians From 371 Health Systems
Comparing Electronic Health Record Use Between US and Non-US Clinicians From 371 Health Systems

EHR indicates electronic health record.

Figure 2.  Distribution of Total Electronic Health Record (EHR) Time per Day Between US and Non-US Clinicians From 371 Health Systems
Distribution of Total Electronic Health Record (EHR) Time per Day Between US and Non-US Clinicians From 371 Health Systems

The brown color represents the overlap between the US and non-US health systems in this overlaid histogram.

Table 1.  Descriptive Statistics of the Sample
Descriptive Statistics of the Sample
Table 2.  Association Between Electronic Health Record Use and US and Non-US Health Systemsa
Association Between Electronic Health Record Use and US and Non-US Health Systemsa
1.
Adler-Milstein  J, Holmgren  AJ, Kralovec  P, Worzala  C, Searcy  T, Patel  V.  Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide.   J Am Med Inform Assoc. 2017;24(6):1142-1148. doi:10.1093/jamia/ocx080 PubMedGoogle ScholarCrossref
2.
Adler-Milstein  J, Jha  AK.  HITECH Act drove large gains in hospital electronic health record adoption.   Health Aff (Millwood). 2017;36(8):1416-1422. doi:10.1377/hlthaff.2016.1651 PubMedGoogle ScholarCrossref
3.
Centers for Medicare & Medicaid Services. Promoting interoperability (Pl) program. Published October 2018. Accessed June 8, 2020. https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/October2018_SummaryReport.pdf
4.
Lin  SC, Jha  AK, Adler-Milstein  J.  Electronic health records associated with lower hospital mortality after systems have time to mature.   Health Aff (Millwood). 2018;37(7):1128-1135. doi:10.1377/hlthaff.2017.1658 PubMedGoogle ScholarCrossref
5.
McCullough  JS, Casey  M, Moscovice  I, Prasad  S.  The effect of health information technology on quality in U.S. hospitals.   Health Aff (Millwood). 2010;29(4):647-654. doi:10.1377/hlthaff.2010.0155 PubMedGoogle ScholarCrossref
6.
Holmgren  AJ, Pfeifer  E, Manojlovich  M, Adler-Milstein  J.  A novel survey to examine the relationship between health IT adoption and nurse-physician communication.   Appl Clin Inform. 2016;7(4):1182-1201. doi:10.4338/ACI-2016-08-RA-0145 PubMedGoogle ScholarCrossref
7.
Holmgren  AJ, Patel  V, Adler-Milstein  J.  Progress in interoperability: measuring US hospitals’ engagement in sharing patient data.   Health Aff (Millwood). 2017;36(10):1820-1827. doi:10.1377/hlthaff.2017.0546 PubMedGoogle ScholarCrossref
8.
Adjerid  I, Adler-Milstein  J, Angst  C.  Reducing Medicare spending through electronic health information exchange: the role of incentives and exchange maturity.   Inf Syst Res. 2018;29(2):341-361. doi:10.1287/isre.2017.0745 Google ScholarCrossref
9.
Bailey  JE, Pope  RA, Elliott  EC, Wan  JY, Waters  TM, Frisse  ME.  Health information exchange reduces repeated diagnostic imaging for back pain.   Ann Emerg Med. 2013;62(1):16-24. doi:10.1016/j.annemergmed.2013.01.006 PubMedGoogle ScholarCrossref
10.
Gawande  A. Why doctors hate their computers. The New Yorker. November 12, 2018. Accessed May 27, 2020. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers
11.
Tai-Seale  M, Olson  CW, Li  J,  et al.  Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine.   Health Aff (Millwood). 2017;36(4):655-662. doi:10.1377/hlthaff.2016.0811 PubMedGoogle ScholarCrossref
12.
Ratwani  RM, Reider  J, Singh  H.  A decade of health information technology usability challenges and the path forward.   JAMA. 2019;321(8):743-744. doi:10.1001/jama.2019.0161 PubMedGoogle ScholarCrossref
13.
Halamka  JD, Tripathi  M.  The HITECH era in retrospect.   N Engl J Med. 2017;377(10):907-909. doi:10.1056/NEJMp1709851 PubMedGoogle ScholarCrossref
14.
Leslie  M, Paradis  E, Gropper  MA, Kitto  S, Reeves  S, Pronovost  P.  An ethnographic study of health information technology use in three intensive care units.   Health Serv Res. 2017;52(4):1330-1348. doi:10.1111/1475-6773.12466 PubMedGoogle ScholarCrossref
15.
Sinsky  C, Colligan  L, Li  L,  et al.  Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties.   Ann Intern Med. 2016;165(11):753-760. doi:10.7326/M16-0961 PubMedGoogle ScholarCrossref
16.
Shanafelt  TD, Hasan  O, Dyrbye  LN,  et al.  Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014.   Mayo Clin Proc. 2015;90(12):1600-1613. doi:10.1016/j.mayocp.2015.08.023PubMedGoogle ScholarCrossref
17.
Han  S, Shanafelt  TD, Sinsky  CA,  et al.  Estimating the attributable cost of physician burnout in the United States.   Ann Intern Med. 2019;170(11):784-790. doi:10.7326/M18-1422 PubMedGoogle ScholarCrossref
18.
Shanafelt  T, Goh  J, Sinsky  C.  The business case for investing in physician well-being.   JAMA Intern Med. 2017;177(12):1826-1832. doi:10.1001/jamainternmed.2017.4340 PubMedGoogle ScholarCrossref
19.
West  CP, Dyrbye  LN, Shanafelt  TD.  Physician burnout: contributors, consequences and solutions.   J Intern Med. 2018;283(6):516-529. doi:10.1111/joim.12752 PubMedGoogle ScholarCrossref
20.
Gajra  A, Bapat  B, Jeune-Smith  Y,  et al.  Frequency and causes of burnout in US community oncologists in the era of electronic health records.   JCO Oncol Pract. 2020;16(4):e357-e365. doi:10.1200/JOP.19.00542 PubMedGoogle ScholarCrossref
21.
Gardner  RL, Cooper  E, Haskell  J,  et al.  Physician stress and burnout: the impact of health information technology.   J Am Med Inform Assoc. 2019;26(2):106-114. doi:10.1093/jamia/ocy145 PubMedGoogle ScholarCrossref
22.
Tai-Seale  M, Dillon  EC, Yang  Y,  et al.  Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records.   Health Aff (Millwood). 2019;38(7):1073-1078. doi:10.1377/hlthaff.2018.05509 PubMedGoogle ScholarCrossref
23.
Zulman  DM, Shah  NH, Verghese  A.  Evolutionary pressures on the electronic health record: caring for complexity.   JAMA. 2016;316(9):923-924. doi:10.1001/jama.2016.9538 PubMedGoogle ScholarCrossref
24.
Melnick  ER, Dyrbye  LN, Sinsky  CA,  et al.  The association between perceived electronic health record usability and professional burnout among US physicians.   Mayo Clin Proc. 2020;95(3):476-487. doi:10.1016/j.mayocp.2019.09.024 PubMedGoogle ScholarCrossref
25.
Sinsky  C. Infographic: date night with the EHR. NEJM Catalyst. Published December 12, 2017. Accessed June 27, 2020. https://catalyst-nejm-org.ezp-prod1.hul.harvard.edu/doi/full/10.1056/CAT.17.0304#
26.
Downing  NL, Bates  DW, Longhurst  CA.  Physician burnout in the electronic health record era: are we ignoring the real cause?   Ann Intern Med. 2018;169(1):50-51. doi:10.7326/M18-0139 PubMedGoogle ScholarCrossref
27.
Adler-Milstein  J.  Electronic health record time among outpatient physicians: reflections on the who, what, and why.   Ann Intern Med. 2020;172(3):212-213. doi:10.7326/M19-3921 PubMedGoogle ScholarCrossref
28.
Overhage  JM, McCallie  D  Jr.  Physician time spent using the electronic health record during outpatient encounters: a descriptive study.   Ann Intern Med. 2020;172(3):169-174. doi:10.7326/M18-3684 PubMedGoogle ScholarCrossref
29.
Melnick  ER, Sinsky  CA, Dyrbye  LN,  et al.  Association of perceived electronic health record usability with patient interactions and work-life integration among US physicians.   JAMA Netw Open. 2020;3(6):e207374. doi:10.1001/jamanetworkopen.2020.7374 PubMedGoogle Scholar
30.
Adler-Milstein  J, Zhao  W, Willard-Grace  R, Knox  M, Grumbach  K.  Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians.   J Am Med Inform Assoc. 2020;27(4):531-538. doi:10.1093/jamia/ocz220 PubMedGoogle ScholarCrossref
31.
Hilliard  RW, Haskell  J, Gardner  RL.  Are specific elements of electronic health record use associated with clinician burnout more than others?   J Am Med Inform Assoc. 2020;27(9):1401-1410. doi:10.1093/jamia/ocaa092PubMedGoogle ScholarCrossref
32.
Sinsky  CA, Rule  A, Cohen  G,  et al.  Metrics for assessing physician activity using electronic health record log data.   J Am Med Inform Assoc. 2020;27(4):639-643. doi:10.1093/jamia/ocz223 PubMedGoogle ScholarCrossref
33.
Moriarty A. Top 10 ambulatory EHR vendors by 2019 market share. Updated May 2019. Accessed June 27, 2020. https://blog.definitivehc.com/top-ambulatory-ehr-systems
34.
Monegain  B. Cerner has almost double EHR global market share of closest rival Epic, Kalorama says. Healthcare IT News. Published May 15, 2018. Accessed July 7, 2020. https://ramaonhealthcare.com/cerner-has-almost-double-ehr-global-market-share-of-closest-rival-epic-kalorama-says
35.
Squires  DA.  The U.S. health system in perspective: a comparison of twelve industrialized nations.   Issue Brief (Commonw Fund). 2011;16:1-14.PubMedGoogle Scholar
36.
Schneider  EC, Sarnak  DO, Squires  D, Shah  A, Doty  MM. Mirror, mirror 2017: international comparison reflects flaws and opportunities for better U.S. health care. Accessed June 16, 2020. https://www.commonwealthfund.org/interactives/2017/july/mirror-mirror/
37.
Kurani  N, McDermott  D, Shanosky  N. How does the quality of the U.S. healthcare system compare to other countries? Published August 20, 2020. Accessed September 2, 2020. https://www.healthsystemtracker.org/chart-collection/quality-u-s-healthcare-system-compare-countries/
38.
Tseng  P, Kaplan  RS, Richman  BD, Shah  MA, Schulman  KA.  Administrative costs associated with physician billing and insurance-related activities at an academic health care system.   JAMA. 2018;319(7):691-697. doi:10.1001/jama.2017.19148 PubMedGoogle ScholarCrossref
39.
Hermanowski  TR, Kowalczyk  M, Szafraniec-Burylo  SI, Krancberg  AN, Pashos  CL.  Current status and evidence of effects of e-prescribing implementation in United Kingdom, Italy, Germany, Denmark, Poland and United States.   Value Health. 2013;16(7):A462-A463. doi:10.1016/j.jval.2013.08.806 Google ScholarCrossref
40.
Richman  M, Joo  J, Rohani  P.  Transitioning to e-prescribing: preformatted prescription forms improve safety, formulary compliance, prescribing satisfaction, and perceived efficiency.   J Patient Saf. 2018;14(4):241-245. doi:10.1097/PTS.0000000000000198 PubMedGoogle ScholarCrossref
41.
Leung  AA, Keohane  C, Lipsitz  S,  et al.  Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool.   J Am Med Inform Assoc. 2013;20(e1):e85-e90. doi:10.1136/amiajnl-2012-001549 PubMedGoogle ScholarCrossref
42.
Moja  L, Kwag  KH, Lytras  T,  et al.  Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis.   Am J Public Health. 2014;104(12):e12-e22. doi:10.2105/AJPH.2014.302164 PubMedGoogle ScholarCrossref
43.
Ancker  JS, Edwards  A, Nosal  S, Hauser  D, Mauer  E, Kaushal  R; HITEC Investigators.  Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system.   BMC Med Inform Decis Mak. 2017;17(1):36. doi:10.1186/s12911-017-0430-8 PubMedGoogle ScholarCrossref
44.
Shanafelt  TD, Dyrbye  LN, Sinsky  C,  et al.  Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction.   Mayo Clin Proc. 2016;91(7):836-848. doi:10.1016/j.mayocp.2016.05.007PubMedGoogle ScholarCrossref
45.
Centers for Medicare & Medicaid Services. Simplifying documentation requirements. Published March 23, 2020. Accessed June 28, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Monitoring-Programs/Medicare-FFS-Compliance-Programs/SimplifyingRequirements
46.
Etheredge  LM.  A rapid-learning health system: what would a rapid-learning health system look like, and how might we get there?   Health Aff (Millwood). 2007;26(2)(suppl 1):w107-w118. doi:10.1377/hlthaff.26.2.w107 PubMedGoogle ScholarCrossref
47.
Berwick D. 5 Big missteps on the patient safety journey. Updated May 24, 2017. Accessed June 28, 2020. https://www.beckershospitalreview.com/quality/dr-don-berwick-5-big-missteps-on-the-patient-safety-journey.html
48.
Berwick  DM.  Era 3 for medicine and health care.   JAMA. 2016;315(13):1329-1330. doi:10.1001/jama.2016.1509 PubMedGoogle ScholarCrossref
49.
Gerrie  BJ, Holbrook  EA.  The evolutionary role of physician assistants across the United States, Canada, and the United Kingdom.   Int J Exerc Sci. 2013;6(1):1-8. https://digitalcommons.wku.edu/ijes/vol6/iss1/1Google Scholar
50.
Maier  CB, Barnes  H, Aiken  LH, Busse  R.  Descriptive, cross-country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential.   BMJ Open. 2016;6(9):e011901. doi:10.1136/bmjopen-2016-011901 PubMedGoogle Scholar
Original Investigation
Physician Work Environment and Well-Being
December 14, 2020

Assessment of Electronic Health Record Use Between US and Non-US Health Systems

Author Affiliations
  • 1Interfaculty Initiative in Health Policy, Harvard University, Cambridge, Massachusetts
  • 2Harvard Business School, Boston, Massachusetts
  • 3Department of Medicine, Stanford University, Stanford, California
  • 4Clinical Excellence Research Center, Stanford University, Stanford, California
  • 5Department of General Internal Medicine, Brigham & Women’s Hospital, Boston, Massachusetts
  • 6Harvard Medical School, Boston, Massachusetts
  • 7Division of Hematology, Department of Medicine, Stanford University, Palo Alto, California
  • 8Department of Economics, Harvard University, Cambridge, Massachusetts
  • 9Graduate School of Business, Stanford University, Stanford, California
JAMA Intern Med. 2021;181(2):251-259. doi:10.1001/jamainternmed.2020.7071
Key Points

Question  Does the use of the electronic health record (EHR) differ between clinicians in the US and those in other countries?

Findings  In this cross-sectional study of the EHR metadata of 371 health systems in the US and abroad, US clinicians vs non-US clinicians were found to spend more time per day actively using the EHR, receive more system-generated messages, write a higher proportion of automatically generated note text, and spend more time using the EHR after hours.

Meaning  Findings from this study suggest that US clinicians compared with non-US clinicians had a higher EHR burden, which could be alleviated by minimizing EHR uncertainties and consolidating documentation requirements.

Abstract

Importance  Understanding how the electronic health record (EHR) system changes clinician work, productivity, and well-being is critical. Little is known regarding global variation in patterns of use.

Objective  To provide insights into which EHR activities clinicians spend their time doing, the EHR tools they use, the system messages they receive, and the amount of time they spend using the EHR after hours.

Design, Setting, and Participants  This cross-sectional study analyzed the deidentified metadata of ambulatory care health systems in the US, Canada, Northern Europe, Western Europe, the Middle East, and Oceania from January 1, 2019, to August 31, 2019. All of these organizations used the EHR software from Epic Systems and represented most of Epic Systems’s ambulatory customer base. The sample included all clinicians with scheduled patient appointments, such as physicians and advanced practice practitioners.

Exposures  Clinician EHR use was tracked by deidentified and aggregated metadata across a variety of clinical activities.

Main Outcomes and Measures  Descriptive statistics for clinician EHR use included time spent on clinical activities, note documentation (as measured by the percentage of characters in the note generated by automated or manual data entry source), messages received, and time spent after hours.

Results  A total of 371 health systems were included in the sample, of which 348 (93.8%) were located in the US and 23 (6.2%) were located in other countries. US clinicians spent more time per day actively using the EHR compared with non-US clinicians (mean time, 90.2 minutes vs 59.1 minutes; P < .001). In addition, US clinicians vs non-US clinicians spent significantly more time performing 4 clinical activities: notes (40.7 minutes vs 30.7 minutes; P < .001), orders (19.5 minutes vs 8.75 minutes; P < .001), in-basket messages (12.5 minutes vs 4.80 minutes; P < .001), and clinical review (17.6 minutes vs 14.8 minutes; P = .01). Clinicians in the US composed more automated note text than their non-US counterparts (77.5% vs 60.8% of note text; P < .001) and received statistically significantly more messages per day (33.8 vs 12.8; P < .001). Furthermore, US clinicians used the EHR for a longer time after hours, logging in 26.5 minutes per day vs 19.5 minutes per day for non-US clinicians (P = .01). The median US clinician spent as much time actively using the EHR per day (90.1 minutes) as a non-US clinician in the 99th percentile of active EHR use time per day (90.7 minutes) in the sample. These results persisted after controlling for organizational characteristics, including structure, type, size, and daily patient volume.

Conclusions and Relevance  This study found that US clinicians compared with their non-US counterparts spent substantially more time actively using the EHR for a wide range of clinical activities or tasks. This finding suggests that US clinicians have a greater EHR burden that may be associated with nontechnical factors, which policy makers and health system leaders should consider when addressing clinician wellness.

Introduction

After the passage of the Health Information Technology for Economic and Clinical Health Act of 2009, health systems in the United States rapidly adopted electronic health records (EHRs).1-3 Electronic health records have shown promise in improving quality,4,5 improving communication,6 and decreasing redundant use.7-9 However, studies suggest that unintended consequences have emerged, including clinician frustration with EHRs and the large amount of time spent working in these systems.10-15 Use of EHRs has been associated with decreased job satisfaction and burnout among clinicians16-20 as well as disruptions of clinician-patient relationships.11,21-24 In addition, EHRs enable clinicians to work outside of the physical walls of their facilities, resulting in so-called desktop medicine that expands to after-hours work.25

It is critical to understand how EHRs have changed clinician work, productivity, and well-being so as to be aware of their association with clinician burnout. Most studies of EHRs have relied on data from a small number of organizations, largely in the US.11,15 An early comparison of EHR documentation found that US clinician notes were 4 times longer than the notes of their counterparts in other countries and that non-US clinicians were more likely to report satisfaction with their EHR systems.26

To date, few large-scale studies of clinician EHR work have been conducted and none have addressed the global variation, with the largest study focusing on cross-specialty variation within the US.27,28 Little is known regarding the variation in time spent on specific EHR tasks, such as messaging (also known as in-basket), ordering, or documentation. To address this, we conducted a large, multiorganizational, cross-national comparison of EHR use by clinicians in ambulatory settings. Using a unique data set of EHR metadata, this study aimed to provide insights into which EHR activities clinicians spend their time doing,21 the EHR tools they use,26,29 the system messages they receive,22,30,31 and the amount of time they spend in the EHR after hours.30,32

Methods

This cross-sectional study was deemed exempt by the institutional review board at Stanford University because it used deidentified data and was not human subject research. The study was conducted from November 1, 2019, to July 1, 2020.

Data

The data sample consisted of deidentified metadata from 371 ambulatory care health systems in the US, Canada, Northern Europe, Western Europe, the Middle East, and Oceania for the study period January 1, 2019, to August 31, 2019, obtained from the EHR software vendor Epic Systems. Epic Systems has the largest ambulatory market share in the US, and the study sample represented most of Epic System’s global ambulatory customer base.33,34 Organizations that used the Epic EHR software but requested that their metadata not be shared were excluded from the sample. The sample included all clinicians with scheduled patient appointments, such as physicians and advanced practice practitioners (eg, physician assistants and nurse practitioners), and excluded nonclinical users of the EHR and clinicians who did not have scheduled appointments (eg, nurses and medical assistants).

In the data set, each organization represented a single EHR installation, which may include multiple suborganizations and facilities. For example, a hospital with several freestanding ambulatory clinics that used the same EHR was counted as 1 organization. Data were aggregated at the organizational level, with the means calculated over the duration of the study.

The data included organizational characteristics provided by Epic Systems. These characteristics included country (with organizations outside of the US grouped into geographic regions), region within the US, organizational structure (ie, ambulatory only, hospital and clinic, or other, such as retail clinic), organizational type (ie, teaching or academic, community, pediatric only, safety net, other ambulatory clinic, or other), whether the health system had an integrated health plan, size (measured by both the number of physicians and the number of outpatient encounters during the study period), and patient volume (ie, mean number of daily scheduled appointments per clinician during the study period).

Measuring EHR Activity

The software has the capacity to monitor EHR activity at an extremely granular level, collecting metadata primarily with the Signal data extraction tool. Those metadata document the time the system is used, as indicated by keystrokes, mouse movements, clicks, scrolling, and interactions with the EHR. In this analysis, time was defined as the time a user was performing active tasks in the EHR. If no activity is detected for 5 seconds, the system stops counting time. This time measure captures active EHR engagement and excludes other time a clinician spends performing nonkeystroke tasks while the EHR is open.

Measurement of clinician work in the context of an EHR is a challenge. In this study, we defined work as the active interaction between the clinician and the tool that excluded other work, such as talking with the patient or digesting information while reading data in the EHR. This work time measure is inherently conservative but is available from metadata in a standardized fashion across all study sites. However, this measure may result in an underestimate of true EHR work time if clinicians spend time reading notes or otherwise performing EHR tasks without directly interacting with the system. For this reason, the EHR active work time should be considered as associated with but distinct from measurement methods, such as audit log data or time and motion studies used in previous studies of clinician EHR time.

EHR Time Distribution

We measured active EHR use time categorized into 4 main activities.11,28 The first activity was clinical review, defined as time spent reviewing test results and patient history. The second activity was notes, defined as time spent documenting the clinical encounters. The third activity was in-basket messages, defined as time spent reading and writing messages and managing the messaging feature of the EHR. In-basket messages included those between clinical team members; messages from patients to the clinician; and a wide array of automatically generated messages regarding available results, orders, and prescribing, among others. The fourth activity was orders, defined as time spent entering orders for patients and other tasks related to such orders, such as associating a diagnosis. All EHR time was measured as the mean time per scheduled day in the study period to account for the differences in workload across clinicians who may not be practicing full-time.

Note Composition, In-Basket Message Distribution, and After-Hours Time

For the notes activity, we measured the source of note text. We identified text generated with a manual process as the note text created by typing, transcription by a scribe or another nonclinician EHR user, or voice input using a dictation or text-to-speech program. We classified text generated with an automated process as the note text created by copying and pasting or using the software templates (eg, NoteWriter, SmartTool, and dot phrases) or any other mechanism to bring text from other parts of the EHR. We then calculated the proportion of note text (by character) generated manually vs automatically at the organizational level for the study period. To describe in detail the level of use of automatically generated text, we chose to measure the proportion of automated compared with manual text in the notes rather than the presence or absence of automated note text.

We measured the mean number of in-basket messages received per clinician per day, both overall and by source. In-basket messages in the EHR can be generated from 7 possible sources: system, team, results, prescription, patient, custom, and other. These categories were defined to be consistent with previous studies.22

We calculated after-hours time in the EHR per day. This metric included only time spent performing clinical work and does not include time spent for purposes of tasks such as research, data analysis, performance measurement, or customization. We defined after-hours EHR time as any time between 5:30 pm and 7:00 am local time on weekdays and any time on weekends, unless the clinician was scheduled during those times, according to the Epic Systems definition of after-hours time, which was broadly consistent with the literature on after-hours EHR work.28,30 Weekend days were established by specific locale for non-US health systems. Because of data limitations, the after-hours time was calculated using data from April 1, 2019, to August 31, 2019.

Statistical Analysis

We compared the descriptive statistics of the organizational characteristics for US and non-US organizations. We tested for statistically significant differences using Fisher exact tests for organizational structure and organizational type and unpaired, 2-tailed t tests with unequal variance for number of physicians, number of outpatient visits, and number of scheduled appointments per clinician per day. We then calculated the descriptive statistics for EHR time per day across the 4 activities (clinical review, notes, in-basket messages, and orders) as well as the total EHR time per day for both US and non-US clinicians using unpaired, 2-tailed t tests with unequal variance to evaluate the statistical significance. We calculated the descriptive statistics and tests of statistical significance in comparing US with non-US health systems for manual and automated note text, in-basket messages received per day (in total and across message categories), and after-hours time per day.

To assess these associations while controlling for observable organizational characteristics, we created 4 ordinary least-squares regression models. Each model used a different dependent variable: total EHR time per day in minutes, after-hours EHR time per day in minutes, percentage of note text generated from automated sources, and EHR system–generated messages received per clinician per day. Each model included an indicator for the US or non-US location of the health system as well as organizational characteristics such as structure, type, health plan integration, size, and daily patient volume. All models had robust standard errors clustered at the health system level. In addition, we conducted a qualitative interview with a non-US hospital chief medical information officer who was previously employed by a health care organization in the study sample.

We conducted several robustness and sensitivity analyses. We calculated the source of the note text with 3 categories (manual or automated process [copy and paste or Epic template]) by disaggregating the copy-and-paste and template text. We ran the regression models several times (first excluding the health systems located in Western Europe and then excluding health systems located in Canada) to ensure that the results were not driven solely by the non-US health systems in areas most represented in the sample. We ran the model of total EHR time with the dependent variable expressed as time per encounter rather than time per day and time per day normalized by number of clinician encounters per day.

A 2-sided P = .05 was used to indicate statistical significance. All statistical calculations and plots were done with Stata, version 16.1 (StataCorp LLC). Data were analyzed from December 1, 2019, to July 1, 2020.

Results

Of the 371 health systems included in the sample, 348 (93.8%) were located in the US and 23 (6.2%) were located in other countries. Western Europe was the region most represented outside of the US, with 11 health systems (47.8%), followed by Canada with 6 systems (26.1%). The Middle East (3 [13.0%]), Northern Europe (2 [8.7%]), and Oceania (1 [4.3%]) composed the remaining health care organizations with the Epic EHR. Full descriptive statistics are available in Table 1.

US clinicians spent a mean time of 90.2 minutes actively using the EHR per day compared with the 59.1 minutes spent per day by non-US clinicians (P < .001). Differences in time spent performing each of the 4 clinical activities were observed between the US and non-US clinicians, such as notes (40.7 minutes vs 30.7 minutes; P < .001), orders (19.5 minutes vs 8.75 minutes; P < .001), in-basket messages (12.5 minutes vs 4.80 minutes; P < .001), and clinical review (17.6 minutes vs 14.8 minutes; P = .01).

Clinicians in the US created more notes that were generated from automated sources compared with non-US clinicians (77.5% vs 60.8% of note text; P < .001), and similar results were found when we disaggregated automated text into copy-and-paste and templated text (eFigure in the Supplement). In addition, US clinicians compared with non-US clinicians received more messages per day in total (33.8 vs 12.8; P < .001) and from various sources: system (11.5 vs 6.0; P < .001), team (11.4 vs 3.27; P < .001), results (6.49 vs 3.01; P < .001), prescription (2.70 vs 0.14; P < .001), patient (1.06 vs 0.10; P < .001), and custom (0.35 vs 0.03; P < .001). US clinicians worked in the EHR for a longer time after hours per day than did non-US clinicians (26.5 minutes vs 19.5 minutes; P = .01) (Figure 1). The distribution of US and non-US clinician total EHR time per day is shown in Figure 2. The median time spent working in the EHR was 90.1 minutes per day for US clinicians compared with 58.3 minutes per day for non-US clinicians. The 99th percentile of EHR work time for US clinicians was 143.4 minutes per day, whereas the 99th percentile for non-US clinicians was 90.7 minutes per day.

The multivariable models found similar differences between US and non-US health systems. Compared with non-US clinicians (reference group), US clinicians spent 23.67 more minutes per day actively using the EHR (β = 23.67; 95% CI, 17.70-29.64; P < .001), spent 7.23 more minutes after hours per day interacting with the EHR (β = 7.23; 95% CI, 2.30-12.16; P < .001), composed 17 percentage points more note text with automated tools (β = 0.17%; 95% CI, 0.11%-0.22%; P < .001), and received 5.28 more system-generated in-basket messages per day (β = 5.28; 95% CI, 3.18-7.38; P < .001) (Table 2). Regression model results were robust to excluding non-US health systems in Western Europe (eTable 1 in the Supplement) or Canada (eTable 2 in the Supplement) and to measuring total EHR time at the encounter level (eTables 3 and 4 in the Supplement). For example, US clinicians spent more time per day in the EHR (β = 32.02 minutes; P < .001), spent more after-hours time per day (β = 10.98 minutes; P < .001), generated a higher proportion of note text from automated sources (β = 0.19%; P < .001), and received more system messages per day (β = 4.03; P = .01) (eTable 1 in the Supplement).

Discussion

We found large differences in EHR use between US and non-US clinicians. This finding is notable given that all of the organizations in the study used the same EHR software, although institutions may customize the system’s functionality. The results suggest that a portion of clinician EHR work was associated with contextual factors unique to national health systems rather than the technical demands of the EHR itself or the clinical demands of delivering care. Although some EHR features provide considerable value to patients and clinicians, quality of care delivered in the US is unlikely to be substantially better than that in the other countries examined in this study.35,36 Furthermore, although health outcomes are associated with many factors ranging from technical clinical skills to social determinants, such as poverty, racism, and lack of access, recent research has shown that US-based health care organizations are not better than their counterparts in other countries at achieving high performance in most process quality measures during care delivery that information technology could help improve, such as reducing rates of medication errors.37 Therefore, the finding that US clinicians spent more time actively using the EHR is concerning. This observation is consistent with other studies that suggested EHR adoption in the US is not associated with decreased administrative burdens38; we found that the average US clinician spent as much time actively using the EHR as a non-US clinician in the 99th percentile of EHR work time. This finding suggests that if US clinicians could decrease their EHR work time to the level of their non-US peers, they may be able to increase the volume of patients seen or improve quality along multiple dimensions, such as longer visits and more patient-centered care.

The differences in EHR use between US and non-US clinicians are associated with multiple factors. For example, the reason that US clinicians spend more time on orders may be the need for diagnostic association, a billing requirement in the US that is absent from most non-US billing practices; alternatively, US clinicians may need to enter individual electronic orders for low-risk tasks, such as ear irrigation or immunizations. Similarly, additional time spent by US clinicians on in-basket messages may reflect policy-driven differences in volume of messages received. The Meaningful Use incentive program in the US mandated both secure messaging with patients and electronic prescribing, resulting in nearly universal adoption of those capabilities by US health systems. Connecting more members of the care team, such as pharmacists, to the EHR led to a higher volume of team messages. Given the low level of patient and prescription messages received by non-US clinicians and the wide variation in the implementation of electronic prescribing worldwide,39 the requirements of the Meaningful Use program were likely a factor in the differences in in-basket messages time. US clinicians also spent more time working in the Notes function despite using more automatically generated text. Previous studies have shown that the length of notes is considerably longer in the US,26 which may illustrate the implication of greater use of automatically generated text for nonclinical purposes such as billing, quality reporting, documentation to minimize legal liability, and other administrative tasks.

Some EHR areas in which US clinicians spend more time likely deliver value to patients, such as secure messaging, which patients may prefer to alternatives. Other functions may improve safety and save time elsewhere in the care delivery process, such as e-prescribing, which was considered EHR work for US clinicians in this study but was less likely to be observed among non-US clinicians who may write prescriptions manually.40 Additional time in the orders activity may reflect higher levels of clinical decision support, which may improve safety,41,42 albeit potentially at the cost of burnout and alert fatigue.43,44 Studies have suggested associations between clinician burnout and total EHR time,21 after-hours EHR time,30 percentage of note text generated from automated sources,26 and system-generated messages received per day.22,30

We believe that findings from this study have important implications for health policy and practice. Although substantial attention has been focused on the role of EHRs in burnout, the results showed that US clinicians used EHRs in different ways compared with their non-US peers, suggesting that at least some of the time burden was associated with nontechnical aspects of EHR implementation that were specific to US market characteristics such as differences in workflow or policy. Factors such as a multipayer billing environment, in which clinicians rely on documentation to ensure their claims are not denied, as well as Meaningful Use and incorporation of quality measurement and administrative functions may have contributed to the results.

Policy makers who are concerned about the association of EHRs with clinician burnout should consider the implications of the study results. Most ambulatory practices contract with multiple payers and must be prepared to submit clinical documentation to substantiate reimbursable services, and they may opt to overdocument to guard against rejected claims. Similarly, US clinicians may overdocument if they are concerned that they may need to defend themselves from malpractice claims. In our qualitative interview with Chris Hayes, MD, a chief medical information officer in a non-US health system (email and telephone communication, May 2020), he indicated that, despite working in a fee-for-service reimbursement setting, there is no need to document anything that is not relevant to clinical care because medical claims are rarely challenged. Although the Centers for Medicare & Medicaid Services has clarified documentation requirements,45 aggressive steps toward reducing inefficiencies in reimbursement are needed. Similarly, EHRs have promised to capture the information necessary to measure and improve quality.46 However, quality reporting has been fragmented, involving multiple stakeholders.47,48 Minimizing uncertainties and consolidating requirements could alleviate the EHR burdens of US clinicians.

Limitations

This study has some limitations. First, this analysis is descriptive and cannot address the causal association between country of practice and EHR use. Second, the data set we obtained only accounted for ambulatory practice; we were unable to evaluate EHR use in an inpatient, emergency, or long-term care setting. Third, because the study used a narrowly defined measure of EHR work that counted only the time that clinicians spent actively working within the system, the results were difficult to compare with those of other investigations that used audit log data or time-and-motion studies and were likely to underestimate work time; the times represented in this study should not be taken as indicative of the length of a clinician workday. Furthermore, the sample included nonphysician clinicians and ambulatory care settings with EHR use patterns that were substantially different from those reported in studies of walk-in clinics, for example; this difference limits this study’s comparability to previous studies. However, the distribution of EHR time across activities in this study was similar to that in other studies.11,28

Fourth, the data set did not include detailed information on clinician schedules, and thus we were unable to standardize the measure of scheduled days; the differences in clinician scheduling may be associated with some of the differences in EHR time between US and non-US clinicians. However, the multivariable analyses we performed, which controlled for the number of appointments per clinician per day, annual outpatient volume, and number of physicians, found similar results as our bivariate comparisons; the sensitivity analysis that compared EHR time per encounter also found results that were consistent with the time-per-scheduled-day measure. Similarly, although the US health systems had substantially greater annual patient volume compared with the non-US systems and only a small number of additional physicians, we did not have data on the exact number of nonphysician clinicians for health system deidentification purposes. However, US health systems are more likely to have more nurse practitioners and physician assistants,49,50 which may explain why the differences in daily patient volume by clinician were much less pronounced in this study. In addition, although the measure of after-hours work time used a vendor-derived metric that combined a standard approach to defining a clinical work day with scheduled visit data that were consistent with data in the literature, we were unable to classify all after-hours time, which may result in some measurement error. Fifth, we had data for a broad range of ambulatory care facilities, but the data set came from a single EHR vendor and from a small number of non-US organizations, and thus the use of another vendor would have likely produced different results, limiting the external generalizability of this study. In addition, although the sample included nearly all Epic Systems customers that used the ambulatory EHR, the health systems that were also Epic customers but asked to be excluded from the sample may have different EHR use patterns. Sixth, the note text data included voice dictation and transcription by medical scribes, but we analyzed only the EHR work time spent by clinicians and not by scribes on behalf of clinicians. This exclusion may bias the results, although the data set indicated that dictation and transcription by medical scribes were much more common in the US, suggesting that we may have underestimated the difference between US and non-US clinicians. Seventh, although we included controls for health system characteristics, we did not have detailed organizational data. Understanding how EHR use varies across organizations is an important area for future research.

Conclusions

This cross-sectional study found that US clinicians spent substantially more time actively using the EHR than their non-US counterparts that interacted with the same technology. US clinicians had a higher EHR burden per day across 4 activities (clinical review, notes, in-basket messages, and orders). Policy makers and health system leaders who seek to address clinician wellness should consider minimizing uncertainties and consolidating documentation requirements to alleviate the burden of clinician EHR work, which is associated with US-specific market and policy factors.

Back to top
Article Information

Accepted for Publication: October 5, 2020.

Published Online: December 14, 2020. doi:10.1001/jamainternmed.2020.7071

Correction: This article was corrected on February 1, 2021, to indicate that Mr Holmgren and Dr Downing contributed equally as co–first authors.

Corresponding Author: A. Jay Holmgren, MHI, Harvard Business School, Soldiers Field Road, 324A Cotting House, Boston, MA 02163 (aholmgren@hbs.edu).

Author Contributions: Mr Holmgren had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Mr Holmgren and Dr Downing contributed equally as co–first authors.

Concept and design: Holmgren, Downing, Bates, Milstein, Sharp, Cutler, Huckman, Schulman.

Acquisition, analysis, or interpretation of data: Holmgren, Downing, Bates, Shanafelt, Sharp, Cutler.

Drafting of the manuscript: Holmgren, Downing, Milstein, Sharp, Cutler, Schulman.

Critical revision of the manuscript for important intellectual content: Holmgren, Downing, Bates, Shanafelt, Sharp, Cutler, Huckman, Schulman.

Statistical analysis: Holmgren, Cutler, Huckman.

Obtained funding: Milstein.

Administrative, technical, or material support: Downing, Cutler, Schulman.

Supervision: Bates, Cutler, Huckman.

Conflict of Interest Disclosures: Dr Bates reported receiving grants and personal fees from EarlySense; personal fees from Center for Digital Innovation (Negev) Ltd; equity from Valera Health, CLEW, and MDClone Ltd; personal fees and other from AESOP; and grants from IBM Watson outside the submitted work. Dr Shanafelt reported being a coinventor of the Well-Being Index instruments and the Participatory Management Leadership Index, for which he receives a portion of any royalties paid to the copyright owner, Mayo Clinic, and reported receiving honoraria for providing grand rounds, keynote lectures, and advice to health care organizations. Dr Milstein reported being a co-founding scientist and paid scientific adviser of Dawnlight Technology and Prealize Health. Dr Huckman reported receiving personal fees from Kaiser Permanente, Partners Healthcare, MD Anderson Cancer Center, OhioHealth, and Ochsner Health; serving as an advisory board member for RubiconMD, Arena, and Carrum Health; and being an uncompensated trustee of Brigham Health and the Brigham and Women's Physicians Organization. Dr Schulman reported being a board member and shareholder for Grid Therapeutics and Reserve Therapeutics; being a managing member and shareholder for Faculty Connection LLC; being a shareholder for Prealize; being an investor in Altitude Ventures Inc and Excelerate Health Ventures; being a consultant for Novartis, Cytokinetics, Business Roundtable, Motley Rice LLC, and Frazier Healthcare Partners; being a speaker for Health Quest LLC and ISMIE Inc; being president of Business School Alliance for Health Management; being senior associate editor of Health Services Research; and being on the advisory board of Civica RX. No other disclosures were reported.

Additional Contributions: Sam Choi, BS, and Josh Holzbauer, BA, Epic Systems, provided assistance with the data. These individuals received no additional compensation, outside of their usual salary, for their contributions.

References
1.
Adler-Milstein  J, Holmgren  AJ, Kralovec  P, Worzala  C, Searcy  T, Patel  V.  Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide.   J Am Med Inform Assoc. 2017;24(6):1142-1148. doi:10.1093/jamia/ocx080 PubMedGoogle ScholarCrossref
2.
Adler-Milstein  J, Jha  AK.  HITECH Act drove large gains in hospital electronic health record adoption.   Health Aff (Millwood). 2017;36(8):1416-1422. doi:10.1377/hlthaff.2016.1651 PubMedGoogle ScholarCrossref
3.
Centers for Medicare & Medicaid Services. Promoting interoperability (Pl) program. Published October 2018. Accessed June 8, 2020. https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/October2018_SummaryReport.pdf
4.
Lin  SC, Jha  AK, Adler-Milstein  J.  Electronic health records associated with lower hospital mortality after systems have time to mature.   Health Aff (Millwood). 2018;37(7):1128-1135. doi:10.1377/hlthaff.2017.1658 PubMedGoogle ScholarCrossref
5.
McCullough  JS, Casey  M, Moscovice  I, Prasad  S.  The effect of health information technology on quality in U.S. hospitals.   Health Aff (Millwood). 2010;29(4):647-654. doi:10.1377/hlthaff.2010.0155 PubMedGoogle ScholarCrossref
6.
Holmgren  AJ, Pfeifer  E, Manojlovich  M, Adler-Milstein  J.  A novel survey to examine the relationship between health IT adoption and nurse-physician communication.   Appl Clin Inform. 2016;7(4):1182-1201. doi:10.4338/ACI-2016-08-RA-0145 PubMedGoogle ScholarCrossref
7.
Holmgren  AJ, Patel  V, Adler-Milstein  J.  Progress in interoperability: measuring US hospitals’ engagement in sharing patient data.   Health Aff (Millwood). 2017;36(10):1820-1827. doi:10.1377/hlthaff.2017.0546 PubMedGoogle ScholarCrossref
8.
Adjerid  I, Adler-Milstein  J, Angst  C.  Reducing Medicare spending through electronic health information exchange: the role of incentives and exchange maturity.   Inf Syst Res. 2018;29(2):341-361. doi:10.1287/isre.2017.0745 Google ScholarCrossref
9.
Bailey  JE, Pope  RA, Elliott  EC, Wan  JY, Waters  TM, Frisse  ME.  Health information exchange reduces repeated diagnostic imaging for back pain.   Ann Emerg Med. 2013;62(1):16-24. doi:10.1016/j.annemergmed.2013.01.006 PubMedGoogle ScholarCrossref
10.
Gawande  A. Why doctors hate their computers. The New Yorker. November 12, 2018. Accessed May 27, 2020. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers
11.
Tai-Seale  M, Olson  CW, Li  J,  et al.  Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine.   Health Aff (Millwood). 2017;36(4):655-662. doi:10.1377/hlthaff.2016.0811 PubMedGoogle ScholarCrossref
12.
Ratwani  RM, Reider  J, Singh  H.  A decade of health information technology usability challenges and the path forward.   JAMA. 2019;321(8):743-744. doi:10.1001/jama.2019.0161 PubMedGoogle ScholarCrossref
13.
Halamka  JD, Tripathi  M.  The HITECH era in retrospect.   N Engl J Med. 2017;377(10):907-909. doi:10.1056/NEJMp1709851 PubMedGoogle ScholarCrossref
14.
Leslie  M, Paradis  E, Gropper  MA, Kitto  S, Reeves  S, Pronovost  P.  An ethnographic study of health information technology use in three intensive care units.   Health Serv Res. 2017;52(4):1330-1348. doi:10.1111/1475-6773.12466 PubMedGoogle ScholarCrossref
15.
Sinsky  C, Colligan  L, Li  L,  et al.  Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties.   Ann Intern Med. 2016;165(11):753-760. doi:10.7326/M16-0961 PubMedGoogle ScholarCrossref
16.
Shanafelt  TD, Hasan  O, Dyrbye  LN,  et al.  Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014.   Mayo Clin Proc. 2015;90(12):1600-1613. doi:10.1016/j.mayocp.2015.08.023PubMedGoogle ScholarCrossref
17.
Han  S, Shanafelt  TD, Sinsky  CA,  et al.  Estimating the attributable cost of physician burnout in the United States.   Ann Intern Med. 2019;170(11):784-790. doi:10.7326/M18-1422 PubMedGoogle ScholarCrossref
18.
Shanafelt  T, Goh  J, Sinsky  C.  The business case for investing in physician well-being.   JAMA Intern Med. 2017;177(12):1826-1832. doi:10.1001/jamainternmed.2017.4340 PubMedGoogle ScholarCrossref
19.
West  CP, Dyrbye  LN, Shanafelt  TD.  Physician burnout: contributors, consequences and solutions.   J Intern Med. 2018;283(6):516-529. doi:10.1111/joim.12752 PubMedGoogle ScholarCrossref
20.
Gajra  A, Bapat  B, Jeune-Smith  Y,  et al.  Frequency and causes of burnout in US community oncologists in the era of electronic health records.   JCO Oncol Pract. 2020;16(4):e357-e365. doi:10.1200/JOP.19.00542 PubMedGoogle ScholarCrossref
21.
Gardner  RL, Cooper  E, Haskell  J,  et al.  Physician stress and burnout: the impact of health information technology.   J Am Med Inform Assoc. 2019;26(2):106-114. doi:10.1093/jamia/ocy145 PubMedGoogle ScholarCrossref
22.
Tai-Seale  M, Dillon  EC, Yang  Y,  et al.  Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records.   Health Aff (Millwood). 2019;38(7):1073-1078. doi:10.1377/hlthaff.2018.05509 PubMedGoogle ScholarCrossref
23.
Zulman  DM, Shah  NH, Verghese  A.  Evolutionary pressures on the electronic health record: caring for complexity.   JAMA. 2016;316(9):923-924. doi:10.1001/jama.2016.9538 PubMedGoogle ScholarCrossref
24.
Melnick  ER, Dyrbye  LN, Sinsky  CA,  et al.  The association between perceived electronic health record usability and professional burnout among US physicians.   Mayo Clin Proc. 2020;95(3):476-487. doi:10.1016/j.mayocp.2019.09.024 PubMedGoogle ScholarCrossref
25.
Sinsky  C. Infographic: date night with the EHR. NEJM Catalyst. Published December 12, 2017. Accessed June 27, 2020. https://catalyst-nejm-org.ezp-prod1.hul.harvard.edu/doi/full/10.1056/CAT.17.0304#
26.
Downing  NL, Bates  DW, Longhurst  CA.  Physician burnout in the electronic health record era: are we ignoring the real cause?   Ann Intern Med. 2018;169(1):50-51. doi:10.7326/M18-0139 PubMedGoogle ScholarCrossref
27.
Adler-Milstein  J.  Electronic health record time among outpatient physicians: reflections on the who, what, and why.   Ann Intern Med. 2020;172(3):212-213. doi:10.7326/M19-3921 PubMedGoogle ScholarCrossref
28.
Overhage  JM, McCallie  D  Jr.  Physician time spent using the electronic health record during outpatient encounters: a descriptive study.   Ann Intern Med. 2020;172(3):169-174. doi:10.7326/M18-3684 PubMedGoogle ScholarCrossref
29.
Melnick  ER, Sinsky  CA, Dyrbye  LN,  et al.  Association of perceived electronic health record usability with patient interactions and work-life integration among US physicians.   JAMA Netw Open. 2020;3(6):e207374. doi:10.1001/jamanetworkopen.2020.7374 PubMedGoogle Scholar
30.
Adler-Milstein  J, Zhao  W, Willard-Grace  R, Knox  M, Grumbach  K.  Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians.   J Am Med Inform Assoc. 2020;27(4):531-538. doi:10.1093/jamia/ocz220 PubMedGoogle ScholarCrossref
31.
Hilliard  RW, Haskell  J, Gardner  RL.  Are specific elements of electronic health record use associated with clinician burnout more than others?   J Am Med Inform Assoc. 2020;27(9):1401-1410. doi:10.1093/jamia/ocaa092PubMedGoogle ScholarCrossref
32.
Sinsky  CA, Rule  A, Cohen  G,  et al.  Metrics for assessing physician activity using electronic health record log data.   J Am Med Inform Assoc. 2020;27(4):639-643. doi:10.1093/jamia/ocz223 PubMedGoogle ScholarCrossref
33.
Moriarty A. Top 10 ambulatory EHR vendors by 2019 market share. Updated May 2019. Accessed June 27, 2020. https://blog.definitivehc.com/top-ambulatory-ehr-systems
34.
Monegain  B. Cerner has almost double EHR global market share of closest rival Epic, Kalorama says. Healthcare IT News. Published May 15, 2018. Accessed July 7, 2020. https://ramaonhealthcare.com/cerner-has-almost-double-ehr-global-market-share-of-closest-rival-epic-kalorama-says
35.
Squires  DA.  The U.S. health system in perspective: a comparison of twelve industrialized nations.   Issue Brief (Commonw Fund). 2011;16:1-14.PubMedGoogle Scholar
36.
Schneider  EC, Sarnak  DO, Squires  D, Shah  A, Doty  MM. Mirror, mirror 2017: international comparison reflects flaws and opportunities for better U.S. health care. Accessed June 16, 2020. https://www.commonwealthfund.org/interactives/2017/july/mirror-mirror/
37.
Kurani  N, McDermott  D, Shanosky  N. How does the quality of the U.S. healthcare system compare to other countries? Published August 20, 2020. Accessed September 2, 2020. https://www.healthsystemtracker.org/chart-collection/quality-u-s-healthcare-system-compare-countries/
38.
Tseng  P, Kaplan  RS, Richman  BD, Shah  MA, Schulman  KA.  Administrative costs associated with physician billing and insurance-related activities at an academic health care system.   JAMA. 2018;319(7):691-697. doi:10.1001/jama.2017.19148 PubMedGoogle ScholarCrossref
39.
Hermanowski  TR, Kowalczyk  M, Szafraniec-Burylo  SI, Krancberg  AN, Pashos  CL.  Current status and evidence of effects of e-prescribing implementation in United Kingdom, Italy, Germany, Denmark, Poland and United States.   Value Health. 2013;16(7):A462-A463. doi:10.1016/j.jval.2013.08.806 Google ScholarCrossref
40.
Richman  M, Joo  J, Rohani  P.  Transitioning to e-prescribing: preformatted prescription forms improve safety, formulary compliance, prescribing satisfaction, and perceived efficiency.   J Patient Saf. 2018;14(4):241-245. doi:10.1097/PTS.0000000000000198 PubMedGoogle ScholarCrossref
41.
Leung  AA, Keohane  C, Lipsitz  S,  et al.  Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool.   J Am Med Inform Assoc. 2013;20(e1):e85-e90. doi:10.1136/amiajnl-2012-001549 PubMedGoogle ScholarCrossref
42.
Moja  L, Kwag  KH, Lytras  T,  et al.  Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis.   Am J Public Health. 2014;104(12):e12-e22. doi:10.2105/AJPH.2014.302164 PubMedGoogle ScholarCrossref
43.
Ancker  JS, Edwards  A, Nosal  S, Hauser  D, Mauer  E, Kaushal  R; HITEC Investigators.  Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system.   BMC Med Inform Decis Mak. 2017;17(1):36. doi:10.1186/s12911-017-0430-8 PubMedGoogle ScholarCrossref
44.
Shanafelt  TD, Dyrbye  LN, Sinsky  C,  et al.  Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction.   Mayo Clin Proc. 2016;91(7):836-848. doi:10.1016/j.mayocp.2016.05.007PubMedGoogle ScholarCrossref
45.
Centers for Medicare & Medicaid Services. Simplifying documentation requirements. Published March 23, 2020. Accessed June 28, 2020. https://www.cms.gov/Research-Statistics-Data-and-Systems/Monitoring-Programs/Medicare-FFS-Compliance-Programs/SimplifyingRequirements
46.
Etheredge  LM.  A rapid-learning health system: what would a rapid-learning health system look like, and how might we get there?   Health Aff (Millwood). 2007;26(2)(suppl 1):w107-w118. doi:10.1377/hlthaff.26.2.w107 PubMedGoogle ScholarCrossref
47.
Berwick D. 5 Big missteps on the patient safety journey. Updated May 24, 2017. Accessed June 28, 2020. https://www.beckershospitalreview.com/quality/dr-don-berwick-5-big-missteps-on-the-patient-safety-journey.html
48.
Berwick  DM.  Era 3 for medicine and health care.   JAMA. 2016;315(13):1329-1330. doi:10.1001/jama.2016.1509 PubMedGoogle ScholarCrossref
49.
Gerrie  BJ, Holbrook  EA.  The evolutionary role of physician assistants across the United States, Canada, and the United Kingdom.   Int J Exerc Sci. 2013;6(1):1-8. https://digitalcommons.wku.edu/ijes/vol6/iss1/1Google Scholar
50.
Maier  CB, Barnes  H, Aiken  LH, Busse  R.  Descriptive, cross-country analysis of the nurse practitioner workforce in six countries: size, growth, physician substitution potential.   BMJ Open. 2016;6(9):e011901. doi:10.1136/bmjopen-2016-011901 PubMedGoogle Scholar
×