Analysis of Electronic Health Record Use and Clinical Productivity and Their Association With Physician Turnover

IMPORTANCE Physician turnover takes a heavy toll on patients, physicians, and health care organizations. Survey research has established associations of electronic health record (EHR) use with professional burnout and reduction in professional effort, but these findings are subject to response fatigue and bias. OBJECTIVE To evaluate the association of physician productivity and EHR use patterns, as determined by vendor-derived EHR use data platforms, with physician turnover. DESIGN, SETTING, AND PARTICIPANTS This retrospective cohort study was conducted among nonteaching ambulatory physicians at a large ambulatory practice network based in New England. Data were collected from March 2018 to February 2020. MAIN OUTCOMES AND MEASURES Physician departure from the practice network; 4 time-based coremeasuresofEHRuse,normalizedto8hoursofscheduledclinicaltime;teamwork,percentage ofaphysician’sordersthatareplacedbyothermembersofthecareteam;andproductivitymeasures of patient volume, intensity, and demand. RESULTS Among 335 physicians assessed for eligibility, 314 unique physicians (89.2%) were included in the analysis (123 [39%] women; 100 [32%] aged 45-54 years), with 5663 physician-monthsofdata.Theturnoverratewas5.1%/year(32of314physicians).Physicianscompletedamean 2.6 appointments/hour (95% CI, 2.5-2.6 appointments/hour) and 206 appointments/month (95% CI, 197-215 appointments/month) with 5.5 hours (95% CI, 5.3-5.8 hours) of EHR time for every 8 hours of scheduled patient time. After controlling for gender, medical specialty, and time, the following variables were associated with turnover: limitations to vendor-derived EHR use data platforms, the findings from this cohort study of physician turnover, productivity, and EHR use, as assessed by vendor-derived EHR use data platforms in a large ambulatory care network, indicate that physician productivity and EHR usemetricswereassociatedwithdeparture.Physicianage,gender,andspecialtywerenotassociated with departure in bivariate or multivariate analyses. After controlling for these factors, less time on the EHR and inbox were both associated with physician departure. The direction of the association between these time-based EHR use metrics and physician departure was opposite of the hypothesized direction based on assumptions about the association between EHR usability and physician burnout. Lower rates of teamwork on order entry were associated with physician departure, indicating that teamwork may help to prevent physician turnover. This finding is consistent with evidence that team-based care can reduce professional burnout and optimize team performance. 26-32 Taken together, these findings suggest that a refined, prospective model of vendor-derived EHR data could help to identify at high risk of departure who might benefit from targeted team-based care interventions. Indeed, low demand for a physician’s service, time on the EHR and inbox, and rates of teamwork on orders may very well be lead indicators for physicians preparing to leave practice.


Introduction
Physician turnover disrupts patients' continuity of care, the lives of physicians and their families, and strains health care organizations' culture, climate, and finances. Physician replacement costs to a health care organization (including recruitment, lost revenue, onboarding, and time to return to optimal efficiency) have been quantified at as much as $1 million per physician departure. 1 This translates to approximately $4.6 billion annually at the national level. 2 These costs are particularly worrisome given high rates of physician burnout 3 and a projected shortfall of 35 000 to 90 000 physicians by 2025. 4,5 Cross-sectional analyses have established associations between professional satisfaction, burnout, intention to leave, [6][7][8][9][10] and actual reduction in professional effort. 11 Given excessive physician time spent on electronic health record (EHR) activities [12][13][14][15] and its association with professional burnout, [16][17][18] it is not surprising that physician dissatisfaction with the EHR has been associated with intention to reduce clinical work in the next 12 months (in national sample of 6880 physicians: OR, 1.44; 95% CI, 1.16-1.80; P = .001) and to leave one's current position in the next 24 months (OR, 1.57; 95% CI, 1.27-1.93; P < .001). 8 In particular, EHR inbox message volume has been strongly associated with physician burnout. 19,20 Survey research is subject to the limitations of self-report, including response fatigue and bias.
EHR audit log data has the potential to overcome these limitations by providing an automated, objective data source for measuring physician productivity and EHR use. 21,22 Recognizing that physician departure is multifactorial (ie, attributed to reasons that are personal and financial and related to work culture, work climate, workload, and so on), we assume an association exists between physician workload and departure. Given this assumption, reliable measurement of physician productivity and EHR use could help to identify physicians at increased risk of departure. 23 In 2020, 7 core EHR use metrics were proposed to standardize ambulatory physician EHR use measurement normalized to 8 hours of scheduled patient time. 24 Five of these metrics could be implemented for ambulatory, nonteaching physicians in 2 health care systems operating different EHR vendor products. 22 To our knowledge, no previous studies have examined physician productivity and EHR use patterns and their association with physician turnover. Therefore, we conducted a study of ambulatory physicians' turnover across an ambulatory care delivery network over a 2-year period to determine the association of physicians' productivity and EHR use patterns with departure. Based on their positive association with physician burnout, 19,20,25 we hypothesized that increased time on EHR activities (in particular inbox management) would be associated with higher rates of physician departure.

Study Design and Setting
This retrospective cohort study of ambulatory physician turnover examined physician turnover, productivity, and EHR use from March 2018 to February 2020 in a large ambulatory practice network in New England. Given known challenges implementing core EHR use metrics with vendor-derived data in the previous implementation of the 7 core metrics for ambulatory physician EHR use, 22 the study cohort was limited to the ambulatory division of a practice network that employs ambulatorybased physicians who are out of training and have no teaching responsibilities. The network includes 141 practice sites in Connecticut, New York, and Rhode Island and operates on a single installation of the Epic EHR (Epic Systems). The study protocol was approved by Yale University's institutional review board. Informed consent was waived, as all study data were deidentified, no private health information was collected, and obtaining consent would jeopardize the scientific validity of the findings given the likelihood of introducing participation bias. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines for cohort studies.

Participants
All nontrainee ambulatory physicians in the practice network were eligible for inclusion in the analysis. The inclusion criteria from the previously reported feasibility study (>30 scheduled ambulatory clinical hours) 22 were applied to each month's data to best approximate which physicians worked exclusively in the ambulatory setting without teaching duties. These inclusion criteria also inherently exclude physicians decreasing their clinical effort in preparation to depart their position.
To identify EHR use patterns associated with departure in time for future interventions, we excluded all data within 3 months prior of departure. Physicians with missing data for variables in the final model were excluded from the analysis.

Data Sources
The primary data source for this study was EHR metadata from vendor-derived data platforms. EHR use data were collected from the Epic Signal platform, and scheduling data were collected from the Epic Clarity database. Signal tracks EHR user actions using a 5-second latency period, ie, once the user is idle for 5 seconds, tracking pauses until user actions resume. User actions are defined as any mouse action (ie, click, movement, scroll) or keyboard action (ie, keystrokes). This 5-second latency period was internally validated by Epic with a time-motion analysis (Provider Efficiency Technical Services team, email, April 6, 2020). A similar time-motion analysis has been performed on other Epic event logs. 12 It is important to note that time on EHR tasks away from the computer will not reliably be captured, eg, a telephone call to patient to resolve an inbox message.
Physician demographic data, including termination status and departure date, were obtained from human resources rosters. These rosters did not include a specific reason for departure (ie, we are unable to determine whether a physician left their position to practice in another location, to leave the profession of medicine, or to retire).

Measurements and Outcomes
Physician characteristics included age, gender, medical specialty, and termination date (if the physician departed their position during the study period). The primary outcome was physician turnover (ie, leaving one's position in the practice network during the study period). To control for temporal trends in EHR use (eg, physician-level variability in EHR proficiency or efficiency over time), each month of data for each physician was assigned a monthly time trend variable corresponding to that physician's study month number.
Physician productivity measures were analyzed to account for differences in clinical workload in terms of patient volume, intensity of work, and demand for a physician's services. Patient volume was defined as the number of completed appointments per month. Intensity was defined as the number of completed appointments per scheduled hour of work. Demand was defined as the percentage of available patient time that was scheduled with appointments.
EHR use measures included the 5 core EHR use metrics that were feasible for implementation in vendor-derived EHR data platforms: total EHR time, normalized to 8 hours of scheduled patient time; work outside of scheduled clinical hours, normalized to 8 hours of scheduled patient time; encounter note documentation time, normalized to 8 hours of scheduled patient time; active time on inbox, normalized to 8 hours of scheduled patient time; and teamwork for orders, the percentage of a physician's orders that are placed by other members of the care team. 22,24 This final non-timebased metric was specifically proposed due to the extensive literature that team-based care can reduce professional burnout. [26][27][28][29][30][31][32]

Statistical Analysis
Data were summarized using standard descriptive statistics for continuous variables and frequencies and percentages for categorical variables. The longitudinal data set consisted of monthly measures for each physician. Bivariate analyses were conducted using 2-sample t tests; associations for continuous variables and χ 2 , 2-sample proportions tests for categorical variables were used for crosssectional data, such as physician characteristics or to compare the means of the mean values for all metrics at the physician level. Mean differences between groups based on repeated measures were also calculated by departure status using linear mixed models to adjust for within-physician association over time. To account for within-physician association due to the multiple observations for each physician, the associations between departure status, physician productivity, and EHR use patterns were examined on a monthly basis for each physician using generalized estimation equations (GEEs) for panel data (xtgee in Stata; family: binomial; link function: logit, exchangeable correlation structure) that allowed robust standard errors. Two models were built to model physician turnover as a function of EHR and productivity metrics: 1 using total EHR time as a variable of primary interest and another using time on inbox, note documentation, and work outside scheduled clinical hours, given that total EHR time is an approximate combination of these 3 metrics. Both models were adjusted by age, gender, physician specialty, and study month. Specialties were grouped into 3 categories: primary care, medical specialties, and surgical specialties. Since both patient volume and intensity include completed appointments per month, we chose to control for intensity in the final model (instead of volume alone) to control for additional dimensions of productivity, thereby potentially explaining more variance in physician turnover that is due to non-EHR factors. Variation in physician turnover that could be explained by the models (McKelvey and Zavoina R 2 ) was computed using multivariable logistic regression models adjusted for correlated physician-level data

Results
Among 335 physicians assessed for eligibility, 314 unique physicians (89.2%) met criteria and were included in the analysis, with a corresponding 5728 physician-months of data ( Figure 1;  Figure 2, stratified by turnover status during the study period.
After controlling for physician age, gender, medical specialty, and study month in the generalized estimation equations model (

Key Results
Despite continued limitations to vendor-derived EHR use data platforms, the findings from this cohort study of physician turnover, productivity, and EHR use, as assessed by vendor-derived EHR use data platforms in a large ambulatory care network, indicate that physician productivity and EHR use metrics were associated with departure. Physician age, gender, and specialty were not associated with departure in bivariate or multivariate analyses. After controlling for these factors, less time on the EHR and inbox were both associated with physician departure. The direction of the association between these time-based EHR use metrics and physician departure was opposite of the hypothesized direction based on assumptions about the association between EHR usability and physician burnout. Lower rates of teamwork on order entry were associated with physician departure, indicating that teamwork may help to prevent physician turnover. This finding is consistent with evidence that team-based care can reduce professional burnout and optimize team performance. [26][27][28][29][30][31][32] Taken together, these findings suggest that a refined, prospective model of vendor-derived EHR data could help to identify physicians at high risk of departure who might benefit from targeted team-based care interventions. Indeed, low demand for a physician's service, time on the EHR and inbox, and rates of teamwork on orders may very well be lead indicators for physicians preparing to leave practice.

Interpretation
Using vendor-derived EHR data to explore physician departure offers several advantages over traditional survey research. Vendor-derived EHR data are more objective, more available (given their EHR-related work. Alternatively, physicians who leave their practice could be more proficient with the EHR and, therefore, more marketable to move to a new position. This would suggest that more proficient EHR users are at an increased likelihood for departure. However, we are unable to conclude from this study any more details of the EHR proficiency of physicians who departed. Future studies could more definitively assess the association of burnout and departure with EHR use by linking survey data with EHR use and productivity metrics with more detailed information on departure. Furthermore, incorporating more detailed analysis of non-time-based EHR use patterns (eg, wrong-patient orders, number of clicks per patient) could help to better determine whether physicians who leave practice and spend less time on EHR work are indeed more proficient than their peers who stay in practice. 23 Despite the effect size of inbox time and teamwork for orders, these metrics' comparatively smaller R 2 indicate that physician characteristics and productivity are likely more strongly associated with physician turnover than EHR use. This finding is consistent with a survey study of 1310 ambulatory clinicians, which found that work culture explained more variation in burnout than the EHR (R 2 , 17.6% vs 1.3%; P < .001). 33 Similarly, physician-perceived EHR usability explained only 5.8% (3) practice setting (including academic or telehealth); (4) medical specialty; and (5) temporal factors, such as the COVID-19 pandemic and vacation time. Additional work in this space would benefit from standardizing vendor-derived EHR data definitions in a way that is clinically relevant and important as well as qualitative exit interviews to provide further insight into existing data (eg, why physicians left their position) and the association between EHR use, burnout, and physician attrition.

Limitations
This study has limitations. The main limitation was that the primary outcome-physician departure-did not include a reason, ie, whether a physician left their position to take a new job, retire, or leave the profession of medicine altogether. Subsequently, the study findings are more likely applicable to departure in general than a specific reason. This study was also subject to limitations related to vendor-derived data quality. For example, some data definitions were changed internally by the vendor during the study period. More importantly, vendor-derived data platforms could not reliably distinguish ambulatory care context and scheduled hours for teaching attendings.
This limitation is not unique to this vendor or practice network. 22 However, we cannot conclude whether the findings from this analysis are generalizable vs specific to the practice settings studied here based on local differences such as culture, climate, and compensation models.

Conclusions
As the first study that we know of to use vendor-derived EHR data to model physician turnover, this study identified physician demand, inbox time, and teamwork on orders as key variables associated with physician departure. Counterintuitively, less time spent on the EHR (in particular on inbox management) was associated with physician departure and warrants further investigation.
Standardizing vendor-derived data definitions (across and within vendor products) with better integration of clinical schedules 23,36,37 could improve data validity and reliability. With greater data validity and reliability, future models could prospectively identify physicians at high risk of departure who would benefit from targeted interventions to improve retention.