Perspectives of VA Primary Care Clinicians Toward Electronic Consultation-Related Workload Burden: A Qualitative Analysis | Electronic Health Records | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Table.  Characteristics of Study Sample, by Clinician Use of e-Consultation
Characteristics of Study Sample, by Clinician Use of e-Consultation
1.
Gupte  G, Vimalananda  V, Simon  SR, DeVito  K, Clark  J, Orlander  JD.  Disruptive innovation: implementation of electronic consultations in a Veterans Affairs Health Care System.   JMIR Med Inform. 2016;4(1):e6. doi:10.2196/medinform.4801 PubMedGoogle Scholar
2.
McAdams  M, Cannavo  L, Orlander  JD.  A medical specialty e-consult program in a VA health care system.   Fed Pract. 2014;31(5):26-31. https://www.mdedge.com/fedprac/article/82261/health-policy/medical-specialty-e-consult-program-va-health-care-systemGoogle Scholar
3.
The Specialty Care Transformational Initiative Evaluation Center. Evaluation of Specialty Care Initiatives: Final E-Consult Evaluation Report. Veterans Health Administration; 2013.
4.
Battaglia  C, Lambert-Kerzner  A, Aron  DC,  et al.  Evaluation of e-consults in the VHA: provider perspectives.   Fed Pract. 2015;32(7):42-48.PubMedGoogle Scholar
5.
Vimalananda  VG, Gupte  G, Seraj  SM,  et al.  Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis.   J Telemed Telecare. 2015;21(6):323-330. doi:10.1177/1357633X15582108 PubMedGoogle ScholarCrossref
6.
Palen  TE, Price  D, Shetterly  S, Wallace  KB.  Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study.   BMC Med Inform Decis Mak. 2012;12:65. doi:10.1186/1472-6947-12-65 PubMedGoogle ScholarCrossref
7.
Liddy  C, Drosinis  P, Keely  E.  Electronic consultation systems: worldwide prevalence and their impact on patient care-a systematic review.   Fam Pract. 2016;33(3):274-285. doi:10.1093/fampra/cmw024 PubMedGoogle ScholarCrossref
8.
Lee  MS, Ray  KN, Mehrotra  A, Giboney  P, Yee  HF  Jr, Barnett  ML.  Primary care practitioners’ perceptions of electronic consult systems: a qualitative analysis.   JAMA Intern Med. 2018;178(6):782-789. doi:10.1001/jamainternmed.2018.0738 PubMedGoogle ScholarCrossref
9.
O’Brien  BC, Harris  IB, Beckman  TJ, Reed  DA, Cook  DA.  Standards for reporting qualitative research: a synthesis of recommendations.   Acad Med. 2014;89(9):1245-1251. doi:10.1097/ACM.0000000000000388 PubMedGoogle ScholarCrossref
10.
Feldstein  AC, Glasgow  REA.  A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice.   Jt Comm J Qual Patient Saf. 2008;34(4):228-243. doi:10.1016/S1553-7250(08)34030-6 PubMedGoogle Scholar
11.
Venkatesh  V, Morris  MG, Davis  GB, Davis  FD.  User acceptance of information technology: toward a unified view.   Manage Inf Syst Q. 2003;27(3):425. doi:10.2307/30036540Google ScholarCrossref
12.
Scientific Siftware Development. Atlas.ti. Accessed October 4, 2020. https://atlasti.com/
13.
Elo  S, Kyngäs  H.  The qualitative content analysis process.   J Adv Nurs. 2008;62(1):107-115. doi:10.1111/j.1365-2648.2007.04569.x PubMedGoogle ScholarCrossref
14.
Venkatesh  V, Thong  JYL, Xu  X.  Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology.   Manage Inf Syst Q. 2012;36(1):157. doi:10.2307/41410412Google ScholarCrossref
15.
Linzer  M, Bitton  A, Tu  S-P,  et al; Association of Chiefs and Leaders in General Internal Medicine (ACLGIM) Writing Group.  The end of the 15–20 minute primary care visit.   J Gen Intern Med. 2015;30(11):1584-1586. doi:10.1007/s11606-015-3341-3 PubMedGoogle ScholarCrossref
16.
Linzer  M, Manwell  LB, Williams  ES,  et al; MEMO (Minimizing Error, Maximizing Outcome) Investigators.  Working conditions in primary care: physician reactions and care quality.   Ann Intern Med. 2009;151(1):28-36, W6-9. doi:10.7326/0003-4819-151-1-200907070-00006 PubMedGoogle ScholarCrossref
17.
Geraghty  EM, Franks  P, Kravitz  RL.  Primary care visit length, quality, and satisfaction for standardized patients with depression.   J Gen Intern Med. 2007;22(12):1641-1647. doi:10.1007/s11606-007-0371-5 PubMedGoogle ScholarCrossref
18.
Simonetti  JA, Sylling  PW, Nelson  K,  et al.  Patient-centered medical home implementation and burnout among VA primary care employees.   J Ambul Care Manage. 2017;40(2):158-166. doi:10.1097/JAC.0000000000000160 PubMedGoogle ScholarCrossref
19.
Parker  PA, Kulik  JA.  Burnout, self- and supervisor-rated job performance, and absenteeism among nurses.   J Behav Med. 1995;18(6):581-599. doi:10.1007/BF01857897 PubMedGoogle ScholarCrossref
20.
Helfrich  CD, Simonetti  JA, Clinton  WL,  et al.  The association of team-specific workload and staffing with odds of burnout among VA primary care team members.   J Gen Intern Med. 2017;32(7):760-766. doi:10.1007/s11606-017-4011-4 PubMedGoogle ScholarCrossref
21.
Rodriguez  KL, Burkitt  KH, Bayliss  NK,  et al.  Veteran, primary care provider, and specialist satisfaction with electronic consultation.   JMIR Med Inform. 2015;3(1):e5. doi:10.2196/medinform.3725 PubMedGoogle Scholar
22.
Simonetti  J, Ho  M.  Veteran Health Administration (VHA) E-Consultation Toolkit. US Department of Veteran Affairs; 2017.
23.
Blijleven  V, Koelemeijer  K, Wetzels  M, Jaspers  M.  Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care, and efficiency of care.   JMIR Hum Factors. 2017;4(4):e27. doi:10.2196/humanfactors.7978 PubMedGoogle Scholar
24.
Liddy  C, Moroz  I, Afkham  A, Keely  E.  Sustainability of a primary care–driven econsult Service.   Ann Fam Med. 2018;16(2):120-126. doi:10.1370/afm.2177 PubMedGoogle ScholarCrossref
25.
Kirsh  S, Carey  E, Aron  DC,  et al.  Impact of a national specialty e-consultation implementation project on access.   Am J Manag Care. 2015;21(12):e648-e654.PubMedGoogle Scholar
26.
Straus  SG, Chen  AH, Yee  H  Jr, Kushel  MB, Bell  DS.  Implementation of an electronic referral system for outpatient specialty care.   AMIA Annu Symp Proc. 2011;2011:1337-1346.PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 1,837
    Citations 0
    Original Investigation
    Health Policy
    October 30, 2020

    Perspectives of VA Primary Care Clinicians Toward Electronic Consultation-Related Workload Burden: A Qualitative Analysis

    Author Affiliations
    • 1Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Eastern Colorado Health Care System, Aurora
    • 2Department of Health Services, University of Washington, Seattle
    • 3Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus, Aurora
    • 4Office of Specialty Care and Specialty Care Transformation, Washington, DC
    • 5Division of Cardiology, University of Colorado School of Medicine, Aurora
    • 6Seattle Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, Seattle, Washington
    • 7Division of Hospital Medicine, University of Colorado School of Medicine, Aurora
    JAMA Netw Open. 2020;3(10):e2018104. doi:10.1001/jamanetworkopen.2020.18104
    Key Points

    Question  What are primary care clinicians’ perceptions regarding electronic consultation and their clinical workload?

    Findings  In this qualitative study including 34 primary care clinicians in the US Veterans Health Administration health care system, 3 themes regarding electronic consultation and workload were identified. The clinicians viewed electronic consultations as an added time burden, as shifting responsibilities from specialists to themselves, and even with the increased workload, as improving timeliness and quality of care.

    Meaning  The findings of this study suggest that further research is warranted to assess whether the increased workload is objective or perceived.

    Abstract

    Importance  Electronic consultation (e-consultation) is increasingly being adopted to expand access to specialty care and reduce health care costs. Little is known about clinicians’ perceptions of using e-consultations, which may be associated with program adoption.

    Objective  To identify perceptions of primary care clinicians in the US Veterans Health Administration (VHA) system about e-consultation and workload.

    Design, Setting, and Participants  A qualitative study using semistructured interviews was conducted from September 2017 through March 2018 in a national sample of VHA primary care clinics in the US. Participants were primary care clinicians who had at least 300 total patient encounters from July 2016 to June 2017, including at least 1 e-consultation request. A convenience sample of participants was recruited using email invitations. Deductive and inductive content analysis were used to identify themes. Data were analyzed from October 2017 to April 2018.

    Exposures  Use of e-consultation.

    Main Outcomes and Measures  Primary care clinician perspectives regarding e-consultation and their workload.

    Results  A total of 34 primary care clinicians enrolled working across 27 VHA clinical sites were included; 9 (26%) were between ages 40-49 years; 23 (68%) were female. Three themes were identified. First, the process of entering, tracking, and following up on e-consultations added a time burden to primary care clinicians. Second, e-consultation was perceived to shift diagnostic and follow-up responsibilities from specialists to primary care clinicians. Third, e-consultations were thought to improve the timeliness and quality of care provided despite a perceived increase in workload.

    Conclusions and Relevance  In this study, participants perceived e-consultation as valuable for patient care but also as an increase in their workload. Further work is warranted to quantify the workload increase on clinician burnout, job satisfaction, and turnover.

    Introduction

    Health care systems are adopting electronic consultation (e-consult) programs to increase access to specialty care services.1,2 The Veterans Health Administration (VHA) identified access to specialty services as a leading clinical priority and adopted e-consults as one mechanism to achieve this goal.2 E-consultation was piloted within several VHA medical centers in 2009 and expanded nationally in 2011.3 However, there is substantial variation in use across VHA service regions and medical specialties.3 Reasons for this variation are unclear, but likely include clinician, specialty, and system-level factors.

    Improving our understanding of clinician-level perceptions regarding e-consultation is important for informing program implementation and adoption, and ensuring that e-consults facilitate delivery of high-quality care. Prior evaluations demonstrated that primary care clinicians and specialists believe that e-consults may improve communication between clinicians.4-7 Primary care clinicians and specialists across several health care systems have reported that use of e-consultations increases the efficiency of care and reduces appointment burden for patients.4-7

    However, little is known regarding challenges to using e-consultations, which could help explain clinician-level variation in their use. In some cases, primary care clinicians have reported that e-consultation templates are difficult to use or are inefficient.4 In a 2018 study, primary care clinicians working in safety-net clinics described e-consultations as “increased administrative burden, broadened clinical responsibility, and restructuring of specialty care delivery.”8(pE1) A perception among primary care clinicians that e-consultation increases their workload may hinder adoption and lead to negative outcomes among the primary care workforce. The aim of this study was to explore perceptions regarding the association between e-consultations and workload among a national sample of VHA primary care clinicians.

    Methods
    Study Design

    We conducted semistructured interviews with primary care clinicians who used e-consultations and were employed by VHA in 2017. The activities were undertaken in support of a VHA operational project and did not constitute research, in whole or in part, in compliance with VHA Handbook 1058.05, which waives the requirement for institutional review board approval.

    Participant Identification and Recruitment

    We identified all VHA primary care clinicians using data from the VHA Corporate Data Warehouse. Primary care clinicians with at least 300 total patient encounters, including at least one e-consultation, between July 2016 and June 2017 were eligible for inclusion. Sites with fewer than 500 total visits, fewer than 3 specialties, or without any e-consultation visits were excluded. We enrolled a national sample of primary care clinicians because of the potential for variations in the implementation of e-consultations at the local level, and because this process was implemented across many clinical settings that likely differ with respect to their organizational climate. We sampled clinicians using a stratified purposeful strategy based on (1) clinician use of e-consultations (frequency, high or low), (2) facility use of e-consultations (frequency, high or low), (3) clinic location (Veteran Affairs Medical Center [VAMC] or community-based outpatient clinic [CBOC]), and (4) medical training (doctor vs nurse practitioner or physician assistant) (Table). We recruited and enrolled participants to balance each stratum until thematic saturation was reached within the overall sample. Primary care clinicians were recruited using email invitations followed by up to 3 reminder emails.

    The e-consultation processes within VHA is depicted in eFigure 1 in the Supplement. An example of VHA e-consultation templates is shown in eFigure 2 in the Supplement.

    Interviews

    Development of the semistructured interview guide was informed by the Practical, Robust, Implementation, and Sustainability Model (PRISM).9,10 Items aimed to elicit information on primary care clinicians’ experiences (including associations between e-consultations and workload), and facilitators and barriers to e-consultation use (eMethods in the Supplement).11 Clinicians who responded to recruitment procedures were scheduled to complete 30-minute telephone interviews with trained interviewers (C.L., M.L., P.G., and E.L.). Interviews were conducted from September 2017 through March 2018. Informed consent was obtained orally before interviews were audio recorded, deidentified, and saved on a secure server. Recordings were transcribed verbatim. Transcriptions were coded and managed in Atlas.ti version 7 (Scientific Software Development).12

    Qualitative Analysis

    We used inductive-deductive content analysis, with predetermined code families based on barriers to using e-consultations. Inductive coding captured emergent themes. Transcripts were coded by qualitative analysts with various backgrounds, including an anthropologist (C.L.), psychologist (P.G.), social worker (R.J.), and health service researchers (M.L., E.L.). We used an iterative, team-based approach to achieve consensus. Each analyst initially independently coded the same 3 transcripts. Together, codes and emergent themes were discussed to review points of divergence and convergence and new code categories, and a refined codebook was subsequently developed. The team continued to meet and discuss new codes, update the codebook, and introduce new findings until all transcripts were coded. All quotations associated with emerging themes were reviewed to refine the theme, identify contrary cases, and assess saturation. Through further analyses and discussions, the team determined thematic saturation was reached when discussions of new notable findings waned. This study followed the Standards for Reporting Qualitative Research (SRQR) reporting guideline for qualitative studies.10,13

    Results

    Of the 34 primary care clinicians enrolled working across 27 VHA clinical sites, 9 (26%) were between the ages of 40-49 years; 23 (68%) were female (Table). We identified 3 themes regarding the association between e-consultation and workload among primary care clinicians, including (1) the process of entering, tracking, and following up on e-consultations added time burden to primary care clinicians; (2) primary care clinicians perceived that e-consultation shifts diagnostic and follow-up responsibilities from specialists to themselves; and (3) e-consultations improve the timeliness and quality of care provided despite the perceived increase in workload.

    Time Burden

    Many primary care clinicians felt that e-consultations were time consuming. Specific processes that were time consuming included using templates that were not user-friendly or that included requirements to provide patient data which they felt were not relevant for care: “To be honest, I think it’s ridiculous how many questions they ask us…I’m like, I don’t know. I have to go look that up…so why do I have to waste my time with a yes or no button…” (high user; high-use VAMC).

    Others noted that templates for some specialties were “incredibly wordy,” required tangential information to complete, or required information that was not readily available in the medical record: “The [mammogram] consult…is so incredibly wordy…why am I wasting my time?” (high user; high-use VAMC).

    When medical information was not readily accessible, primary care clinicians reported using appointment time to navigate patients’ medical records: “They expect you to put in all of the physical findings and exams. I’m like, give me a break, in 30 minutes, if I were to do that… I can’t be doing their diabetes, their hypertension, their cholesterol and everything, so that’s where, you know, sometimes I get the burden of being a primary care physician, where I’m expected to do all the initial work for the specialist” (high user; high-use VAMC).

    In some cases, the time burden required to place e-consultations forced primary care clinicians to complete them outside of allotted appointment times and experienced challenges in transferring information from the patient medical record to the e-consultation template: “Well, the ones I don’t like is when I’m in the middle of trying to do it and they’re asking all these questions that I don’t have the answers to, and I can’t get back to the chart” (low user; low-use VAMC).

    Tracking e-consultations added to primary care clinicians’ workload, and sometimes made it difficult to close the communication loop with the specialist and patient: “I think the biggest downside for e-consults…is that…instead of having them be seen by…a specialist, you get sort of recommendations back that then require a lot more work on your part in terms of ordering tests, following up with the patient, communicating things, and I think that workload shift from specialty to primary care is my biggest barrier” (low user; high-use VAMC).

    In some cases, primary care clinicians felt that specialists did not respond to e-consultations in a timely manner, or e-consultations were discontinued requiring the primary care clinician to re-enter the consultation. One primary care clinician described the burden of tracking e-consultations: “I think (e-consults) being discontinued and then re-entering it is just needless work…it falls on me to go and enter, click all the buttons… and then I have to make sure that I see all the consults and have to be alerted because if doctor makes a comment that tells me to do something, [and] if I'm not getting all the alerts, I'll miss stuff... you have to see every little note, patient scheduled, talked to patient, patient no-showed, and you’re getting every single alert because you never know which ones are pertinent…so it increases the workload of alerts that you have to review” (high user; high-use VAMC).

    Similarly, another clinician said: “(My e-consultations) could go out to potentially up to 3 months of chart review which is beyond acceptable… so that has generated a mechanism whereby I must find some sort of tracking system so those people are not forgotten” (high user; low-use VAMC).

    Many primary care clinicians described a risk of losing track of the e-consultation if specialists did not direct their response specifically to them: “…the return response really should come by way of an additional view alert… they’ll respond back and forget to tag us on, and then it would be so easy to forget that you’ve either put it in or further not know that it had been responded to…and only serendipitously would you find the response” (high user; low-use VAMC).

    Increased Responsibility

    Participants reported that using e-consultations resulted in a shift of work from specialists to themselves. Specialists often recommended that primary care clinicians obtain diagnostic testing that they would typically not be responsible for ordering or interpreting. In some cases, primary care clinicians did not have the approval to order specialized tests: “…they want us to order testing that is not appropriate for primary care… I’m not the one to order a CT myelogram…Yes, you want an MRI, it’s a different story, but a CT myelogram, what am I supposed to do with it, to put another consult to them?” (high user; high-use VAMC).

    Some primary care clinicians were uncomfortable with the responsibility of ordering and being responsible for specialized tests: “… I’m working up someone with potential leukemia, (and) ordering all these diagnostic tests, some of which, it’s not within my scope of practice to interpret…So, that’s a little bit stressful, the whole idea of doing some of this workup that’s recommended through e-consults and not being educated exactly on what it means or what I’m doing, that kind of increases the workload” (low user; CBOC).

    One clinician seemed to feel that this shift of responsibility has become normalized: “I have a patient… [with] chronic anemia… I pretty much do some of the basic workup...and I ask them, what more do I still need to do? That’s when I get the input from them saying, OK, do this mutation or send a special, peripheral blood smear, and they would tell me who to send it to. That’s a little extra work for us, but I guess that’s how it is” (high user; high-use VAMC).

    In some instances, there was uncertainty in who was responsible for ordering tests: “…they’ll say they want labs… I don’t know if they sent them to the lab there, or if they’re gonna come back to [me] and I’m supposed to do it” (high user; CBOC). One primary care clinician said, “they would just sort of wash their hands of the problem and make it somebody’s else’s problem” (high user; CBOC).

    Electronic Consultation Value

    Despite the increased workload, primary care clinicians expressed that e-consultations were generally effective at coordinating timely care without trade-offs to quality or continuity: “I do get a more timely response…rather than having to have the veteran wait for a week or 2 weeks or sometimes 4 weeks to get in with the specialist” (low user; CBOC).

    E-consultation has alleviated some primary care clinician concerns about the uncertainty of patients following through with in-person appointments because the “person doesn’t have to come back and forth to multiple appointments” (high user; low-use VAMC). Several clinicians reported that e-consultations have reduced unnecessary specialty appointments and patient travel. In addition, 1 primary care clinician felt that the additional workload carried by primary care clinicians could cut down the work for specialists, who he perceived as overworked: “It’s added to the primary provider…but it’s cutting down on our… overworked specialists… when we don’t have the money or we can’t pay them enough to come, and they’ve got a huge amount of work on their plate…” (low user, low-use VAMC).

    Patient satisfaction, quality of care, and increasing access to specialty care that would otherwise be unavailable were reasons primary care clinicians continued using e-consultations despite their increased workload: “E-consults have been very good for the patient. Patients are happier and they improve patient care and patient safety. The e-consult process has made more work for us…but on the other hand, before we had them… you just had no one else to help you with it, so it’s taken some pressure off me and I think the patients are getting better care, but it’s more work” (high user; CBOC).

    Discussion

    In this study of a national sample of VHA primary care clinicians who use e-consultation, most perceived e-consultations as increasing their clinical workload. We identified specific mechanisms by which e-consultations added to their workload, including time needed to enter and track consultations, and shifting responsibility for the diagnosis and management of conditions that would typically fall outside their scope of practice. That e-consultation may increase clinical workload among primary care clinicians has several important implications for VHA and other health care systems which have prioritized this mechanism of care delivery.

    A perception that e-consultations create additional work may hinder their adoption and use.11,14 Prior studies have already shown that primary care clinicians have limited time to complete their clinical tasks, which has been exacerbated by increases in patient complexity, new responsibilities related to team-based care, and technologic advances (eg, e-consultations).15,16 Additional studies among clinicians who have abandoned this technology or have yet to adopt it may be helpful in improving our understanding of these challenges.

    An important question is what trade-offs are occurring to allow for this increase in workload. For example, it is unknown whether clinicians are avoiding other recommended clinical tasks (eg, dietary counseling) to allow for the completion and tracking of e-consultations. If clinicians are completing e-consultation requests during clinical encounters, this process may reduce the amount of time they are able to spend in face-to-face communication with their patients, which may have an adverse effect on patient satisfaction.17 Alternatively, primary care clinicians may adapt to this workload change by increasing work hours or completing work during nonclinical time. This trade-off could have important implications for the primary care workforce given the high prevalence of burnout among VHA primary care staff,18 and that burnout is associated with staff absenteeism19 and turnover.20

    An important question is whether the increased workload reported by primary care clinicians in this sample is objective or perceived, and how specialists consider e-consultations to affect their workload. Prior studies revealed that specialists report time constraints in completing consultations, in addition to increased e-consultation volumes over time.1,21 One study found that specialists perceived that 27% of their e-consultations were new work.2 That is, those consultations would not have occurred in the absence of an e-consultation mechanism.22 Additional studies using survey or time-in-motion methodology could be helpful in validating clinicians’ perceptions about workload. If e-consultation is adding to workload among primary care clinicians or specialists, the VHA and other health care systems may need to consider how to account for that work within productivity requirements and reimbursement plans.

    We identified several practical ways in which primary care clinicians’ concerns could potentially be addressed. First, most clinicians described ways in which templates could facilitate or hinder e-consultation. Future work could aim to develop specialty-specific high-quality templates that facilitate efficient and effective communication between clinicians. Clinicans in this sample also reported several communication challenges, including confusion regarding test ordering and interpretation responsibilities, and communication with patients. Facility-specific and specialty-specific service agreements developed in a collaborative manner may be useful in addressing these issues. Consistent with this approach, the VHA has developed a clinician toolkit which outlines processes by which services and clinical sites might coordinate e-consultation procedures.22

    Several participants reported concerns about tracking e-consultations and ways in which responses could be lost. Primary care clinicians described developing workarounds to manage this, such as creating their own tracking system to ensure completion of e-consultations. These workarounds were also described as contributing to their workload, and it is unknown how effective they may be in preventing adverse events (eg, test results lost to follow-up). Prior studies illustrated that clinical workarounds, although potentially necessary when clinicians are faced with unintended challenges, are generally suboptimal and may jeopardize patient safety, efficiency, and effectiveness of care.23

    Despite these challenges, primary care clinicians generally agreed that e-consultations improved quality and access to care. This agreement is consistent with other studies among VHA and non-VHA clinicians, which have demonstrated improved timeliness for specialty care input, reduced unnecessary face-to-face visits, improved clinician communications and patient care, and that e-consultations have an educational benefit for clinicians.2,8,21,24-26

    This study was conducted within the VHA. Thus, some findings (eg, frustration with e-consultation templates) may be specific to the VHA. However, the main finding of this study is consistent with an abundance of organizational function literature demonstrating that new processes and technology often have adverse and unexpected effects on clinicians.2,8,21,24-26 Other health care systems may need to consider assessing the extent to which implementing e-consultation may shift clinical responsibilities among clinicians, and develop strategies to mitigate or account for that shift.

    Limitations

    This study has limitations. The generalizability of findings is limited given the qualitative study design, and we enrolled a small sample of clinicians working in VHA clinics. Although we used several methods to limit interpreter bias, the identified themes could have been affected by interpreter bias. Similarly, participants’ reports of experiences using e-consultations are subject to recall and social desirability biases.

    Conclusions

    To our knowledge, this study provides the first insights into perceptions of workload associated with e-consultation among a national sample of clinicians. That many primary care clinicians feel that e-consultations increase their workload has implications for the future use of this technology and the well-being of patients and clinicians. Additional work is warranted to quantify the potential time burden reported by primary care clinicians, capture specialists’ perspectives, and assess workload increases with regard to clinician outcomes such as burnout and turnover.

    Back to top
    Article Information

    Accepted for Publication: June 17, 2020.

    Published: October 30, 2020. doi:10.1001/jamanetworkopen.2020.18104

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Lee M et al. JAMA Network Open.

    Corresponding Author: Marcie Lee, MPH, MA, Qualitative Analyst, Denver Seattle Center of Innovation (COIN), Eastern Colorado Healthcare System, 1700 N Wheeling St, Building P, Aurora, CO 80045 (marcie.lee@va.gov).

    Author Contributions: Dr Simonetti had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Lee, Leonard, Greene, Whittington, Kirsh, Ho, Sayre, Simonetti.

    Acquisition, analysis, or interpretation of data: Lee, Leonard, Greene, Kenney, Simonetti.

    Drafting of the manuscript: Lee, Leonard, Greene.

    Critical revision of the manuscript for important intellectual content: Leonard, Greene, Kenney, Whittington, Kirsh, Ho, Sayre, Simonetti.

    Statistical analysis: Greene, Simonetti.

    Obtained funding: Kenney, Kirsh, Simonetti.

    Administrative, technical, or material support: Leonard, Greene, Kenney, Whittington, Sayre, Simonetti.

    Supervision: Leonard, Greene, Kirsh, Sayre, Simonetti.

    Conflict of Interest Disclosures: The authors were employed by or affiliated with the VA. Dr Whittington reported receiving grants from VA during the conduct of the study; personal fees from Ipsen, personal fees from Merck, and personal fees from Sanofi outside the submitted work. Dr Ho reported receiving other support from VHA during the conduct of the study, and receiving by grants from NHLBI, VA HSR&D, and University of Colorado School of Medicine; having a research agreement with Bristol-Myers Squibb through the University of Colorado, and serving as the Deputy Editor for Circulation: Cardiovascular Quality and Outcomes. No other disclosures were reported.

    Funding/Support: This work was supported and awarded by the Office of the Under Secretary for Health (USH), Office of Veterans Access to Care (OVAC), and the Office of Rural Health (ORH).

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Veterans Affairs.

    Additional Contributions: We thank Emily Lawrence, MPH, and Rachel Johnson, MSW, LCSW, both of VA Denver-Seattle COIN, for data collection and analysis related to this project. They received compensation as HSR&D employees.

    References
    1.
    Gupte  G, Vimalananda  V, Simon  SR, DeVito  K, Clark  J, Orlander  JD.  Disruptive innovation: implementation of electronic consultations in a Veterans Affairs Health Care System.   JMIR Med Inform. 2016;4(1):e6. doi:10.2196/medinform.4801 PubMedGoogle Scholar
    2.
    McAdams  M, Cannavo  L, Orlander  JD.  A medical specialty e-consult program in a VA health care system.   Fed Pract. 2014;31(5):26-31. https://www.mdedge.com/fedprac/article/82261/health-policy/medical-specialty-e-consult-program-va-health-care-systemGoogle Scholar
    3.
    The Specialty Care Transformational Initiative Evaluation Center. Evaluation of Specialty Care Initiatives: Final E-Consult Evaluation Report. Veterans Health Administration; 2013.
    4.
    Battaglia  C, Lambert-Kerzner  A, Aron  DC,  et al.  Evaluation of e-consults in the VHA: provider perspectives.   Fed Pract. 2015;32(7):42-48.PubMedGoogle Scholar
    5.
    Vimalananda  VG, Gupte  G, Seraj  SM,  et al.  Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis.   J Telemed Telecare. 2015;21(6):323-330. doi:10.1177/1357633X15582108 PubMedGoogle ScholarCrossref
    6.
    Palen  TE, Price  D, Shetterly  S, Wallace  KB.  Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study.   BMC Med Inform Decis Mak. 2012;12:65. doi:10.1186/1472-6947-12-65 PubMedGoogle ScholarCrossref
    7.
    Liddy  C, Drosinis  P, Keely  E.  Electronic consultation systems: worldwide prevalence and their impact on patient care-a systematic review.   Fam Pract. 2016;33(3):274-285. doi:10.1093/fampra/cmw024 PubMedGoogle ScholarCrossref
    8.
    Lee  MS, Ray  KN, Mehrotra  A, Giboney  P, Yee  HF  Jr, Barnett  ML.  Primary care practitioners’ perceptions of electronic consult systems: a qualitative analysis.   JAMA Intern Med. 2018;178(6):782-789. doi:10.1001/jamainternmed.2018.0738 PubMedGoogle ScholarCrossref
    9.
    O’Brien  BC, Harris  IB, Beckman  TJ, Reed  DA, Cook  DA.  Standards for reporting qualitative research: a synthesis of recommendations.   Acad Med. 2014;89(9):1245-1251. doi:10.1097/ACM.0000000000000388 PubMedGoogle ScholarCrossref
    10.
    Feldstein  AC, Glasgow  REA.  A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice.   Jt Comm J Qual Patient Saf. 2008;34(4):228-243. doi:10.1016/S1553-7250(08)34030-6 PubMedGoogle Scholar
    11.
    Venkatesh  V, Morris  MG, Davis  GB, Davis  FD.  User acceptance of information technology: toward a unified view.   Manage Inf Syst Q. 2003;27(3):425. doi:10.2307/30036540Google ScholarCrossref
    12.
    Scientific Siftware Development. Atlas.ti. Accessed October 4, 2020. https://atlasti.com/
    13.
    Elo  S, Kyngäs  H.  The qualitative content analysis process.   J Adv Nurs. 2008;62(1):107-115. doi:10.1111/j.1365-2648.2007.04569.x PubMedGoogle ScholarCrossref
    14.
    Venkatesh  V, Thong  JYL, Xu  X.  Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology.   Manage Inf Syst Q. 2012;36(1):157. doi:10.2307/41410412Google ScholarCrossref
    15.
    Linzer  M, Bitton  A, Tu  S-P,  et al; Association of Chiefs and Leaders in General Internal Medicine (ACLGIM) Writing Group.  The end of the 15–20 minute primary care visit.   J Gen Intern Med. 2015;30(11):1584-1586. doi:10.1007/s11606-015-3341-3 PubMedGoogle ScholarCrossref
    16.
    Linzer  M, Manwell  LB, Williams  ES,  et al; MEMO (Minimizing Error, Maximizing Outcome) Investigators.  Working conditions in primary care: physician reactions and care quality.   Ann Intern Med. 2009;151(1):28-36, W6-9. doi:10.7326/0003-4819-151-1-200907070-00006 PubMedGoogle ScholarCrossref
    17.
    Geraghty  EM, Franks  P, Kravitz  RL.  Primary care visit length, quality, and satisfaction for standardized patients with depression.   J Gen Intern Med. 2007;22(12):1641-1647. doi:10.1007/s11606-007-0371-5 PubMedGoogle ScholarCrossref
    18.
    Simonetti  JA, Sylling  PW, Nelson  K,  et al.  Patient-centered medical home implementation and burnout among VA primary care employees.   J Ambul Care Manage. 2017;40(2):158-166. doi:10.1097/JAC.0000000000000160 PubMedGoogle ScholarCrossref
    19.
    Parker  PA, Kulik  JA.  Burnout, self- and supervisor-rated job performance, and absenteeism among nurses.   J Behav Med. 1995;18(6):581-599. doi:10.1007/BF01857897 PubMedGoogle ScholarCrossref
    20.
    Helfrich  CD, Simonetti  JA, Clinton  WL,  et al.  The association of team-specific workload and staffing with odds of burnout among VA primary care team members.   J Gen Intern Med. 2017;32(7):760-766. doi:10.1007/s11606-017-4011-4 PubMedGoogle ScholarCrossref
    21.
    Rodriguez  KL, Burkitt  KH, Bayliss  NK,  et al.  Veteran, primary care provider, and specialist satisfaction with electronic consultation.   JMIR Med Inform. 2015;3(1):e5. doi:10.2196/medinform.3725 PubMedGoogle Scholar
    22.
    Simonetti  J, Ho  M.  Veteran Health Administration (VHA) E-Consultation Toolkit. US Department of Veteran Affairs; 2017.
    23.
    Blijleven  V, Koelemeijer  K, Wetzels  M, Jaspers  M.  Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care, and efficiency of care.   JMIR Hum Factors. 2017;4(4):e27. doi:10.2196/humanfactors.7978 PubMedGoogle Scholar
    24.
    Liddy  C, Moroz  I, Afkham  A, Keely  E.  Sustainability of a primary care–driven econsult Service.   Ann Fam Med. 2018;16(2):120-126. doi:10.1370/afm.2177 PubMedGoogle ScholarCrossref
    25.
    Kirsh  S, Carey  E, Aron  DC,  et al.  Impact of a national specialty e-consultation implementation project on access.   Am J Manag Care. 2015;21(12):e648-e654.PubMedGoogle Scholar
    26.
    Straus  SG, Chen  AH, Yee  H  Jr, Kushel  MB, Bell  DS.  Implementation of an electronic referral system for outpatient specialty care.   AMIA Annu Symp Proc. 2011;2011:1337-1346.PubMedGoogle Scholar
    ×