Telemedicine vs Telephone Consultations and Medication Prescribing Errors Among Referring Physicians

Key Points Question Does the use of video telemedicine for pediatric consultations to referring hospital emergency departments (EDs) result in less frequent medication errors than the current standard of care, telephone consultations? Findings In this cluster randomized crossover trial that included 696 acutely ill children presenting to 15 community and rural EDs, there were no statistically significant differences in physician-related medication errors between children assigned to receive telephone consultations vs video telemedicine consultations. Meaning These findings suggest that the use of video telemedicine to conduct consultations for acutely ill children in rural and community EDs does not result in less frequent medication errors than consultations done by telephone.


INTERVENTIONS
Participating EDs were randomized to use video telemedicine or telephone for consultations with pediatric critical care physicians according to 1 of 4 unbalanced (3 telemedicine to 1 telephone) crossover treatment assignment sequences.

MAIN OUTCOMES AND MEASURES
Pharmacists reviewed medical records to document physicianrelated medication errors using a previously validated instrument.Multilevel logistic regression analyses were performed to create models with the medication order as the unit of analysis and adjusting for age, the log-transformed Revised Pediatric Emergency Assessment Tool score, and hospital study period.
Of the 2414 medication orders, errors occurred in 124 cases (5.1%), including 26 of 513 orders (5.1%) in the telephone cohort and 98 of 1901 orders (5.2%) in the telemedicine cohort.In the multivariable analysis, the adjusted odds ratio of experiencing a medication error among those assigned to telemedicine was 0.86 (95% CI, 0.49-1.52;P = .61).

CONCLUSIONS AND RELEVANCE
This cluster randomized crossover trial found no statistically significant differences in physician-related medication errors between critically ill children assigned to receive telephone consultations vs video telemedicine consultations.

Introduction
Medication errors are a leading preventable cause of adverse events in health care. 1 Patients presenting to emergency departments (EDs) are at high risk of medication errors for several reasons, including that EDs are traditionally chaotic environments 2,3 and often lack access to pharmacist review. 4Children are at even higher risk, up to 3 to 4 times more likely than adults to experience a medication error during an ED encounter. 4,5Children require weight-based dosing, and because of relatively low volumes of children evaluated in non-children's hospital EDs, practitioners may be less familiar and less confident in ordering medications for children. 6,7One study 8 found that 39% of medication orders from encounters with critically ill children presenting to rural EDs in northern California had at least 1 medication error.
Consultation with a pediatric specialist by either telephone or video telemedicine has the potential to decrease the frequency of medication errors, given that pediatric specialists are more experienced in pediatric prescribing.A nonrandomized study 9 of 8 critical access hospitals found that pediatric ED encounters with telemedicine consultations were associated with fewer medication errors than ED encounters with telephone consultations and ED encounters with no consultations.
To our knowledge, there have been no randomized trials evaluating whether telemedicine consultations provide additional benefit over telephone consultations with respect to physicianrelated medication errors in community EDs.To address this gap, we conducted a large, multiinstitutional, 2-year randomized trial to determine the impact of pediatric video telemedicine emergency consultations compared with telephone consultations on physician-related medication errors.We hypothesized that critically ill children who received care in these EDs during periods where video telemedicine consultations were assigned would experience fewer physician-related medication errors than children who received care during periods where telephone consultations were assigned.

Study Design and Participating Hospitals
We sought to conduct a pragmatic trial, conducted in a clinical practice setting and with broad patient inclusion criteria and clinicians, designed to evaluate the impact of telemedicine consultations on physician-related medication errors in a robust clinical trial. 10The resulting trial design was a pragmatic, cluster randomized trial that used an unbalanced crossover design in which EDs, the units of randomization, were stratified by size and geographic location.This study received institutional review board approval from University of California, Davis.We followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline for randomized clinical trials.The need for informed consent was waived because the patients' data were deidentified, in accordance with 45 CFR §46.The study protocol is shown in Supplement 1.
From the more than 30 hospital EDs in the University of California Davis Children's Hospital (UCDCH) catchment area with video telemedicine capabilities, we selected 16 EDs to ensure a diverse sample.These included 6 critical access hospitals and 12 hospitals located in Health Resources and Services Administration-designated rural geographic areas.Eight of the EDs had access to on-call consulting pediatricians, but none of the hospitals had local pediatric hospitalist physicians, pediatric emergency medicine physicians, or pediatric critical care physicians.All EDs had electronic health records that included computerized physician order entry (CPOE).We chose a crossover design over a parallel group design (in which each ED would be randomized to either telemedicine or telephone for the entire duration of the study) for methodological, ethical, and statistical reasons.By assigning EDs to 1 of 4 period sequences where each sequence contained 3 periods assigned to telemedicine consultation and 1 period assigned to telephone consultation, the study design allows each ED to serve as its own control.This required a smaller sample size than a parallel cluster randomized trial would need, while also avoiding the ethical hurdle that could arise from assigning some EDs to a telephone-only group.We choose a 3:1 unbalanced allocation of the study periods because, from previous experience and because of the additional workflow burden, we anticipated that when EDs were in periods assigned to video telemedicine, there would still be a high rate of telephone use.

JAMA Network Open | Emergency Medicine
The 16 hospitals were binned into 4 strata.Within each stratum, EDs were randomized into 1 of the four 4-period sequences (ie, schedules) that covered an enrollment period of 2 years for each hospital.In each schedule, a letter refers to a 6-month block.With T representing video telemedicine time blocks and P representing telephone time blocks, the 4 possible schedules were PTTT, TPTT, TTPT, or TTTP.To remind and encourage consulting physicians to use the assigned modality, interfacility transfer request cards, used to document all consultation requests, included participating ED names and modality assignments.In addition, as a nudge strategy, emails were sent to each consulting physician following every consultation and were shared at weekly physician meetings with protocol adherence rates for all physicians.These efforts were meant to encourage the use of the assigned modality throughout the trial.

Patient Population
Children were included in the study if they were younger than 15 years and if the treating emergency medicine physician placed a telephone call to the UCDCH transfer center to request a consultation from a pediatric critical care physician and/or request to transfer to the pediatric intensive care unit.
The age of 15 years was selected as a threshold to reduce potential bias introduced by the fact that, in some instances, children aged 16 years and older are routed to adult critical care physicians.Children with acute injuries were routed to the pediatric trauma surgeons and were not included in this study.

Video Telemedicine Intervention
Video telemedicine units consisted of pole-mounted, high-resolution video conferencing units with remote pan-tilt-zoom capabilities.All consultations were initiated by a telephone call from the outside ED to the UCDCH transfer center to connect to a UCDCH pediatric critical care physician.
During the telemedicine-assigned periods, the UCDCH pediatric critical care physicians were encouraged to complete the consultations over video telemedicine.During the telephone-assigned consultations, the physicians were encouraged to complete the consultations over the telephone.
Although physicians were made aware of the assigned consultation modality, deviations were permitted, and the consultation modality was ultimately determined by agreement between physicians.In neither case did the UCDCH pediatric critical care physicians have access to the electronic health records at the participating ED.Pediatric critical care physicians could initiate the telemedicine call at a workstation in the UCDCH pediatric intensive care unit.Video telemedicine consultations typically involved some combination of the treating physician, nurses, and respiratory therapists in addition to the patient and parent or guardian (when present).More details on the technological specifications of the video telemedicine setup and integration into workflow have been reported elsewhere. 11

Sample Size
Our trial was designed to satisfy several objectives, including assessing impacts on medication errors.
From previous experience conducting pediatric telemedicine emergency consultations, we expected approximately 2% of all pediatric ED encounters at participating EDs to result in pediatric specialty consultation, and 25% of these to receive a pediatric critical care consultation.Sample size  11 The planned enrollment of at least 448 patients provided at least 80% power to detect a clinically important adjusted odds ratio of 0.3 for the medication error outcome.This calculation assumed that approximately 10% of medications during the telephone-only periods would result in a physician-related error (defined later), on the basis of previously reported error incidences. 8,9nce, an odds ratio of 0.3 applied to an incidence of 10% results in an incidence lowered to 3.2%.

Study Outcomes: Medication Errors
Medication errors were defined using the definitions and taxonomy from the National Coordinating Council for Medication Error Reporting and Prevention. 12We used Lexi-Comp, Inc, guidelines 13 for appropriate medication dosing ranges.From these published definitions and guidelines, a medication error evaluation instrument was developed and validated by our research group. 8We focused on physician-related ED medication errors because we hypothesized that telemedicine and telephone consultations would most likely affect emergency medicine physician prescribing rather than administration and delivery of the medication.Physician-related ED medication errors were defined as those involving a wrong dose, defined as between 10% and 25% of appropriate dose, a wrong or inappropriate medication for the patient's condition, a wrong route of administration, a wrong dosage form, or errors regarding patient information, such as a known allergy. 8,9The instrument was applied to all patient records and was used to evaluate all medication orders, including prescription and nonprescription medications.The data collection instrument and guidelines used to determine medication errors are provided in the eAppendix in Supplement 2.

Data Collection
Hospitals were enrolled for 2 years, with staggered start dates between September 2014 to March entered into a REDCap database. 14Severity of illness was measured using the validated Revised Pediatric Emergency Assessment Tool. 15The entire medical record from the ED visit, including physician documentation, vital signs, nursing notes, laboratory results, medication orders, and other records, were available for pharmacist review.Three pediatric pharmacists (A.C.V.A.P., and A. S.P.), who were not involved in the patient's care, independently determined medication errors and assigned error severity using the redacted electronic medical records; 90% of the records were reviewed by 1 of the 3 pediatric pharmacists.To estimate interrater reliability, 10% of patient records were reviewed by all 3 pediatric pharmacists.

Statistical Analysis
Analyses were conducted from May 2022 to January 2023.We used Pearson χ 2 test to perform bivariable comparisons of the 2 consultation modes for categorical variables and the Wilcoxon-Mann-Whitney test for continuous, nonnormal variables, with significance set at 2-sided P < .05.The units of analysis for this study were individual medications, which were nested within patients seen in the originating EDs.We used 3-level mixed-effects logistic regression with random intercepts for patient and for originating ED.For the patients rated by the 3 pediatric pharmacists, we fit a mixed-effects model with separate random intercepts for originating EDs, patients, and medication and used the sum of the variance components for these terms relative to the sum of all 4 variance components (including residual error) to estimate interrater reliability.Covariates were chosen a priori and included patient age, the natural log-transformed severity of illness as measured by Revised Pediatric Emergency Assessment Tool, and hospital time-period in hospital (eg, first, second, third, and fourth) to adjust for secular trends.Because this was a pragmatic trial, we anticipated deviations in the treatment groups and report intention-to-treat, treatment-received, and per-protocol analyses.

Physician-Related Medication Errors
Two encounters from the medication error analyses that were assigned to the video telemedicine consultation cohort were excluded because medication data were not able to be extracted from the electronic health record owing to a loss of electronic health record data (Figure ).In treatment-received and per-protocol analyses, the frequencies of medication errors among patients and medication orders were similar to the results in the intention-to-treat analyses and were not significantly different between the telephone and video telemedicine cohorts (Table 2).After adjusting for age, severity of illness, and hospital study period, the adjusted odds ratio of experiencing a medication error among those assigned to video telemedicine was 0.86 (95% CI, 0.49-1.52;P = .61).The results were similar in the treatment-received and per-protocol analyses (Table 3).

Medication error characteristics
No. 159 535 Patients with medication orders 130 (

JAMA Network Open | Emergency Medicine
Telemedicine vs Telephone Consultations and Medication Prescribing Errors

Discussion
In this pragmatic, cluster randomized, unbalanced crossover trial, we did not find a statistically significant difference in the odds of medication errors among critically ill children receiving care in community and rural EDs who received a pediatric critical care consultation by video telemedicine vs telephone.Medication error rates were similar between comparison groups in the unadjusted and adjusted models, including the intention-to-treat, treatment received, and per-protocol analyses.
Although other research has suggested that video telemedicine consultations are helpful among this population with respect to practitioner and caregiver experience, triage decisions, and appropriateness of transfer, 11 these consultations did not impact physician-related medication errors.
In this study, we found that the overall rate of medication errors was similar to those in other studies of EDs that have CPOE 5 and lower than those demonstrated in older studies and those in EDs without CPOE. 8,9We found that the most common type of physician-related medication error was the wrong dose, which has been a well-documented error in children. 4,7,16Despite the assistance from CPOEs, dosing errors were common in this study.This may be due to the inaccuracies, rounding errors by the CPOE, or even the discrepancies in acceptable dose by different pharmacologic references. 7,16though research suggests that telemedicine consultations can improve communication and quality of care and reduce unnecessary transfers, 11,17 the evidence supporting telemedicine to reduce medication errors is limited.One previous study 9 by our group found that using telemedicine consultations in the ED was associated with decreased physician-related medication errors.
However, that study took place in EDs during a time without readily available CPOE.It is possible that errors with telemedicine are now being reduced by electronic health record safety features and CPOE.In addition, the overall incidence of medication errors per patient was substantially higher in our previous study at 1.22 errors per patient, 11 in comparison to the frequency of 0.125 errors per patient in the current study, further highlighting reduction of errors associated during time periods with CPOE in place.For this study, the lack of a difference in medication errors was affected by the overall low incidence of medication errors, leading to a decrease in study power.

Limitations
Our study has some limitations.First, the agreement between pharmacist reviewers was only moderate.We determined medication errors from medical records, which meant that we could only identify errors that were documented.Consultations in this study also occurred approximately 2.5 hours after the patient presented to the ED, during which time most medications were already ordered, limiting the opportunity for the telehealth consultation to have a positive impact.
Nevertheless, this is the pragmatic nature in which telemedicine consultations are obtained.We also did not collect information on the recommendations that were given by the physician and, therefore, could not identify which consultations explicitly resulted in medication suggestions from the consulting pediatric critical care practitioner.9][20] The ability of the emergency medicine and pediatric intensive care unit physicians to deviate from the assigned consultation mode indicates there may be confounding by indication.In this study, there was a larger proportion of deviations in the telemedicine group than in the telephone group, perhaps reflecting workflow difficulties in incorporating telemedicine or resistance in adoption of telemedicine. 21,22Although unmeasured confounders and protocol deviations could have contributed to biased results, the previously published baseline characteristics comparing patients who received consultations that adhered to the study group and those whose consultations deviated from the study group intervention were similar. 11We also did not identify any significant differences by intention-to-treat, per-protocol, or treatment-received analyses.In addition, despite use of a previously validated medication error instrument, medication errors and their outcomes are somewhat subjective in nature, and the interrater reliability between pharmacists was only moderate, potentially impacting the validity of the findings.It is unlikely that the raters were biased toward 1 consultation or the other because they were blinded.However, the imprecision of this measure may have biased the overall estimate toward the null.In addition, because the use of telemedicine to provide consultations to pediatric patients in remote EDs varies, further research is needed to better understand the impact of these consultations on measures of quality and patient safety.

Conclusions
In this cluster randomized crossover trial, among a cohort of critically ill children presenting to community and rural EDs, the use of video telemedicine to deliver pediatric critical care consultations was not associated with fewer physician-related medication errors compared with the use of telephones to deliver these consultations.The role of video telemedicine in providing remote acute care continues to emerge.Medication errors may or may not be amenable to improvement through this modality as electronic prescribing and safety protocols are established at more hospitals.Further research is needed to determine what aspects of pediatric acute care consultations are most likely to be improved with video telemedicine.

QuestionFindings
Does the use of video telemedicine for pediatric consultations to referring hospital emergency departments (EDs) result in less frequent medication errors than the current standard of care, telephone consultations?In this cluster randomized crossover trial that included 696 acutely ill children presenting to 15 community and rural EDs, there were no statistically significant differences in physicianrelated medication errors between children assigned to receive telephone consultations vs video telemedicine consultations.Meaning These findings suggest that the use of video telemedicine to conduct consultations for acutely ill children in rural and community EDs does not result in less frequent medication errors than consultations done by telephone.
2016 and end dates between September 2016 and March 2018.Deidentified electronic medical records were obtained from the originating EDs with any direct and indirect references to the type of consultation (telephone or video telemedicine) redacted.Demographic data and encounter characteristics, including use of emergency medical services for transportation to the ED, were

Table 1 .
Baseline Characteristics by Study Allocation (Intention-to-Treat)

Table 2 .
Medication Errors and Types at the Medication Level a Includes intravenous, per os, and nasogastric.bRefers to allergy, drug interaction, and kidney or liver disease.