eTable 1. Potential Alerts That Could Be Generated by a Bar Code Medication Scan in the Electronic Health Record
eTable 2. Examples of Reported Medication Administration Errors From the Hospital Reporting System
eTable 3. Sensitivity Analysis Results Varying the Interruption Exposure Window Duration Prior to Medication Administration Attempt From 5 to 30 Minutes and Using Conditional Logistic Regression to Analyze the Isolated Within-Nurse or Within-Patient Effect Only
eFigure 1. Data Flow Diagram for Nurses Included in the Study
eFigure 2. Data Flow Diagram for Patients Included in the Study
eFigure 3. Data Flow Diagram for Medication Administrations Included in the Study
eFigure 4. Data Flow Diagram for Incoming Telephone Calls Included in the Study
eFigure 5. Data Flow Diagram for Incoming Text Messages Included in the Study
eFigure 6. Data Flow Diagram for Bar Code Medication Alerts Included in the Study
eFigure 7. Data Flow Diagram for Reported Medication Administration Errors Included in the Study
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Bonafide CP, Miller JM, Localio AR, et al. Association Between Mobile Telephone Interruptions and Medication Administration Errors in a Pediatric Intensive Care Unit. JAMA Pediatr. 2020;174(2):162–169. doi:10.1001/jamapediatrics.2019.5001
Are mobile telephone interruptions temporally associated with pediatric intensive care unit nurses’ errors during medication administration?
In this cohort study of 257 nurses and 3308 patients in a pediatric intensive care unit, incoming calls on nurses’ institutional mobile telephones occurring in the 10 minutes before medication administration were significantly associated with increased risk of error. The risk was higher during night shifts and among nurses with fewer than 6 months’ experience, and it also varied by nurse to patient ratio and level of patient care required.
This study’s findings suggest that, although communication-related interruptions cannot be eliminated, interventions to reduce the frequency and adverse consequences of interruptions should include consideration of time of day, nurse experience, nurse to patient ratio, and level of patient care required.
Incoming text messages and calls on nurses’ mobile telephones may interrupt medication administration, but whether such interruptions are associated with errors has not been established.
To assess whether a temporal association exists between mobile telephone interruptions and subsequent errors by pediatric intensive care unit (PICU) nurses during medication administration.
Design, Setting, and Participants
A retrospective cohort study was performed using telecommunications and electronic health record data from a PICU in a children’s hospital. Data were collected from August 1, 2016, through September 30, 2017. Participants included 257 nurses and the 3308 patients to whom they administered medications.
Primary exposures were incoming telephone calls and text messages received on the institutional mobile telephone assigned to the nurse in the 10 minutes leading up to a medication administration attempt. Secondary exposures were the nurse’s PICU experience, work shift (day vs night), nurse to patient ratio, and level of patient care required.
Main Outcomes and Measures
Primary outcome, errors during medication administration, was a composite of reported medication administration errors and bar code medication administration error alerts generated when nurses attempted to give medications without active orders for the patient whose bar code they scanned.
Participants included 257 nurses, of whom 168 (65.4%) had 6 months or more of PICU experience; and 3308 patients, of whom 1839 (55.6%) were male, 1539 (46.5%) were white, and 2880 (87.1%) were non-Hispanic. The overall rate of errors during 238 540 medication administration attempts was 3.1% (95% CI, 3.0%-3.3%) when nurses were uninterrupted by incoming telephone calls and 3.7% (95% CI, 3.4%-4.0%) when they were interrupted by such calls. During day shift, the odds ratios (ORs) for error when interrupted by calls (compared with uninterrupted) were 1.02 (95% CI, 0.92-1.13; P = .73) among nurses with 6 months or more of PICU experience and 1.22 (95% CI, 1.00-1.47; P = .046) among nurses with less than 6 months of experience. During night shift, the ORs for error when interrupted by calls were 1.35 (95% CI, 1.16-1.57; P < .001) among nurses with 6 months or more of PICU experience and 1.53 (95% CI, 1.16-2.03; P = .003) among nurses with less than 6 months of experience. Nurses administering medications to 1 or more patients receiving mechanical ventilation and arterial catheterization while caring for at least 1 other patient had an increased risk of error (OR, 1.21; 95% CI, 1.03-1.42; P = .02). Incoming text messages were not associated with error (OR, 0.97; 95% CI, 0.92-1.02; P = .22).
Conclusions and Relevance
This study’s findings suggest that incoming telephone call interruptions may be temporally associated with medication administration errors among PICU nurses. Risk of error varied by shift, experience, nurse to patient ratio, and level of patient care required.
Interruptions during clinical work occur frequently and can impair the performance of nurses and physicians.1-4 Interruptions to nurses on hospital wards have been shown to be associated with procedural failures and clinical errors during medication administration, measured using direct in-person observation.1 Direct observation of clinical work is highly informative but has limitations; it requires trained clinical observers1 and monitoring of intraobserver and interobserver reliability,5 and it may introduce Hawthorne-like effects.6
Interruptions from electronic devices, most notably incoming calls and text messages on mobile telephones, have been shown to be hazardous when the recipients are operating motor vehicles.7-9 The increasing availability and functionality of mobile telephones in hospitals has facilitated direct communication between nurses and other clinical staff and enabled immediate notification of nurses when alarms occur. Although telephone calls are a leading source of interruptions,4 no association between telephone interruptions and downstream hazards has yet been established, to our knowledge.
In this study, our objective was to examine whether incoming mobile telephone call and text message interruptions are associated with subsequent medication administration errors among pediatric intensive care unit (PICU) nurses. We aimed to accomplish this objective without using direct observational methods, while acknowledging that our approach would not be able to capture all error types that can be identified using direct observation.
We performed this retrospective cohort study in a 55-bed, medical-surgical PICU, using data from August 1, 2016, through September 30, 2017. The Institutional Review Board of Children’s Hospital of Philadelphia approved the study with a waiver of the requirement of informed consent from participants because it determined that the study plan met the criteria in 45 CFR 46.116(d); specifically, that the study presented minimal risk, the waiver would not adversely affect the rights or welfare of the participants, and the research could not practicably be carried out without the waiver or alteration.
The primary participants were nurses based in the PICU who were identified by using a staff register. Secondary participants were patients who were hospitalized in the PICU between August 1, 2016, and September 30, 2017. We extracted demographic data, such as the reported race/ethnicity of the patient and family, from the institutional electronic health record (EHR) system. During the study period, all nurses were using assigned institutional mobile telephones (Ascom d62; Ascom Holding AG) as their primary communication devices.
Use of a bar code medication administration (BCMA) system linked to the EHR was required for the administration of 99% of medications in the PICU. Prior to the start of the study period, PICU nurses had achieved 99% compliance with the guidelines for BCMA use. The procedure for BCMA use included the following steps: scan the bar code on the patient’s ID band, scan the bar codes on the medications to be administered, and then administer the medications. If the system detected an error, the EHR displayed an alert.
We extracted data from the hospital databases that store telephone call and text message data from every incoming call and message sent to nurses’ telephones. The hospital telecommunications system time-stamps all calls and messages to the second, using the institution’s universal system clock. We manually verified the consistency of time stamps across platforms prior to data extraction. To obtain the telephone call data, we directly queried the back-end database (Innovaphone; Innovaphone AG) that stores call detail records by user. The call detail records include indicators for call direction (eg, incoming, outgoing), alert signal (indicating that the call made the telephone ring), and connection status (connected, transferred, forwarded, released, or busy). In the analytic data set used in this study, 91% of calls had a status of “connected,” indicating that most incoming calls were answered by the nurse. Considering that, prior to each medication administration, there is a variable time window during which a medication is obtained, checked, prepared, and brought to the bedside, in collaboration with nurse experts we made the a priori decision that telephone call and text message interruptions occurring in the 10 minutes leading up to a medication administration attempt would be included as exposures.
We used a composite outcome of errors during medication administration as the primary outcome of analysis, comprising both (1) BCMA system–generated alerts and (2) reported medication administration errors, because both end points reflect risk of medication-related harm to the patient. We refer to the composite outcome simply as “errors.”
We extracted BCMA alerts from the hospital’s data warehouse. Among 16 potential alerts that could be generated by a bar code medication scan in the EHR during the study period, we focused on the types that our pharmacy was using to track events that were likely to be “near misses,” or attempts to give medications without orders present. These included alerts for “order is not active” and “no order for patient” (eg, attempt to give morphine to an infant who did not have an active order for it), “admin on discontinued med” and “admin on completed med” (eg, attempt to give midazolam to a child whose order for the drug was recently discontinued), and “order for wrong patient” (eg, attempt to give another patient’s vancomycin to the patient whose bar code was scanned). The 16 types of alerts and their relative frequencies, categories, and descriptions are in eTable 1 in the Supplement.
We extracted reported medication administration errors occurring during the study period that were labeled in the hospital’s online reporting system as related to the medication administration process. Because those labels might not always be accurate, a physician (C.P.B.) and nurse (S.S.) independently reviewed the descriptions of 50 randomly selected reports and categorized them as errors if they (1) occurred during nurse medication administration, and (2) were traceable to a specific minute. In independent review, the physician and nurse agreed on the classification of 49 of 50 events; the nurse performed single review for the remaining reports. We also extracted the severity of each reported error, which was categorized by using a framework adapted from the National Coordinating Council for Medication Error Reporting and Prevention Index.10 Severity was included in the online reporting system, categorized independently by a nurse safety and quality specialist from the unit where the event occurred.
We analyzed the data at the level of the successful medication administration or failed administration attempt. Successful administrations were entered in the EHR with 1 of the following actions: “given,” “new bag,” or “new syringe.” Eligible failed administration attempts were BCMA alerts and reported administration errors that were intercepted without associated completed administrations.
We defined administration time by the EHR time stamp for that action. We excluded administrations for which the timing was untrustworthy. We considered the timing of administrations trustworthy if the nurse’s manual time entry was within 10 minutes of the corresponding EHR time stamp. We excluded medication administrations during EHR downtimes. We merged multiple attempts by the same nurse to administer medication to the same patient during the same minute because administration time was available with only 1-minute granularity and we assumed that receiving a BCMA alert would lead to increased vigilance in subsequent administration attempts. Because we did not know the administration sequence within individual minutes, we made the decision to consider each administration attempt a success or failure at the 1-minute level. We excluded data from a 30-minute (washout) period following each error.
We categorized nurses’ clinical experience as less than 6 months or as 6 months or more to characterize the least experienced nurses, among whom we hypothesized we would see the strongest interruption effects. We classified shifts as day (7 am to 6:59 pm) or night (7 pm to 6:59 am). We categorized the nurse to patient ratio at the time of administration either as 1:1 or as 1:2 and higher. Intervention data (as an indicator of the level of patient care required) were extracted at the day level from our hospital’s instance of the Virtual Pediatric Systems PICU database.11,12 To characterize the level of patient care required, we included a dichotomous variable indicating patients requiring a high level of advanced critical care, as evidenced by receipt of mechanical ventilation and arterial catheterization, vs patients not requiring that level of care.
We used a multivariable logistic regression model with the interactions we hypothesized might be important as our primary analytic model. We included a 2-way interaction between nurse to patient ratio and patient-specific level of care and a 3-way interaction between work shift, nurse experience level, and presence of telephone interruption. We chose these interactions based on existing literature13 and our clinical understanding of the complex relationships between these variables. We standardized probabilities of error given certain sets of exposures using predictive margins.14 Regression models accounted for clustering of observations at the level of the nurse. We examined the design effects of this clustering to assess nurse-to-nurse variability in rates of error.15 We did not separately account for patient clustering nested within nurse clusters because accounting for the higher, nurse-level clustering also accounts for any lower level of correlation that exists within each cluster.16
We used R software, version 3.5.3 (R Foundation for Statistical Computing), for data management and Stata, version 15.1 (StataCorp LLC), for analyses. Statistical significance was indicated by P < .05.
We performed 3 sensitivity analyses. First, we varied the interruption exposure window in the primary analysis between 5 and 30 minutes and compared the results with results obtained with the 10-minute window selected a priori. Second, we explored the potential problem of confounding by nurse, using conditional logistic regression to examine the association of interruption and error within the data for each individual nurse. Third, we explored the potential problem of confounding by patient, using conditional logistic regression within the data for each individual patient.
The final data set included 257 nurses, of whom 168 (65.4%) had 6 months or more of PICU experience; and 3308 patients, of whom 1839 (55.6%) were male, 1539 (46.5%) were white, and 2880 (87.1%) were non-Hispanic (Table 1). We included 238 540 medication administration attempts, 19 891 (8.3%) of which were interrupted by 1 or more of the 173 225 incoming telephone calls, and 106 660 (44.7%) of which were interrupted by 1 or more of the 3 272 427 incoming text messages in the 10 minutes leading up to the administration attempt. We also included 7513 BCMA system–generated alerts and 42 reported medication administration errors. Examples of reported errors that were included in the analysis are listed in eTable 2 in the Supplement. Physiologic monitor alarms generated 86.8% of the text messages. Characteristics of medication administration attempts, text messages, and alerts are described in Table 2. The Supplement includes CONSORT-style17 data flow diagrams for nurses (eFigure 1), patients (eFigure 2), medication administrations (eFigure 3), telephone calls (eFigure 4), text messages (eFigure 5), BCMA system alerts (eFigure 6), and reported medication administration errors (eFigure 7).
The overall rate of errors within 238 540 medication administration attempts was 3.2% (95% CI [after adjustment for data clustering at the level of the nurse], 3.0%-3.3%). The unadjusted rate of errors when nurses were not interrupted by incoming telephone calls was 3.1% (95% CI, 3.0%-3.3%) compared with 3.7% (95% CI, 3.4%-4.0%) when they were interrupted by such calls. The unadjusted rate of errors when nurses were not interrupted by incoming text messages was 3.2% (95% CI, 3.0%-3.3%) compared with 3.1% (95% CI, 3.0%-3.3%) when they were interrupted by incoming text messages. Individual nurses’ personal error rates ranged from 0.5% to 11.6% (median, 3.2%; IQR, 2.3%-3.9%).
In unadjusted analyses, we found preliminary evidence that the presence of telephone call interruptions and 1 or more prior errors committed by the same nurse during the same shift both increased the odds of committing an error, and that working during a night shift appeared to decrease the odds of error (Table 3).
In multivariable analysis, the estimated error rate when nurses were uninterrupted by telephone calls significantly differed by time of day, with a rate of 3.4% (95% CI, 3.2%-3.6%) during the day shift and 2.8% (95% CI, 2.6%-2.9%) during the night shift (P < .001 for the contrast). However, odds ratios (ORs) for error when telephone interruptions occurred were greater during the night shift than during the day shift and greater with less experienced nurses than with more experienced ones (Table 4). During the day shift, the OR for error when nurses were interrupted by calls (compared with uninterrupted) was 1.02 (95% CI, 0.92-1.13; P = .73) among nurses with 6 months or more of experience and 1.22 (95% CI, 1.00-1.47; P = .046) among nurses with less than 6 months of experience. During the night shift, the OR for error when nurses were interrupted by calls was 1.35 (95% CI, 1.16-1.57; P < .001) among nurses with 6 months or more of experience and 1.53 (95% CI, 1.16-2.03; P = .003) among nurses with less than 6 months of experience. The differential effects of these exposure conditions on the association between telephone interruptions and errors resulted in similar estimated error rates of 3.7% (95% CI, 3.3%-4.0%) during the day shift and 3.8% (95% CI, 3.3%-4.3%) during the night shift (P = .57 for the contrast). Nurses administering medications to 1 or more patients receiving mechanical ventilation and arterial catheterization while caring for at least 1 other patient had an increased risk of error (OR, 1.21; 95% CI, 1.03-1.42; P = .02). Incoming text messages were not associated with error (OR, 0.97; 95% CI, 0.92-1.02; P = .22).
Predicted probabilities of committing medication administration errors in different subgroups of nurses are shown in Table 5, with the main contrast being the presence or absence of a telephone interruption.
Design effects ranged from 1.05 to 2.09 for all the variables in the logistic model, indicating that individual nurses’ personal error rates sometimes varied more than one would expect at random and that data clustering at the level of the nurse was appropriate. In sensitivity analyses, we observed stability of the odds ratio point estimate direction and approximate magnitude (eTable 3 in the Supplement).
The results of our study suggest that interruptions due to incoming telephone calls were temporally associated with nurses’ errors during the medication administration process. Given the very low frequency of reported medication administration errors, this finding was attributable almost entirely to BCMA system alerts. Our main findings included the following: (1) telephone call interruptions were associated with significantly increased odds of committing errors; (2) although the overall baseline error rate without interruptions was lower during the night shift than during the day shift, the increase associated with telephone interruptions was greater at night, especially among less experienced nurses; (3) the nurse to patient ratio was important in the context of the level of patient care, with the highest risk among nurses caring for 2 or more patients and administering medications to patients with mechanical ventilation and an arterial catheter; and (4) text message interruptions were not associated with errors, possibly because text messages do not require immediate response or because nurses have adapted to their frequent occurrence (they preceded nearly half of all medication administration attempts).
Interruptions previously were shown to be associated with medication errors. In a large, direct observation study of 98 nurses on Australian hospital wards administering more than 4000 medications, Westbrook and colleagues1 found associations between interruptions and observed procedural failures and clinical errors. In their study, the observed baseline procedural failure rate was high at 72.3%, increasing to 82.1% with 1 interruption. Their baseline clinical error rate was also high at 36.1%, increasing to 43.5% with 1 interruption. Major errors similar to the primary outcome in our study, such as attempting to administer an unordered drug, occurred in 2.1% of administrations, increasing to 2.8% when 1 interruption occurred. Our results, obtained by using a large data set instead of direct observation, appear similar to the major error results in their study.
We did not aim to validate our method in comparison with direct observation for several reasons. First, direct observation is not a criterion standard for assessing medication administration errors. A composite measure combining nurses’ reports, patients’ reports, reports from third-party in-person observers (or discreet video recordings reviewed later by expert observers), and EHR data would likely be required to achieve a definitive assessment. Second, if we are to solve the problems that many observers suspect are caused by interruptions, we need more efficient, less costly, and more easily reproducible methods for measuring interruptions and outcomes over time. Electronic health record–based approaches to studying the association between interruptions and errors have the potential to offset the need for trained observers. These approaches also eliminate any potential Hawthorne-like effects, although the existence of these effects in direct clinical observation has been questioned.18 Later, when interventions are put into place to reduce interruptions and the results of the interventions must be measured, EHR-based approaches offer relative efficiency for measuring the same outcomes in the same way, over and over again.
Despite the findings of our study and the work of others, interruptions are not universally bad. They are a necessary aspect of clinical care, as they often stem from the need to communicate important information between clinical staff. Unfortunately, the highest rates of interruptions occur while nurses are performing medication-related tasks,19 activities that require concentration, during which interruptions can be particularly harmful. Yet, there may be opportunities to reduce the frequency of interruptions or to alter the context in which they occur. In an analysis of more than 5000 interruptions measured using direct observation, only 11% were found to result in a positive outcome, suggesting that many interruptions could be eliminated.20
Our study had several limitations. First, we focused on a small subset of errors and procedural failures. As a result, error rates were low and the absolute increase in risk associated with telephone interruptions was small. However, like the low rates of serious errors found in the study by Westbrook et al,1 our findings likely represent the tip of an iceberg of errors associated with interruptions. Therefore, we believe that they warrant action to reduce interruptions. Second, we considered only interruptions from nurses’ institutional mobile telephones and did not examine events that could only be measured using direct observation, such as personal mobile telephone calls and in-person interruptions. Third, we do not know the frequency of BCMA alert overrides, or in how many erroneous medication administration attempts that triggered BCMA alerts the medication ultimately was administered to the patient, or the clinical severity of the error when it was. The emphasis in the present study was on interruptions associated with initial errors in administration of medications; we were not primarily focused on what happened after the initial error. Fourth, we excluded administration attempts for which the nurse’s manual time entry was separated by 10 minutes or more from the corresponding EHR time stamp. It is possible that this action resulted in the exclusion of administration attempts performed hastily during times when nurses were so busy that their documentation was delayed, and that these administration attempts might have been more prone to errors. If such attempts were indeed excluded, we would expect our findings to be biased toward the null hypothesis.
In this analysis of a large EHR and telecommunications data set, we found an association between incoming mobile telephone calls and subsequent medication administration errors by PICU nurses. This finding suggests a potential role for telephone call diversion strategies during peak medication administration times.21,22 The rates of errors associated with such interruptions varied by shift (night vs day), nurses’ experience level, and the nurse to patient ratio in combination with the level of patient care required. The contribution of each of these factors should be considered when designing and prioritizing interventions to reduce the frequency of interruptions and potentially adverse consequences for patient safety.
Accepted for Publication: August 14, 2019.
Corresponding Author: Christopher P. Bonafide, MD, MSCE, Section of Pediatric Hospital Medicine, Children’s Hospital of Philadelphia, Buerger Center for Advanced Pediatric Care, 3500 Civic Center Blvd, Philadelphia, PA 19104 (email@example.com).
Published Online: December 20, 2019. doi:10.1001/jamapediatrics.2019.5001
Author Contributions: Dr Bonafide and Mr Miller had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Bonafide, Miller, Localio, Holmes, Nadkarni, Keren.
Acquisition, analysis, or interpretation of data: Bonafide, Miller, Localio, Khan, Dziorny, Mai, Stemler, Chen, Holmes.
Drafting of the manuscript: Bonafide, Miller.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Bonafide, Miller, Localio, Dziorny, Chen.
Obtained funding: Bonafide.
Administrative, technical, or material support: Khan, Stemler, Chen.
Supervision: Nadkarni, Keren.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was funded by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute (5-K23HL116427) (Dr Bonafide).
Role of the Funder/Sponsor: The National Heart, Lung, and Blood Institute had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: Dr Keren is Deputy Editor of JAMA Pediatrics but was not involved in any of the decisions regarding review of the manuscript or its acceptance. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Create a personal account or sign in to: