To implement and evaluate the impact of computerized provider order entry (CPOE) on reducing ordering errors in pediatric chemotherapy.
Before-and-after study from 2001 to 2004.
Pediatric Oncology in an academic medical center.
Implementation of a CPOE system guided by multidisciplinary failure modes and effects analysis into pediatric chemotherapy.
Main Outcome Measures
Completion data on chemotherapy steps of high morbidity/mortality potential if missed (as determined by attending oncologists) from 1259 pre-CPOE paper and 1116 post-CPOE pediatric chemotherapy orders.
After CPOE deployment, daily chemotherapy orders were less likely to have improper dosing (relative risk [RR], 0.26; 95% confidence interval [CI], 0.11-0.61), incorrect dosing calculations (RR, 0.09; 95% CI, 0.03-0.34), missing cumulative dose calculations (RR, 0.32; 95% CI, 0.14-0.77), and incomplete nursing checklists (RR, 0.51; 95% CI, 0.33-0.80). There was no difference in the likelihood of improper dosing on treatment plans and a higher likelihood of not matching medication orders to treatment plans (RR, 5.4; 95% CI, 3.1-9.5).
Failure modes and effects analysis–guided CPOE reduced ordering errors in pediatric chemotherapy and provided data for further improvements.
The complexity of pediatric chemotherapy makes it vulnerable to errors.1 As a specialized medication process, it combines the risks for errors in pediatric drug dosage calculation (which is dependent on age, weight, and body surface area) and the risks associated with the narrow therapeutic profile and high potential for acute and cumulative toxicities of antineoplastic agents.2 Those risks, combined with potential interruptions in the execution of detailed protocols (due to the labile physiological conditions of patients), create a higher probability and impact of errors and worse outcomes3 in an already vulnerable population.
Because of these inherent vulnerabilities, error reduction and prevention in pediatric chemotherapy is also complex and requires a systematic approach, strong leadership, and multidisciplinary cooperation for success.4,5 Error management in this domain requires global and specific consideration of the medication process and the information that controls it and coordination and communication among physicians, nurses, pharmacists, and allied health professionals. In addition, there must be ongoing surveillance and data collection and analysis to evaluate the effect of interventions6 on the medication process7 and to identify the introduction of new types of errors.8
One approach to reduce and prevent errors in medicating patients is to model the process as a series of steps performed by individuals (physicians, nurses, clerks, and pharmacists) who pass information from one to another.9 With such a model, domain experts may be able to assess failure modes (mechanisms, likelihood, and impacts of the failure of individual steps)10 for the process. With this assessment, interventions may be planned to reduce variation in the human performance (of repetitive or complex tasks) that leads to errors.11 For medication processes, one highly promoted information technology (IT) application advocated by health care industry leaders to reduce errors is computerized provider order entry (CPOE)12 because of its ability to reduce variability through automation and clinical decision support.
We herein describe the incorporation of CPOE into the pediatric chemotherapy process and its impact on process errors at an academic medical center.
This study was conducted in Pediatric Oncology of the Johns Hopkins Children's Center/Sidney Kimmel Comprehensive Cancer Center, Baltimore, Md, which provides chemotherapy to more than 200 children with a variety of neoplastic disorders. In response to a sentinel event, and as part of a safety and quality improvement program (the Quality Improvement Initiative) in pediatric oncology, paper-based order entry of pediatric chemotherapy was converted to CPOE.
As part of a hospital leadership–driven initiative to reduce errors in chemotherapy, a multidisciplinary team consisting of staff from pediatric oncology (physicians, nurses, and physician assistants), pharmacy, and hospital information systems conducted a failure modes and effects analysis (FMEA) of the pediatric chemotherapy process and made recommendations to guide the implementation of functions for CPOE.
Recommendations of the multidisciplinary team, based on the FMEA findings and a pre-CPOE audit (described in “Data Collection”), were used to guide the modification of an available pharmacy system (RxTFC Pharmacy Information System; GE Medical Systems Information Technologies, BDM Information Systems Ltd, Saskatoon, Saskatchewan) to meet the needs of pediatric chemotherapy and to address the identified failure modes. A sample of the interface is shown in the Figure.
Sample screen shot of the pediatric chemotherapy computerized provider order entry interface.
To evaluate the impact of the introduction of CPOE into the pediatric chemotherapy process and to create a benchmark for further improvements, a 2-phase daily audit of pediatric chemotherapy treatments was administered before and after CPOE deployment. Before CPOE deployment, audit data were collected from July 31 to August 1, 2001, and August 14, 2001, to August 22, 2002 (241 days) using a paper-based survey tool from 1259 sequential chemotherapy orders (representing 176 patients). After CPOE deployment, audit data were collected from February 3, 2003, to February 12, 2004 (296 days) using a Web-based direct-entry tool (part of the CPOE application) from 1116 orders (representing 167 patients). Use of these data was approved by the Quality Assurance committee and the Division Chair at The Johns Hopkins University School of Medicine.
Using all chemotherapy orders for an individual patient in 1 day as the unit of analysis, we captured successful correct completion rates of specific steps of “high importance” (as determined by attending oncologists according to their potential for morbidity and mortality if not correctly completed) from the pre- and post-CPOE periods. The specific steps measured and compared were correct order format on the treatment plan (proper medication dose on the associated chemotherapy protocol/treatment plan), correct order format on the order (proper medication dose on an individual order), order and treatment plan match (proper match of a medication dose to its associated protocol), cumulative dose on the treatment plan (verification of the cumulative dose of a medication on a protocol), correct calculation (verification of the medication dose calculation), and nursing checklist (documentation of the nurse review).
Medication process steps, their failure modes (the potential severity and probability of occurrence), and the consensus recommendation to reduce errors are summarized in Table 1.
Of the different medication steps (prescription, ordering, transcription, dispensing, and administration), failure modes of high severity were identified in the ordering and administration steps. Of those, error types of medium to medium-high probability involved the misidentification of patients, patient variables (height and weight), and medications. Consensus recommendations to reduce these error types were to (1) limit choices (by predefined menus instead of free-text entry), (2) enforce entry of required data (height and weight), and (3) provide alerts for abnormal values. Failure modes of low to medium-high probability involved dosing errors due to mismatch of drug orders to chemotherapy protocols or miscalculation. Consensus recommendations to reduce error probability were to (1) enforce protocol specification and (2) automate calculations.
Completion rates of oncologist-identified steps of high importance from the pre- and post-CPOE periods are summarized in Table 2.
After CPOE deployment, daily chemotherapy orders were less likely to have improper dosing on orders (relative risk [RR], 0.26; 95% confidence interval [CI], 0.11-0.61), incorrect dosing calculations (RR, 0.09; 95% CI, 0.03-0.34), missing cumulative dose calculations (RR, 0.32; 95% CI, 0.14-0.77), and incomplete nursing checklists (RR, 0.51; 95% CI, 0.33-0.80). There was no difference in the likelihood of improper dosing on treatment plans and a higher likelihood of not matching medication orders to treatment plans (RR, 5.4; 95% CI, 3.1-9.5).
The risks to patients posed by potential errors in the pediatric chemotherapy process are multidimensional, owing to inherent toxicities of the sometimes experimental drugs used, complexities of the protocols followed, and clinical lability of the patients' disease processes. In addition, children, possibly owing to the additional need for individualized dosing based on weight and body surface area, are at higher risk for medication errors (approximately 3 times that of adults13) in general.7
For these reasons, the maxim primum non nocere14 mandates systematic error management in the pediatric chemotherapy process as an essential part of improving pediatric oncology outcomes.15 Such management includes an informed and iterative approach to (1) model the clinical process in which errors occur; (2) identify the errors, their mechanisms, probabilities of occurrence, and impacts; (3) modify the process to eliminate or reduce the errors; (4) evaluate the effect of modifications on the process (and outcomes); and (5) update clinical process models on the basis of evaluation data.16 Systematic error management is vital to good clinical care in complex processes (such as patient medication) and complex settings (such as hospitals).
Other proactive and reactive approaches to error management in pediatric chemotherapy include (manual) multidisciplinary check of order sets,17 integrated incident and error reporting,18 knowledge acquisition and computer-aided planning of therapy with decision support for individualized therapy,19 standardization of the terminology used,20 and education of caregivers,21 in addition to standard approaches such as root cause analysis22 (mandated by the Joint Commission on Accreditation of Healthcare Organizations for the investigation of sentinel events in accredited hospitals) of errors when they occur.
Failure modes effects analysis is a methodology adapted from the military23 and recommended by the Joint Commission on Accreditation of Healthcare Organizations for redesigning health care processes24 as a proactive, systematic approach for identifying “the ways that a process or design can fail, why it might fail, and how it can be made safer.”25 Failure modes effects analysis is directly applicable to the incorporation of IT such as CPOE into a complex process such as pediatric chemotherapy. Using domain and IT expertise, the assessment identifies and prioritizes specific process errors (failure modes) and suggests ways that IT (in this case CPOE) can help to mitigate them. The collection of data (also determined by domain expertise, in this case attending oncologists) provides evidence and benchmarks for further improvement efforts.
According to literature and to the experience of the team, most medication process errors (in general) and pediatric chemotherapy errors (in particular) occur at the prescribing/ordering step,26 to which the team recommendations were directed to (1) automate calculations (via embedded calculators), (2) reduce free-text and handwritten entry (via drop-down medication lists), (3) reduce prescriber memory load (via autocompletion of specific data fields), and (4) enforce prescriber completion of essential data fields (via pop-up alerts). These functions were incorporated into the design and implementation of the application and its interface.
Incorporation of CPOE has been advocated as an effective way to reduce medication errors,27,28 but its implementation and deployment must be tailored to the needs of specific clinical processes. In pediatrics and neonatology, CPOE has been demonstrated to mitigate errors in specialized medication processes (such as total parenteral nutrition29 and continuous intravenous infusions30), and these deployments have succeeded through guidance by domain and IT experts who are familiar with the processes.
Error management must consist of continued surveillance for persistence of old errors and anticipation of the introduction of new ones. In the first iteration of CPOE error, there was an unanticipated but statistically significant decrease in the matching of orders (chemotherapy drug doses) to specific protocols. A review of the data suggested several reasons for this:
No automated drug-protocol linkage. At the time of this study, this feature was not included.
User interface problem. New or experimental drugs did not appear on the predefined menu after CPOE deployment, requiring users to enter them manually, resulting in drug-protocol nonmatch. Before CPOE deployment, all orders were handwritten, which may have also resulted in underreporting of drug-protocol nonmatches.
Human transcription failure. After CPOE deployment, protocols were still paper based, forcing prescribers to switch back and forth between electronic and print interfaces.
These considerations provide a basis for the next iteration of the CPOE application, which will include automated linkage of chemotherapy order sets to specific protocols at the Johns Hopkins Children's Center (planned for 2006).
A strength of this study was the inclusion of audit data from all patient encounters in the pre- and post-CPOE time frames. A limitation was the focus on process and not outcomes. Inclusion of outcome measures is projected for a future iteration.
In conclusion, pediatric chemotherapy is a complex medication process that is vulnerable to human error. Computerized provider order entry is a recognized IT intervention that can reduce errors, but its incorporation into the process must be guided by domain and IT expertise. We have described an FMEA-guided implementation of CPOE in pediatric chemotherapy at an academic medical center with data that demonstrate its impact on reducing process errors.
Correspondence: George R. Kim, MD, The Johns Hopkins School of Medicine, 600 N Wolfe St, Nelson 2-133, Baltimore, MD 21287-3200 (firstname.lastname@example.org).
Accepted for Publication: November 22, 2005.
Author Contributions: All of the authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding/Support: This study was supported by the Center for Innovation and Quality Patient Care at The Johns Hopkins Hospital.
Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, Lehmann CU. Error Reduction in Pediatric ChemotherapyComputerized Order Entry and Failure Modes and Effects Analysis. Arch Pediatr Adolesc Med. 2006;160(5):495-498. doi:10.1001/archpedi.160.5.495