[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.163.129.96. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Article
May 2006

Error Reduction in Pediatric ChemotherapyComputerized Order Entry and Failure Modes and Effects Analysis

Author Affiliations

Author Affiliations: Divisions of Health Sciences Informatics (Drs Kim and Lehmann) and Pediatric Oncology (Drs Chen and Arceci and Mss Kokoszka and Daniel), The Johns Hopkins School of Medicine, and Department of Pharmacy, The Johns Hopkins Hospital (Ms Mitchell), Baltimore, Md.

Arch Pediatr Adolesc Med. 2006;160(5):495-498. doi:10.1001/archpedi.160.5.495
Abstract

Objective  To implement and evaluate the impact of computerized provider order entry (CPOE) on reducing ordering errors in pediatric chemotherapy.

Design  Before-and-after study from 2001 to 2004.

Setting  Pediatric Oncology in an academic medical center.

Intervention  Implementation of a CPOE system guided by multidisciplinary failure modes and effects analysis into pediatric chemotherapy.

Main Outcome Measures  Completion data on chemotherapy steps of high morbidity/mortality potential if missed (as determined by attending oncologists) from 1259 pre-CPOE paper and 1116 post-CPOE pediatric chemotherapy orders.

Results  After CPOE deployment, daily chemotherapy orders were less likely to have improper dosing (relative risk [RR], 0.26; 95% confidence interval [CI], 0.11-0.61), incorrect dosing calculations (RR, 0.09; 95% CI, 0.03-0.34), missing cumulative dose calculations (RR, 0.32; 95% CI, 0.14-0.77), and incomplete nursing checklists (RR, 0.51; 95% CI, 0.33-0.80). There was no difference in the likelihood of improper dosing on treatment plans and a higher likelihood of not matching medication orders to treatment plans (RR, 5.4; 95% CI, 3.1-9.5).

Conclusion  Failure modes and effects analysis–guided CPOE reduced ordering errors in pediatric chemotherapy and provided data for further improvements.

The complexity of pediatric chemotherapy makes it vulnerable to errors.1 As a specialized medication process, it combines the risks for errors in pediatric drug dosage calculation (which is dependent on age, weight, and body surface area) and the risks associated with the narrow therapeutic profile and high potential for acute and cumulative toxicities of antineoplastic agents.2 Those risks, combined with potential interruptions in the execution of detailed protocols (due to the labile physiological conditions of patients), create a higher probability and impact of errors and worse outcomes3 in an already vulnerable population.

Because of these inherent vulnerabilities, error reduction and prevention in pediatric chemotherapy is also complex and requires a systematic approach, strong leadership, and multidisciplinary cooperation for success.4,5 Error management in this domain requires global and specific consideration of the medication process and the information that controls it and coordination and communication among physicians, nurses, pharmacists, and allied health professionals. In addition, there must be ongoing surveillance and data collection and analysis to evaluate the effect of interventions6 on the medication process7 and to identify the introduction of new types of errors.8

One approach to reduce and prevent errors in medicating patients is to model the process as a series of steps performed by individuals (physicians, nurses, clerks, and pharmacists) who pass information from one to another.9 With such a model, domain experts may be able to assess failure modes (mechanisms, likelihood, and impacts of the failure of individual steps)10 for the process. With this assessment, interventions may be planned to reduce variation in the human performance (of repetitive or complex tasks) that leads to errors.11 For medication processes, one highly promoted information technology (IT) application advocated by health care industry leaders to reduce errors is computerized provider order entry (CPOE)12 because of its ability to reduce variability through automation and clinical decision support.

We herein describe the incorporation of CPOE into the pediatric chemotherapy process and its impact on process errors at an academic medical center.

METHODS

This study was conducted in Pediatric Oncology of the Johns Hopkins Children's Center/Sidney Kimmel Comprehensive Cancer Center, Baltimore, Md, which provides chemotherapy to more than 200 children with a variety of neoplastic disorders. In response to a sentinel event, and as part of a safety and quality improvement program (the Quality Improvement Initiative) in pediatric oncology, paper-based order entry of pediatric chemotherapy was converted to CPOE.

FAILURE MODES AND EFFECTS ANALYSIS

As part of a hospital leadership–driven initiative to reduce errors in chemotherapy, a multidisciplinary team consisting of staff from pediatric oncology (physicians, nurses, and physician assistants), pharmacy, and hospital information systems conducted a failure modes and effects analysis (FMEA) of the pediatric chemotherapy process and made recommendations to guide the implementation of functions for CPOE.

IMPLEMENTATION OF CPOE

Recommendations of the multidisciplinary team, based on the FMEA findings and a pre-CPOE audit (described in “Data Collection”), were used to guide the modification of an available pharmacy system (RxTFC Pharmacy Information System; GE Medical Systems Information Technologies, BDM Information Systems Ltd, Saskatoon, Saskatchewan) to meet the needs of pediatric chemotherapy and to address the identified failure modes. A sample of the interface is shown in the Figure.

Figure.
Sample screen shot of the pediatric chemotherapy computerized provider order entry interface.

Sample screen shot of the pediatric chemotherapy computerized provider order entry interface.

DATA COLLECTION

To evaluate the impact of the introduction of CPOE into the pediatric chemotherapy process and to create a benchmark for further improvements, a 2-phase daily audit of pediatric chemotherapy treatments was administered before and after CPOE deployment. Before CPOE deployment, audit data were collected from July 31 to August 1, 2001, and August 14, 2001, to August 22, 2002 (241 days) using a paper-based survey tool from 1259 sequential chemotherapy orders (representing 176 patients). After CPOE deployment, audit data were collected from February 3, 2003, to February 12, 2004 (296 days) using a Web-based direct-entry tool (part of the CPOE application) from 1116 orders (representing 167 patients). Use of these data was approved by the Quality Assurance committee and the Division Chair at The Johns Hopkins University School of Medicine.

DATA ANALYSIS

Using all chemotherapy orders for an individual patient in 1 day as the unit of analysis, we captured successful correct completion rates of specific steps of “high importance” (as determined by attending oncologists according to their potential for morbidity and mortality if not correctly completed) from the pre- and post-CPOE periods. The specific steps measured and compared were correct order format on the treatment plan (proper medication dose on the associated chemotherapy protocol/treatment plan), correct order format on the order (proper medication dose on an individual order), order and treatment plan match (proper match of a medication dose to its associated protocol), cumulative dose on the treatment plan (verification of the cumulative dose of a medication on a protocol), correct calculation (verification of the medication dose calculation), and nursing checklist (documentation of the nurse review).

RESULTS
FMEA RECOMMENDATIONS

Medication process steps, their failure modes (the potential severity and probability of occurrence), and the consensus recommendation to reduce errors are summarized in Table 1.

Table 1. 
Abbreviated Failure Modes and Effects Analysis Matrix for Steps of High Importance
Abbreviated Failure Modes and Effects Analysis Matrix for Steps of High Importance

Of the different medication steps (prescription, ordering, transcription, dispensing, and administration), failure modes of high severity were identified in the ordering and administration steps. Of those, error types of medium to medium-high probability involved the misidentification of patients, patient variables (height and weight), and medications. Consensus recommendations to reduce these error types were to (1) limit choices (by predefined menus instead of free-text entry), (2) enforce entry of required data (height and weight), and (3) provide alerts for abnormal values. Failure modes of low to medium-high probability involved dosing errors due to mismatch of drug orders to chemotherapy protocols or miscalculation. Consensus recommendations to reduce error probability were to (1) enforce protocol specification and (2) automate calculations.

AUDIT DATA ANALYSIS

Completion rates of oncologist-identified steps of high importance from the pre- and post-CPOE periods are summarized in Table 2.

Table 2. 
Audit Data Analysis of High Importance Steps
Audit Data Analysis of High Importance Steps

After CPOE deployment, daily chemotherapy orders were less likely to have improper dosing on orders (relative risk [RR], 0.26; 95% confidence interval [CI], 0.11-0.61), incorrect dosing calculations (RR, 0.09; 95% CI, 0.03-0.34), missing cumulative dose calculations (RR, 0.32; 95% CI, 0.14-0.77), and incomplete nursing checklists (RR, 0.51; 95% CI, 0.33-0.80). There was no difference in the likelihood of improper dosing on treatment plans and a higher likelihood of not matching medication orders to treatment plans (RR, 5.4; 95% CI, 3.1-9.5).

COMMENT

The risks to patients posed by potential errors in the pediatric chemotherapy process are multidimensional, owing to inherent toxicities of the sometimes experimental drugs used, complexities of the protocols followed, and clinical lability of the patients' disease processes. In addition, children, possibly owing to the additional need for individualized dosing based on weight and body surface area, are at higher risk for medication errors (approximately 3 times that of adults13) in general.7

For these reasons, the maxim primum non nocere14 mandates systematic error management in the pediatric chemotherapy process as an essential part of improving pediatric oncology outcomes.15 Such management includes an informed and iterative approach to (1) model the clinical process in which errors occur; (2) identify the errors, their mechanisms, probabilities of occurrence, and impacts; (3) modify the process to eliminate or reduce the errors; (4) evaluate the effect of modifications on the process (and outcomes); and (5) update clinical process models on the basis of evaluation data.16 Systematic error management is vital to good clinical care in complex processes (such as patient medication) and complex settings (such as hospitals).

Other proactive and reactive approaches to error management in pediatric chemotherapy include (manual) multidisciplinary check of order sets,17 integrated incident and error reporting,18 knowledge acquisition and computer-aided planning of therapy with decision support for individualized therapy,19 standardization of the terminology used,20 and education of caregivers,21 in addition to standard approaches such as root cause analysis22 (mandated by the Joint Commission on Accreditation of Healthcare Organizations for the investigation of sentinel events in accredited hospitals) of errors when they occur.

Failure modes effects analysis is a methodology adapted from the military23 and recommended by the Joint Commission on Accreditation of Healthcare Organizations for redesigning health care processes24 as a proactive, systematic approach for identifying “the ways that a process or design can fail, why it might fail, and how it can be made safer.”25 Failure modes effects analysis is directly applicable to the incorporation of IT such as CPOE into a complex process such as pediatric chemotherapy. Using domain and IT expertise, the assessment identifies and prioritizes specific process errors (failure modes) and suggests ways that IT (in this case CPOE) can help to mitigate them. The collection of data (also determined by domain expertise, in this case attending oncologists) provides evidence and benchmarks for further improvement efforts.

According to literature and to the experience of the team, most medication process errors (in general) and pediatric chemotherapy errors (in particular) occur at the prescribing/ordering step,26 to which the team recommendations were directed to (1) automate calculations (via embedded calculators), (2) reduce free-text and handwritten entry (via drop-down medication lists), (3) reduce prescriber memory load (via autocompletion of specific data fields), and (4) enforce prescriber completion of essential data fields (via pop-up alerts). These functions were incorporated into the design and implementation of the application and its interface.

Incorporation of CPOE has been advocated as an effective way to reduce medication errors,27,28 but its implementation and deployment must be tailored to the needs of specific clinical processes. In pediatrics and neonatology, CPOE has been demonstrated to mitigate errors in specialized medication processes (such as total parenteral nutrition29 and continuous intravenous infusions30), and these deployments have succeeded through guidance by domain and IT experts who are familiar with the processes.

Error management must consist of continued surveillance for persistence of old errors and anticipation of the introduction of new ones. In the first iteration of CPOE error, there was an unanticipated but statistically significant decrease in the matching of orders (chemotherapy drug doses) to specific protocols. A review of the data suggested several reasons for this:

  • No automated drug-protocol linkage. At the time of this study, this feature was not included.

  • User interface problem. New or experimental drugs did not appear on the predefined menu after CPOE deployment, requiring users to enter them manually, resulting in drug-protocol nonmatch. Before CPOE deployment, all orders were handwritten, which may have also resulted in underreporting of drug-protocol nonmatches.

  • Human transcription failure. After CPOE deployment, protocols were still paper based, forcing prescribers to switch back and forth between electronic and print interfaces.

These considerations provide a basis for the next iteration of the CPOE application, which will include automated linkage of chemotherapy order sets to specific protocols at the Johns Hopkins Children's Center (planned for 2006).

A strength of this study was the inclusion of audit data from all patient encounters in the pre- and post-CPOE time frames. A limitation was the focus on process and not outcomes. Inclusion of outcome measures is projected for a future iteration.

In conclusion, pediatric chemotherapy is a complex medication process that is vulnerable to human error. Computerized provider order entry is a recognized IT intervention that can reduce errors, but its incorporation into the process must be guided by domain and IT expertise. We have described an FMEA-guided implementation of CPOE in pediatric chemotherapy at an academic medical center with data that demonstrate its impact on reducing process errors.

Back to top
Article Information

Correspondence: George R. Kim, MD, The Johns Hopkins School of Medicine, 600 N Wolfe St, Nelson 2-133, Baltimore, MD 21287-3200 (grkim@jhmi.edu).

Accepted for Publication: November 22, 2005.

Author Contributions: All of the authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Funding/Support: This study was supported by the Center for Innovation and Quality Patient Care at The Johns Hopkins Hospital.

References
1.
Scavuzzo  JGamba  N Bridging the gap: the virtual chemotherapy unit J Pediatr Oncol Nurs 2004;2127- 32
PubMedArticle
2.
Womer  RBTracy  ESoo-Hoo  WBickert  BDiTaranto  SBarnsteiner  JH Multidisciplinary systems approach to chemotherapy safety: rebuilding processes and holding the gains J Clin Oncol 2002;204705- 4712
PubMedArticle
3.
Taylor  REBailey  CCRobinson  K  et al. International Society of Paediatric Oncology; United Kingdom Children's Cancer Study Group, Results of a randomized study of preradiation chemotherapy versus radiotherapy alone for nonmetastatic medulloblastoma: the International Society of Paediatric Oncology/United Kingdom Children's Cancer Study Group PNET-3 Study J Clin Oncol 2003;211581- 1591
PubMedArticle
4.
Branowicki  PO’Neill  JBDwyer  JLMarino  BLHoulahan  KBillett  A Improving complex medication systems: an interdisciplinary approach J Nurs Adm 2003;33199- 200
PubMedArticle
5.
Dinning  CBranowicki  PO’Neill  JBMarino  BLBillett  A Chemotherapy error reduction: a multidisciplinary approach to create templated order sets J Pediatr Oncol Nurs 2005;2220- 30
PubMedArticle
6.
Apkon  MMattera  JALin  Z  et al.  A randomized outpatient trial of a decision-support information technology tool Arch Intern Med 2005;1652388- 2394
PubMedArticle
7.
Lehmann  CUKim  GR Prevention of medication errors Clin Perinatol 2005;32107- 123
PubMedArticle
8.
Ash  JSBerg  MCoiera  E Some unintended consequences of information technology in health care: the nature of patient care information system–related errors J Am Med Inform Assoc 2004;11104- 112
PubMedArticle
9.
Coiera  E Four rules for the reinvention of health care BMJ 2004;3281197- 1199
PubMedArticle
10.
Bonnabry  PCingria  LAckermann  MSadeghipour  FBigler  LMach  N Use of a prospective risk analysis method to improve the safety of the cancer chemotherapy process Int J Qual Health Care 2006;189- 16
PubMedArticle
11.
Norman  DA Design rules based on analyses of human error Commun ACM 1983;26254- 258Article
12.
 Computer physician order entry  Leapfrog Group Web site. Washington, DC Leapfrog Group2005;Available athttp://www.leapfroggroup.org/for_hospitals/leapfrog_safety_practices/cpoeUpdated 2005. Accessed November 16, 2005
13.
Kaushal  RBates  DWLandrigan  C  et al.  Medication errors and adverse drug events in pediatric inpatients JAMA 2001;2852114- 2120
PubMedArticle
14.
Smith  CM Origin and uses of primum non nocere—above all, do no harm! J Clin Pharmacol 2005;45371- 377
PubMedArticle
15.
Kaushal  RShojania  KGBates  DW Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review Arch Intern Med 2003;1631409- 1416
PubMedArticle
16.
Pape  TMGuerra  DMMuzquiz  M  et al.  Innovative approaches to reducing nurses' distractions during medication administration J Contin Educ Nurs 2005;36108- 116
PubMed
17.
Marino  BLBranowicki  PBennett  JA  et al.  Evaluating process changes in a pediatric hospital medication system Outcomes Manag 2002;610- 15
PubMed
18.
France  DJMiles  PCartwright  J  et al.  A chemotherapy incident reporting and improvement system Jt Comm J Qual Saf 2003;29171- 180
PubMed
19.
Garde  SBaumgarten  BBasu  O  et al.  A meta-model of chemotherapy planning in the multi-hospital/multi-trial-center–environment of pediatric oncology Methods Inf Med 2004;43171- 183
PubMed
20.
Merzweiler  AEhlerding  HCreutzig  U  et al.  Standardizing terminology in pediatric oncology: the basic data set [in German] Klin Padiatr 2002;214212- 217
PubMedArticle
21.
Kline  NE Promoting patient safety through the development of a pediatric chemotherapy and biotherapy provider program [editorial] J Pediatr Oncol Nurs 2004;2165- 66
PubMedArticle
22.
Wald  HShojania  KGShojania  KGedDuncan  BWedMcDonald  KMed  et al.  Root cause analysis Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Rockville, Md: Agency for Healthcare Research and Quality July 2001. Evidence Report/Technology Assessment No. 43. AHRQ Publication No. 01-E058. Available athttp://www.ahrq.gov/clinic/ptsafety/Accessed November 18, 2005
23.
Haapenen  PHelminen  A Failure mode and effects analysis of software-based automation systems STUK-YTO- TR 190. Helsinki, Finland; 2002. Available athttp://www.fmeainfocentre.com/download/softwarefmea.pdfAccessed November 18, 2005
24.
O’Leary  D Reducing medical errors: a review of innovative strategies to improve patient safety Testimony before the House Committee on Energy and Commerce Subcommittee on Health, May 8, 2002Available athttp://www.jcaho.org/news+room/on+capitol+hill/oleary_test.htmAccessed November 16, 2005
25.
Joint Commission on Accreditation of Healthcare Organizations, Team identifies ways to mitigate risks in new accreditation process Jt Comm Perspect 2004;2415- 16
PubMed
26.
Barber  NRawlins  MDean Franklin  B Reducing prescribing error: competence, control, and culture Qual Saf Health Care 2003;12(suppl 1)i29- i32
PubMedArticle
27.
Kohn  LedCorrigan  JedDonaldson  Med To Err is Human: Building a Safer Health System: Institute of Medicine.  Washington, DC National Academy Press1999;
28.
 Patient safety Leapfrog Group Web site. Available at http://www.leapfroggroup.org/safety1.htmAccessed July 26, 2004
29.
Lehmann  CUConner  KGCox  JM Preventing provider errors: online total parenteral nutrition calculator Pediatrics 2004;113748- 753
PubMedArticle
30.
Giannone  G Computer-supported weight-based drug infusion concentrations in the neonatal intensive care unit Comput Inform Nurs 2005;23100- 105
PubMedArticle
×