Effects of Computerized Physician Order Entry and Clinical Decision Support Systems on Medication Safety: A Systematic Review

Background  Iatrogenic injuries related to medications are common, costly, and clinically significant. Computerized physician order entry (CPOE) and clinical decision support systems (CDSSs) may reduce medication error rates.

Methods  We identified trials that evaluated the effects of CPOE and CDSSs on medication safety by electronically searching MEDLINE and the Cochrane Library and by manually searching the bibliographies of retrieved articles. Studies were included for systematic review if the design was a randomized controlled trial, a nonrandomized controlled trial, or an observational study with controls and if the measured outcomes were clinical (eg, adverse drug events) or surrogate (eg, medication errors) markers. Two reviewers extracted all the data. Discussion resolved any disagreements.

Results  Five trials assessing CPOE and 7 assessing isolated CDSSs met the criteria. Of the CPOE studies, 2 demonstrated a marked decrease in the serious medication error rate, 1 an improvement in corollary orders, 1 an improvement in 5 prescribing behaviors, and 1 an improvement in nephrotoxic drug dose and frequency. Of the 7 studies evaluating isolated CDSSs, 3 demonstrated statistically significant improvements in antibiotic-associated medication errors or adverse drug events and 1 an improvement in theophylline-associated medication errors. The remaining 3 studies had nonsignificant results.

Conclusions  Use of CPOE and isolated CDSSs can substantially reduce medication error rates, but most studies have not been powered to detect differences in adverse drug events and have evaluated a small number of "homegrown" systems. Research is needed to evaluate commercial systems, to compare the various applications, to identify key components of applications, and to identify factors related to successful implementation of these systems.

MEDICATION ERRORS and adverse drug events (ADEs) are common, costly, and clinically important problems.^1-7 Each year, an estimated 770 000 people are injured or die in hospitals from ADEs, which are injuries resulting from drug use.^4,5,8 Adult hospital incidence rates of ADEs have ranged from 2 to 7 per 100 admissions,^2,4,9,10 although determination of a precise national estimate is difficult because studies have used varying definitions.¹¹ Approximately 28% of ADEs are associated with a medication error and therefore are judged to be preventable.² Of preventable ADEs, 56% occurred during drug ordering.²

Two inpatient studies, 1 in adults² and 1 in children,⁷ found that medication errors occurred at rates of more than 5% and that approximately half of all medication errors occurred at the stage of drug ordering. The principal types of medication errors include missing a dose and incorrect medication doses, frequencies, or routes.² The frequency and type of medication errors found depend on the method used to detect them. Other studies,¹ which used a direct observation method to assess how accurately orders are carried out, found high rates of drug administration errors.

Analysis of medication errors suggests that prevention strategies targeting systems rather than individuals are most effective in reducing errors.¹² Computerized physician order entry (CPOE) and clinical decision support systems (CDSSs) are promising interventions that target the ordering stage of medications, where most medication errors and preventable ADEs occur. Despite growing public mandates and the obvious theoretical advantages of these systems, organizational adoption of CPOE and CDSSs has been limited. The much publicized Institute of Medicine report To Err Is Human¹³ drew attention to this "digital divide" between health care and other industrial sectors and called for more widespread adoption of information technology solutions to improve medication safety. Results of recent surveys^14,15 of US hospitals indicate that only 4.3% to 15.0% of hospitals have an electronic medication order-entry system in place. If partially implemented systems and institutions in the process of acquiring systems are included, this figure may be as high as 30%.^14,15 The degree to which health care has lagged behind other sectors in automating complex and hazardous processes is striking. For example, adoption of information technology interventions such as bar coding and automated drug delivery systems to reduce drug administration errors has also been slow. Therefore, we undertook this study to systematically review the cumulative evidence on the effects of CPOE and CDSSs on medication safety.

Computerized physician order entry refers to a variety of computer-based systems that share the common features of automating the medication ordering process and that ensure standardized, legible, and complete orders. Clinical decision support systems are built into almost all CPOE systems to varying degrees. Basic clinical decision support provides computerized advice regarding drug doses, routes, and frequencies, and more sophisticated CDSSs can perform drug allergy checks, drug–laboratory value checks, and drug-drug interaction checks and can provide reminders about corollary orders (eg, prompting the user to order glucose checks after ordering insulin) or drug guidelines.¹⁶

Clinical decision support systems may also be implemented without CPOE. Basic CDSSs often assist in tasks such as drug selection, dosing, and duration, and more refined CDSSs can incorporate patient- or pathogen-specific information. The ordering physician may view such advice and then proceed with a conventional handwritten medication order.

Studies were identified by searching the US National Library of Medicine MEDLINE electronic bibliographic database and the electronic Cochrane Library. The MEDLINE search strategy was performed using the following MeSH terms: hospital information systems; decision support systems, clinical; and drug therapy, computer-assisted. In addition, we searched for key title words related to computerized order entry and combined the results of these searches with MeSH terms capturing adverse events and medical errors: medical error, iatrogenic disease, sentinel surveillance, and safety. The Cochrane Library was searched using similar key terms and title words. Reference lists from all relevant articles, including 2 systematic reviews,^17,18 were reviewed to identify additional primary studies.

Specifically, we sought articles describing computerized systems for performing general order entry or CDSSs for guiding physicians in the order-writing process. Computerized programs that screen for potential ADEs were not included, unless they interact with users during the order-writing process,^10,19,20 and neither were CDSSs built into programmable intravenous infusion pumps.^21-23 Although both of these practices play a role in improving medication safety, they do not affect the stage of order writing, which is the focus of this review.

Two of us (R.K. and K.G.S.) reviewed all the articles to determine the level of evidence for practice effectiveness using frameworks developed by the University of California San Francisco–Stanford Evidence-Based Practice Center for the evaluation of study design (Table 1) and measured outcomes (Table 2).²⁴ This classification scheme was developed because of the heterogeneous nature of the studies evaluating CPOE and CDSSs. The scheme incorporates features of existing frameworks and recommendations for evaluating and synthesizing evidence.^25-31 We included articles with a minimum level 3 study design (observational studies with controls) and level 2 outcomes (surrogate clinical outcomes). Studies that reported a mixture of level 2 and level 3 outcomes (outcomes with an indirect or unestablished connection to the target safety outcome) were included, as had been decided prospectively. Disagreements were resolved by discussion.

Medication errors are errors in the process of ordering, transcribing, dispensing, administering, or monitoring medications. One example is an order written for acetaminophen without a route of administration. Medication errors include a mixture of errors with differing potentials for patient injury.

Potential ADEs are medication errors with significant potential to harm a patient that may or may not actually reach a patient. An example of an intercepted potential ADE is an order written for a morphine overdose that is noticed and corrected by a pharmacist before the drug is administered. An example of a nonintercepted potential ADE is an administered overdose of morphine to a patient who does not have any sequelae. Medication errors and potential ADEs were considered surrogate outcomes (level 2).

Adverse drug events are injuries resulting from drug use and therefore constitute clinical outcomes (level 1). Adverse drug events associated with a medication error are considered preventable, whereas those not associated with a medication error are considered nonpreventable. An example of a preventable ADE is the development of a rash after the administration of penicillin to a known penicillin-allergic patient. In contrast, a nonpreventable ADE is the development of a penicillin-associated rash in a patient with no known previous allergies.

Nonintercepted serious medication errors include nonintercepted potential ADEs and preventable ADEs (ie, medication errors that either have the potential to or actually cause harm to a patient). Errors that are not intercepted and have the potential to or actually cause injury are the most important from the perspective of patient safety.

Studies were grouped into 2 categories: those evaluating CPOE with CDSSs and those evaluating CDSSs alone. No studies were found assessing CPOE alone. We did not quantitatively score the quality of the studies owing to the recognized difficulties in quality scoring in general^29,32 and especially for a heterogeneous group of studies such as those included in this review. Nonetheless, both reviewers abstracted each study for prospectively determined elements pertaining to methodological quality.³³ In addition, the reviewers described the study design, setting, outcomes, and results. Other extracted information included data regarding potential harm from the practice and issues regarding cost and implementation.

Twelve studies met the inclusion criteria for study design and measured outcomes. The 5 studies listed in Table 3 evaluated CPOE with CDSSs.^34-38 In the first study,³⁴ investigators at the Regenstrief Institute for Health Care (affiliated with the Indiana University School of Medicine, Indianapolis) conducted a randomized controlled trial of 2181 patients evaluating the effects of CPOE on corollary order prescribing. The remaining 4 studies^35-38 evaluated the CPOE system at Brigham and Women's Hospital (BWH). The first BWH study³⁵ was a cross-sectional analysis of 6771 patients comparing an intervention period of CPOE with CDSSs with a historical period, the next 2 BWH studies^36,37 were time series analyses, and the final BWH study³⁸ was a randomized controlled trial with a crossover design of 7490 patients.

Table 4 lists 7 studies^39-45 that evaluated isolated CDSSs; 6 were randomized controlled trials and 1 was a prospective before-after analysis. All of these studies were conducted in the inpatient setting. Burton et al³⁹ assessed the use of a computerized aminoglycoside dosing program for 75 patients at the Dallas Veterans Affairs Medical Center in Texas. Evans et al,^40,41 at LDS Hospital, Salt Lake City, Utah, performed 2 studies on antibiotic CDSSs: a randomized controlled trial⁴⁰ of empiric antibiotic drug selection using CDSSs with 451 patients and a cross-sectional analysis⁴¹ comparing an intervention period of a computer-assisted anti-infective drug management program with a historical control period for 1136 patients in the intensive care unit. Casner et al⁴² and Hurley et al⁴³ performed randomized controlled trials of computerized theophylline dosing programs with 17 and 48 patients, respectively. Mungall et al⁴⁴ evaluated a heparin dosing program for 25 inpatients, and White et al⁴⁵ evaluated a warfarin dosing program for 39 inpatients. In summary, many of the CDSS studies had small sample sizes and consequently were underpowered.

Most of the included CPOE studies primarily measured level 2 and level 3 outcomes because level 1 outcomes are significantly less frequent. Therefore, larger and longer studies are necessary to measure the effects of an intervention on ADE rates, and the costs of such studies are very high. The first 2 BWH studies^35,36 primarily measured nonintercepted serious medication errors (level 2) and medication errors (level 2) but also included ADEs as a secondary outcome (level 1). The other 3 CPOE studies reported level 2 and level 3 outcomes (ie, prescribing practices,³⁷ corollary orders,³⁴ and appropriate drug dose and frequency³⁸). Corollary orders (level 2 and level 3 outcomes) are orders needed to detect or ameliorate potential effects of a trigger order, for example, ordering regular laboratory tests of coagulation status after starting a patient on intravenous heparin therapy.

Similarly, the studies evaluating CDSSs report level 1 and level 2 outcomes, with level 1 outcomes often a secondary end point. Burton et al³⁹ reported rates of toxic serum aminoglycoside levels (level 2). Evans et al determined rates of pathogen susceptibility to an antibiotic drug regimen (level 2)⁴⁰ and rates of anti-infective drug–associated ADEs (level 1).⁴¹ Casner et al⁴² and Hurley et al⁴³ reported rates of toxic serum theophylline levels (level 2). Mungall et al⁴⁴ and White et al⁴⁵ reported bleeding complications (level 1), and White et al⁴⁵ also reported overanticoagulation rates (level 2).

The first BWH study³⁵ assessing the impact of CPOE with CDSSs demonstrated a 55% decrease in nonintercepted serious medication errors (P = .01). As a secondary outcome, this study found a 17% decrease in the preventable ADE rate, which was not statistically significant (P = .37). The CPOE application at the time of this study included only basic decision support, with limited checking for allergies and drug-drug interactions. The second study,³⁶ a time series analysis, evaluated medication error rates before CPOE and in the 3 years subsequent to its implementation. It demonstrated an 81% decrease in medication errors and an 86% decrease in nonintercepted serious medication errors (P<.001 for both). This study found a decrease in the rate of ADEs per 1000 patient-days from 14.7 to 9.6 during the study (P = .09) and a decrease in the number of preventable ADEs from 5 to 2 (P = .05).

The remaining 3 studies assessed more specific types of medication errors. Overhage et al³⁴ demonstrated a greater than 25% improvement in the rates of corollary orders with implementation of computerized reminders. Teich et al³⁷ demonstrated 5 prescribing improvements in types, doses, and frequencies of drug use with the implementation of computerized clinical decision support. Finally, Chertow et al³⁸ demonstrated a 13% decrease in inappropriate dose and a 24% decrease in inappropriate frequency for nephrotoxic drugs in patients with renal insufficiency (P<.001 for both).

Three of the studies assessing isolated CDSSs evaluated computerized antibiotic drug advice and demonstrated lower rates of toxic levels, improved pathogen susceptibility, and a decreased anti-infective drug–associated ADE rate. Burton et al³⁹ evaluated a computerized aminoglycoside dosing program and demonstrated lower rates of toxic levels in intervention patients, but the results were not statistically significant (P = .40). Evans et al⁴⁰ demonstrated a 17% greater pathogen susceptibility to an antibiotic drug regimen suggested by a computer consultant vs a physician (P<.001). In another study, Evans et al⁴¹ reported a 70% decrease in ADEs caused by anti-infective agents through use of a computer-based anti-infective drug management program (P = .02).

Two other studies evaluated theophylline dosing. Casner et al⁴² demonstrated no difference in rates of toxic serum levels. In contrast, Hurley et al⁴³ demonstrated significantly lower rates of toxic levels in intervention patients (18.9%) than in control patients (37.8%) (P = .04).

The final 2 studies evaluated anticoagulation agents. Evaluation of a heparin dosing system demonstrated lower rates of bleeding events in intervention patients (4.2%) vs control patients (7.7%), but without statistical significance (P = .6).⁴⁴ Similarly, evaluation of a warfarin (Coumadin) dosing program demonstrated lower rates of bleeding complications (0% vs 8%) and overanticoagulation rates (5% vs 17%), but neither result was statistically significant (P = .11).⁴⁵

These studies provide evidence that the use of CPOE with CDSSs significantly decreases medication error and serious medication error rates at 2 institutions with "homegrown" systems. However, the effect on ADE rates has not been adequately tested because studies with sufficient power have not been performed. There is a strong correlation between medication errors and ADEs, so such applications will almost certainly reduce ADE rates. Nevertheless, medication errors have widely varying potential for harm, and it seems easiest to prevent those that rarely cause injury.^35,36

Some of the CDSS studies,^40,41,43 particularly those evaluating antibiotic drug–associated programs, demonstrated focal reductions in medication errors with statistical significance as well as some decreases in ADE rates. Although other CDSSs tended to have statistically insignificant results, these studies^39,42,44,45 were underpowered, with sample sizes of 17 to 75 patients. Comprehensive applications, including CPOE and sophisticated decision support, will likely have the greatest effect.

One important question is whether the currently available data are sufficiently compelling that CPOE should be widely adopted or whether further research is required. We believe that further studies targeted at a few critical questions are desirable but not a requirement before widespread adoption. For example, a multicenter study evaluating the impact of CPOE on ADE rates would cost tens of millions of dollars and would be hard to perform because CPOE is a complex application touching on so many parts of the clinical and information systems. Instead, research should focus on questions such as the following: What are the differences among various CPOE systems? What are barriers to adoption? What are the key decision support elements? How effective are specific pieces of decision support? How should these applications be implemented in community hospitals?

Most studies of CPOE have assessed only 2 internally developed (homegrown) systems. To date, dissemination of these systems has been limited for a variety of reasons. Most hospitals use commercial systems. Relatively few vendors have CPOE applications that have broad use at more than a handful of hospitals. As with evaluations of therapeutic agents, there is a reasonable expectation of a class effect⁴⁶ with many CPOE systems, but classes of CPOE remain to be established. For example, CPOE systems with no decision support will almost certainly decrease error rates less than systems with sophisticated decision support. Thus, one area for further research consists of developing tools to assess the extent to which a specific commercial CPOE application will reduce the medication error rate or the preventable ADE rate. Comparisons among such commercial products will likewise be informative.

Organizational adoption of CPOE has been limited. One survey¹⁴ of 668 hospitals indicated that 15% had at least partially implemented CPOE. A more recent survey¹⁵ of pharmacy directors at 1091 acute care hospitals in the United States (49% response rate) reported that 4.3% of hospitals had an electronic medication order-entry system in place.

Many barriers to CPOE adoption exist. Rogers⁴⁷ suggests that perceived attributes of an innovation and organizational social context strongly affect the innovation adoption rate. Perhaps most important, a health care institution must garner financial and organizational support before introducing CPOE with CDSSs. Computerized physician order entry requires large up-front capital investment with more remote, albeit substantial, returns. Such investment is especially challenging when organizations are losing money. In addition to the financial obstacles, implementing sophisticated new clinical information systems presents substantial organizational challenges owing to the impact on institutional culture and clinical workflow^48-51 and the need to accommodate existing institutional systems used for billing, laboratory, and pharmacy data.⁵²

In addition, the efficacy of individual decision support elements warrants further investigation. Many of the CDSS studies included in this review produced nonsignificant results. Yet, it is difficult to draw definitive conclusions because of the small sample sizes. Larger studies need to be performed, as do studies identifying key, successful decision support elements.

Purchasing commercial CPOE systems is generally more expensive than is internally developing systems. Brigham and Women's Hospital has reported costs of $1.9 million for developing and implementing CPOE in 1992, with ongoing maintenance costs of $500 000 per year, although this was incremental to what was already a highly developed clinical system.⁵³ Fewer data are available regarding the cost of purchasing and implementing large commercial systems, but it may be on the order of tens of millions of dollars, especially if related clinical applications such as a clinical data repository must be upgraded. Several studies^34,37,54 report that only minimal resources are needed to introduce or maintain decision support programs into existing order-entry programs.

The beneficial effects of CPOE systems extend beyond medication safety and include reduced costs and quality improvement. These benefits have been achieved by providing feedback about the appropriateness and costs of laboratory and radiologic tests, easy implementation of clinical pathways, improved quality measurement, and improved coding and billing. Brigham and Women's Hospital estimated net savings of $5 to $10 million per year for the CPOE system.⁵⁵ In a randomized controlled clinical trial, Tierney et al⁵⁶ demonstrated that CPOE linked to a comprehensive electronic medical record system resulted in charges that were $887 (12.7%) lower per admission. Cost savings associated with averted ADEs may be considerable. For example, BWH, a 720-bed academic institution, estimated costs before CPOE implementation of $2.8 million annually for preventable ADEs.⁵⁷ Evans et al⁵⁸ reported a $100 000 per year cost avoidance with a computer-assisted antibiotic drug dosing program attributable to decreased antibiotic drug use and avoided ADEs.

As with any other technology, CPOE and CDSSs may introduce different types of medication errors. Incorrect default dosing or route suggestions may lead to potentially erroneous orders. For example, the first time series analysis³⁶ at BWH demonstrated an initial increase in intercepted potential ADEs attributable to the ordering screen structure for potassium chloride, which made it easy to order large doses of intravenous potassium. Once identified, this error was rectified, but this event underscores the importance of ongoing close scrutiny of CPOE and CDSSs. In general, as users become accustomed to CPOE and CDSSs, they are likely to accept computer suggestions with minimal reflection,⁵⁹ emphasizing the importance of testing decision support default settings and suggestions.

When CPOE systems are not electronically linked to computerized pharmacy systems, pharmacists must manually reenter orders into the pharmacy system, with a resultant increase in chance of error. Pedersen et al¹⁵ found that 25.7% of surveyed hospitals with electronic prescribing lacked information system linkages to pharmacy systems.

The trigger level for computerized warnings must be set to the appropriate sensitivity. In situations with a potential for significant harm, it is important that providers receive warnings without being overwhelmed by alarms of marginal value. Hardware outages and software instability pose further risks. In particular, the reliability needed for CPOE is much higher than that required for systems that simply report laboratory test results. Finally, physicians can electronically write an order in the wrong patient's record, analogous to handwriting an order in the wrong patient's medical chart.

In the meantime, public and private groups are increasingly demanding implementation of CPOE and other information technologies. The Leapfrog Group, a consortium of companies that belong to the Business Roundtable, has endorsed CPOE in hospitals as 1 of 3 changes that would most improve patient safety in the United States.⁶⁰ A Medicare Payment Advisory Commission report⁶¹ suggested instituting financial incentives for CPOE implementation. Legislation has also been introduced at the federal and state levels. United States Senators Bob Graham (D-Fla) and Olympia Snowe (R-Maine) recently introduced a bill, titled the "Medication Errors Reduction Act of 2001," to establish an informatics system grant program for hospitals and skilled nursing facilities.⁶² At the state level, California recently enacted legislation stipulating that acute-care hospitals implement information technology such as CPOE to reduce medication-related errors.⁶³

In conclusion, use of CPOE and isolated CDSSs significantly decreases medication error rates and provides other important benefits related to medication use. Funding constraints and public pressure to bridge the digital divide in health care make a large trial comparing paper ordering to sophisticated CPOE systems unlikely. However, a need exists for research evaluating commercial systems, the relative benefits of different classes of systems, and factors related to successful implementation of these systems.

Corresponding author and reprints: Rainu Kaushal, MD, MPH, Division of General Internal Medicine, Brigham and Women's Hospital, PBBA3, 75 Francis St, Boston, MA 02115 (e-mail: rkaushal@partners.org).

Accepted for publication August 16, 2002.

Portions of this article are based on work performed by the authors in collaboration with the University of California San Francisco–Stanford Evidence-Based Practice Center under contract 290-97-0013 to the Agency for Healthcare Research and Quality, Rockville, Md.

Dr Bates has received honoraria for speaking from the Eclipsys Corp (Boca Raton, Fla), which has licensed the rights to the BWH Clinical Information System. The hospital no longer has a financial relationship with Eclipsys. Dr Bates is a coinventor on patent No. 6029138 held by BWH on the use of decision support software for medical management, licensed to Medicalis Corp (Boston, Mass). He holds a minority equity position in the privately held company Medicalis, which develops Web-based decision support for radiology test ordering, and he serves as a consultant to Medicalis. He is also a consultant and serves on the advisory board for McKesson MedManagement (Minneapolis, Minn), a company that assists hospitals in preventing adverse drug events.

We thank Robert M. Wachter, MD, Bradford W. Duncan, MD, Kathryn M. McDonald, MSc, and Amy J. Markowitz, JD, for their careful review of early drafts of this manuscript.

The authors are responsible for the contents of this article, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or of the US Department of Health and Human Services.