Customize your JAMA Network experience by selecting one or more topics from the list below.
Koppel R, Metlay JP, Cohen A, et al. Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. JAMA. 2005;293(10):1197–1203. doi:10.1001/jama.293.10.1197
Author Affiliations: Department of Sociology
(Dr Koppel), Department of Medicine, Cardiovascular Division (Dr Kimmel) and
General Medicine Division (Drs Metlay and Strom), Center for Clinical Epidemiology
and Biostatistics (Drs Koppel, Metlay, Cohen, Kimmel, and Strom and Mr Localio),
Department of Biostatistics and Epidemiology (Drs Metlay, Kimmel, and Strom
and Mr Localio), Department of Pharmacology (Dr Strom), Center for Education
and Research in Therapeutics (Drs Metlay and Strom and Mr Localio), University
of Pennsylvania School of Medicine (Mr Abaluck), Philadelphia; and Center
for Health Equity Research and Promotion, Department of Veterans Affairs,
Philadelphia (Dr Metlay).
Context Hospital computerized physician order entry (CPOE) systems are widely
regarded as the technical solution to medication ordering errors, the largest
identified source of preventable hospital medical error. Published studies
report that CPOE reduces medication errors up to 81%. Few researchers, however,
have focused on the existence or types of medication errors facilitated by
Objective To identify and quantify the role of CPOE in facilitating prescription
Design, Setting, and Participants We performed a qualitative and quantitative study of house staff interaction
with a CPOE system at a tertiary-care teaching hospital (2002-2004). We surveyed
house staff (N = 261; 88% of CPOE users); conducted 5 focus groups
and 32 intensive one-on-one interviews with house staff, information technology
leaders, pharmacy leaders, attending physicians, and nurses; shadowed house
staff and nurses; and observed them using CPOE. Participants included house
staff, nurses, and hospital leaders.
Main Outcome Measure Examples of medication errors caused or exacerbated by the CPOE system.
Results We found that a widely used CPOE system facilitated 22 types of medication
error risks. Examples include fragmented CPOE displays that prevent a coherent
view of patients’ medications, pharmacy inventory displays mistaken
for dosage guidelines, ignored antibiotic renewal notices placed on paper
charts rather than in the CPOE system, separation of functions that facilitate
double dosing and incompatible orders, and inflexible ordering formats generating
wrong orders. Three quarters of the house staff reported observing each of
these error risks, indicating that they occur weekly or more often. Use of
multiple qualitative and survey methods identified and quantified error risks
not previously considered, offering many opportunities for error reduction.
Conclusions In this study, we found that a leading CPOE system often facilitated
medication error risks, with many reported to occur frequently. As CPOE systems
are implemented, clinicians and hospitals must attend to errors that these
systems cause in addition to errors that they prevent.
Adverse drug events (ADEs) are estimated to injure or kill more than
770 000 people in hospitals annually.1 Prescribing
errors are the most frequent source.2-5 Computerized
physician order entry (CPOE) systems are widely viewed as crucial for reducing
prescribing errors2,3,6-17 and
saving hundreds of billions in annual costs.18,19 Computerized
physician order entry system advocates include researchers, clinicians, hospital
administrators, pharmacists, business councils, the Institute of Medicine,
state legislatures, health care agencies, and the lay public.2,3,6-10,12,14-17,20-22 These
systems are expected to become more prevalent in response to resident working-hour
limitations and related care discontinuities23 and
will supposedly offset causes (eg, job dissatisfaction) and effects (eg, ADEs)
of nursing shortages.24,25 Such
a system is increasingly recommended for outpatient practices (Box).
Free of handwriting identification problems
Faster to reach the pharmacy
Less subject to error associated with similar drug
More easily integrated into medical records and decision-support
Less subject to errors caused by use of apothecary
Easily linked to drug-drug interaction warnings
More likely to identify the prescribing physician
Able to link to ADE reporting systems
Able to avoid specification errors, such as trailing
Available and appropriate for training and education
Available for immediate data analysis, including postmarketing reporting
Claimed to generate significant economic savings
With online prompts, CPOE systems can
Link to algorithms to emphasize cost-effective medications
Reduce underprescribing and overprescribing
Reduce incorrect drug choices
Abbreviations: ADE, adverse drug event; CPOE, computerized physician
Adoption of CPOE perhaps gathered such strong support because its promise
is so great, effects of medication error so distressing, circumstances of
medication error so preventable, and studies of CPOE preliminary yet so positive.21,26-28 Studies
of CPOE, however, are constrained by its comparative youth, continuing evolution,
need to focus on potential rather than actual errors, and limited dissemination
(in 5% to 9% of US hospitals).29-36 Two
critical studies21,30 examined
distinctions between reductions in possible ADEs vs actual reductions in ADEs;
the former are well documented and often cited, but the latter are largely
undocumented and unknown. Studies of CPOE efficacy (17% to 81% error reduction)
usually focus on its advantages2,3,6-11,14-16 and
are generally limited to single outcomes, potential error reduction, or physician
studies combine CPOE and clinical support systems in their analyses.30,40,41
In the past 3 years, though, a few studies21,26-28,30,31,33,42-46 suggested
some ways that CPOE might contribute to medication errors (eg, ignored false
alarms, computer crashes, orders in the wrong medical records). Several decades
of human-factors research, moreover, highlighted unintended consequences of
technologic solutions, with recent discussions on hospitals.32,33,42-44,47-52
We undertook a comprehensive, multimethod study of CPOE-related factors
that enhance risk of prescription errors.
We performed a quantitative and qualitative study incorporating structured
interviews with house staff, pharmacists, nurses, nurse-managers, attending
physicians, and information technology managers; real-time observations of
house staff writing orders, nurses charting medications, and hospital pharmacists
reviewing orders; focus groups with house staff; and written questionnaires
administered to house staff. Qualitative research was iterative and interactive
(ie, interview responses generated new focus group questions; focus group
responses targeted issues for observations).
We studied a major urban tertiary-care teaching hospital with 750 beds,
39 000 annual discharges, and a widely used CPOE system (TDS) operational
there from 1997 to 2004. Screens were usually monochromatic with pre-Windows
interfaces (Eclipsys Corp, Boca Raton, Fla). The system was used on almost
all services and integrated with the pharmacy’s and nurses’ medication
This study was approved by the University of Pennsylvania institutional
review board. The researchers were not involved in CPOE system design, installation,
Intensive One-on-One House Staff Interviews. To
develop our initial questions, we conducted 14 one-on-one house staff interviews.
An experienced sociologist (R.K.) conducted the open-ended interviews, focusing
on stressors and other prescribing-error sources (mean interview time, 26
minutes; range, 14-66 minutes).
Focus Groups. We conducted 5 focus groups with
house staff on sources of stress and prescribing errors, moderated by an experienced
sociologist (R.K.) and audiorecorded. Participants were reimbursed $40 (average
group size, 10; range, 7-18; and average length, 1.75 hours; range, 1.4-2
Expert Interviews. We interviewed the surgery
chair, pharmacy and technology directors, clinical nursing director, 4 nurse-managers,
5 nurses, an infectious disease fellow, and 5 attending physicians. All interviews,
except 1, were privately conducted by the same investigator (R.K.).
Shadowing and Observation. During a discontinuous
4-month period (2002-2003), we shadowed 4 house staff, 3 attending physicians,
and 9 nurses engaged in patient care and CPOE use. We observed 3 pharmacists
reviewing orders. The researcher (R.K.) wore a faculty identification badge.
Observation notes were freehand but guided by the interview findings.
Survey. From 2002 to the present, we distributed
structured, self-administered questionnaires to house staff who order medications
via CPOE. The 71-item questionnaire focused on working conditions and sources
of error and stress. We report here on 10 CPOE-related questions. We constructed
the survey after our interviews and focus groups, leading us to provide separate
answer options about sources of error and sources of stress; add questions
on CPOE as a possible source of error risk, an issue that emerged in our qualitative
research; and quantify the frequency of these error risks. Not all CPOE-related
error risks are amenable to survey questions. We have robust survey results
on 10 of the 22 identified error risks; these findings are presented with
the qualitative findings.
The sampled population (N = 291) included house staff who
typically enter more than 9 medication orders per month. The target study
population excluded 648 residents in services that seldom use CPOE: pathology,
podiatry, occupational medicine, anesthesia, radiology, radiation oncology,
ophthalmology, and dermatology.
More than 70% of the questionnaires were administered at routine house
staff meetings. Other house staffwere located via departmental coordinators
or pagers. Participants received $5 coupons for local coffee shops. Two hundred
sixty-one house staff (88% of the target population) completed the questionnaire.
Characteristics of the house staff were as follows. Of 94 interns contacted,
85 (90.4%) participated; of 96 second-year residents, 84 (87.5%) participated;
and of 107 third- through fifth-year residents, 92 (85.9%) participated. The
participating sample was 44.8% female, 66.3% white, and 32.5% were interns.
Participants’ mean age was 29.6 years. These data did not differ significantly
from characteristics of nonparticipants.
Our qualitative and quantitative research identified 22 previously unexplored
medication-error sources that users report to be facilitated by CPOE. We group
these as (1) information errors generated by fragmentation of data and failure
to integrate the hospital’s several computer and information systems
and (2) human-machine interface flaws reflecting machine rules that do not
correspond to work organization or usual behaviors.
Assumed Dose Information. House staff often
rely on CPOE displays to determine minimal effective or usual doses. The dosages
listed in the CPOE display, however, are based on the pharmacy’s warehousing
and purchasing decisions, not clinical guidelines. For example, if usual dosages
are 20 or 30 mg, the pharmacy might stock only 10-mg doses, so 10-mg units
are displayed on the CPOE screen. Consequently, some house staff order 10-mg
doses as the usual or “minimally effective” dose. Similarly, house
staff often rely on CPOE displays for normal dosage ranges.
House staff regularly use CPOE to determine dosages (Table). In the last 3 months, 73% of house staff reported using
CPOE displays to determine low doses for medications they did not usually
prescribe; 82% used CPOE displays to determine range of doses (Table). Two fifths (38%-41%) used CPOE displays to determine dosages
at least a few times weekly; 10% to 14% used CPOE displays in this misleading
Medication Discontinuation Failures. Ordering
new or modifying existing medications is usually a separate process from canceling
(“discontinuing”) an existing medication. Without discontinuing
the current dose, physicians can increase or decrease medication (giving a
“double” total dose, eg, every 6 hours and every 8 hours), add
new but duplicative medication, and add conflicting medication. Medication-canceling
ambiguities are exacerbated by the computer interface and multiple-screen
displays of medications; as discussed below, viewing 1 patient’s medications
may require 20 screens.
Discontinuation failures “for at least several hours” from
not seeing patients’ complete medication records were reported by 51%
(Table). Twenty-two percent indicated
that this failure occurs a few times weekly, daily, or more frequently.
Procedure-Linked Medication Discontinuation Faults. Procedures and certain tests are often accompanied by medications.
If procedures are canceled or postponed, no software link automatically cancels
Immediate Orders and Give-as-Needed Medication Discontinuation
Faults. NOW (immediate) and PRN (give as needed) orders may not enter
the usual medication schedule and are seldom discussed at handoffs. Also,
because medication charting is so cumbersome and displays so fragmented, NOW
and PRN orders are less certain to be charted or canceled as directed. Failure
to chart or cancel can result in unintended medications on subsequent days
or reordering (duplications) on the same day.
Antibiotic Renewal Failure. To maximize appropriate
antibiotic prescribing, house staff are required to obtain approval by infectious
disease fellows or specialist pharmacists. Lack of coordination among information
systems, however, can produce gaps in therapy because antibiotics are generally
approved for 3 days. Before the third day, house staff should request continuation
or modification. To aid this process, reapproval stickers are placed on paper
charts on the second day. However, when house staff order medications, they
primarily use electronic charts, thus missing warning stickers. No warning
is integrated into the CPOE system, and ordering gaps expand until noticed.
Some unintentional “gaps” continue indefinitely because it is
unknown whether antibiotics were intentionally halted.
In the last 3 months, 83% of house staff observed gaps in antibiotic
therapy because of unintended delays in reapproval. Twenty-seven percent reported
this occurrence a few times weekly; 13%, once daily or more frequently (Table).
Diluent Options and Errors. A recent CPOE innovation
requires house staff to specify diluents (eg, saline solution) for administering
antibiotics. A few diluents interact with antibiotics, generating precipitates
or other problems. Many house staff are unaware of impermissible combinations.
Pharmacists catch many such errors, but their interventions are time-consuming
and not ensured.
Allergy Information Delay. CPOE provides feedback
on drug allergies, but only after medications are ordered. Some house staff
ignored allergy notices because of rapid scrolling through screens, the need
to order many medications, difficulties discontinuing and reordering medications,
possibility of false allergy information, and, most important, post hoc timing
of allergy information. House staff claimed post hoc alerts unintentionally
encourage house staff to rely on pharmacists for drug-allergy checks, implicitly
shifting responsibility to pharmacists.
Conflicting or Duplicative Medications. The
CPOE system does not display information available on other hospital systems.
For example, only the pharmacy’s computer provides drug interaction
and lifetime limit warnings. Pharmacists call house staff to clarify questionable
orders, but this additional step costs time and increases error potential.
House staff and pharmacists reported that this method generates tension.
Patient Selection. It is easy to select the
wrong patient file because names and drugs are close together, the font is
small, and, most critical here, patients’ names do not appear on all
screens. Different CPOE computer screens offer differing colors and typefaces
for the same information, enhancing misinterpretation as physicians switch
among screens. Patients’ names are grouped alphabetically rather than by house
staff teams or rooms. Thus, similar names (combined with small fonts, hectic
workstations, and interruptions) are easily confused.
Fifty-five percent of house staff reported difficulty identifying the
patient they were ordering for because of fragmented CPOE displays; 23% reported
that this happened a few times weekly or more frequently (Table).
Wrong Medication Selection. A patient’s
medication information is seldom synthesized on 1 screen. Up to 20 screens
might be needed to see all of a patient’s medications, increasing the
likelihood of selecting a wrong medication.
Seventy-two percent of house staff reported that they were often uncertain
about medications and dosages because of “difficulty in viewing all
the medications on 1 screen.”
Unclear Log On/Log Off. Physicians can order
medications at computer terminals not yet “logged out” by the
previous physician, which can result in either unintended patients receiving
medication or patients not receiving the intended medication.
Failure to Provide Medications After Surgery. When
patients undergo surgery, CPOE cancels their previous medications. When surgeons
order new or renewed medications, however, the orders are “suspended”
(not sent to the pharmacy) until “activated” by postanesthesia-care
nurses. But these “activations” still do not dispense medications.
Physicians must reenter CPOE and reactivate each previously ordered medication.
Surgery residents reported that they sometimes overlooked this extra process.
Postsurgery “Suspended” Medications. Physicians
ordering medications for postoperative patients whom they actually observe
on hospital floors can be deceived by patients’ real location vs patients’
computer-listed location. If patients were not logged out of postanesthesia
care, the CPOE will not process medication orders, labeling them “suspended.”
Physicians must renegotiate the CPOE and resubmit orders for patients to receive
Loss of Data, Time, and Focus When CPOE Is Nonfunctional. CPOE is shut down for periodic maintenance, and crashes are common.
Backup systems prevent loss of data previously entered. However, orders being
entered when the system crashes are lost and cannot be reentered until the
system is restarted. House staff reported that the need to wait for the system’s
revival and order reentry increases error risks.
Eighty-four percent reported delayed medication orders because of system
shutdowns. Forty-seven percent reported that shutdowns occur a few times weekly
to more than once daily (Table). The
CPOE manager confirmed house staff downtime estimates; 2 or 3 weekly crashes
of at least 15 minutes are common.
Sending Medications to Wrong Rooms When the Computer
System Has Shut Down. If the computer system is down when a patient
is moved within the hospital, CPOE does not alert the pharmacy, and medications
are sent to the “old” room, thus being lost or delayed. Also,
wrong medications might be administered to “new” patients in “old”
Late-in-Day Orders Lost for 24 Hours. When
patients leave surgery or are admitted late in the day, medications and laboratory
orders might be requested for “tomorrow” at, for example, 7 AM. By the time the intern enters the orders, however, it might already
be “tomorrow” (ie, after midnight). Therefore, patients do not
receive medications or tests for an extra day.
Role of Charting Difficulties in Inaccurate and Delayed
Medication Administration. Nurses are required to record (chart) administration
of medications contemporaneously. However, contemporaneous charting requires
time when there is little time available. Computerized physician order entry
systems compound this challenge considerably. To chart drug administrations,
nurses must stop administering medications, find a terminal, log on, locate
that patient’s record, and individually enter each medication’s
administration time. If medications are not administered (eg, patient was
out of the room), nurses must scroll through several additional screens to
record the reason(s) for nonadministration.
Nurses reported that up to 60% of their medications are not recorded
contemporaneously but are charted at shift end or post hoc by the nurse manager
via global computer commands.
Many house staff, aware of recording inaccuracies, seek nurses to determine
real administration times of time-sensitive drugs (eg, aminoglycosides). House
staff reported that these additional steps are distracting and time-consuming.
Interrupted ordering or medication reviews can increase error risks.
Moreover, because of cumbersome charting, some medications, especially
insulin, are recorded on parallel systems (ie, paper chart, separate paper
sheets, or directly in CPOE). Multiple systems cause confusion, and off-system
information is sometimes lost.
Inflexible Ordering Screens, Incorrect Medications. House staff reported that because of CPOE inflexibility, nonstandard
specifications (eg, test modifications or specific scan angles) are often
impossible to enter. Medications accompanying procedures must be stopped and
reordered, with dangers linked to uncertain canceling and reordering.
Similarly, nonformulary medications can be lost because they must be
entered on separate screen sections, might not be sent to the pharmacy, and
might escape nurses’ notice (eg, nonformulary medication to prevent
organ rejection was not listed among medications in CPOE, was not sent to
the pharmacy, and was ignored for 6 days).
Ninety-two percent reported that CPOE is inflexible, generating difficulties
in specifying medications or ordering off-formulary medications. Thirty-one
percent reported that this occurred a few times weekly; 24% said daily or
more frequently (Table).
Our qualitative research identified 22 situations in which CPOE increased
the probability of prescribing errors. Our quantitative data reveal that several
CPOE-enhanced error risks appear common (ie, observed by 50% to 90% of house
staff) and frequent (ie, repeatedly observed to occur weekly or more often).
We broadly grouped the error risks as information errors generated by fragmentation
of data and failure to integrate the hospital’s several computer and
information systems (10 error types) and human-machine interface flaws reflecting
machine rules that do not correspond to work organization or usual behaviors
(12 error types). Although this schema is not exhaustive, it informs both
administrative and programming solutions.
Perhaps CPOE-facilitated error risks received limited attention because
the methodologies and foci of previous studies addressed CPOE’s role
in error reduction2,3,6-11,14-16,42 and
seldom its role in error facilitation.21,26-28,31,32,45 One
key study27 examined errors but was entirely
qualitative, with no frequency estimates. Other reasons CPOE’s problems
may have escaped larger examination include the orientation of medical personnel
to solve or work around problems, beliefs that problems are due to insufficient
training or noncompliance, erratic error-reporting mechanisms, and focus on
technology rather than on work organization.30,32,42,43,52,53 Our
multimethod, triangulated approach explored wider ranges of CPOE’s effects.33,42,48,54
That CPOE use might increase the likelihood of medication errors was
an unanticipated finding, which would not have surfaced without open-ended
qualitative research. Survey data provided a different type of validation
and strengthened our confidence in the findings. Our error risk frequency
estimates are from a robust sample of house staff.
We conducted research at only 1 hospital. Although the CPOE system we
examined (TDS) has comprised as much as 60% of the market,55-57 it
is possible that several CPOE-facilitated errors discussed here may not be
widely generalizable. Also, TDS, like all complex CPOE systems, is “customized”
and undergoes repeated improvements. Our qualitative findings are not from
random house staff samples. Identified error risks may be overstated or understated.
However, our survey findings are based on an almost 90% sample of relevant
house staff and are less likely susceptible to sample bias.
House staff may have misinterpreted our questions or response categories.
Despite extensive pretests, focus groups, and poststudy interviews, the process
is hardly foolproof.
Although house staff in one-on-one interviews and focus groups discussed
actual errors, the survey data reflect house staff responses or statements
about medication error likelihood, not actual ADEs. Thus, our survey analysis
focuses on features of error-prone systems rather than errors themselves.
Also, we stress that hospital pharmacists review every order and reject about
4%; many errors existed with paper-based systems, and without direct comparative
studies we cannot contrast their relative advantages; there is no reason to
suspect that TDS is inferior to any other CPOE system; and it is badly designed
and poorly integrated CPOE systems that are at issue.
CPOE is widely regarded as the crucial technology for reducing hospital
medication errors.2,3,6-22,30,31,58,59 As
with any new technology, however, initial assessments may insufficiently consider
risks and organizational accommodations.30,32-34,42-44,46,48-52,60 The
literature on CPOE, with few exceptions,21,26-28,34,39,45 is
enthusiastic. Our findings, however, reveal that CPOE systems can facilitate
error risks in addition to reducing them. Without studies of the advantages
and disadvantages of CPOE systems, researchers are looking at only one edge
of the sword. This limitation is especially noteworthy because many problems
we identified are easily corrected.
Our recommendations concentrate on organizational factors. (1) Focus
primarily on the organization of work, not on technology; CPOE must determine
clinical actions only if they improve, or at least do not deteriorate, patient
care. (2) Aggressively examine the technology in use; problems are obscured
by workarounds, the medical problem-solving ethos, and low house staff status.
(3) Aggressively fix technology when it is shown to be counterproductive because
failure to do so engenders alienation and dangerous workarounds in addition
to persistent errors; substitution of technology for people is a misunderstanding
of both. (4) Pursue errors’ “second stories” and multiple
causations to surmount the barriers enhanced by episodic and incomplete error
reporting, which is standard, and management belief in these error reports,
which obfuscates and compounds problems. (5) Plan for continuous revisions
and quality improvement, recognizing that all changes generate new error risks.
In our work, use of multiple qualitative and survey methods identified
and quantified error risks not previously considered, offering many opportunities
for error reduction. As CPOE systems are implemented, clinicians and hospitals
must attend to the errors they cause, in addition to the errors they prevent.
Corresponding Author: Ross Koppel, PhD,
Center for Clinical Epidemiology and Biostatistics, Room 106, Blockley Hall,
School of Medicine, University of Pennsylvania, Philadelphia, PA 19104 (email@example.com).
Author Contributions: Dr Koppel had full access
to all of the data in the study and takes responsibility for the integrity
of the data and the accuracy of the data analysis.
Study concept and design: Koppel, Metlay, Localio,
Acquisition of data: Koppel, Cohen, Abaluck,
Analysis and interpretation of data: Koppel,
Cohen, Abaluck, Localio.
Drafting of the manuscript: Koppel, Cohen.
Critical revision of the manuscript for important
intellectual content: Koppel, Metlay, Cohen, Abaluck, Localio, Kimmel,
Statistical analysis: Koppel, Cohen.
Obtained funding: Koppel, Metlay, Localio,
Administrative, technical, or material support:
Koppel, Cohen, Localio, Strom.
Study supervision: Koppel, Cohen, Strom.
Financial Disclosures: None reported.
Funding/Support: This research was supported
by a grant from the Agency for Healthcare Research and Quality (AHRQ), P01
HS11530-01, Improving Patient Safety Through Reduction in Medication Errors.
Dr Metlay is also supported through an Advanced Research Career Development
Award from the Health Services Research and Development Service of the Department
of Veterans Affairs.
Role of the Sponsors: Neither AHRQ nor the
Department of Veterans Affairs had any role in the design and conduct of the
study; collection, management, analysis, and interpretation of the data; or
the approval of the manuscript.
Project Advisory Committee: Linda H. Aiken,
PhD, University of Pennsylvania; David W. Bates, MD, MSc, Harvard School of
Medicine; Shawn Becker, RN, US Pharmacopeia; Marc L. Berger, MD, Merck &
Co; Steven L. Sauter, PhD, National Institute for Occupational Safety and
Health; Thomas Snedden, PA, Department of Aging; Paul D. Stolley, MD, MPH,
University of Maryland; Joel Leon Telles, PhD, Delaware Valley Healthcare
Council of Hospital Association of Pennsylvania.
Acknowledgment: We thank Charles Leonard, PharmD;
Frank Sites, MHA, RN; Joel Telles, PhD; Edmund Weisburg, MS; and Ruthann Auten,
Create a personal account or sign in to: