Raschke RA, Gollihare B, Wunderlich TA, Guidry JR, Leibowitz AI, Peirce JC, Lemelson L, Heisler MA, Susong C. A Computer Alert System to Prevent Injury From Adverse Drug EventsDevelopment and Evaluation in a Community Teaching Hospital. JAMA. 1998;280(15):1317-1320. doi:10.1001/jama.280.15.1317
From Good Samaritan Regional Medical Center (Drs Raschke, Leibowitz, and Peirce and Mss Gollihare and Susong), Samaritan Health System (Messrs Wunderlich and Lemelson), and Desert Samaritan Medical Center (Drs Guidry and Heisler), Phoenix, Ariz.
Context.— Adverse drug events (ADEs) are the most common type of iatrogenic injury
occurring in hospitalized patients. Errors leading to ADEs are often due to
restricted availability of information at the time of physician order writing.
Objectives.— To develop, implement, and evaluate a computer alert system designed
to correct errors that might lead to ADEs and to detect ADEs before maximum
Design.— Prospective case series.
Setting.— A 650-bed community teaching hospital in Phoenix, Ariz.
Patients.— Consecutive sample of 9306 nonobstetrical adult patients admitted during
the last 6 months of 1997.
Interventions.— Thirty-seven drug-specific ADEs were targeted. Our hospital information
system was programmed to generate alerts in clinical situations with increased
risk for ADE-related injury. A clinical system was developed to ensure physician
notification of alerts.
Main Outcome Measures.— A true-positive alert was defined as one in which the physician wrote
orders consistent with the alert recommendation after alert notification.
Results.— During the 6-month study period, the alert system fired 1116 times and
596 were true-positive alerts (positive predictive value of 53%). The alerts
identified opportunities to prevent patient injury secondary to ADEs at a
rate of 64 per 1000 admissions. A total of 265 (44%) of the 596 true-positive
alerts were unrecognized by the physician prior to alert notification.
Conclusions.— Clinicians can use hospital information systems to detect opportunities
to prevent patient injury secondary to a broad range of ADEs.
ADVERSE DRUG EVENTS (ADEs) are the most common type of iatrogenic injury
occurring in hospitalized patients.1,2
Adverse drug events have been reported to occur during 1% to 30% of hospital
admissions, depending on the operational definition of ADE and the rigor with
which they are sought.2- 9
A recent meta-analysis reported an overall incidence of 6.7% for serious adverse
drug reactions (a term that excludes injury secondary to errors in prescribing
and administration).10 For every 1000 patients
admitted to a hospital, approximately 3 will die3,10,11
and 1 will suffer serious long-term disability2
due to ADEs. The mean direct cost of an inpatient ADE ranges from $1900 to
From 28% to 56% of ADEs are preventable,3,4,6,7
and these are most commonly caused by errors in order writing.3,13
Such errors occur in up to 5% of medication orders.14,15
Prescription of the wrong drug or wrong dose is often due to lack of information
regarding the drug or the patient.3,13,14
A recent study concluded that 78% of errors leading to ADEs are due to systems
failures that could be corrected by improved information systems.13
We have developed a computer alert system that provides patient-specific
information to clinicians, with the specific aim of correcting prescription
errors that might lead to ADEs (primary prevention) and detecting ADEs before
harm occurs (secondary prevention).
Good Samaritan Regional Medical Center (GSRMC) is a 650-bed teaching
hospital and regional referral center in Phoenix, Ariz. In 1994, a group convened
at GSRMC to develop a method of reducing ADE-related injury using the decision
support capabilities of our hospital information system (Discern Expert, Cerner
Corp, Kansas City, Mo). The group included physicians and representatives
from pharmacy, clinical pharmacy, nursing, laboratory, and information services.
Our hospital information system contained integrated patient-specific
data including demographics, pharmacy orders, drug allergies, radiology orders,
and laboratory results. Other clinical information such as major diagnoses
andphysicians' notes were not part of this database. Efforts focused on using
information from the integrated databases to detect situations that might
lead to ADE-related patient injury. The group devised a plan to do so through
primary prevention alerts, which detect prescription errors with high potential
for resulting in ADEs (eg, inappropriate dosing of imipenem in a patient with
renal failure), and secondary prevention alerts, which detect potential ADEs
before maximal patient injury has occurred (eg, new-onset thrombocytopenia
in a patient receiving heparin sodium).
Specific ADEs were selected for inclusion based on clinical significance
and the presence of specific risk factors for injury in our databases. Adverse
drug events resulting from drug interactions and allergies were already being
addressed at our institution through computerized decision support, and therefore
were not included.
Thirty-seven drug- or drug class–specific ADEs were targeted.
These are listed in Table 1 and
represent the most common categories of ADEs described in the Harvard Medical
Practice Study,2 with the exception of allergic
We developed and pilot tested computer programs to generate alerts for
each of the targeted ADEs. The logic statements within the programs each contained
a trigger premise (describing a clinical situation in which injury secondary
to an ADE might be imminent) and a recommendation to avoid injury (eg, if
a verified serum potassium level exceeds 6.0 mmol/L and the patient is receiving
potassium chloride, then print an alert recommending discontinuation). Table 1 includes simplified logic for each
Programs involving drug dose adjustment in renal failure use the method
described by Jelliffe16 to estimate creatinine
clearance17 and the American College of Physicians'
recommendations for appropriate drug dosing.18
Programming was performed using Cerner Rule Editor (Cerner Corp). Systems
for physician alert notification were developed.
Pharmacists evaluated each alert that printed out in the pharmacy. This
involved confirmation of the information that triggered the alert and discussion
with nurses regarding the patient's clinical condition when necessary. The
pharmacist contacted the attending physician when the alert recommendations
seemed appropriate given the clinical situation. Alerts designed to prevent
radiocontrast media nephrotoxicity were evaluated by radiology technicians
and brought to the attention of the attending radiologist when appropriate.
We collected data on consecutive alerts that fired between July 1, 1997,
and January 1, 1998. The pharmacist or radiology technician who contacted
the physician recorded (1) whether the physician had already recognized the
problem identified by the alert, (2) whether the physician made order changes
consistent with alert recommendations, (3) the reason for disagreement if
the physician did not make order changes, and (4) the time spent evaluating
A research nurse prospectively collected this information and confirmed
physician order changes by paper chart review. Each firing was classified
as a true-positive or false-positive alert based on whether the attending
physician wrote orders consistent with the alert recommendations. Systat version
5.2.1 (Systat Inc, Evanston, Ill) was used for all descriptive statistical
During the 6-month study period, there were 13521 admissions at GSRMC,
of which 4215 were labor and delivery admissions. Consistent with a published
observation that ADEs are extremely uncommon in obstetrical patients,7 there were only 7 alert firings among these patients.
The following results apply only to the 9306 nonobstetrical admissions.
The ADE alert system fired 1116 times. In 794 cases, the evaluator felt
the alert warranted physician notification. Physicians were not notified when
the adverse event was clearly not drug related (eg, thrombocytopenia secondary
to disseminated intravascular hemolysis), or when the triggering laboratory
result was misleading (eg, hyperkalemia secondary to hemolysis of blood specimen).
A total of 596 (53%) of 1116 alerts were true positives. Thus, opportunities
to potentially reduce patient injury secondary to ADEs were identified at
a rate of 64 per 1000 admissions (596/9306). Physicians stated they were previously
unaware of the potentially dangerous clinical situations leading to alert
firings in 265 (44%) of the 596 true-positive alerts. The order changes in
these patients were directly attributable to alert notification and occurred
at a rate of 29 per 1000 admissions (265/9306).
Primary prevention alerts fired 803 times, identifying 490 potential
opportunities to prevent ADEs (positive predictive value of 61%) (Table 2). Of these, 238 (49%) were unrecognized
by the clinician before alert notification. The most common prevention alert
firings were for radiocontrast media nephrotoxicity and digoxin toxicity.
Secondary prevention (early detection) alerts fired 313 times, identifying
106 cases in which the physician agreed that action was required to evaluate
or treat a possible ongoing ADE (positive predictive value of 34%) (Table 3). Twenty-seven (25%) of these were
previously unrecognized. The most common detection alert firings were for
possible pseudomembranous colitis and drug nephrotoxicity.
The most common reasons for false-positive primary prevention alerts
were (1) importance of the radiocontrast media study was felt to outweigh
the risk of nephrotoxicity (n = 81), (2) disagreement that renal drug clearance
was inadequate (n = 26), and (3) planned short-term or as-needed-only use
of medications (n = 20). For secondary prevention alerts, the most common
cause for a false-positive alert was the determination that the observed complication
was not drug related (n = 127).
Incidentally, true-positive alerts were associated with appropriate
reductions in drug dosages in 135 patients. Eighty-four of these were previously
unrecognized, and resulted in a savings of 254 drug doses (146 doses of antibiotics
and 108 doses of nonantibiotic medications). The mean time spent by pharmacy
technicians evaluating each alert was 15.9 minutes (SD, 12.8 minutes; range,
Our system detected opportunities to reduce ADE-related injury at a
rate of 64 per 1000 patient admissions. Previous measures of this rate are
not available for comparison because this is the first study to prospectively
evaluate a computer support system with real-time intervention for reducing
injury from a broad range of ADEs. However, previous noninterventional studies
have quantified the rate of opportunities to prevent ADEs. Leape and colleagues13 combined preventable ADEs and potential ADEs (medication
errors with the potential to cause ADEs) to determine the total number of
preventable events. A rate of 69 per 1000 patient admissions can be calculated
from their reported data,3,13
which is similar to the rate in our study. Others have reported preventable
event rates of 106 per 1000 admissions15 and
117 per 1000 admissions.7 Although we would
strive to develop a system to circumvent all such preventable events, our
set of alerts represents only a subset, and probably includes events that
would not be classified as preventable or potential ADEs by other researchers.
Nevertheless, several examples in which our preventive intervention
failed to illustrate the serious potential consequences of a true-positive
alert. In one instance, an alert identified an elderly woman with renal insufficiency
and hyperkalemia who was receiving potassium chloride and quinapril. Use of
the medications was discontinued on alert notification; however, the patient
suffered a fatal cardiac arrest less than 1 hour later with a serum potassium
level of 7.0 mmol/L. Another patient, identified by a pilot alert, was receiving
metformin and had a serum creatinine level of more than 350 µmol/L.
Within 24 hours, the patient developed fatal lactic acidosis.
Cost considerations are important in determining the generalizability
of our approach. In a 1993 survey of 166 hospitals, 83% reported the ability
to identify patients based on medications received, but only 30% could integrate
this information with laboratory data (a prerequisite for an alert system
such as ours).19 An integrated hospital information
system with decision-support capability may cost several hundred thousand
to several million dollars, depending on the size of the institution (Steve
Hawthorne, Cerner Corp, written communication, September 4, 1998). Our working
group spent approximately 400 person-hours developing the specific system
described in this article, but the entire process need not be duplicated at
every institution implementing such a system. The overall positive predictive
value (53%) and the average time spent evaluating an alert (15.9 minutes)
suggest that the average incremental cost of each true-positive alert is approximately
30 minutes of pharmacist or radiology technician work time. Given the rate
of alert firings, this amounts to approximately one fourth of a full-time
equivalent at our institution.
The benefit of an highly effective ADE prevention program can be estimated
for a hypothetical 650-bed hospital. If ADEs occur in approximately 7% of
admissions,3,10 1800 would be
expected annually. A conservative estimate is that 28% of ADEs, in general,
and 42% of life-threatening ADEs are preventable.3,6
Therefore, a fully functional system might avert 500 ADEs and save 36 lives
per year. The average preventable ADE adds $5857 to the cost of hospitalization,6 therefore cost savings as high as $3 million annually
might be achieved. Prevention of ADEs should also reduce indirect costs associated
with disability and medical-legal liability.
Previous studies have demonstrated the utility of computer systems in
Researchers from LDS Hospital in Salt Lake City, Utah, have developed a computerized
ADE monitor that detected 80 times more ADEs than conventional self-reporting
methods.8 The most common method by which this
system identified ADEs was the administration of antidotes (such as naloxone).
Detection of this type of ADE would not qualify for our definition of secondary
prevention, since the ADE in question has already been recognized and treated.
A subsequent study showed that reporting of computer-detected ADEs to
physicians resulted in a 65% reduction of severe ADEs compared with historic
controls.22 These impressive results were achieved
with a focus on allergic and idiosyncratic ADEs. In contrast, we chose to
focus on nonallergic ADEs in which real-time intervention might benefit the
patient. Studies in which computer-assisted antibiotic dosing has been shown
to decrease antibiotic-related ADEs23,24
exemplify this approach.
We felt it unethical to design our study with a concurrent, nonintervention
control group. Reliable historical controls were not available because the
established method of ADE detection at our hospital (self-report) is highly
Therefore this study did not directly measure a reduction in ADE-related injury.
Instead, our study relied on changes in physician behavior as the main outcome
variable, a common limitation of published research regarding computer-based
clinical decision support.27
Caution is warranted when interpretating our aggregate data. Adverse
drug event alerts are quite heterogeneous. Some detect rare but immediately
life-threatening ADEs (eg, metformin-induced lactic acidosis), and others
detect common situations with a lower potential to result in injury (eg, hypokalemia
in a patient receiving digoxin). Classen and colleagues4
have shown tremendous variation in the clinical and economic impact of various
types of ADEs.
Several of our ADE alerts appear to have low sensitivity. Cognitive
impairment is a common and important ADE in the elderly,28,29
but our delirium alert only identified a few cases. This alert uses pharmacy
orders for haloperidol in elderly patients to identify delirium. We are developing
methods to enter clinical data (such as alterations in mental status) into
our computer database to improve the ability of our system to detect important
clinical outcomes that are not well represented in our current databases.
We are also attempting to reduce false-positive alerts through refinement
of alert-trigger logic. Excluding chemotherapy patients from thrombocytopenia
alerts is an example of how this can be accomplished. Other systems improvements
we are implementing include alerts to detect drug-induced pancreatitis and
vancomycin administration in patients with methicillin-sensitive Staphylococcus aureus infections.
Computer alert systems can be used to identify opportunities to prevent
or reduce patient injury associated with a broad range of ADEs. Prerequisites
for a computer ADE alert system such as the one described herein include (1)
an integrated computerized database (including clinical, pharmacy, and laboratory
data), (2) the ability to program the system to generate alerts when opportunities
to prevent injury occur, and (3) reliable clinical systems for physician notification.
Opportunity exists for greatly increasing the scope of computer-assisted
decision making in clinical practice. Computer-aided diagnosis, preventive
care reminders, and computer-aided quality assurance are examples of computer-based
clinical decision support systems that have improved quality.30
Computer systems with online physician order entry would enable decision-support
systems to provide potentially critical information to the physician close
to the moment of decision making.31- 33
Improvements in hospital information systems and increasing utilization of
this powerful tool by physicians should have an enormous beneficial impact
on the quality of medical care.