High Rates of Adverse Drug Events in a Highly Computerized Hospital

Background  Numerous studies have shown that specific computerized interventions may reduce medication errors, but few have examined adverse drug events (ADEs) across all stages of the computerized medication process. We describe the frequency and type of inpatient ADEs that occurred following the adoption of multiple computerized medication ordering and administration systems, including computerized physician order entry (CPOE).

Methods  Using explicit standardized criteria, pharmacists classified inpatient ADEs from prospective daily reviews of electronic medical records from a random sample of all admissions during a 20-week period at a Veterans Administration hospital. We analyzed ADEs that necessitated a changed treatment plan.

Results  Among 937 hospital admissions, 483 clinically significant inpatient ADEs were identified, accounting for 52 ADEs per 100 admissions and an incidence density of 70 ADEs per 1000 patient-days. One quarter of the hospitalizations had at least 1 ADE. Of all ADEs, 9% resulted in serious harm, 22% in additional monitoring and interventions, 32% in interventions alone, and 11% in monitoring alone; 27% should have resulted in additional interventions or monitoring. Medication errors contributed to 27% of these ADEs. Errors associated with ADEs occurred in the following stages: 61% ordering, 25% monitoring, 13% administration, 1% dispensing, and 0% transcription. The medical record reflected recognition of 76% of the ADEs.

Conclusions  High rates of ADEs may continue to occur after implementation of CPOE and related computerized medication systems that lack decision support for drug selection, dosing, and monitoring.

Multiple broad-based studies during the past 15 years have demonstrated that adverse drug events (ADEs) account for up to 41%¹ of all hospital admissions and more than $2 billion annually in inpatient costs.^2-4 Several of these studies have also estimated that as many as a quarter of inpatient ADEs may be preventable through interventions such as computerized physician order entry (CPOE) and related systems.^5-7 On the basis of these projections and the proven success of these systems in identifying ADEs and reducing medication errors,^8-11 computerized medication processes have been widely promoted as essential to preventing actual ADEs.^9,12,13

Recently, some researchers have questioned the extent to which currently available CPOE and related systems are preventing ADEs.^14-16 There are concerns that features of commercial CPOE products vary widely and that few can match the sophistication of custom systems developed at institutions that have successfully reduced targeted ADEs.^15,17-21 Moreover, broad-based surveys of ADEs in institutions that have implemented multiple computerized medication systems have not been published; it is unclear how these interventions together have affected the occurrence of ADEs linked to problems across stages of medication processing (ie, ordering, transcription, dispensing, administration, and monitoring).⁵

The Veterans Administration (VA) Healthcare System, one of the largest integrated delivery systems in the country, is a leader in patient safety and has actively sought to reduce medication errors using multiple computerized interventions such as CPOE,^22-26 bar code–controlled medication delivery,^9,27,28 a complete electronic medical record,^1,29-31 automated drug-drug interaction checking,^32-35 and computerized allergy tracking and alerting.^36-38 The White House has recently praised the advanced technologies of the VA computer system and suggested that it could be widely distributed to private medical practices.³⁹ Hospitals in several European and African countries already use the VA computer system.⁴⁰ Both the broad implementation of computer-assisted medication processes and the potential national significance of the VA’s computer systems make the VA an important setting for a new survey of ADEs.

The VA Medical Center in Salt Lake City is a 110-bed, tertiary-care teaching facility. At the time of the study, the VA computerized medical record included all orders, results, medications, and notes. It did not include images and flow sheets used by nurses and anesthesiologists. Computerized order checking was fully functional for allergies, many drug-drug interactions, and limited drug-disease interactions; it did not feature sophisticated decision support algorithms. In particular, it did not offer drug selection, dosing, or monitoring advice. Bar code medication administration, a method by which patients’ identification bands and prepackaged drugs are scanned prior to administration, was active in most units, and 81% of study patient–days were fully covered by this technology. As in other VA medical centers, this center had implemented additional programs that have been demonstrated to reduce ADEs, such as full-time patient safety coordinators,^34,38 unit dosing,³⁴ clinical pharmacists who perform rounds with the medical team,^22,34,41,42 and care by resident physicians supervised by faculty.⁴³

During a 20-week period (August 13, 2000, through December 31, 2000), newly admitted patients to all wards (medical, surgical, intensive care, rehabilitation, and psychiatric) were randomly assigned to prospective case review by 1 of 2 full-time clinical pharmacists. Every other day, the pharmacists reviewed all new notes, orders, laboratory test results, and discharge summaries available in the electronic medical record and examined other sections of the record as needed.

The Food and Drug Administration (FDA), World Health Organization, and International Committee on Harmonisation define an adverse drug reaction as “a response to a drug which is noxious and unintended and occurs at doses used in man for prophylaxis, diagnosis, therapy, or modification of physiologic functions.”^44(p1255) We chose to study ADEs, defined as “injury resulting from the use of a drug.”^5,24(p88),45 Specifically for this study, ADEs included all traditional adverse drug reactions plus harm from overdoses, harm from inappropriate dose reductions or discontinuations, and intolerable harm from dose titration.⁴⁶ Self-limited harm and harm from titration, such as extreme glucose and electrolyte abnormalities, that did not result in a serious event according to the FDA or in drug discontinuation were excluded. The harm must have started in the hospital and have been probably or certainly⁴⁴ caused by drug therapy. All harm reflected by laboratory findings was also defined by explicit criteria and based on training material from a prior study in Boston, Mass,⁵ with the addition of electrolyte and glucose abnormalities. Deaths were not attributed to an ADE if the patient had an underlying medical condition independent of the ADE that significantly contributed to the death.

Classifications were formulated from the literature with an emphasis on national or international standards for pharmacological typology,⁴⁴ causality assessment,⁴⁴ error type,⁴⁷ event terminology,⁴⁸ drug class,⁴⁹ seriousness index,⁵⁰ and medication error category index.⁴⁷ The set of error types was expanded to include failure to provide prophylaxis against common drug reactions, such as not prescribing potassium with a higher-dose loop diuretic. Errors were graded on a 4-point confidence scale, and only those judged probable or certain were included in the analysis. For the error subanalysis, we included only errors associated with ADEs.

Other classifications not commonly found in the literature were also used. Additional resource utilization in response to ADEs was scored as a sole, sufficient, or necessary cause of additional monitoring and interventions. Additional monitoring was scored when the occurrence of an ADE resulted in orders for laboratory tests, additional nursing time, follow-up appointments, diagnostic procedures, prolongation of hospitalization, or transfer to a higher level of care. Additional interventions were scored when an ADE required treatment, such as drugs for counteracting an effect, therapeutic procedures, or other treatments, but not when the only intervention was to reduce or discontinue offending drugs or to change to a new class of drug. To distinguish among a large category of moderately serious events, moderately serious outcomes were classified according to whether the patient had to undergo additional tests and monitoring. For an ADE to be included for analysis, it either must have resulted in a serious outcome, monitoring, interventions, or adjustments to the offending drug beyond routine titration or should have resulted in monitoring or interventions. Consensus was reached on these classification criteria at weekly confirmation committee meetings attended by 2 physicians, 2 clinical pharmacists, a nurse-researcher, and the project coordinator.

Finally, health care professional documentation and recognition of ADEs were evaluated. Recognition of an ADE was judged to have occurred when symptoms or signs were documented or an order was written to treat the ADE. The ADEs identified through laboratory results and other findings required a reference in the progress notes (eg, “low potassium”); merely copying the laboratory value was not judged to be sufficient evidence for recognizing harm. A documented association between the manifestation and the drug in the progress notes by any health care professional was scored as recognition of an association or link between the event and the ADE.

All statistics were calculated using Stata SE version 8.2 statistical software (College Station, Tex). Categorical variables were analyzed with the χ² test or Fisher exact test as appropriate, and adjustments for multiple comparisons were made using the Holm method. Agreement between pharmacists for the presence of an ADE was calculated in a sample of patients toward the end of the study using the κ statistic.

Of 2306 admissions, 937 (41%) were randomly selected for review, accounting for 6856 patient-days. Overall, 483 clinically significant ADEs were identified, corresponding to an incidence density of 70 ADEs per 1000 patient-days or 52 ADEs per 100 admissions. Two-hundred forty-one (26%) of the reviewed admissions had at least 1 ADE, among which were identified a mean of 3.2 ADEs, a median of 2 ADEs, and a range of 1 to 20 ADEs. Using World Health Organization criteria, 294 events (60%) were categorized as certain ADEs and 189 (39%) as probable ADEs. For the ADE to be clinically significant, it required a change in the treatment plan; 337 (67%) of the ADEs had changes for which the ADE was judged to be the sole cause. Interrater agreement for the presence of an ADE in an admission was 100% (κ = 1.00; P < .001).

Adverse drug reactions accounted for 448 events (93%), whereas the remaining 35 ADEs (7%) resulted from overdosing or underdosing of medications. Adverse drug reactions were composed of 2 standard pharmacological types: 402 (90%) dose-dependent events and 46 (10%) idiosyncratic events (43 nonallergic and 3 allergic). Of the dose-dependent adverse drug reactions, 14 (3%) occurred at doses near the lower end of recommended dose ranges for the offending drug.

The ADEs commonly occurred from various drug interactions. Additive drug-drug interactions, in which the physiologic effects of 2 or more drugs were similar, accounted for 189 (39%) of the ADEs (eg, cardiovascular-diuretic combinations caused 10 episodes of hypotension). Drug-condition interactions accounted for 142 ADEs (29%) (eg, renal failure increasing the drug’s effect or congestive heart failure exacerbated by a β-blocker). Only 3 ADEs (<1%) were attributed to drug-age interactions and 4 (<1%) to drug-drug metabolic reactions, in which one drug increased the level of another.

Using the LDS Hospital classification scale,⁵⁰ 438 ADEs (91%) were scored as moderate and 45 (9%) as serious. Further classifying moderate events by actual treatments for the ADE, 105 (22%) required interventions and monitoring procedures, 154 (32%) required only interventions without additional monitoring, 51 (11%) required monitoring without additional interventions, and 128 (27%) required only discontinuation or adjusting the dose of the offending drug. Using National Coordinating Council for Medication Error Reporting and Prevention indexing,⁴⁷ 421 ADEs (87%) required treatment (category E), 22 (4%) required prolonged hospitalization (category F), 1 (<1%) resulted in permanent harm (category G), 16 (3%) resulted in a near-death experience (category H), and 6 (1%) were fatal (category I). Fatal reactions included narcotics leading to apnea (2 ADEs), narcotics leading to stupor, vomiting, and aspiration (1 ADE), toxic blood levels of lidocaine leading to progressive bradycardia and asystole (1 ADE), and combinations of nonsteroidal anti-inflammatory drugs and heparin analogues leading to upper gastrointestinal bleeding and either aspiration (1 ADE) or myocardial infarction (1 ADE).

A total of 172 errors contributed to 129 ADEs (27%). There was an average of 1.4 errors for ADEs with at least 1 error. Thirty-five ADEs (27%) were categorized as execution errors, and 107 (83%) were categorized as planning errors.³⁴ The most common error types were failure to provide prophylaxis for expected adverse drug reactions (36%), failure to start or complete adequate monitoring for common adverse drug reactions (33%), and prescription of improper doses (33%) or inappropriate medications (7%). With respect to prophylaxis errors, the most common were failure to prescribe an as-needed or routine bowel regimen for narcotics (10% of errors) and failure to prescribe potassium with diuretics when the serum potassium level was in a low-normal range (8% of errors). Errors occurred at the following stages of care⁵: 61% ordering, 0% transcription, 1% dispensing, 13% administration, and 25% monitoring.

The most common drug classes associated with ADEs (Table 1), either alone or in conjunction with other agents, were narcotic analgesics (26%), diuretics (18%), and cardiovascular or renal agents (17%). These 3 classes also had high error rates, but the difference in rates compared with other drug classes did not reach statistical significance after adjusting for multiple comparisons. Reactions to narcotic analgesics and diuretics required an intervention, such as a procedure or a counteracting drug, significantly more often than reactions to other classes. The 3 most common event syndromes (Table 2) were constipation (14%), hypokalemia (10%), and hypotension (10%). Significantly higher percentages of these ADEs led to additional interventions. Error rates were significantly higher for constipation (55%; P < .001) and hypokalemia (47%; P = .04). The proportion of ADEs across the Medical Dictionary for Regulatory Activities⁴⁸ system organ classes was 24% gastrointestinal, 21% metabolism and nutrition, 14% vascular (including bleeding), 7% nervous system, 6% respiratory, 6% psychiatric, 5% cardiac, 4% infections, and 2% renal and urinary.

Documentation by a clinical professional of an accurate association between the ADE and a candidate drug was identified for 61% of the ADEs. Of the notes documenting specific ADEs in association with a drug, 18% included causality assessment statements. Only 1% of ADEs were described in the adverse-reaction section of the medical record. No indication of recognition of the ADE was found in any note or order for 24% of ADEs.

High rates of clinically significant ADEs, ordering errors, and unrecognized ADEs were noted at a VA hospital that had adopted a wide range of computer technologies and personnel strategies designed to improve medication safety. In interpreting our findings, several factors should be considered.

The incidence density of 6.6 serious and 0.9 fatal ADEs per 1000 patient-days highlights the frequency with which serious iatrogenic injuries can result during inpatient care. Annually, our 110-bed VA hospital is expected to experience 40 deaths from ADEs. The overall incidence density of 70 ADEs per 1000 patient-days is 5⁵¹ to 19⁵⁰ times higher than that reported in prior studies, whereas the incidence density of serious ADEs is many times higher than that reported previously,^5,50,51 with rare exceptions.⁴ Differences in definitions and specific criteria were unlikely to permit excessive case inflation, since more than 93% of our ADEs met the relatively restrictive FDA definition of an adverse drug reaction, and mild reactions were excluded. Notwithstanding, this study does not provide evidence that the true rate of ADEs is higher than in other hospitals previously studied. It is likely that the case finding in this study was facilitated by legible and accessible electronic data, iterative case review, and use of clinical pharmacists, who typically find higher rates of ADEs.^41,51,52

This hospital’s rates of error at various stages of medication use differed from those reported at a Boston hospital before the implementation of CPOE.⁵ The proportion of errors at the ordering stage was higher in Salt Lake City (74% vs 56%), whereas the proportions of errors at the transcription and administration stages were lower (0% vs 6% and 11% vs 24%, respectively). It appears from the shift in distributions of error-associated ADEs that the VA computerized interventions worked to reduce error-related ADEs almost exactly as designed. For example, lack of decision support for drug selection, dosage, and monitoring permitted high rates of errors in these stages. On the other hand, CPOE appears to have virtually eliminated ADEs associated with transcription errors, and bar code medication administration, although not shown in studies to prevent ADEs,¹⁵ may have resulted in relatively fewer ADEs associated with administration errors at the study site.

As reported in a previous Salt Lake City study,⁵⁰ recognition was lacking and documentation of ADEs was rare. In our study, only 1% of all ADEs were documented in the allergy or adverse drug reaction section of the record. Medical residents at our site frequently reported that they assumed that documentation was appropriate only for hypersensitivity reactions. Although the patient care team may not have recognized one quarter of the ADEs, we found evidence in this study that the treating team recognized more ADEs per patient-day than were detected under the scrutiny of research teams in most other studies of ADEs. Such high clinical recognition may have been facilitated by the easy access to legible patient data provided by the VA’s computerized record system.

The limitations of our study should be acknowledged. First, capabilities and implementations of various CPOE systems differ markedly.^13,15 However, the VA health care system uses only 1 electronic medical record platform, and the embedded ADE safeguards have changed little since this study was completed. Second, although case-based assignments of causality, error, and harm are subject to bias, the use of standardized causation criteria and generally conservative scoring of error and harm was designed to minimize this bias.^15,53 Third, the lack of a control group precludes strong inferences about how computerized systems reduced or increased rates of ADEs. An unbiased quasi-experimental pre-post design would be impractical because computerized systems for medication management and note documentation were deployed over several years. Finally, this study did not examine potential ADEs; that is, medication errors that did not cause harm. However, because current computerized medication systems have been shown to reduce potential ADEs or medication errors¹³ but have inconsistently reduced actual ADEs^15,22 and because three quarters of ADEs are not caused by error, a focus on assessing harm is warranted.¹⁵

The high rate of ADEs and the seriousness of the ADEs in this study provide empirical support for improvements in computerized interventions. The CPOE systems must address dosing, prophylaxis, and monitoring errors through such approaches as standard order sets or automated suggestions for prophylaxis and monitoring strategies.⁸ For example, at the time a physician enters an order for a loop diuretic, CPOE should suggest an order for a potassium supplement and orders for monitoring serum creatinine and potassium levels. Bar code medication administration data on dosage and effectiveness of as-needed medications should feed into computerized algorithms for medication titration, such as insulin for diabetes or narcotics for pain control. Drug order checking could be revised to de-emphasize rare drug-drug interactions and emphasize common, additive drug-drug interactions.⁵⁴ Finally, ADE documentation could be improved through computerized documentation systems that facilitate recording and displaying causality, seriousness, or dosage information, which would improve the usefulness of ADE reports to regulatory agencies and alerts at the time of drug ordering.⁴⁵ For example, because 93% of ADEs were dose related, the inability to integrate individualized, “unsafe” dosage ranges into order checking is a missed opportunity for patient safety efforts.

In conclusion, our study found high rates of clinically important ADEs related to problems in drug selection, dosage, and monitoring in a VA medical center after the adoption of computerized systems that offered minimal decision support for these specific aspects of the medication process. Because the VA’s computerized patient record system likely makes ADEs more visible than would a paper-based system, this study does not support the interpretation that the VA computerized patient record system induces ADEs. However, our findings do imply that purchasers of CPOE systems should not rely on generic CPOE and bar code medication administration systems alone to dramatically reduce ADE rates. Rather, health care organizations desirous of preventing ADEs should consider whether candidate computerized medication systems offer decision support functions that address the most troublesome aspects of the medication administration process.

Correspondence: Jonathan R. Nebeker, MS, MD, GRECC (182), 500 Foothill Dr, Salt Lake City, UT 84148 (Jonathan.Nebeker@hsc.utah.edu).

Accepted for Publication: February 7, 2005.

Financial Disclosure: None.

Funding/Support: This study was supported by grant SAF 98-122-2 from the Veterans Administration Health Services Research and Development (HSR&D) Service (Washington, DC) (principal investigator, Dr Hurdle, and coprincipal investigator, Dr Nebeker). The study received additional support from VA HSR&D grants TRP 02-147 and RCD 02-176; the Geriatric Research, Education, and Clinical Center and the Salt Lake Informatics, Decision Enhancement, and Surveillance Center (VA Salt Lake City Health Care System, Salt Lake City); and the VA Cooperative Studies Program (Albuquerque, NM).

Additional Information: This study was conducted with the approval of the institutional review boards for the University of Utah and the Salt Lake City VA Health Care System (IRB 7389-99).

Acknowledgment: We thank Matthew H. Samore, MD, Beverly Roth, PharmD, Michelle Felgar, MA, Scott Capps, BS, and Brett South, MA, for critical review, data collection, administrative support, and data management.