Study design and participant flow. EMR indicates electronic medical record.
Sample warfarin-interacting medication alert.
Time series of warfarin-interacting medication prescription rates by month (warfarin-interacting medication prescriptions per 10 000 warfarin users per month). A, Interacting medications (combined); B, warfarin-acetaminophen. Fitted trend lines show predicted values from the segmented regression analysis. Transition period is the intervention implementation period.
Feldstein AC, Smith DH, Perrin N, Yang X, Simon SR, Krall M, Sittig DF, Ditmer D, Platt R, Soumerai SB. Reducing Warfarin Medication InteractionsAn Interrupted Time Series Evaluation. Arch Intern Med. 2006;166(9):1009-1015. doi:10.1001/archinte.166.9.1009
Computerized decision support reduces medication errors in inpatients, but limited evidence supports its effectiveness in reducing the coprescribing of interacting medications, especially in the outpatient setting. The usefulness of academic detailing to enhance the effectiveness of medication interaction alerts also is uncertain.
This study used an interrupted time series design. In a health maintenance organization with an electronic medical record, we evaluated the effectiveness of electronic medical record alerts and group academic detailing to reduce the coprescribing of warfarin and interacting medications. Participants were 239 primary care providers at 15 primary care clinics and 9910 patients taking warfarin. All 15 clinics received electronic medical record alerts for the coprescription of warfarin and 5 interacting medications: acetaminophen, nonsteroidal anti-inflammatory medications, fluconazole, metronidazole, and sulfamethoxazole. Seven clinics were randomly assigned to receive group academic detailing. The primary outcome, the interacting prescription rate (ie, the number of coprescriptions of warfarin-interacting medications per 10 000 warfarin users per month), was analyzed with segmented regression models, controlling for preintervention trends.
At baseline, nearly a third of patients had an interacting prescription. Coinciding with the alerts, there was an immediate and continued reduction in the warfarin-interacting medication prescription rate (from 3294.0 to 2804.2), resulting in a 14.9% relative reduction (95% confidence interval, −19.5 to −10.2) at 12 months. Group academic detailing did not enhance alert effectiveness.
This study, using a strong and quasi-experimental design in ambulatory care, found that medication interaction alerts modestly reduced the frequency of coprescribing of interacting medications. Additional efforts will be required to further reduce rates of inappropriate prescribing of warfarin with interacting drugs.
Coprescribing interacting medications can result in serious patient consequences, particularly in patients who receive warfarin, because of warfarin's narrow therapeutic index.1 Medication interactions that accentuate the anticoagulant effect of warfarin increase patient risk for intracerebral hemorrhage, other bleeding episodes, and death.1 Over 1 million warfarin prescriptions are dispensed annually in the United States, ranking it among the top 15 prescription drugs by volume, and its use is expected to increase as the population ages.2 The coprescription of warfarin and drugs associated with significant interactions is common—during a 1-year period, more than two thirds of patients receiving warfarin in a large pharmacy benefits manager database received at least 1 interacting medication that increased bleeding risk.3
Decision support through alerts and reminders at the time of prescribing has been shown to be an effective method for reducing medication errors for inpatients.4 Limited sample size has hindered an evaluation of the effect of decision support on the coprescribing of interacting medications, however, and even less is known about the effects of computerized decision support to improve medication safety in the outpatient setting. Whether clinician education, such as academic detailing,5 can improve the effectiveness of medication interaction alert systems is unknown. Academic detailing has been shown to be a successful intervention for improving many types of prescribing.5
In a health maintenance organization (HMO), using an electronic medical record (EMR) with computerized order entry and decision support for medications, we evaluated the effect of EMR alerts for selected coprescriptions of medications that interact with warfarin and compared coprescribing rates and trends before and after the alerts. This study also experimentally evaluated the effect of group academic detailing on response to the alert recommendations.
The protocol for this study was approved by the institutional review board within the study HMO. Clinicians provided informed consent for structured interviews and usability testing of draft EMR alerts during intervention development. Health maintenance organization members provide consent on enrollment to use their medical records for research.
The study was conducted in a nonprofit group model HMO with 15 primary care clinics and about 450 000 members. Nearly all the HMO patients taking warfarin have a prescription drug benefit, and most have been enrolled in the HMO's anticoagulation clinic to manage warfarin dosing using international normalized ratio (INR) monitoring. The remainder of the care for patients taking warfarin takes place in the standard inpatient and outpatient settings.
Electronic databases include patient and health care provider demographics and pharmacy dispensing data linked through the health record number that each member receives on enrollment in the HMO. The databases capture over 95% of all medical care and pharmacy services members receive.6
This group-randomized trial compared interventions to reduce the coprescribing of warfarin and interacting medications. The unit of randomization was the primary care clinic; the unit of intervention was the primary care provider; and the unit of analysis was time (study month). The primary outcome was the “interacting prescription rate,” defined as the number of coprescriptions of warfarin-interacting medications per 10 000 warfarin users per month. The effect of the interventions was evaluated using an interrupted time series design, analyzed with segmented regression models that control for preintervention trends.7
Figure 1 shows the study design and participant flow during the trial. The 15 primary care clinics were block randomized by the study statistician according to their baseline prescribing error rates generated from data from a prior study8 using a binary random number generator. All clinics received the EMR alerting interventions as intended. Of the 113 clinicians eligible to receive academic detailing, 96 (85%) received it. Data from patients who received any dispensing of warfarin during the study period (n = 9910) were included in the calculation of the interacting prescription rate for the analyses.
The intervention development and details have been described elsewhere.9,10 We reviewed the study HMO and published data regarding the frequency and severity of warfarin medication interactions.2,11 Based on these data, an expert subgroup of the formulary committee of the HMO selected the medication interactions to target. Table 1 gives the 5 targeted interacting medications or medication classes.
We conducted structured interviews with 20 prescribers to elicit clinician pREFERENCES about proposed alerts and academic detailing.9 We conducted alert discount usability testing12 with 5 clinicians from one clinic assigned to the intervention. In response to the findings, all the medication interaction alerts were clearly identified as safety alerts, included a short description of the clinical issue or risk, and recommended medication alternatives. When warfarin and a targeted interacting medication were coprescribed, an alert would appear, whereupon the clinician could click “OK” and prescribe the interacting medication or change the medication. Or the clinician could type “alt (medication name)” to select the diagnosis that necessitated the treatment and then select from among several specific treatment, dose, and schedule options that appeared in the order entry field. Figure 2 presents a sample warfarin-acetaminophen medication interaction alert. Clinicians enter orders and receive alerts only for prescription medications. The warfarin-acetaminophen alert provides alternatives for the most common scenario—prescribing a narcotic combined with acetaminophen.
The academic detailing program was tailored for delivery to small groups of clinicians and included the key aspects of academic detailing.13 We hypothesized that academic detailing would increase the likelihood of prescribers following the advice in the EMR medication interaction alerts. Two internal medicine physicians who were well known and respected within the HMO delivered the 40-minute educational sessions. The sessions addressed identified barriers to the use of alerts (eg, clinician tendency to click though the alerts) using interactive discussion; referenced credible information sources; and provided materials describing the clinical risks, alerts, and tools for later reference.13 A second copy was mailed to clinicians 16 weeks after the educational sessions.
January 2000 through November 2002 served as the preintervention (baseline) period. From December 2002 to March 2003, the warfarin-interacting medication alerts were implemented and the academic detailing was delivered. The alerts remained active during the postintervention period of April 2003 through August 2004.
The study analyst assessing the outcomes was blinded to treatment group assignment. The primary outcome was the “interacting prescription rate” or number of coprescriptions of warfarin-interacting medications per 10 000 warfarin users per month. A warfarin user was defined as any patient who had a supply of warfarin from a prescription written by a study primary care provider. Coprescription was defined as any overlap in the days supplied of medication for warfarin and the targeted medications.
To examine the comparability of the 2 study groups at baseline, we compared characteristics of clinicians and patients using unpaired, 2-tailed t tests for continuous variables and χ2 tests for categorical variables.
Given the preplanned use of all of the available primary care clinics and primary care providers and the availability of many years of prescription data relevant to patients taking warfarin, we did not preselect a specific number of time points, prescriptions, or patients. We determined that the number of available time points would exceed the range of 12 points before and after an intervention used in similar segmented regression analyses and that it was adequate to detect modest effects.7,14
The effect of the EMR alerts with and without academic detailing was estimated using segmented regression models, controlling for preintervention trends.7,14 Calendar month served as the unit of analysis, with the primary outcome measure being the interaction event rate. There were 56 monthly intervals: 34 prior to the intervention, 4 when the interventions were delivered (the transition period), and 18 after the intervention. The 4 transition months were not included in the analyses. We used intention-to-treat principles, so error rates were calculated using all primary care clinicians and their patients that were prescribed warfarin regardless of clinician participation in the interventions.
The models included a constant term, a term for baseline linear trend, a group variable to indicate study group, and terms to estimate change in level and change in trend and interaction terms (group by baseline trend, group by change in level, and group by change in trend) to test the effect of group academic detailing.14 We used the main effect models to estimate the absolute and relative differences in warfarin-interacting prescription rates 12 months after the intervention. Analyses were conducted for each of the interacting drugs alone and the 5 targeted drugs combined using SAS Proc Autoreg (SAS Institute, Cary, NC) for time series models.
Among interaction events, we used t tests to compare the mean number of days of prescription overlap with warfarin for each interacting medication in the 12-month preintervention and postintervention periods.
The baseline characteristics by study group of the participating primary care clinics, clinicians, and patients are presented in Table 2. In the 12 months prior to the intervention, the mean (monthly) interacting prescription rate was 2812 in the alerts plus academic detailing group and 3259 in the alerts-only group. Warfarin-acetaminophen coprescriptions accounted for approximately 80% of the events. Other group characteristics were similar.
In the segmented regression models, the interaction representing group differences in baseline trend (P = .22) and level change after the alerts were implemented (P = .87) was not significant. The groups were different in slope after the alerts were implemented but not in the anticipated direction, with the control group showing a sharper decline compared with the academic detailing group (P = .002). The group variable and associated interaction terms were dropped from the remainder of the analyses because academic detailing did not have the hypothesized effect, and the effect of the alerts was tested across both groups.
The effect of the alerts on interacting prescription rates is displayed in Figure 3. When baseline trends were controlled for, the overall interacting prescription rate decreased immediately after the alerts were implemented, with an estimated reduction of 329.7 interacting prescriptions per 10 000 warfarin users in the first month (P = .002). The alerts also significantly changed the trend in the interacting prescription rate, with a preintervention increasing rate of 1.1 and a postintervention decreasing rate of 21.3 (slope change, −22.4; P = .01).
The main contributor to the effect of the alerts was the effect of the warfarin-acetaminophen alert (Figure 3B). This alert alone resulted in an estimated reduction of 311 warfarin-acetaminophen interacting prescriptions per 10 000 warfarin users in the first month (P = .002) and a slope change of −22.2 compared with preintervention trends (P = .009). There was also a significant level change in the warfarin-fluconazole interacting prescription rate at the time of the intervention (17.8 fewer events per 10 000 warfarin users in the first month; P = .04) but a nonsignificant slope change. The models revealed nonsignificant level changes for warfarin–nonsteroidal anti-inflammatory medications (−28.9; P = .60), warfarin-sulfamethoxazole (−4.94; P = .97), and warfarin-metronidazole (0.86; P = .94) interacting prescription rates at the time of the intervention, as well as nonsignificant slope changes for all 3 interacting medications.
Without the alert, the total interacting prescription rate at 12 months after alert implementation was estimated to be 3294.0. With the alert, the estimated total interacting prescription rate was 2804.2, representing an absolute reduction of 489.8 events per month (95% confidence interval [CI], −664.9 to −314.7) and a 14.9% relative reduction (95% CI, −19.5% to −10.2%). For the warfarin-acetaminophen coprescription, the comparable interacting prescription rates were 2711.0 and 2207.6, representing an absolute reduction of 503.4 interacting prescriptions per month (95% CI, −670.0 to −336.8) and an 18.6% relative reduction (95% CI, −23.8% to −13.4%).
Among warfarin-interacting medication prescriptions, the mean prescription overlap ranged from 4.5 days for fluconazole to 23.9 days for nonsteroidal anti-inflammatory drugs; there were no significant differences in the length of the overlap periods before and after the interventions.
We found that coprescribing warfarin with interacting medications was common in the outpatient setting. More than a third of patients receiving warfarin were coprescribed 1 of the 5 study medications, and warfarin-acetaminophen coprescription accounted for the majority of the interacting prescriptions. These findings are consistent with those from a retrospective review of a large pharmacy benefits manager database, in which 64.8% of patients taking warfarin received a concurrent prescription for at least 1 potentially interacting drug, and acetaminophen-containing products accounted for 22.7% of these.3
After adjusting for baseline trends, implementation of the EMR alerts was associated with a significant immediate and continuing reduction in the interacting prescription rate, leading to an estimated 15% relative reduction in the overall rate at 12 months. Our findings are consistent with prior research from the inpatient setting, in which computerized physician order entry with decision support substantially reduced medication errors overall.4 However, in the prior analyses, the small number of interacting prescriptions did not allow for the separate evaluation of the effect of decision support on those events. A prior study of a drug interaction alert system in the outpatient setting did not control for preintervention trends.15
Our findings may be important when considering prior work that supports the potential severity of warfarin interactions. Drug interactions are the most common factor associated with a critically high INR16 and an increased risk of bleeding.17 Acetaminophen has been reported by Hylek et al11 to be an unrecognized hazard for warfarin takers—as little as 1300 mg/d for 7 days increased by 10-fold the odds of an INR greater than 6. However, it is important to note that the clinical significance of the acetaminophen-warfarin interaction is not without controversy. The mechanism of this interaction has only recently been elucidated18 and likely results from the independent inhibitory effect of an acetaminophen metabolite on enzymes of the vitamin K cycle. Although several case reports and controlled studies have reported that acetaminophen potentiates the anticoagulant effect of warfarin,11,19,20 others have not found a clinically relevant interaction.21,22 We found a significant reduction in warfarin-acetaminophen coprescribing after the alert was implemented despite the controversy surrounding the clinical significance of the interaction. This highlights the importance of the careful selection of medication interactions to target with alerts.
We found no improved effect in the study group that received group academic detailing. The detailing aimed to assist clinicians with the use of the medication interaction alerts and remove barriers to following the alert advice. However, well-constructed alerts may be self-explanatory and provide the just-in-time training that is thought to be a key feature of successful computerized decision support to improve prescribing.23 The absence of an additional intervention effect from the education may not be especially surprising given the strength of the EMR alerting intervention. A study of inpatient computerized physician order entry with computerized decision support24 revealed a strong effect on reducing potential adverse drug events, and a robust intervention that better integrated pharmacists into the care team did not enhance that effect. Despite our findings, academic detailing remains a potentially important intervention to improve prescribing safety. Also, it is interesting to note that we found a significant slope change (improvement over time) in the interacting prescription rate after alert implementation. This may indicate that the alerts themselves improve clinician awareness of medication interactions, leading to some improvement of the effect over time while the alerts are active.
Although the reduction of the interacting prescription rate was sudden and progressive, some may ask why the rate did not fall further after the alerts were implemented. Coprescribing warfarin and interacting medications is not contraindicated in all situations. In fact, acetaminophen alone or in combination with narcotics may often be the best medication choice for pain for patients taking warfarin.25 Similarly, the antibiotics targeted here may at times be the best or only choice for an individual patient. If an interacting medication is coprescribed, it would be appropriate to counsel the patient on the recommended dose limitations and to increase the frequency of INR monitoring. We did not assess these appropriate actions in response to alerts; nor did we assess warfarin dose adjustment. Also, our definition of coprescribing was conservative, that is, prescription overlap of even a single day was counted. This could have led to false-positive alerting in cases for which a short period of prescription overlap was not thought to be clinically significant and a secondary attenuation of the effect of the alerts.
This study has several other limitations. There is evidence that clinicians may be more responsive to alerts that address more severe medication interactions than we evaluated,26 thus our findings may not be generalizable to all interaction alerts. The only statistically significant effect found was on the rate of coprescribing of acetaminophen and warfarin. It is likely that the study was underpowered to detect changes in less frequent coprescribing events. Also, the study alerts comprised the majority of medication interaction alerts presenting to clinicians during the study period. In some settings, clinicians report that they can be overwhelmed by the sheer volume of alerts,9,26 suggesting that warfarin-interacting medication alerts may be less effective if competing for attention with alerts for many medication interactions. Our results thus may not be completely generalizable to other settings.
We were unable to ascertain patient use of over-the-counter acetaminophen or how the alerts affected clinician counseling of patients regarding this use. We did not evaluate the selection of medication alternatives and their inherent safety, and the study was not designed to evaluate the effect of the interventions on patient outcomes. Future research should include these key areas.
While one can never eliminate the possibility that a cointervention explains the changes we observed in interaction rates, we worked closely with the pharmacy, information technology, and anticoagulation clinic, and the programs for patients receiving warfarin remained stable throughout the study period. Therefore, such a cointervention is unlikely to explain the sudden and substantive reduction in the interaction event rate immediately following the alerts.
In summary, to our knowledge, this is the first study in ambulatory care with a strong quasi-experimental design that found that EMR alerts reduced the frequency of coprescribing of interacting medications. We found that coprescribing interacting medications with warfarin is common. Given that prior work supports the association between warfarin-interacting medication, increased INR, and resultant patient bleeding risk, implementing EMR alerts for medications interacting with warfarin should reduce patient morbidity and mortality. Group academic detailing did not improve the effectiveness of the alerts. Additional studies will be necessary to determine how to maximize the effectiveness of medication interaction alerts in the outpatient setting.
Correspondence: Adrianne C. Feldstein, MD, MS, Center for Health Research, Kaiser Permanente, 3800 N Interstate Ave, Portland, OR 97227 (Adrianne.C.Feldstein@kpchr.org).
Accepted for Publication: December 15, 2005.
Financial Disclosure: None
Funding/Support: This study was supported through cooperative agreement U18 HS11843 from the Agency for Healthcare Research and Quality. Drs Simon, Platt, and Soumerai were also supported by a grant from the Harvard Pilgrim Healthcare Foundation.
Role of the Sponsor: The funding organization was not involved in the design or conduct of the research; the collection, management, analysis, or interpretation of the data; or the preparation or approval of this manuscript.
Acknowledgment: We thank the many individuals who assisted with the design of the intervention and who were critical organizational advocates for the project: Homer Chin, MD; Ron Potts, MD; Sean Jones, MD; Maureen Wright, MD; Nancy Louie-Lee, MS, RPh; Lisa Shook; Christine Kovach, RPh; Nan Robertson, RPh; and Debra Wright, RPh. We also thank Martha Swain for editorial assistance and Debra Burch and Elizabeth Sheeley for administrative support.