Effect of Pharmacist Email Alerts on Concurrent Prescribing of Opioids and Benzodiazepines by Prescribers and Primary Care Managers: A Randomized Clinical Trial

Key Points

Question  Can pharmacist email alerts to practitioners reduce concurrent prescribing of opioids and benzodiazepines?

Findings  In this randomized clinical trial of 2237 patients coprescribed opioids and benzodiazepines and 789 practitioners who treated them, email alerts failed to detectably reduce concurrent prescribing of opioids and benzodiazepines, which can put patients at risk of overdose. The email alerts had no statistically significant effect on patient receipt of these medications or on practitioner prescribing.

Meaning  These findings suggest that alternative strategies may be more fruitful targets for efforts to make opioid prescribing safer and demonstrate that pairing randomization with quality improvement activities can generate evidence for stakeholders.

Importance  Policy makers have sought to discourage concurrent prescribing of opioids and benzodiazepines (coprescribing) because it is associated with overdose. Email alerts sent by pharmacists may reduce coprescribing, but this intervention lacks randomized evidence.

Objective  To investigate whether pharmacist emails to practitioners caring for patients who recently received opioids and benzodiazepines reduce coprescribing of these medications.

Design, Setting, and Participants  Randomized clinical trial (intention to treat) conducted in 2019-2021 of patients and their practitioners (prescribers and primary care managers) in the National Capital Region of the Military Health System. Participants were 2237 patients who were recently coprescribed opioids and benzodiazepines. These patients had 789 practitioners eligible for emails.

Interventions  Patients were randomized to email alerts to their practitioners or as-usual care. Clinical pharmacists sent the email alert. Messages were standardized and designed to facilitate coordination between practitioners, increase awareness of guidelines, and provide action steps and resources.

Main Outcomes and Measures  The primary outcomes were patients’ days received of opioids, benzodiazepines, and concurrent opioids and benzodiazepines during the 90 days following enrollment evaluated using 1-sided hypothesis tests. Secondary outcomes included total prescribing of opioids and benzodiazepines by patients’ practitioners, including to patients outside the study, to test for broader outcomes on their prescribing.

Results  Of 2237 patients, 1187 were assigned to treatment and 1050 to control; 1275 (57%) were women. Patients received a mean (SD) of 31 (44) days of opioids and 33 (34) days of benzodiazepines in the 90 days before enrollment. There were no detected differences in the primary end points, including patients’ receipt of opioids (adjusted difference, 1.1 days; 95% CI, -∞ to 3.0; P = .81), benzodiazepines (adjusted difference, −0.6 days; 95% CI, -∞ to 1.4; P = .30), and opioids and benzodiazepines together (adjusted difference, −0.1 days; 95% CI, -∞ to 0.7; P = .41). Of 789 practitioners, 429 were considered the treatment group, 325 were considered controls, and 35 were excluded. There were no detected differences in practitioners’ total prescribing of opioids, benzodiazepines, or both drug classes together.

Conclusions and Relevance  In this randomized clinical trial of pharmacist emails to practitioners, email alerts failed to detectably reduce coprescribing, highlighting the value of alternative approaches. Combining randomization with quality improvement activities may help stakeholders seeking evidence-based interventions to encourage guideline-concordant care.

Trial Registration  ClinicalTrials.gov Identifier: NCT03887247

During the past 2 decades, opioid overdoses and deaths have increased substantially in what is widely described as a public health crisis.^1-4 Harms from benzodiazepines have followed a similar trajectory but have attracted less attention.^5-7 These medications heighten opioid-induced respiratory depression, the cause of opioid overdose.⁸ Concurrent receipt of prescribed opioids and benzodiazepines is associated with adverse patient outcomes.^9-11 One-third to one-half of prescription opioid overdose deaths involve a benzodiazepine.^12,13 In 2017, more than 1 in 5 patients prescribed an opioid also received a benzodiazepine.^14,15 While this rate has declined in recent years, 3 million adults still receive concurrent prescriptions (coprescriptions) annually.¹⁶

These developments have led policy makers to discourage coprescribing of these medications. Recommendations to avoid coprescribing appear in guidelines from the Centers for Disease Control and Prevention and the Department of Veterans Affairs and the Department of Defense,^17,18 Choosing Wisely guidance from the American Society of Anesthesiologists,¹⁹ and the Beers Criteria from the American Geriatrics Society.²⁰ The US Food and Drug Administration also requires black box warnings about overdose on all opioid and benzodiazepine product labeling.²¹

The ongoing receipt of opioids and benzodiazepines together highlights the need for evidence-based approaches to encourage safer prescribing. Nudges, or interventions that seek to change behavior without directly limiting choices or changing incentives, provide one approach.^22,23 There are several examples of successful opioid prescribing nudges, including peer comparison feedback on pills per opioid prescription,²⁴ reduced default duration or quantity for new opioid prescriptions,^25-28 and letters to practitioners informing them that one of their patients overdosed.²⁹ Nudgelike interventions have also successfully reduced benzodiazepine prescribing.^30,31 Nonrandomized studies of engaging pharmacists to deliver interventions to the rest of the care team have reported the interventions as effective strategies,^32-34 as have clinical trials with pharmacists as participants in the intervention.^31,35 Yet there is little randomized evidence on using nudges to decrease opioid-benzodiazepine coprescribing. Evidence is also lacking on whether including pharmacists in efforts to reduce coprescribing could make them more successful.

We therefore conducted a randomized clinical trial (intent to treat) of email alerts from clinical pharmacists to practitioners after their patients filled coprescriptions of opioids and benzodiazepines. If these messages reduced subsequent prescribing, they could be valuable tools for policy makers, organizations, and practitioners seeking to make health care delivery safer.

The trial used a parallel-group design in which patients were randomized weekly to the treatment (email alert to their practitioners) or control (no alert) arms. The allocation ratio was 1:1 except for the first 421 patients, when we were testing an additional similar email alert. Further information appears in the Trial Protocol included as Supplement 1. At this time, the ratio was 1:1:1 to the 2 alert arms or control, or 2:1 to either alert vs control.

The intervention was considered quality improvement (QI) by the Walter Reed National Military Medical Center institutional review board. The evaluation was overseen by this institutional review board as human participant research and was exempt from informed consent requirements. This study follows the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

Study participants were patients in the National Capital Region (NCR) of the Military Health System and their associated NCR practitioners, defined as their opioid prescribers, benzodiazepine prescribers, and primary care manager. Patients who had recently filled overlapping opioid and benzodiazepine prescriptions were identified weekly in the Military Health System Population Health Portal Opioid Management Registry, which provides near real-time data on opioid prescribing in the Military Health System. Prescriptions written by non-NCR practitioners and community practitioners were counted when enrolling patients. However, only NCR practitioners were eligible for emails. Only primary care managers with a prior care relationship with the patient, defined as having at least 1 appointment in the past year, were eligible for emails.

To meet study inclusion criteria, the patient needed at least 1 opioid-benzodiazepine overlap day in the last month according to the dates of service and days’ supply of their prescription drug fills. We also required the patient have at least 1 practitioner eligible for emails (for the first 421 patients, we required at least 2 practitioners). Patients receiving hospice care, with a cancer diagnosis, or under age 18 years were excluded from enrollment, as were those who had already been enrolled. We capped weekly enrollment to limit pharmacist workload, with patients with more recent overlaps enrolled first.

Calculations from pre-intervention data indicated that a sample of 2500 patients would yield 80% statistical power to detect reductions in opioid prescribing of 2.0 days, benzodiazepine prescribing of 1.9 days, and concurrent prescribing of 1.1 days at the 5% significance level using 1-sided tests.

The intervention was an email alert to the patient’s practitioners. The message (available in the Trial Protocol Supplement 1) informed the practitioners that their patient may be at risk for a life-threatening opioid-benzodiazepine interaction. It encouraged them to take immediate action and, when multiple practitioners were identified, to coordinate with each other to revise the patient’s treatment plan. It sought to facilitate this coordination by allowing practitioners to reply all to the group and by listing practitioners’ phone numbers. The message noted the Veterans Affairs/Department of Defense clinical guideline and risk to patient, action steps including tapering and prescribing naloxone, and relevant resources about how to safely modify the patient’s treatment plan. It also included a report of the patient’s recent opioid and benzodiazepine prescriptions and information on risk factors and recent health care use related to overdose risk.

Initially, we also tested a version of the alert in which emails were sent to each practitioner individually and did not include practitioners’ contact information. Given their similarity, we do not distinguish between the 2 alerts in analyses and consider the 140 patients assigned to this alert treated.

Clinical pharmacists sent the message to practitioners as an encrypted email with the report attached. Enrollment occurred weekly from June 2019 to March 2021 except weeks when pharmacists were on vacation (8 weeks) or computer systems were undergoing maintenance (16 weeks). The prespecified analysis plan was finalized in January 2021. Researchers were unblinded to study data after enrollment ended in March 2021.

Patients were allocated to treatment and control arms each week using a random sequence of numbers. Randomization was stratified by ordering patients by their volume of opioid-benzodiazepine overlap during the previous 90 days, measured in days. We then divided the sorted list into blocks of 3 (first 421 patients) or 2 (subsequent patients). The enrollment list was sent to the clinical pharmacist team along with a packet of prepopulated emails and reports. The pharmacists then emailed the patients’ practitioners.

Our analyses use 2019-2021 prescription fill and military enrollment records reported in the Military Health System Management and Reporting Tool database. We observe fills at military pharmacies as well as community pharmacies covered by TRICARE, the military health insurance benefit. Analyses were conducted in Stata/MP version 17 (StataCorp).

The primary analysis was conducted at the patient level. We prespecified 3 primary end points owing to the letter’s focus on opioids, benzodiazepines, and overlap thereof. The first was the patient’s receipt of opioids measured as the total days’ supply of all opioid prescription drug fills during the 90 days following enrollment. The second end point was the patient’s receipt of benzodiazepines measured analogously. The final end point was the number of days the patient received overlapping opioids and benzodiazepines during the 90-day period. To measure overlap, we assumed a patient’s opioid or benzodiazepine fill became active on its fill date and lasted for its days’ supply.

Secondary patient-level end points included measuring the primary end points at shorter and longer durations; measuring these end points as the number of fills or the morphine or diazepam milligram equivalents; and tracking days’ supply of naloxone, substance use disorder treatment drugs, and other psychoactive and analgesic medications.

To assess effects of the intervention on practitioners, including changes in prescribing to their other patients who were not mentioned in the email alerts, we prespecified secondary analyses at the practitioner level capturing their prescribing to all patients. We assembled cohorts of treatment and control group practitioners. The cohorts consisted of the opioid prescribers, benzodiazepine prescribers, and/or primary care managers of study patients. We grouped practitioners into treatment or control based on the randomized treatment assignment of their first patient enrolled in the study. We considered the date a practitioner’s first patient was enrolled in the study as the practitioner’s enrollment date for outcome measurement. Practitioners in the control group could later receive emails if their patients were enrolled in subsequent weeks. For this reason, we consider the practitioner-level evaluation an imperfect compliance trial yielding intention-to-treat estimates.

Analyses at the practitioner level consisted of 3 end points. The first was prescribing of opioids to all patients as measured by the total days’ supply of all fills prescribed by the practitioner and dispensed during the 90 days following enrollment. The second was prescribing of benzodiazepines measured analogously. The last was the number of opioid-benzodiazepine overlap days to which the practitioner contributed during the 90-day postenrollment period. We calculated this end point by assuming a patient started taking a prescription on the fill date and continued for its days’ supply. For each patient receiving opioids and/or benzodiazepines from the practitioner, we counted the number of days during the 90-day period when they were taking medications in both classes and the practitioner prescribed at least one of them.

Patient-level analyses use multivariable linear regression models with inverse probability of treatment weighting to account for differing allocation ratios early vs late in the study period. All regressions adjust for opioid days, benzodiazepine days, and overlap days during the 90-day period immediately prior to enrollment. Other secondary end points are additionally adjusted for the end point as measured during the 90-day pre-enrollment period. Regressions include fixed effects for randomization strata. Practitioner-level analyses use the same models, assigning each practitioner the weighting of their first enrolled patient and including fixed effects for first enrollment week rather than randomization strata.

Analyses use robust variance estimates for inference. Practitioner-level analyses are clustered at the level of the first enrolled patient. We hypothesized that the intervention would lower prescribing and that, regardless, it was highly unlikely to increase it. Therefore, for each primary end point, we conducted a 1-sided hypothesis test with the alternative hypothesis that the effect was negative (implying a reduction in prescribing) and considered P < .05 significant. To address the multiplicity of testing, we also conducted hypothesis tests that effects on the 3 primary end points were jointly 0 and reported the single 1-sided P value.³⁶ We use the same approach for the 3 analogous practitioner-level outcomes. All other secondary end points are treated as exploratory, use 2-sided tests with P < .05 considered significant, and do not adjust for multiple testing.

A total of 2237 patients were enrolled in the study, with 1050 allocated to the control group and 1187 allocated to the treatment group (Figure 1). After exclusions, 2235 patients remained in the analysis sample. The mean (SD) number of contactable practitioners per patient was 1.6 (0.8) (Table 1; eTable 1 in Supplement 2). Patients received a mean (SD) of 31 (44) days of opioids and 33 (34) days of benzodiazepines during the 90 days before enrollment with 9 (15) days of opioid-benzodiazepine overlap during that time. More than half of patients had a mental health disorder diagnosis prior to enrollment and 1 in 9 had a substance use disorder diagnosis.

These patients had 1830 practitioners when also counting those who worked outside the NCR but whose prescriptions were covered by the military health benefit. A total of 789 of these practitioners were in the NCR and thus eligible for emails and inclusion in practitioner-level analyses (eFigure 1 in Supplement 2). Based on the treatment allocation of their first patient enrolled in the study, 325 of these practitioners were in the control group for these analyses and 429 were in the treatment group. A total of 35 practitioners had multiple patients enrolled in their first week and were excluded.

As expected given the imperfect compliance design of the practitioner analyses, 27.5% of control practitioners had at least 1 patient in the treatment group in the subsequent 90 days (Table 1). Practitioners were mostly physicians, more than half of whom were primary care physicians. Nonphysician practitioners were mostly physician assistants and nurse practitioners.

Following enrollment, patients’ mean receipt of opioids and benzodiazepines was similar in the control and treatment groups (Figure 2). Consistent with the visual evidence, there was no detected difference in opioid days (control mean: 20.2 days; treatment mean: 21.1 days, adjusted difference, 1.1 days; P = .81, 95% CI, -∞ to 3.0), benzodiazepine days (control mean: 20.2 days; treatment mean: 18.8 days, adjusted difference, −0.6 days, P = .30, 95% CI, -∞ to 1.4), or overlap days (control mean: 10.4 days; treatment mean: 10.0 days, adjusted difference, −0.1; days, P = .41, 95% CI, -∞ to 0.7) (Table 2). We also conducted a test of the joint hypothesis that effects on these endpoints were all 0. We failed to reject this joint hypothesis (t = .15; P = .56).

Results were similar at 30-day and 180-day durations as well as when measuring receipt in morphine or diazepam equivalent milligrams or number of fills. There were no detected impacts on receipt of naloxone, substance use disorder treatment medications, nonbenzodiazepine sleep medications, gabapentinoids, muscle relaxants, antipsychotics, or nonsteroidal anti-inflammatory drugs (eTable 3 in Supplement 2). Exploratory subgroup analyses by care team contact, including for patients whose entire care teams were contactable, also failed to detect any effects (eTable 4 in Supplement 2).

Mean prescribing of opioids, benzodiazepines, and drugs in both classes together was similar for treated and control practitioners during the 90 days after their first patients were enrolled (Figure 3). We failed to detect differences in prescribing of opioids (control mean: 170.2 days, treatment mean: 185.0 days, adjusted difference, −5.5 days; P = .34, 95% CI, -∞ to 15.7), benzodiazepines (control mean: 128.6 days, treatment mean: 134.8 days, adjusted difference, 9.4 days; P = .82, 95% CI, -∞ to 26.6), or opioid-benzodiazepine overlaps (control mean: 17.3 days, treatment mean: 20.4 days, adjusted difference: 0.0 days; P = .49, 95% CI, -∞ to 2.8). We failed to reject a test of the joint hypothesis that effects on these endpoints were all 0 (t = .21; P = .58). We found similar results at shorter and longer durations and using alternative measures of prescribing volume and opioid-benzodiazepine overlaps. We also failed to detect any changes in prescribing of other relevant medications (eTable 5 in Supplement 2).

In this randomized clinical trial, sending email alerts to practitioners after their patients filled opioid and benzodiazepine prescriptions failed to detectably reduce patient receipt of these medications. These results are estimated precisely and we can rule out effects that are small but would have been clinically meaningful. For instance, using 2-sided tests we can reject that the intervention reduced patients’ receipt of opioids by more than 1.3 days, a 6.5% relative reduction compared with the control mean of 20.2 days.

We also hypothesized that the alerts would encourage practitioners to reduce coprescribing throughout their patient panels. If the alerts were particularly effective at curtailing new initiations of these medications, this effect could materialize even without detected effects on the patients mentioned in the emails. To test this hypothesis, we used the fact that a practitioner’s first patient enrolled in the study was randomly assigned to the treatment. However, this analysis also did not detect any prescribing effects.

These results contrast with prior successful opioid nudge interventions.^25,26,29 However, interventions in this class are not universally effective. A recent systematic review of clinician nudges found that interventions that focused on providing or framing information often were not successful.³⁷ Our findings also did not align with prior work suggesting that pharmacist interventions encourage safer prescribing. Given the earlier evidence and organizational experience suggesting their utility in encouraging safe prescribing,³⁸ nudges engaging pharmacists in the care team remain a key topic for future research.

Our null findings demonstrate that combining QI activities with rigorous evaluation designs can produce valuable evidence for stakeholders. Traditionally, QI efforts have been evaluated with time series methods, but this approach can be at risk of confounding due to the phenomenon of regression to the mean.³⁹ As an example, in the current study, practitioners in the control group prescribed approximately 10% fewer days of opioids and benzodiazepines in the 90 days after their first patient was enrolled compared with the 90 days before. A pre-post analysis would spuriously note benefits on these end points even though these prescribers received no email alert for that patient. Embedding randomization into our QI activities facilitated a rigorous evaluation with a null result. While the evaluation did not uncover a benefit of the intervention, it was still useful for the organization because it showed that limited health care resources could be more effectively deployed elsewhere.

This approach may also help organizations seeking to become learning health systems, defined by the Institute of Medicine as when “knowledge generation is so embedded into the core of the practice of medicine” that it “leads to continual improvement in care.”⁴⁰ Randomized QI may help learning health systems identify practices with and without clinical benefits, facilitating a process of continuous improvement.^41-43

This study has several limitations. We considered why these email alerts did not lead to detectable reductions in prescribing. First, there was a secular decline in coprescribing in the NCR in the leadup to this study. These successful efforts by stakeholders to reduce use of opioids and benzodiazepines may have limited the ability of our intervention to further change behavior. In addition, prescribers may have already integrated warnings about the dangers of coprescribing into their clinical practice,¹⁴ leaving less scope for this intervention to provide practice-changing information.⁴⁴

Second, benefits of the intervention may have been too small or focused on a subset of patients to be detected. Most enrolled patients did not experience long-term coprescribing during which overdose risk and potential intervention benefits would have been highest. The alerts had limited potential benefits for patients who would not otherwise go on to fill more prescriptions, and several practitioners responded to the emails noting that they did not plan to continue prescribing. Alerts that occurred earlier in the prescribing process may have been more effective, as some research has suggested.⁴⁵

Third, the alerts encouraged practitioners to coordinate with each other, and evidence suggests patients who receive coprescriptions from multiple practitioners or payors are at greater risk.^46,47 While the message sought to promote safer prescribing through several channels, its ability to improve coordination was limited by the fact that 51% of study patients had one practitioner eligible for email. Efforts to ease coordination between practitioners separated geographically or across delivery systems may warrant further research; here, half of patients had practitioners who we could not contact because they worked outside the NCR or were community practitioners who could not be securely emailed.

Additional limitations include that actual enrollment was slightly below planned enrollment; that the treatment group includes the 140 patients randomized to similar email alerts in which practitioners were sent individual, not group messages; and that we could not ascertain whether practitioners opened the emails.

The findings of this randomized clinical trial highlight the value of rigorous testing for health care organizations and policy makers. Ineffective interventions take staff away from valuable clinical activities and can even harm the efficacy of beneficial interventions through alert fatigue. Our results cast doubt on the utility of this intervention and others like it. By focusing on other avenues to make prescribing safer, organizations can avoid wasting scarce resources and creating unnecessary burdens for practitioners. The success of other interventions based on behavioral science suggests many opportunities remain for stakeholders seeking cost-effective approaches to promote safe prescribing of opioids, benzodiazepines, and other medications.

Accepted for Publication: August 5, 2022.

Published: September 30, 2022. doi:10.1001/jamahealthforum.2022.3378

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Sacarny A et al. JAMA Health Forum.

Corresponding Author: Adam Sacarny, PhD, Assistant Professor, Department of Health Policy and Management, Columbia University Mailman School of Public Health, 722 W 168th St, 4th Floor, New York, NY 10032 (ajs2102@columbia.edu).

Author Contributions: Dr Sacarny had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Sacarny, Safran, Steffel, Sim, Brutcher, Spevak.

Acquisition, analysis, or interpretation of data: Sacarny, Safran, Steffel, Dunham, Abili, Mohajeri, Oh.

Drafting of the manuscript: Sacarny, Steffel.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Sacarny.

Obtained funding: Safran.

Administrative, technical, or material support: Sacarny, Safran, Dunham, Abili, Oh, Sim, Brutcher, Spevak.

Supervision: Sacarny, Safran, Brutcher, Spevak.

Conflict of Interest Disclosures: Drs Sacarny and Steffel reported receiving salary support from the US General Services Administration (GSA) during the conduct of the study through an Intergovernmental Personnel Act (IPA) agreement between their academic institutions and GSA. Both also reported being affiliated with GSA through the IPA. Mr Dunham reported at the time of the study being affiliated with Vista Defense Technologies, a contractor that has received funding from DHA, and currently being an employee of NATGO Data Group, a contractor that has received funding from the Military Health System, the entity that oversees DHA. NATGO Data Group had no involvement in this study. No other disclosures were reported.

Funding/Support: The quality improvement effort used no outside funding. The evaluation of this effort was funded by the Defense Health Agency via an interagency agreement with the US General Services Administration (DHA-2017-R-274).

Role of the Funder/Sponsor: The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the official policy of the Department of Defense, the US General Services Administration, or the US Government.

Additional Contributions: We thank Kara Beck, PhD (Veterans Health Administration), Mary Clair Turner, PhD (US General Services Administration), and Miles Williams, PhD (Denison University), for administrative, technical, and/or material support. None received compensation.