Effect of Peer Comparison Letters for High-Volume Primary Care Prescribers of Quetiapine in Older and Disabled Adults: A Randomized Clinical Trial

Key Points

Question  Can behavioral nudges reduce inappropriate prescribing of antipsychotic agents and raise clinical quality for older and disabled patients, who often receive these drugs?

Findings  In this randomized clinical trial, a peer comparison letter randomized across the 5055 highest Medicare prescribers of the antipsychotic quetiapine fumarate reduced prescribing for at least 2 years. Effects were larger than those observed in existing large-scale behavioral interventions, potentially because of the content of the peer comparison letter, which mentioned the potential for a review of prescribing activity.

Meaning  Behavioral nudge interventions can raise the quality of prescribing, but research is still needed on how to most precisely target unsafe prescribing behavior.

Importance  Antipsychotic agents, such as quetiapine fumarate, are frequently overprescribed for indications not supported by clinical evidence, potentially causing harm.

Objective  To investigate if peer comparison letters targeting high-volume primary care prescribers of quetiapine meaningfully reduce their prescribing.

Design, Setting, and Participants  Randomized clinical trial (intent to treat) conducted from 2015 to 2017 of prescribers and their patients nationwide in the Medicare program. The trial targeted the 5055 highest-volume primary care prescribers of quetiapine in 2013 and 2014 (approximately 5% of all primary care prescribers of quetiapine).

Interventions  Prescribers were randomized (1:1 ratio) to receive a placebo letter or 3 peer comparison letters stating that their quetiapine prescribing was high relative to their peers and was under review by Medicare.

Main Outcomes and Measures  The primary outcome was the total quetiapine days supplied by prescribers from the intervention start to 9 months. Secondary outcomes included quetiapine receipt from all prescribers by baseline patients, quetiapine receipt by patients with low-value or guideline-concordant indications for therapy, mortality, and hospital use. In exploratory analyses, the study followed outcomes to 2 years.

Results  Of the 5055 prescribers, 231 (4.6%) were general practitioners, 2428 (48.0%) were in family medicine, and 2396 (47.4%) were in internal medicine; 4155 (82.2%) were male. All were included in the analyses. Over 9 months, the treatment arm supplied 11.1% fewer quetiapine days per prescriber vs the control arm (2456 vs 2864 days; percentage difference, 11.1% fewer days; 95% CI, −13.1% to −9.2% days; P < .001; adjusted difference, −319 days; 95% CI, −374 to −263 days; P < .001), which persisted through 2 years (15.6% fewer days; 95% CI, −18.1% to −13.0%; P < .001). At the patient level, individuals in the treatment arm received 3.9% (95% CI, −5.0% to −2.9%; P < .001) fewer days of quetiapine from all prescribers over 9 months, with a larger decrease among patients with low-value vs guideline-concordant indications (−5.9% [95% CI, −8.0% to −3.9%] vs −2.4% [95% CI, −4.0% to −0.9%], P = .01 for test that effects were equal for both patient groups). There was no evidence of substitution to other antipsychotics, and 9-month mortality and hospital use were similar between the treatment vs control arms.

Conclusions and Relevance  Peer comparison letters caused substantial and durable reductions in quetiapine prescribing, with no evidence of negative effects on patients.

Trial Registration  ClinicalTrials.gov identifier: NCT02467933

Every year, millions of older adults are prescribed atypical antipsychotic agents for off-label use beyond the indications approved by the US Food and Drug Administration (FDA), which are limited to schizophrenia, bipolar disorder, and some cases of depression.¹ Off-label prescribing to older adults for other indications, such as behavioral symptoms in dementia, anxiety, and insomnia, has continued^2-4 despite a large body of evidence that the use of atypical antipsychotics is associated with significant harm in these populations.^5-9 These harms include a host of adverse outcomes, such as increased risk of death, cognitive decline, extrapyramidal symptoms, and sedation.^7,10-12

This evidence has contributed to a broad consensus among psychiatric experts that excessive off-label use of antipsychotic medications in older adults, particularly those with dementia, is a serious problem. Multiple Choosing Wisely recommendations from the American Psychiatric Association target off-label use of antipsychotics.¹³ The FDA has warned against the use of antipsychotics for the treatment of elderly individuals with dementia.¹⁴ The American Geriatrics Society recommends that these drugs be used only when other interventions have failed and the patient threatens self-harm or harm to others.¹⁵

Quetiapine fumarate is an atypical antipsychotic that is prescribed at a particularly high frequency for off-label use. In the United States, 2.8 million patients fill a prescription for quetiapine annually,¹⁶ but as much as 75% of quetiapine prescribing lacks a basis in clinical evidence, making it an attractive target for interventions to reduce off-label prescribing.¹⁷

Quiz Ref IDThe widespread off-label use of antipsychotics in spite of clear guidelines has attracted the attention of the Centers for Medicare & Medicaid Services (CMS) and federal oversight agencies.^2,18 However, there is a gap between the need to curb antipsychotic overprescribing and the evidence base of effective interventions to change prescriber behavior. One existing approach focuses on changing health care professionals’ beliefs about the clinical benefits of prescribing; this intensive education can raise the quality of psychiatric medication prescribing.^19,20 Another set of techniques based on behavioral economics involves harnessing peer comparison messaging to nudge physicians to change behavior without financial incentives.^21-26 Yet, there is limited evidence on bringing health care professional education or behavioral nudges to a national scale. To our knowledge, no large-scale randomized behavioral interventions have targeted antipsychotic prescribing.

We performed a randomized clinical trial (intent to treat) of peer comparison letters to high quetiapine-prescribing primary care physicians with the goal of reducing excessive prescribing to Medicare program beneficiaries. Because peer comparison letters are inexpensive and easily scaled, they could be a powerful approach to improve the safety of antipsychotic prescribing.

Quiz Ref IDThis study used a placebo-controlled, parallel-group design with balanced randomization (1:1 ratio) to the control arm (placebo letter) and treatment arm (peer comparison letter). The study was overseen by an interdisciplinary team at CMS and the US Office of Evaluation Sciences (Washington, DC), as well as institutional review boards at Columbia University (New York, New York), Harvard University (Boston, Massachusetts), and the Massachusetts Institute of Technology (Cambridge). The institutional review boards each waived informed consent for prescribers. The trial protocol can be found in Supplement 1.

Study participants were primary care practitioners (PCPs) or prescribers chosen by a CMS analysis of quetiapine prescribing in Medicare Part D (prescription drug coverage) in 2013 and 2014. We chose PCPs (prescribers with a specialty of general practice, family medicine, or internal medicine) because the lack of psychiatric specialization suggested less formal training in prescribing of antipsychotics. We defined quetiapine prescriptions as prescriptions for branded Seroquel (AstraZeneca Pharmaceuticals LP), Seroquel XR (AstraZeneca Pharmaceuticals LP), or generic quetiapine.

Power calculations indicated that a sample of 5000 would have 80% statistical power to detect an intervention effect of 1.5% to 1.7% on overall prescribing at the 5% significance level. Study participants were identified from the pool of PCPs with at least 10 quetiapine prescriptions in 2013 and 2014 who prescribed significantly more quetiapine than other such prescribers in their state. The PCPs were classified as high prescribers if their prescribing was at or above the 75th percentile plus a multiplier factor of the interquartile range vs other PCPs in the same state (a modified Tukey outlier method²⁷) on 2 measures of quetiapine prescribing. These measures were (1) the number of quetiapine prescription fills supplied and (2) the total days of quetiapine supplied regardless of the number of patients (Supplement 1). A multiplier factor of 0.25 identified the 5055 highest-volume primary care prescribers (approximately 5% of all PCP prescribers of quetiapine) exceeding the outlier threshold for both measures in 2013 and 2014, which met our power calculations and became the study sample.

Quiz Ref IDThe intervention was a mailed peer comparison letter using social norms from the Center for Program Integrity (Baltimore, Maryland) within CMS on PCPs’ quetiapine-prescribing behavior.²³ Its message and format drew on insights from previous randomized evaluations of letter interventions.^25,28,29 The letter (Supplement 1) indicated that the prescriber’s quetiapine prescribing was under review by CMS and was extremely high relative to the within-state peers. The text of the letter discussed that high quetiapine prescribing could be appropriate but was concerning for medically unjustified use. The letter encouraged PCPs to review their prescribing patterns and explained that PCPs could expect to receive future communications from CMS. The placebo intervention was a letter and pamphlet discussing an unrelated Medicare enrollment regulation, sent to allow CMS to observe whether letters were returned to sender in the full sample.

Placebo and intervention letters were mailed in April 2015. Drawing on literature³⁰ that found that effects of letters grow when they are sent repeatedly, 2 follow-up intervention letters with more recent prescribing data were sent in August and October 2015 to treatment arm prescribers. An additional notice was sent to the control arm in June 2015 clarifying the enrollment process and the regulation.

The trial ended after the second follow-up letter on the request by CMS that the study team report the effect of the intervention. The prespecified analysis plan was finalized in March 2016, and researchers were then unmasked to the postintervention data.

Prescribers were allocated by the first study author (A.S.) to control and treatment arms. A random sequence of numbers and a prespecified rerandomization procedure were used (Supplement 1).

We analyzed prescribers and patients using 100% Medicare claims data from 2013 to 2017, enrollment data from 2015 to 2017, and risk-adjustment data from 2013 and 2014. Data were analyzed using statistical software (Stata/MP, version 13; StataCorp LP).

The primary outcome was measured at the prescriber level and was prespecified as the cumulative total number of quetiapine days supplied by PCPs in the 9 months after the intervention start (the initial mailing of letters). This outcome measure counts the number of quetiapine fills at pharmacies paid by Medicare Part D that were attributed to the targeted prescriber, quantified using the total number of days of quetiapine in the prescription fills. We chose the total number of days of quetiapine to integrate both changes in prescribing to continuing patients and initiations to new patients. As an exploratory outcome, we also assessed total number of days of quetiapine over an extended duration of 2 years.

We prespecified several additional secondary outcomes at the prescriber level and the patient level; we highlight several herein and provide the full set in Supplement 1. At the prescriber level, we also examined new quetiapine starts by PCPs, defined as all quetiapine days supplied to patients who had not received quetiapine from the study PCP during the last year. We also examined possible substitution toward similar atypical antipsychotic agents, the same drug class as quetiapine, as well as other psychiatric medications.

For patient-level outcomes, we defined a baseline cohort of patients as those receiving quetiapine from any study prescriber in the year before the intervention (Table 1 and Supplement 1). For this cohort, we examined the number of quetiapine fills over 9 months and 2 years, measured in days of quetiapine from all prescribers, divided into the following 3 mutually exclusive sources: the patient’s baseline study prescriber, other nonpsychiatric prescribers, and other psychiatric prescribers. We further examined health care use after the intervention, including inpatient admissions, emergency department visits, and psychiatrist outpatient visits, all cumulative to 9 months.

Across several outcomes, we also assessed the effect of the intervention based on the likely indication for quetiapine prescribing. We defined the following 2 cohorts of patients: (1) those whose indications likely fell under the FDA’s quetiapine black box warning (low-value prescribing) and (2) those with FDA-approved indications (guideline-concordant prescribing),^14,31 which also aligns with existing clinical guidelines.¹⁵ Using preintervention diagnoses in 2013 and 2014, quetiapine prescribing for patients with schizophrenia, bipolar disorder, or major depression without dementia or Alzheimer disease was deemed guideline concordant, whereas quetiapine prescribing for patients with dementia or Alzheimer disease but none of the major psychiatric illnesses above was considered low value (eTable 1 in Supplement 2). Patients in the low-value and guideline-concordant groups comprised 23 490 of 89 500 (26.2%) and 25 680 of 89 500 (28.7%) of the total baseline patient cohort, respectively (Table 1 and eTable 2 in Supplement 2). The residual group was composed of patients with no history of either category of diagnoses or with a history of diagnoses in both categories (Supplement 1); exploratory analyses of this group showed effects similar to the overall effects.

We used multivariable linear regression models to evaluate the effect of the intervention. To increase the statistical power of our analyses, we prespecified multivariable adjustment for the level of the outcome before the start of the intervention and for several additional characteristics (Supplement 1).^32,33 We used robust variance techniques in all statistical models, and patient-level analyses accounted for intraprescriber correlation with clustering at the prescriber level. Two-sided hypothesis tests with P < .05 were considered significant. To facilitate comparisons of outcomes with different levels, in some analyses we estimated a percentage effect by dividing the absolute effect (eg, absolute difference in quetiapine days supplied) and 95% CI by the control arm mean outcome.

Of the 5055 study prescribers, 2528 prescribers were allocated to the control arm (placebo letter), and 2527 prescribers were allocated to the treatment arm (peer comparison letter). Two prescribers were not sent follow-up letters because they had died. Of the 5055 prescribers, 231 (4.6%) were general practitioners, 2428 (48.0%) were in family medicine, and 2396 (47.4%) were in internal medicine; 900 (17.8%) were female. All 5055 prescribers were included in analyses (Figure 1). The baseline patient cohort contained 89 500 patients, 45 589 aligned to the control arm and 43 911 aligned to the treatment arm (Table 1 and eFigure 1 in Supplement 2).

The average prescriber in the study was responsible for supplying 2916 days (97 months) of quetiapine during the 9 months before the intervention (or about 3 months of quetiapine per week). On average, 820 (28.1%) of these days were to patients for likely low-value indications, and 778 (26.7%) were to patients for likely guideline-concordant indications. The average baseline patient received 193 days (6 months) of quetiapine during the 9-month preintervention period.

Quiz Ref IDDuring the 9-month postintervention period, the average treatment arm prescriber supplied 2456 days (82 months) of quetiapine vs 2864 days (96 months) in the control arm, an adjusted difference of −319 days (95% CI, −374 to −263 days) per prescriber or an 11.1% (95% CI, −13.1% to −9.2%; P < .001) decrease vs control (Table 2 and Figure 2A). Extending the postintervention period to 2 years, the cumulative effect was a 15.6% (95% CI, −18.1% to −13.0%; P < .001) relative decrease vs control. The intervention was also associated with a significant decrease of 27.1% (95% CI, −31.1% to −23.1%; P < .001) relative to control in the volume of new quetiapine prescriptions over 9 months, which persisted cumulative to 2 years (−24.3% relative decrease; 95% CI, −28.0% to −20.6%; P < .001) (Table 2 and Figure 2B).

At the prescriber level, the intervention reduced quetiapine prescribing to both low-value and guideline-concordant patients (Table 2 and eFigure 2A in Supplement 2). There was a smaller decrease in prescribing to guideline-concordant patients, although the effect was not statistically different compared with the decrease for low-value patients (P = .25 for test that effects were equal over 9 months and P = .17 cumulative to 2 years).

We also examined quetiapine prescribing at the patient level (ie, how the intervention affected the average baseline patient’s receipt of quetiapine from all prescribers over the outcome period). The intervention was associated with a reduction of 6.7 days of quetiapine (95% CI, −8.5 to −4.9 quetiapine days; P < .001) received per patient over 9 months or a 3.9% (95% CI, −5.0% to −2.9%; P < .001) relative decrease (Table 2). The cumulative effect at 2 years grew to a 5.6% relative decrease (95% CI, −6.8% to −4.3%; P < .001).

There was a significantly smaller reduction in the receipt of quetiapine for guideline-concordant patients than for low-value patients (P = .01 for difference in percentage reduction between the 2 patient groups both over 9 months and cumulative to 2 years; tests compare effects for low-value patients in this paragraph with effects for guideline-concordant patients in the following paragraph) (Table 2 and eFigure 2B in Supplement 2). For low-value patients, the intervention was associated with a 5.9% (95% CI, −8.0% to −3.9%; P < .001) reduction in quetiapine receipt over 9 months and with a larger 7.9% (95% CI, −10.4% to −5.4%; P < .001) decrease cumulative to 2 years. An exploratory analysis showed that the entirety of this effect came from the study prescribers, with no compounding or offsetting change from other (nonbaseline) prescribers (Figure 3A).

For guideline-concordant patients, there was a relative reduction of 2.4% (95% CI, −4.0% to −0.9%; P = .002) over 9 months and 4.0% (95% CI, −5.7% to −2.3%; P < .001) over 2 years in quetiapine receipt (Table 2). In exploratory analyses, we found that 39.8% of the reduction for guideline-concordant patients from study physicians was offset by shifting prescriptions to other prescribers (Figure 3B and eTable 3 in Supplement 2). Most of the offset was because of an increase in quetiapine receipt from other (nonbaseline) physicians with psychiatric specialization (the remainder came from other prescribers, including study prescribers from whom the patient did not previously receive quetiapine and nonpsychiatric prescribers outside of the study).

To test for effects on the total cessation of quetiapine, we considered whether patients received any quetiapine in each quarter in an exploratory analysis (eFigure 3 in Supplement 2). Percentage effects on the total cessation were roughly twice as large for low-value patients as for guideline-concordant patients.

There was no statistically significant effect of the intervention on PCPs prescribing or patients receiving other antipsychotics, antianxiety drugs, sleep aids, and antidepressants (eTable 4 and eTable 5 in Supplement 2). We studied the receipt of all antipsychotics for the low-value and guideline-concordant patient groups in an exploratory analysis (eTable 6 in Supplement 2). While both patient groups experienced increases in the receipt of other antipsychotics, the magnitudes were small, leaving the qualitative effect of the intervention on the total receipt unchanged.

There was no significant change in mortality, inpatient admissions, emergency department visits, or psychiatrist outpatient visits for baseline patients during the 9-month outcome period. Exploratory analyses of the patient groups detected only a reduction in emergency department visits for guideline-concordant patients (eTable 7 and eFigure 4 in Supplement 2).

In this randomized clinical trial, we found that peer comparison letters targeting the 5055 highest quetiapine-prescribing PCPs nationwide in the Medicare program led to statistically significant, persistent decreases in quetiapine prescribing. The decrease was pronounced for new quetiapine prescribing, suggesting a particular effect on physicians’ decision making about whether to initiate quetiapine treatment. The intervention was associated with reductions in prescribing to both low-value and guideline-concordant patients at the prescriber level; however, at the patient level, low-value patients had a significantly greater decline in quetiapine receipt. We detected no adverse effects of the letters on baseline patients according to mortality data and health care use. These results provide encouraging evidence that high prescribers of antipsychotics can decrease quetiapine prescribing, without adverse clinical consequences, in response to a letter highlighting their overall high rates of prescribing.

Compared with existing work on prescribing quality, this study provides a unique example of a large-scale intervention yielding clinically meaningful, persistent effects. For example, a recent antibiotic prescribing nudge targeting general practitioners throughout England reduced inappropriate prescribing by 3.3%.²⁴ Effects in the present study were smaller than those of other promising behavioral interventions on prescribing that targeted a more limited number of health care professionals (eg, where a peer comparison message reduced inappropriate antibiotic prescribing by 22% and effects endured after the intervention²³), although those interventions involved more complex changes, such as modifying electronic health record systems.^22,23,34

The findings herein also contrast with the null effect of a similar intervention performed by several members of our study team targeting high prescribers of controlled substances, including opioids.²⁵ The present study incorporated lessons from that work that could have contributed to the more substantial effect we observed here. First, our study targeted a wider range of high prescribers (approximately 5% of quetiapine-prescribing PCPs) vs the top 0.3% of all schedule II controlled substance prescribers in the previous study. Second, the letters in the present study had stronger wording regarding the possibility that prescribing was inappropriate and could be reviewed, which may have led physicians to take them more seriously. This finding can guide future evaluations of randomized letters with a variety of framings to find optimally effective approaches to communication.

In many domains we did not observe evidence consistent with significant unintended consequences from the present intervention, such as substitution away from quetiapine toward another antipsychotic agent. We observed reductions in the receipt of quetiapine among guideline-concordant patients, which could represent negative effects from PCPs cutting quetiapine use indiscriminately, even for patients who may need it. If this represented a harmful change for patients, we may have expected to see higher rates of adverse outcomes in the guideline-concordant patient group as prescribing rates decreased. However, if anything, guideline-concordant patients experienced lower rates of hospital encounters after the intervention. Although there are negative outcomes beyond these that we may not have observed, these results suggest that PCPs may be able to target guideline-concordant patients for whom stopping quetiapine treatment may be clinically justifiable while maintaining access for patients who experience clinical benefits (by continuing to prescribe to these patients or by shifting them to psychiatrists). In future interventions, it will be important to specifically target low-value care (eg, by selecting physicians not only by their high overall prescribing but also by their high rates of low-value prescribing).

Quiz Ref IDThis study has several limitations. First, our analysis included only prescribing covered by Medicare Part D. The letters may have encouraged physicians to reevaluate their prescribing to patients with private insurance, Medicaid, or no insurance coverage. This spillover effect could amplify or dampen the magnitude of our findings, depending on the nature of the spillovers. Second, another limitation concerns the external validity of the study if it was scaled or repeated in a different population. The effectiveness of the letters may have come from their novelty, and the magnitude of effects may decline if letters are used frequently or across multiple settings (eg, antibiotics, opioids, and benzodiazepines) similar to the well-documented phenomenon of alert fatigue.³⁵ Letters sent to other populations, such as prescribers who were not high-volume outliers, could have different effects. Third, we classify low-value and guideline-concordant prescribing using administrative data, which may have measurement error. Validation studies would enable future interventions to use these data more confidently. Fourth, our outcomes did not measure quality of life or mental health directly, which may have been the most likely domains for detecting a negative effect if the intervention caused harm.

Fifth, because of limitations in data access, we could not estimate effects for patients who were classified as neither low value nor guideline concordant. Imputed effects for this patient group were similar to the overall effects, but we did not report them because it was not possible to impute 95% CIs. We also were not able to assess the characteristics of the psychiatric (and nonpsychiatric) care providers who offset reductions in quetiapine prescribing by study PCPs.

We found that a low-cost series of peer comparison letters targeting PCPs who were high prescribers of quetiapine in the Medicare program resulted in large, sustained decreases in prescribing. We observed greater decreases in likely low-value, off-label prescribing than in potentially guideline-concordant prescribing, with little evidence that prescribers simply switched patients to other similar drugs and with no detected negative effects on patients. With increasing awareness of the dangers of inappropriate prescribing, this study provides evidence that peer comparison letters targeted at high-risk medications could effectively and efficiently create durable improvements in prescribing patterns.

Accepted for Publication: May 31, 2018.

Corresponding Author: Adam Sacarny, PhD, Department of Health Policy and Management, Mailman School of Public Health, Columbia University, 722 W 168 St, Fourth Floor, New York, NY 10032 (ajs2102@columbia.edu).

Published Online: August 1, 2018. doi:10.1001/jamapsychiatry.2018.1867

Author Contributions: Dr Sacarny had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Sacarny, Tetkoski, Yokum, Agrawal.

Acquisition, analysis, or interpretation of data: Sacarny, Barnett, Le, Yokum, Agrawal.

Drafting of the manuscript: Sacarny.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Sacarny, Barnett.

Obtained funding: Sacarny, Yokum.

Administrative, technical, or material support: Sacarny, Le, Tetkoski, Yokum, Agrawal.

Supervision: Sacarny, Agrawal.

Conflict of Interest Disclosures: None reported.

Funding/Support: We acknowledge the support of the Robert Wood Johnson Foundation, Abdul Latif Jameel Poverty Action Lab (J-PAL) North America, and the Laura and John Arnold Foundation (all to Dr Sacarny).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The views expressed in this publication represent those of the authors and not their respective organizations, including the Centers for Medicare & Medicaid Services (CMS). The contents of this publication were reviewed for compliance by CMS.