A Randomized Trial Testing US Food and Drug Administration “Breakthrough” Language

In colloquial terms, “breakthrough” connotes an important, definitive advance. Since the 2012 US Food and Drug Administration (FDA) Safety and Innovation Act became law, however, the FDA can assign the breakthrough designation to a drug that “treats a serious or life-threatening condition” and “may demonstrate a substantial improvement…over available therapies” based only on preliminary evidence (eg, uncontrolled studies, surrogate outcomes).¹ Such drugs often receive “accelerated approval.”^2-4

All FDA press releases announcing approval of breakthrough-designated drugs use the term breakthrough; about half use promising.^4,5 Patients can easily find these press releases searching the Internet or hearing about them in the news. Unless patients understand the FDA’s usage of breakthrough, they may have unwarranted confidence in the evidence supporting drug claims. In a randomized trial, we test how these terms affect lay judgments (NCT02428556).

We recruited an online sample of 597 Americans (mean age, 36 years [range, 19-83 years]; 41% were women; 55% had a college and/or graduate degree) in June 2014 from an online service (Amazon’s Mechanical Turk [MTurk]). Participants received $1 for completing “a 10-minute medical drug survey.”

Participants were randomized to 1 of 5 vignettes—short descriptions of a recently approved drug (Table 1), based on an FDA press release for a metastatic lung cancer breakthrough drug conditionally approved based on the surrogate outcome tumor shrinkage. The facts-only condition described the drug as meeting the breakthrough criteria, but without using the term. The breakthrough and promising conditions added those terms. The tentative explanation used FDA-required language for professional labeling. The definitive explanation changed “may be contingent” to “is contingent.” Participants judged the drug’s benefit, harm, and strength of evidence (Table 2 includes full question text). A Kruskal-Wallis test was performed followed by Mann-Whitney U-tests comparing individual groups. A Bonferroni-correction accounted for multiple comparisons, α = .01 (IBM SPSS Statistics; version 23.0).

Adding either description (promising or breakthrough) increased the percentage of participants rating the drug as “very” or “completely effective” compared with facts-only: 23% and 25% vs 11%; P = .002 and P = .001 (Table 2). It significantly increased the percentage believing that the evidence supporting the drug is “strong” or “extremely strong”: 59% and 63% vs 43%; P = .006 and P = .003. Adding either explanation reduced the breakthrough effect on judged drug effectiveness and evidence quality. Both explanations significantly reduced the percentage of respondents incorrectly believing the drug had been “proven to save lives”: 16% (tentative) and 10% (definitive) vs 31% (breakthrough); P = .006 and P <.001. Perceived drug safety was similar in all groups. A final question in all conditions asked participants which of 2 drugs they would take for a potentially deadly condition: one described as “breakthrough,” the other described as meeting the breakthrough criteria. Ninety-two chose the “breakthrough drug.”

The terms breakthrough and promising increased people’s beliefs in a drug’s effectiveness and strength of supporting evidence compared with describing the drug as meeting the breakthrough criteria, but without using those descriptors. The breakthrough effect was mitigated by explaining the regulatory meaning of accelerated approval (as required in the professional label).

One study limitation is the sample, who were volunteers, younger, and better educated than the general public. Although MTurk samples often respond similarly to nationally representative ones⁶ we can only speculate about possible differences herein. A second limitation is using a hypothetical drug, albeit with information adapted from an actual FDA press release, hence similar to what people might encounter in their everyday lives.

While the name “breakthrough therapy designation” is specified by law, FDA is not required to use the name or terms like promising in press releases. Press releases with neutral terms—and that routinely explain the limited evidence supporting accelerating approval—might help consumers make more accurate judgments about these drugs.

Corresponding Author: Tamar Krishnamurti, PhD, Department of Engineering and Public Policy, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213 (tamar@cmu.edu).

Published Online: September 21, 2015. doi:10.1001/jamainternmed.2015.5355.

Author Contributions: Dr Krishnamurti had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: Krishnamurti, Schwartz, Fischhoff.

Drafting of the manuscript: Krishnamurti, Woloshin, Schwartz.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Krishnamurti.

Administrative, technical, or material support: Krishnamurti.

Study supervision: Krishnamurti, Woloshin, Schwartz.

Conflict of Interest Disclosures: Drs Woloshin and Schwartz are cofounders and shareholders of Informulary Inc, a provider of data about the benefits, harms, and uncertainties of prescription drugs. No other disclosures are reported.

Funding/Support: We gratefully acknowledge support of the Swedish Foundation for the Humanities and Social Sciences for supporting Dr Fischhoff while he revised the manuscript. Discretionary funds from Dartmouth Medical School covered the costs of collecting data, including compensation for research assistance.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Previous Presentation: Selected findings from this research (“What Evidence is Essential for New Medical Products? Implications for Patients and Health Policy”; Talk title: “Communicating uncertainty about new drugs”) were presented at the American Association for the Advancement of Science Meeting; June 13, 2014; Washington, DC.

Additional Contributions: We thank David Zimmerman, BS (Science and Humanities Scholars Program, Carnegie Mellon University), for his data collection research assistance, for which he was compensated, and Thomas Moore, PhD, Institute for Safe Medication Practices, Alexandria, Virginia, Department of Epidemiology and Biostatistics, The George Washington University School of Public Health and Health Services, Washington, DC, for very helpful comments on an earlier draft. Dr Moore received no compensation for his comments.