eAppendix. Full text and survey questions
Kesselheim AS, Woloshin S, Eddings W, Franklin JM, Ross KM, Schwartz LM. Physicians’ Knowledge About FDA Approval Standards and Perceptions of the “Breakthrough Therapy” Designation. JAMA. 2016;315(14):1516-1518. doi:10.1001/jama.2015.16984
Before US patients can use new prescription drugs, the US Food and Drug Administration (FDA) reviews the clinical trial results to confirm that benefits outweigh harms for the indication. Approval may involve superiority to placebo, not to an active comparator or standard of care (although approval can be based on uncontrolled or historically controlled studies). Numerous pathways expedite drug development and approval for serious or life-threatening conditions. For example, since 2012, the FDA can designate a drug as a “breakthrough therapy” if preliminary clinical evidence—such as an improvement in a pharmacodynamic biomarker—suggests an advantage over existing options.1 Through April 2015, the FDA designated 76 “breakthrough” drugs,2 and the term is routinely used in press releases3 and prescribing resources.4
Although the term breakthrough leads consumers to overly optimistic beliefs about drug effectiveness,5 it is not known how physicians understand this term—or more generally, what FDA approval means.
We selected a random sample of 300 clinically active internists and 900 specialists (endocrinology, hematology, and infectious diseases) from the American Board of Internal Medicine’s diplomate list; 52 lacked updated contact information. Our questions (eAppendix in the Supplement) were part of a larger survey, which was approved by the Brigham and Women’s Hospital's institutional review board, about prescription drugs. Consent was implied by participation after reading a description of survey goals. The sample was contacted via email, with nonresponders receiving 2 reminders, then a mailed version, and a final email reminder.
We asked 3 questions about FDA approval and 5 about breakthrough therapies (Table 1). Respondents included those who answered at least 1 question.
Physicians then chose between prescribing 2 new, identical hypothetical drugs: both met the statutory definition of breakthrough therapy, but only 1 was named a “breakthrough” drug. Finally, physicians were randomized by computer to read 1 of 4 versions of a mock FDA press release about a hypothetical new drug (Procampa) for lung cancer, approved based on tumor shrinkage: (1) the “facts alone” version described FDA approval data, (2) the “breakthrough” version stated that the FDA “designated it as a breakthrough drug,” (3) the “breakthrough/expedited” version added the drug’s review speed, and (4) the “breakthrough/warning” version also warned that the drug’s effect on survival was unknown (Table 2).
We calculated confidence intervals using the Wilson method (proportions) and the Wald method (differences in proportions), and tested associations using the Pearson χ2 test (nominal variables) and the Cochran-Mantel-Haenszel row mean scores statistic (ordinal variables). All analyses were done in Stata (StataCorp), version 13.1. A 2-tailed P value less than .05 was considered significant.
Of 1148 physicians contacted, 692 physicians (60%) responded: mean age, 46 years (SD, 10); women, 45%; specialists, 79%. Nonrespondents were similar: mean age, 49 years; women, 43%, and specialists, 80%. Respondents showed limited knowledge of FDA approval: 73% (95% CI, 69%-76%) incorrectly believed FDA approval meant comparable effectiveness to other approved drugs; 70% (95% CI, 66%-73%) incorrectly believed approval required both a statistically significant and clinically important effect. Specialists averaged slightly more correct answers than internists (1.2 vs 1.1, P = .02). Among the 3 breakthrough knowledge questions, 52% incorrectly believed that strong evidence (randomized trials) is needed to earn the breakthrough designation. Specialists averaged more correct answers than internists (1.5 vs 1.2, P = .01) (Table 1).
In the hypothetical scenario shown in Table 1, physicians preferred the breakthrough drug Axabex (94% [95% CI, 91%-95%]) over the drug Zykanta, which had no breakthrough designation (6% [95% CI, 5%-9%]). In the trial of the mock press release formats, compared with the facts alone version of the press release, the breakthrough/warning version decreased the percentage of physicians agreeing there was strong evidence of benefit (61% vs 48%, P = .01; between-group difference, 13% [95% CI, 3%-24%]) or that the hypothetical drug improved survival (64% vs 50%, P = .01; between-group difference, 14% [95% CI, 4%-24%]) (Table 2).
A national survey of board-certified internists and specialists revealed substantial deficits in knowledge of the meaning of FDA approval. Physicians tended to overestimate the minimum evidence of efficacy required of new drugs. Similarly, many misinterpreted the term breakthrough—believing these drugs were supported by stronger evidence than required by the statute.
Our results are limited by social desirability and other inherent survey response biases and may not be generalizable beyond internists and medical specialists. The misconceptions identified may lead physicians to overprescribe newly approved drugs—particularly breakthrough therapies—and inadequately communicate how well these drugs work to the patients who will use them.
Corresponding Author: Aaron S. Kesselheim, MD, JD, MPH, 1620 Tremont St, Ste 3030, Boston, MA 02120 (firstname.lastname@example.org).
Author Contributions: Dr Kesselheim had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Kesselheim, Woloshin, Schwartz.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Kesselheim, Woloshin, Schwartz.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Kesselheim, Woloshin, Eddings, Franklin, Schwartz.
Obtained funding: Kesselheim.
Administrative, technical, or material support: Kesselheim, Ross.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Schwartz and Woloshin are cofounders of Informulary, a company that provides data about the benefits, harms, and uncertainties of prescription drugs. No other disclosures were reported.
Funding/Support: Dr Kesselheim’s work was funded by the Greenwall Foundation.
Role of the Funder/Sponsor: The Greenwall Foundation had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.