Availability of Comparative Efficacy Data at the Time of Drug Approval in the United States

Context Comparative effectiveness is taking on an increasingly important role in US health care, yet little is known about the availability of comparative efficacy data for drugs at the time of their approval in the United States.

Objective To quantify the availability of comparative efficacy data for new molecular entities (NMEs) approved in the United States.

Data Sources Approval packages publicly available through the online database of drug products approved by the US Food and Drug Administration (FDA).

Study Selection Identification of efficacy studies that supported approval of each NME approved by FDA between 2000 and 2010.

Data Extraction We determined whether eligible studies were head-to-head active controlled trials and whether the results of such studies were available in the approval packages. We recorded the approved indication, whether the NME was an orphan product, whether the NME had undergone priority review, and whether the control group was a specific active comparator or standard care.

Results Of 197 NMEs identified that met eligibility criteria, 100 (51% [95% confidence interval {CI}, 44%-58%]) met criteria for having comparative efficacy data available at the time of market authorization. After excluding NMEs designated as orphan products (n = 37) and those approved for indications for which no alternative treatments existed (n = 17), this proportion increased to 70% (95% CI, 62%-77%). The proportions of NMEs with available comparative efficacy data varied widely by therapeutic area, from 33% (95% CI, 9%-67%) for hormones and contraceptives to 89% (95% CI, 56%-99%) for diabetes medications.

Conclusion Publicly available FDA approval packages contain comparative efficacy data for about half of NMEs recently approved in the United States and for more than two-thirds of NMEs for which alternative treatment options exist. We did not investigate the extent to which available comparative efficacy information is useful for clinical guidance.

In 2009, Congress allocated $1.1 billion to comparative effectiveness research through the American Recovery and Reinvestment Act. According to the Institute of Medicine, such research is defined as the “generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.”¹

A key objective of comparative effectiveness research is to inform evidence-based treatment decisions made by patients and prescribers.² Comparative effectiveness information is also crucial for formulary and coverage decisions.³ Large-scale, randomized, head-to-head comparisons of multiple treatments are generally regarded as the gold standard for comparing the efficacy of medical interventions but are costly and time consuming. The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT), which compared the incidence of cardiovascular events and death among patients treated with 1 of 4 antihypertensive medications, with or without lipid-lowering therapy, cost US taxpayers more than $100 million and took 8 years to complete.⁴

Observational studies of comparative effectiveness are increasingly conducted, because they allow for simultaneous comparisons of multiple medical products, represent care in routine practice, and can be completed at a fraction of the cost of most clinical trials.⁵ In most situations, observational studies also may be more quickly completed than randomized trials.

However, comparative effectiveness information on drugs is most useful to decision-makers shortly after marketing approval, when observational data from routine care and data from large head-to-head trials comparing multiple treatments are not yet available. This raises the question about the extent to which preapproval data could inform comparative effectiveness decisions in the early marketing phase. Although reliance on placebo-controlled studies without active control groups generates much debate,⁶ little systematic evidence is available regarding the accessibility of comparative efficacy data when drugs are approved. Such data were available for roughly half of the drugs approved in Europe between 1999 and 2005.⁷ However, the European Medicines Agency recommends the use of both active and placebo comparators when possible,⁸ whereas the US Food and Drug Administration (FDA) does not.⁹

We sought to determine the proportion of recently approved drugs that had comparative efficacy data available at the time of market authorization in the United States and to examine trends in availability of this information over time and by therapeutic indication.

We obtained data from the publicly accessible Drugs@FDA database.¹⁰ We identified all new molecular entities (NMEs) approved from 2000 through 2010, excluding diagnostic agents, hyaluronidase, and sunscreens. We included therapeutic biologic products contained in the database, such as monoclonal antibodies, cytokines, growth factors, enzymes, immunomodulators, and thrombolytics, but the database does not include all biologics approved during the study period.

For each eligible NME, we accessed the drug approval packages and determined whether any of the efficacy studies that supported the approved indication used active comparators. The approval packages often contain multiple documents, including approval letters, medical reviews, chemistry reviews, pharmacology reviews, and statistical reviews. We considered an NME to have active comparator data if at least 1 efficacy study, described in any of these documents, included an active comparator. We required that the efficacy study be one that supported the approved indication; that is, the eligible trials were generally phase 3, unless only phase 2 studies were the basis for approval (as with darunavir), and the eligible trials studied end points relevant to the approved indication(s). The active comparator must have been approved by the FDA by the time the NME was approved. In some cases data were only available from ongoing clinical studies, which we included if the interim results were reported. We did not include studies that used historical controls. Furthermore, we required that efficacy results of the comparison between the NME and the active comparator be reported in the approval documents.

We determined whether the NME of interest (drug A) was compared with a specific alternative treatment option (eg, drug A vs drug B) or standard care alone (eg, drug A + standard care vs placebo + standard care). We also extracted information on the approval date, whether the drug was an orphan product, the review classification (ie, priority or standard), the approved indication(s), and whether the approval decision was based primarily on placebo-controlled or active drug–controlled trial data. Two investigators (N.H.G., M.K.K.) independently extracted data and made each determination. Discrepancies were discussed and resolved by consensus.

We calculated the proportion (and 95% confidence interval [CI]) of approved NMEs with available comparative efficacy data and cross-tabulated the proportion by therapeutic indication and year of authorization. We inspected the variation in proportions over time by plotting them by year of authorization. We then fit a multivariable logistic regression model to adjust for temporal effects that could be attributable to trends in priority review approvals and trends in types of drugs approved, with respect to therapeutic indication. We conducted this analysis at the individual NME level, with availability of comparative efficacy data as the dependent variable and with indicators of year of market authorization as our main explanatory variables of interest.

We further included indicators of review status and therapeutic indication to adjust for temporal changes in priority review designations and types of drugs approved. We transformed estimated log odds for each year from the logistic regression model to probabilities and plotted them. We performed all statistical analyses in SAS version 9.2 (SAS Institute Inc, Cary, North Carolina).

We identified 197 eligible approved NMEs between 2000 and 2010, of which a high of 24 were approved in 2000 and 2004 and a low of 14 were approved in 2005 and 2007 (Table). One hundred NMEs (51% [95% CI, 44%-58%]) had comparative efficacy data available at the time of market authorization. FDA approval was based primarily on active-control data for 59 NMEs, which was 59% (95% CI, 49%-68%) of NMEs approved with any comparative efficacy data available and 30% (95% CI, 24%-37%) of all NMEs approved during the study period.

After excluding orphan products (n = 37) and other NMEs approved for indications for which no alternative treatments existed (n = 17), the proportion with available comparative efficacy data increased to 70% (95% CI, 62%-77%). On a yearly basis, the proportion of NMEs with comparative efficacy data (excluding orphan drugs and those for which no alternative treatment existed) varied between 50% (95% CI, 21%-79%) in 2008 and 92% (95% CI, 65%-100%) in 2010.

Availability of comparative efficacy data was more common for some therapeutic indications, including diabetes mellitus (89% [95% CI, 56%-99%]) and infectious diseases (73% [95% CI, 54%-87%]), than others, such as hormones and contraceptives (33% [95% CI, 9%-67%]), cancer (35% [95% CI, 18%-54%]), and genitourinary tract conditions (38% [95% CI, 16%-66%]). After excluding orphan drugs and products approved for indications for which no alternative treatments existed, the proportions by therapeutic indications were more similar. After adjusting for therapeutic indication and priority review status in the multivariable model, the estimated trend in proportion of NMEs with comparative efficacy data over time was not meaningfully different from the unadjusted trend. Compared with those drugs that received standard review designations, NMEs that received priority review designations were much less likely to have comparative efficacy data (odds ratio, 0.24 [95% CI, 0.11-0.53]).

Overall, about half of all new drugs approved in the United States since 2000 were compared with an alternative treatment prior to market authorization, and the results of this comparison were publicly available in the FDA approval packages. For NMEs approved for indications with existing alternative treatment options, the proportion of approvals with comparative efficacy data increased to roughly two-thirds. Drugs with priority review designations were less likely to have comparative efficacy data available. Whether active comparators are used in clinical trials likely depends not only on the availability of a reasonable alternative or established standard of care but also on the ethics of placebo controls for the given indications. For example, when an alternative treatment existed, most NMEs approved for cancer and human immunodeficiency virus/AIDS had comparative efficacy information available.

Although comparative efficacy data meeting our minimal criteria were available for approximately half of all newly approved NMEs, we did not assess the extent to which the publicly available data are informative enough to provide a basis for prescribing and coverage decisions. The FDA and the Centers for Medicare & Medicaid Services recently issued a memorandum of understanding stating that the organizations will work together to promote initiatives related to the review and use of FDA-regulated products.¹¹ While the memorandum lacks details, one stated goal of the partnership is to establish processes that meet common needs for evaluating, among other things, the efficacy and coverage of medications. A step in that direction would be to further increase the proportion of NMEs with preapproval comparative efficacy studies and to improve the accessibility of such information at the time of FDA approval. Another avenue for broader accessibility and dissemination would be independent drug information providers, such as http://www.RxFacts.org. Making the data more accessible to clinicians and payers can help maximize their utility in prescribing and coverage decisions. To our knowledge, approval information is not widely used for making decisions in the postapproval setting.

Our definition of what constituted availability of comparative efficacy data was quite liberal. We required a minimum of 1 study with a single active comparator, although prescribers and payers often must make decisions among multiple alternatives, even within a single class.¹² Further, the proportion of drugs with available comparative efficacy information may be slightly overestimated by our exclusion of therapeutic biologic products not available in the Drugs@FDA database, because we expect that only a small proportion of approved biotechnological products would have head-to-head data available in approval packages.⁷ Nevertheless, our results, based on US approvals from 2000 through 2010, are in line with those of a similar study based on European approvals between 1999 and 2005. Many of the approved products overlap in the 2 studies, and US and European approvals are likely based on the same premarketing trials.

In conclusion, we found that publicly available documents include results of at least 1 head-to-head trial with an approved alternative for approximately half of all newly approved NMEs. Strategies are needed to enhance the accessibility of, and ultimately the use of, this information, particularly in the early marketing experience, when comparative effectiveness data from other sources are scarce or nonexistent.

Corresponding Author: Joshua J. Gagne, PharmD, MS, Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Harvard Medical School, 1620 Tremont St, Ste 3030, Boston, MA 02120 (jgagne1@partners.org).

Author Contributions: Dr Gagne had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Goldberg, Schneeweiss, Gagne.

Acquisition of data: Goldberg, Kowal, Gagne.

Analysis and interpretation of data: Goldberg, Schneeweiss, Kowal, Gagne.

Drafting of the manuscript: Goldberg, Gagne.

Critical revision of the manuscript for important intellectual content: Schneeweiss, Kowal, Gagne.

Statistical analysis: Goldberg, Gagne.

Obtained funding: Schneeweiss.

Administrative, technical, or material support: Kowal, Gagne.

Study supervision: Schneeweiss, Gagne.

Funding/Support: This study was funded by internal resources of the Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital.

Role of the Sponsor: Brigham and Women's Hospital had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

Additional Contributions: We gratefully acknowledge Judith Jones, MD, PhD (Degge Group Ltd), for helpful discussions about an earlier draft of the manuscript. Dr Jones did not receive any compensation for her contributions.