Analysis of Control Arm Quality in Randomized Clinical Trials Leading to Anticancer Drug Approval by the US Food and Drug Administration

Key Points

Question  How often are anticancer drugs approved by the US Food and Drug Administration (FDA) based on randomized clinical trials designed with a suboptimal control arm?

Findings  In this quality improvement study, 143 anticancer drug approvals granted by the FDA between January 1, 2013, and July 31, 2018, were reviewed; 16 (17%) of 95 anticancer drugs approved for market were approved based on comparing them with suboptimal control arms.

Meaning  Despite the increase in the number of drug approvals by the FDA, a substantial number of anticancer drugs are receiving market authorizations based on data that does not prove their superiority over standard-of-care therapy, thereby leaving both clinicians and patients unsure of the benefit conveyed by these agents.

Importance  To date, an empirical evaluation of the quality of control arms in randomized clinical trials (RCTs) leading to anticancer drug approvals by the US Food and Drug Administration (FDA) has not been undertaken.

Objective  We sought to estimate the percentage of RCTs that used a control arm deemed suboptimal and led to FDA approval of anticancer drugs from January 1, 2013, to July 31, 2018.

Design, Setting, and Participants  This quality improvement study included 143 anticancer drug approvals granted by the FDA from January 1, 2013, to July 31, 2018. All approvals based on single-arm studies (48 approvals) were excluded. Approvals based on RCTs were further investigated and each trial was analyzed for design, time of patient accrual, control arm, and primary end point. Standard-of-care therapy was determined by evaluating the literature and published guidelines 1 year prior to the start of trial enrollment. The percentage of approvals based on RCTs that used suboptimal control arms was then calculated. The quality of the control arm was deemed suboptimal if the choice of control agent was restricted to exclude a recommended agent, the control arm was specified but the recommended agent was unspecified, and if prior RCT data had demonstrated that the control agent was inferior to an available alternative.

Main Outcomes and Measures  Estimated percentage of RCTs that used suboptimal control arms that led to FDA approval of anticancer agents between January 1, 2013, to July 31, 2018.

Results  A total of 145 studies that led to 143 drug approvals between January 1, 2013, and July 31, 2018, were included. Of these studies, 48 single-arm studies were excluded. The remaining 97 studies led to 95 drug approvals. Of these 95 approvals, 16 (17%) were based on RCTs with suboptimal control arms; 15 were international trials, and 1 was conducted in the United States. The type of approval was regular in 15 trials and accelerated in 1 trial. When categorized by the nature of suboptimal control, 4 (25%) trials omitted active treatment in control arm by limiting investigator’s choice, 11 (63%) trials omitted active treatment in the control arm by using a control agent known to be inferior to other available agents or not allowing combinations, and 1 (13%) trial used a previously used treatment in the control arm with a known lack of benefit associated with reexposure.

Conclusions and Relevance  Although anticancer drug approvals are increasing, a proportion of these drugs are reaching the market without proven superiority to what is considered the standard of care at the time of patient enrollment in pivotal trials. The choice of control arm should be optimized to ensure that new anticancer agents being marketed are truly superior to what most clinicians would prescribe outside a clinical trial setting.

Most randomized clinical trials (RCTs) of novel anticancer drugs leading to marketing authorization by the US Food and Drug Administration (FDA) are designed, conducted, and funded by the biopharmaceutical industry. One potential concern with these studies is that in an effort to increase the likelihood of obtaining favorable results, the control arm of these studies may be suboptimal (“straw man” comparators).^1-4 For instance, Sharman et al⁵ note that the RESONATE-2 trial, a randomized trial of ibrutinib vs chlorambucil in CLL, relied on chlorambucil as control agent, a drug that has been repeatedly beaten by alternatives and has poor real-world use. Tao and Prasad¹ note other examples of trials with suboptimal control arms, including an RCT comparing nivolumab with dacarbazine in metastatic melanoma that enrolled patients after ipilimumab had been demonstrated to be superior in this setting.

Using a suboptimal control arm may bias a trial in favor of the experimental arm and reduces a trial’s external validity—it is no longer capable of answering the pertinent clinical question of whether the novel drug is superior to prevailing practice. Despite concern for suboptimal control arms in oncology, to our knowledge, there is no empirical analysis assessing how often they occur. For these reasons, we analyzed the percentage of RCTs that used a suboptimal control arm and led to regulatory approval for anticancer drugs.

This study of published reports did not involve patient data and was not submitted for institutional review board approval. Written patient informed consent was not required. We sought to assess what percentage of RCTs use a suboptimal control arm and lead to new or supplemental marketing authorization of anticancer drugs by the FDA.

We examined all hematology/oncology drug approvals by the FDA from January 1, 2013, through July 31, 2018, listed on the FDA website. We identified the accrual period, sites of accrual (national vs international), control arm used, primary end point, date of approval, and pathway of approval (accelerated vs regular). We excluded all marketing authorizations made on the basis of nonrandomized data.

We considered the quality of the control arm to be suboptimal if (1) restrictions were placed on the choice of control that excluded a recommended agent, (2) the control arm was specified but not the recommended agent (eg, the control arm was a single agent when doublet therapy is recommended), and (3) prior RCT data had demonstrated the control agent was inferior to an available alternative.

We assessed alternative control arms using 2 independent methods. First, the first and second authors (T.H. and M.B.S.) performed a search of the National Comprehensive Cancer Network (NCCN) guidelines and published review articles for a particular cancer to determine the standard-of-care (SOC) therapy for a specific cancer. These articles were published at least 1 year prior to start of accrual of an RCT of interest that led to FDA approval. Second, the first and second authors separately and independently read the published RCT data as well as the appendices, when relevant, and determined the adequacy of control arm. Conflicts were resolved by referring to the third author (V.P.).

Descriptive statistics are reported throughout. The study data was analyzed from October 1, 2018, to October 15, 2018.

Between January 1, 2013, and July 31, 2018, the FDA granted 143 anticancer drug marketing authorizations. Of these authorizations, 48 were excluded because the pivotal trial was nonrandomized, and 95 anticancer drugs were approved on the basis of 97 RCTs (Figure 1). Of the 95 approvals based on 97 RCTs, 11 (11%) were for genitourinary malignant neoplasms, 8 (8%) for gynecologic malignant neoplasms, 11 (11%) for gastrointestinal and hepatobiliary malignant neoplasms, 16 (18%) for lung and head and neck malignant neoplasms, 15 (16%) were for melanoma and sarcoma, 11 (11%) were for breast neoplasms, 5 (5%) were for myeloid malignant diseases, 12 (13%) were for lymphoid cancers, 6 (6%) were for plasma cell neoplasms (Figure 2 and eTable 1 in the Supplement). Of the 2 approvals for drugs to treat multiple myeloma (daratumumab and lenalidomide), which were based on 2 separate RCTs each,^6-9 all 4 trials used optimal controls.

Of these 95 anticancer drug approvals, 16 (17%) were based on trials with suboptimal controls^{10-15,17,20-22,24-26,28,31,32} (Table). Of these 16 approvals, 15 were based on international trials,^{10-15,17,20,22,24-26,28,31,32} and 1 was based on a trial²¹ conducted in the United States. The FDA approval pathway was regular in 15 trials,^{10-12,14,15,17,20-22,24-26,28,31,32} and accelerated in 1 trial.¹³ The primary end point for these studies was progression-free survival in 11 (69%),^{10,12,15,17,21,22,24,25,31,32} overall response rate in 1 (13%),¹⁴ another surrogate end point (eg, metastasis-free survival, major molecular response) in 3 (13%),^11,13,20 and overall survival in 1 (6%).^26,28

The Table provides a list of all cancer drug approvals determined to have suboptimal control arms, and eTable 2 in the Supplement outlines the reasons they were deemed suboptimal. When categorized by the nature of suboptimal control, 4 (25%) of 16 trials omitted active treatment in control arm by limiting investigator’s choice,^12,14,17,25 11 (63%) of 16 trials omitted active treatment in control arm by using control known to be inferior to other available agents or not allowing combinations,^{11,13,15,20-22,24,26,28,31,32} and 1 (13%) of 16 used a previously used treatment in the control arm with known lack of benefit associated with reexposure¹⁰ (eTable 2 in the Supplement).

When the threshold for determining the SOC therapy was extended to 2 years prior to start of accrual of an RCT of interest, 14 of the 16 clinical trials^{10-15,17,20-22,24,25,28,31} had control arms considered suboptimal.

During our study period, 16 of 95 anticancer drugs (17%) gained FDA approval based on RCTs that used a suboptimal control arm for the SOC in the United States. These findings raise important observations. First, international RCTs conducted in other countries may use a suboptimal control arm owing to lack of availability of what would be considered standard therapy in the United States. For example, the ALCANZA study¹⁴ allowed use of oral methotrexate or bexarotene (investigator’s choice) but prohibited the use of histone deacetylase (HDAC) inhibitors, which were approved by the FDA at the time of trial enrollment. This was justified by the lack of approval of HDAC inhibitors in the EU for cutaneous anaplastic large cell lymphoma. Yet, the results of this trial¹⁴ were also submitted for FDA approval. Apart from the ALCANZA trial, international RCT accrual did not directly explain the choice of control arm in 13 out of 16 trials.^{10-15,17,20-22,24,25,28}

Second, most approvals based on RCTs with suboptimal control arms receive regular or full FDA approval, and thereby do not require additional RCTs to verify clinical benefit. This is problematic because when an experimental agent has not been proven to be superior to the established SOC treatment, clinicians are potentially offering patients an agent that may be the equivalent or even inferior to established standard therapy, usually at a higher cost and alternate toxicity profile. The relevant question for the practicing physician, that is, whether a new drug is better than the current best therapy, may not be answered.

Third, the frequency of having a suboptimal control arm was similar between RCTs for hematologic and solid tumors (6 of 25 [24%] vs 10 of 73 [14%]). This lack of difference may be surprising because it appears that options for SOC therapy, particularly for hematologic cancers, can be numerous, allowing for laxity in the choice of control when designing RCTs. For example, TOURMALINE-MM1 compared the addition of ixazomib vs placebo to lenalidomide/dexamethasone in relapsed and/or refractory multiple myeloma.²⁶ Despite some guidelines listing doublet therapy regimens for relapsed multiple myeloma,²⁷ many experts used a triplet regimen³⁴ for relapsed multiple myeloma in the real world setting before the TROUMALINE-MM1 trial started enrolling patients.

One reason the FDA would approve agents that are compared with suboptimal control arms is the subjective nature of the decision that goes into assessing the control arm. In many instances, one can construct a justification for the control arm. Moreover, suboptimal control arms may accrue events (eg, progression of disease or death) faster, leading to successful results sooner, and hastening drugs to market. Another reason involves external influences on stakeholders, such as incentives from pharmaceutical companies, the highly lethal nature of a disease, and scarcity of active agents on the market. Therefore, it is essential to design trials that are capable of being flexible. Such trials with adaptive designs use results to modify the trial’s trajectory in accordance with prespecified criteria, which in some cases, may be more efficient than traditional design.³⁵

Our analysis is the first of its kind that attempts to empirically evaluate the quality of control arms in clinical trials leading to marketing authorizations over a 5-year period. The main limitation of our analysis is some inherent subjectivity in the assessment of acceptable SOC therapy before a particular trial started enrolling patients. We attempted to limit subjectivity by individually and separately reviewing the literature and guidelines and establishing consensus SOC treatment for each cancer. Given that the judgment of whether a control arm is suboptimal requires interpretation, we chose to publish our results so that future investigators can directly examine the trials presented in the Table.

The aim of our analysis was to evaluate the quality of control arms in RCTs leading to anticancer drug approvals by the FDA. We found that, between January 1, 2013, and July 31, 2018, FDA approval of 16 (17%) of 95 anticancer drugs for the market were based on RCTs with suboptimal control arms. Future regulatory trials should be optimized to ensure that new anticancer agents being marketed are truly superior to what most clinicians would prescribe outside a clinical trial setting.

Accepted for Publication: January 11, 2019.

Published Online: May 2, 2019. doi:10.1001/jamaoncol.2019.0167

Correction: This article was corrected on June 20, 2019, to fix errors in the number of US Food and Drug Administration approvals and trials described in the Abstract; Results, Discussion, and Conclusion sections; Figures, Table, and eTables. Forty-eight single-arm studies were excluded. The remaining 97 studies led to 95 drug approvals. The type of approval in 15 trials was regular and accelerated in 1 trial. Eleven trials omitted active treatment in the control arm by using a control agent known to be inferior to other available agents or not allowing combinations. One trial used a previously used treatment in the control arm with a known lack of benefit associated with reexposure. In Figure 1, 48 anticancer drug approvals were excluded, 95 anticancer drug authorizations were reviewed, and 79 approvals were based on randomized clinical trials with optimal controls. In Figure 2, the suboptimal control bars for genitourinary and lung/head and neck tumors should be 4% and 2%, respectively. In the Table, data from the ALTA trial was replaced with data from the PROSPER trial. The PROSPER trial was deleted from eTable 1 and added to eTable 2, and the ALTA trial was deleted from eTable 2.

Corresponding Author: Talal Hilal, MD, Mayo Clinic, 5881 E Mayo Blvd, Phoenix, AZ 85054 (hilal.talal@mayo.edu).

Author Contributions: All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Hilal, Prasad.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Hilal, Sonbol.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hilal.

Administrative, technical, or material support: Prasad.

Study supervision: Prasad.

Conflict of Interest Disclosures: Dr Prasad reported grants from Laura and John Arnold Foundation during the conduct of the study, royalties from his book Ending Medical Reversal, honoraria for Grand Rounds/lectures from several universities, medical centers, nonprofit groups, and professional societies, and serves as a writer for Medscape. He runs the podcast Plenary Session, which has Patreon backers. No other disclosures were reported.