Evolution of the Randomized Clinical Trial in the Era of Precision Oncology

Key Points

Question  What are the characteristics that define modern randomized clinical trials (RCTs) in oncology, and do they differ from characteristics of RCTs from the preceding decades?

Findings  In this cohort study of 298 RCTs, progression-free survival was the predominant end point of oncology RCTs, and median survival gains remain modest. Almost all RCTs are now funded by industry; this is accompanied by a substantial increase in use of professional medical writers.

Meaning  The oncology community needs to consider new approaches to study design to ensure that treatments offer important benefits to patients; parallel to this, the current funding model for cancer clinical trials requires urgent attention.

Importance  The randomized clinical trial (RCT) in oncology has evolved since its widespread adoption in the 1970s. In recent years, concerns have emerged regarding the use of putative surrogate end points, such as progression-free survival (PFS), and marginal effect sizes.

Objective  To describe contemporary trends in oncology RCTs and compare these findings with earlier eras of RCT design and output.

Design, Setting, and Participants  Retrospective cohort study of systemic therapy RCTs in breast, colorectal, and non–small cell lung cancer published in 7 major journals between 2010 and 2020. This strategy replicates prior work and allows for comparison of trends with RCTs published between 1995 to 2004 and 2005 to 2009.

Main Outcomes and Measures  Data on RCT design, funding, results, and reporting were extracted from the published RCT report. Findings from the current period (2010-2020) were compared with data from RCTs published from 1995 to 2004 and 2005 to 2009. Descriptive and bivariate statistics were used to analyze temporal trends.

Results  The cohort included 298 RCTs (132 [44%] breast, 111 [37%] non–small cell lung cancer, 55 [19%] colorectal cancer). Experimental treatment included molecular inhibitor (171 of 298 [57%]), cytotoxic (83 of 298 [28%]), hormone (15 of 298 [5%]), and immune (24 of 298 [8%]) therapies. Sixty-nine percent (206 of 298) of RCTs were of palliative intent. The most common primary end point is now PFS; this has increased substantially over time (from 0% [0 of 167] to 18% [25 of 137] to 42% [125 of 298]; P < .001). Of 298 RCTs, 265 (89%) are now funded by industry (previously 95 of 167 [57%] and 107 of 137 [78%]; P < .001). Fifty-eight percent (173 of 298) of trials met their primary end point. Among positive trials, median improvement in overall survival and PFS was 3.4 and 2.9 months, respectively. More than one-third (117 of 298 [39%]) of reports used a professional medical writer; this increased substantially during the study period (from 3 of 27 [11%] in 2010 to 12 of 18 [67%] in 2020; P < .001).

Conclusions and Relevance  This cohort study suggests that contemporary oncology RCTs now largely measure putative surrogate end points and are almost exclusively funded by the pharmaceutical industry. The increasing role of medical writers warrants attention. To demonstrate that new cancer treatments are high value, the oncology community needs to consider the extent to which study end points and target effect size provide meaningful benefit to patients.

For the past 5 decades, the randomized clinical trial (RCT) has been the standard to establish efficacy of new cancer therapies. However, there have been important changes in RCT design and end points since its widespread adoption in the 1970s. In 2008, our group described characteristics of RCTs published in major journals during the era of cytotoxic chemotherapy (1975-2004) for breast, colorectal, and non–small cell lung cancers.¹ During this period, the size of RCTs increased substantially; primary end points shifted from response rate to overall survival (OS); the proportion of trials with a statistically significant difference in favor of the experimental arm doubled (23% to 42%); and, although the magnitude of benefit did not change over time, authors of modern recent RCTs were more likely to strongly endorse the experimental arm as a new standard of care. We also reported a major shift in funding: while government grants supported most RCTs (60%) in the 1970s and 1980s, by the late 1990s and early 2000s, 57% of trials were funded by industry. Finally, our original overview demonstrated sponsorship bias (ie, independent of effect size and statistical significance, industry-funded trials are more likely to be “positive”—defined in the present study as RCTs in which the primary end point has a statistically significant difference in favor of the experimental arm).

Using the same methods, we subsequently described RCTs published from 2005 to 2009 in the early years of targeted therapy.² As expected, trials increasingly tested targeted agents. During these years, median sample size increased substantially, and there was a major shift in the primary end point from OS to putative surrogate end points, such as disease-free and progression-free survival (PFS). During this era, industry funded 78% of RCTs.

This prior body of work described trends in oncology RCTs during the cytotoxic and early targeted therapy eras. The US Food and Drug Administration (FDA) approval of ipilimumab for metastatic melanoma in 2011 began a new era of immune therapy.³ Building on our previous work in this area, we sought to describe RCT design, results, and outputs in the past decade of oncology trials during the modern era of precision oncology. We used a methodologic approach that will allow comparison across treatment eras. Observations from the current era, in light of prior decades, will offer useful insights for the design and interpretation of future RCTs.

A literature search was performed to identify all phase 3 RCTs of systemic therapy in breast, colorectal, and non–small cell lung cancer published between January 1, 2010, and December 31, 2020, in the high-impact journals that publish a large proportion of practice-changing RCTs in oncology: New England Journal of Medicine, Lancet, Lancet Oncology, Journal of the American Medical Association (JAMA), JAMA Oncology, Journal of Clinical Oncology, and Journal of the National Cancer Institute. These journals and disease sites formed the basis of our prior work, ensuring some consistency in evaluating temporal trends.^1,2

An electronic MEDLINE search was conducted using search terms: journal name, disease type, clinical trial type, and year of publication (eFigure 1 in the Supplement). Exclusion criteria included early phase (ie, 1 or 2) or pilot studies; studies that did not involve an anticancer drug; studies on cancer screening/prevention; studies comparing dose, route, or schedule of the same drug; reports of pooled data from multiple RCTs; and articles that did not report primary efficacy results of the full study population. Two authors (J.D.P. and J.B.) reviewed trials for exclusion; disagreement was resolved by the senior author (C.B.). As this study included data from published RCTs and did not include any patient-level details, institutional review board approval was not required.

We used previously designed data abstraction variables to capture information regarding study design, results, and output.¹ Involvement of a cooperative trials group was identified based on explicit statement in the article, affiliations, or title. Superiority studies were classified as having met the primary end point if there was a statistically significant (P < .05) difference in favor of the experimental arm. Noninferiority and equivalence studies were considered to have met the primary end point if noninferiority/equivalence was established. Use of a professional medical writer was based on an explicit statement in the article.

Two authors (J.D.P. and J.B.) initially piloted the abstraction tool on 10 RCTs and subsequently modified the database. A single author (J.B.) captured data on all RCTs. To maintain quality assurance, a second author (J.D.P.) periodically reviewed a total of 50 randomly selected studies to ensure there were no systematic errors in data capture. Upon final completion of data capture, a second author (J.D.P.) repeated abstraction for a random selection of 10% (n = 31) of the final cohort of RCTs to measure quality.

Descriptive results are provided for RCTs within the current study period (2010-2020). Trends in RCT characteristics were directly compared across the eras of cytotoxic therapy (1995-2004), targeted therapy (2005-2009), and precision oncology (2010-2020). Categorical data were compared using the Pearson χ² or Fisher exact test depending on cell sizes, and continuous data were compared using the Mann-Whitney U or the Kruskal-Wallis test. All P values were from 2-sided tests, and results were deemed statistically significant at P < .05; no adjustments for multiple comparisons were made. All analyses were performed using IBM SPSS, version 26.0 for Windows (IBM Corp).

The search strategy yielded 1078 results; 780 articles were excluded (eFigure 2 in the Supplement). The final study cohort consisted of 298 trials (list available from corresponding author on request) including a total of 333 618 patients. Interobserver agreement between the 2 data abstractors based on a random extraction of 10% (31 of 298) of RCTs was 99% for all data points (1819 of 1829).

Characteristics of RCT design during 2010 to 2020 are summarized in Table 1; to highlight temporal trends, data from 1994 to 2004 and 2005 to 2009 are also shown.^1,2 Over time, there has been a relative increase in non–small cell lung cancer trials (from 23% to 26% to 37%; P = .003) and decrease in colorectal cancer trials (28% to 23% to 19%; P = .05). Trials are increasingly likely to be palliative in nature (from 60% to 56% to 69%; P = .02).

The median sample size from 2010 to 2020 was 682; this has been stable since 2005 to 2009 (722) but substantially larger than in 1995 to 2004 (446). Median follow-up has decreased over the 3 time periods from 47 months (1995-2004) to 37 months (2005-2009) to 25 months (2010-2020). Among the 272 superiority trials published during 2010 to 2020, the median hazard ratio (HR) that a trial was powered to detect was 0.74. As shown in Figure 1A, there was a modest increase in the effect size a study was powered to detect over time (from an HR of 0.77 in 2010 to an HR of 0.70 in 2020; P = .001).

In 2010 to 2020, PFS was the most commonly used primary end point (125 of 298 [42%]); only 86 of 298 (29%) RCTs used OS as the primary end point. Over the 3 study periods, there has been a marked shift away from OS as primary end point (from 49% to 36% to 29%; P < .001); use of PFS increased substantially in 2010 to 2020 compared with 2005 to 2009 (from 18% [25 of 137] to 42% [125 of 298]; P < .001) (Figure 2).

Targeted agents (including molecular inhibitors and immune therapies) represented 65% (195 of 298) of all experimental agents in the study cohort (Table 2). Immune therapies were tested in 8% (24 of 298) of trials. The use of a targeted anticancer drug was more common in non–small cell lung cancer trials (89 of 111 [80%]) compared with breast (70 of 132 [53%]) and colorectal cancer (36 of 55 [65%]) (P = .001), as well as in trials funded by industry (187 of 265 [71%] vs 8 of 33 [24%]; P < .001). Among the 195 trials of targeted or immune therapies, 25 of 195 (13%) were restricted to specific histologic types (all in non–small cell lung cancer), while 116 of 195 (60%) were restricted to a specific biomarker. The use of biomarker-driven treatment increased over the study period (26% in 2010 vs 61% in 2020; P = .003).

During 2010 to 2020, 265 of 298 (89%) phase 3 RCTs were funded by industry; this proportion increased substantially from 1995 to 2004 (57%) and 2005 to 2009 (78%) (P < .001) (Figure 2). Ninety-six percent (187 of 195) of RCTs testing targeted agents were funded by industry compared with 74% (61 of 83) of trials testing experimental cytotoxic therapy (P < .001). Industry-funded trials were more likely to study treatments in the palliative setting than nonindustry studies (188 of 265 [71%] vs 18 of 33 [55%]; P = .05). Among palliative studies, industry-funded trials were more likely to use PFS as the primary end point (118 of 188 [63%] vs 7 of 18 [39%]; P = .048). One-third (97 of 298 [33%]) of current studies involved cooperative trials groups; these trials were less likely to be funded by industry than trials without cooperative group involvement (77 of 97 [79%] vs 188 of 201 [94%]; P = .001). Cooperative groups were less likely to study cancers in the palliative setting (53 of 97 [55%] vs 153 of 201 [76%]; P < .001).

Half of all trials (173 of 298 [58%]) met their primary outcome; 153 of 272 (56%) superiority trials had a statistically significant difference in favor of the experimental arm. Superiority trials with palliative intent (compared with adjuvant/neoadjuvant) were more likely to be positive (114 of 188 [61%] vs 39 of 84 [46%]; P = .03). Palliative trials with PFS as the primary end point were also more likely to be positive compared with trials using an OS end point (84 of 115 [73%] vs 28 of 66 [42%]; P < .001).

Among all positive superiority trials in the contemporary cohort, the median HR observed for the primary outcome measure was 0.68. There was a temporal increase in effect size over the study period (HR of 0.73 in 2010 vs HR of 0.59 in 2020; P = .047; Figure 1B). The OS results were reported in 187 of 272 (69%) superiority trials. One-third of these trials (64 of 187 [34%]) showed a statistically significant improvement in OS outcome; median (interquartile range) improvement in OS was 3.5 (2.5-6.6) months. The PFS results were reported in 169 of 272 (62%) superiority trials. Two-thirds (117 of 169 [69%]) showed a statistically significant PFS benefit; median (interquartile range) improvement in PFS was 2.8 (1.5-5.1) months. Quality-of-life (QOL) data were captured in 37% (110 of 298) of all trials in the study cohort and 46% (95 of 206) of all palliative intent trials.

During 2010 to 2020, an increasing proportion of oncology RCTs (from 11% in 2010 to 44% in 2020; P < .001) were published in the major general medical journals (New England Journal of Medicine, Lancet, JAMA). Positive RCTs were substantially more likely than negative RCTs to be published in a major journal (63 of 173 [37%] vs 6 of 125 [5%]; P < .001). Thirty-nine percent (117 of 298) of RCT reports used a medical writer; this was observed almost exclusively in industry-funded RCTs (115 of 265 [43%] vs 2 of 33 [6%]; P < .001). There was a substantial increase in use of medical writers during the study era (from 3 of 27 [11%] in 2010 to 12 of 18 [67%] in 2020; P < .001) (Figure 3).

In this study, we describe the design and results of oncology RCTs conducted during 2010 to 2020 in the era of precision oncology. We compare these findings with RCTs from prior eras (targeted therapy 2005-2009, and cytotoxic therapy 1995-2004). Several important findings have emerged. First, industry now funds 89% of all RCTs, and this has increased over time. Parallel to this, we identify tremendous uptake in use of medical writers: from 11% of RCT reports in 2010 to 67% in 2020. Second, the majority of RCTs test new therapies in the palliative setting. Third, almost half of trials use PFS as the primary end point, and this continues to increase over time. Fourth, palliative trials and trials using PFS as the primary end point are more likely to be positive. Fifth, although most trials are now testing targeted therapies, only 60% of these trials use biomarker enrichment. Finally, while there is some signal of increased effect size in the HR used in both RCT power calculations and study results, the median improvement in OS and PFS among positive trials remains modest at only 3.5 and 2.8 months, respectively.

Despite important advances in oncology that have led to substantial improvements in patient outcomes, there is growing recognition that many new treatments do not provide meaningful benefit to patients,^4,5 have important clinical toxic effects,⁶ and are associated with rapidly escalating drug costs.^7,8 These observations led to the contemporary movement promoting value in cancer care with important frameworks developed by the American Society of Clinical Oncology and the European Society for Medical Oncology.^9,10 Within our study cohort, half of the trials were positive. While the effect size in power calculations and observed HRs among positive trials have slightly increased, median gains in OS and PFS are modest. The observed improvements in PFS and OS are consistent with a review of solid tumor drugs approved by the FDA in 2002 to 2014 in which median gains in OS and PFS were 2.1 months and 2.4 months, respectively.¹¹ Prior work has shown that most new cancer therapies do not offer meaningful clinical benefit, including both FDA-approved and European Medicines Agency–approved treatments.^12-14 Among all positive RCTs of cancer therapy published in 2014 to 2017, only one-third identified treatments that met the European Society for Medical Oncology–Magnitude of Clinical Benefit Scale threshold for substantial clinical benefit.¹⁵ Moreover, with their very high prices, the value of many modern anticancer therapies is questionable.¹⁶

The marked rise in use of PFS is of further concern, particularly given that PFS trials are more likely to be positive. Temporal trends in end points and industry funding over the past 5 decades are shown in Figure 2. While there are some contexts in which PFS is an appropriate end point, this is the exception and not the rule; in most contexts, PFS is not a valid surrogate for QOL or OS.^17-20 In the present cohort, QOL data were only evident in half of palliative intent trials. We believe that all trials in noncurative settings should measure QOL; failure to do so severely limits application of results to patient care.

There has been a continued shift toward agents that target specific molecular features of cancer; however, only 60% of RCTs involving targeted agents restrict their populations to a specific biomarker. Next-generation sequencing, which has allowed for rapid genomewide evaluation of DNA variations, is increasingly being used to guide treatment decisions, despite a paucity of phase 3 data establishing the efficacy of this approach.²¹ A further problem with the current paradigm was identified in a recent review of ClinicalTrials.gov,²² which found that only a small proportion of planned, ongoing, or completed precision oncology trials are randomized by design. To fully realize the potential of precision medicine, our community will need to more carefully consider its implications on study design.

Our body of work shows that, over the past 5 decades, there has been a massive shift in the funding of oncology clinical trials (Figure 3). Almost all phase 3 RCTs are now funded by industry. Given the cost of new agents and the resources required to run oncology RCTs, this observation is not unexpected. Industry involvement in RCTs influences all stages of RCT conduct, including design, data collection, analysis, and manuscript drafting.^23-26 Accordingly, this complex relationship needs to be carefully considered by the oncology community, as we recognize its inherent risks and conflicts of interest. It also speaks to a fundamental problem in how cancer research funds are allocated: there are many important questions that may not be of interest to industry. Randomized clinical trials to answer these questions will require funding from other sources, such as government grants and philanthropic groups. Otherwise, there is a risk that our field will continue to investigate only novel, expensive therapeutics. This risk was demonstrated in our recent overview²⁷ of all oncology RCTs published globally during 2014 to 2017: the majority of trials test new medicines in the palliative setting, and only 13% of trials test new approaches to radiotherapy or surgery.

Finally, more than one-third of RCTs in the present study cohort involved a medical writer, a trend that has skyrocketed in recent years, with 67% of RCTs published in 2020 from the cohort using medical writers (Figure 3). This is consistent with recent work by Kouzy et al,²⁸ who reported that 43% of oncology RCTs used professional medical writers. There is reason to be concerned that medical writers may unduly influence the interpretation of trials. Additionally, their role is contrary to accepted scientific principles whereby first authors should take responsibility for writing their own manuscripts.²⁹ This is an issue that requires serious discussion by clinicians and journal editors, as it is unlikely that medical writers have a neutral effect on the clinical trial reporting.

Our study has methodologic limitations. The search strategy replicates our prior work^1,2 and allows us to describe trends in oncology RCTs over 5 decades; however, restricting the cohort to 3 cancers and 7 high-profile journals may limit the generalizability of the findings to other cancers. Our primary goal, however, was to establish a similar cohort to build on our previous reports on the characteristics of oncology RCTs for 3 common cancers. Furthermore, we sought to describe practice-changing RCTs, the majority of which would be published in these high-profile journals. Because much of the initial work in immune therapy took place in other cancers (ie, melanoma), the present cohort includes a relatively small proportion of RCTs (24 of 298 [8%]) that test agents of this class. We also dichotomized funding as “industry: yes/no”; this classification does not distinguish partial funding (ie, only provision of study drug) from full funding and study sponsorship. Finally, our definition of a positive RCT—which was restricted to the primary end point having a statistically significant difference in favor of the experimental arm (ie, P < .05)—does not account for the more complex interpretation of positivity necessary when clinically applying RCT results to general patient populations.³⁰

The current study provides insights into the design, results, and reporting of systemic therapy RCTs in the contemporary era. Over the past 5 decades, we have seen a major shift in primary end points and funding of RCTs. We are concerned with the widespread adoption of medical writers in high-profile oncology RCTs and believe this practice should be questioned; journal editors will need to consider if this practice is consistent with International Committee of Medical Journal Editors policies for authorship. Given current interests in promoting high-value care, our community needs to reflect on the widespread use of PFS and whether effect sizes targeted in the trial designs are large enough to change clinical practice. To fulfill the promise of precision medicine in oncology, we will also need to further understand if drugs with specific targets have activity in specific groups of patients rather than “all-comers.” Finally, the oncology community needs to consider how we can answer fundamental questions in our field that will be of low priority for the pharmaceutical industry. This will require additional funding streams to ensure that the cancer research enterprise is positioned to answer questions that matter most to patients.

Accepted for Publication: February 10, 2021.

Published Online: March 25, 2021. doi:10.1001/jamaoncol.2021.0379

Corresponding Author: Christopher M. Booth, MD, Division of Cancer Care and Epidemiology, Queen’s Cancer Research Institute, Queen’s University, 10 Stuart St, Kingston, ON K7L 3N6, Canada (booth@queensu.ca).

Author Contributions: Drs Del Paggio and Booth had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Del Paggio, Prasad, Booth.

Acquisition, analysis, or interpretation of data: Del Paggio, Berry, Hopman, Eisenhauer, Gyawali, Booth.

Drafting of the manuscript: Del Paggio, Prasad, Booth.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hopman.

Supervision: Prasad, Booth.

Conflict of Interest Disclosures: Dr Prasad reported grants from Arnold Ventures; royalties from Johns Hopkins Press, Medscape, and MedPage; consulting fees from UnitedHealthcare and New Century Health; speaking fees from eviCore and New Century Health; honoraria for grand rounds/lectures from universities, medical centers, nonprofits, and professional societies; and having Patreon backers for the Plenary Session podcast from Patreon outside the submitted work. Dr Booth is supported as the Canada Research Chair in Population Cancer Care. No other disclosures were reported.