Characteristics of Oncology Clinical Trials: Insights From a Systematic Analysis of ClinicalTrials.gov

Importance Clinical trials are essential to cancer care, and data about the current state of research in oncology are needed to develop benchmarks and set the stage for improvement.

Objective To perform a comprehensive analysis of the national oncology clinical research portfolio.

Design All interventional clinical studies registered on ClinicalTrials.gov between October 2007 and September 2010 were identified using Medical Subject Heading terms and submitted conditions. They were reviewed to validate classification, subcategorized by cancer type, and stratified by design characteristics to facilitate comparison across cancer types and with other specialties.

Results Of 40 970 interventional studies registered between October 2007 and September 2010, a total of 8942 (21.8%) focused on oncology. Compared with other specialties, oncology trials were more likely to be single arm (62.3% vs 23.8%; P < .001), open label (87.8% vs 47.3%; P < .001), and nonrandomized (63.9% vs 22.7%; P < .001). There was moderate but significant correlation between number of trials conducted by cancer type and associated incidence and mortality (Spearman rank correlation coefficient, 0.56 [P = .04] and 0.77 [P = .001], respectively). More than one-third of all oncology trials were conducted solely outside North America.

Conclusions and Relevance There are significant variations between clinical trials in oncology and other diseases, as well as among trials within oncology. The differences must be better understood to improve both the impact of cancer research on clinical practice and the use of constrained resources.

The cost of cancer care is rising quickly in the United States.¹ However, our understanding of the value of the care being delivered has not kept pace owing to a number of factors, including the lack of comparative effectiveness research,² inefficiencies in trial execution,^3,4 increasing use of off-label agents,⁵ and increasing reliance on less robust trial designs.^6,7 Concerns about clinical research in the United States already exist, prompting the reorganization of the cancer cooperative groups,^4,8 the creation of the Patient-Centered Outcomes Research Institute,⁹ and the development of research metrics such as success in accrual of intended study sample size.^10,11 However, to our knowledge, the data needed to truly understand the characteristics of oncology trials being conducted and their ability to advance clinical care do not yet exist.

ClinicalTrials.gov represents a unique resource with which to explore the research enterprise.^12-16 It is the most robust of the international clinical trial registries, currently containing detailed information on more than 120 000 clinical research studies conducted in more than 175 countries.¹⁷ Originally created to increase public awareness of clinical trials and improve the conduct and monitoring of research,¹⁸ the registry now serves as a mandatory repository for information on most clinical studies conducted under US regulatory auspices.^19,20 In addition, registration with ClinicalTrials.gov or another comparable registry is a prerequisite for publishing study results in many peer-reviewed journals.²¹

A systematic evaluation of the database has been hampered by a lack of access to the complete, annotated data. Members of the Clinical Trials Transformation Initiative bridged this gap by creating a high-quality, publicly available, searchable database of the information contained in ClinicalTrials.gov—the database for Aggregate Analysis of ClinicalTrials.gov (AACT).²² The addition of oncology and other disease-specific search capabilities enhance the utility of AACT.

By leveraging information now available through the AACT database, we present an initial snapshot of oncology research, establishing a necessary and timely baseline to inform health policy decisions and allow the development of metrics meant to maximize the return on the clinical research investment and on health. By elucidating the characteristics and outlining differences from other areas of medicine, we hope to facilitate a national discussion regarding the unique aspects of oncology research and areas in which reorientation is critically needed. The ultimate goal is to ensure that patients receive the best care possible by advancing the development of evidence.

We downloaded records of 96 346 clinical studies registered with ClinicalTrials.gov as of September 27, 2010—a date chosen to coincide with the third anniversary of the enactment of the legal obligation for sponsors to register interventional trials.¹⁹ The data set was then locked, and the relational AACT database was designed in Oracle to facilitate aggregate analysis.

The oncology study data set (Figure 1) was created using disease condition terms (both Medical Subject Heading [MeSH] and non-MeSH) provided by the persons entering data and additional condition MeSH terms generated by a National Library of Medicine algorithm. Terms from both the 2010 MeSH thesaurus²³ and non-MeSH disease conditions that appeared in at least 5 studies were reviewed and annotated by oncologists at the Duke Cancer Institute, Durham, North Carolina. Terms were annotated according to their relevance to oncology (as relevant or nonrelevant). A total of 1443 MeSH identifiers corresponding to 546 unique MeSH terms (18 491 MeSH identifiers reviewed) and 436 non-MeSH terms (1220 reviewed) were annotated as relevant for the oncology domain; studies with at least 1 term annotated as relevant were included in the initial data set (n = 9047). This classification method was evaluated for false-positive and false-negative findings; trials with conflicting MeSH terms (both relevant and nonrelevant terms for the same trial) were manually reviewed. The detailed methods used to develop this specialty data set have been described elsewhere.²⁴

After identifying the oncology data set, each trial was subcategorized according to cancer type based on its brief title. If the category was unclear, conditions and keywords were assessed, or, if necessary, the full study record at ClinicalTrials.gov was reviewed. During this process, 105 trials (1.2%) deemed not to be oncology related were excluded, reducing the sample to 8942. Trials that included 3 or more cancer types were grouped into a “general” category because their nonspecific nature could interfere with reliability of analyses if included in calculations for every disease covered.

Oncology trials were examined on 10 dimensions: (1) number of arms, (2) masking, (3) randomization, (4) phase, (5) comparator arms, (6) purpose of intervention, (7) enrollment status, (8) location of trial sites, (9) sponsor, and (10) completion status. The results were compared between oncology and nononcology trials and across cancer types. Descriptors of study location were reported by region and then grouped as within North America exclusively, outside North America exclusively, and in both locations. The clinical trial portfolio by cancer type was compared with the relative incidence and mortality for that cancer type.²⁵ Funding source was derived from submitted information on lead sponsors and collaborators using the classification of sponsoring agencies provided by the National Library of Medicine. A study was classified as industry funded if the lead sponsor was from industry or a collaborator was from industry and no collaborators were from the National Institutes of Health (NIH) or National Cancer Institute (NCI). A study was classified as NIH/NCI funded if an NIH or NCI agency was either the lead sponsor or collaborator and the lead sponsor was not from industry. Funding for remaining studies was classified as “other.”

Descriptive statistics were used to characterize the trials. When the interventional model was “single-group” or the number of study arms recorded as “1,” the value of allocation (if missing) was assigned as “nonrandomized” and the value of blinding was assigned as “open.” Unless otherwise noted, missing values were excluded when calculating descriptive statistics. The Pearson χ² test and Wilcoxon rank sum test were used to compare, respectively, categorical and continuous characteristics of trials in oncology with those in other specialties. Correlations between (1) mortality rate or (2) incidence rate and number of clinical trials by cancer type were calculated using the Spearman rank correlation statistic.

Of the 40 970 studies registered with ClinicalTrials.gov in the study period, oncology trials accounted for 21.8% of all trials (n = 8942) and constituted the single largest clinical discipline, followed by mental health (9.0%), infectious disease (8.3%), diabetes mellitus (6.1%), and cardiology (5.7%). Oncology studies were significantly more likely to be single arm (62.3% vs 23.8%; P < .001), open label (87.8% vs 47.3%; P < .001), and nonrandomized (63.9% vs 22.7%; P < .001) (Table). Oncology trials were also smaller than those of other clinical specialties, with median accrual of 51 vs 72 patients (P < .001) in other disease states. Among trials reporting phase, early-phase trials (those with phase 1 and/or 2 components) were more common in oncology than in other specialties (83.0% vs 51.6%; P < .001). Further comparisons between oncology and other medical specialties have found that oncology trials were more likely to have ongoing recruitment (60.7% vs 42.3%; P < .001) and were less likely to report completion of trials (10.7% vs 29.9%; P < .001) as of September 27, 2010.²⁶

Figure 2 shows the number of trials in common cancer types in association with their relative incidence and mortality in 2010. There was moderate correlation between the number of clinical trials conducted in a given cancer subtype and the associated incidence and mortality of those cancers (Spearman rank correlation coefficient, 0.56 [P = .04] and 0.77 [P = .001], respectively). Lung cancer has the highest incidence, with 14.5% (n = 222 520) of all new diagnoses, and the highest mortality, accounting for 27.6% (n = 157 300) of all cancer deaths in 2010,²⁵ but it is the focus of only 9.2% (n = 824) of cancer trials. Bladder cancer also had a low representation, at 1.1% (n = 100) of all trials despite incidence and mortality rates of 4.6% (n = 70 530) and 2.6% (n = 14 680), respectively. Meanwhile, the representation of breast cancer is proportionate to its incidence, despite a low ratio of incidence to mortality. The representation for lymphoma, at 6.6% (n = 590), is higher than either its incidence or mortality (4.8% [n = 74 030] and 3.8% [n = 21 530], respectively).

Selected trial attributes among the top 10 cancer subtypes by mortality in 2010 are shown in eTable 1). Treatment-oriented trials predominate in the data set related to the legal obligation to register interventional trials.¹⁹ However, trials vary in orientation; 1 in 4 registered breast cancer trials evaluate supportive care (9.6%), diagnosis (8.5%), or prevention (6.7%), whereas these same categories account for less than 1 in 10 pancreatic cancer trials.

For any given cancer subtype, the relationship between the spectrum of available treatments and the design of ongoing trials is also inconsistent. Figure 3 shows 5 cancer types with wide variations in 5-year survival.²⁷ Among these cancers, the proportion of trials that are single arm, open, randomized, or phase 3 does not consistently vary with the severity of the disease (eTable 2). As an example, although the efficacy of available treatment options for breast cancer is much higher than for lung cancer, the proportion of single-arm trials is nearly the same. Furthermore, while there is variation in mean trial size, with a range from 413 patients in prostate cancer trials to 82 in lymphoma trials, the median trial size does not show a similar distribution (50 patients for prostate cancer vs 45 in lymphoma). In half of the cancer types presented, the majority of trials were single site rather than multisite.

TheTable shows that 65.1% of oncology trials include a North American study site, with 34.9% conducted purely in other regions. Among sites outside North America, there is wide geographic dispersion. When trials were examined by cancer subtype (eTable 2), less than 50% of trials were conducted only in North America for 5 of 10 subtypes.

Across all oncology clinical trials, 41.8% of trials were funded primarily by industry, 15.3% by government, and 42.9% by other funders (Table). The “other” category consisted primarily of academic institutions, cooperative groups, and foundations. There are 1608 separate entities listed in this category. The breakdown across nononcology trials was similar, with 47.1% of trials funded primarily by industry, 6.8% by government, and 46.1% by other funders. When compared by funding source, oncology trials were similar in the percentages of blinded trials (10.8%, 13.2%, and 13.8% for industry, other funders, and government, respectively), single-arm trials (62.2%, 61.1%, and 66.0%), and trials expected to enroll more than 100 patients (25.0%, 29.0%, and 26.9%). The percentages of phase 3 and 4 trials for the 3 funding sources were 14.9%, 20.0%, and 11.5%, respectively.

Oncology trials are predominantly early-phase studies that evaluate surrogate end points and are small, single arm, and open label. This orientation toward less robust design differs significantly from trials in other areas of medicine. Despite a wide variation in treatment options and survival between cancer types, the proportion of small, single-arm studies does not vary significantly between cancer types, and there is only moderate correlation between the number of trials for a given cancer type and relative incidence or mortality.

It is not clear whether these attributes of cancer research can be justified by the underlying differences in biology, treatment patterns, and mortality. Although the role of industry is often pointed to as a reason for many of the trial characteristics, government and other sponsors fund more than half of the research performed across oncology clinical trials. As the costs of cancer care in the United States continue to rise, reaching $125 billion in 2010, with 70% spent on agents approved within the past decade,^28,29 we must ensure that the research being done will adequately answer the critical clinical questions. Similarly, because the United States accounted for more than half of the $217 billion spent worldwide on direct cancer care in 2009,³⁰ the majority of data being generated from trials represented in ClinicalTrials.gov should be relevant to US populations. More than one-third of all oncology trials do not include a North American site, and the percentage of patients accrued in the United States is even less.

The relative abundance of small, early-phase, single-arm, and open-label studies in oncology can be partially explained by a few selection pressures. First, accelerated approval was embraced by the US Food and Drug Administration (FDA) in 1992 to improve access to treatments for life-threatening diseases³¹ and subsequently expanded to cover applications for new molecular entities in oncology.³² As part of the accelerated approval process, early-phase trials using surrogate end points are often submitted to the FDA. This results in variations in the number of phase 3 trials that are performed after approval. Second, regulations support the off-label use and reimbursement of antineoplastic agents. Because postmarketing evidence requirements for off-label use are less stringent than those for FDA approval, evidence development is inconsistent in this space and frequently ends at phase 2.^5-7

Although these policies can be defended, we must ensure that the resulting research can answer the pressing questions in oncology. There is an inherent tension between the desire to use new, life-saving treatments and the imperative to develop the evidence base that patients, clinicians, regulatory agencies, and advocacy groups need to make sound decisions. Unfortunately, the high prevalence of small trials that lack comparator arms, rely on historical controls, and lack randomization limit the ability to assess the evidence supporting specific treatments through systematic reviews and comparative effectiveness research. These problems were highlighted during recent discussions regarding sipuleucel-T (Provenge; Dendreon) and bevacizumab (Avastin; Genentech/Roche) when the reliability of early-phase trials that use surrogate end points was called into question.^33-35

As the national debate regarding research priorities continues to unfold, the research community must take into account competing priorities, the importance of particular research questions, the urgency of disease, and the availability of trials across geographic regions and disadvantaged populations. Although discovering breakthrough treatments must be a priority, understanding and improving the use of existing treatments is likewise essential. Nations such as the United Kingdom³⁶ attempt to coordinate and manage their approach to clinical research to align it better with public health priorities. The United States has not made similar progress.

Through this work, we hope to stimulate a discussion of the present state of oncology research, one that addresses questions of how best to distribute constrained resources to achieve maximum impact. Clinical research requires substantial financial and intellectual resources from industry, academia, foundations, and the federal government, along with the personal investment of thousands of patients with cancer. Are modifications appropriate, and, if so, how and to what extent?

The optimal collection of oncology trials is not known. However, some key themes have emerged from this work. First, we must ensure that the ClinicalTrials.gov database is as complete, up to date, and useful as possible. Second, there is a need for a more nuanced approach to clinical trial design. Although open, single-arm, nonrandomized trials that rely on historical controls are appropriate for some cancer types and settings, it is difficult to argue that they should be the norm, especially as more agents enter the market. The lack of variation in design by funding status and cancer type is of particular interest. One might anticipate that cancer types with more developed treatment paradigms, longer overall survival, and government trials would be more likely to have clinical trials that are blinded, later-phase, and larger, but this association was not consistently seen in the data. There should be a regulatory push to match the trial design to the indication and to identify appropriate transition points in the drug development process when trial designs should mature in line with changes in survival by tumor type. Attributes of clinical trials for a given cancer type should evolve in areas such as phase and size to ensure that research reflects the state of the treatment landscape and of the science.

Unfortunately, there is no normative standard because the clinical research needs in areas such as cardiology or orthopedic surgery differ from those in oncology owing to the acuity of the disease and the quickly evolving treatment paradigm. The first step in moving the field forward is to use the curated AACT database to elucidate critical attributes of existing trials. Our vision is to then use it to monitor ongoing clinical research, maximizing its impact through a consortium of sponsors, researchers, policymakers, and advocacy organizations. Metrics must be developed that reflect a thoughtful balance between competing priorities. To date, however, attempts by the project team to develop metrics have been stymied, in part because of the poor reporting standards referenced earlier. Metrics should not be developed in a vacuum, because critical data are needed to inform the agenda, such as the accrual success of different trial designs and the choice of appropriate end points. For instance, advocating for new trial networks and increased randomization would be inappropriate if doing so would undermine accrual to trials. The project team has found it to be difficult to gather critical data on both ongoing and completed trials, such as accrual rates, despite coupling publicly available data with direct-to-investigator surveys. The rate of negative results that go unpublished further complicates analytic efforts. Improved reporting will help to overcome these issues. Establishing rigorous metrics and ensuring consistent reporting standards in the ClinicalTrials.gov database will foster collaboration across borders to answer globally directed scientific questions and to provide study results that are applicable across patient populations and research communities.

As noted elsewhere by Califf and colleagues,²⁶ there are significant limitations to the ClinicalTrials.gov data set. There are limits to the registry's completeness, because registration is required only for studies that had been initiated as of October 2007 and met the following criteria: (1) interventional study of drugs, biologics, or devices; (2) trial phase 2, 3, or 4; (3) inclusion of at least 1 US site or trial conducted under an investigational new drug application or investigational device exemption. Missing data, the medical sophistication of persons entering data, the combination of ambiguous terminology and free-text input options, and an unrealistic implementation timeline imposed by Congress in the late 1990s³⁷ have complicated analysis efforts. The inability to differentiate the trials designed to support accelerated approval further complicate a nuanced understanding of the data.

The lack of a standard ontology is also a major concern. On generating a frequency table from the outcomes field, we identified more than 25 000 outcomes across oncology trials that occurred only once or twice. This not only presents enormous challenges in data analysis but more importantly it limits the ability of clinicians and patients to synthesize available evidence and reach reliable conclusions. Nonetheless, the use of ClinicalTrials.gov and the AACT database to approximate the composition of cancer clinical trials is a valid first step. As data elements improve, so will understanding. We must iteratively improve this resource.

The data contained in the ClinicalTrials.gov data set afford a remarkable opportunity to better understand clinical research in oncology. In the midst of an era marked by the reorganization of the cooperative group system and implementation of health reform legislation that includes establishment of the Independent Payment Advisory Board and the Patient-Centered Outcomes Research Institute, it is essential to prioritize research questions appropriately and to understand the ideal mix of trials so as to maximize the generation of actionable evidence. Accurately characterizing the state of clinical research is the first step toward an effective leveraging of limited resources. Subsequent insights and metric development will allow us to monitor the activity and advance new approaches. Future analyses of the AACT data set will include more granular analyses of specific cancer types and sponsors to provide further insights.

Correspondence: Amy P. Abernethy, MD, Duke Cancer Care Research Program, 25165 Morris Bldg, Campus Box 3436, Durham, NC 27710 (amy.abernethy@duke.edu).

Accepted for Publication: January 31, 2013.

Published Online: April 29, 2013. doi:10.1001/jamainternmed.2013.627

Author Contributions:Study concept and design: Hirsch, Califf, Tasneem, Dilts, and Abernethy. Acquisition of data: Hirsch and Tasneem. Analysis and interpretation of data: Hirsch, Cheng, Horton, Chiswell, Schulman, Dilts, and Abernethy. Drafting of the manuscript: Hirsch, Califf, Horton, Dilts, and Abernethy. Critical revision of the manuscript for important intellectual content: Califf, Cheng, Tasneem, Horton, Chiswell, Schulman, Dilts, and Abernethy. Statistical analysis: Hirsch, Cheng, Horton, and Chiswell. Obtained funding: Califf. Administrative, technical, and material support: Califf, Tasneem, Schulman, Dilts, and Abernethy. Study supervision: Hirsch, Schulman, Dilts, and Abernethy.

Funding/Support: Financial support for this project was provided by grant U19FD003800 from the FDA, awarded to Duke University for the Clinical Trials Transformation Initiative.

Previous Presentations: The results of this study were presented at the American Society of Clinical Oncology annual meeting; June 5, 2011; Chicago, Illinois; and the Society of Clinical Trials meeting; May 21, 2012; Miami, Florida.

Additional Contributions: Jean Bolte, project leader of the Clinical Trials Transformation Initiative, provided assistance with this project. Suanna Bruinooge at the American Society of Clinical Oncology reviewed the manuscript, and Jonathan McCall provided editorial assistance as part of his regular duties as a medical editor employed by the Duke Clinical Research Institute.

This article was corrected for errors on August 6, 2013.