An Analysis of Contemporary Oncology Randomized Clinical Trials From Low/Middle-Income vs High-Income Countries

Key Points

Question  To what extent do oncology randomized clinical trials (RCTs) reflect the global cancer burden?

Findings  This systematic review–based cohort study of all 694 phase 3 RCTs published from 2014 to 2017 found that trials are conducted predominantly in high-income countries and study cancers that do not match the global burden of disease. Even though RCTs from low-middle and upper-middle–income countries are more likely to study new treatments that benefit patients, these RCTs are published in journals with lower impact factors.

Meaning  Policy makers, research funders, and journals need to address these challenges with a range of measures including building capacity and capability in RCTs.

Importance  The burden of cancer falls disproportionally on low-middle–income countries (LMICs). It is not well known how novel therapies are tested in current clinical trials and the extent to which they match global disease burden.

Objectives  To describe the design, results, and publication of oncology randomized clinical trials (RCTs) and examine the extent to which trials match global disease burden and how trial methods and results differ across economic settings.

Design, Setting, and Participants  In this retrospective cohort study, a literature search identified all phase 3 RCTs evaluating anticancer therapies published from 2014 to 2017. Randomized clinical trials were classified based on World Bank economic classification. Descriptive statistics were used to compare RCT design and results from high-income countries (HICs) and low/middle-income countries (LMICs). Statistical analysis was conducted in January 2020.

Main Outcomes and Measures  Differences in the design, results, and output of RCTs between HICs and LMICs.

Results  The study cohort included 694 RCTs: 636 (92%) led by HICs and 58 (8%) led by LMICs. A total of 601 RCTs (87%) tested systemic therapy and 88 RCTs (13%) tested radiotherapy or surgery. The proportion of RCTs relative to global deaths was higher for breast cancer (121 RCTs [17%] and 7% of deaths) but lower for gastroesophageal cancer (38 RCTs [6%] and 14% of deaths), liver cancer (14 RCTs [2%] and 8% of deaths), pancreas cancer (14 RCTs [2%] and 5% of deaths), and cervical cancer (9 RCTs [1%] and 3% of deaths). Randomized clinical trials in HICs were more likely than those in LMICs to be funded by industry (464 [73%] vs 24 [41%]; P < .001). Studies in LMICs were smaller than those in HICs (median, 219 [interquartile range, 137-363] vs 474 [interquartile range, 262-743] participants; P < .001) and more likely to meet their primary end points (39 of 58 [67%] vs 286 of 636 [45%]; P = .001). The observed median effect size among superiority trials was larger in LMICs compared with HICs (hazard ratio, 0.62 [interquartile range, 0.54-0.76] vs 0.84 [interquartile range, 0.67-0.97]; P < .001). Studies from LMICs were published in journals with lower median impact factors than studies from HICs (7 [interquartile range, 4-21] vs 21 [interquartile range, 7-34]; P < .001). Publication bias persisted when adjusted for whether a trial was positive or negative (median impact factor: LMIC negative trial, 5 [interquartile range, 4-6] vs HIC negative trial, 18 [interquartile range, 6-26]; LMIC positive trial, 9 [interquartile range, 5-25] vs HIC positive trial, 25 [interquartile range, 10-48]; P < .001).

Conclusions and Relevance  This study suggests that oncology RCTs are conducted predominantly by HICs and do not match the global burden of cancer. Randomized clinical trials from LMICs are more likely to identify effective therapies and have a larger effect size than RCTs from HICs. This study suggests that there is a funding and publication bias against RCTs led by LMICs. Policy makers, research funders, and journals need to address this issue with a range of measures including building capacity and capability in RCTs.

Low- and middle-income countries will account for 75% of global cancer deaths by 2030.¹ In this context, there is an urgent need for building collaboration and capacity in these regions. Randomized clinical trials (RCTs) remain the most powerful tool to change clinical practice and improve outcomes. Randomized clinical trials are increasingly multinational, funded by industry, and more likely to use putative surrogate end points and identify modest effect size.^2-5 Despite these aspects, only a minority of new cancer drugs offer substantial clinical benefit for patients.^6-9 This body of work supports the assertion that research priorities are shaped by the pharmaceutical industry in high-income countries (HICs) rather than by population-specific and context-specific needs.

There is increasing recognition that the global cancer research ecosystem needs recalibration to address imbalances across disease settings and between preclinical and clinical efforts.^10-12 Patients enrolled in clinical trials often do not adequately represent the diversity of the global population. A 2013 Cochrane review of 12 340 clinical trials found that 89% of trials and 82% of participants were from HICs.¹³ Although bibliometric analysis has exposed the dominance of HICs in global cancer research, to our knowledge, there are very limited data regarding the extent to which RCT design and results vary based on the economic setting in which they are conducted. It is also unknown whether the current trends of “megatrials” and marginal effect sizes apply to RCTs from low-resource settings. In this study, we analyze all oncology RCTs conducted globally from 2014 to 2017. The objectives were to understand the extent to which trials match global disease burden and how trial methods and results differ across economic settings.

We performed a retrospective systematic review and cohort study to evaluate characteristics of all oncology RCTs published from 2014 to 2017. A structured PubMed literature search used the following terms: Neoplasms/drug therapy [MeSH] OR Neoplasms/radiotherapy [MeSH] OR Neoplasms/surgery [MeSH] OR Neoplasms/therapy [MeSH] OR Neoplasms/transplantation [MeSH], sorted by best match and filtering for phase 3 clinical trials. Studies were included if they were English-language reports of a phase 3 study of any cancer and tested a cancer-directed therapy. There was no minimum sample size. Studies were excluded if they reported only subset or pooled analyses, reported interim analyses, or assessed cancer screening or prevention. Studies of supportive care (ie, antiemetics or growth factors) or integrative medicine (ie, yoga or vitamins) were excluded. As this study included data from published RCTs and did not include any patient-level details, institutional review board approval was not required.

All eligible studies were reviewed using a standardized data abstraction form to capture information regarding authorship, funding, study design, results, and journal of publication. Data abstraction was performed independently by 2 of us (J.C.W. and S.S.). One of us (C.M.B.) performed random duplicate abstraction throughout the process to ensure that data abstraction was of high quality. At completion of data collection, 30 studies were randomly chosen for double review; only 11 of 1020 variables (1%) were found to be discordant with the original assessment. One of us (J.C.D.P.) with extensive experience using the European Society of Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) derived grades for all superiority studies of systemic therapy that met their primary end point.

Studies were classified into country of origin based on the institutional affiliation of the first author. The country of origin was used to further divide studies into income level classifications based on the World Bank income classification.¹⁴ There were no RCTs from low-income countries. Because of a paucity of studies from low-middle–income countries (LMICs) (n = 7), they were combined with upper-middle–income countries (n = 51) and collectively referred to as LMICs; these trials are compared with those from HICs. Global cancer mortality statistics were obtained from the GLOBOCAN database.¹⁵

Descriptive results were generated for the full study cohort. Comparisons were made between studies led by HICs and those led by LMICs. Journal impact factor (IF) was also compared using the IF from 2016, as reported by the Journal Citation Reports Impact Factor.¹⁶ We also compared the primary end point effect size (ie, hazard ratio [HR]) of “positive” superiority RCTs (ie, trials with a statistically significant difference in favor of the experimental group) between HICs and LMICs. Version 1.1 of the ESMO-MCBS was used to derive a grade based on the positive end point for systemic therapy.¹⁷ Grades of A and B (curative setting) and 5 and 4 (palliative setting) were considered to be substantial benefit.

Statistical analysis was conducted from January 20 to 21, 2020, using IBM SPSS, version 26.0 for Windows (IBM Corp). Outcomes were compared using the Pearson χ² test or the Fisher exact test, and independent-samples t tests or the Mann-Whitney test as appropriate. All P values were from 2-sided tests and results were deemed statistically significant at P < .05; no adjustments for multiple comparisons were made.

The search strategy identified 2275 publications. Reasons for exclusion were: subset or pooled analysis (n = 883), not phase 3 RCT (n = 250), not anticancer intervention (n = 217), protocol/interim analysis (n = 134), or additional report of included study (n = 97) (eFigure in the Supplement). The final study cohort included 694 RCTs (list available from corresponding author on request).

Most RCTs (636 [92%]) were led by HICs; 58 (8%) were led by LMICs. A total of 565 RCTs (81%) reported countries that enrolled patients: 238 (42%) enrolled patients from LMICs. Among trials led by HICs, 182 of 509 (36%) enrolled participants from LMICs. Among HICs, the most common leading countries were the US (174 of 636 [27%]), France (64 of 636 [10%]), Germany (62 of 636 [10%]), Japan (59 of 636 [9%]), and the United Kingdom (57 of 636 [9%]). The most common countries among LMICs were China (42 of 58 [72%]) and India (6 of 58 [10%]).

Characteristics of the study cohort are presented in Table 1. The most common cancers studied were breast cancer (121 of 694 [17%]), lung cancer (104 of 694 [15%]), and colorectal cancer (58 of 694 [8%]). The extent to which cancers studied in RCTs align with global cancer mortality is shown in Figure 1.¹⁵ There is general alignment between the proportion of all global cancer deaths and the proportion of all global RCTs for lung cancer (104 RCTs [15%] and 18% of deaths), colorectal cancer (58 RCTs [8%] and 9% of deaths), and prostate cancer (37 RCTs [5%] and 4% of deaths). The proportion of oncology RCTs relative to global cancer deaths is substantially higher for breast cancer (121 RCTs [17%] and 7% of deaths), leukemia (47 RCTs [7%] and 3% of deaths), and lymphoma (43 RCTs [6%] and 3% of deaths); the proportion of RCTs is substantially lower for gastroesophageal cancer (38 RCTs [6%] and 14% of deaths), liver cancer (14 RCTs [2%] and 8% of deaths), pancreas cancer (14 RCTs [2%] and 5% of deaths), and cervical cancer (9 RCTs [1%] and 3% of deaths).

A total of 448 RCTs (65%) were conducted in the palliative setting; 601 trials (87%) tested systemic therapy (207 cytotoxic [34%], 117 tyrosine kinase inhibitor [19%], 49 monoclonal antibody [8%], 35 hormone [6%], 136 multiagent [23%], and 57 other [9%]) and 88 (13%) tested radiotherapy or surgery. A total of 416 RCTs (60%) tested palliative systemic therapies. The most common primary end points were progression-free survival (220 [32%]), overall survival (215 [31%]), and disease-free survival, event-free survival, or relapse-free survival (149 [22%]). A total of 488 RCTs (70%) were supported by industry.

Comparisons of RCT design between HICs and LMICs are presented in Table 1. The primary end point of disease-free survival, event-free survival, or relapse-free survival was more common in HICs than LMICs (142 [22%] vs 7 [12%]; P = .07); response rate was a more common primary end point in LMICs than HICs (9 of 58 [16%] vs 35 of 636 [6%]; P = .003). Trials conducted by HICs were more likely than LMICs to be industry funded (464 [73%] vs 24 [41%]; P < .001).

Details regarding the conduct and results of RCTs are shown in Table 2. The median number of participants across all RCTs was 443 (interquartile range [IQR], 246-718). The median time of accrual was 36 months (IQR, 24-60 months). Forty-seven percent (325 of 694) of all trials met their primary end point. Primary end point results were reported for 607 of 610 (99%) of superiority trials. A total of 262 of 607 superiority trials (43%) met their primary end point (ie, P < .05 for primary end point); among these trials, the median HR was 0.63 (IQR, 0.51-0.74). Among the 166 positive superiority trials of systemic therapy for which an ESMO-MCBS grade could be calculated, 55 (33%) met the threshold for substantial benefit.

Studies in LMICs were smaller than HIC trials (median, 219 [IQR, 137-363] vs 474 [IQR, 262-743] participants; P < .001) and were more likely to meet their primary end points (39 of 58 [67%] vs 286 of 636 [45%]; P = .001) (Table 2). Among superiority trials, the effect size was larger in LMICs compared with HICs (all RCTs: median HR, 0.62 [IQR, 0.54-0.76] vs 0.84 [IQR, 0.67-0.97]; P < .001; “positive” trials: median HR, 0.59 [IQR, 0.43-0.66] vs 0.65 [IQR, 0.52-0.75]; P = .02). The proportion of trials identifying treatments with substantial clinical benefit (ESMO-MCBS grades 4, 5, A, or B) was 48% in LMICs (10 of 21 RCTs) and 31% in HICs (45 of 145) (P = .13).

A total of 686 RCTs (99%) were published in journals with IFs. The median IF for all RCTs was 21 (IQR, 7-27). Studies from LMICs were published in journals with lower median IFs than studies from HICs (7 [IQR, 4-21] vs 21 [IQR, 7-34]; P < .001) (Figure 2). This publication bias persisted when adjusted for whether a trial was positive or negative (median IF: positive LMIC trial, 9 [IQR, 5-25] vs positive HIC trial, 25 [IQR, 10-48]; and negative LMIC trial, 5 [IQR, 4-6] vs negative HIC trial, 18 [IQR, 6-26]; P < .001).

To our knowledge, this is the first study based on a systematic overview of published oncology phase 3 RCTs. Many important findings have emerged. First, the RCT landscape is dominated by investigators in HICs and the diseases studied do not match the global burden of cancer. Gastroesophageal, liver, pancreas, and cervical cancers are substantially underrepresented in RCTs. Second, most RCTs are funded by the HIC-based pharmaceutical industry and focused disproportionally on systemic therapies in the palliative setting. Third, RCTs testing new approaches in surgery and radiotherapy account for only 13% of all RCTs. Fourth, use of putative surrogate end points is pervasive. Fifth, only one-third of positive trials identify a new treatment that is associated with substantial clinical benefit. Sixth, compared with RCTs from HICs, RCTs from LMICs are more likely to be positive and identify a larger magnitude of benefit. Seventh, we have identified a very substantial publication bias; despite being more likely to be positive and having a larger magnitude of benefit, RCTs from LMICs are published in journals with far lower IFs than trials from HICs.

Our study identifies an imbalance in the number of RCTs conducted in HICs and LMICs; we also describe substantial discordance in the cancers studied in these RCTs compared with the global burden of disease. These results are consistent with prior work.^11,18,19 This imbalance reflects a historical colonial approach (prioritizing HICs over LMICs) to global health and is perpetuated by substantial structural barriers to conducting clinical research in low-resource settings. These barriers are compounded by the global cancer funding paradox whereby 5% of global resources for cancer are spent in LMICs, where 64% of global deaths as a result of cancer occur.^20,21

Our results demonstrate that the global research agenda is dominated by the pharmaceutical industry in HICs; this finding is not unexpected given that the primary goal of industry is to maximize profit. There is an urgent need for philanthropic and government funding agencies to recalibrate this imbalance. The marked difference in rates of industry funding may also reflect a reluctance of industry to sponsor studies in countries with less established research infrastructure. A related theme is the practice of investigators from HICs leading a drug registration trial that predominantly enrolls patients in LMICs where the treatment would have no chance of being available after the clinical trial; this in itself represents a distinct form of research parachutism.^22-24

There are substantial barriers to conducting cancer research in LMICs, including limited research infrastructure (ie, research staff, ethics, regulatory, legal and contracts, data capture and analysis, and pharmacy and laboratory accreditation), limited funding opportunities, and the remarkable clinical workloads that make protected time for research nonexistent in most parts of the world.^25-27 Because patients in many LMICs do not have access to universal health coverage, the prohibitive costs of even the standard of care preclude participation in research studies. In many LMICs, trainees receive very limited teaching in principles of critical appraisal or research methodology; this limitation makes it even more difficult to launch investigator-initiated local studies. Despite these barriers, there are encouraging signs of improvement. For example, the National Cancer Grid of India leads a highly successful week-long intensive research methodology workshop for emerging leaders in academic medicine.²⁸ The National Cancer Grid has also leveraged substantial funding to support Indian-led clinical research and is establishing research hubs across the country to provide logistic and methodological support.

Despite barriers to research in LMICs, a series of high-profile RCTs from LMICs published in 2014 and 2015 have changed clinical practice globally.^29-31 Data from the current study suggest that RCTs led by researchers in LMICs may be less prone to one of the greatest pitfalls of contemporary oncology: the overpowered “megatrial” that detects marginal improvements in outcome. Our data show that RCTs from LMICs are smaller, more likely to be “positive,” and are associated with a larger magnitude of benefit. This likely reflects a level of pragmatism in the design of RCTs in LMICs that has been lost in HICs where it is commonplace to adopt a cancer treatment that costs $100 000 per year and extends median survival by a few weeks.

Although the primary objective of our study was to compare and contrast RCTs conducted in HICs and LMICs, a number of important findings have emerged about the overall state of oncology RCTs. Consistent with other reports,^2-5 we found that RCTs are largely funded by industry, testing palliative systemic therapy, using putative surrogate end points (requiring frequent imaging studies that may not be feasible in many settings), and are associated with marginal therapeutic benefits. This is not an encouraging portrait for the global cancer research ecosystem. With the growing interest in high-value cancer care it is imperative that clinical trials raise the bar so that patients have access to new treatments that lead to meaningful improvements in outcome and are of broad value to health systems.

Our data demonstrate that, since the initial report of negative publication bias in oncology in 2003,³² this phenomenon remains a problem in the field. However, the most striking result of our study is the observation of a strong publication bias against studies led by LMICs. Positive trials from LMICs were published in journals with lower IFs than journals that published negative trials from HICs. The bias against nononcology studies from LMICs has been previously described in general medicine^33,34; however, this is the first study, to our knowledge, to identify this phenomenon in oncology. These observations highlight an important form of bias within the current system. This “publication prejudice” needs to be recognized and addressed to ensure a fair and equitable research ecosystem that leads to improvements in health for all global citizens. The American Society of Oncology’s Journal of Global Oncology has provided an important venue for research from LMICs. The recent announcement by ecancer³⁵ that it will publish work only from LMICs will also mitigate some of this bias. However, one of the unintended consequences of having dedicated global cancer journals is that it may relegate any article from a non-HIC to one of these journals with a lower IF. As investigators who work extensively in global oncology, our perception is that many cancer journals are reluctant to publish anything that is not from an HIC; editors of these journals may feel that the Journal of Global Oncology and ecancer offer the mainstream journals an opportunity to reject articles from LMICs. It will be important that the major cancer journals be mindful of their role in supporting research from all countries and ensuring that they publish reports based on scientific merit and human health impact regardless of where the study was conducted.

Our results should be interpreted in light of methodological limitations. Studies were classified into country of origin based on the institutional affiliation of the first author. This is only a surrogate marker for location of the trial and all other authors. The study was restricted to RCTs and therefore may not reflect the entire research ecosystem; however, RCTs shape policy and practice, and for that reason, the findings from this study are highly relevant to patient care. Our study did not include reports in languages other than English, and our search strategy would also have missed RCTs published in nonindexed journals. However, to ascertain the validity of the search strategy process for RCTs in LMICs, we queried ClinicalTrials.gov for phase 3 trials completed from 2014 to 2017. Twelve LMICs were randomly selected; 91% (104 of 114) of studies identified in our ClinicalTrials.gov query were captured by our search strategy, suggesting that our cohort is likely representative of the overall body of RCTs. Finally, given the paucity of RCTs from low-middle income countries, they were grouped with upper-middle–income countries; policy issues may vary widely across the economic continuum.

Our results offer policy-relevant insights into the cancer research enterprise that require attention in the next 5 years. First, there is an urgent need to correct the global research funding paradox; philanthropic and government funding agencies should uphold their moral and scientific duty to support research in LMICs. Second, initiatives to build research infrastructure and human capacity in low-resource settings must be promoted and supported, with a particular emphasis on creating networks of countries with similar context-specific needs (ie, South-South partnerships). Third, the pressing clinical volumes in LMICs need to be recognized and supported with additional health care professionals to allow clinical investigators to develop their own independent research programs. This should be an integral component of health systems strengthening. Fourth, our finding of a publication bias against RCTs from LMICs cannot be ignored. Oncology journals need to recognize their role in promoting and disseminating high-quality research regardless of the country of origin. Increasing LMIC representation on editorial boards is one important first step in this regard. Global oncology-specific journals also play an important role but do not absolve other journals from the responsibility to ensure that high-quality work is published. Fifth, sustained effort is needed to increase RCTs that address new surgical and radiotherapy techniques. Sixth, we make an urgent plea to oncologists and investigators worldwide to recognize the importance and synergies that can emerge from global research efforts that are truly collaborative in spirit, bidirectional, and mutually beneficial. Living and working in very different contexts provides a unique opportunity for clinicians and health systems to learn from each other. With appropriate support, investigators in LMICs may in fact be better positioned than colleagues in HICs to answer fundamental research questions that can substantially improve patient outcomes globally.

This study found that oncology RCTs are predominantly led by HICs, funded by HIC-based pharmaceutical industry, focus on palliative systemic therapy, and study cancers that do not match the global burden of disease. Randomized clinical trials from LMICs were more likely to identify new treatments that benefit patients. Finally, we identified evidence of a substantial publication bias against RCTs from LMICs. Cancer is now recognized as a global disease; these data support the call for capacity building and research support directed at cancer clinical trials in LMICs that are context relevant.

Accepted for Publication: October 16, 2020.

Published Online: January 28, 2021. doi:10.1001/jamaoncol.2020.7478

Corresponding Author: Christopher M. Booth, MD, Division of Cancer Care and Epidemiology, Queen’s University Cancer Research Institute, 10 Stuart St, Kingston, ON K7L 3N6, Canada (booth@queensu.ca).

Author Contributions: Dr Booth had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Wells, Sharma, Gyawali, Pramesh, Booth.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Wells, Sharma, Gyawali, Mukherji, Sullivan, Booth.

Critical revision of the manuscript for important intellectual content: Wells, Sharma, Del Paggio, Hopman, Gyawali, Mukherji, Hammad, Pramesh, Aggarwal, Booth.

Statistical analysis: Wells, Sharma, Hopman, Pramesh, Booth.

Administrative, technical, or material support: Mukherji, Sullivan.

Supervision: Pramesh, Aggarwal, Sullivan, Booth.

Conflict of Interest Disclosures: Dr Wells reported receiving nonfinancial support from Pfizer outside the submitted work. No other disclosures were reported.