Characteristics and Research Waste Among Randomized Clinical Trials in Gastric Cancer

This cross-sectional study examines the characteristics and presence of research waste, defined as unpublished data, inadequate reporting, or avoidable design limitations, among randomized clinical trials in gastric cancer.


Introduction
Gastric cancer (GC) is the fifth most common human malignant tumor, ranking third in cancer-related mortality. 1 More than 1 million GC diagnoses are made globally each year. 2 To improve the prognosis of patients with GC, increased numbers of randomized clinical trials (RCTs) have been carried out to explore new and potentially more effective treatments. In the past 20 years, these RCTs have provided high-level medical evidence for the clinical diagnosis and treatment of GC. Based on these findings, postoperative recovery and long-term survival of patients with GC have improved significantly. [3][4][5][6] Although RCTs provide high-level evidence and have made important contributions to the advancement of treatment, research waste is inevitably a major challenge to evidence-based medicine. This means that wasteful RCTs used resources while the risks to participants increased.
Chapman et al 7 found that 85.2% of surgical-related RCTs had research waste. This waste may occur at any stage of the research cycle. First, RCTs may be wasteful owing to avoidable design flaws. For example, improper implementation of randomization or blindness procedures is associated with decreased credibility of research results. Waste may also be associated with a failure to consider previous evidence. Second, if results were not published for any reason, the research was also wasted. Third, published RCTs may be considered wasteful owing to insufficient reporting, making it difficult or impossible to use and repeat research reports. Research wastefulness may ultimately mean that RCT results cannot be adopted in the clinical practice guidelines of the relevant field.
Currently, to our knowledge, research waste has not been explored for GC RCTs. With increasing incidence of treatment-resistant GC subtypes, 8 it is important to develop more RCTs aimed at the development of new and effective treatments for GC. Minimizing research waste is essential to ensure that new treatments are appropriately, safely, and effectively translated into clinical practice.
We completed one of the most comprehensive analyses to date of GC RCTs, focusing on characteristics and components of research waste among GC RCTs in the past 20 years to identify potential targets for improvement. In addition, we explored whether published RCTs were referenced in guidelines and whether relevant prospective data were reused.

Methods
Fujian Medical University Union Hospital Ethics Committee determined that because this crosssectional study was a review of registered RCTs, it was exempt from research ethics approval and the requirement for informed consent. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Design and Data Sources
The data used in this study were derived from ClinicalTrials.gov, a publicly available online trial registry. 9 We searched the ClinicalTrials.gov database using the keyword gastric cancer on a single specific definition for this value. 7,[11][12][13][14][15][16] In this study, we predefined RCTs with sample sizes smaller than the smallest 25% samples as small-sample RCTs. Because the incidence of GC is highly variable by region and culture, with higher incidence rates overall in Asian countries, we classified RCTs into Asian and non-Asian trials based on the location of the principal investigator (PI). A positive RCT was defined as an RCT in which the primary end point had a statistically significant difference in favor of the experimental group. 17

Status of Publication
To determine publication status, we performed searches using the ClinicalTrials.gov identifier (National Clinical Trial [NCT] number), name of the PI, and several keywords related to the RCT on PubMed and Scopus. We then checked the manuscript's information, including interventions, study population, and dates of recruitment, to further confirm publication status. If no corresponding manuscripts from PubMed and Scopus were available, we contacted the corresponding PI to further determine publication status. When no response was received, the RCT was determined to be unpublished by default. A trial was considered to be published if a full-text manuscript (available in print or online) was identified in a peer-reviewed journal. 7 The search was performed by 2 independent investigators (M.L. and J.-B. W.). The last search was conducted on June 30, 2020.

Reporting Adequacy Assessment
The assessment of the adequacy of reporting for each manuscript was performed using the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline statement. 18,19 There were 37 items for each manuscript involving pharmacological interventions and 40 items for each manuscript involving nonpharmacological interventions. 18,19 Therefore, to conform to the CONSORT statement, we divided RCTs into pharmacological interventions and nonpharmacological interventions, with 1 point added to the reporting adequacy score for each item. A single investigator (J.-X.L.) downloaded all manuscripts and supplementary correspondence materials. Then, using Adobe Acrobat Pro PDF software version XI Pro (Adobe), which could transform PDF documents into Word documents, the single investigator deleted the author and journal information and reformatted the text to reduce biases. Finally, the files were printed and 2 independent investigators (B.-B.X. and H.-L.Z.) reviewed and scored all manuscripts independently according to the CONSORT 2010 checklist. Discrepancies were discussed and resolved through consensus after every 3 manuscripts reviewed. Because the CONSORT reporting guideline statement has many items and each item was assigned a point, the interrater agreement was not recorded. The full-text manuscript and correspondence supplementary materials were reviewed to determine whether a protocol was available. 7 Manuscripts that met 75% of items (ie, 28 of 37 pharmacological items or 30 of 40 nonpharmacological items) were considered to be reported adequately, as previously described. 7

Design Flaw Assessment
Using the Cochrane tool, 20 2 independent investigators (B.-B.X. and H.-L.Z.) reviewed masked manuscripts and assessed the risk of bias, including selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases. Each item was determined to be low risk, unclear, or high risk. Discrepancies were discussed and resolved through consensus after every 3 manuscripts reviewed. Items with an unclear risk of bias were considered to be high-risk items in the statistical analyses. This was done because unclear descriptions of key methods are associated with judgments of an RCT's ability to inform practice. Additionally, the presence of a relevant systematic review, or justification as to why one was not necessary in novel settings, was also assessed. This had to be cited in the full-text manuscript and be considered capable of informing the necessity of carrying out the RCT to be counted as present. Articles that presented 1 of the previously listed biases or lacked a relevant systematic review citation were considered to have an avoidable design flaw.

Whether Referenced in Guidelines and Reuse of Prospective Data
For each published RCT, we first searched articles that cited the published RCT in the Google Scholar databases. 21 Subsequently, 2 independent investigators (B.-B.X. and H.-L.Z.) manually checked each article that referred to the RCT to determine whether treatment or practice guidelines were present.
We also assessed whether prospective data were reused for post hoc analysis (ie, data from the RCT were reanalyzed for outcomes other than preset primary or secondary end points). [22][23][24] We assumed that the post hoc analysis always cited the original RCT.

Outcomes
The primary outcomes were to elaborate characteristics of RCTs in the past 20 years and to explore research waste (ie, nonpublication, inadequate reporting, and avoidable design flaws). The secondary outcomes were to explore potentially modifiable research waste. Additionally, we determined whether the published RCTs were referenced in guidelines and explored the reuse of prospective data. Publication of an RCT is the precondition for assessments of report adequacy and design flaws. Therefore, in the analyses of research waste, we included all published RCTs and excluded RCTs completed after June 2016 without publication.

Statistical Analysis
Differences between groups in categorical variables were compared using the χ 2 or Fisher test (if the number of samples was <5). 25 A second-order polynomial equation was fitted to the data by a multiple regression method. Simple and multivariable logistic regression models were used to determine independent risk factors associated with research waste. Variables with a value of P < .05 in the simple analysis were subsequently included in a multivariable analysis. All statistical analyses were performed using SPSS statistical software version 18.0 for Windows (IBM) and R statistical software version 4.0.2 (R Project for Statistical Computing). P values < .05 were considered statistically significant, and all tests were 2-sided. Data were analyzed from August through December 2020.

Characteristics of RCTs in the Past 20 Years
From January 2000 to December 2019, there were 306 clinical trials that met inclusion criteria that were retrieved. After we excluded 33 nonrandomized clinical trials, 3 trials that were not phase 3 or 4 trials, 9 trials not related to GC, and 1 duplicated trial, 262 GC RCTs were included in the analysis. participants. Therefore, we defined the RCTs with sample sizes of less than 200 individuals as small RCTs. There were 77 RCTs (29.4%) with external funding ( Table 1).  with publication status, RCTs developed in non-Asian regions or with multicenter designs or external funds were more likely to be published (eTable 1 in the Supplement).

Referenced in Guidelines and Reuse of Prospective Data
Among 81 published RCTs, 50 trials (61.7%) reported positive results, and 31 RCTs (38.3%) reported negative results. We excluded 10 RCTs that were published in 2 recent years (ie, 2019 and 2020) and found that 35 trials (49.3%) were referenced in corresponding guidelines. Trials that included 200 participants or more and reported positive results were more likely to be referenced in guidelines (eTables 5 and 6 in the Supplement). In addition, 18 RCTs (22.2%) had prospective data that were reused. Trials developed in non-Asian regions or that had multicenter designs or positive results were more likely to have prospective data that were reused (eTables 7 and 8 in the Supplement).

Discussion
In this cross-sectional study, the characteristics of 262 GC RCTs in the past 20  design, a greater number of patients included, and external funding were associated with decreased odds of research waste.
The best way to minimize bias when evaluating novel treatments in health care is through RCTs. 26 The degree of disease burden determines which RCTs should be performed to improve the   29 New research should not be conducted unless the questions to be resolved at the beginning cannot be answered satisfactorily with existing evidence. We conducted analyses of the characteristics of GC RCTs carried out in the past 20 years and found that the number of RCTs increased by year and then the trend stabilized. This also suggests that the phenomenon of research waste deserves attention. When high-level medical evidence for surgical treatment, especially minimally invasive surgical treatment for patients with GC, has been established, 30,31 it is not difficult to speculate that systemic treatment will comprise the main research direction in the future. We also found that the proportion of systematic treatment was as high as 67% in the past 5 years, with greater proportions for targeted therapy or immunotherapy. Although this may be an important factor associated with whether breakthroughs can be made in the treatment of GC in the future, attention should be paid to designing RCTs that are compatible with the latest results and development directions to avoid research waste.
Publication of RCTs can provide high-level evidence for practice guidelines for GC. In this study, we found that although there were more RCTs with PIs based in Asia than those with PIs based outside of Asian (which was expected), the latter had a higher proportion of multicenter RCTs and published RCTs than the former. Although these findings may be only statistically associated, the smaller number of GC diagnoses in non-Asian regions may be associated with the development of more multicenter RCTs, with an indirect association with the quality and publication rate of GC RCTs.
These findings suggest that, in the future, scholars should strengthen the collaboration among multiple centers when designing RCTs, especially in Asia.
Additionally, to consider similar research conducted at the same time when designing novel research, design features may also be associated with the size of the effect estimate. For RCTs, allocation concealment, study blindness, and improper use of random allocation sequence methods may be associated with the effect estimate, especially when the study end point is a subjective result. 32 Overall, this study found that 77.8% of RCTs had avoidable design flaws. Common design flaws included flaws in random sequence allocation concealment and evaluator blindness. Trials with design flaws were more likely to be registered earlier and have a smaller sample size. In addition, a lack of external funding was associated with design flaws.
The sufficiency of results reporting is critical for readers to judge whether research applies to their practice. In general, most RCTs (80.2%) were adequately reported. However, it is particularly noteworthy that reporting quality was unsatisfactory for study protocol and randomization.
Approximately 30% of trials provided their study protocols. The lack of a study protocol may reduce readers' understanding of the entire RCT design. The unreasonable use of randomization methods may also be associated with potential bias. We additionally found that in surgical-related RCTs, there was a lack of detailed descriptions of surgical interventions and methods of intervention (22.2%) had prospective data that were reused. Trials that had multicenter designs, were based in non-Asian regions, or had positive results were more likely to have their prospective data reused. We considered that having weak efficacy results (ie, negative results) may be a factor associated with the lack of prospective data reuse. The potential value of these negative data has yet to be exploited fully. This suggests that further efforts to make the best use of available data in combination with specific prospective data collection are warranted.
Conducting an RCT usually requires a great deal of labor and material and financial resources.
Although there has been a similar report, 7 this study is the first, to our knowledge, to focus on clinical trials with regard to a specific field (ie, GC). Most of the findings obtained are original, which may contribute to the proper conduct of clinical trials.

Limitations
This study has several limitations. First, quantifying research waste is complex. Such waste is by no means limited to the 3 elements defined in this study. For example, repeated RCTs that explore low-priority research topics with clear medical evidence could also have research waste. Second, various end points were collected manually, which may be associated with measurement errors.
However, we had 2 independent investigators evaluate each RCT and resolve differences through discussion so that measurement errors could be minimized as much as possible. Third, although ClinicalTrials.gov is a comprehensive clinical trial registry, which accounted for more than 60% of trials in the past 20 years, 38 the World Health Organization has also recognized other countryspecific and region-specific registries, 10 and RCTs in those registries were not included in this study.
Our study presented only a preliminary viewpoint for scientists. Fourth, this study could not further