Well-researched and methodologically sound study protocols are important for the credibility of randomized clinical trials (RCTs).1 This is true for the main analysis and subgroup analyses.2 A 2014 study3 of RCT protocols approved by ethics committees between 2000 and 2003 found that almost 30% of protocols specified at least 1 subgroup analysis. However, most of them lacked essential details, such as the definition of subgroup variables, scientific rationales, hypotheses, or a description of statistical methods. In the present study, we compared these findings with 2 more recent samples of RCT protocols approved in 2012 and 2016 to assess the prevalence and reporting quality of planned subgroup analyses over time. In addition, we determined the proportion of planned subgroup analyses based on molecular and genetic markers.
This cross-sectional study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. Approval by the ethics committee of Northern West and Central Switzerland and informed consent were waived because the study did not involve patients or the public in the design, conduct, reporting, or dissemination plans of the research.
This study uses data from 3 retrospective cohorts of RCT protocols approved between 2000 and 2003,3 2012, and 2016.1 The examined protocols were approved by research ethics committees in Switzerland, Germany, and Canada. They constitute random samples of all approved RCT protocols at participating ethics committees. Investigators trained in clinical research methods (MSc or PhD) recorded, independently and in duplicate, RCT characteristics and details about subgroup analyses.1,3 Disagreements were resolved by discussion and consensus. We descriptively summarized the characteristics of the 3 cohorts focusing on the planning of subgroup analyses in RCT protocols in November and December 2020; comparative statements are not inferential. The present study is 1 of 5 prespecified subprojects of the Adherence to SPIRIT Recommendations (ASPIRE) study.1
This study included 894 protocols approved between 2000 and 2003, 257 protocols approved in 2012, and 292 protocols approved in 2016. At all 3 time points, approximately one-third of RCT protocols included plans for at least 1 subgroup analysis (2000-2003: 252 [28.2%]; 2012: 93 [36.2%]; 2016: 96 [32.9%]) (Table 1). At each time point, RCT protocols planning subgroup analyses were more frequently industry sponsored, had a multicenter design, and had a larger sample size than RCT protocols without planned subgroups. Subgroup analyses were particularly frequent in protocols of oncology and cardiovascular RCTs. The number of subgroup analyses per study, although frequently not reported, likely increased over time (2000 to 2003: median, 3 [IQR, 1-6]; 2012: median, 6 [IQR, 4-23.5]; 2016: median, 6 [IQR, 3-13]) (Table 2). The most frequent subgroup defining variables used in 2012 and 2016 (not assessed in the oldest sample) were age (2012: 44 of 93 [47.3%]; 2016: 42 of 96 [43.7%]) and sex (2012: 37 of 93 [39.7%]; 2016: 38 or 96 [39.5%]). Molecular or genetic markers were subgroup-defining variables in 13 of 93 (14.0%) RCT protocols approved in 2012 and 16 of 96 (16.7%) RCT protocols approved in 2016. The reporting of subgroup-specific hypotheses increased over time (2000 to 2003: 17 of 252 protocols [6.7%]; 2012: 9 of 93 protocols [9.7%]; 2016: 16 of 96 protocols [16.7%]) as did the number of plans that included a hypothesis sufficiently detailed to anticipate a direction of effect (2000 to 2003: 10 of 252 protocols [4.0%]; 2012: 9 of 93 protocols [9.7%]; 2016: 16 of 96 protocols [14.7%]). At all 3 time points, approximately one-third of subgroup analysis plans specified a statistical test for interaction (2000 to 2003: 87 of 252 protocols [34.5%]; 2012: 31 of 93 protocols [33.3%]; 2016: 26 of 96 protocols [27.1%]).
The proportion and characteristics of RCT protocols with planned subgroup analyses appeared stable over time. Although the increasing proportion of hypothesis-supported subgroup analyses is encouraging, basic scientific principles, such as researching prior knowledge, limiting the number of analyses, and using appropriate statistics, continue to be violated in the majority of RCT protocols with planned subgroups. This is remarkable given the abundance of methodological guidance available.2,4 Study limitations include the poor reporting of subgroup analysis plans in some trial protocols and the lack of access to statistical analysis plans developed in later phases of trials. Considering the increasing importance of subgroup analyses to inform precision medicine,5,6 investigators and regulators should pay more attention to the methodological quality of subgroup analysis plans.
Accepted for Publication: August 26, 2021.
Published: October 27, 2021. doi:10.1001/jamanetworkopen.2021.31503
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Taji Heravi A et al. JAMA Network Open.
Corresponding Author: Ala Taji Heravi, MSc, Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, and University of Basel, Spitalstrasse 12, CH- 4031 Basel, Switzerland (ala.tajiheravi@usb.ch).
Author Contributions: Ms Taji Heravi and Dr Briel had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Ms Taji Heravi, Drs Gryaznov and Schandelmaier contributed equally.
Concept and design: Taji Heravi, Schandelmaier, Kasenda, Briel.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Taji Heravi, Gryaznov, Briel.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Taji Heravi, Schandelmaier.
Obtained funding: Briel.
Administrative, technical, or material support: Taji Heravi, Gryaznov, Briel.
Supervision: Kasenda, Briel.
Conflict of Interest Disclosures: Dr Kasenda is currently employed by iOMEDICO AG, Freiburg, Germany and reported receiving personal fees from Roche, Riemser, and Astellas outside the submitted work. Dr Gryaznov reported being currently employed by Idorsia Pharmaceuticals. Dr Briel reported receiving grants from the Swiss Federal Office of Public Health. No other disclosures were reported.
Funding/Support: The Adherence to SPIRIT Recommendations (ASPIRE) study was supported in part by the Swiss Federal Office of Public Health. This substudy did not receive any specific funding.
Role of the Funder/Sponsor: The Swiss Federal Office of Public Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We are grateful to Prof Doug Altman (University of Oxford) who was instrumental in developing the initial concept of the ASPIRE study and who sadly passed away before it came to fruition. We thank all participating research ethics committees from Germany (Freiburg), Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St. Gallen, Frauenfeld, Zurich), Canada (Hamilton), and the UK (National Health Service Health Research Authority) for their support and cooperation.
Additional Information: All participating ethics committees were project partners.
1.Gryaznov
D, Odutayo
A, von Niederhäusern
B,
et al. Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to Spirit Recommendations (ASPIRE) study and associated projects.
Trials. 2020;21(1):896. doi:
10.1186/s13063-020-04808-yPubMedGoogle ScholarCrossref 2.Schandelmaier
S, Briel
M, Varadhan
R,
et al. Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses.
CMAJ. 2020;192(32):E901-E906. doi:
10.1503/cmaj.200077PubMedGoogle ScholarCrossref 3.Kasenda
B, Schandelmaier
S, Sun
X,
et al; DISCO Study Group. Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.
BMJ. 2014;349:g4539. doi:
10.1136/bmj.g4539PubMedGoogle ScholarCrossref 6.Yang
JC-H, Wu
Y-L, Schuler
M,
et al. Afatinib versus cisplatin-based chemotherapy for EGFR mutation-positive lung adenocarcinoma (LUX-Lung 3 and LUX-Lung 6): analysis of overall survival data from two randomised, phase 3 trials.
Lancet Oncol. 2015;16(2):141-151. doi:
10.1016/S1470-2045(14)71173-8PubMedGoogle ScholarCrossref