Customize your JAMA Network experience by selecting one or more topics from the list below.
Woloshin S, Schwartz LM, Bagley PJ, Blunt HB, White B. Characteristics of Interim Publications of Randomized Clinical Trials and Comparison With Final Publications. JAMA. 2018;319(4):404–406. doi:10.1001/jama.2017.20653
Publication of interim results of randomized trials can be consequential, potentially undermining trial integrity by unblinding or encouraging dropouts and crossovers,1 introducing uncertainty because results based on fewer outcomes are less precise, or overstating true treatment effects as demonstrated primarily in trials terminated early2,3 but also in ongoing trials.4 Nevertheless, because interim results are new and often promising, they may generate substantial interest, which can be misleading if results change. We describe the characteristics of interim publications from ongoing randomized trials and compare their consistency and prominence with those of final publications.
We searched PubMed for randomized trials (between January 1, 2006 and December 31, 2015) with “interim,” “not mature,” or “immature” in the title or abstract, excluding pilot studies, protocols, and cancer trials reporting final primary (progression-free survival) but interim secondary (overall survival) outcomes. To identify final publications, we searched PubMed, ClinicalTrials.gov, and Web of Science through December 1, 2016, and used trial registration numbers, trial and author names, randomized groups, and planned sample sizes to match with interim reports. We emailed authors of interim reports when no final publication was identified (20% [13/66] responded between January and September 2017).
Two reviewers (S.W. and L.M.S.) independently abstracted data, resolving disagreements by discussion. For interim and final publications reporting the same efficacy or safety outcome, we compared trial characteristics and prominence (journal impact factor, Altmetric attention score for news and social media) using paired difference tests (McNemar or signed rank) to generate 2-tailed P values (P < .05 was considered statistically significant) in Stata version 14.2 (StataCorp). In addition, we categorized abstract conclusions (not different, beneficial, or harmful) and compared changes between interim and final publications.
Interim results were reported in 613 of 1267 screened publications. Of these, 72% (n = 442) reported on trials stopped early (for benefit, n = 105; harm, n = 67; futility, n = 224; or other problems, n = 46). The remaining 171 ongoing trials (mostly in oncology, surgery, or cardiology) reported interim efficacy or safety results (Table 1). Forty percent (n = 68) of the publications stated that the interim analysis was protocol specified but half (n = 86) provided no reason.
Final results were published for 61% (n = 98) of the 160 trials for which sufficient time elapsed for publication (ie, >1 year beyond trial registry–specified study completion date). Among the 73 matched pairs reporting the same efficacy or safety outcome, interim analyses had fewer participants (median, 205 vs 295 participants) and shorter follow-up (median, 48 vs 104 weeks) than final analyses (Table 2). Interim and final publications had similar prominence, and most (79%; n = 58) abstract conclusions did not change: 32 consistently concluded the intervention was not different from the comparator and 26 consistently concluded the intervention was beneficial. Twenty-one percent (n = 15) changed: 4 from not different to beneficial, 3 from not different to harmful (or possibly harmful), 6 from beneficial to not different, 1 from beneficial to harmful, and 1 from inconclusive to noninferior (see examples in Table 2). The change from beneficial to harmful occurred because the harm of high-dose imatinib, which reduced event-free survival vs the standard dose, emerged in the final publication.5,6
Many interim publications reported analyses without mention of prespecification or any justification. Frequent nonpublication of final results means true treatment effects often remain unknown. Interim and final publications had similar journal and media prominence, and most reached similar conclusions. However, in 21%, clinicians could have been misled about whether an intervention was beneficial, harmful, or ineffective. Limitations include underestimation of interim publications (full text not searched), the final publication rate (missed publications or misclassified trials eligible for publication), and the extent of interim to final changes (only 1 key outcome analyzed).
Interim publication should be limited to protocol-prespecified analyses performed when enough outcomes occurred for statistical stability and to scenarios least likely to undermine trial integrity (eg, crossovers cannot happen in a trial testing 2 surgical techniques after all patients have had surgery). Routinely including the word “interim” in the title and justifying the reason in the publication would help alert readers to inherent uncertainties. Journals, authors, and funders should commit to making final results accessible by linking interim publications to final reports whenever available.
Accepted for Publication: December 8, 2017.
Corresponding Author: Lisa M. Schwartz, MD, MS, Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH 03756 (email@example.com).
Author Contributions: Drs Woloshin and Schwartz had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Woloshin, Schwartz, Bagley, Blunt.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Woloshin, Schwartz.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Woloshin, Schwartz, White.
Administrative, technical, or material support: White.
Supervision: Woloshin, Schwartz.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Drs Woloshin and Schwartz report having served as medical experts in testosterone litigation and were the cofounders of Informulary Inc, a company that provided data about the benefits and harms of prescription drugs, which ceased operations in December 2016. No other disclosures were reported.
Funding/Support: Drs Woloshin and Schwartz were supported in part by the Agency for Healthcare Research and Quality’s Comparative Health System Performance Initiative under grant 1U19HS024075, which studies how health care delivery systems promote evidence-based practices and patient-centered outcomes research in delivering care.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Meeting Presentation: The results of this study were presented at the Eighth International Congress on Peer Review and Scientific Publication; September 10, 2017; Chicago, Illinois.