A major aim of trial registration is to help identify and deter the selective reporting of outcomes based on the results.1,2 However, it is unclear whether registered outcomes accurately reflect the trial protocol and whether registration improves the reporting of primary outcomes in publications. We evaluated adherence to trial registration and its association with subsequent publication and reporting of primary outcomes.
We conducted a cohort study of all initiated clinical trial protocols approved in 2007 by the research ethics committee for the region of Helsinki and Uusimaa, Finland. Registry records and articles published up to February 2017 were identified using keywords to search trial registries, PubMed, EMBASE, Cochrane Central, Finnish databases (Medic, ARTO, TUHAT), and Google. Trial characteristics and outcomes were extracted in duplicate from each protocol (including amendments), registry record, and publication.
Using descriptive statistics and multivariable logistic regression adjusting for characteristics in Table 1, we determined the prevalence of and variables associated with prospective registration (within 1 month after the trial start date to allow for incomplete start dates and processing delays in the registry); the proportion of trials with at least 1 discrepant primary outcome in the protocol compared with (1) the registry and (2) the publication; and the association between prospective registration and subsequent publication without discrepant primary outcomes compared with the protocol. A 2-sided P value of less than .05 was used for statistical significance, and odds ratios (ORs) with 95% CIs were calculated using Stata/SE (StataCorp), version 12.1.
Discrepancies were defined as (1) a new primary outcome being reported that was not specified as primary in the protocol; or (2) a protocol-defined primary outcome being omitted or downgraded (reported as secondary or unspecified) in the registry or publication. For comparison with registries, we used the primary outcomes defined in the most recent protocol or amendment dated before the initial registration date. For comparison with publications, we used the most recent protocol version regardless of amendment date.
Among 113 trials, 69 (61%) were prospectively registered and 64 (57%) were published. Trials involving drug or biologic interventions and larger sample sizes were more likely to be registered (Table 1).
A primary outcome was not defined in 23 protocols (20%). Discrepancies were found in at least 1 primary outcome defined in the registry for 16 of 69 prospectively registered trials (23%) when compared with the protocol, whereas 9 of 58 published trials (16%) with defined primary outcomes had discrepancies between the publication and the protocol (Table 2). Discrepancies between the protocol and publication were more common in unregistered trials (6 of 11 trials [55%]) than registered trials (3 of 47 [6%]) (P < .001). Only 1 published article acknowledged the changes to primary outcomes.
Prospective registration was significantly associated with subsequent publication (68% of registered trials vs 39% of unregistered trials; adjusted OR, 4.53 [95% CI, 1.12-18.34]) (Table 1). Registered trials were also significantly more likely than unregistered trials to be subsequently published with the same primary outcomes as defined in the protocol (64% of registered trials vs 25% of unregistered trials; adjusted OR, 5.79 [95% CI, 1.42-23.65]).
Clinical trials were often unregistered, unpublished, and discrepant in the reporting of primary outcomes across information sources. Limitations include the unclear generalizability beyond the Finnish jurisdiction and the limited sample size.
Although discrepancies are commonly found between registries and publications,3 which may reflect selective outcome reporting, the rationale is less clear for different primary outcomes appearing between the registry and protocol prior to results being known. Potential reasons for such discrepancies include clerical oversight or intentional suppression from disclosure. The original protocol and amendments should be made publicly available so that editors, peer reviewers, and readers can identify any unacknowledged changes to protocol-defined outcomes in the registry or publication.4,5 The protocol should provide a complete description of the primary outcomes and other key elements of the study plans.6 Amendments should be transparently reported.
Prospective registration was associated with publication and publication without discrepancies in the primary outcomes. Journal editors, regulators, research ethics committees, funders, and sponsors should implement policies mandating prospective registration for all clinical trials. Only with accessible, complete information can interventions be adequately evaluated for patient care.
Accepted for Publication: August 11, 2017.
Corresponding Author: An-Wen Chan, MD, DPhil, Women’s College Research Institute, 76 Grenville St, Room 6416, Toronto, ON, Canada M5S 1B2 (anwen.chan@utoronto.ca).
Published Online: September 11, 2017. doi:10.1001/jama.2017.13001
Author Contributions: Dr Chan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Chan, Hemminki.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Chan, Hemminki.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Chan.
Obtained funding: Chan, Hemminki.
Administrative, technical, or material support: Kitchen, Axentiev, Virtanen, Liu, Hemminki.
Supervision: Chan, Hemminki.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: This project was supported in part by grant MET 117434 from the Canadian Institutes of Health Research Dissemination Events and grant 28356 from the Academy of Finland.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Meeting Presentation: This article was presented at the Peer Review Congress; September 10-12, 2017; Chicago, Illinois.
1.Dwan
K, Gamble
C, Williamson
PR, Kirkham
JJ; Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review.
PLoS One. 2013;8(7):e66844.
PubMedGoogle ScholarCrossref 2.Chan
A-W, Hróbjartsson
A, Haahr
MT, Gøtzsche
PC, Altman
DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
JAMA. 2004;291(20):2457-2465.
PubMedGoogle ScholarCrossref 3.Jones
CW, Keil
LG, Holland
WC, Caughey
MC, Platts-Mills
TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review.
BMC Med. 2015;13:282.
PubMedGoogle ScholarCrossref 4.Zarin
DA, Tse
T, Williams
RJ, Carr
S. Trial reporting in ClinicalTrials.gov—the final rule.
N Engl J Med. 2016;375(20):1998-2004.
PubMedGoogle ScholarCrossref 5.Chan
A-W, Song
F, Vickers
A,
et al. Increasing value and reducing waste: addressing inaccessible research.
Lancet. 2014;383(9913):257-266.
PubMedGoogle ScholarCrossref 6.Chan
A-W, Tetzlaff
JM, Altman
DG,
et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials.
Ann Intern Med. 2013;158(3):200-207.
PubMedGoogle ScholarCrossref