Association of Cardiovascular Trial Registration With Positive Study Findings: Epidemiological Study of Randomized Trials (ESORT)

Trial registration has been proposed to reduce selective publication and outcome reporting, thereby increasing accountability in the conduct of research.¹ Since 2005, policy makers, journal editors, and research funders have increasingly endorsed and mandated trial registration.² However, evidence to support the proposed benefits of trial registration is lacking. Analysis of a select group of randomized trials (RTs) in oncology found that registered and unregistered trials were equally likely to reach conclusions favoring new oncology drugs.³ We conducted a cross-sectional analysis of published cardiovascular RTs to compare RTs reported as registered with those not reported as registered.

An RT was eligible for analysis if it was published on PubMed in December 2012 and focused on a cardiovascular disease, as defined by the International Classification of Diseases. Our research group⁴ has previously reported a detailed description of the process. Briefly, 2 reviewers independently screened all abstracts and full texts. No language restrictions were applied. Study characteristics and methodological characteristics (Table 1) were extracted in duplicate. The International Committee of Medical Journal Editors definition of trial registration was used.¹ The primary outcome was the reported study findings for each trial, categorized as significant positive, nonsignificant, or significant negative. Trials not reported as registered were searched for registry information using the World Health Organization Trials Registry Platform. The χ² test, Fisher exact test, and Wilcoxon rank-sum test were applied as appropriate for analysis.

We identified 4190 abstracts of possible reports of RTs, among which 191 cardiovascular RTs were identified. Of these, 86 (45.0%) were reported as registered. Registry information was found for 6 (5.7%) of the 105 trials not reported as registered. Trials reported as registered (median sample size, 111; interquartile range [IQR], 49-360) were larger than those not reported as registered (median sample size, 59; IQR, 24-106) (P < .001). Trials reported as registered were also more likely to report a power calculation, explicitly define the primary outcome, and report attrition among study participants. Specific data are reported for all characteristics in Table 2; P < .05 for all comparison.

Seventy-four of 105 trials not reported as registered (70.5%) reported positive findings, whereas 45 of 86 trials reported as registered (52%) reported positive findings (Table 2). Trials reported as registered were less likely than those not reported as registered to report positive findings even after adjustment for confounders associated with larger treatment effects: random sequence generation, allocation concealment, blinding, and sample size⁵ (unadjusted odds ratio [OR], 0.46; 95% CI, 0.25-0.83; adjusted OR, 0.50; 95% CI, 0.26-0.95). The estimated association was consistent when industry funding and journal impact factor were included in the model in a post hoc analysis (OR, 0.38; 95% CI, 0.18-0.80; model restricted to 174 trials for which a 2012 impact factor was available).

Less than 50% of cardiovascular trials published in December 2012 were reported as registered. These studies were larger and more likely to report strong methodological characteristics but were less likely to report significant positive findings.

To our knowledge, this is the first study to demonstrate an inverse association between reporting of trial registration and significant positive outcomes in cardiovascular RTs. These findings suggest that reporting of registration may reduce selective outcome reporting or selective publication.

Although some countries have mandatory trial registration policies, these are often limited in scope. For instance, many policies focus on pharmacologic trials, although only 42% of trials in our cohort tested a drug intervention. The majority of our cohort would therefore be exempt.

Our study has 2 limitations. First, we assessed the reported characteristics of cardiovascular trials, which may not perfectly reflect the conduct of the RTs, the conduct of trials conducted in other areas of medicine, or the conduct of trials testing specific types of interventions. Second, we included only journals that were indexed in PubMed.

Taken together, registration of all clinical trials should be a mandatory requirement for ethical and regulatory approval.

Corresponding Author: Kazem Rahimi, DM, The George Institute for Global Health, Oxford Martin School, University of Oxford, 34 Broad St, Oxford OX1 3DB, England (kazem.rahimi@georgeinstitute.ox.ac.uk).

Published Online: December 29, 2014. doi:10.1001/jamainternmed.2014.6924.

Authors Contributions: Mr Emdin and Dr Odutayo contributed equally as first authors, had full access to all the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Emdin, Odutayo, Hsiao, Hopewell, Altman.

Acquisition, analysis, or interpretation of data: Emdin, Odutayo, Hsiao, Shakir, Rahimi.

Drafting of the manuscript: Emdin, Odutayo, Hsiao, Rahimi, Altman.

Critical revision of the manuscript for important intellectual content: Emdin, Odutayo, Hsiao, Shakir, Hopewell.

Statistical analysis: Emdin, Odutayo, Hsiao.

Administrative, technical, or material support: Odutayo.

Study supervision: Hopewell, Rahimi, Altman.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding was provided by the Rhodes scholarship (Emdin, Odutayo, and Hsiao) and by grant CDF-2013-06-012 from the National Institute for Health Research and the Oxford Biomedical Research Center (Rahimi).

Role of the Funder/Sponsor: The funding institutions had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank Max Emmerich, BM BCh, for his translation support; he received no compensation for his translation support.