Ross JS, Mocanu M, Lampropulos JF, Tse T, Krumholz HM. Time to publication among completed clinical trials. JAMA Intern Med. Published online March 4, 2013. doi:10.1001/jamainternmed.2013.136
eFigure. Time to publication after completion among clinical trials registered in ClinicalTrials.gov and published in the biomedical literature (cited in MEDLINE)
Joseph S. Ross, Marian Mocanu, Julianna F. Lampropulos, Tony Tse, Harlan M. Krumholz. Time to Publication Among Completed Clinical Trials. JAMA Intern Med. 2013;173(9):825–828. doi:10.1001/jamainternmed.2013.136
Author Affiliations: Section of General Internal Medicine (Dr Ross), Section of Cardiovascular Medicine (Dr Krumholz), and Robert Wood Johnson Foundation Clinical Scholars Program (Drs Ross and Krumholz), Department of Medicine, Yale University School of Medicine, New Haven, Connecticut; Center for Outcomes Research and Evaluation, Yale–New Haven Hospital, New Haven (Drs Ross, Lampropulos, and Krumholz); Section of Cardiovascular Medicine, Department of Medicine, Mayo Clinic, Rochester, Minnesota (Dr Mocanu); Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, Maryland (Dr Tse); and Section of Health Policy and Administration, Yale University School of Epidemiology and Public Health, New Haven (Dr Krumholz).
Prior studies have shown that 25% to 50% of clinical trials are never published.1- 4 However, among those published, we know little about the length of time required for publication in the peer-reviewed biomedical literature after study completion. Ioannidis5 previously demonstrated that a sample of randomized phase 2 and 3 trials conducted between 1986 and 1996 required nearly 2.5 years for publication, while our more recent study of National Institutes of Health (NIH)-funded trials found that the average time to publication was almost 2 years.4 We sought to determine time to publication for a recent and representative sample of trials published in 2009.
We examined clinical trials published during the 2009 calendar year that were indexed via MEDLINE and linked to a ClinicalTrials.gov identifier (National Clinical Trial [NCT] number).6 We excluded articles that were not the first publication reporting on the clinical trial by reviewing article text for citations to and searching MEDLINE for earlier trial publications. We imputed completion dates based on the subject enrollment dates and the longest period of follow-up required for any outcome measure reported in the article. For trials without enrollment dates, we used the completion date provided on ClinicalTrials.gov; this date was not used uniformly because our prior work demonstrated its unreliability.3 We determined publication dates using MEDLINE, using online publication dates for articles made available ahead of print. Because ClinicalTrials.gov reports completion dates in month/year format, publication dates were similarly recorded for purposes of calculating time to publication.
We obtained additional information through the National Library of Medicine, using the ClinicalTrials.gov identifier, including data elements reporting lead sponsor, study design, and condition studied (Table). We obtained journal impact factors from Web of Knowledge (Thomson Reuters). We used Kruskal-Wallis tests to compare median times to publication between trial characteristic categories using a type I error rate of 0.003 to account for multiple comparisons (n = 16). Statistical analysis was performed using JMP 7.0.1 software (SAS Institute Inc).
There were 1336 clinical trials published during the 2009 calendar year, indexed via MEDLINE and linked to a ClinicalTrials.gov identifier. The majority of trials were funded by nonprofit organizations and enrolled more than 100 subjects (Table). Nearly half were published in journals with an impact factor less than 5.
Median time to publication was 21 months, with an interquartile range of 13 to 32 months (eFigure) and modest differences across types of trials (Table). For instance, median time to publication was longer among trials funded by industry compared with trials funded by government and nonprofit organizations (24 vs 20 months; P < .001), but was shorter among trials enrolling 1000 subjects or more compared with trials enrolling 100 subjects or fewer and trials enrolling between 100 and 1000 subjects (18, 20, and 23 months, respectively; P < .001) and was shorter among trials published in journals with an impact factor greater than 10 compared with trials published in journals with an impact factor less than 10 (17 vs 23 months; P < .001).
We found, on average, that nearly 2 years had passed between completion and publication of clinical trials, across all trial funders. Moreover, given that our study was necessarily limited to examining time to publication among completed trials that were eventually published, this estimate is conservative. First, we studied trials registered in and linked to ClinicalTrials.gov, a select group of studies. Because of policies in place as of 2005, many may have been registered to ensure compliance with International Committee of Medical Journal Editors requirements for publication7 and thus more likely to publish in a timely manner. Second, we only studied trials that were published (and indexed via MEDLINE and linked to ClinicalTrials.gov identifiers). Between 50% and 70% of studies registered in ClinicalTrials.gov are eventually published many years after trial completion3,4; 30% to 50% are never published.
Differences across trial types were generally modest and indicate that timely dissemination of research needs to be uniformly prioritized to enhance science. Many steps must be successfully taken between trial completion and publication, including data management, statistical analysis, and writing, along with processes out of investigators' control, such as peer review. The substantial variation in publication time, from months to years, suggests that there may be opportunities to improve the speed and efficiency with which investigators publish completed studies.
We cannot rule out that study findings may have been disseminated through means other than publication, including scientific meeting presentations. However, with the exception of public results reporting, these alternative dissemination strategies lead to limited public awareness of the research. Given the time required to publish results from these clinical trials, our findings support current federal initiatives requiring results reporting of clinical studies within 12 months of trial completion8 to ensure the timely dissemination of clinical science.
Correspondence: Dr Ross, Section of General Internal Medicine, Yale University School of Medicine, PO Box 208093, New Haven, CT 06520 (firstname.lastname@example.org).
Published Online: March 4, 2013. doi:10.1001/jamainternmed.2013.136
Author Contributions: All authors had full access to all the data in the study and Dr Ross takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Ross, Tse, and Krumholz. Acquisition of data: Ross, Mocanu, Lampropulos, and Tse. Analysis and interpretation of data: Ross, Mocanu, Lampropulos, and Krumholz. Drafting of the manuscript: Ross. Critical revision of the manuscript for important intellectual content: Ross, Mocanu, Lampropulos, Tse, and Krumholz. Obtained funding: Krumholz. Administrative, technical, and material support: Mocanu and Tse.
Conflict of Interest Disclosures: Dr Ross reports that he is a member of a scientific advisory board for FAIR Health Inc. Dr Krumholz reports that he chairs a scientific advisory board for UnitedHealthcare. Dr Tse reports that he is a Program Analyst for ClinicalTrials.gov
Funding/Support: The analyses on which this publication is based were performed under Contract No. HHSN276201000788P, entitled “Evaluating Time-to-Publication Following Completion of Clinical Trials andRelated Factors,” funded by the National Library of Medicine, Department of Health and Human Services. Drs Ross and Krumholz receive support from Medtronic Inc and from the Centers of Medicare & Medicaid Services to develop and maintain performance measures that are used for public reporting. Dr Ross is supported by the National Institute on Aging (K08 AG032886) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr Tse is supported by the Intramural Research Program of the National Institutes of Health, National Library of Medicine. Dr Krumholz is supported by a National Heart Lung Blood Institute Cardiovascular Outcomes Center Award (1U01HL105270-02).
Disclaimer: The ideas and opinions expressed are the authors' own. The content of this publication does not necessarily reflect the views or policies of the US National Institutes of Health, Public Health Service, or Department of Health and Human Services. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Additional Contributions: Deborah A. Zarin, MD, of the Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, provided comments on an early version of the manuscript. Dr Zarin received no compensation for her review.