[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.211.168.204. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Publication Bias
June 5, 2002

Association Between Time Interval to Publication and Statistical Significance

Author Affiliations

Author Affiliations: Department of Community Health, Brown University, Providence, RI (Dr Dickersin and Ms Zhu); JAMA, Chicago, Ill (Drs Olson, Rennie, and Cook, Mss Flanagin and Reiling, and Mr Pace); Department of Medicine, Division of Emergency Medicine, University of Washington, Seattle (Dr Olson); Institute for Health Policy Studies, University of California-San Francisco (Dr Rennie); Departments of Medicine, Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario (Dr Cook).

JAMA. 2002;287(21):2829-2831. doi:10.1001/jama.287.21.2829
Context

Context Studies have shown that reports from clinical trials with statistically significant results tend to be submitted and published more rapidly than reports from studies with null or nonsignficant findings.

Methods We wanted to determine whether manuscripts reporting positive results from controlled clinical trials are published more quickly than those reporting negative results. We tracked manuscripts from submission to JAMA until the publication decision. We classified results as positive if a statistically significant difference was reported for the primary outcome. Manuscripts were further classified according to indicators of methodologic quality and other study characteristics. We included manuscripts if they were submitted from February 1996 through August 1999, reported results of a prospective study in which participants were assigned to a treatment or comparison group, used statistical tests to compare differences between groups, and were accepted for publication.

Results One hundred thirty-three manuscripts met our inclusion criteria, 78 (59%) reported positive results, 51 (38%) reported negative results. We were unable to classify the direction of results for 4 articles (3%). The time interval between submission and publication was not associated with positive results: median time between submission and publication was 7.8 months for reports with positive vs 7.6 months for reports with negative results (P = .44). Time to publication also was not associated with any marker of study quality or study characteristic.

Conclusions Among 133 published controlled trials, time to publication was not associated with statistical significance, methodologic quality, or other study characteristics.

Publication bias is the tendency to publish research results based on the strength and direction of a study's findings. Data from a number of studies1 have clearly shown that publication bias exists. At least 3 studies have indicated that studies with positive results may be published more quickly than those with null or negative findings.24 However, these studies did not examine whether this increase in publication time resulted from investigators' taking less time to submit manuscripts that report positive results or resulted from the editorial review's and revision process's taking less time, or both.

METHODS

We hypothesized that controlled clinical trial articles reporting positive results are published more rapidly than those reporting negative results. The details of our methods are reported in detail elsewhere.5

Briefly, we tracked a cohort of manuscripts from submission to JAMA to acceptance. Articles were included in our study if they were submitted from February 1996 through August 1999, reported results of a prospective study in which participants were assigned to a treatment or comparison group, reported use of statistical tests to compare differences between groups, and were accepted for publication.

Outcomes were classified as positive when P<.05, the 95% confidence interval [CI] for a risk difference excluded 0, or when an odds ratio or relative risk excluded 1.0 for the primary outcome (or the majority of outcomes if a primary outcome could not be identified). Otherwise, manuscripts were classified as having negative results. Manuscripts were further classified according to indicators of methodologic quality, including reporting a sample size calculation, the method by which subjects were assigned to treatment, whether participants were masked to group assignment, whether withdrawals were reported, and whether results were analyzed by treatment received or treatment assigned. Other study characteristics included number of study centers recruiting patients, countries in which participants were enrolled, funding source, sample size, and the sex of the first author. Data were abstracted from each manuscript independently by 2 of 3 investigators (C.M.O., D.R., or D.C.) and disagreements were resolved by telephone. Additional information on publication was extracted from JAMA's database.

To examine the association between study findings and time to publication, we performed Kaplan Meier analysis, which included all published manuscripts with findings that could be classified as positive or negative. We also performed Kaplan Meier analyses examining the association between each of the study and methodologic quality characteristics for association with time interval to publication.

The University of Washington's human subjects review committee approved the study.

RESULTS

One hundred thirty-three manuscripts met our inclusion criteria, 78 (59%) reported positive results and 51 (38%) reported negative results. We were unable to classify the direction of results for 4 articles (3%). After omitting studies with unclear results, we found no evidence that articles with positive findings were published at a faster pace than those reporting negative findings: the median time interval between submission and publication was 7.8 months for positive vs 7.6 months for negative studies; (P = .44) (Figure 1).

Figure. Time Interval Between Submission and Publication by Results
Image description not available.
Kaplan Meier survival analysis P = .44.

We also found no association between each of the methodologic quality and study characteristics and the time interval between submission and publication (data not shown).

COMMENT

Authors, funders, patients, policymakers, and other stakeholders in research findings are legitimately concerned about dissemination of the results of controlled trials as quickly as possible. Several initiatives are underway to shorten the time interval between study completion to publication and from submission to publication. For example, many journals have established fast track policies for accelerating the publication process for studies in certain cases. At JAMA, peer reviewers are asked to return their comments within 24 to 48 hours for fast track reports and within 21 days for reports undergoing the standard review process. None of the articles in our study had been designated as fast track because JAMA's fast tract option was not initiated until late 1999.

In this study, we found no evidence that the time interval between manuscript submission and publication is influenced by study results, methodologic quality, or other study characteristics. Examination of factors leading to delays in publication have not tended to differentiate between the time authors take to write and submit study results from the time interval between submission and publication. Specifically, other analyses24 have included all studies undertaken, including those not published. Since negative findings are less likely to be published, analyses of time to publication that include unpublished reports focus on a different question than the one we addressed. Our study is the first to focus on published articles, the time interval between submission and publication, and possible time differences between reports describing positive and negative findings.

In a study of 66 completed randomized human immunodeficiency virus trials, time from study completion to publication was shorter for trials with positive results than it was for trials with negative results (a median of 1.7 years vs 3.0 years, respectively; P<.001).3 Trials with statistically significant results favoring the experimental arm (positive results) were submitted more quickly after completion than those with negative findings (median 1.0 vs 1.6 years, respectively; log rank, P = .001) and reached publication somewhat more quickly after submission (median 0.8 vs 1.1 years, respectively; P = .04). Of the 45 trials submitted for publication, 17 were rejected by 1 or more journals, and there was a nonsignificant tendency for positive results to be accepted more often on first submission (OR, 1.6; 95% CI; 0.5-5.6). Stern and colleagues also showed that clinical trials with null or nonsignificant results were slower to reach publication than those with significant findings (using the date of the ethics committee's approval as a start date).2 Misakian and Bero4 examined the number of years between start of funding and first publication for studies of passive smoking and found a statistically significant shorter interval for studies with statistically significant results compared with those with nonsignificant or mixed results.

We limited our definition of positive findings to those that were statistically significant for the primary outcome or the majority of outcomes examined when the primary outcome was unclear. Our results apply to a general population of trials submitted to JAMA and may not be applicable to other journals or trials in specific fields. We did not examine any discrete times such as the time taken by referees for review or by authors for revisions. Our study does not account for other features of journal articles that may influence time to publication, such as whether the results are clinically important or timely in some other way. It is possible but unlikely that our failure to observe any differences between groups in the time from submission to publication is attributable to a small sample size.

Conclusion

Among 133 published controlled trials, time to publication was not associated with statistical significance, methodologic quality, or other study characteristics.

References
1.
Dickersin K. How important is publication bias? a synthesis of available data.  AIDS Educ Prev1997;9(suppl A):15-21.
2.
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects.  BMJ.1997;315:640-645.
3.
Ioannidis JP, Cappelleri JC, Sacks HS, Lau J. The relationship between study design, results, and reporting of randomized clinical trials of HIV infection.  Control Clin Trials.1997;18:431-44.
4.
Misakian AL, Bero LA. Publication bias and research on passive smoking. comparison of published and unpublished studies  JAMA.1998;280:250-253.
5.
Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, Hogan JW, Zhu Q, Reiling J, Pace B. Publication bias in editorial decision making.  JAMA.2002;287:2825-2828.
×