[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Publication Bias
June 5, 2002

Publishing Protocols of Systematic Reviews: Comparing What Was Done to What Was Planned

Author Affiliations

Author Affiliations: Monash Institute of Health Services Research, Monash Medical Centre, Clayton, Victoria, Australia (Dr Silagy); and UK Cochrane Centre, Oxford, England (Mss Middleton and Hopewell). Dr Silagy is deceased.

JAMA. 2002;287(21):2831-2834. doi:10.1001/jama.287.21.2831

Context Publication of research protocols minimizes bias by explicitly stating a priori hypotheses and methods without prior knowledge of results.

Methods We conducted a retrospective comparative study to assess the extent to which the content of published Cochrane reviews had changed compared with their previously published protocols and to assess any potential impact these changes may have had in introducing bias to the study. We identified previously published protocols for new Cochrane reviews appearing in The Cochrane Library; 2000, issue 3. The texts of published protocols and completed reviews were compared. Two raters independently identified changes to the different sections of the protocol and classified the changes as none, minor, or major.

Results Of the 66 new Cochrane reviews, we identified a previously published protocol for 47 reviews. Of these, 43 reviews had at least 1 section that had undergone a major change compared with the most recently published protocol. The greatest variation between protocols and reviews was in the methods section, in which 68% of reviews (n = 32) had undergone a major change. Changes made in other sections that may have resulted in the introduction of bias included narrowing of objectives, addition of comparisons or new outcome measures, broadening of criteria for the types of study design included, and narrowing of types of participants included.

Conclusions Research protocols, even if published, are likely to remain, at least to some extent, iterative documents. We found that a large number of changes were made to Cochrane reviews, some of which could be prone to influence by prior knowledge of results. Even if many of the changes between protocol and review improve the overall study, the reasons for making these should be clearly identified and documented within the final review.