Systematic reviews are foundational to evidence-based patient care, yet these reviews are criticized for including only statistically significant findings to estimate summary effects, which may lead to publication bias.1,2 The exclusion of nonsignificant and/or unpublished results can lead to an overrepresentation of an intervention’s actual effect and potentially imprecise clinical decision making.3,4 Although many methods exist to minimize publication bias in systematic reviews, the use of clinical trial registries (CTRs) is particularly encouraging.
Clinical trial registries index new and ongoing studies, and may provide access to ongoing trial data prior to a study’s print release, even if nonsignificant findings arise.5 Given the potential of CTRs to identify unpublished studies for mitigating the potential of publication bias, we investigated the use of CTRs in systematic reviews published in the top otolaryngology journals and evaluated whether additional studies could have been located for systematic reviews that omitted CTR searches.
Using Google Scholar h5-index scores, the following journals were identified: American Journal of Otolaryngology–Head and Neck Medicine and Surgery, Clinical Otolaryngology, Current Opinions in Otolaryngology & Head and Neck Surgery, International Journal of Otolaryngology, JAMA Otolaryngology–Head & Neck Surgery, Journal of the Association for Research in Otolaryngology, Journal of Otolaryngology–Head & Neck Surgery, The Laryngoscope, and Otolaryngology–Head and Neck Surgery. A PubMed search was performed for relevant systematic reviews from January 1, 2008, to December 8, 2017. On December 19, 2017, we searched for systematic reviews published by the Cochrane Ear, Nose and Throat Disorders Group using the Cochrane Database of Systematic Reviews with the same date limiter. Eligible reviews were examined to identify whether a CTR was searched, and if so, which CTR(s) were reported. We also extracted title, author(s), journal, country of origin, and source (PubMed or Cochrane). We used the World Health Organization’s International Clinical Trials Registry Platform (ICTRP) Version 2.1 list of eligible CTRs and also included ClinicalTrials.gov.6
We next selected a random sample of 30 journal-published systematic reviews that excluded trial registry searches, and searched ClinicalTrials.gov for completed primary studies that could have been further considered by systematic reviewers had they conducted CTR searches. For each review, we sought to identify the number of clinical trials that were possibilities for inclusion. ClinicalTrials.gov searches were conducted from December 2 to 9, 2018.
After exclusions, 515 PubMed journal-published systematic reviews and 39 Cochrane reviews were included. Of the 515 journal-published reviews, 26 (5%) reported a search of 1 or more CTRs. ClinicalTrials.gov was cited most frequently (n = 24), followed by the ICTRP (n = 8), the European Clinical Trials Register (n = 2), and the Chinese Clinical Trial Registry (n = 1). In comparison, of the 39 Cochrane reviews, 35 (90%) reported searching a CTR. The Cochrane reviews reported searches of 3 registries: ICTRP (n = 35), ClinicalTrials.gov (n = 31), and ISRCTN/IRCT (n = 28).
From our random sample of 30 systematic reviews, 14 (46.7%) could have found at least 1 relevant study for inclusion in the systematic reviews had a CTR search been performed.
Only a minority of systematic reviews published in top-ranked otolaryngology journals reported searching a CTR (26 of 515 [5%]). By comparison, systematic reviews published by the Cochrane Collaboration almost always reported searching trial registries. Furthermore, almost 50% of systematic reviews in our second analysis would have located relevant studies for further consideration. Given that CTRs are freely available in the public domain and are not time consuming to search, we recommend that they be routinely included as part of a larger gray literature search strategy and represent a promising solution to address publication bias.
Corresponding Author: Kaleb Fuller, BS, Center for Health Sciences, Oklahoma State University, 1111 W 17th St, Tulsa, OK 74107 (kaleb.fuller@okstate.edu).
Accepted for Publication: February 23, 2019.
Conflict of Interest Disclosures: None reported.
Published Online: April 25, 2019. doi:10.1001/jamaoto.2019.0521
Author Contributions: Kaleb Fuller had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Vassar.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: Fuller, Vassar.
Statistical analysis: Fuller, Riddle.
Administrative, technical, or material support: Arthur, Riddle, Vassar.
Study supervision: Vassar.
2.Institute of Medicine, Board on Health Care Services, Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust. National Academies Press; Washington, DC 2011.
5.Committee on Clinical Trial Registries, Board on Health Sciences Policy, Institute of Medicine. Content of a Clinical Trial Registry. US: National Academies Press; Washington, DC 2006.