Assessment of Duplicate Evidence in Systematic Reviews of Imaging Findings of Children With COVID-19 | Infectious Diseases | JAMA Network Open | JAMA Network
[Skip to Content]
[Skip to Content Landing]
Figure.  Matrix of Evidence Displaying the Number of Systematic Reviews and Primary Studies Included in Each Review
Matrix of Evidence Displaying the Number of Systematic Reviews and Primary Studies Included in Each Review

References can be found in the eReferences in the Supplement.

1.
Tugwell  P, Welch  VA, Karunananthan  S,  et al.  When to replicate systematic reviews of interventions: consensus checklist.   BMJ. 2020;370:m2864. doi:10.1136/bmj.m2864PubMedGoogle ScholarCrossref
2.
Cui  X, Zhao  Z, Zhang  T,  et al.  A systematic review and meta-analysis of children with coronavirus disease 2019 (COVID-19).   J Med Virol. Published online August 6, 2020. doi:10.1002/jmv.26398Google Scholar
3.
Solla  F, Bertoncelli  CM, Rampal  V.  Does the PROSPERO registration prevent double review on the same topic?   BMJ Evid Based Med. 2020;(February):bmjebm-2020-111361. doi:10.1136/bmjebm-2020-111361PubMedGoogle Scholar
4.
Moher  D, Booth  A, Stewart  L.  How to reduce unnecessary duplication: use PROSPERO.   BJOG. 2014;121(7):784-786. doi:10.1111/1471-0528.12657PubMedGoogle ScholarCrossref
5.
Glasziou  PP, Sanders  S, Hoffmann  T.  Waste in COVID-19 research.   BMJ. 2020;369:m1847. doi:10.1136/bmj.m1847PubMedGoogle Scholar
6.
Page  MJ, Welch  VA, Haddaway  NR, Karunananthan  S, Maxwell  LJ, Tugwell  P.  “One more time”: why replicating some syntheses of evidence relevant to COVID-19 makes sense.   J Clin Epidemiol. 2020;125:179-182. doi:10.1016/j.jclinepi.2020.05.024PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,726
    Citations 0
    Research Letter
    Pediatrics
    January 7, 2021

    Assessment of Duplicate Evidence in Systematic Reviews of Imaging Findings of Children With COVID-19

    Author Affiliations
    • 1Sinaloa Pediatric Hospital’s Cochrane Associate Centre, Culiacan, Mexico
    • 2Epistemonikos Foundation, Santiago, Chile
    • 3Department of Pediatrics, University of Antioquia, Medellin, Colombia
    JAMA Netw Open. 2021;4(1):e2032690. doi:10.1001/jamanetworkopen.2020.32690
    Introduction

    Formulating evidence-based recommendations for children affected by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is challenging. Identifying and synthesizing the evidence to inform these recommendations has become difficult. With the explosion of publications on preprint servers and in journals, waste in coronavirus disease 2019 (COVID-19) research is common. While replication of systematic reviews may be appropriate in some instances, duplication refers to needless repetition of the same review.1 Answering simple questions, such as the most common findings in children with COVID-19, requires an enormous effort. We aimed to map 1 of these questions (ie, what is the spectrum and frequency of imaging findings in children with COVID-19?) to illustrate the overlap and shortcomings of the evidence syntheses in this area.

    Methods

    This cross-sectional study began with systematic searches in the Living Overview of Evidence platform for COVID-19, a system that maps population, intervention, comparison, and outcome (PICO) questions to a repository maintained through regular searches in more than 40 sources, including databases, preprint servers, trial registries, and others. An artificial intelligence algorithm maintains the repository, and the information is transmitted in real time. Search methods for COVID-19 in the Living Overview of Evidence are detailed in the eAppendix in the Supplement. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline, modified for overviews. Because this was a secondary analysis of already-published articles, no institutional review board approval was sought.

    We included systematic reviews describing imaging findings in children younger than 18 years (excluding neonates) with SARS-CoV-2 infection and primary studies with more than 30 children included. Two reviewers (G.P.-G. and F.V.-P.) independently evaluated potentially eligible studies. Searches had no language restrictions and covered the period until September 1, 2020.

    We built a matrix of evidence using Epistemonikos Database (Epistemonikos) to compare the studies included in the reviews. A matrix of evidence is a table displaying all systematic reviews answering the same question as well as the studies answering the question of interest included in these reviews. We provide a narrative description of the results. No statistical testing was conducted.

    Results

    We identified 25 systematic reviews, including 17 primary studies, answering the question of interest (Figure). Only 6 of the 25 systematic reviews identified (24%) had been previously registered in PROSPERO or other registries. The number of primary studies identified by each particular review ranged from 1 to 9. Our search found 11 eligible primary studies that were not identified by any of the reviews.

    The most recent review with the largest number of included studies (Cui et al2) had only 9 of 28 articles (32%) that were eligible according to our analysis. We explored whether this review explicitly excluded these studies or whether there were additional criteria explaining why they were not included. Four studies (21%) were probably missed during the review process, and the other 15 (79%) were published after their search.

    Discussion

    This study presents a particular case in which, in less than 6 months, the literature was flooded with more systematic reviews than primary studies trying to answer a very specific clinical question, such as the imaging findings in children with COVID-19. Replication of systematic reviews may be appropriate to verify their findings or to extend or narrow the question they are trying to answer.1 However, needless repetition is wasteful. Initiatives like the PROSPERO database were created so authors could identify ongoing systematic reviews and perhaps stop the development of a new, unnecessary study.3,4

    Duplication at a massive level, which has happened with COVID-19, is unjustified and may be unethical.1,5,6 None of the systematic reviews included the totality of primary studies, which may be partly explained by the rapid rate of reporting of new studies but also by limitations of search strategies. This also highlights how quickly published reviews can become obsolete if they are not continuously updated.

    This study has limitations. Our analysis cannot provide clinical guidance regarding the imaging findings of children with COVID-19. The findings are also limited to the date of our search.

    Back to top
    Article Information

    Accepted for Publication: November 13, 2020.

    Published: January 7, 2021. doi:10.1001/jamanetworkopen.2020.32690

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Pérez-Gaxiola G et al. JAMA Network Open.

    Corresponding Author: Giordano Pérez-Gaxiola, MD, MSc, Sinaloa Pediatric Hospital’s Cochrane Associate Centre, Blvd Constitución y Donato Guerra, SN, Culiacan 80200, Mexico (giordano@sinestetoscopio.com).

    Author Contributions: Dr Pérez-Gaxiola had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Pérez-Gaxiola, Rada, Flórez.

    Acquisition, analysis, or interpretation of data: Pérez-Gaxiola, Verdugo-Paiva, Flórez.

    Drafting of the manuscript: Pérez-Gaxiola, Verdugo-Paiva, Flórez.

    Critical revision of the manuscript for important intellectual content: Pérez-Gaxiola, Rada, Flórez.

    Statistical analysis: Pérez-Gaxiola, Rada.

    Administrative, technical, or material support: Verdugo-Paiva, Rada.

    Supervision: Rada, Flórez.

    Conflict of Interest Disclosures: None reported.

    Additional Contributions: Valeria Contreras, Graphic Designer (Epistemonikos Foundation), designed the figure. No compensation was received.

    References
    1.
    Tugwell  P, Welch  VA, Karunananthan  S,  et al.  When to replicate systematic reviews of interventions: consensus checklist.   BMJ. 2020;370:m2864. doi:10.1136/bmj.m2864PubMedGoogle ScholarCrossref
    2.
    Cui  X, Zhao  Z, Zhang  T,  et al.  A systematic review and meta-analysis of children with coronavirus disease 2019 (COVID-19).   J Med Virol. Published online August 6, 2020. doi:10.1002/jmv.26398Google Scholar
    3.
    Solla  F, Bertoncelli  CM, Rampal  V.  Does the PROSPERO registration prevent double review on the same topic?   BMJ Evid Based Med. 2020;(February):bmjebm-2020-111361. doi:10.1136/bmjebm-2020-111361PubMedGoogle Scholar
    4.
    Moher  D, Booth  A, Stewart  L.  How to reduce unnecessary duplication: use PROSPERO.   BJOG. 2014;121(7):784-786. doi:10.1111/1471-0528.12657PubMedGoogle ScholarCrossref
    5.
    Glasziou  PP, Sanders  S, Hoffmann  T.  Waste in COVID-19 research.   BMJ. 2020;369:m1847. doi:10.1136/bmj.m1847PubMedGoogle Scholar
    6.
    Page  MJ, Welch  VA, Haddaway  NR, Karunananthan  S, Maxwell  LJ, Tugwell  P.  “One more time”: why replicating some syntheses of evidence relevant to COVID-19 makes sense.   J Clin Epidemiol. 2020;125:179-182. doi:10.1016/j.jclinepi.2020.05.024PubMedGoogle ScholarCrossref
    ×