Publication of Randomized Clinical Trials in Pediatric Research A Follow-up Study

IMPORTANCE Nonpublication of research results in considerable research waste and compromises medical evidence and the safety of interventions in child health. OBJECTIVE To replicate, compare, and contrast the findings of a study conducted 15 years ago to determine the impact of ethical, editorial, and legislative mandates to register and publish findings. DESIGN, SETTING, AND PARTICIPANTS In this cohort study, abstracts accepted to the Pediatric Academic Societies (PAS) meetings from May 2008 to May 2011 were screened in duplicate to identify phase 3 randomized clinical trials enrolling pediatric populations. Subsequent publication was ascertained through a search of electronic databases in 2017. Study internal validity was measured using the Cochrane Risk of Bias Tool, the Jadad scale, and allocation concealment, and key variables (eg, trial design and study stage) were extracted. Associations between variables and publication status, time to publication, and publication bias were examined. MAIN OUTCOMES AND MEASURES Publication rate, trial registration rate, study quality, and risk of bias. RESULTS A total of 177 787 abstracts were indexed in the PAS database. Of these, 3132 were clinical trials, and 129 met eligibility criteria. Of these, 93 (72.1%; 95% CI, 53.8%-79.1%) were published. Compared with the previous analysis, the current analysis showed that fewer studies remained unpublished (183 of 447 [40.9%] vs 36 of 129 [27.9%], respectively; odds ratio [OR], 0.56; 95% CI, 0.36-0.86; P = .008). Fifty-one of 129 abstracts (39.5%) were never registered. Abstracts with larger sample sizes (OR, 1.92; 95% CI, 1.15-3.18; P = .01) and those registered in ClinicalTrials.gov (OR, 13.54; 95% CI, 4.78-38.46; P < .001) were more likely to be published. There were no differences in quality measures, risk of bias, or preference for positive results (OR, 1.60; 95% CI, 0.58-4.38; P = .34) between published and unpublished studies. Mean (SE) time to publication following study presentation was 26.48 (1.97) months and did not differ between studies with significant and nonsignificant findings (25.61 vs 26.86 months; P = .93). Asymmetric distribution in the funnel plot (Egger regression, −2.77; P = .04) suggests reduced but ongoing publication bias among published studies. Overall, we observed a reduction in publication bias and in preference for positive findings and an increase in study size and publication rates over time. CONCLUSIONS AND RELEVANCE These results suggest a promising trend toward a reduction in publication bias and nonpublication rates over time and positive impacts of trial registration. Further efforts are needed to ensure the entirety of evidence can be accessed when assessing treatment effectiveness. JAMA Network Open. 2018;1(1):e180156. doi:10.1001/jamanetworkopen.2018.0156 Key Points Question What proportion of abstracts submitted to a major annual pediatric scientific meeting are subsequently published? Findings Among 129 phase 3 randomized clinical trials identified in this cohort study, 27.9% were not subsequently published, and 39.5% were never registered, with previous trial registration and sample size associated with greater likelihood of publication. Mean (SE) time to publication from study presentation was 26.48 (1.97) months, and there was evidence of publication bias among published studies. Meaning Further encouragement and follow-up are needed to ensure that the totality of evidence is made available to inform clinical decision making. Author affiliations and article information are listed at the end of this article. Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2018;1(1):e180156. doi:10.1001/jamanetworkopen.2018.0156 May 18, 2018 1/9


Introduction
Publication bias, the selective publication of results or studies favoring positive outcomes, presents a critical threat to the validity and effectiveness of evidence-based medicine. 1 First referred to as the "file-drawer problem" nearly 4 decades ago, 2 selective publication has significant economic impacts, affects the safety of drugs and therapies, and stifles the implementation of true evidence-based medicine. This includes exposing study participants to potential harm due to duplication of previously unreported studies that were unsuccessful and creating a skewed evidence base for clinical decision making [3][4][5] and knowledge translation efforts. The existence of publication bias may be even more problematic in the pediatric population because of the unique challenges of conducting controlled clinical trials in children, including small sample sizes, editorial bias of pediatric trials, hesitation to test interventions in children, challenges with informed consent, and limited funding allotted to pediatric research. 6 Randomized clinical trials (RCTs) involve the enrollment of human participants to test a new treatment or intervention and provide the highest level of evidence to inform clinical practice. 7 Studies at the phase 3 stage are typically designed to assess the efficacy of an intervention and its value in clinical practice and are generally the most expensive, time-consuming, and difficult trials to conduct, as they enroll a large number of patients. 7 Similarly, studies at earlier stages are essential to assess safety and efficacy and to inform larger phase 3 trials. 7 Thus, nonpublication at any stage of pediatric clinical trials has a direct impact on patients' and populations' health, represents a waste of human and financial resources, and violates our ethical imperative to share results and reduce harm, particularly among vulnerable segments of the population. [8][9][10] A 2002 study examining abstracts accepted to the annual Pediatric Academic Societies (PAS) meetings between 1992 and 1995 and following up until 2000 to determine which studies were published found that 40.9% of abstracts reporting phase 3 RCTs with pediatric outcomes went unpublished, and that those with negative findings were less likely to be published. 11 Since this study's publication, a number of strategies have been implemented to minimize publication bias and increase transparency in clinical research, 9,12 in particular the refusal by major journals to publish studies without prior trial registration. 13 Despite these enhanced editorial mandates, publication bias persists. [14][15][16][17][18] The purpose of this study was to replicate the methods of the previous study by (1) measuring the subsequent publication of abstracts presented at a major annual pediatric scientific meeting, (2) investigating trial quality and factors associated with publication, and (3) examining the association of trial registration with subsequent publication.

Selection of Abstracts and Articles
Conference abstracts presented at the PAS meetings between May 2008 and May 2011 and their identifying information were extracted from the PAS website. In collaboration with a knowledge synthesis librarian, a modified version of the Cochrane Highly Sensitive Search Strategy 19 was developed and implemented to identify controlled clinical trials. Titles and abstracts of identified citations were then screened in duplicate using prespecified criteria to identify phase 3 RCTs that enrolled the pediatric population (neonatal, infants, children, and adolescents) and reported on pediatric outcomes. Studies reporting outcomes on pregnant women, animal studies, and non-RCT studies were excluded. No restrictions were imposed on treatment duration. Between March 20, 2017, and April 20, 2017, a search for trial registry results and full-text peer-reviewed publications corresponding to the citations that met the broad inclusion criteria was conducted in ClinicalTrials.gov and PubMed. A secondary search was then conducted in Embase, Web of Science, and Scopus for full-text articles not identified in PubMed. At least 1 of the a priori established primary outcomes must have been identified in the abstract and article to be considered as the corresponding article. The full-text search was conducted by a single researcher, with a second and third researcher conducting random checks for quality assurance. Ethical review and informed consent were not required by the University of Manitoba Health Research Ethics Board, as this was a review of a publicly available database and its associated articles in which aggregated data could not be associated with any individuals.

Assessment of Methodological Quality
Internal validity of abstracts and articles was assessed using multiple methods. For consistency with the previous study, study quality was first scored using the validated 5-point Jadad scale, 20 assessing randomization method, blinding, and description of dropouts and withdrawals. Second, study risk of bias was assessed as high, low, or unclear using the Cochrane Risk of Bias Tool 21 according to 6 domains: sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting, and other sources of bias; a final overall assessment within or across studies was then based on the responses to individual domains. 21 Third, the method used to prevent foreknowledge of group assignment by patients and investigators was assessed. Allocation concealment was rated as adequate (eg, centralized or pharmacy-controlled randomization), inadequate (eg, any procedure that was transparent before allocation, such as an open list of random numbers), or unclear. It should be noted that allocation concealment is also assessed as part of the risk of bias assessment. Finally, any reported funding source was documented as government, pharmaceutical, private, other, or unclear.

Data Extraction
Information extracted from identified abstracts and articles included abstract and journal citation, year of publication, trial design (parallel or crossover), study type (efficacy or equivalency), study stage (interim or final analyses), pilot or full trial, and primary study outcomes. Study type was noted based on the authors' primary hypothesis: intent to demonstrate a significant difference between treatments (efficacy) or intent to show noninferiority between treatments (equivalence). A study was noted as efficacious (favoring the intervention) based on the authors' statements and primary outcome significance level.

Statistical Analysis
As per the previous study, medians and interquartile ranges were used to describe nonparametric data, means and standard deviations were used for normal continuous data, and percentages and 95% confidence intervals were used for dichotomous and categorical data. Pearson χ 2 tests (dichotomous and categorical data), t tests (normal continuous data), and Wilcoxon rank sum tests (nonparametric data) were used to examine association between variables and publication status.
Variables associated with publication were assessed using logistic regression, time to publication was assessed using log-rank tests, and publication bias was assessed using a funnel plot and Egger regression. Statistical significance was set at P < .05 (2-tailed test).

Sample
We identified 177 787 abstracts indexed in the PAS database for the annual PAS conference from 2008 to 2011. Using a modified version of the Cochrane Highly Sensitive Search Strategy, we selected 3132 citations of controlled clinical trials for further review. Following title and abstract screening by 2 independent researchers, 129 citations met the eligibility criteria. Of these, 93 (72.1%; 95% CI, 53.8%-79.1%) were found as published articles.

Quality of Abstracts and Articles
Quality measures for abstracts and articles are summarized in Table 2. There were no distinguishable differences in quality between abstracts that were subsequently published and those that were not with the exception of an assessment of high risk of bias, which was more easily distinguishable in unpublished vs published abstracts. Because most abstracts did not contain sufficient information to score quality measures, full manuscript text was examined to generate the Jadad and Risk of Bias scores for published and unpublished articles and identify allocation concealment and funding source for published articles.

Variables Associated With Publication
Variables associated with publication based on the univariate analysis are summarized in Table 3.
Trial registration was significantly associated with study publication compared with no study

Variables That Changed From Abstract to Published Article
Ninety-three abstracts were paired with their subsequently published articles. Of these, 50 (53.8%; 95% CI, 43.7%-63.5%) had conclusions favoring treatment, which was similar to abstracts that went

Time to Publication
The time to publication by overall study conclusions (favoring treatment or not) was assessed.

Discussion
The   underpowered, methodologically poor, wasteful of scarce research resources, and ultimately unlikely to be published; however, we are seeing positive steps forward in this regard.
In this sample, most abstracts were not detailed enough to adequately rate study quality and most studies reporting funding sources at the abstract stage were funded by the pharmaceutical industry. At the abstract stage, most studies were rated of low quality based on the Jadad scale or had an unclear risk of bias; this did not differ between published and unpublished studies. However, the use of these tools for quality assessment was intended for articles, with limited space to adequately detail required quality measures in abstracts, and as such, articles were rated of high quality at publication stage. Furthermore, we observed an increase in quality scores from the previous sample.
Mean (SE)  Unlike the previous study, study significance was not found to be associated with subsequent publication among abstracts presented between 2008 and 2011 at the PAS scientific meeting. However, it is unclear how many RCTs performed in children were never submitted or accepted to an annual meeting. Our study found an overall trend of publication bias among published studies. However, this was analyzed only on a portion of all included studies and, as such, should be interpreted with caution.
Furthermore, the overall evidence for publication bias in this analysis is modest.
Nonpublication of research findings has implications. Failure to publish results of clinical research involving participants and families who have consented to participate disrespects their personal contribution and decreases public trust in clinical science. 28 The persistence of nonpublication of trial findings nearly 15 years later 11 violates the Declaration of Helsinki, which states that every research study involving human participants must be registered in a publicly accessible database before the recruitment of the first participant and that researchers, authors, editors, and publishers have an ethical obligation to publish research results. 9 Timely publication of the findings of clinical trials is essential not only to support evidence-based decision making by patients and clinicians, but also to ensure that all children receive the best possible care.

Limitations
Several limitations should be noted. Although the PAS meetings serve as a useful sample with which to examine publication bias and nonpublication of research findings, they do not capture the full breadth of existing trials. Recent studies have also examined the discontinuation and nonpublication of RCTs conducted in children based on trials registered in ClinicalTrials.gov. 15 Our study was able to capture additional trials that were never registered (51 of 129 [39.5%]) and demonstrates that trial registration, although associated with subsequent publication, does not guarantee publication.
Furthermore, our findings suggest that a significant proportion of trials remain unregistered. This is an important addition to the literature. Second, our replication study yielded a lower sample of phase 3 RCTs compared with the previous study. 11 However, predefined inclusion criteria were followed by 2 independent reviewers. Furthermore, exact definitions used in the previous study were used at the stage of variable extraction. Variation in sample size between the 2 studies could be due to more stringent inclusion criteria or a shift in the research realm or conference interests over time. Finally, because of limited space within abstracts, it is difficult to judge study quality at this stage. However, quality did not differ between abstracts that were subsequently published and those that were not.

Conclusions
Our study found that among phase 3 pediatric RCTs presented at a major scientific meeting, a large proportion of studies remain unregistered and unpublished. However, we observed promising trends in the reduction of nonpublication and positive impact of editorial policy changes. Of studies that were published, previous trial registration and larger sample sizes were significantly associated with subsequent publication. These findings suggest that ongoing efforts are needed to enhance publication of all trials and to enforce mandatory trial registration, particularly in pediatric trials, to ensure that participation and knowledge generation can benefit all children.