Discrepancies in the Registries of Diet vs Drug Trials

ClinicalTrials.gov was established in 2000 in response to the Food and Drug Administration Modernization Act of 1997, which called for registration of trials of investigational new drugs for serious diseases. Subsequently, the scope of ClinicalTrials.gov expanded to all interventional studies, including diet trials. Presently, prospective trial registration is required by the National Institutes of Health for grant funding and many clinical journals for publication.¹ Registration may reduce risk of bias from selective reporting and post hoc changes in design and analysis.^1,2 Although a study³ of trials with ethics approval in Finland in 2007 identified numerous discrepancies between registered protocols and subsequent publications, the consistency of diet trial registration and reporting has not been well explored.

This cross-sectional study compared the registries of drug and diet trials published in selected prominent clinical journals in the last decade. A literature search, conducted June 26, 2019, retrieved trials with obesity-related outcomes in 5 general medical journals and 1 nutrition journal with the highest impact factors for their fields (The New England Journal of Medicine, JAMA, The BMJ, The Lancet, Annals of Internal Medicine, and The American Journal of Clinical Nutrition). Results were limited to clinical trials in the 10-year period ending June 1, 2019. For drug trials, the search terms were “drug OR placebo OR pharmaceutical” with exclusion if the intervention did not focus on a specific drug (eg, dietary supplement, food extract). For diet trials, the search terms were “diet OR dietary” with exclusion if the intervention did not focus on a specific diet (eg, dietary supplement, food extract, dietary pattern such as meal skipping, or multicomponent intervention such as one including exercise). Additionally, trials in both categories were excluded if the article was not in the indicated journals (eg, BMJ Open) or was not an original randomized clinical trial; the primary registry was not in ClinicalTrials.gov; the primary outcome was not related to body weight, adiposity, or energy balance; or the primary outcome was measured in less than 28 days. For each included trial, we examined the current ClinicalTrials.gov registry, the history of changes at the ClinicalTrials.gov archive site, and the final published article, with reference to changes in or discrepancies involving the prespecified primary outcome. We calculated an odds ratio and 95% confidence interval for the number of diet vs drug trials with changes or discrepancies using SAS statistical software version 9.4 (SAS Institute). A 2-tailed P < .05 was considered statistically significant. The ethical review board of Boston Children’s Hospital does not require review of this type of study because no human participants were involved.

Our literature search retrieved 148 drug studies and 343 diet studies, from which 9 and 21, respectively, were included in our sample after applying exclusion criteria. As shown in Table 1 and Table 2, 2 drug trials (22%) and 18 diet trials (86%) had a substantive discrepancy from initial registration, typically involving a change in time frame of the primary outcome or the number of co–primary outcomes. The odds ratio for a discrepancy for diet vs drug trials was 21.0 (95% CI, 2.9-153.8; P = .002). Among diet trial registries, 2 appeared to have been post hoc and 1 was not available.

This study found that, among top-rated clinical journals, most diet trials would not satisfy an essential criterion for prospective registration, as judged by standards inferred from drug trials. This problem extends beyond low registration rates in behavioral research,⁴ as registration for only 1 diet trial was missing.

Limitations of this study include a focus on just 1 (though central) aspect of registration and extrapolation of the findings to the general literature. We did not examine registry data involving interventions, participant number (anticipated vs achieved), or statistical treatments. Although our findings derive from a small subset of published trials, the highlighted problems are likely general to the field, in light of the highly selective nature of the journals examined. Furthermore, we did not control for measures of trial quality because these could be considered inherently related to, rather than confounders of, the observed association.

Problems with diet trial registries may arise from the their greater heterogeneity and lower budgets vs drug studies and the inadequacy of infrastructural support for nutrition research.⁵ Ultimately, high-quality diet trials of the type needed to develop effective prevention and treatment for chronic disease will require substantial investment of financial and personnel resources from the National Institutes of Health and philanthropy. More immediate and specific remedies for the deficiencies of diet trial registries include (1) posting a detailed final statistical analysis plan before unmasking random group assignments or beginning data analyses as a minimum quality criterion; (2) declaring substantive changes to the original registry in the main manuscript (rather than infrequently read online supplementary material); and (3) creating specialized registries for diet (and possibly other behavioral) trials to reflect their special challenges beyond those of drug trials (for which current registries were originally intended).

Accepted for Publication: September 23, 2019.

Published: November 13, 2019. doi:10.1001/jamanetworkopen.2019.15360

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Ludwig DS et al. JAMA Network Open.

Corresponding Author: David S. Ludwig, MD, PhD, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115 (david.ludwig@childrens.harvard.edu).

Author Contributions: Drs Ludwig and Ebbeling had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: Ludwig, Ebbeling.

Drafting of the manuscript: Ludwig, Heymsfield.

Critical revision of the manuscript for important intellectual content: All authors.

Administrative, technical, or material support: Ludwig, Ebbeling.

Supervision: Ludwig.

Conflict of Interest Disclosures: Dr Ludwig reported receiving grants from the National Institutes of Health, Arnold Ventures, Nutrition Science Initiative, and New Balance Foundation outside the submitted work; and royalties for books on diet and obesity. Dr Ebbeling reported receiving grants from the National Institutes of Health, Arnold Ventures, Nutrition Science Initiative, and New Balance Foundation outside the submitted work. Dr Heymsfield reported receiving personal fees from Medifast Medical Advisory Board and Tanita Medical Advisory Board outside the submitted work.