[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Table 1.  
Drug Registries Involving Obesity in Selected Major Medical Journals
Drug Registries Involving Obesity in Selected Major Medical Journals
Table 2.  
Diet Registries Involving Obesity in Selected Major Medical Journals
Diet Registries Involving Obesity in Selected Major Medical Journals
1.
DeAngelis  CD, Drazen  JM, Frizelle  FA,  et al; International Committee of Medical Journal Editors.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors.  JAMA. 2004;292(11):1363-1364. doi:10.1001/jama.292.11.1363PubMedGoogle ScholarCrossref
2.
Rennie  D.  Trial registration: a great idea switches from ignored to irresistible.  JAMA. 2004;292(11):1359-1362. doi:10.1001/jama.292.11.1359PubMedGoogle ScholarCrossref
3.
Chan  AW, Pello  A, Kitchen  J,  et al.  Association of trial registration with reporting of primary outcomes in protocols and publications.  JAMA. 2017;318(17):1709-1711. doi:10.1001/jama.2017.13001PubMedGoogle ScholarCrossref
4.
Azar  M, Riehm  KE, Saadat  N,  et al.  Evaluation of journal registration policies and prospective registration of randomized clinical trials of nonregulated health care interventions.  JAMA Intern Med. 2019;179(5):624-632. doi:10.1001/jamainternmed.2018.8009PubMedGoogle ScholarCrossref
5.
Ludwig  DS, Ebbeling  CB, Heymsfield  SB.  Improving the quality of dietary research.  JAMA. 2019;322(16):1549-1550. doi:10.1001/jama.2019.11169PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    EXPAND ALL
    Culture Change is Possible
    David Mellor, PhD, Ecology and Evolution | Center for Open Science
    I applaud the authors for this important work. The practices that they are uncovering adds to the evidence that reduces credibility of empirical research findings. This adds to the evidence accumulated by Goldacre and many others.

    The authors provide three specific recommendations for addressing the problems that they have quantified: posting analysis plans, disclosure of deviations, and specialized registries. I concur with all three but would like to suggest several others.

    First: Journals can address this problem directly by verifying the adherence (or transparent disclosures) to pre-registered plans (see recommendations in the TOP Guidelines, Level 2 for study
    and analysis plan preregistration: cos.io/top).

    Second: Journals can also train or ask reviewers to evaluate such adherence. Simple checklists or instructions can guide reviewers.

    Third: Journals can make such disclosures normalized by asking for it whenever registered research is report.

    Fourth: Journals can directly address the incentives for such deviations (that is, the desire to report only the most clean story) by adopting Registered Reports, in which peer review occurs before results are known: cos.io/rr.

    I would encourage all journals to apply these standards and offer these formats for all hypothesis-testing, inferential research studies.
    CONFLICT OF INTEREST: I work for the non-profit Center for Open Science, that advocates for increased study registration and publishing formats to address shortcomings that are noted in this study.
    READ MORE
    Research Letter
    Nutrition, Obesity, and Exercise
    November 13, 2019

    Discrepancies in the Registries of Diet vs Drug Trials

    Author Affiliations
    • 1New Balance Foundation Obesity Prevention Center, Boston Children’s Hospital, Harvard Medical School, Boston, Massachusetts
    • 2Metabolism & Body Composition Laboratory, Pennington Biomedical Research Center, Baton Rouge, Louisiana
    JAMA Netw Open. 2019;2(11):e1915360. doi:10.1001/jamanetworkopen.2019.15360
    Introduction

    ClinicalTrials.gov was established in 2000 in response to the Food and Drug Administration Modernization Act of 1997, which called for registration of trials of investigational new drugs for serious diseases. Subsequently, the scope of ClinicalTrials.gov expanded to all interventional studies, including diet trials. Presently, prospective trial registration is required by the National Institutes of Health for grant funding and many clinical journals for publication.1 Registration may reduce risk of bias from selective reporting and post hoc changes in design and analysis.1,2 Although a study3 of trials with ethics approval in Finland in 2007 identified numerous discrepancies between registered protocols and subsequent publications, the consistency of diet trial registration and reporting has not been well explored.

    Methods

    This cross-sectional study compared the registries of drug and diet trials published in selected prominent clinical journals in the last decade. A literature search, conducted June 26, 2019, retrieved trials with obesity-related outcomes in 5 general medical journals and 1 nutrition journal with the highest impact factors for their fields (The New England Journal of Medicine, JAMA, The BMJ, The Lancet, Annals of Internal Medicine, and The American Journal of Clinical Nutrition). Results were limited to clinical trials in the 10-year period ending June 1, 2019. For drug trials, the search terms were “drug OR placebo OR pharmaceutical” with exclusion if the intervention did not focus on a specific drug (eg, dietary supplement, food extract). For diet trials, the search terms were “diet OR dietary” with exclusion if the intervention did not focus on a specific diet (eg, dietary supplement, food extract, dietary pattern such as meal skipping, or multicomponent intervention such as one including exercise). Additionally, trials in both categories were excluded if the article was not in the indicated journals (eg, BMJ Open) or was not an original randomized clinical trial; the primary registry was not in ClinicalTrials.gov; the primary outcome was not related to body weight, adiposity, or energy balance; or the primary outcome was measured in less than 28 days. For each included trial, we examined the current ClinicalTrials.gov registry, the history of changes at the ClinicalTrials.gov archive site, and the final published article, with reference to changes in or discrepancies involving the prespecified primary outcome. We calculated an odds ratio and 95% confidence interval for the number of diet vs drug trials with changes or discrepancies using SAS statistical software version 9.4 (SAS Institute). A 2-tailed P < .05 was considered statistically significant. The ethical review board of Boston Children’s Hospital does not require review of this type of study because no human participants were involved.

    Results

    Our literature search retrieved 148 drug studies and 343 diet studies, from which 9 and 21, respectively, were included in our sample after applying exclusion criteria. As shown in Table 1 and Table 2, 2 drug trials (22%) and 18 diet trials (86%) had a substantive discrepancy from initial registration, typically involving a change in time frame of the primary outcome or the number of co–primary outcomes. The odds ratio for a discrepancy for diet vs drug trials was 21.0 (95% CI, 2.9-153.8; P = .002). Among diet trial registries, 2 appeared to have been post hoc and 1 was not available.

    Discussion

    This study found that, among top-rated clinical journals, most diet trials would not satisfy an essential criterion for prospective registration, as judged by standards inferred from drug trials. This problem extends beyond low registration rates in behavioral research,4 as registration for only 1 diet trial was missing.

    Limitations of this study include a focus on just 1 (though central) aspect of registration and extrapolation of the findings to the general literature. We did not examine registry data involving interventions, participant number (anticipated vs achieved), or statistical treatments. Although our findings derive from a small subset of published trials, the highlighted problems are likely general to the field, in light of the highly selective nature of the journals examined. Furthermore, we did not control for measures of trial quality because these could be considered inherently related to, rather than confounders of, the observed association.

    Problems with diet trial registries may arise from the their greater heterogeneity and lower budgets vs drug studies and the inadequacy of infrastructural support for nutrition research.5 Ultimately, high-quality diet trials of the type needed to develop effective prevention and treatment for chronic disease will require substantial investment of financial and personnel resources from the National Institutes of Health and philanthropy. More immediate and specific remedies for the deficiencies of diet trial registries include (1) posting a detailed final statistical analysis plan before unmasking random group assignments or beginning data analyses as a minimum quality criterion; (2) declaring substantive changes to the original registry in the main manuscript (rather than infrequently read online supplementary material); and (3) creating specialized registries for diet (and possibly other behavioral) trials to reflect their special challenges beyond those of drug trials (for which current registries were originally intended).

    Back to top
    Article Information

    Accepted for Publication: September 23, 2019.

    Published: November 13, 2019. doi:10.1001/jamanetworkopen.2019.15360

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Ludwig DS et al. JAMA Network Open.

    Corresponding Author: David S. Ludwig, MD, PhD, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115 (david.ludwig@childrens.harvard.edu).

    Author Contributions: Drs Ludwig and Ebbeling had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: All authors.

    Acquisition, analysis, or interpretation of data: Ludwig, Ebbeling.

    Drafting of the manuscript: Ludwig, Heymsfield.

    Critical revision of the manuscript for important intellectual content: All authors.

    Administrative, technical, or material support: Ludwig, Ebbeling.

    Supervision: Ludwig.

    Conflict of Interest Disclosures: Dr Ludwig reported receiving grants from the National Institutes of Health, Arnold Ventures, Nutrition Science Initiative, and New Balance Foundation outside the submitted work; and royalties for books on diet and obesity. Dr Ebbeling reported receiving grants from the National Institutes of Health, Arnold Ventures, Nutrition Science Initiative, and New Balance Foundation outside the submitted work. Dr Heymsfield reported receiving personal fees from Medifast Medical Advisory Board and Tanita Medical Advisory Board outside the submitted work.

    References
    1.
    DeAngelis  CD, Drazen  JM, Frizelle  FA,  et al; International Committee of Medical Journal Editors.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors.  JAMA. 2004;292(11):1363-1364. doi:10.1001/jama.292.11.1363PubMedGoogle ScholarCrossref
    2.
    Rennie  D.  Trial registration: a great idea switches from ignored to irresistible.  JAMA. 2004;292(11):1359-1362. doi:10.1001/jama.292.11.1359PubMedGoogle ScholarCrossref
    3.
    Chan  AW, Pello  A, Kitchen  J,  et al.  Association of trial registration with reporting of primary outcomes in protocols and publications.  JAMA. 2017;318(17):1709-1711. doi:10.1001/jama.2017.13001PubMedGoogle ScholarCrossref
    4.
    Azar  M, Riehm  KE, Saadat  N,  et al.  Evaluation of journal registration policies and prospective registration of randomized clinical trials of nonregulated health care interventions.  JAMA Intern Med. 2019;179(5):624-632. doi:10.1001/jamainternmed.2018.8009PubMedGoogle ScholarCrossref
    5.
    Ludwig  DS, Ebbeling  CB, Heymsfield  SB.  Improving the quality of dietary research.  JAMA. 2019;322(16):1549-1550. doi:10.1001/jama.2019.11169PubMedGoogle ScholarCrossref
    ×