Discrepancies in the Registries of Diet vs Drug Trials

This cross-sectional study examines discrepancies between registered protocols and subsequent publications for drug and diet trials whose findings were published in prominent clinical journals in the last decade.


Introduction
ClinicalTrials.gov was established in 2000 in response to the Food and Drug Administration Modernization Act of 1997, which called for registration of trials of investigational new drugs for serious diseases. Subsequently, the scope of ClinicalTrials.gov expanded to all interventional studies, including diet trials. Presently, prospective trial registration is required by the National Institutes of Health for grant funding and many clinical journals for publication. 1 Registration may reduce risk of bias from selective reporting and post hoc changes in design and analysis. 1,2 Although a study 3  were "drug OR placebo OR pharmaceutical" with exclusion if the intervention did not focus on a specific drug (eg, dietary supplement, food extract). For diet trials, the search terms were "diet OR "Relative change in body weight (%) wk 0, wk 52" As initially registered 29945727 NCT02548585 "Percent change from baseline in MMT glucose AUC (up to 240 minutes post MMT) to end of treatment"; "change from baseline in body weight in kg to end of treatment"; both outcomes for "Cohort 4 only" Primary outcome tested in phase 2a part of study, not dose escalation part Dose escalation cohorts indicated by numbers (eg, cohort 4) in registry and letters (cohorts A-E) in published study 26840133 NCT01273584 "Birthweight centile (z-score) at birth" As initially registered Presented as median difference in published study 26284720 NCT01272232 "Change from baseline in body weight (fasting) wk 0, wk 56"; "proportion of subjects losing at least 5% of baseline body weight at 56 wk"; "proportion of subjects losing more than 10% of baseline body weight at 56 wk" As initially registered 26132939 NCT01272219 "Change from baseline in body weight (fasting) wk 0, wk 56"; "proportion of subjects losing at least 5% of baseline body weight at 56 wk"; "proportion of subjects losing more than 10% of baseline body weight at 56 wk"; "proportion of subjects with onset of type 2 diabetes at 160 wk" As initially registered Co-primary outcomes 1-3 presented in this published study; co-primary outcome 4, related to diabetes, published in Lancet 2017; 389:1399-1409; PMID: 28237263 21481449 NCT00553787 "Mean percent loss of baseline body weight and percent of subjects with at least 5% weight loss at 56 wk" As initially registered 20673995 NCT00532779 "The percentage of total body weight lost and the percentage of subjects who achieve a weight decrease of ≥5% at 56 wk" As initially registered 20647200 NCT00395135 "Proportion (%) of patients achieving ≥5% weight reduction at the end of the first year of treatment (wk 52)"; "proportion of patients maintaining ≥5% weight reduction at the end of year 2 (wk 104)" 5 Co-primary outcomes listed, including 1 with a threshold of 10% weight reduction Primary outcomes amended in registry after publication (submitted January 4, 2013) "Change from baseline in weight at 12 mo"; "baseline, 3 mo, 6 mo, 12 mo" Change from baseline in weight at 12 mo Interim points (3, 6 mo) not included; primary outcome amended in registry (submitted July 27, 2017) 28747328 NCT00938808 "Weight, No. of patients operated with knee alloplasty 1 y, 3 y" Change in body weight after 3 y and the number of participants who received total knee alloplasty during the 3-y intervention 1-y data not included in the primary outcome (provided instead in supplement table) 28298396 NCT01066806 "Weight 5 y" Percent change in body fat at 1 y Primary outcome amended in registry (submitted November 3, 2011) 27903520 NCT01750021 "Changes in adipose tissue"; "CT, body composition, molecular analyses of adipose tissue"; "time frame: baseline and 3 months" Study powered on "change of visceral fat mass measured by computed tomography" Numerous outcomes presented as primary; time frame amended in registry (submitted January 5, 2014) to include 6 mo data; published study presents only 3 mo data 26075751 NCT01152359 "Body weight 48 wk" As initially registered Registry amended after publication to "percent change in body weight " (submitted January 9, 2019) 25178568 NCT00609271 "Body weight"; "body composition"; both outcomes measured at "randomization, 3, 6, 12 mo"

Multiple anthropometric variables and disease risk factors
Study powered on body weight; authors state: "Because of the number of tests performed in the primary analyses, statistically significant results should be interpreted with caution"; registry amended after publication to include 16 discrete primary outcomes (submitted April 25, 2018) 24257725 NCT01195610 "Change in body weight "; "body fat percentage"; "dietary compliance level"; "components of metabolic syndrome"; all outcomes measured at 26 wk Change in body weight from wk 0 to wk 26 Primary outcome amended in registry (submitted April 25, 2012); other initial primary outcomes presented as secondary outcomes in published study 23255569 NCT01068197 "Change in insulin sensitivity between the 2 dietary groups"; "change in BMI z-score between the 2 dietary groups"; both outcomes measured at 3-, 12-, and 24 mo BMI z-score Manuscript states "BMI z score … was the predetermined primary efficacy variable used to assess the effectiveness of treatments"; insulin resistance (sensitivity) secondary outcome 22998340 NCT00893529 "Body weight 0, 6, 12, and 18 mo" BMI z-score change from baseline to 18 mo Primary outcome amended in registry to "body mass index z-score 0, 6, 12, and 18 months" (submitted February 4, 2011); interim points not included in calculation of primary outcome in published study 22998339 NCT00381160 "Body mass index" BMI change through 2 y Primary outcome amended in registry (submitted November 29, 2007); in the published study, 2-y data presented as primary, 1-y data as secondary 22743313 NCT00194428 "Body weight and body composition in overweight and obese person" Multiple outcomes at 6 and 18 mo, none specified as primary Registry amended to specify 6-mo but not 18-mo point (submitted January 3, 2008) 22301929 NCT01017783 "Weight change 0, 3, 6 mo" As initially registered Manuscript states "primary hypothesis … was that participants assigned to the beverage substitution groups would achieve greater weight loss at 6 mo"; both 3 and 6 mo data presented in published study 22205311 NCT00777647 "Body weight; MR spectroscopy; MRI; DEXA scan 6 mo"

Multiple anthropometric variables and disease risk factors
No outcome specified as primary in published study 21715516 NCT01266330 "Body weight"; "body composition DEXA assessment"; "oxidative stress plasma malonaldehyde, 8-isoprostane F2α and oxidized LDL"; "inflammatory stress plasma L-6 [sic], IL-15, MCP, CRP, adiponectin and TNF-α"; all measured at 12 weeks "Oxidative and inflammatory biomarkers" at 0, 1, 4, and 12 wk Published study states power calculation based on preliminary data involving CRP; body weight and composition indicated as secondary outcomes; registered after participants recruited (initial enrollment indicated as "actual") 21105792 NCT00390637 "For adults: body weight loss maintained, body composition, proportion of subjects maintaining >0, 5 and 10% weight loss, and dropout rate" Body weight loss maintenance after 6 mo Registry amended to specify 6-mo point (submitted November 3, 2007)  dietary" with exclusion if the intervention did not focus on a specific diet (eg, dietary supplement, food extract, dietary pattern such as meal skipping, or multicomponent intervention such as one including exercise). Additionally, trials in both categories were excluded if the article was not in the indicated journals (eg, BMJ Open) or was not an original randomized clinical trial; the primary registry was not in ClinicalTrials.gov; the primary outcome was not related to body weight, adiposity, or energy balance; or the primary outcome was measured in less than 28 days. For each included trial, we examined the current ClinicalTrials.gov registry, the history of changes at the ClinicalTrials.gov archive site, and the final published article, with reference to changes in or discrepancies involving the prespecified primary outcome. We calculated an odds ratio and 95% confidence interval for the number of diet vs drug trials with changes or discrepancies using SAS statistical software version 9.4 (SAS Institute). A 2-tailed P < .05 was considered statistically significant. The ethical review board of Boston Children's Hospital does not require review of this type of study because no human participants were involved.

Results
Our literature search retrieved 148 drug studies and 343 diet studies, from which 9 and 21, respectively, were included in our sample after applying exclusion criteria. As shown in Table 1 and Table 2, 2 drug trials (22%) and 18 diet trials (86%) had a substantive discrepancy from initial registration, typically involving a change in time frame of the primary outcome or the number of co-primary outcomes. The odds ratio for a discrepancy for diet vs drug trials was 21.0 (95% CI, 2.9-153.8; P = .002). Among diet trial registries, 2 appeared to have been post hoc and 1 was not available.

Discussion
This study found that, among top-rated clinical journals, most diet trials would not satisfy an essential criterion for prospective registration, as judged by standards inferred from drug trials. This problem extends beyond low registration rates in behavioral research, 4 as registration for only 1 diet trial was missing.
Limitations of this study include a focus on just 1 (though central) aspect of registration and extrapolation of the findings to the general literature. We did not examine registry data involving interventions, participant number (anticipated vs achieved), or statistical treatments. Although our findings derive from a small subset of published trials, the highlighted problems are likely general to the field, in light of the highly selective nature of the journals examined. Furthermore, we did not control for measures of trial quality because these could be considered inherently related to, rather than confounders of, the observed association.
Problems with diet trial registries may arise from the their greater heterogeneity and lower budgets vs drug studies and the inadequacy of infrastructural support for nutrition research. 5 Ultimately, high-quality diet trials of the type needed to develop effective prevention and treatment for chronic disease will require substantial investment of financial and personnel resources from the National Institutes of Health and philanthropy. More immediate and specific remedies for the deficiencies of diet trial registries include (1) posting a detailed final statistical analysis plan before unmasking random group assignments or beginning data analyses as a minimum quality criterion; (2) declaring substantive changes to the original registry in the main manuscript (rather than infrequently read online supplementary material); and (3) creating specialized registries for diet (and possibly other behavioral) trials to reflect their special challenges beyond those of drug trials (for which current registries were originally intended).