[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
Views 11,516
Citations 0
August 12, 2019

Improving the Quality of Dietary Research

Author Affiliations
  • 1New Balance Foundation Obesity Prevention Center, Boston Children’s Hospital and Harvard Medical School, Boston, Massachusetts
  • 2Metabolism and Body Composition Laboratory, Pennington Biomedical Research Center, Baton Rouge, Louisiana
JAMA. Published online August 12, 2019. doi:10.1001/jama.2019.11169

Compared with pharmaceutical research, studies evaluating diet or dietary interventions for chronic diseases like obesity (to be distinguished from micronutrient deficiency syndromes like scurvy or rickets) have far greater challenges in terms of consistency, quality control, confounding, and interpretation. For instance, a placebo for an active drug can be easily prepared, but dietary assignment can rarely be truly masked. Moreover, diet is highly heterogeneous, with myriad interacting and potentially confounding factors. If an intervention increases intake from one food category, participants may well eat less from other food categories.

In addition, for any diet type, many possible variations exist. For example, does a low-fat diet include high amounts of sugar? Is a high-protein diet plant or animal based? Does a ketogenic diet have unrealistically low saturated fat content? Whereas a drug typically acts through one or a limited number of biological pathways, even discrete changes in diet (such as the ratio of dietary fat to carbohydrate) will directly affect numerous hormones and metabolic pathways involving many organ systems. The lack of uniformity across studies and the difficulty in establishing mechanisms make translation into public policy exceedingly difficult, a problem now being addressed by the 2020 Dietary Guidelines Advisory Committee.1

Virtually all nutrition research must also contend with the challenge of behavior modification. Changing a diet is much more difficult than taking a pill, and adherence to dietary interventions is more difficult to assess (ie, compared with measurement of drug levels in serum or urine). Feeding studies conducted on metabolic wards provide continuous observation of participants in a research unit and aim to circumvent this problem, but do so under artificial environmental conditions that affect eating behavior, physical activity, stress level, and likely other factors that affect diet-related outcomes.

Furthermore, many environmental, psychological, and behavioral factors can influence eating behavior over the short-term, but in the long-term biological factors likely predominate. With calorie restriction, hunger increases and metabolic rate slows, and these adaptive biological responses antagonize ongoing weight loss and undermine long-term dietary adherence.2 For this reason, short-term studies (<1 month for mechanistic research and <1 year for behavioral trials), the vast majority in nutrition, may reveal only transitory phenomena of unclear significance to the understanding of how a specific diet moderates a chronic disease over many years.3

Complicating matters further, funding for diet studies is a small fraction of that for drug discovery and development, not only because of the limited National Institutes of Health (NIH) budget but also because few large companies stand to profit from novel dietary treatments for chronic disease. In addition, diets are not easily protected by intellectual property and their success may decrease the need for new drugs. When food companies do fund research, they tend to focus on their own products and, without the protections now standard in pharmaceutical research, the findings may be fraught with bias.4

Consequently, most diet studies have a limited budget, often with inadequate infrastructure support (ie, lacking involvement of commercial research organizations or specific mechanisms to ensure high-quality registries, databases, and data reporting). The few major NIH-sponsored multicentered trials (eg, Look Ahead study, Women’s Health Initiative Dietary Modification Trial, and Child and Adolescent Trial for Cardiovascular Health) have tested conventional low-fat diets, and each has shown no difference between the control and experimental groups for the primary outcome. Promising novel approaches, such as the very low–carbohydrate diet for diabetes,5 have not been tested in any study with comparable size and budget.

An additional challenge is the lack of standardization in study design. Drug development begins with preclinical proof of principle, then proceeds to assess efficacy, effectiveness, and real-world experience in phase 1 through 4 trials. This orderly progression may not always be practical or desirable in nutrition. However, the interpretation of diet trials often exceeds the limitations of study design, as occurs when pilot studies with low power or effectiveness studies with low-treatment intensity reach conclusions about the comparative efficacy of 2 or more diets.

For instance, in the Pounds Lost study,6 811 participants with overweight or obesity were assigned to 1 of 4 diets differing in protein, carbohydrate, and fat content, and were provided counseling sessions to promote behavior change. After 2 years, the diet groups did not differ in weight loss, a finding widely interpreted to mean that macronutrient composition does not matter in obesity treatment.

However, the maximum self-reported macronutrient differences among the groups was less than half the target, and even these small differences may have been inflated by social desirability bias (ie, when a participant instructed to follow a diet overreports or underreports consumption of compliant or noncompliant foods, respectively). Moreover, biomeasures of adherence (serum level of triglycerides, urinary nitrogen excretion) suggested little differentiation in diet among the 4 treatment groups.

Although the Pounds Lost study underscores the need for more intensive intervention to promote long-term behavior change in the modern food environment, the findings cannot inform current understanding of the true efficacy of diets when people consume them as intended. Even for costly diseases like type 2 diabetes, dietary trials have characteristically used comparatively low–intensity interventions, biasing comparison with other treatment options such as bariatric surgery.7

There is a path forward that may improve the quality of dietary research:

  • Recognize that the design features of phase 3 drug studies are not always feasible or appropriate in nutrition research, and clarify the minimum standards necessary for diet studies to be considered successful.

  • Distinguish among study design categories, including mechanistic, pilot (exploratory), efficacy (explanatory), effectiveness (pragmatic), and translational (with implications for public health and policy). Each of these study types is important for generating knowledge about diet and chronic disease, and some overlap may invariably exist; however, the findings from small-scale, short-term, or low-intensity trials should not be conflated with definitive hypothesis testing.

  • Acknowledge that changes to, or discrepancies in, clinical registries of diet trials are commonplace, and update final analysis plans before unmasking random study group assignments and initiating data analysis.

  • Define diets more precisely when feasible (eg, with quantitative nutrient targets and other parameters, rather than qualitative descriptors such as Mediterranean) to allow for rigorous and reproducible comparisons.

  • Improve the methods for addressing common design challenges, such as how to promote adherence to dietary prescriptions (ie, with feeding studies and more intensive behavioral and environmental intervention), and reduce dropout or loss to follow-up.

  • Develop sensitive and specific biomeasures of adherence (eg, metabolomics), and use available methods when feasible (eg, doubly labeled water method for total energy expenditure).

  • Create and adequately fund local (or regional) cores to enhance research infrastructure.

  • Standardize practices to mitigate the risk of bias related to conflicts of interest in nutrition research, including independent oversight of data management and analysis, as has been done for drug trials.

  • Make databases publicly available at time of study publication to facilitate reanalyses and scholarly dialogue.

  • Establish best practices for media relations to help reduce hyperbole surrounding publication of small, preliminary, or inconclusive research with limited generalizability.

Improvement of the quality of nutrition research will likely require substantial financial and human resource investment, but holds promise to reduce the massive human toll and financial burden of diet-related chronic disease.

Back to top
Article Information

Corresponding Author: David S. Ludwig, MD, PhD, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115 (david.ludwig@childrens.harvard.edu).

Published Online: August 12, 2019. doi:10.1001/jama.2019.11169

Conflict of Interest Disclosures: Dr Ludwig reported receiving grants from the National Institutes of Health, the Laura and John Arnold Foundation, the Nutrition Science Initiative, and the New Balance Foundation; and receiving royalties for books on obesity and nutrition that recommend a carbohydrate-modified diet. Dr Ebbeling reported receiving grants from the National Institutes of Health, the Laura and John Arnold Foundation, the Nutrition Science Initiative, and the New Balance Foundation. Dr Heymsfield reported receiving personal fees from Medifast, Tanita Corp, and Janssen; and being a current member of the 2020 US Dietary Guidelines Committee.

Disclaimer: The views expressed are those of the authors and not the US Dietary Guidelines Committee.

Dietary guidelines for Americans, work under way: review of the science. https://www.dietaryguidelines.gov/work-under-way/review-science. Accessed July 6, 2019.
Leibel  RL, Rosenbaum  M, Hirsch  J.  Changes in energy expenditure resulting from altered body weight.  N Engl J Med. 1995;332(10):621-628. doi:10.1056/NEJM199503093321001PubMedGoogle ScholarCrossref
Ludwig  DS, Astrup  A, Bazzano  LA,  et al.  Ultra-processed food and obesity: the pitfalls of extrapolation from short studies.  Cell Metab. 2019;30(1):3-4. doi:10.1016/j.cmet.2019.06.004PubMedGoogle ScholarCrossref
Litman  EA, Gortmaker  SL, Ebbeling  CB, Ludwig  DS.  Source of bias in sugar-sweetened beverage research: a systematic review.  Public Health Nutr. 2018;21(12):2345-2350. doi:10.1017/S1368980018000575PubMedGoogle ScholarCrossref
Feinman  RD, Pogozelski  WK, Astrup  A,  et al.  Dietary carbohydrate restriction as the first approach in diabetes management: critical review and evidence base.  Nutrition. 2015;31(1):1-13. doi:10.1016/j.nut.2014.06.011PubMedGoogle ScholarCrossref
Sacks  FM, Bray  GA, Carey  VJ,  et al.  Comparison of weight-loss diets with different compositions of fat, protein, and carbohydrates.  N Engl J Med. 2009;360(9):859-873. doi:10.1056/NEJMoa0804748PubMedGoogle ScholarCrossref
Ludwig  DS, Ebbeling  CB, Livingston  EH.  Surgical vs lifestyle treatment for type 2 diabetes.  JAMA. 2012;308(10):981-982. doi:10.1001/2012.jama.10156PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Don't Forget Enteral Nutrition
    Carlo Pedrolli, M.D. | S. Chiara Hospital, Clinical Nutrition Unit , Trento, Itali
    I read with much interest this paper.

    I would suggest to add a proposal to improve quality of dietary research.

    We have thousands of patients fed by enteral nutrition in which the variability of nutrients is very little; in a way there are many patients in whom we know exactly how many calories we give, how many proteins, how many carbohydrates, how much fiber and how many micronutrients we give as nutritional intake and it is possible to calculate exactly how much they eat in a day.

    I think we use this information too infrequently in research
    and I would advise to do it to enhance the quality of dietary research; do you agree?