Association of Industry Sponsorship With Outcomes of Nutrition Studies: A Systematic Review and Meta-analysis

Key Points

Question  Is food industry sponsorship of nutrition studies associated with outcomes that favor the sponsor?

Findings  This systematic review and meta-analysis examined 12 reports and found that 8, which included 340 studies, could be combined in a meta-analysis. Although industry-sponsored studies were more likely to have conclusions favorable to industry than non–industry-sponsored studies, the difference was not significant. There was also insufficient evidence to assess the quantitative effect of industry sponsorship on the results and quality of nutrition research.

Meaning  These findings suggest but do not establish that industry sponsorship of nutrition studies is associated with conclusions that favor the sponsors, and further investigation of differences in study results and quality is needed.

Importance  Food industry sponsorship of nutrition research may bias research reports, systematic reviews, and dietary guidelines.

Objective  To determine whether food industry sponsorship is associated with effect sizes, statistical significance of results, and conclusions of nutrition studies with findings that are favorable to the sponsor and, secondarily, to determine whether nutrition studies differ in their methodological quality depending on whether they are industry sponsored.

Data Sources  OVID MEDLINE, PubMed, Web of Science, and Scopus from inception until October 2015; the reference lists of included reports.

Study Selection  Reports that evaluated primary research studies or reviews and that quantitatively compared food industry–sponsored studies with those that had no or other sources of sponsorship.

Data Extraction  Two reviewers independently extracted data from each report and rated its quality using the ratings of the Oxford Centre for Evidence-Based Medicine, ranging from a highest quality rating of 1 to a lowest of 5.

Main Outcomes and Measures  Results (statistical significance and effect size) favorable to the sponsor and conclusions favorable to the sponsor. If data were appropriate for meta-analysis, we used an inverse variance DerSimonian-Laird random-effects model.

Results  Of 775 reports reviewed, 12, with quality ratings ranging from 1 to 4, met the inclusion criteria. Two reports, with data that could not be combined, assessed the association of food industry sponsorship and the statistical significance of research results; neither found an association. One report examined effect sizes and found that studies sponsored by the food industry reported significantly smaller harmful effects for the association of soft drink consumption with energy intake and body weight than those not sponsored by the food industry. Eight reports, including 340 studies, assessed the association of industry sponsorship with authors’ conclusions. Although industry-sponsored studies were more likely to have favorable conclusions than non–industry-sponsored studies, the difference was not significant (risk ratio, 1.31 [95% CI, 0.99-1.72]). Five reports assessed methodological quality; none found an association with industry sponsorship.

Conclusions and Relevance  Although industry-sponsored studies were more likely to have conclusions favorable to industry than non–industry-sponsored studies, the difference was not significant. There was also insufficient evidence to assess the quantitative effect of industry sponsorship on the results and quality of nutrition research. These findings suggest but do not establish that industry sponsorship of nutrition studies is associated with conclusions that favor the sponsors, and further investigation of differences in study results and quality is needed.

Dietary guidelines provide recommendations to reduce the risk of conditions such as obesity, diabetes, and cardiovascular disease. Even when dietary guidelines have been based on systematic reviews,¹ the evidence has been criticized for being biased² and guidelines contain conflicting recommendations.^3,4 Recent scrutiny of the funding practices of transnational food companies⁵ has heightened concerns about the credibility of nutrition research and how sponsorship affects the findings.^6-9 It is important to know whether funding source influences the statistical significance of the results or the effect sizes of nutrition studies and should, therefore, be considered when biases in these studies are assessed.¹⁰

Considerable evidence suggests that industry sponsorship of research is associated with outcomes that favor the sponsor.^11-14 Examinations of pharmaceutical and tobacco industry–sponsored research show that, even when controlling for methodological biases, industry-sponsored studies are more likely to have results that favor the sponsor’s product than studies with other sources of sponsorship.^11,15,16 Industry sponsors can influence the outcomes of a study in many ways, including the framing of the research questions, the design and conduct of the study, selective reporting of results, and “spin” on conclusions.^17-19 Food companies appear to use tactics similar to those of the tobacco industry to influence research.^13,20,21

Prior assessments of the influence of industry sponsorship and conflicts of interest in nutrition research have had conflicting results.^22,23 It is unclear whether studies of sponsorship bias in nutrition research have controlled for other potential biases, such as methodological quality, that could also influence research outcomes. We conducted a systematic review of studies examining the association of industry sponsorship with the statistical significance of results, effect sizes, and conclusions of nutrition research.

Our objectives were to determine whether (1) published nutrition studies with food industry sponsors are more likely to have results and/or conclusions that are favorable to the industry and (2) published nutrition studies sponsored by industry differ in their methodological quality compared with studies with other or no sponsors.

This review includes published reports that were designed to quantitatively compare food industry– and non–food industry–sponsored samples of primary nutrition research studies (such as cohort studies) or reviews. We excluded conference presentations, opinion pieces, and letters to the editor. We had no language restrictions.

We hypothesized that studies with food industry sponsorship would be more likely to have favorable results and conclusions than those without industry sponsorship. We assessed 2 primary outcomes:

For studies of health benefits, favorable results were defined as those that were statistically significant (eg, P < .05 or 95% confidence interval excluding the possibility of no difference) in favor of the sponsor’s product(s) or diet. For studies of harms, favorable results were defined as those in which harms were not statistically significant (eg, P > .05 or 95% confidence interval including the possibility of no difference) or results had a statistically significant measure of harm in the comparator group.

We also determined whether each report assessed the magnitude of effect size estimates as an outcome. The effect size measures the standardized mean difference between groups; an effect size of 0 means that there is no difference. Because the effect size is a measure of the magnitude of an effect, it can be compared across different outcome measures.

Conclusions that suggested that the nutrition intervention or exposure being studied was beneficial to health and/or safe were considered favorable to the study sponsor. Otherwise, the conclusions were considered unfavorable.

We determined whether each report compared the methodological quality of industry- vs non–industry-sponsored studies.

We searched Ovid MEDLINE, PubMed, Web of Science, and Scopus (inception to October 2015) (eMethods in the Supplement). We hand-searched the reference lists of all included reports to identify any additional relevant reports that the electronic searches missed.

Two investigators (N.C. and L.A.B.) independently screened the titles and abstracts of all retrieved records for obvious exclusions and then applied our inclusion criteria to the full text of the remaining reports. Any discrepancies were resolved by consensus. Reasons for exclusion are presented in eTable 1 in the Supplement.

Two assessors (N.C. and A.F.) independently extracted data from each included report; a third assessor (L.A.B.) adjudicated any disagreements. We contacted the authors of 2 reports to acquire missing data.

Two investigators (N.C. and L.A.B.) independently rated the quality of the included reports using the Oxford Centre for Evidence-based Medicine ratings, with a highest rating of 1 to a lowest of 5. The quality ratings are as follows: 1, properly powered and conducted randomized clinical trial; systematic review with meta-analysis; 1a, systematic review without meta-analysis; 2, well-designed controlled trial without randomization; prospective comparative cohort trial; 3, case-control studies; retrospective cohort study; 4, case series with or without intervention; cross-sectional study; 5, opinion of respected authorities; case reports.²⁴

To test our hypothesis that studies with food industry sponsorship would be more likely to have favorable conclusions than those without industry sponsorship, we conducted a meta-analysis using Review Manager 5.3 software (Cochrane Collaboration). We assessed statistical heterogeneity using the I² statistic, a statistic that quantifies the variability in effect estimates that is due to heterogeneity rather than chance. Because heterogeneity was substantial (defined as an I² > 50%), we used an inverse variance DerSimonian-Laird random-effects model for the meta-analysis. Due to the lack of homogeneous data on statistical significance of results or effect size, we could not quantitatively synthesize data (ie, conduct a meta-analysis) on these outcomes.

As shown in Figure 1, 775 references were identified and 12 reports met the inclusion criteria. Table 1 summarizes the characteristics of the included reports. The quality of the reports ranged from 1 to 4. The 12 reports were published between 2003 and 2014. The median number of included studies was 68.5 (range, 17-2539). Four reports included randomized controlled trials (RCTs) only, 2 included reviews only, and 6 included a mix of study designs. Four reports focused on the effects of sugar-sweetened beverage consumption, and 4 focused on a broad range of interventions to reduce obesity.

The reports defined industry sponsorship in different ways (Table 1). Nine reports examined associations of industry sponsorship and reported outcomes. Three reports examined both industry sponsorship and author conflicts of interest together, while 1 of these examined industry sponsorship and author conflicts of interest separately.

The most commonly studied outcome was the association of industry sponsorship with conclusions (8 reports); 5 reports assessed only conclusions. eTable 2 in the Supplement shows how conclusions that were favorable to the sponsor were defined and measured in the reports. Only 1 report assessed the association of industry sponsorship with effect size estimates, and 2 measured the association with statistical significance of the results.

Of the 12 reports, 1 was industry funded and 8 were not; 1 report had no external funding, and for 2 reports funding was not disclosed (Table 2). Authors of 3 reports had financial ties to the food industry; for 6 reports, the authors stated that they had no conflicts of interest and for 3 reports author conflicts of interest were not disclosed.

Methodological quality was assessed in 5 reports using a variety of definitions and tools (Table 3).

Neither of the 2 reports that examined the association between industry sponsorship and the statistical significance of results found an association. Both of these reports were systematic reviews (quality rating 1a). The results of the reports could not be combined because they measured statistical significance in different ways (per study vs all individual outcomes). One report containing 70 RCTs measuring the efficacy and harm of synbiotics, probiotics, and prebiotics (foods or supplements that aim to stimulate the growth of beneficial gut bacteria) found no significant association between funding source and statistically significant results for 7 of the 8 clinical outcomes examined. Overall, industry-sponsored studies reported 20.6% (73 of 354) of all clinical outcomes as favorable compared with non–industry-sponsored studies, which reported 16.7% (9 of 54) as favorable.²⁹ The second report, examining 19 RCTs assessing calcium supplementation in healthy children, found that there was insufficient variability in the study results to measure any association between study sponsorship and results; almost all study results found a statistically significant improvement in bone health outcomes.³¹

Only 1 report including 88 observational studies and RCTs examining sugar-sweetened beverages and various health outcomes assessed the relationship between industry sponsorship and effect size.³³ The report was a systematic review that analyzed RCTs and observational studies in separate meta-analyses (quality rating 1). For the harmful outcome of energy intake, overall effect size was smaller in industry-sponsored (r = 0.05; 95% CI, 0.04-0.07) compared with non–industry-sponsored (r = 0.23; 95% CI, 0.22-0.24; P < .006) studies, and for the outcome of body weight, effect size was also smaller in industry-sponsored (r = 0.02; 95% CI, 0.01-0.04) vs non–industry-sponsored (r = 0.10; 95% CI, 0.09-0.11; P < .006) studies. However, no significant difference in effect size was observed among RCTs.

Eight reports, including 340 studies, examined the association of sponsorship and conclusions, and all could be combined in a meta-analysis (Figure 2). Although industry-sponsored studies were more likely to have favorable conclusions than non–industry-sponsored studies, the difference was not significant (risk ratio [RR], 1.31; 95% CI, 0.99-1.72).

We conducted 2 additional analyses to explore heterogeneity. Two of the 8 reports defined industry sponsorship as a combination of study sponsorship and author conflicts of interest, and these could not be separated for analysis.^22,27 We conducted a sensitivity analysis excluding these 2 reports and found similar results (RR, 1.20; 95% CI, 0.93-1.54; I² = 42%). In addition, 2 reports included only reviews and not primary research studies.^22,28 Exclusion of these from the analysis produced similar results (RR, 1.11; 95% CI, 0.92-1.34; I² = 5%).

One report, with quality rating 1a, examined the association of author conflicts of interest and conclusions.²³ This report examined the health risks and nutritional value of genetically modified foods and found a significant association between author conflicts of interest and favorable study conclusions; 100% (41 of 41) of studies with author conflicts of interest reached favorable conclusions, compared with 76% (39 of 51) without author conflicts of interest (RR, 1.31; 95% CI, 1.12-1.52).

Five reports compared the methodological quality of industry-sponsored with non–industry-sponsored studies (Table 3). No reports examined the association of authors’ conflicts of interest with methodological quality. One report assessed risk of bias of the included studies using Cochrane methodology³⁸ and found that there was no significant association of industry sponsorship and random sequence generation, allocation concealment, blinding, or selective reporting.²⁹ Industry-sponsored studies had significantly less missing data than non–industry-sponsored studies.³⁸ Three reports used different tools to assess methodological quality using a score (eg, primary and review Quality Criteria Checklist and Chalmers method) (Table 3) and found no differences in quality scores between industry- and non–industry-sponsored studies.^25,28,30 One report measured quality using Consolidated Standards of Reporting Trials (CONSORT) and found that reporting was equivalent, regardless of funding.³² However, CONSORT is a guideline for reporting trials and does not assess how they are actually conducted or the means to reduce bias.^39,40

Our review identifies a gap in empirical evidence on the association of industry sponsorship or authors’ conflicts of interest and the outcomes of nutrition research. The majority of the reports examined only the effects of sponsorship on conclusions. Influence on conclusions is important to study because the relationship between industry sponsorship and conclusions favorable to the study sponsor has been previously demonstrated in tobacco,¹³ pharmaceutical,¹¹ and environmental toxin research.⁴¹ Spin on conclusions, which has been identified as a tactic used in other industries,^16,42 can influence how research is interpreted^19,42 and can undermine the credibility of research reports. From the standpoints of developing systematic reviews, dietary guidelines, and other evidence-based advice, the results are more relevant than the conclusions; for example, only the results are included in systematic reviews.

Our findings suggest that there is insufficient evidence to assess the quantitative effect of industry sponsorship on the results of nutrition research and, thus, account for this bias in systematic reviews. The 2 reports that assessed the association of sponsorship and the statistical significance of research results found no association.^29,31 This may be because there was insufficient power to compare industry- and non–industry-sponsored studies, as most of the studies were industry sponsored. In addition, funding sources of nutrition studies are often not disclosed.⁵ Improved disclosure of funding sources and larger samples for analysis should make it possible to assess the association of funding source with statistical significance of study results, as well as effect sizes. It is important to determine whether industry sponsorship affects the results of nutrition research, as has been shown for pharmaceutical industry funding of drug research.¹¹

Our review found that industry-sponsored studies were of equal or better quality than those with other funding sources. However, methodological quality was usually measured using tools that derived quality scores. The use of quality scores can be problematic because the choice of scale can influence the results of meta-analyses. Individual study domains should be assessed instead.⁴³ These findings are consistent with previous examinations of pharmaceutical and tobacco research showing that industry-sponsored studies are of equal or better quality than non–industry-funded studies.^11,15,44

Industry sponsorship can influence research results in a variety of ways. Methodological quality is only one characteristic that can influence study outcomes. Sponsors can also frame research questions to produce a desirable outcome or to generate research that diverts attention from certain questions. For example, the tobacco industry funded research on the adverse health effects of indoor air components other than tobacco smoke to distract from the evidence on harms associated with environmental tobacco smoke exposure.¹⁵ Sponsors can influence how the study is actually conducted and whether the results of the study are published in full or not.⁴⁵ Although industry sponsorship has been associated with selective reporting of research outcomes that favor the sponsor,⁴⁶ this practice was not assessed in any of the reports that we reviewed. The association of research sponsorship with the design and reporting of nutrition research should be examined.

We conducted a comprehensive search and followed explicit and well-defined inclusion and exclusion criteria for the reports. Authors of reports were also contacted for additional data. We reported on all outcomes and rated the quality of all the reports that we included.

The limited number of studies that met our inclusion criteria prevented the conduct of statistical analyses of the relationship between industry sponsorship and study results. We could not quantitatively synthesize data for all outcomes because the reports were heterogeneous. They included different topics and designs of studies and classified industry sponsorship in different ways. In addition, we only included data on sponsorship that was disclosed, and did not seek to identify industry funding or other associations that were not disclosed in the publications.

The scrutiny of the funding practices of large transnational food companies^6,7 has threatened the credibility of nutrition research and researchers.⁵ However, without empirical work examining the association of industry sponsorship with the results of nutrition research, researchers, policy makers, and the public have no way of quantifying and understanding the extent of industry influence on the data. It is challenging to rigorously assess the association of industry sponsorship with research outcomes. The quality of the reports that we examined varied. Research to quantify the influence of industry sponsorship on effect estimates can be improved by obtaining complete and accurate data on sponsors of research and conflicts of interest of sponsors and authors, and focusing on specific research questions and study designs. Thus, bias in study methods, as well as bias related to sponsorship, can be measured.

Most of the studies included in our review focused on sponsorship by large transnational food companies. However, conflicts of interest in nutrition research are complex because they encompass more than financial relationships with the manufacturers of the food products being tested.⁴⁷ For example, there is a conflict of interest if an investigator receives royalties from selling his or her own dietary advice. In addition, trade organizations representing different food groups also sponsor nutrition research.^48,49 Therefore, it is important to know whether the extent and mechanisms of bias are similar across different types of sponsors.

Previous research documenting the influence of industry sponsorship on research in other health-related fields has led to international reforms to make data more accessible, conflicts of interest and funding more transparent, and to calls for stricter standards and policies for managing conflicts of interest, critiquing and reporting evidence, and conducting systematic reviews.^10,50,51 Similar research is needed to help refine methods for evaluating studies used in systematic reviews that form the basis of dietary guidelines. Such research should also determine whether (1) biases associated with industry conflicts of interest necessitate policies for disclosure and management similar to those now widely accepted in clinical research; (2) mechanisms to reduce publication bias, such as study registries or open access data, should be considered for nutrition studies; and (3) research agendas should be revised to produce studies that are relevant to population health.

Although industry-sponsored studies were more likely to have conclusions favorable to industry than non–industry-sponsored studies, the difference was not significant. There was also insufficient evidence to assess the quantitative effect of industry sponsorship on the results and quality of nutrition research. These findings suggest but do not establish that industry sponsorship of nutrition studies is associated with conclusions that favor the sponsors, and further investigation of differences in study results and quality is needed.

Accepted for Publication: September 12, 2016.

Corresponding Author: Lisa A. Bero, PhD, University of Sydney, D17 Charles Perkins Centre Level 6, Sydney, NSW 2006, Australia (lisa.bero@sydney.edu.au).

Published Online: October 31, 2016. doi:10.1001/jamainternmed.2016.6721

Authors Contributions: Dr Bero had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Chartres, Bero.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Chartres.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Chartres, Bero.

Study supervision: Bero.

Conflict of Interest Disclosures: Mr Chartres has received a scholarship (James Milner PhD scholarship) from the University of Sydney. Dr Fabbri has received a scholarship from the University of Insubria (Varese, Italy). No other disclosures are reported.

Additional Contributions: We thank Lorraine Evison, University of Sydney, for advice on the search strategy and Quinn Grundy, PhD, and Barbara Mintzes, PhD, University of Sydney; Andreas Lundh, MD, PhD, Odense University Hospital and University of Southern Denmark; and Marion Nestle, DrPH, New York University, for comments on the manuscript. They received no compensation for these contributions.