Evaluation of the Quality of Evidence of the Association of Foods and Nutrients With Cardiovascular Disease and Diabetes

This systematic review examines meta-analyses of randomized clinical trials and cohort studies for the magnitude of effect of foods, beverages, and nutrients for cardiometabolic disease.

The following principles, focusing on meta-analyses of prospective cohort studies and/or randomized controlled trials, guided the scoring for each of 9 Bradford-Hill criteria: Consistent evidence from several well-designed studies with relatively few limitations; Consistent evidence from several studies but with some important limitations; Emerging evidence from a few studies or conflicting results from several studies; -criterion not met. Definitions for each of the 9 criteria and adaptations to the general scoring system were as follows:

1.
Strength: magnitude of association, including RRs for protective factors of >0.9 ( ), 0.8-0.89 ( ), or <0.8 ( ); and for harmful factors, of <1.11 ( ), 1.25 ( ), and >1.25 ( ). Since magnitude is directly dependent on both the selected serving size and frequency of consumption, we utilized serving sizes most similar to standard dietary guidelines and frequencies of consumption representing modest, standardized differences in intake (e.g., 1 serving/d of fruit) that are easily communicated and could be feasibly achieved by an intervention.

2.
Consistency: association is repeatedly observed in different populations and circumstances, including ≥80% of included study-specific estimates being in the expected direction ( ); ≥60 -<80% ( ); ≥40 -<60% ( ); and <40% (not meeting criteria). (Though some other grading frameworks use statistical heterogeneity, this is not optimal to assess consistency as characterized by Bradford-Hill. Statistical measures of heterogeneity are influenced by both magnitudes of differences and also the numbers of studies and precision of each estimate. Thus, diet-disease relationships with few studies could have lack of consistency but fail to achieve statistical heterogeneity due to low power; while diet-disease relationships having many studies with high precision could exhibit statistical heterogeneity yet still be consistent in terms of their overall inference for the effect of the dietary factor on disease.)

3.
Temporality: exposure precedes outcome. Because all evidence was based on longitudinal studies, this was a necessary criterion ( ); when relatively few overall studies were available (<5), we graded this criterion conservatively as .

4.
Coherence: interpretation of association does not conflict with known natural history and biology of the disease, for example based on pathways of disease occurrence and laboratory findings on the dietary factor.

eMethod. Searches for Identifying Meta-analyses of the Associations of Specified Dietary Risk Factors on Cardiometabolic Diseases
For each identified diet-disease relationship, we performed multiple systematic searches of PubMed to identify meta-analyses of randomized controlled trials or prospective cohort studies evaluating these specific dietary factors and total cardiovascular disease, coronary heart disease, stroke including subtypes (ischemic, hemorrhagic), or diabetes. For sodium, and sugar-sweetened beverages and non-nutritive sweetened beverages, we also reviewed effects on blood pressure and obesity, respectively, based on randomized trials demonstrating primary effects on these risk pathways. We did not search for individual papers/studies across multiple dietary risk factors and outcomes, rather we only included published, peer-reviewed meta-analyses. Based on our and other recent reviews 5,6 , we did not include multiple other factors for which the initial appraisal identified one or more key limitations that would limit meeting the criteria for quality of

eFigure 2. Estimates of Etiologic Associations of Sodium and Systolic Blood Pressure
The boxes in the plot show the effect estimates from the meta-analyses and the horizontal lines through the boxes show the length of the confidence interval. NR: Not reported Dietary factors with probable or convincing evidence, based on the Bradford-Hill criteria, for associations on cardiometabolic (CMD) outcomes including cardiovascular disease (CVD), coronary heart disease (CHD), stroke, and type 2 diabetes. Diet-CMD relationships with <3 studies and papers that did not reference the individual studies included in the dose-response meta-analysis were not included. Number of estimates can be higher than the number of studies if more than one arm in a randomized controlled trial, if estimates were separated by age or sex in prospective cohort studies, or more than one prospective cohort study was included in a single study. Although we identified several meta-analyses on the relationship between dietary sodium intake, and SBP and DBP [7][8][9][10][11] , no studies adjusted for age-, race-, and hypertension-status interactions which have been shown to mediate the association. For this relationship, we selected the study included in our previous review paper. Available evidence suggest that sodium increases mortality from CHD, stroke, and other-blood pressure related cardiovascular diseases through effects on SBP. For every year above or below age 50, there was 0.105 mm HG (95% CI: 0.047, 0.164) larger or smaller BP reduction, respectively. Effects on CVD vs. SBP were separately identified and are not independent (i.e., effects on CVD are at least partly mediated by SBP effects). associations on cardiometabolic (CMD) outcomes including cardiovascular disease (CVD), coronary heart disease (CHD), stroke, and type 2 diabetes. Diet-CMD relationships with <3 studies and papers that did not reference the individual studies included in the dose-response meta-analysis were not included. Fruits-exclude 100% juices, and vegetables exclude vegetable juices, starchy vegetables such as potatoes and corn, and salted or pickled vegetables. Because individual studies may include potatoes in the vegetable category, the associations identified for vegetables should be considered as representing the outcome of vegetables, including potatoes.
Associations of potatoes were also evaluated separately. Evidence suggests that SSBs are associated with increased risk because they affect both BMI and BMI-independent factors of type 2 diabetes and cardiovascular outcomes. Several meta-analyses found an association between SSB intake and incident overweight or obesity, but none reported on the association between changes in SSB intake and weight gain. For this association, we selected a pooled analysis of 3 prospective cohort studies 29 .
Glycemic load is calculated as the glycemic index of a food multiplied by its carbohydrate content. Higher values reflect both higher glycemic index and higher quantities of refines grains, starches, and sugars. Evidence of associations of dietary fiber was also identified. Glycemic load and dietary fiber overlap with foods in Figure 2, including fruits, vegetables, potatoes, beans or legumes, nuts or seeds, whole grains, and refined grains. Although the Reynolds et al 31 meta-analysis contained 1 additional primary study than Livesey et al 34 , we did not select it because it included several primary studies with poor dietary instrument validity that resulted in a null association between glycemic index and diabetes. Sodium was assessed by 24-hr dietary recall, food frequency questionnaire, or 24-hr urine excretion.
No meta-analyses were identified for potassium. For this association, the study identified in the previous review 6 was selected.

eTable 3. Reasons for Excluding Dietary Factor-CMD Relationships
Dietary Factor Outcome Reason(s) for Exclusion Beans/legumes 42 CVD Consistency could not be assessed because the metaanalysis did not reference the primary studies or include a forest plot Nuts/seeds 43 Hemorrhagic stroke Consistency could not be assessed because the metaanalysis did not reference the primary studies or include a forest plot Fruit juice 44 CVD <3 primary studies included in the meta-analysis Stroke <3 primary studies included in the meta-analysis SSBs 45 MI <3 primary studies included in the meta-analysis Non-nutritive sweetened beverages 27,28 CVD Lack of biological plausibility, and supportive experiment from RCTs and cohorts of risk factors CHD Lack of biological plausibility, and supportive experiment from RCTs and cohorts of risk factors Stroke Lack of biological plausibility, and supportive experiment from RCTs and cohorts of risk factors Diabetes Lack of supportive experiment, including from RCTs and cohorts of risk factors Vegetable fiber 31 CHD <3 primary studies included in the meta-analysis PUFA 46 CVD Lack of biological plausibility, and supportive experiment from RCTs and cohorts of risk factors PUFA replacing SFA 47

Diabetes
Lack of biological plausibility, and supportive experiment from RCTs and cohorts of risk factors Coffee 48 Diabetes Lack of supportive experiment, including from RCTs and cohorts of risk factors SFA 49,50 CVD Included trials were mostly PUFA replacing SFA Stroke Lack of biological plausibility Hemorrhagic stroke Consistency could not be assessed because the metaanalysis did not reference the primary studies or include a forest plot Sodium 51 CVD Consistency could not be assessed because the metaanalysis did not reference the primary studies or include a forest plot Although meta-analyses were available for several diet-CMD relationships, no significant associations were found, such as eggs 52,53 , SFA 36,46,54 , milk 22,26,55,56 , legumes 17,19,42 , refined grains 15,17,19 , nuts 14,17,43 , MUFA 36,46,47,54 , cheese 22,55,56 , and yogurt 55,56 . Our updated search did not identify dose-response meta-analyses on lean fish, fatty fish, seafood omega-3, plant omega-3, dietary cholesterol, dietary calcium, or total energy.