Customize your JAMA Network experience by selecting one or more topics from the list below.
Balk EM, Bonis PAL, Moskowitz H, et al. Correlation of Quality Measures With Estimates of Treatment Effect in Meta-analyses of Randomized Controlled Trials. JAMA. 2002;287(22):2973–2982. doi:10.1001/jama.287.22.2973
Author Affiliations: Evidence-based Practice Center, Division of Clinical Care Research, Tufts University School of Medicine, New England Medical Center, Boston, Mass (Drs Balk, Bonis, Moskowitz, Schmid, Wang, and Lau); and the Biomedical Research Institute, Foundation for Research and Technology Hellas, Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece (Dr Ioannidis). Dr Moskowitz is now with the Division of General Pediatrics, Mount Sinai Hospital, Mount Sinai School of Medicine, New York, NY.
Context Specific features of trial quality may be associated with exaggeration
or shrinking of the observed treatment effect in randomized studies. Therefore,
assessment of trial quality is often used in meta-analysis. However, the degree
to which specific quality measures are associated with treatment effects has
not been well established across a broad range of clinical areas.
Objective To determine if quality measures are associated with treatment effect
size in randomized controlled trials (RCTs).
Design Quality measures from published quality assessment scales were evaluated
in RCTs included in meta-analyses from 4 medical areas (cardiovascular disease,
infectious disease, pediatrics, and surgery). Included meta-analyses incorporated
at least 6 RCTs, examined dichotomous outcomes, and demonstrated significant
between-study heterogeneity in the odds ratio (OR) scale.
Main Outcome Measures Relative ORs comparing overall treatment effect (summary OR) of high
vs low-quality studies, as determined by each quality measure, with relative
ORs less than 1 indicating larger treatment effect in low-quality studies.
Results Twenty-four quality measures were analyzed for 276 RCTs from 26 meta-analyses.
Relative ORs of high vs low-quality studies for these quality measures ranged
from 0.83 to 1.26; none was statistically significantly associated with treatment
effect. The proportion of studies fulfilling specific quality measures varied
widely in the 4 medical areas. In analyses limited to specific medical areas,
placebo control, multicenter studies, study country, caregiver blinding, and
statistical methods were significantly associated with treatment effect on
7 occasions. These relative ORs ranged from 0.40 to 1.74. However, the directions
of these associations were not consistent.
Conclusions Individual quality measures are not reliably associated with the strength
of treatment effect across studies and medical areas. Although use of specific
quality measures may be appropriate in specific well-defined areas in which
there is pertinent evidence, findings of associations with treatment effect
cannot be generalized to all clinical areas or meta-analyses.
Create a personal account or sign in to: