In Reply: We agree with Dr Vosk that RCTs can be considered reliable only if properly designed, conducted, analyzed, and interpreted, a fundamental principle that applies equally to observational studies. Observational studies can test large effects, but RCTs of adequate size, dose, and duration are necessary to reliably detect the most plausible small to moderate effects. In all observational studies, no matter how large and well designed, the amount of uncontrolled and uncontrollable confounding can be as large as the effect sizes being sought. Meta-analysis of small trials may introduce confounding as well as small to moderate biases due to differences in designs, adherence, follow-up, and outcomes. Thus, for small to moderate effects, observational studies and meta-analyses of small trials are useful primarily to formulate, not test, hypotheses.
Hennekens CH, DeMets D. Evidence From Randomized Controlled Trials, Meta-analyses, and Subgroup Analyses—Reply. JAMA. 2010;303(13):1253-1255. doi:10.1001/jama.2010.367