Margaret A.WinkerMD, Senior EditorIndividualAuthorPhil B.FontanarosaMD, Senior EditorIndividualAuthor
In Reply.—The purpose of our article was not to show that in comparing large trials and meta-analyses one approach was flawed and the other sound. On the contrary, we tried to describe the potential strengths and weaknesses of these complementary approaches that, when scrutinized, lead to similar conclusions. Still, Drs LeLorier and Gregoire question specifically the meaning and representativeness of the Villar and Cappelleri approach as compared with their "external comparison" approach. However, we believe, particularly for meaning and representativeness, it is mostly the external comparison approach that suffers from bias. A large trial is likely to be conducted after a meta-analysis on exactly the same clinical question, if the meta-analysis is either nonconclusive, but shows important trends, or is conclusive (reaches formal statistical significance), but seems unreliable. In both cases, a subsequent adequately powered large trial may often "disagree" with the meta-analysis at the level of attained statistical significance. Retrieving large trials from influential journals further magnifies this selection bias: such journals would not favor publishing "no news" squaring with old evidence. If, on the other hand, the meta-analysis is considered both reliable and conclusive, then a subsequent large trial is likely to be looking at a different question or a different patient population; then the
"external comparison" approach lacks meaning, since purposefully heterogeneous research is compared.
Ioannidis JPA, Cappelleri JC, Lau J. Comparing Results From Meta-analyses vs Large Trials—Reply. JAMA. 1998;280(6):518-519. doi:10-1001/pubs.JAMA-ISSN-0098-7484-280-6-jac057001