[Skip to Content]
[Skip to Content Landing]
October 23, 1996

Large Trials vs Meta-analysis of Smaller TrialsHow Do Their Results Compare?

Author Affiliations

From the Division of Clinical Care Research (Drs Cappelleri, Schmid, de Ferranti, Chalmers, and Lau, and Mr Aubert), the Division of Geographic Medicine and Infectious Diseases (Dr loannidis), the Biostatistics Research Center (Dr Schmid), Department of Medicine, New England Medical Center, Boston, Mass. Dr Cappelleri is now with Pfizer Central Research, Groton, Conn; Dr loannidis is now with the National; Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Md; and Dr de Ferranti is now with Children's Hospital, Boston.

JAMA. 1996;276(16):1332-1338. doi:10.1001/jama.1996.03540160054033

Objective.  —To evaluate the results of large clinical trials vs the pooled results of smaller trials.

Data Identification.  —Mata-analyses with at least 1 "large" study were identified from the Cochrane Pregnancy and Childbirth Database and from MEDLINE (1966-1995).

Study Selection.  —We used a sample size approach to select 79 meta-analyses with at least 1 large study of 1000 or more patients. We used a statistical power approach to select 61 meta-analyses with at least 1 large study based on statistical power considerations.

Data Extraction.  —The outcome of interest for each meta-analysis was the primary one stated in the original publication or, when not clearly specified, was decided on clinically.

Data Synthesis.  —By random effects calculations, we found agreement between large and smaller trials in 90% of the meta-analyses selected by the sample size approach and in 82% of the meta-analyses selected by the statistical power approach. Twice as many disagreements appeared when the variability among large studies and among smaller studies was not considered (ie, fixed effects calculations). Of the 15 disagreements between results of large and smaller trials using the random effects model, plausible explanations were identified in 10 meta-analyses: 5 with differences in the control rate of events between large and smaller trials, 4 with specific protocol or study differences, and 1 with potential publication bias. Two other disagreements were not clinically important, and tentative reasons could be identified for 2 of the remaining 3 disagreements.

Conclusion.  —Results of smaller studies are usually compatible with the results of large studies, but discrepancies do occur even when the diversity among both large studies and smaller studies is considered. Clinically important differences without a potential explanation are extremely uncommon. Future research should further examine sources of heterogeneity between the results of large and smaller trials.