[Skip to Content]
[Skip to Content Landing]
Views 834
Citations 0
September 5, 2019

Reproducible Findings in Systematic Reviews and Meta-analyses in Oncology: Verify, Then Trust

Author Affiliations
  • 1Fred Hutchinson Cancer Research Center, Seattle, Washington
  • 2SWOG Statistics and Data Management Center, Seattle, Washington
JAMA Oncol. Published online September 5, 2019. doi:10.1001/jamaoncol.2019.2664

The topic of reproducible findings in science has gained particular traction in the past few decades.1 The main concern is that key positive findings from influential studies may actually represent false-positive results owing to chance or bias, raising doubt about whether the mechanism under study was truly effective. In a well-known study, the Open Science Collaboration2 attempted to reproduce the findings of 100 experiments reported in the psychology literature. The replication findings were only half of the magnitude of the original findings, and only 36% of replications showed statistically significant findings. Indeed, in 2005, Ioannidis3 explained how most published research findings are false. This declaration sounds radical on its face—seeming to call into question the entire scientific endeavor—but in fact it synthesized characteristics of research that are readily appreciated. First, investigations of complex (especially biological) systems are difficult given the numerous unknown factors influencing the systems, making it difficult to isolate the mechanism under study. The only reliable way to account for both known and unknown factors in experimentation is through randomization; however, in the real world randomization is not always possible or desired. Second, chance findings happen regularly. In fact not replicating false-positive findings is actually advantageous for science. Conversely, not replicating true-positive findings is harmful, and happens often just by chance; if an experiment demonstrates that group A is superior to group B at P = .05, then if the experiment is repeated assuming the prior result was true, the probability of again showing P < .05 is only 50%. Taken together, the idea that most research findings are false seems suddenly reasonable.