The topic of reproducible findings in science has gained particular traction in the past few decades.1 The main concern is that key positive findings from influential studies may actually represent false-positive results owing to chance or bias, raising doubt about whether the mechanism under study was truly effective. In a well-known study, the Open Science Collaboration2 attempted to reproduce the findings of 100 experiments reported in the psychology literature. The replication findings were only half of the magnitude of the original findings, and only 36% of replications showed statistically significant findings. Indeed, in 2005, Ioannidis3 explained how most published research findings are false. This declaration sounds radical on its face—seeming to call into question the entire scientific endeavor—but in fact it synthesized characteristics of research that are readily appreciated. First, investigations of complex (especially biological) systems are difficult given the numerous unknown factors influencing the systems, making it difficult to isolate the mechanism under study. The only reliable way to account for both known and unknown factors in experimentation is through randomization; however, in the real world randomization is not always possible or desired. Second, chance findings happen regularly. In fact not replicating false-positive findings is actually advantageous for science. Conversely, not replicating true-positive findings is harmful, and happens often just by chance; if an experiment demonstrates that group A is superior to group B at P = .05, then if the experiment is repeated assuming the prior result was true, the probability of again showing P < .05 is only 50%. Taken together, the idea that most research findings are false seems suddenly reasonable.
Unger JM. Reproducible Findings in Systematic Reviews and Meta-analyses in Oncology: Verify, Then Trust. JAMA Oncol. Published online September 05, 2019. doi:10.1001/jamaoncol.2019.2664
Customize your JAMA Network experience by selecting one or more topics from the list below.
Create a personal account or sign in to: