To the Editor: The article by Dr Ioannidis1 reported that it is not uncommon for highly cited studies of clinical interventions and their outcomes to be contradicted by subsequent research. The author noted several possible factors for this finding, including study-to-study variability and publication bias.
An additional contributor to the observed discrepancy among study results concerns the meaning of the typical marker of research significance, the P value. The P value is the probability of a finding, assuming that there is in truth no effect; it is not simply the probability that there is no effect. As noted by Browner and Newman,2 research studies are analogous to diagnostic tests and P values are like false-positive rates when what we really seek is akin to a positive predictive value.
Contradictions in Highly Cited Clinical Research. JAMA. 2005;294(21):2695-2696. doi:10.1001/jama.294.21.2695-a