To the Editor We are deeply concerned about the article by Hartz et al1 published in JAMA Psychiatry. Most of our concerns have to do with the extremely small P values reported in the text proper and in the tables. Several of them appeared to be beyond the precision capability of the statistical software (SAS) that was used. For example, in their Table 3, an odds ratio of 3.96 (95% CI, 3.61-4.35) is said to have an associated P value of 1.2 × 10−188. The other 4 P values in that table were even smaller. Whether or not those P values have been correctly calculated, there is no reason for reporting anything other than P < .0001, which is the default of SAS for very small P values. (The authors also vacillated between P being equal or less than a certain value.) Furthermore, there is no need for both confidence intervals and P values. If an odds ratio of 1 is not inside the confidence interval, the obtained odds ratio is statistically significant. The odds ratio is the measure of the effect size; the P value is not. And the former takes precedence over the latter.
Knapp TR, Hayat MJ. Very Small P Values. JAMA Psychiatry. 2014;71(8):968-969. doi:10.1001/jamapsychiatry.2014.546