Margaret A.Winker, MDIndividualAuthorPhil B.FontanarosaMD, Senior EditorsIndividualAuthor
Copyright 1999 American Medical Association.
All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.1999
To the Editor: We congratulate Dr Scherer and Ms
Crawley1 for the first published study to use the
Consolidated Standards of Reporting Trials (CONSORT) statement as an
instrument to systematically evaluate the quality of the reporting of
clinical trials. However, we have some methodological concerns
regarding this study.
The authors did not prospectively estimate the sample size that would
have resulted in adequate power, and they fail to emphasize the low
power as an important limitation, especially since the study in general
failed to find significant differences in regard to its hypotheses. The
authors mention that "it is possible that our sample size was
insufficient to detect subtle changes in reporting," suggesting
incorrectly that there was adequate power to detect meaningful changes.
Power was further limited by the fact that proportions of certain
potential confounders were significantly different across journals;
comparisons were made separately for each journal, further lowering the
sample size. For example, Ophthalmology had the largest sample
size of 51 articles, 28 of which had structured abstracts and 23 did
not. If a criterion was met in 50% of the articles without structured
abstracts and if a 25% difference (ie, 50% greater or lesser than
that in articles without structured abstracts) was
considered to be meaningful, then the study had only 34% power to
detect this difference, even without correcting for multiple
comparisons. For comparisons in the Archives of Ophthalmology,
the power was even lower because the sample size was only 26.
Mago R, Crits-Christoph P. Evaluating Reporting and Abstracts of Clinical Trials. JAMA. 1999;281(1):34-35. doi:10-1001/pubs.JAMA-ISSN-0098-7484-281-1-jbk0106