CONSORT Revised—Improving the Reporting of Randomized Trials

If physicians are to base treatment decisions on the evidence in the medical literature, all the relevant results of trials must be available easily and consistently. Yet it is common to have trouble identifying the hypothesis, the research question, and the design of a published trial. It is even more common to lose count of the participants or to be unable to tell who received what therapies and the type of analysis used. As a result, it is often impossible to know whether the conclusions are justified by the data.

In February 1995, Schulz and colleagues¹ published an important article that drew attention to this sad state of affairs and to the importance of complete reporting of clinical trials if bias was to be avoided. At that time 2 groups, responding to the widespread perception that reporting of study results was highly variable, had held meetings to try to articulate the standards of good reporting. The first, the Standards of Reporting Trials (SORT) group, met in Ottawa, Ontario, in 1993 and published its recommendations in December 1994.² The second group met at Asilomar, Calif, in 1994 and also published its recommendations for reporting of clinical trials at the end of that year.³

In 1995, JAMA editors persuaded the authors of a clinical trial already accepted for publication to rewrite their article to conform with all the SORT recommendations.^4,5 The report turned out to be an extraordinary, one-of-a-kind prototype. The article was structured into more than 30 parts, was apparently agonizing to write, and was certainly a torture to read. But the experiment was instructive because the authors, and numerous readers, let us know their reactions and suggestions. It was apparent that the SORT recommendations were too inflexible, too mechanistic, and too little concerned with the external validity or applicability of the trial results.⁵

Under the leadership of David Moher, the 2 groups then pooled their results.⁵ In 1996, the result of this cooperation—the Consolidated Standards of Reporting Trials (CONSORT) statement—was published.^6,7 The statement described the process, provided the rationale for the various reporting requirements listed, and, crucially, allowed a mechanism that permitted considerable individual freedom on the part of journal editors in the way they set forth the information. In addition, CONSORT recommended a flow diagram so that the reader might easily follow the progress of participants through the various stages of the trial, and in every arm of the study.

Some trialists confused CONSORT with the earlier SORT and so damned CONSORT for the very reasons that had brought about the changes found in CONSORT.^8,9 They objected to what they perceived as unilateral imposition of excessively prescriptive rules by nontrialists.⁸ This view ignored all the experienced trialists who had taken part in the lengthy process of feedback, of trial and error, that had informed the final document.⁹ Fortunately, no substantive criticisms were raised⁹ and the general reaction was strongly favorable.

CONSORT was republished in and endorsed by many journals, in several languages (http://www.consort-statement.org). Its use is recommended by the International Committee of Medical Journal Editors, the Council of Science Editors, and the World Association of Medical Editors. CONSORT has been so successful that similar groups of scientists and editors have set up standards, using a similar template and based on empirical evidence whenever possible, to increase the quality of reporting of meta-analysis of randomized trials (QUOROM),¹⁰ and the reporting of the meta-analysis of observational studies (MOOSE).¹¹ An effort led by Jeroen Lijmer to establish standards to improve the reporting of studies assessing diagnostic tests (STARD) is proceeding, and this group is expected to publish its recommendations soon. Matthias Egger and colleagues are setting up similar groups for improving the reporting of case-control and cohort studies.

Anyone who has read through many hundreds of randomized controlled trials (RCTs) is immediately struck by the fact that when the authors have used the CONSORT checklist and flow diagram, it takes a fraction of the time to get the essential information necessary to assess the quality of a trial. In this issue of THE JOURNAL, Moher et al¹² show in 3 large medical journals, including JAMA, that adopted the CONSORT statement, study reporting was improved more than in one such journal that has not chosen to do so. However, before concluding that CONSORT is some sort of panacea, Moher et al also demonstrate that even in journals that have made a strong commitment to CONSORT, reporting was deficient in many ways, for example, in detailing concealment of allocation.¹² This shows that editors have as much difficulty as trialists in learning new behaviors.

But the CONSORT statement, developed after a great deal of discussion and some experimentation, was never intended to be set in concrete. From the first, it was assumed that CONSORT would have to change as new evidence accumulated on the importance of various items of reporting. In 1998, Meinert¹³ published a detailed and cogent critique of CONSORT and made several important suggestions for its improvement. In that same issue of JAMA, Moher detailed the often woeful ways in which trials were still being reported and stressed the importance of evidence in improving what he considered to be "an evolving tool," CONSORT.¹⁴

Also in this issue of THE JOURNAL is an article by Egger et al,¹⁵ who examine the use of flow diagrams recommended in the CONSORT statement to show the path of participants from enrollment to analysis. Egger et al found that use of the diagrams was associated with more complete reporting and recommend that all reports of RCTs include them. However, the authors noted problems with the use of the recommended flow diagram, with few of the reports they studied including the number of participants who received treatments as allocated. Moreover, as Meinert pointed out,¹³ there were deficiencies in the diagram itself, such as failure to recommend that the number of participants included in the main analysis be specified, a number needed to assess whether an intention-to-treat analysis had been performed. Egger et al make some sensible suggestions for revision of the flow diagram¹⁵ and these revisions have been incorporated in the revised CONSORT statement.

Publication of the revised CONSORT statement in this issue of THE JOURNAL¹⁶ coincides with its simultaneous publication in the Annals of Internal Medicine and The Lancet, 2 journals that have, in addition to JAMA, strongly supported CONSORT. Just as the original SORT and Asilomar statements were revised in response to public comments and experience, so the CONSORT statement, flow diagram, and the checklist have all been revised. The checklist, as before, consists of items that empirical evidence has shown must be included if reporting bias is to be minimized. The changes are listed in the new statement.¹⁶ To help with the use and dissemination of the revised statement, an explanatory and elaboration article is being published with the CONSORT statement in the Annals of Internal Medicine.¹⁷ The revision of CONSORT, which is clearer and more flexible, should make it even easier for authors and editors to use and should greatly improve the transparency of reporting. And since the reporting of a trial is inseparable from the rest of its conduct, I expect these revised standards eventually to serve a valuable educational function and improve the way trials are conducted.¹⁸

Finally, in this issue of THE JOURNAL Devereaux and colleagues¹⁹ report that there is a long way to go to achieve an acceptable degree of precision in thinking about as well as in reporting clinical trials. The authors assessed whom physicians understood to have been blinded (masked) when a study was reported as using single, double, and triple blinding. They found that physicians' interpretations were quite variable, with the respondents offering 10, 17, and 15 unique interpretations, respectively, of the 3 sorts of blinding. When the authors looked at recently published textbooks, they found 5, 9, and 7 different interpretations of each. Lewis Carroll's Humpty Dumpty could say: "When I use a word . . . it means just what I choose it to mean—neither more nor less,"²⁰ but in the interpretation of science, there is no place for such ambiguity. As Devereaux et al suggest, the answer must be for authors to describe exactly and completely what they did, which is what CONSORT and these other initiatives are all about.

The whole of medicine depends on the transparent reporting of clinical trials. There is plenty of evidence for biased reporting due to commercial influences.²¹ To retain credibility, trialists, other researchers, and editors have to show the profession and the public evidence that we are making an earnest attempt to achieve the very highest standards of transparent reporting.