[Skip to Content]
[Skip to Content Landing]
Download PDF
Table. Characteristics of Highly Cited and Recent RCTs and Cluster RCTs
Table. Characteristics of Highly Cited and Recent RCTs and Cluster RCTs
1.
Campbell MK, Elbourne DR, Altman DG.CONSORT group.  CONSORT statement: extension to cluster randomised trials.  BMJ. 2004;328(7441):702-708PubMedArticle
2.
Bland JM. Cluster randomised trials in the medical literature: two bibliometric surveys.  BMC Med Res Methodol. 2004;4:21PubMedArticle
3.
Platt R. Time for a culture change?  N Engl J Med. 2011;364(15):1464-1465PubMedArticle
4.
Jain R, Kralovic SM, Evans ME,  et al.  Veterans Affairs initiative to prevent methicillin-resistant Staphylococcus aureus infections.  N Engl J Med. 2011;364(15):1419-1430PubMedArticle
5.
Huskins WC, Huckabee CM, O’Grady NP,  et al; STAR*ICU Trial Investigators.  Intervention to reduce transmission of resistant bacteria in intensive care.  N Engl J Med. 2011;364(15):1407-1418PubMedArticle
Research Letters
Feb 25, 2013

Characteristics of Cluster Randomized Trials: Are They Living Up to the Randomized Trial?

Author Affiliations

Author Affiliations: Feinberg School of Medicine, Northwestern University, Chicago, Illinois (Mr Selvaraj); and Medical Oncology Branch, National Cancer Institute, National Institutes of Health, Bethesda, Maryland (Dr Prasad).

JAMA Intern Med. 2013;173(4):313-315. doi:10.1001/jamainternmed.2013.1638

Cluster randomized control trials (RCTs) are a form of prospective study where groups of individuals are allocated to an intervention. They offer the unique advantage of rigorously evaluating practices that cannot feasibly be randomized to the individual—such as public health or quality programs.1 While cluster RCTs can test questions traditional RCTs cannot, the design requires more participants to achieve equivalent statistical power.1 Over the last decade, the number of cluster RCTs have grown dramatically,2 but some researchers remain uncertain of how to interpret this study design.

A recent editorial highlights the debate regarding where to place cluster RCTs in the research hierarchy.3 Two paired articles in a high-impact journal reached different conclusions regarding routine screening and gown and glove precautions for patients with multidrug-resistant bacterial colonization. One article,4 a quasiexperimental before-and-after study, found that the practice worked, while another,5 a multicenter cluster RCT, found no benefit. If an observational study reaches a different result than an RCT, most would conclude the RCT got it right (ie, hormone therapy and cardiovascular risk, beta carotene therapy and cancer prevention). Yet, in the case of contact precautions, the editorial was ambivalent.3 Ambivalence would be reasonable if cluster RCTs are more likely to reach negative conclusions than RCTs. We sought to examine this hypothesis. Herein, we provide a comparison of cluster RCTs and traditional RCTs for the 50 highest-cited articles (to compare high-impact work) and the most recent 50 articles (to compare a random sampling).

Methods

We used ISI Web of Science to identify cluster and traditional RCTs based on citation count and date of appearance. Topic and title searches performed for cluster RCTs included the following: cluster randomized trial, cluster randomized controlled trial, cluster randomized study, cluster randomized controlled study, and British spellings of these terms. A similar search strategy was performed for traditional RCTs. We retrieved 200 total articles, split evenly as cluster RCTs and traditional RCTs. Within each of these types, the 50 most highly cited articles and 50 most recent articles were reviewed.

We extracted the following information for each publication: journal name, year of publication, number of times cited, total number of clusters (if applicable), total number of participants, whether the results were positive, whether mortality was examined, and if mortality was positively affected by the intervention. The Wilcoxon-Mann-Whitney test and χ2 test (or Fisher exact test when appropriate) were used to compare continuous and categorical variables, respectively. P ≤ .05 was considered statistically significant. Analysis was performed using Stata v.12 statistical software (StataCorp).

Results

Descriptive characteristics of the 4 types of articles (N = 200) are displayed in the Table. While highly cited RCTs appeared from 1991 to 2008, the earliest highly cited cluster RCT occurred in 1999. The citation count was significantly higher for highly cited RCTs than cluster RCTs (median, 1980 vs 108; P < .001). New England Journal of Medicine and British Medical Journal published the most highly cited RCTs and cluster RCTs, respectively. Highly cited cluster RCTs enrolled the same number of participants as RCTs (median, 1837 vs 1272; P = .53), whereas recent cluster RCTs enrolled more participants than recent traditional RCTs (median, 936 vs 84; P < .001). Cluster RCTs and traditional RCTs reached positive conclusions with equal frequency among highly cited studies (72% vs 80%; P = .58) and recent studies (76% vs 72%; P = .88). Highly cited cluster RCTs did not examine mortality as an end point as often as traditional RCTs (26% vs 80%; P < .001), whereas recent studies examined mortality in equal numbers (12% vs 8%; P = .74). When mortality was assessed, the results of highly cited cluster RCTs and traditional RCTs found improved mortality at equal frequency (85% vs 75%; P = .50).

Comment

Cluster RCTs address a gap in contemporary study design, and, to make sense of these trials, it is important to know whether they are comparable to time-tested RCTs. Our study demonstrates that cluster RCTs and traditional RCTs achieve the same frequency of positive study findings (both for highly cited work and a random sampling). We provide no evidence to support the belief that cluster RCTs are more likely to reach negative conclusions. Moreover, in the cases where cluster RCT findings are negative, examining the confidence interval may clarify the plausible effects of the therapy. If cluster RCTs reach different conclusions than quasiexperimental work, we find no reason why traditional experimental design hierarchies would not apply.

Notably, our study found that mortality is less often an end point in highly cited cluster RCTs than in highly cited RCTs. This remains a deficit of this burgeoning methodology. When cluster RCTs do address mortality, however, they reach positive findings as often as traditional RCTs.

In conclusion, if cluster RCTs reach negative conclusions, our study provides no reason to doubt those results. Meanwhile, cluster RCTs should more often assess mortality, a hard and important end point, to match their RCT counterparts.

Back to top
Article Information

Correspondence: Dr Prasad, Medical Oncology Branch, National Cancer Institute, National Institutes of Health, 10 Center Dr, Building 10, Room 12N226, Bethesda, MD 20892 (vinayak.prasad@nih.gov).

Published Online: January 21, 2013. doi:10.1001/jamainternmed.2013.1638

Author Contributions:Study concept and design: Prasad. Acquisition of data: Selvaraj. Analysis and interpretation of data: Selvaraj and Prasad. Drafting of the manuscript: Selvaraj and Prasad. Critical revision of the manuscript for important intellectual content: Selvaraj and Prasad. Statistical analysis: Selvaraj. Study supervision: Prasad.

Conflict of Interest Disclosures: None reported.

Disclaimer: The views and opinions of Dr Prasad do not necessarily reflect those of the National Cancer Institute or National Institutes of Health.

References
1.
Campbell MK, Elbourne DR, Altman DG.CONSORT group.  CONSORT statement: extension to cluster randomised trials.  BMJ. 2004;328(7441):702-708PubMedArticle
2.
Bland JM. Cluster randomised trials in the medical literature: two bibliometric surveys.  BMC Med Res Methodol. 2004;4:21PubMedArticle
3.
Platt R. Time for a culture change?  N Engl J Med. 2011;364(15):1464-1465PubMedArticle
4.
Jain R, Kralovic SM, Evans ME,  et al.  Veterans Affairs initiative to prevent methicillin-resistant Staphylococcus aureus infections.  N Engl J Med. 2011;364(15):1419-1430PubMedArticle
5.
Huskins WC, Huckabee CM, O’Grady NP,  et al; STAR*ICU Trial Investigators.  Intervention to reduce transmission of resistant bacteria in intensive care.  N Engl J Med. 2011;364(15):1407-1418PubMedArticle
×