Selvaraj S, Prasad V. Characteristics of Cluster Randomized Trials: Are They Living Up to the Randomized Trial?. JAMA Intern Med. 2013;173(4):313-315. doi:10.1001/jamainternmed.2013.1638
Author Affiliations: Feinberg School of Medicine, Northwestern University, Chicago, Illinois (Mr Selvaraj); and Medical Oncology Branch, National Cancer Institute, National Institutes of Health, Bethesda, Maryland (Dr Prasad).
Cluster randomized control trials (RCTs) are a form of prospective study where groups of individuals are allocated to an intervention. They offer the unique advantage of rigorously evaluating practices that cannot feasibly be randomized to the individual—such as public health or quality programs.1 While cluster RCTs can test questions traditional RCTs cannot, the design requires more participants to achieve equivalent statistical power.1 Over the last decade, the number of cluster RCTs have grown dramatically,2 but some researchers remain uncertain of how to interpret this study design.
A recent editorial highlights the debate regarding where to place cluster RCTs in the research hierarchy.3 Two paired articles in a high-impact journal reached different conclusions regarding routine screening and gown and glove precautions for patients with multidrug-resistant bacterial colonization. One article,4 a quasiexperimental before-and-after study, found that the practice worked, while another,5 a multicenter cluster RCT, found no benefit. If an observational study reaches a different result than an RCT, most would conclude the RCT got it right (ie, hormone therapy and cardiovascular risk, beta carotene therapy and cancer prevention). Yet, in the case of contact precautions, the editorial was ambivalent.3 Ambivalence would be reasonable if cluster RCTs are more likely to reach negative conclusions than RCTs. We sought to examine this hypothesis. Herein, we provide a comparison of cluster RCTs and traditional RCTs for the 50 highest-cited articles (to compare high-impact work) and the most recent 50 articles (to compare a random sampling).
We used ISI Web of Science to identify cluster and traditional RCTs based on citation count and date of appearance. Topic and title searches performed for cluster RCTs included the following: cluster randomized trial, cluster randomized controlled trial, cluster randomized study, cluster randomized controlled study, and British spellings of these terms. A similar search strategy was performed for traditional RCTs. We retrieved 200 total articles, split evenly as cluster RCTs and traditional RCTs. Within each of these types, the 50 most highly cited articles and 50 most recent articles were reviewed.
We extracted the following information for each publication: journal name, year of publication, number of times cited, total number of clusters (if applicable), total number of participants, whether the results were positive, whether mortality was examined, and if mortality was positively affected by the intervention. The Wilcoxon-Mann-Whitney test and χ2 test (or Fisher exact test when appropriate) were used to compare continuous and categorical variables, respectively. P ≤ .05 was considered statistically significant. Analysis was performed using Stata v.12 statistical software (StataCorp).
Descriptive characteristics of the 4 types of articles (N = 200) are displayed in the Table. While highly cited RCTs appeared from 1991 to 2008, the earliest highly cited cluster RCT occurred in 1999. The citation count was significantly higher for highly cited RCTs than cluster RCTs (median, 1980 vs 108; P < .001). New England Journal of Medicine and British Medical Journal published the most highly cited RCTs and cluster RCTs, respectively. Highly cited cluster RCTs enrolled the same number of participants as RCTs (median, 1837 vs 1272; P = .53), whereas recent cluster RCTs enrolled more participants than recent traditional RCTs (median, 936 vs 84; P < .001). Cluster RCTs and traditional RCTs reached positive conclusions with equal frequency among highly cited studies (72% vs 80%; P = .58) and recent studies (76% vs 72%; P = .88). Highly cited cluster RCTs did not examine mortality as an end point as often as traditional RCTs (26% vs 80%; P < .001), whereas recent studies examined mortality in equal numbers (12% vs 8%; P = .74). When mortality was assessed, the results of highly cited cluster RCTs and traditional RCTs found improved mortality at equal frequency (85% vs 75%; P = .50).
Cluster RCTs address a gap in contemporary study design, and, to make sense of these trials, it is important to know whether they are comparable to time-tested RCTs. Our study demonstrates that cluster RCTs and traditional RCTs achieve the same frequency of positive study findings (both for highly cited work and a random sampling). We provide no evidence to support the belief that cluster RCTs are more likely to reach negative conclusions. Moreover, in the cases where cluster RCT findings are negative, examining the confidence interval may clarify the plausible effects of the therapy. If cluster RCTs reach different conclusions than quasiexperimental work, we find no reason why traditional experimental design hierarchies would not apply.
Notably, our study found that mortality is less often an end point in highly cited cluster RCTs than in highly cited RCTs. This remains a deficit of this burgeoning methodology. When cluster RCTs do address mortality, however, they reach positive findings as often as traditional RCTs.
In conclusion, if cluster RCTs reach negative conclusions, our study provides no reason to doubt those results. Meanwhile, cluster RCTs should more often assess mortality, a hard and important end point, to match their RCT counterparts.
Correspondence: Dr Prasad, Medical Oncology Branch, National Cancer Institute, National Institutes of Health, 10 Center Dr, Building 10, Room 12N226, Bethesda, MD 20892 (firstname.lastname@example.org).
Published Online: January 21, 2013. doi:10.1001/jamainternmed.2013.1638
Author Contributions:Study concept and design: Prasad. Acquisition of data: Selvaraj. Analysis and interpretation of data: Selvaraj and Prasad. Drafting of the manuscript: Selvaraj and Prasad. Critical revision of the manuscript for important intellectual content: Selvaraj and Prasad. Statistical analysis: Selvaraj. Study supervision: Prasad.
Conflict of Interest Disclosures: None reported.
Disclaimer: The views and opinions of Dr Prasad do not necessarily reflect those of the National Cancer Institute or National Institutes of Health.