Concerns About Reliability in the Trial to Assess Chelation Therapy (TACT)

Randomized controlled trials (RCTs) are considered the most robust source of scientific evidence to inform the medical community about the benefits and risks of therapeutic interventions. In recommendations for practitioners, treatment guidelines recognize the special value of RCTs by designating such studies as the highest level of evidence in assessing the efficacy of various therapeutic strategies. However, despite the acknowledged importance of RCTs, all randomized trials are not equivalent in reliability, credibility, and value. Every trial has limitations that can compromise the study's interpretability and undermine the strength of its conclusions. In extreme cases, a poor-quality RCT can lead to important patient and societal harms.^1,2

In this issue of JAMA, the report by Lamas et al of the Trial to Assess Chelation Therapy (TACT)³ represents a situation in which many important limitations in the design and execution of a clinical trial compromise the reliability of the study and render the results difficult to interpret. Unfortunately, the efforts of these investigators fell short of the minimum level of quality necessary to adequately answer the question they sought to investigate. Nonetheless, all RCTs should be published because even failed trials provide valuable scientific lessons for the medical community. Accordingly, TACT provides useful insights into the overwhelming challenges faced when trying to determine the effectiveness of an unusual and controversial therapy.

The evolution of clinical trial design over the past 4 decades is based on the principle that a high-quality RCT must effectively minimize bias and variability. Bias is reduced by randomization of patients to alternative treatment strategies, blinding (masking) of all participants (patients and caregivers) to the treatment assignment, and use of an intention-to-treat approach that analyzes patients in their originally assigned treatment group. High levels of patient retention are essential to maintain the integrity of randomization. Validity in clinical trials is enhanced by selecting a sample size large enough to adequately test the hypothesis and through central adjudication of important and objective patient outcomes.

Execution of a high-quality RCT requires skilled investigators and study coordinators who understand these critical scientific principles. For TACT, more than 60% of patients were randomized at enrolling centers described as complementary and alternative medicine sites. Many of these centers have websites that describe their services, which include an array of unproven therapies ranging from stem cell therapy to regrow breasts after mastectomy, high-dose intravenous vitamin C to treat cancer, and use of cinnamon for treating diabetes to treatment of influenza with antimicrobial essential oils or homeopathic remedies (while warning patients not to undergo immunization). Other sites offer chelation to treat or cure a variety of conditions including autism in children. A common theme of these centers is evident—they appear to attempt to appeal to vulnerable patients who have challenging diseases by offering a variety of unscientific and unproven therapies. Whether a high-quality RCT can be performed at such sites is questionable.

Not surprisingly, with a high fraction of such study sites, TACT showed some important deviations from adherence to the scientific principles of a well-controlled trial. The study randomized 1708 patients, but 311 (18%) were lost to follow-up, nearly all because of withdrawal of consent (289 patients), and importantly, these withdrawals were not equally distributed between the treatment groups. Significantly more patients (n=174) withdrew from the placebo group compared with the chelation group (n=115; hazard ratio, 0.66; P = .001). A similar imbalance in discontinuation from randomized treatment was observed—281 in the placebo group and 233 in the chelation group.

In some RCTs, more patients stop study treatment in the active treatment group because of toxicity or adverse drug effects. However, in TACT, why would patients differentially withdraw in such large numbers from the placebo group? A logical explanation is unmasking of treatment assignments. If either the investigators or the patients knew who was receiving chelation, patients assigned to the placebo group would likely be influenced to withdraw or stop study treatment, particularly when some investigators were advocates for chelation therapy.

The substantial nonretention of study participants is sufficient to compromise the validity of the study results. The primary end point occurred in only 39 fewer patients in the chelation treatment group compared with the control group, with a P =.035 that just barely reached statistical significance (adjusted P =.036 for significance after interim analyses). The occurrence of the primary end point in just a few more patients in the chelation treatment group would yield a statistically nonsignificant result. The authors try to make a case that nonretention would not likely have changed the study results, but their methods of imputing data are not sufficient to definitively make this conclusion. No imputation strategy can successfully recover missing outcome data when the missing data are unequally distributed between treatment groups and the treatment benefit is barely statistically significant. Nonretention is such a critical problem in clinical trials that the Institute of Medicine (IOM) convened a working group that issued a comprehensive report on the problem.⁴ In describing the IOM's findings, one statistician summarized, “The preferred and often only satisfactory approach to addressing missing data is to prevent it.”⁵

Differential dropout in TACT suggests unmasking, but the problem of intentional unblinding is more concerning. The sponsors of the trial, the National Heart, Lung, and Blood Institute (NHLBI) and the National Center for Complementary and Alternative Medicine (NCCAM), were unblinded throughout the trial. The National Institutes of Health policy unwisely allows the sponsor access to unblinded trial data, and both organizations sent observers to the closed sessions of the data monitoring committee. This gave them access to confidential data during each of the 11 interim analyses. The unblinding of the study sponsor represents a serious deviation from acceptable standards of conduct for supervision of clinical trials. If a pharmaceutical company sponsoring a trial were allowed access to actual outcome data during the study, there would be major objections. Like any sponsor, the NHLBI and NCCAM cannot be considered unbiased observers. These agencies made major financial commitments to the trial and may intentionally or inadvertently influence study conduct if inappropriately unblinded during the study.

Other limitations in the design of TACT further undermine its reliability. In studying a controversial therapy, the primary end point should include the most objective and reliable components, such as death, stroke, and myocardial infarction. In TACT, the study included 2 less reliable end points, coronary revascularization and hospitalization for angina. These “softer” end points represent 318 of 483 events reported as primary end point events. If any unblinding occurred, investigator biases could potentially influence the decision to hospitalize or revascularize individual patients. During the study, enrollment proceeded at such a slow rate that the trial design was altered midway through the study, which is never desirable. In addition, conducting 11 interim analyses is highly unusual and increases the risk that the study was stopped exactly at the point when marginal “significance” was reached.

Given the numerous concerns with this expensive, federally funded clinical trial, including missing data, potential investigator or patient unmasking, use of subjective end points, and intentional unblinding of the sponsor, the results cannot be accepted as reliable and do not demonstrate a benefit of chelation therapy. The findings of TACT should not be used as a justification for increased use of this controversial therapy.

Corresponding Author: Steven E. Nissen, MD, Department of Cardiovascular Medicine, Cleveland Clinic Foundation, 9500 Euclid Ave, Room J2-230, Cleveland, OH 44195 (nissens@ccf.org).

Editorials represent the opinions of the authors and JAMA and not those of the American Medical Association.