Figure 1. Flow of trial selection. DMARD indicates disease-modifying antirheumatic drugs; RA, rheumatoid arthritis.
Figure 2. Network of treatment comparisons across clinical trials of biologic disease-modifying antirheumatic drugs (DMARDs). The 102 comparisons are according to the prior treatment status of eligible patients. A, DMARD-naive trials (18 comparisons). B, Biologic-naive trials (71 comparisons). C, Biologic–second-line trials (13 comparisons). The area of the circles is proportional to the sum of patients planned to be enrolled in corresponding arms. Grey indicates treatments approved for rheumatoid arthritis (RA) by the US Food and Drug Administration (FDA) or the European Medicines Agency (EMEA); black, treatment not approved for RA by the FDA and EMEA (number in parentheses, treatment not approved/treatment approved). The thickness of the lines is proportional to the number of comparisons. Grey lines indicate comparisons with intervention treatment approved for RA by the FDA or the EMEA; black lines, comparisons with intervention treatment not approved for RA by the FDA and EMEA (number, treatment not approved/treatment approved). NA indicates not available.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Estellat C, Ravaud P. Lack of Head-to-head Trials and Fair Control Arms: Randomized Controlled Trials of Biologic Treatment for Rheumatoid Arthritis. Arch Intern Med. 2012;172(3):237–244. doi:10.1001/archinternmed.2011.1209
Author Affiliations: INSERM U738 Institut National de la Santé et de la Recherche Médicale (Drs Estellat and Ravaud), Département d’Epidémiologie et Recherche Clinique, site Bichat, Hôpitaux Universitaire Paris Nord Val de Siene, Assistance Publique–Hôpitaux de Paris, and Université Paris Diderot (Dr Estellat), and Centre d’Epidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique–Hôpitaux de Paris, and Université Paris Descartes (Dr Ravaud), Paris, France.
Background One of the key elements of comparative treatment effectiveness research is head-to-head trials. We herein describe the control arms and the treatment received by patients in recently conducted or ongoing randomized controlled trials of biologic disease-modifying antirheumatic drugs (DMARDs) for rheumatoid arthritis.
Methods We identified all protocols recorded in ClinicalTrials.gov to October 1, 2009. We extracted trial length and funding, prior treatment, disease activity in eligible patients, and the treatment received in both trial arms.
Results Among the 91 trials identified (15 DMARD-naive trials, 63 biologic-naive trials, and 13 biologic–second-line trials) involving 18 554 patients in control arms (3059, 13 095, and 2400 patients, respectively), only 5 compared biologic DMARDs head-to-head (2 of 7 noncommercially funded trials and 3 of 84 commercially funded trials). Two-thirds (66%) of these trials are ongoing. Networks of treatment comparisons reflect a predominant use of placebo as a comparator (81 of 102 comparisons among the 91 trials). In all 15 DMARD-naive trials, all control patients received a new treatment. In 54 of the 63 biologic-naive trials, 9224 of the 13 095 control patients received their previously ineffective treatment, 3848 for more than 6 months, despite high levels of disease activity and contrary to guidelines. In biologic–second-line trials, 851 of the 2400 control patients received treatment comparable to their previously ineffective one.
Conclusions Head-to-head trials of biologic DMARDs are still exceptions. Exposing patients in control arms who had a previous partial response or nonresponse to an inadequate treatment could lead to irreversible deterioration in condition.
Comparative effectiveness research refers to “the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings.”1(p951),2 It implies going beyond the proof of efficacy (doing more good than harm under ideal circumstances) to address the effectiveness (doing more good than harm under the usual circumstances of health care practice) of a treatment but also comparing the different treatments available to determine which one works best.3
Evidence from indirect comparisons across trials is weaker than evidence from direct randomized head-to-head trials.4-6 Results of placebo-only trials do not answer many clinicians' and patients' questions about the comparative benefits and harms of treatment, and therefore head-to-head trials are needed to determine the real comparative risk/benefit profiles of different treatments.7-10
The choice of the comparator (placebo or active) also raises ethical concerns when patients in the control arm of a trial receive a placebo and/or suboptimal treatment and not the current proven treatment. The 1996 and 2000 revisions of the Declaration of Helsinki allowed the use of placebo when no proven diagnostic or therapeutic method exists.11 After controversy, a note of clarification in 2002 stated that placebo-controlled trials may be ethically acceptable when, for compelling and scientifically sound methodological reasons, use of a placebo is necessary to determine the efficacy or safety of a prophylactic, diagnostic, or therapeutic method or, for minor conditions, when the patients who receive placebo will not be subject to any additional risk of serious or irreversible harm. The latest revision, in 2008, requested the 2 conditions together: methodological reasons and no risk of serious or irreversible harm.12
Rheumatoid arthritis (RA) is a potentially severe condition but has effective treatments. Disease-modifying antirheumatic drugs (DMARDs) can reduce or prevent joint damage, thus preserving joint integrity and function.13,14 Patients deprived of effective DMARDs, even for a short time, are exposed to serious or irreversible harm. Since 2002, biologic DMARDs or conventional DMARDs not previously received have been officially recommended by the American College of Rheumatology to prevent or slow RA damage for patients in whom methotrexate sodium is contraindicated or has failed to achieve satisfactory disease control because of either lack of efficacy (in dosages up to 25 mg/wk) or intolerance.15-17
In this study, we aimed to demonstrate that we lack head-to-head trials providing clinically useful comparative effectiveness data on biologics for RA and that, in placebo-controlled trials, many patients with RA could be exposed to an inappropriate control treatment.
We identified all protocols of randomized controlled trials of biologic DMARDs for RA that ended after 2002 and were reported in ClinicalTrials.gov to October 1, 2009. We searched ClinicalTrials.gov with the terms interventional [study types] AND rheumatoid arthritis [disease] AND adult OR senior [age group] AND phase 3 OR phase 4 [phase].
We excluded trials not meeting our eligibility criteria (Figure 1). If a trial had 2 successive treatment periods, we considered only the first period because the primary outcome is assessed before the second period. In trials with multiple arms, we merged the arms with the same treatment but a different dosage in a single arm.
We automatically extracted the study title, identification number, recruitment status, planned patient enrollment, funding, trial phase, first date treatment was received in the registry, and trial start and completion dates from ClinicalTrials.gov. One of us (C.E.) extracted ClinicalTrials.gov data on intervention treatment, control treatments, cointerventions, primary outcome, treatment length (anticipated maximum time of treatment or, if not available, time frame of the primary outcome), recruitment location (country), and prior treatment of eligible patients.
Sulfasalazine, hydroxychloroquine sulfate, minocycline hydrochloride, and methotrexate were considered conventional DMARDs. Anti–tumor necrosis factor (TNF) (certolizumab pegol, etanercept, adalimumab, infliximab, and golimumab), anti-CD4 (zanolimumab), anti-CD20 (rituximab, ocrelizumab, and ofatumumab), fusion protein (abatacept), anti–interleukin 1 (anakinra), anti–interleukin 1β (ACZ 885), anti–interleukin 6 (tocilizumab), P2X7 receptor antagonist (CE-224535), calcineurin inhibitor (cyclosporine), JAK-3 inhibitor (CP-690550), and T-614 were considered biologic DMARDs.15
If no cointervention was mentioned, we considered that no specific RA treatment was given as a cointervention. If a conventional DMARD was used in all arms of a trial, we considered it a cointervention and the trial an add-on trial. For example, in comparing a combination of methotrexate and a biologic DMARD vs methotrexate, placebo was the control arm and methotrexate was the cointervention.
All data available in ClinicalTrials.gov, including inclusion and exclusion criteria, the official title, and the study description, were used to classify the trial according to prior treatment of eligible patients.
DMARD-naive trials included patients who had not received conventional or biologic DMARDs (usually patients with early RA).
Biologic-naive trials included patients who previously received conventional DMARDs (methotrexate and/or other agents) but not biologic DMARDs.
Biologic–second-line trials included patients who previously received biologic DMARDs.
One of us (C.E.) extracted all data relevant to determine the RA activity of eligible patients from the eligibility criteria, official title, and study description. From these data and according to American College of Rheumatology recommendations,15 2 expert rheumatologists independently rated the maximum level of RA activity (low, moderate, or high) of eligible patients for each trial. Discrepancies were resolved by consensus.
Studies with a pharmaceutical company as one of the reported sponsors were considered commercially funded; otherwise, funding was considered noncommercial. Trials “not yet recruiting,” “recruiting” (participants were being recruited and enrolled), or “active, not recruiting” (the study is ongoing but enrollment has been completed) were considered ongoing; other studies were considered completed. If no start date was mentioned, we considered the first date the report was received by ClinicalTrials.gov as the start date.
We performed a descriptive analysis. Categorical data are described with frequencies as percentages and 95% CIs. Continuous data are described with medians and first and third quartiles.
To describe the main characteristics of trials, our unit of analysis was the trial. Then, we considered comparisons of treatments. We represented all treatment comparisons across the studies by a graphical network, with the thickness of lines proportional to the number of comparisons (Figure 2). Multiple-arm trials were split into subtrials to reflect each comparison. For example, a 3-arm trial of 2 biologic DMARDs and a placebo was considered to have 2 control arms (the biologic DMARD and the placebo) but 3 comparisons. Comparisons not involving biologic DMARDs in any arm were not considered. For example, a 3-arm trial with 2 control arms (hydroxychloroquine vs sulfasalazine vs etanercept) was considered to have 3 comparisons (hydroxychloroquine vs etanercept, sulfasalazine vs etanercept, and hydroxychloroquine vs sulfasalazine), with the latter comparison not considered. Comparisons involving the same biologic DMARD in the 2 arms (eg, etanercept vs etanercept and methotrexate) were also excluded. For network representations, we used the R software package with igraph, version 0.5.4-1 (http://igraph.sourceforge.net).
Finally, we focused on the treatment received by patients in the control arms. To compute the numbers of patients in each arm, we hypothesized a ratio of allocation of 1:1 in the trial unless otherwise stated in ClinicalTrials.gov. The treatment received consisted of the comparator and the cointervention, if any.
We initially retrieved 306 trials; 215 were excluded, and we finally identified 91 trials meeting our criteria (Figure 1). The main characteristics of the selected trials are provided in Table 1. Sixty trials (66%) are ongoing. Fifty trials started before 2007 and 41 after.
In total, 53 trials (58%) investigated anti-TNF agents and 41 trials investigated other biologic DMARDs. Three trials investigated anti-TNF agents head-to-head against other biologic DMARDs (2 a fusion protein and 1 a JAK-3 inhibitor).
Five of the 91 trials (5%) were head-to-head trials, including 3 biologic-naive trials (commercially funded) and 2 biologic–second-line trials (noncommercially funded). Two head-to-head trials began in 2005, 1 in 2008, and 2 in 2009. Among the 60 ongoing trials, 4 (7%) are head-to-head trials.
Graphical representation of treatment networks showed a lack of comparison across biologic DMARDs and the predominant place of placebo for all 3 trial types (DMARD naive, biologic naive, and biologic second-line). The networks are clearly star-shaped, focused on placebo (Figure 2).
In the 15 DMARD-naive trials corresponding to 18 comparisons, placebo was the comparator for 13, always with a cointervention patients had never received (a conventional DMARD or a corticosteroid). For the 5 remaining comparisons, a conventional DMARD was the comparator for 4 and an intravenous corticosteroid for 1. No trial involved a direct comparison of biologic DMARDs.
In the 63 biologic-naive trials corresponding to 71 comparisons, placebo was the comparator for 57. Placebo was used as a comparator alone for 9 comparisons, with a previously ineffective cointervention for 44 and with a cointervention never previously received for 4. For the 14 remaining comparisons, the comparator was a conventional DMARD for 11 and a biologic DMARD for 3 (direct comparisons).
For the 13 biologic–second-line trials corresponding to 13 comparisons, the comparator was a placebo in 11 of 13 control arms. Placebo was used as a comparator alone for 3 of these comparisons, with a previously ineffective cointervention for 2 and with a cointervention never previously received for 6. The 2 remaining trials involved direct comparisons of biologic DMARDs.
For DMARD-naive trials, 3059 control patients were randomized to receive the following treatments never previously received: conventional DMARD (2856 patients), methotrexate with intravenous corticosteroid (23 patients), methotrexate with intra-articular corticosteroid (170 patients), or intra-articular/intramuscular corticosteroid (10 patients). Among these 3059 potential patients, the treatment plan lasted at least 6 months for 2481, the maximal disease activity at baseline was high for 2340, and it was not assessable for 720.
For biologic-naive trials, the treatment for 13 095 control patients according to the main characteristics of the trials is listed in Table 2. For 9224 patients (70.44%) from 54 trials, the treatment plan was a placebo alone, a placebo plus their previous treatment, or their previous treatment alone (1385, 7297, and 542 patients, respectively) despite an inadequate response to this previous treatment at entry in the trial. When considering only the ongoing trials, the treatment plan for 6573 control patients was their previously ineffective treatment, with duration of treatment for 6 months or longer for 3848.
For biologic–second-line trials, 2400 patients were randomized to control arms; for 1549, the treatment was new for them, and for 851, the treatment was the same as their previously ineffective one (treatment for ≤3 months for 549; 3-6 months for 250; and >6 months for 52). These 851 patients are in ongoing trials with high-activity RA patients eligible.
For all 3 trial types, the sole treatment plan for 6518 control patients was their previously ineffective treatment for more than 6 months; 4557 of these patients participated in 17 trials with the primary outcome being a structural outcome.
We identified 91 recent or current trials of RA and biologic DMARDs with protocols reported in ClinicalTrials.gov. Among these trials, only 5 compared biologic DMARDs head-to-head, including 3 biologic-naive trials and 2 biologic–second-line trials. The comparator was a placebo for 81 of 102 comparisons (alone for 12, with a previously ineffective cointervention for 46, and with a cointervention never previously received for 23). Despite recommendations to give biologic DMARDs to patients with an inadequate response to conventional DMARDs, 9879 patients were randomized to control arms to receive no treatment or their previously ineffective treatment. In usual-care conditions, patients with an inadequate response to conventional DMARDs most likely receive biologic DMARDs according to all RA guidelines.15,17
Our networks of treatment comparisons show that placebo is the preferred comparator in these trials, which results in a lack of direct comparison of biologic DMARDs. All these trials involving more than 13 000 patients will not provide the correct information for making evidence-based decisions. One may also question the principle of equipoise in these trials. Does genuine uncertainty remain about which arm is the best, for example, in a trial of golimumab vs placebo 10 years after the first anti-TNF agent was approved for RA? Fries and Krishnan18 address the same question by showing that results were favorable to the sponsor in every abstract with industry sponsorship accepted for the 2001 American College of Rheumatology meetings.
Carpenter et al19 took the example of controlled trials for treatment of schizophrenia and proposed the following criteria for judging the ethical acceptability of a placebo control: the likelihood that the intervention tested will have clinically significant advantages over existing treatment, the presence of compelling reasons for placebo use (mainly issues of assay sensitivity), subject selection that minimizes the possibility of serious adverse consequences, and a risk-vs-benefit analysis that favors the advantages of placebo over the risk to subjects. None of these criteria are met in our analysis. In the field of osteoporosis, Levine20 advocated that placebo use should be permitted only with demonstration that, under the conditions of the trial (exclusion of patients with poor prognosis factors and closely monitored studies of short duration), withholding the known effective therapy would unlikely result in a serious adverse consequence. When the therapy could result in a nontrivial adverse consequence, the investigator should clearly explain to the subject that the investigator's principal occupation is to conduct clinical research and not to pursue a therapeutic objective.20
Despite ethical limits, pharmaceutical companies usually favor placebo trials because they are cheaper, require fewer subjects, and are less hazardous. In addition, results are easier to use in marketing compared with findings of active comparator trials, which may usually at best demonstrate noninferiority compared with a competitor.
Regulators have largely encouraged this inertia in the choice of comparator by their guidelines with which pharmaceutical companies must comply. The US Food and Drug Administration and the European Medicines Agency also favor superiority trials, which contain in themselves evidence of assay sensitivity (the ability of a study to distinguish between active and inactive treatment) and give measures of “absolute” efficacy and safety.21-24 There is a gap between recommendations for managing RA in clinical care and the “active comparator” in research approval requirements. Anti-TNF therapy has been approved for RA since 1998 and is advised for patients with prior methotrexate therapy failure in the 2002 American College of Rheumatology recommendations17 but is not mentioned as an established comparator in the 2003 European Medicines Agency recommendations.25 The 2003 European Medicines Agency recommendations approve studies in which patients receive, in addition to methotrexate, the new agent or a placebo, without any time limit, even for patients with prior methotrexate therapy failure. A revision of this recommendation is in progress to limit the duration of placebo therapy to 3 months and to “highly advise” but “not formally recommend” real head-to-head investigation of biologic DMARDs.26
After completion of a phase 3A trial showing efficacy of a new agent against a placebo, a phase 3B trial testing for effectiveness against another molecule of proven efficacy in the same indication could be required. These 2 phases could be merged in a single trial, as suggested by Boers,27,28 with a short first period (3 months) vs placebo and a second longer period vs standard therapy. New drugs are currently disseminated but without any reliable evidence of their effectiveness relative to existing treatments,8,29 and prescribers must rely on biased indirect comparisons to choose the best treatment for their patients.30 Despite the recommendations of a 2004 European Medicines Agency guidance document31 to perform, when possible, trials of active comparators and placebos to help establish the efficacy and risk of new drugs, this recommendation does not seem to be followed. Moreover, when conducted, most head-to-head trials do not intend to prove superiority but rather try to show similar efficacy to point out the advantage of the sponsored drug on other outcomes, such as specific adverse events or for certain subgroups.32
Our study has some limitations. First, ClinicalTrials.gov records are probably not always comprehensive, and we had to extrapolate from available information; for example, we assumed no specific RA cointervention if none was reported. We could have sought published articles of completed trials for complete information, but our results would have lacked consistency across trials because these data would not have been available for ongoing trials. Second, we did not search for trials in registries other than ClinicalTrials.gov, but this is the largest registry of ongoing trials. Of the 1466 trials on RA recorded in the World Health Organization International Clinical Trials Registry, 1308 (89.22%) are recorded via ClinicalTrials.gov.33 Searching other registries also may have led to heterogeneity in data availability across trials. Third, we focused on only a single disease, RA, a severe condition that can lead to major disability, because of clear existing recommendations for RA and consensus on the need to treat to prevent damage. Fourth, the author who extracted all the study data was not blinded to the trial sponsor. Because classifications involved predefined objective criteria, this bias could have had only a minimal effect on the results. Fifth, the patients in ongoing trials were not yet enrolled but were to be enrolled, and we cannot exclude that these trials may not have enrolled all planned patients. Sixth, our search ended with trials registered in ClinicalTrials.gov until October 1, 2009. In an additional search with the same eligibility criteria, we screened all protocols recorded in ClinicalTrials.gov from October 1, 2009, through October 1, 2011, and found 6 new head-to-head trials among 34 new recorded trials, which is a slightly better frequency.
Our belief that patients in control arms of RA trials of biologic DMARDs are not receiving a proven efficacious treatment could be confirmed by a survey among rheumatologists, asking them which treatment they would prescribe in usual care for similar patients.
Overall, this study shows that, in many recent or ongoing trials assessing biologic DMARDs, RA patients in the control group are being subjected to a known inadequate treatment, although other proven treatments exist and are recommended. Such studies are of questionable clinical interest because they avoid direct comparison between effective treatments and thus do not help practitioners choose the best treatment among all available for patients.
Correspondence: Candice Estellat, MD, Département d’Epidémiologie et Recherche Clinique, Hôpital Bichat 46 rue Henri Huchard, 75018 Paris, France (firstname.lastname@example.org).
Accepted for Publication: October 31, 2011.
Author Contributions: Dr Estellat had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Estellat and Ravaud. Acquisition of data: Estellat. Analysis and interpretation of data: Estellat and Ravaud. Drafting of the manuscript: Estellat. Critical revision of the manuscript for important intellectual concept: Estellat and Ravaud. Statistical analysis: Estellat. Administrative, technical, or material support: Estellat and Ravaud. Study supervision: Ravaud.
Financial Disclosure: None reported.
Additional Contributions: Adeline Abbe, MSc, and Cécile Pino, MSc, helped in updating the database and creating the network figures. Raphaèle Seror, MD, PhD, and Stéphanie Emilie, MD, helped in rating the disease activity of eligible patients.
Create a personal account or sign in to: