[Skip to Content]
[Skip to Content Landing]
Table 1.  
Characteristics of Included Studies
Characteristics of Included Studies
Table 2.  
Comparison of Study Purpose as Described in the Informed Consent Form vs Protocol or Statistical Analysis Plan (SAP)
Comparison of Study Purpose as Described in the Informed Consent Form vs Protocol or Statistical Analysis Plan (SAP)
Table 3.  
Proportion of Informed Consent Forms That Accurately Conveyed Study Purpose
Proportion of Informed Consent Forms That Accurately Conveyed Study Purpose
Table 4.  
Rationale for Selection of Noninferiority Study Design and Margin
Rationale for Selection of Noninferiority Study Design and Margin
1.
World Medical Association. Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects; 2013. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. Accessed July 1, 2017.
2.
National Commission for the Protection of Human Subjects. Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. US Government Printing Office; 1978. https://videocast.nih.gov/pdf/ohrp_belmont_report.pdf. Accessed July 1, 2017.
3.
Department of Health and Human Services. Protection of human subjects. 2009;45 CFR 46. https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html. Accessed July 1, 2017.
4.
Food and Drug Administration. Non-Inferiority Clinical Trials to Establish Effectiveness: Guidance for Industry; 2016. https://www.fda.gov/downloads/Drugs/.../Guidances/UCM202140.pdf. Accessed July 1, 2017.
5.
Food and Drug Administration.  Draft guidance for industry on non-inferiority clinical trials; availability.  Fed Regist. 2010;75(39):9228-9229.Google Scholar
6.
Kaul  S, Diamond  GA.  Good enough: a primer on the analysis and interpretation of noninferiority trials.  Ann Intern Med. 2006;145(1):62-69.PubMedGoogle ScholarCrossref
7.
Fleming  TR.  Current issues in non-inferiority trials.  Stat Med. 2008;27(3):317-332.PubMedGoogle ScholarCrossref
8.
Friedman  LM, Furberg  CD, DeMets  DL, Reboussin  DM, Granger  CB.  Fundamentals of Clinical Trials. New York, NY: Springer; 2015.Crossref
9.
Government Accountability Office. New Drug Approval: FDA’s Consideration of Evidence from Certain Clinical Trials (GAO-10-798); 2010. http://www.gao.gov/new.items/d10798.pdf. Accessed July 1, 2017.
10.
Committee on Strategies for Responsible Sharing of Clinical Trial Data, Board on Health Sciences Policy, Institute of Medicine.  Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk. Washington, DC: National Academies Press; 2015.
11.
Persaud  N, Doshi  P.  North American regulatory agencies can and should make clinical trial data publicly available.  CMAJ. 2016;188(2):96-97.PubMedGoogle ScholarCrossref
12.
Doshi  P, Hur  P, Jones  M,  et al. Dataset for "Informed Consent to Study Purpose in Randomized Clinical Trials of Antibiotics, 1991 Through 2011". doi:10.5281/zenodo.825517
13.
Higgins  JPT, Green  S, eds.  Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. http://handbook.cochrane.org. Accessed July 1, 2017.
14.
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Choice of Control Group and Related Issues in Clinical Trials (ICH E10); 2000. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E10/Step4/E10_Guideline.pdf. Accessed July 1, 2017.
15.
Food and Drug Administration. Points to Consider: Clinical Development and Labeling of Anti-Infective Drug Products; 1992. https://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm070975.pdf. Accessed July 1, 2017.
16.
Food and Drug Administration. Justification of Non-Inferiority Margin for the Treatment of Complicated Skin and Skin Structure Infections; 2002. https://www.fda.gov/ohrms/dockets/ac/08/briefing/2008-4394b1-01-FDA.pdf. Accessed July 1, 2017.
17.
Department of Health and Human Services. Adequate and Well-Controlled Studies. 2010;21 CFR 314.126. https://www.gpo.gov/fdsys/granule/CFR-2010-title21-vol5/CFR-2010-title21-vol5-sec314-126. Accessed July 1, 2017.
18.
Nuremberg Code. Trials of War Criminals Before the Nuremberg Military Tribunals Under Control Council Law No. 10. vol 2; 1949:181-182. https://www.loc.gov/rr/frd/Military_Law/pdf/NT_war-criminals_Vol-II.pdf. Accessed July 1, 2017.
19.
Djulbegovic  B, Clarke  M.  Ethical issues in the reporting of clinical trials.  JAMA. 2001;286(13):1577-1578.PubMedGoogle ScholarCrossref
20.
Pinto  VF.  Non-inferiority clinical trials: concepts and issues.  J Vasc Bras. 2010;9(3):145-151.Google ScholarCrossref
21.
Powers  JH.  Noninferiority and equivalence trials: deciphering ‘similarity’ of medical interventions.  Stat Med. 2008;27(3):343-352.PubMedGoogle ScholarCrossref
22.
Downing  NS, Aminawung  JA, Shah  ND, Krumholz  HM, Ross  JS.  Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012.  JAMA. 2014;311(4):368-377.PubMedGoogle ScholarCrossref
23.
Li  Y, He  Y, Sheng  Y,  et al.  Systematic evaluation of non-inferiority and equivalence randomized trials of anti-infective drugs.  Expert Rev Anti Infect Ther. 2013;11(12):1377-1389.PubMedGoogle ScholarCrossref
24.
Djulbegovic  B, Clarke  M.  Scientific and ethical issues in equivalence trials.  JAMA. 2001;285(9):1206-1208.PubMedGoogle ScholarCrossref
25.
Zarin  DA, Tse  T, Williams  RJ, Carr  S.  Trial reporting in ClinicalTrials.gov—the final rule.  N Engl J Med. 2016;375(20):1998-2004.PubMedGoogle ScholarCrossref
26.
Casadevall  A, Pirofski  L-A.  Microbiology: ditch the term pathogen.  Nature. 2014;516(7530):165-166.PubMedGoogle ScholarCrossref
27.
Prasad  P, Sun  J, Danner  RL, Natanson  C.  Excess deaths associated with tigecycline after approval based on noninferiority trials.  Clin Infect Dis. 2012;54(12):1699-1709.PubMedGoogle ScholarCrossref
28.
Food and Drug Administration. FDA Drug Safety Communication: FDA warns of increased risk of death with IV antibacterial Tygacil (tigecycline) and approves new Boxed Warning. https://www.fda.gov/Drugs/DrugSafety/ucm369580.htm. Accessed December 1, 2016.
29.
Food and Drug Administration. Safety Alerts for Human Medical Products - Doribax (doripenem): Drug Safety Communication - Risk When Used to Treat Pneumonia on Ventilated Patients. 2014. https://wayback.archive-it.org/7993/20170112164151/http:/www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/ucm388328.htm. Accessed July 1, 2017.
30.
Committee for Medicinal Products for Human Use; Efficacy Working Party; Committee for Release for Consultation.  Committee for Medicinal Products for Human Use (CHMP) guideline on the choice of the non-inferiority margin.  Stat Med. 2006;25(10):1628-1638.PubMedGoogle ScholarCrossref
31.
Clinton  WJ, Gore  A.  Reinventing Regulation of Drugs and Medical Devices. Washington, DC: National Performance Review; 1995.
32.
Schultz  WB; Food and Drug Administration.  Statement regarding the demonstrations of effectiveness of human drug products and devices.  Fed Regist. 1995;60(147):39181-39182.Google Scholar
33.
Knottnerus  BJ, Geerlings  SE, Moll van Charante  EP, ter Riet  G.  Women with symptoms of uncomplicated urinary tract infection are often willing to delay antibiotic treatment: a prospective cohort study.  BMC Fam Pract. 2013;14:71. doi:10.1186/1471-2296-14-71PubMedGoogle ScholarCrossref
Original Investigation
October 2017

Informed Consent to Study Purpose in Randomized Clinical Trials of Antibiotics, 1991 Through 2011

Author Affiliations
  • 1University of Maryland School of Pharmacy, Pharmaceutical Health Services Research Department, Baltimore
  • 2University of Queensland School of Public Health, Brisbane, Queensland, Australia
  • 3Centre for Evidence Based Medicine, University of Oxford, Oxford, England
  • 4University of Maryland School of Medicine, VA Maryland Healthcare System, Baltimore
  • 5Center for Disease Dynamics, Economics and Policy, Washington, DC
  • 6North Carolina State University, College of Veterinary Medicine, Raleigh
  • 7now with Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill
  • 8George Washington University School of Medicine, Washington, DC
JAMA Intern Med. 2017;177(10):1452-1459. doi:10.1001/jamainternmed.2017.3820
Key Points

Question  How often is the study purpose explained to potential participants in randomized clinical trials of antibiotics?

Findings  In this cross-sectional analysis of trial documents submitted to regulators, no informed consent forms consistently conveyed the primary hypothesis of the study.

Meaning  Patients enrolling in antibiotic clinical trials are not accurately informed of study purpose, raising fundamental questions of the ethics of consent in antibiotic drug trials.

Abstract

Importance  Potential research participants may assume that randomized trials comparing new interventions with older interventions always hypothesize greater efficacy for the new intervention, as in superiority trials. However, antibiotic trials frequently use “noninferiority” hypotheses allowing a degree of inferior efficacy deemed “clinically acceptable” compared with an older effective drug, in exchange for nonefficacy benefits (eg, decreased adverse effects). Considering these different benefit-harm trade-offs, proper informed consent necessitates supplying different information on the purposes of superiority and noninferiority trials.

Objective  To determine the degree to which the study purpose is explained to potential participants in randomized clinical trials of antibiotics and the degree to which study protocols justify their selection of noninferiority hypotheses and amount of “clinically acceptable” inferiority.

Design and Setting  Cross-sectional analysis of study protocols, statistical analysis plans (SAPs), and informed consent forms (ICFs) from clinical study reports submitted to the European Medicines Agency. The ICFs were read by both methodologists and patient investigators.

Main Outcomes and Measures  Protocols and SAPs were used as the reference standard to determine prespecified primary hypothesis and record rationale for selection of noninferiority hypotheses and noninferiority margins. This information was cross-referenced against ICFs to determine whether ICFs explained the study purpose.

Results  We obtained trial documents from 78 randomized trials with prespecified efficacy hypotheses (6 superiority, 72 noninferiority) for 17 antibiotics conducted between 1991 and 2011 that enrolled 39 407 patients. Fifty were included in the ICF analysis. All ICFs contained sections describing study purpose; however, none consistently conveyed study hypothesis to both methodologists and patient investigators. Methodologists found that 1 of 50 conveyed a study purpose. Patient investigators found that 11 of 50 conveyed a study purpose, 7 accurately and 4 inaccurately compared with the reference standard. Seventy-one of 72 noninferiority trial protocols or SAPs provided no rationale for selection of noninferiority hypothesis. None provided a clinical rationale for the chosen amount of decreased efficacy.

Conclusions and Relevance  Patients were not accurately informed of study purpose, which raises questions regarding the ethics of informed consent in antibiotic trials. Noninferiority and superiority trials entail different benefit-harm trade-offs that must be conveyed for ethical informed consent.

Introduction

Ethical clinical research is based on a respect for persons. The importance of obtaining informed consent from potential research participants is an unquestioned aspect of research involving humans. The Declaration of Helsinki,1 Belmont Report,2 and Common Rule3 all agree that a key element necessary to convey to participants is the “purposes”2,3 (or “aims”1) of the research.

Randomized clinical trials with active controls, sometimes called “head-to-head” studies, can evaluate “superiority” or “noninferiority” hypotheses of an experimental intervention compared with best standard of care. In superiority trials, the primary hypothesis is to evaluate improved efficacy of new interventions by ruling out any amount of inferior efficacy of experimental interventions compared with existing best standard of care. Noninferiority hypotheses allow an amount of “clinically acceptable” inferior efficacy of new interventions compared with older effective interventions,4-6(pp89-121)7 in exchange for potential nonefficacy benefits of new interventions, such as being “less toxic, less invasive, less costly, require fewer doses, improve quality of life, or have some other value to patients.”8(p109) Noninferiority is determined by ruling out an amount of inferior efficacy chosen prior to the trial, called the “noninferiority margin.” Between 2002 and 2009, one-quarter of new drug approvals included evidence from noninferiority trials, with the majority being antimicrobial studies.9

To our knowledge, there has been no systematic evaluation of information provided to potential research participants and whether it is sufficient to distinguish the differing study purposes of superiority and noninferiority trials. If potential research participants assume that they are enrolled in superiority trials that are actually noninferiority trials or vice versa, they may incorrectly assess the balance of benefits and harms to which they may be exposed based on the study’s intended purpose.

Using informed consent forms (ICFs) obtained from regulatory filings for antibiotics, we evaluated (1) how often study purpose was conveyed to potential trial participants. We also assessed study protocols and statistical analysis plans (SAPs) of noninferiority trials to determine the rationale provided for choice of (2) noninferiority hypothesis and (3) noninferiority margin.9

Methods

Institutional review board approval was not sought for this study as this was non–human subjects research based on publicly available data. There was no need for anonymization because we did not evaluate patient-level data.

Data Sources

In 2013 through 2014, we made a freedom of information request to the European Medicines Agency (EMA) for ICFs, study synopses, protocols, and SAPs (including any amendments) for all antibiotic trials in the EMA’s holdings. After an initial denial, our appeal was successful. The EMA reported that of 17 authorized antibiotics, it held study documents for 13 (retapamulin, inhaled aztreonam, inhaled colistin, daptomycin, fidaxomicin, doripenem, ertapenem, telithromycin [Ketek; Sanofi-Aventis], inhaled tobramycin, trovafloxacin [Trovan; Pfizer], tigecycline, telavancin, ceftaroline). The EMA did not possess data for the other 4 (telithromycin [Levviax; Aventis Pharma S.p.A.], trovafloxacin [Turvel; Laboratorios Almirall], alatrofloxacin [Turvel IV; Laboratorios Almirall], and alatrofloxacin mesylate [Trovan IV; Pfizer]), which are the same active molecule as drugs in the initial 13, and regulators did not require additional studies. In 2015, we requested 4 additional antibiotics approved subsequent to our initial request (ceftazidime-avibactam, dalbavancin, oritavancin, tedizolid). We did not initiate a similar request with the US Food and Drug Administration (FDA) because the FDA currently does not release original sponsor-submitted clinical trial documents, which it treats as exempt from disclosure under the Freedom of Information Act.10,11 All of the underlying data for this study, along with our extraction sheets, is freely available in an online repository.12

Selection Criteria

We included all antibiotic randomized clinical trials that had a prespecified efficacy hypothesis. We had no exclusion criteria at the study level.

Study documents for some trials did not include ICFs. For other studies, we received multiple ICF versions. For the latter, we selected the most recent ICF dated prior to the start of patient enrollment. If identically dated ICFs existed for differently aged populations, we chose the form used with the oldest population. When multiple ICFs met these criteria, for example, multiple ICFs for different study sites, we chose the first form that appeared in the documents.

Study Outcomes

Our primary outcome measure was the proportion of ICFs that accurately conveyed sufficient detail to explain the primary hypothesis (superiority or noninferiority) using the protocols or SAPs as the reference standard. For noninferiority studies, we recorded any reasons given in trial documents for the selection of noninferiority hypotheses and noninferiority margins. We also recorded the noninferiority margin.

Informed Consent Form Review

Five raters (2 methodologists with expertise in clinical trial design [T.J., D.J.M.] and 3 patient investigators [P.A.S., A.W., K.A.]) judged whether ICFs explained the study purpose to potential participants, and if so, assigned the study purpose as superiority or noninferiority. Raters were provided with ICFs only and no other study documents. We aimed to increase raters’ comprehension by highlighting the following words that could be relevant to judging study purpose and statistical hypothesis wherever they appeared in consent forms: purpose, aim, goal, objective, hypothesis, evaluate, demonstrate, determine, compare, intend, intention, plan, similar, superior, inferior, better, worse, effective, less, more, safe, tolerate, tolerable, justification. We did this because relevant information is not necessarily always located in the section of ICFs where study purpose is described (eg, sometimes labeled “Purpose” or “Why is this research study being done?”). Highlighting was automated with Adobe Acrobat Professional XI.

Patient investigators (P.A.S., A.W., K.A.) were selected by convenience and given a $500 honorarium. As we expected them to be unfamiliar with the terms “noninferiority” and “superiority,” we designed a questionnaire that enabled us to gauge their determination of study hypothesis without using explicit terms. No training was provided other than requesting their participation in a pilot of 2 trials selected on the basis of being the only 2 trials that named the study design in the study title printed on the ICF.

Patient investigators were provided with ICFs and asked to answer the following question: “Is the primary purpose of this trial to see whether the NEW DRUG is more effective than the CONTROL DRUG or is the primary purpose to see whether the NEW DRUG is not substantially worse than the CONTROL DRUG? Or can you not tell?” For the purpose of comparing patient investigator responses to methodologist responses, an answer of “more effective” was taken to indicate a superiority study and “not substantially worse” a noninferiority study.

Disagreements among rater pairs were reported descriptively and with κ statistics. Subsequently, to compare methodologist and patient investigator judgments, we obtained a consensus methodologist and consensus patient investigator judgment for use in final data analysis by following Cochrane methods13 for reaching consensus.

Methodologists were asked whether the ICF described study purpose sufficiently to distinguish between superiority and noninferiority design, and if so, to judge the design.

One author (P.D.) also searched ICFs for keywords describing study purpose (keywords: purpose, aim, goal, objective, justification) and whether this was phrased to evaluate superiority (superior, better, more effective), similarity (similar), or inferiority (inferior, worse, less effective). Searching was automated with Adobe Acrobat Professional XI and hand reviewed.

Data Analysis

We classified all included trials as superiority or noninferiority hypotheses based on the primary efficacy hypothesis as stated in the synopsis, protocol, or SAP. We classified trials called “equivalence” in reports as “noninferiority” trials, in line with international guidance defining noninferiority trials.14 Many active clinical trials called “equivalence” are noninferiority studies, as they specify only an amount of acceptable inferiority.14 None described an upper bound on greater efficacy as in equivalence trials. Studies were also categorized as studying a “serious” or “nonserious” disease based on whether the FDA has granted Qualified Infectious Disease Product (QIDP) status under the Generating Antibiotic Incentives Now (GAIN) Act in relation to the indication under investigation. The QIDP designation requires serious or life-threatening disease and a qualified pathogen.

Final judgments by methodologist and patient raters were compared with those based on the reference standard of study reports (study synopsis, protocol, or SAP). We present our analyses stratified by statistical hypothesis of the trial’s primary objective (superiority or noninferiority).

We tested 2 hypotheses using the Fisher exact test: first, that the proportion of ICFs that convey their study purpose clearly was equal for superiority and noninferiority trials, stratified by methodologists and patient investigators; second, that ICFs convey their study purpose equally clearly to both methodologists and patient investigators (as assessed by the proportion of correct ratings). We used P < .05 to indicate statistical significance and 2-sided testing.

Results

The EMA provided study documents for 107 trials of 17 antibiotics, of which 78 trials (for 17 drugs) met inclusion criteria (6 superiority trials, 72 noninferiority trials). Of these, 26 trials were excluded from the ICF analysis, primarily because the EMA did not possess ICFs. We also excluded from the ICF analysis 2 trials used in our pilot to develop our methods, leaving a total of 50 trials included in ICF analysis. Details of trials included and excluded, with reasons, are provided in the eFigure in the Supplement and our dataset.12

A total of 39 407 patients were enrolled in 78 trials. Patients enrolled in the 52 trials for which we had ICFs totaled 28 548. Approximately one-half of the included trials (38 of 78) started between 1991 and 2000, and half (36 of 78) commenced between 2001 and 2011. Most superiority trials (5 of 6) studied serious diseases. Three-quarters of noninferiority trials (53 of 72) studied serious diseases (Table 1).

Automated keyword searching showed that all ICFs included a study purpose section. Five ICFs included words such as “better” or “superior,” 4 of which were noninferiority trials (eTable in the Supplement). For example, 1 ICF informed patients that, “The study doctor will give PAR-101 to some people in this study to see if it is safe and tolerable, and can help them with their CDAD [Clostridium difficile–associated diarrhea]. Another purpose of this study is to find out if taking PAR-101 is better than taking vancomycin.” However, the SAP defined the primary hypothesis as noninferiority. See the eBox in the Supplement for other examples.

In the ICF analysis, prior to consensus the 2 methodologist raters’ independent judgments on study hypothesis were concordant (mostly “cannot tell”) in 42 of 50 (84% agreement; unweighted κ < 0) trials. After consensus, they judged 49 of 50 ICFs as containing insufficient information to judge study hypothesis. For the 3 patient investigator raters, agreement was 12 of 50 (24%; unweighted κ = 0.07), 16 of 50 (32%; unweighted κ = 0.23), and 31 of 50 (62%; unweighted κ = 0.18). After consensus, patient investigator raters judged 38 of 50 (76%) ICFs as containing insufficient information to judge study hypothesis.

None of the ICFs consistently conveyed study purpose simultaneously to both methodologist and patient investigator raters. For the methodologist raters, 1 of 50 (2%) ICFs conveyed study purpose. This ICF was rated as a superiority trial whereas the protocol or SAP defined the primary hypothesis as noninferiority. This was the aforementioned trial in which the ICF included wording stating, “the purpose of this study is to find out if taking PAR-101 is better than taking vancomycin.” For patient investigators, 7 of 50 (14%) ICFs conveyed study purpose accurately (5 noninferiority and 2 superiority trials) and 4 (8%) conveyed the incorrect hypothesis (all 4 were noninferiority trials judged as superiority trials) compared with the protocols or SAPs. The remaining 38 (76%) trials’ ICFs did not convey a study purpose (Table 2).

Because none of the ICFs rated by methodologists consistently conveyed study purpose accurately, we could not reject the hypothesis that ICFs from superiority and noninferiority trials convey study purpose equally clearly. For patient advocate raters, ICFs from superiority trials conveyed study purpose accurately more commonly than noninferiority trials (50% vs 11%; P = .09). Finally, we rejected the null hypothesis that ICFs convey their study purpose equally clearly to both methodologists and patient investigators (0% vs 14%; P = .01) (Table 3).

In protocols or SAPs, we could not locate a rationale for selection of noninferiority hypotheses in 71 of 72 (99%) noninferiority trials. The single trial that provided a justification explained that a placebo-controlled trial would be unethical given the type of disease (severe acute bacterial skin and skin structure infections with large lesions and systemic signs of infection) and an active-controlled superiority trial would be unethical “because highly effective antibiotic treatment exists” (Table 4).

We located a written explanation of the choice of noninferiority margin in 51 of 72 (71%) trials, whereas 21 (29%) trials did not include a justification for the noninferiority margin choice. Thirty-seven (51%) trial documents indicated that the noninferiority margin was chosen in accordance with a guidance from the FDA, commonly citing 2 specific FDA documents15,16 (Table 4). Six (8%) trials referenced the effect of the control drug from previous placebo-controlled trials, as delineated in FDA regulation and international guidance.14,17 No trials provided a clinical rationale explaining why the chosen amount of inferiority should be considered “clinically acceptable.”

Among noninferiority studies, the median noninferiority margin was −10% (interquartile range, −15% to −10%) (Table 1).

Discussion

Our analysis of 50 antibiotic trials submitted to regulatory agencies for marketing approval over 2 decades found that potential participants were not accurately informed of study purpose in ICFs. Neither patient investigators nor experienced methodologists could determine the purpose for the majority of trials. These results raise questions about the ethics of antibiotic development. Clarity on study purpose is central to the ethical principles of respect for persons, as well as minimizing risk; it is also crucial to understanding alternative treatments noted in foundational documents regarding experimentation involving humans, and a requirement for proper informed consent in the Common Rule.1-3,18

Others have noted19,20 that informed consent should differ between superiority and noninferiority trials, given differences in trade-offs between efficacy and harm. In superiority trials, investigators hypothesize increased efficacy compared with best standard of care. In noninferiority trials, investigators hypothesize that some decreased effectiveness compared with effective best standard of care would be acceptable to potential research participants in exchange for potential nonefficacy benefits such as decreased adverse effects or improved convenience.

Whereas all ICFs included a study purpose section (usually labeled “Purpose”), none consistently conveyed the primary hypothesis of noninferiority or superiority. One reason may be historical: when randomized trials became the standard, few effective therapies existed and placebo controls were common. The important ethical documents1-3 were drafted when superiority hypotheses were the norm. The term “noninferiority” did not exist until the 1980s.21

Today, nearly one-third of premarketing trials have active controls.22 For antibiotics, the proportion is higher and noninferiority trials are common.9,23 Living ethical guidance documents need updating to reflect this changing landscape. Investigators need guidance on how to ensure that ICFs convey study purpose. We are unaware of ICF model language for noninferiority trials, although some have offered sample language for “equivalence” trials.24 However, investigators should avoid wording that implies that a noninferiority trial evaluates equality of interventions because noninferiority trials cannot show that 2 interventions are not different by any amount (as in superiority trials). To aid efforts to develop model language, investigators, sponsors, and institutional review boards could prospectively publish ICFs and study protocols in trial registries. The recent US Department of Health and Human Services Final Rule (effective January 2017) indicates that ClinicalTrials.gov will accommodate optional posting of ICFs and requires posting of protocols and SAPs.25 In the Box, we offer sample language for ICFs for noninferiority trials.

Box Section Ref ID
Box.

Sample Language to Describe Study Purpose in Informed Consent Form of Noninferiority Trials

  • Generic language:

You are being asked to enroll in a clinical trial comparing a new experimental (drug, vaccine, diagnostic, etc) with a standard treatment (drug, vaccine, diagnostic, etc). The standard treatment is a (regulator) approved (drug, etc) that is already known to be effective in (treating, preventing, diagnosing) your disease. This trial does not seek to demonstrate that (new drug, etc) is more effective than the (standard drug). Instead, investigators will test to determine whether the new (drug) is no more than X% (eg, 10%) less effective than the standard treatment. A finding of X% less effective would mean that 1 in Y (eg, 10) patients had a worse outcome [explain what the outcome is] with the new experimental (drug etc) compared with the standard treatment. The new (drug) might be better for patients because it might have (demonstrated or hypothesized nonefficacy benefit described here—fewer adverse effects, more convenient, etc—ideally with the specific hypothesis related to less adverse effects (eg, “might have 10% less nausea” etc) [and whether that nonefficacy benefit is known or will be tested in the trial]).

  • Example language in uncomplicated urinary tract infection for a noninferiority hypothesis comparing a new antibiotic with an older effective antibiotic with a −10% noninferiority margin in which the new drug is hypothesized to have fewer adverse effects:

You are being asked to enroll in a clinical trial comparing a new experimental antibiotic, Supermycin, with a standard drug treatment called Standardcillin. The standard treatment is FDA approved and already known to be effective in treating urinary tract infections in young healthy women. This trial does not seek to demonstrate that Supermycin is more effective than Standardcillin. Instead, investigators will test to determine whether Supermycin is no more than 10% less effective than Standardcillin in treating urinary tract infections. A finding of 10% less effective would mean that for 1 in 10 women, it will take longer for the symptoms of their disease to go away compared with Standardcillin. Supermycin might be better for patients because it might have fewer adverse effects, such as an up to 10% decrease in nausea, diarrhea, and rash, compared with Standardcillin. This possibility of fewer adverse effects is not yet proven and will also be tested in this trial.

We found that 99% of protocols or SAPs lacked a rationale for the selection of noninferiority hypotheses. One reason may be that investigators believe that “noninferior” means “equal to” or “as effective as,” thus assuming that they are not exposing participants to increased risk of harm. Ruling out no loss of effectiveness at all necessitates superiority trials.1

In the single trial that offered a rationale, investigators stated that a placebo-controlled superiority trial would be unethical in severe skin infections and that an active-controlled superiority trial would be unethical “because highly effective antibiotic treatment exists.” However, the reason placebo-controlled trials are considered unethical is that they may expose patients who have effective options to increased harm from a less effective intervention (placebo). The primary hypothesis in noninferiority trials also allows for lesser effectiveness with a new drug in patients who have effective options (the control drug). Therefore, noninferiority trials raise the same ethical issues as placebo-controlled trials.1 Challenges in demonstrating added benefits with new antibiotics do not make superiority trials “unethical” but highlight either a lack of unmet medical need for improved efficacy in patients for whom effective therapy exists or the need to focus on nonantibiotic interventions that might improve outcomes.26

Clarity of study purpose is particularly important in serious and/or life-threatening diseases. More than 90% of analyzed trials used noninferiority hypotheses, with deaths in 64% of them, and recent antibiotics approved via noninferiority have increased morbidity and mortality.27-29 In life-threatening disease, potential research participants may accept increased adverse effects for potentially prolonged and/or improved life investigated in superiority trials. However, potential participants may not find noninferiority trials acceptable if they are not willing to give up any amount of efficacy, even in exchange for fewer adverse effects or improved convenience, particularly in acute life-threatening infections where decreased efficacy implies increased morbidity or mortality compared with the active control. The EMA has noted, “Where the treatment under consideration is used for the prevention of death or irreversible morbidity...it can be very difficult to justify a noninferiority margin of any size.”30 The FDA has stated that “it is essential for public health protection that a new therapy be as effective as alternatives that are already approved for marketing when...the disease to be treated is life-threatening or... the disease to be treated is a contagious illness.”31,32 This indicates that noninferiority is not an appropriate study design for serious diseases because there is no “clinically acceptable” inferior efficacy in such cases.5

Noninferiority trials may be acceptable in non–life-threatening diseases in which research participants might accept some amount of lesser effectiveness not resulting in death or irreversible harm in exchange for superiority on fewer adverse effects, with the amounts determined by valid patient surveys.33

We found that 71% of noninferiority studies provided a rationale for selection of noninferiority margins. Many provided this rationale in the discussion of sample size derivation. Half mentioned FDA guidance,15,16 but none provided clinical reasoning or involvement from patients in selecting the amount of decreased effectiveness considered “acceptable.” Noninferiority margins that allow for absolute decreases in effectiveness of −15% to −10% mean that as many as 1 in 6 to 10 patients may receive less effective new interventions compared with older effective therapy. This suggests that major trial decisions are driven by convention, sample size considerations, or following regulatory suggestions, leaving unclear what, if any, consideration is given to patient perspectives. The patient perspective is important because noninferiority margins are a central factor to determining whether a trial ultimately meets its primary objective, but also because the margin reflects subjective judgments regarding the amount of decreased efficacy that patients would find acceptable, as well as the type and amount of nonefficacy benefits that constitute an acceptable trade-off.

Limitations

Our study has limitations. Because we only assessed ICFs, we do not know what other information was provided to participants, how well they understood the ICF, or whether they read it. We also cannot generalize our findings beyond antibiotic trials. Finally, we included study documents irrespective of their date stamp. The degree that study hypotheses might have changed over time and were not reflected consistently across documents may have caused misclassification.

Conclusions

Patients enrolling in clinical trials of antibiotics are not accurately informed of study purpose. Because noninferiority trials do not intend to demonstrate superior efficacy of new interventions and entail trade-offs of lesser efficacy for other benefits, this study raises fundamental questions of the ethics of consent in antibiotic trials.

Back to top
Article Information

Corresponding Author: Peter Doshi, PhD, Pharmaceutical Health Services Research Department, University of Maryland, School of Pharmacy, 220 Arch St, 12th Floor, Room 01-228, Baltimore, MD 21201 (pdoshi@rx.umaryland.edu).

Accepted for Publication: March 15, 2017.

Published Online: August 21, 2017. doi:10.1001/jamainternmed.2017.3820

Author Contributions: Dr Doshi had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Doshi, Powers.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Doshi, Powers.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hur, Jones.

Administrative, technical, or material support: Doshi.

Study supervision: Doshi.

Conflict of Interest Disclosures: Drs Doshi, Jones, and Jefferson were co-recipients of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of 2 Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children—https://www.journalslibrary.nihr.ac.uk/programmes/hta/108001#/. Drs Doshi, Jones, and Jefferson are also in receipt of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. Dr Doshi received €1500 from the European Respiratory Society in support of his travel to the society’s September 2012 annual congress in Vienna, where he gave an invited talk on oseltamivir. Dr Doshi has received funding support ($11 000) from the American Association of Colleges of Pharmacy for a study to analyze written medical information regarding the possible harms of statins. Dr Doshi is also an associate editor of the BMJ and an unpaid member of the IMEDS steering committee at the Reagan-Udall Foundation for the FDA, which focuses on drug safety research. Dr Hur is a postdoctoral fellow at the University of Maryland; this fellowship is funded by Novartis. Mr Albarmawi reports working as a research assistant on projects for Bayer Healthcare and Amgen, Inc (recipient: University of Maryland, Baltimore) outside the submitted work. Dr Jefferson receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. Dr Jefferson is occasionally interviewed by market research companies for anonymous interviews about phase 1 or 2 pharmaceutical products. In 2011 through 2013, Dr Jefferson acted as an expert witness in a litigation case related to oseltamivir phosphate (Tamiflu [Roche]) and in a labor case on influenza vaccines in health care workers in Canada. In 1997 to 1999 Dr Jefferson acted as a consultant for Roche, in 2001 to 2002 for GlaxoSmithKline, and in 2003 for Sanofi-Synthelabo for pleconaril (an antirhinoviral, which did not get approval from the US FDA). Dr Jefferson was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific advisor to a legal team acting on the drug oseltamivir (Tamiflu [Roche]). In 2014 to 2015, Dr Jefferson was a member of 2 advisory boards for Boerhinger. Dr Jefferson has a potential financial conflict of interest in the investigation of the drug oseltamivir. Dr Jefferson was a member of an Independent Data Monitoring Committee for a Sanofi Pasteur clinical trial. Dr Jefferson is a co-signatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of human papillomavirus vaccines and consequent complaints to the European Ombudsman. Dr Morgan reports grants from the Centers for Disease Control and Prevention, the National Institutes of Health, the Agency for Healthcare Research and Quality, and the Veterans Affairs Health Services Research and Development Service, other from the Infectious Diseases Society of America, American Society for Microbiology, Lown, and the Society for Healthcare Epidemiology of America for expenses to organize or present at national meetings, and received honoraria from Springer Inc for journal and book editing, outside the submitted work. Ms Spears reports personal fees from Pfizer and other from Hoffmann La Roche/Genentech, outside the submitted work. Dr Powers reports personal fees from Abbvie, Cardeas, Cempra, Contrafect, Gilead, Johnson & Johnson, MedImmune, Novartis, Otsuka, Pfizer, Roche, and Trius outside the submitted work. No other disclosures are reported.

Funding/Support: This study was funded out of nonspecific faculty research funds provided by the University of Maryland, Baltimore, to Dr Doshi.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank our 2 patient investigator raters, Alan Wu, BA, and Katura Alwyn, BA (who received compensation from Dr Doshi’s faculty research funds). We also thank Deborah Zarin, MD, for a helpful early discussion on our study design. Dr Zarin received no compensation. We are also grateful for data provided by the European Medicines Agency.

References
1.
World Medical Association. Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects; 2013. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. Accessed July 1, 2017.
2.
National Commission for the Protection of Human Subjects. Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. US Government Printing Office; 1978. https://videocast.nih.gov/pdf/ohrp_belmont_report.pdf. Accessed July 1, 2017.
3.
Department of Health and Human Services. Protection of human subjects. 2009;45 CFR 46. https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html. Accessed July 1, 2017.
4.
Food and Drug Administration. Non-Inferiority Clinical Trials to Establish Effectiveness: Guidance for Industry; 2016. https://www.fda.gov/downloads/Drugs/.../Guidances/UCM202140.pdf. Accessed July 1, 2017.
5.
Food and Drug Administration.  Draft guidance for industry on non-inferiority clinical trials; availability.  Fed Regist. 2010;75(39):9228-9229.Google Scholar
6.
Kaul  S, Diamond  GA.  Good enough: a primer on the analysis and interpretation of noninferiority trials.  Ann Intern Med. 2006;145(1):62-69.PubMedGoogle ScholarCrossref
7.
Fleming  TR.  Current issues in non-inferiority trials.  Stat Med. 2008;27(3):317-332.PubMedGoogle ScholarCrossref
8.
Friedman  LM, Furberg  CD, DeMets  DL, Reboussin  DM, Granger  CB.  Fundamentals of Clinical Trials. New York, NY: Springer; 2015.Crossref
9.
Government Accountability Office. New Drug Approval: FDA’s Consideration of Evidence from Certain Clinical Trials (GAO-10-798); 2010. http://www.gao.gov/new.items/d10798.pdf. Accessed July 1, 2017.
10.
Committee on Strategies for Responsible Sharing of Clinical Trial Data, Board on Health Sciences Policy, Institute of Medicine.  Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk. Washington, DC: National Academies Press; 2015.
11.
Persaud  N, Doshi  P.  North American regulatory agencies can and should make clinical trial data publicly available.  CMAJ. 2016;188(2):96-97.PubMedGoogle ScholarCrossref
12.
Doshi  P, Hur  P, Jones  M,  et al. Dataset for "Informed Consent to Study Purpose in Randomized Clinical Trials of Antibiotics, 1991 Through 2011". doi:10.5281/zenodo.825517
13.
Higgins  JPT, Green  S, eds.  Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. http://handbook.cochrane.org. Accessed July 1, 2017.
14.
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Choice of Control Group and Related Issues in Clinical Trials (ICH E10); 2000. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E10/Step4/E10_Guideline.pdf. Accessed July 1, 2017.
15.
Food and Drug Administration. Points to Consider: Clinical Development and Labeling of Anti-Infective Drug Products; 1992. https://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm070975.pdf. Accessed July 1, 2017.
16.
Food and Drug Administration. Justification of Non-Inferiority Margin for the Treatment of Complicated Skin and Skin Structure Infections; 2002. https://www.fda.gov/ohrms/dockets/ac/08/briefing/2008-4394b1-01-FDA.pdf. Accessed July 1, 2017.
17.
Department of Health and Human Services. Adequate and Well-Controlled Studies. 2010;21 CFR 314.126. https://www.gpo.gov/fdsys/granule/CFR-2010-title21-vol5/CFR-2010-title21-vol5-sec314-126. Accessed July 1, 2017.
18.
Nuremberg Code. Trials of War Criminals Before the Nuremberg Military Tribunals Under Control Council Law No. 10. vol 2; 1949:181-182. https://www.loc.gov/rr/frd/Military_Law/pdf/NT_war-criminals_Vol-II.pdf. Accessed July 1, 2017.
19.
Djulbegovic  B, Clarke  M.  Ethical issues in the reporting of clinical trials.  JAMA. 2001;286(13):1577-1578.PubMedGoogle ScholarCrossref
20.
Pinto  VF.  Non-inferiority clinical trials: concepts and issues.  J Vasc Bras. 2010;9(3):145-151.Google ScholarCrossref
21.
Powers  JH.  Noninferiority and equivalence trials: deciphering ‘similarity’ of medical interventions.  Stat Med. 2008;27(3):343-352.PubMedGoogle ScholarCrossref
22.
Downing  NS, Aminawung  JA, Shah  ND, Krumholz  HM, Ross  JS.  Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012.  JAMA. 2014;311(4):368-377.PubMedGoogle ScholarCrossref
23.
Li  Y, He  Y, Sheng  Y,  et al.  Systematic evaluation of non-inferiority and equivalence randomized trials of anti-infective drugs.  Expert Rev Anti Infect Ther. 2013;11(12):1377-1389.PubMedGoogle ScholarCrossref
24.
Djulbegovic  B, Clarke  M.  Scientific and ethical issues in equivalence trials.  JAMA. 2001;285(9):1206-1208.PubMedGoogle ScholarCrossref
25.
Zarin  DA, Tse  T, Williams  RJ, Carr  S.  Trial reporting in ClinicalTrials.gov—the final rule.  N Engl J Med. 2016;375(20):1998-2004.PubMedGoogle ScholarCrossref
26.
Casadevall  A, Pirofski  L-A.  Microbiology: ditch the term pathogen.  Nature. 2014;516(7530):165-166.PubMedGoogle ScholarCrossref
27.
Prasad  P, Sun  J, Danner  RL, Natanson  C.  Excess deaths associated with tigecycline after approval based on noninferiority trials.  Clin Infect Dis. 2012;54(12):1699-1709.PubMedGoogle ScholarCrossref
28.
Food and Drug Administration. FDA Drug Safety Communication: FDA warns of increased risk of death with IV antibacterial Tygacil (tigecycline) and approves new Boxed Warning. https://www.fda.gov/Drugs/DrugSafety/ucm369580.htm. Accessed December 1, 2016.
29.
Food and Drug Administration. Safety Alerts for Human Medical Products - Doribax (doripenem): Drug Safety Communication - Risk When Used to Treat Pneumonia on Ventilated Patients. 2014. https://wayback.archive-it.org/7993/20170112164151/http:/www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/ucm388328.htm. Accessed July 1, 2017.
30.
Committee for Medicinal Products for Human Use; Efficacy Working Party; Committee for Release for Consultation.  Committee for Medicinal Products for Human Use (CHMP) guideline on the choice of the non-inferiority margin.  Stat Med. 2006;25(10):1628-1638.PubMedGoogle ScholarCrossref
31.
Clinton  WJ, Gore  A.  Reinventing Regulation of Drugs and Medical Devices. Washington, DC: National Performance Review; 1995.
32.
Schultz  WB; Food and Drug Administration.  Statement regarding the demonstrations of effectiveness of human drug products and devices.  Fed Regist. 1995;60(147):39181-39182.Google Scholar
33.
Knottnerus  BJ, Geerlings  SE, Moll van Charante  EP, ter Riet  G.  Women with symptoms of uncomplicated urinary tract infection are often willing to delay antibiotic treatment: a prospective cohort study.  BMC Fam Pract. 2013;14:71. doi:10.1186/1471-2296-14-71PubMedGoogle ScholarCrossref
×