Customize your JAMA Network experience by selecting one or more topics from the list below.
Individuals who enroll in clinical trials do so with the belief that their participation will help to advance medical science. However, many trials are designed, conducted, and reported in ways that stymie this objective, a problem that can be called “uninformativeness.” From the perspective of researchers, this is a form of research inefficiency.1 But from the perspective of participants, preventable uninformativeness is a serious breach of trust and a violation of research ethics.
An uninformative trial is one that provides results that are not of meaningful use for a patient, clinician, researcher, or policy maker. The following are necessary conditions for a trial to be informative (eTable in the Supplement): (1) the study hypothesis must address an important and unresolved scientific, medical, or policy question; (2) the study must be designed to provide meaningful evidence related to this question; (3) the study must be demonstrably feasible (eg, it must have a realistic plan for recruiting sufficient participants); (4) the study must be conducted and analyzed in a scientifically valid manner; and (5) the study must report methods and results accurately, completely, and promptly. Trials that do not meet all of these conditions are very likely to be uninformative.
Scholars of clinical research have identified many of these problems, labeled them as ethically questionable, and called for change.2,3 Nevertheless, recent studies have shown a continued high prevalence of trials that fail 1 or more of the necessary conditions, despite current funding, review, and reporting processes (eTable in the Supplement). For example, a 2013 systematic review of otitis media had to exclude 24 of 96 eligible trials because of an excess “risk of bias.”4 If a randomized trial is judged to be so biased that it cannot be included in a systematic review, then participants in that trial effectively made no contribution to science. Similarly, a review of the largest unreported trials registered in ClinicalTrials.gov identified 67 trials that had no results reported in the public domain a median of 9 years after completion, effectively minimizing the contributions of more than 87 000 participants.5 Subtler examples are trials that unnecessarily addressed settled questions, which added no new knowledge, or trials that use very short-term outcome measures to assess treatments for long-term illness, rendering the trial findings misleading or irrelevant for clinical decision-making.
Uninformative trials present a challenge to ethics, science, and medical practice. First, such trials do not fulfill the well-established principle of social value that provides justification for asking people to participate in clinical research.6 Second, the spirit of informed consent is not met when patients enroll in trials expecting to contribute to medical progress when it is manifestly unlikely. The coexistence of uninformative trials alongside informative ones, even in the same medical center, makes it very difficult for prospective participants and their advisors to make fully informed decisions about trial participation. Third, uninformative trials divert participants, researchers, and other resources from other endeavors, including more informative trials. Fourth, uninformative trial reports compete for the limited attention of physicians or policy makers, who might not detect the design flaws and may draw invalid inferences.
Strong incentives to conduct research along with inadequate research training promotes the initiation of uninformative trials. These trials can survive the many layers of review and oversight because no oversight mechanism assesses all 5 elements of informative trials. For example, while ethics committees are instructed to assess risk/benefit, they do not typically assess scientific merit beyond that needed to justify risk. As pointed out by Altman,2 assessment of study value requires more expertise than simple assessment of study validity, and many review bodies lack the time, expertise, motivation, or charge to do that. Funders presumably consider scientific merit, but frequently do not review the full trial protocol and thus cannot identify critical design or operational details that could render the study uninformative. Quality of reporting cannot be directly addressed prior to study initiation, and funders typically do not consider past actions or current reporting processes when considering a new study.
Trials without any external funding may be at particular risk for being uninformative. As of March 18, 2019, there were 9484 open clinical trials registered in ClinicalTrials.gov that were enrolling over 5 million US participants and had no evidence of external funding. Unless academic medical centers (AMCs) and other groups that sponsor research, such as independent research organizations, have their own review infrastructure, there is no assurance that these trials had any independent scientific review. Independent scientific review alone is not sufficient, but is one mechanism for identifying correctable design flaws and determining whether the question being posed is unsettled and answerable by the trial.1 AMCs and funders also have a mixed record for assuring results reporting for trials they fund or sponsor. Reviews in the past 5 years have found that only about half of nonindustry trials have publicly reported results, and the US Food and Drug Administration Amendments Act 2007 (FDAAA) trial tracker website shows high rates of noncompliance with the FDAAA reporting from most US AMCs.7
Various incentives within the current research system point in the direction of conducting more trials, with few counterincentives for uninformative trials. Even though the phenomenon and the ethical and policy implications are patent, solutions will not be simple. Reforms need to be evidence based and must avoid adding burdens to researchers that do not confer associated benefits. Changes will need to be sensitive to the risk of deterring informative research.
Four broad recommendations can be made. First, AMCs and other groups that sponsor research should embrace their responsibilities as research sponsors by ensuring that each trial receives meaningful scientific review by a funder or another body identified by the AMC prior to its initiation. This review should ensure that each new trial is informed by the body of relevant completed and ongoing studies. The review should also include scrutiny of the study design to identify and remediate any serious design flaws. Such a process might focus on those trials that are not likely to get reviewed through other processes and those perceived to be at greatest risk for being uninformative (eg, trials with no external funding, trials by less experienced investigators, or trials with design characteristics that carry high risk of bias). Second, in the spirit of “you can’t improve it if you can’t measure it,” priority should be given to the development of metrics that reflect the extent to which the 5 conditions of informative trials are being met. Third, incentives need to be developed that reward researchers, trial sponsors, or AMCs for conducting informative trials, and that more strongly discourage the conduct of uninformative trials. Fourth, funders similarly have a responsibility to ensure that the trials that they fund are likely to be informative. This would require a more detailed review of the trial protocol, in some cases, and a process for holding the researchers they are funding accountable for ensuring that the conduct, analysis, and reporting are done in scientifically appropriate and timely manner.8
The current clinical research enterprise has insufficient safeguards against uninformative clinical trials that do not fulfill the contract between researchers and research participants. Uninformative trials squander scarce resources, divert participants from better research, and dilute the evidence base of medicine. Rather than relying on intensive efforts to screen uninformative trials late in their life cycle, AMCs, funders, and others involved in the conduct and oversight of human research need to devise incentives and plug oversight gaps so that fewer patients are induced to participate in uninformative research. As Altman noted, “As the system encourages poor research it is the system that should be changed. We need less research, better research, and research done for the right reasons.”2
Corresponding Author: Jonathan Kimmelman, PhD, Biomedical Ethics Unit, Department of Social Studies of Medicine, McGill University, 3647 Peel St, Montreal, QC H3A 1X1, Canada (email@example.com).
Published Online: July 25, 2019. doi:10.1001/jama.2019.9892
Conflict of Interest Disclosures: Dr Zarin reported consulting for and being the former director of ClinicalTrials.gov. Dr Kimmelman reported receiving personal fees from serving on a data and safety monitoring committee for Ultragenyx Inc. No other disclosures were reported.
eTable. Conditions for Trial Informativeness
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Zarin DA, Goodman SN, Kimmelman J. Harms From Uninformative Clinical Trials. JAMA. Published online July 25, 2019. doi:10.1001/jama.2019.9892
Browse and subscribe to JAMA Network podcasts!
Create a personal account or sign in to: