Optimizing the Trade-off Between Learning and Doing in a Pandemic

The world is united regarding the goal of ending the coronavirus disease 2019 (COVID-19) pandemic but not the strategy to achieve that goal. One stark example is the debate over whether to prescribe available therapies, such as quinine-based antimalarial drugs (eg, chloroquine or hydroxychloroquine), or test these drugs in randomized clinical trials (RCTs). At the heart of the problem is one of the oldest dilemmas in human organizations: the “exploitation-exploration” trade-off.¹ Exploitation refers to acting on current knowledge, habits, or beliefs despite uncertainty. This is the “just do it” option: give various therapies (eg, chloroquine) to affected patients based on current knowledge or a hunch. Exploration refers to actions taken to generate new knowledge and reduce uncertainty, eg, testing therapies in an RCT. This is the “must learn” option.

Currently, these approaches are framed as a choice: do something (treat the patient) or learn something (test the drug). This dilemma is now playing out across the world, with many clinicians recommending treatments (eg, antiviral agents or immunomodulating drugs), even while researchers and regulators emphasize that evidence is limited and the need for RCTs is paramount. The problem is that exploitation/exploration trade-offs are almost always best solved by a strategy that blends both: simultaneously learning while doing. The joint goal of this integrated effort is to maximize short-term outcomes (eg, the best possible recovery of patients who must be treated now) and long-term outcomes (eg, the fastest path to discovery and dissemination of new treatments). This balance is elusive, and potentially impossible without an integrated approach—a single system that “learns while doing,” with alacrity.

However, medicine is organized to do each task separately: clinical practice (doing) and clinical research (learning) are addressed by separate institutions, procedures, and funding. Hospitals and physician groups practice medicine while academic institutions, entities like the National Institutes of Health, and pharmaceutical companies focus on scientific discovery. There are many reasons for this division of labor, not the least of which was the Belmont Report, which codified that research should be kept separate from the practice of medicine, or patients’ interests will be usurped.² There are, however, huge costs to this division, including delays in knowledge acquisition and dissemination. In normal times, these costs are somewhat suppressed or ignored, but in a crisis such as the COVID-19 pandemic, they come into sharp focus. Thus, as pressure mounts to “just do something,” the challenge to learn simultaneously and efficiently becomes increasingly difficult, placing both individuals and society in peril.

The chief tool in the learning toolkit is the RCT, primarily because randomization is such a powerful mechanism for inferring causal effects. It is not perfect, and there are alternatives, but in the absence of a miracle drug that dramatically eradicates the disease, randomization will be crucial to determine what therapies work. There are, however, 3 major challenges.

Randomization is profoundly uncomfortable. Kalil has suggested that a clinician who wishes to administer chloroquine (rather than defer to randomized assignment) may not necessarily serve the patient’s best interest (eg, by administering an intervention that is in fact harmful) and squanders an opportunity to learn.³ This argument is correct. However, it ignores that many clinicians, policy makers, and health care administrators will experience profound discomfort and distress, especially while trying to manage a pandemic. Even if a physician agrees that the evidence is uncertain, if they believe that the chance of benefit outweighs the chance of harm, they will feel compelled to “just do it.” The consequences for the patient, for whom the physician feels responsible, are salient and immediate. In contrast, the benefits of accelerated learning through participation in the trial, as well as the consequences of delayed knowledge generation through failure to participate, feel abstract, remote, difficult to calibrate, and beyond the physician’s responsibility.

The deployment of an RCT is too cumbersome. Because the clinical research apparatus is executed by a process parallel to care delivery, there is no easy way to inculcate randomization at the point that clinical decisions are being made. The trial has to be designed, funded, and approved. All operational aspects have to be custom built and implemented. Even when the trial is live, the activities required to convert from “just order chloroquine” to “randomize to chloroquine or placebo” include many steps that intrude on clinical operations, distract from other critical clinical actions, and require research staff who may be furloughed in an epidemic. Meanwhile, thousands of patients are treated outside any RCT, and time to learn is lost.

Those who would fund or conduct RCTs cannot present a unified plan. Although well-motivated, the entire clinical trial enterprise—investigators, drug companies, and funding agencies—is currently in worldwide chaos. Hundreds of COVID-19 trials have been registered on ClinicalTrials.gov, intending to test a wide array of interventions. Clinicians and hospitals are bombarded with requests to participate. Pharmaceutical trials are moving quickly, but in competition with each other. Funding agencies are articulating varying views on priorities and processes. Everywhere, those who would design and conduct trials are competing for funds and priority review. Already, there has been intense clashing of competing ideas around which approaches are appropriate, which therapies should be tested, and other pressing issues. This threat is global, but most efforts are regional and national, with little international cooperation. In the midst of this chaos, it is difficult to imagine any coordinated integrated relationship between the clinical research and clinical practice enterprises. The effort required is simply immense.

The goal must be a system that has a more integrated approach to learning while doing. Currently, those who are doing are increasingly becoming overwhelmed. The first onus, therefore, should be on the research community to “lean in” toward the clinical practice community. Practically speaking, that can include several steps.

Promote designs that make randomization more comfortable. The desire to do something is real and valid. Against that backdrop, RCTs that assign half of the patients to a control group with no active agent intended to mitigate COVID-19 and offer no near-term reward such as new information on treatment effectiveness are asking a lot of clinicians—but designs that simultaneously randomize to several treatment options lower the proportion of individuals assigned to control care. Trials that adapt to quickly discontinue poorly performing therapies may also be reassuring, as clinicians will know that an assigned “recipe” will be increasingly likely to be superior as knowledge accrues. International, large-scale, high-enrolling RCTs could conceivably adapt as frequently as weekly.

Promote designs that facilitate “1-stop shopping” at the point of care for the evaluation of different therapies. There is not time, either during the review process or at the bedside, to be weighing the pros and cons of multiple competing trials. It would be far more efficient to use designs that leverage a common platform for trial entry, data collection, and testing of multiple therapies.

Consider sacrificing sacred cows of clinical research. In a rapidly changing pandemic, perfection will be the enemy of the good. For example, the use of placebo in a control group can increase trial rigor, but it is not necessarily crucial and will add logistical burden, hampering study design (challenging to have multiple intervention groups with multiple placebos), launch (difficult to produce placebo instantly), and execution (more complex dispensary logistics). A good trial, even if not perfect, is better than no trial.

Cast a big (international) tent. None of the above will happen without leadership and commitment to create an integrated environment with incentives to work together. Funding agencies responsible to taxpayers need the political cover and authority to support international studies; pharmaceutical companies need support and incentives from regulatory authorities to participate in collaborative trials; and academic investigators need a structure that provides academic credit and incentive to collaborate in efforts where they might otherwise perceive anonymity and loss of control.

Health care systems can help implement study logistics within clinical information systems. Health care, especially in the US, largely runs on electronic health record (EHR) systems. These systems can facilitate many aspects of clinical research. However, because there is no common platform or cross-system interoperability, there is limited ability for researchers to integrate smoothly with local system EHRs without the cooperation of hospital information technology staff. If hospitals could nonetheless prioritize integration, several research tasks (eg, screening, enrollment, treatment assignment, data collection) could be facilitated.

Physicians can engage in self-reflection and seek consensus around equipoise. Physicians often hold highly divergent views on whether to give, withhold, or test any particular therapy. Even if there is equipoise on average, if that average is because half the physicians believe a therapy works and the other half believe it will be harmful, then it will be difficult to conduct a trial. Now is a valuable time to reflect on how much uncertainty exists around different therapies and the value of reducing that uncertainty. A hospital that unites around its commitment to generate knowledge as fast as possible regarding a suite of potential therapies will be far more ready to collaborate with the clinical research enterprise.

Physicians can have an active role in discussions with patients. COVID-19 is a new disease. Thus, essentially any therapy prescribed in its treatment is experimental. It is crucial, therefore, that patients participate fully in a process of informed consent. Physicians’ roles can include actively referring patients, answering patients’ questions when appropriate, or joining the research team as local investigators. Each physician has an important responsibility to be informed, think clearly, and communicate wisely.

There is a need to do (treat patients) and learn (test therapies) at the same time—they are intertwined endeavors. However, these approaches have been disaggregated and assigned to 2 disparate enterprises. Each enterprise must make changes to lean in to the other. All of society is profoundly affected by social distancing protocols. If the problem of learning while doing cannot be solved, the period until effective treatments are discovered and implemented will be prolonged, in turn prolonging the period that society must endure these blunt public health measures. This is not a problem for researchers alone: it is incumbent on the public, private companies, public officials, health care centers, clinicians, and the research community to solve. The world faced a similar epidemic in 1919, and millions died. Now facing another, it is useful to reflect that physicians tried to treat the Spanish flu with quinine. A century later, it is shocking to still be asking the same question about the same drugs. An integrated approach of learning while doing is essential.

Corresponding Author: Derek C. Angus, MD, MPH, University of Pittsburgh, 3550 Terrace St, 614 Scaife Hall, Pittsburgh, PA 15261 (angusdc@upmc.edu).

Published Online: March 30, 2020. doi:10.1001/jama.2020.4984

Conflict of Interest Disclosures: None reported.

Additional Contributions: I thank Julie Donohue, PhD, University of Pittsburgh, for her input. Dr Donohue received no compensation for her contributions.