Consequences of Slow Progress Toward Pragmatism in Randomized Clinical Trials: It Is Time to Get Practical

The randomized clinical trial (RCT) is the gold standard for deriving evidence-based practices in medicine. For common chronic diseases with multifactorial causative mechanisms, the modest treatment effects that can be expected from new therapies usually require randomization to draw a causal inference that observed differences are attributable to the treatment. Despite this, most multisociety cardiovascular disease guidelines rely on inconclusive data from small trials, observational studies, and expert consensus, rather than definitive RCT results; in fact, a recent study found that less than 10% of guideline recommendations are supported by the highest-quality evidence (level of evidence A), which is typically derived from an RCT.¹ In essence, clinicians most often depend on consensus opinion rather than firm evidence for decisions that affect life and death and have a huge effect on quality of life.

It is impractical to expect that all day-to-day clinical questions can be answered with traditional RCTs; over time, RCTs have accrued methodologic baggage that is costly and inefficient and does not clearly accomplish the mission of informing decisions. The current designs make recruitment and retention of participants challenging. Furthermore, it is difficult to enroll patients from diverse demographics, and this limits the generalizability of study findings to real-world populations, while much of the process of clinical trials makes limited contribution to answering the questions for which evidence is needed. While most other industries have moved to a much more efficient digital infrastructure, RCTs are mired in traditions that distract from the primary simple concepts that underlie appropriate causal inference: proper randomized treatment assignment, measurement of adherence, and complete ascertainment of critical outcomes.

Clinical trialists have developed a construct known as the Pragmatic Explanatory Continuum Indicator Summary (PRECIS), which quantifies the degree to which a trial incorporates pragmatic principles.² The updated version of this tool (PRECIS-2) scores 9 elements of trial design: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.³ Each element is scored from 1 (very explanatory) to 5 (very pragmatic). In this issue of JAMA Cardiology, Sepehrvand et al⁴ use the PRECIS construct to assess how the degree of clinical trial pragmatism has changed in the past 20 years. The authors critically examined cardiovascular disease–associated RCTs published in the top 3 medical and top 3 cardiovascular journals in 2000, 2005, 2010, and 2015. Over time, the degree of pragmatism increased only modestly in the 616 RCTs included; the increase was most notable in the domains of eligibility, setting, intervention delivery, and primary end points.

This slow pace of transformation of the clinical trials enterprise has consequences for people with chronic diseases. The United States has experienced a decline in life expectancy in 4 consecutive annual assessments, and an uptick is occurring in cardiovascular and stroke mortality after decades of steady reductions.⁵ It is ironic that at a time when researchers know so much about disturbing epidemiological trends and the enormous evidence gap is becoming clear, they are so slow to make the necessary changes to accelerate the production of high-quality evidence to guide efforts to stem the reduction in life expectancy.

Today’s rapid pace of discovery and these concerning trends necessitate more creative ways to conduct RCTs. The good news is that the digital transformation of health care and other parts of society is providing the tools to move quickly to change the field. Although researchers have access to increasingly large observational data sets (big data) that provide context and contribute to comparative effectiveness, randomization is still the best choice for inferring treatment effects between or among diagnostic or therapeutic choices.⁶ Accordingly, it is necessary to focus on using randomization efficiently. This was anticipated in the 21st Century Cures Act, which mandates a transition to more use of “real-world data”⁷ and “real-world evidence,”⁷ carefully defined to include randomization in the context of real life.

In contrast with explanatory and mechanistic trials, pragmatic clinical trials are designed to answer questions that inform decision-makers about health and health care.⁸ Unfortunately, the nomenclature has become muddied by 2 different definitions, one focused on streamlining procedures and the other focused on deriving knowledge in a manner that mimics clinical practice. Both types of pragmatic trials are useful for understanding how different interventions compare across heterogeneous study populations. Frequently encountered clinical questions about therapeutic effects and care delivery approaches lend themselves well to such trials, given the large numbers of patients affected by these diseases and modest treatment effects. For example, the Aspirin Dosing: a Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) clinical trial seeks to answer an important yet simple and unanswered clinical question: what is the appropriate dosage of aspirin for patients with established cardiovascular disease?⁹ The study, conducted by the Patient-Centered Outcomes Research Network (PCORnet), has randomized approximately 15 000 participants across a few dozen US health care systems, using electronic health records as the primary data source and thereby streamlining enrollment and data extraction. Furthermore, simple tools such as smartphone applications have the potential to enable rapid enrollment, consent, and interactive participation of tens of thousands of patients at a fraction of the time and cost it would take to conduct a traditional clinical trial.¹⁰ Such use of technology combined with reform of the human activities can also enable diversification of the participants and improve their engagement by regularly capturing activity levels and patient-reported outcomes.

Sepehrvand et al⁴ found that neutral trials and those further along in the study process (phase III or IV trials) were more likely to be pragmatic. This finding deserves careful thought. The goal of PRECIS is not to strive for the highest score. Rather, the criteria should be used to evaluate the best design for the question addressed by the trial, an approach known as quality by design. For example, for new therapies, there is good reason to improve adherence to assigned therapy as part of the study design, but a collection of minor adverse events beyond the first 1500 patients is an expensive distraction from what matters in trials. On the other hand, most regulated clinical trials for evaluation of products for marketing use inclusion and exclusion criteria and extreme focus on adherence to optimize the likelihood of a beneficial treatment effect in a way that cannot be replicated in practice. The key is to use tools like PRECIS to come to a rational judgment on trial design.

Interestingly, the authors⁴ found that there were no differences in PRECIS-2 scores by funding source (industry vs government). This finding raises the question of whether the National Institutes of Health, as the dominant government funder of clinical trials, is investing enough in stimulating the transformation of this field. The National Institutes of Health Collaboratory and similar efforts by individual institutes at the National Institutes of Health provide important efforts to learn about the best methods. However, there appears to be a shrinking proportional commitment to cardiovascular clinical trials, even in the face of rising mortality and evidence gaps.

We commend the authors⁴ for investigating an important and timely topic in clinical trial design. Their straightforward descriptive study carefully outlines the characteristics of pragmatic RCTs over the past 2 decades and points out that pragmatism and exploration exist along a continuum. Clinicians, health systems, and policy makers continue to struggle to use the limited high-quality evidence in daily decisions around therapeutic and diagnostic choices. Clearly, new methods are needed to accelerate the efficient generation of evidence using the RCT design. The concept of pragmatic clinical trials is hardly new, yet as Sepehrvand et al⁴ show, more work is needed to break from traditional trial design. As they point out, it would be advisable for clinical trialists to self-rate the degree of pragmatism of their proposed trial using a tool like PRECIS-2, and when appropriate for the study question, they should make every effort to increase the degree of pragmatism. For example, recruiting patients through their usual sites of care and using electronic, patient-centered consent forms can greatly improve a trial’s applicability to real-world settings.

Overall, this study⁴ reports only a very modest improvement in published cardiovascular RCT pragmatism over the past 20 years. Now is the time to think critically about how to leverage digital tools and large data platforms to effectively, efficiently, and inclusively generate the practical cardiovascular evidence needed for clinical decision-making and health care policy-making. The deterioration in health outcomes in the United States calls on the biomedical community to accelerate pragmatic evidence development to inform decisions.

Corresponding Author: Fatima Rodriguez, MD, MPH, Division of Cardiovascular Medicine, Stanford University School of Medicine, 870 Quarry Rd, Falk Cardiovascular Research Center, Stanford, CA 94305 (frodrigu@stanford.edu).

Published Online: September 1, 2019. doi:10.1001/jamacardio.2019.3922

Conflict of Interest Disclosures: Dr Harrington reports receiving research grants in his name from Apple, Verily, and the Patient-Centered Outcomes Research Institute and to Stanford from Sanofi, AstraZeneca, Portola, Janssen, Bristol-Myers Squibb, Novartis, CSL, and The Medicines Company; fees for consulting/advisory board services to Element Science, MyoKardia, and WebMD; and membership on the board of directors at Signal Path, the American Heart Association, and Stanford HealthCare. Dr Rodriguez reports personal fees from NovoNordisk, consulting fees and equity from HealthPals, and grants from Verily Life Sciences outside the submitted work. Dr Califf reports employment with Verily Life Sciences and board membership at Cytokinetics; personal fees from Merck, Lilly, Genentech, Boehringer Ingelheim, and Biogen during the conduct of the study; and status of former commissioner of the US Food and Drug Administration. No other disclosures were reported.