Feasibility of Using Real-world Data to Emulate Postapproval Confirmatory Clinical Trials of Therapeutic Agents Granted US Food and Drug Administration Accelerated Approval

Under the accelerated approval pathway of the US Food and Drug Administration (FDA), therapeutic agents targeting serious or life-threatening diseases can receive approval on the basis of surrogate markers that are reasonably likely to predict clinical benefit conditional on the conduct of postapproval confirmatory trials.¹ As required by the 21st Century Cures Act of 2016, the FDA has developed guidance on the use of observational methods and real-world data (RWD) (eg, billing, claims, and electronic health record [EHR] data) to generate clinical evidence to fulfill postapproval study requirements.² Prior evaluations have suggested important limitations to feasibly emulating trials using RWD because of difficulties in reliably ascertaining interventions, indications, trial inclusion and exclusion criteria, and primary end points from claims and/or structured EHR data.³

A better understanding of the feasibility of emulating FDA-required postapproval trials conducted to verify clinical benefit is critical because these studies often face recruitment challenges, continue to focus on surrogate markers as end points,¹ and are delayed for years after approval.^1,4 Therefore, we conducted this cross-sectional study to examine the feasibility of using RWD to emulate FDA-required postapproval confirmatory trials for all new therapeutic agents that received accelerated approval between 2009 and 2018.

This study followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) reporting guideline. This study did not require institutional review board approval because it was based on publicly available information, in accordance with 45 CFR §46. Informed consent was not needed because no patient data were used.

We used the Drugs@FDA database to identify all FDA-required postapproval confirmatory trials for new molecular entity drugs and biologics that were regulated by the FDA Center for Drug Evaluation and Research and granted accelerated approval between 2009 and 2018.⁵ For all postapproval confirmatory trials designed to verify efficacy, we reviewed study descriptions, ClinicalTrials.gov information, and abstracts of publications to determine the proportion of trials for which the (1) clinical indication, (2) at least 80% of the clinical inclusion and exclusion criteria, (3) the comparator, and (4) the primary end point(s) could be routinely ascertained from RWD using previously described methods (eTable in the Supplement).³ These characteristics were considered unlikely to be routinely ascertained from observational data if researchers would find it difficult to develop a computable phenotype using available RWD sources.³ The data were analyzed in Microsoft Excel.

Between 2009 and 2018, the FDA approved 41 new therapeutic agents via the accelerated approval pathway, requiring a total of 50 postapproval confirmatory trials (Table). Twenty postapproval confirmatory trials (40%) were ongoing at the time of accelerated approval.

Among the 50 postapproval confirmatory trials, 12 (24%) had a clinical indication that could be routinely ascertained from claims and/or structured EHR data, whereas 38 (76%) required nonroutinely ascertainable disease severity or treatment-related qualifiers (Figure). Of the 46 trials for which clinical inclusion and exclusion criteria were available on ClinicalTrials.gov, 2 (4%) had at least 80% of their criteria that could be routinely ascertained. Of the 41 trials with a comparator arm, 22 (54%) used active comparators, all of which could be ascertained using RWD. Of the 49 trials for which primary end point information was available, 20 (40%) had at least 1 end point that could be routinely ascertained from RWD. Overall, none of the FDA-required postapproval confirmatory trials had all of the following: (1) a clinical indication, (2) at least 80% of the clinical inclusion and exclusion criteria, (3) a comparator, and (4) at least 1 primary end point that could be routinely ascertained from RWD.

The findings of this cross-sectional study suggest that none of the 50 FDA-required postapproval confirmatory trials for therapeutic agents granted accelerated approval between 2009 and 2018 could have been feasibly emulated using currently available claims and/or structured EHR data. In particular, the narrowly defined indications and strict inclusion and exclusion criteria of the FDA-required postapproval confirmatory trials precluded emulation using RWD. Although RWD can be used to ascertain clinical outcomes for real-world populations receiving therapeutic agents, our findings suggest that current observational methods and RWD can complement, but are unlikely to replace, postapproval confirmatory trial requirements.

This study had some limitations. Certain determinations were subjective, we did not consider the relative importance of an individual criterion, and we did not consider accelerated approvals for supplemental indications. Our findings suggest that to use RWD for regulatory evaluations, initiatives are necessary to standardize data elements across different facilities, clinicians, and EHRs.⁶

Accepted for Publication: September 9, 2021.

Published: November 9, 2021. doi:10.1001/jamanetworkopen.2021.33667

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Wallach JD et al. JAMA Network Open.

Corresponding Author: Joshua D. Wallach, PhD, MS, Department of Environmental Health Sciences, Yale School of Public Health, 60 College St, Floor 4, Room 411, New Haven, CT 06510 (joshua.wallach@yale.edu).

Author Contributions: Dr Wallach had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Wallach, Bartlett, Ross.

Acquisition, analysis, or interpretation of data: Wallach, Zhang, Skydel, Dhruva, Shah, Ross.

Drafting of the manuscript: Wallach.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Wallach.

Supervision: Wallach.

Conflict of Interest Disclosures: Dr Wallach reported receiving funding from the US Food and Drug Adaministration (FDA). Dr Zhang reported receiving research funding from the FDA. Dr Dhruva reported receiving research funding from Arnold Ventures, the FDA, the Greenwall Foundation, the National Evaluation System for Health Technology Coordinating Center, and the National Heart, Lung, and Blood Institute (NHLBI). Dr Shah reported receiving research support through the Mayo Clinic from the FDA, the Centers for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, the National Science Foundation, and the Patient-Centered Outcomes Research Institute. Dr Ross reported receiving research support through Yale University from Johnson & Johnson to develop methods of clinical data sharing, from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology, from the FDA for the Yale University–Mayo Clinic Center for Excellence in Regulatory Science and Innovation, from the Agency for Healthcare Research and Quality, from the NHLBI, and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International. No other disclosures were reported.

Funding/Support: This study was supported by award K01AA028258 from the National Institute on Alcohol Abuse and Alcoholism of the NIH (Dr Wallach).

Role of the Funder/Sponsor: The National Institute on Alcohol Abuse and Alcoholism of the NIH had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.