Cohort criteria included receiving a prescription for ramipril or telmisartan between January 2003 (start of available data) and September 2009 (prior to the Food and Drug Administration approval of the supplemental indication). ACE-I indicates angiotensin converting enzyme inhibitor; ARB, angiotensin receptor blocker; CAD, coronary artery disease; CHF, congestive heart failure; CVD, cerebrovascular disease; MI, myocardial infarction; PAD, peripheral arterial disease; PCI, percutaneous coronary intervention; T2DM, type 2 diabetes mellitus; and TIA, transient ischemic attack.
eAppendix. Methodology for Selecting Telmisartan
eTable 1. Covariate Definitions
eTable 2. Additional Baseline Characteristics
Customize your JAMA Network experience by selecting one or more topics from the list below.
Fralick M, Kesselheim AS, Avorn J, Schneeweiss S. Use of Health Care Databases to Support Supplemental Indications of Approved Medications. JAMA Intern Med. 2018;178(1):55–63. doi:10.1001/jamainternmed.2017.3919
Can health care databases be used to confirm a supplemental indication that has been demonstrated in a randomized clinical trial for an approved medication?
This cohort study replicated the results of a randomized clinical trial that established the supplemental indication for telmisartan by using data from a US health care database (insurance claims data) available at the time that the supplemental indication was approved. Similar to the randomized clinical trial, our study revealed a decreased risk of angioedema with telmisartan compared with ramipril.
In certain clinical scenarios, database studies may support supplemental effectiveness applications for already approved medications.
Manufacturers of US Food and Drug Administration–approved prescription drugs often apply for additional indications based on randomized clinical trials. Real-world database analyses on a medication’s use and outcomes in routine settings of care might help to inform decision making regarding such supplemental indications.
To examine whether longitudinal data from a health care database can support the results of a randomized clinical trial that led to a supplemental indication for telmisartan.
Design, Setting, and Participants
This cohort study of patients newly prescribed telmisartan or ramipril used insurance claims data from a nationwide health care database from January 1, 2003, through September 30, 2009, to compare patient outcomes. This study replicated the inclusion and exclusion criteria used in the Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial (ONTARGET) and used propensity score matching to balance 74 patient characteristics. Data analysis was performed from February 15, 2017, to May 24, 2017.
Telmisartan use vs ramipril use.
Main Outcomes and Measures
The primary outcome was a composite of myocardial infarction, stroke, or hospitalization for congestive heart failure.
Of the 640 951 patients included in the study, 48 053 were newly prescribed ramipril (mean [SD] age, 68.29 [9.52] years; 31 940 male [66.5%]) and 4665 were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 male [51.7%]). After propensity score matching, a total of 4665 patients were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 [51.7%]), and 4665 patients were newly prescribed ramipril (mean [SD] age, 69.36 [9.67] years; 2343 male [50.2%]). As seen in ONTARGET, the composite risk of stroke, myocardial infarction, and hospitalization for congestive heart failure was similar for the 2 medications (hazard ratio, 1.0; 95% CI, 0.9-1.1). In addition, the study found that telmisartan was associated with a substantially decreased risk of angioedema (hazard ratio, 0.1; 95% CI, 0.03-0.56) compared with ramipril.
Conclusions and Relevance
Real-world data analyses of patients receiving routine care provided findings similar to those found in the randomized clinical trial that established telmisartan’s supplemental indication. In certain situations, database studies may support supplemental applications for effectiveness for already approved medications.
In December 2016, the 21st Century Cures Act was signed into law in the United States.1 It contained a provision intended to promote real-world data studies of medication use and outcomes in routine clinical settings in US Food and Drug Administration (FDA) authorization of additional indications for already approved prescription drugs.1 Such data, with or without randomization, are drawn from health care use data, insurance claims, registry studies, and/or electronic health record systems in typical clinical settings of care.2-4 Although the FDA has long used such data to clarify the safety of medications, the data can seldom establish a drug’s effectiveness. Well-designed randomized clinical trials are the criterion standard for assessing whether a drug is efficacious because random treatment assignment and a controlled research environment can more readily support causal inferences.
In recent years, new methodologic approaches have improved the validity and reproducibility of nonrandomized data, including new-user designs,5 active comparators, propensity score (PS) matching, and controlling for disease risk scores.6,7 Other important aspects include assessing covariates before cohort entry (to avoid adjusting for intermediate variables) and defining cohort entry as the time when the patient first receives the exposure of interest (to decrease the possibility of immortal time bias).6-8
Can such analytic techniques confirm supplemental indications for already approved drugs? Approximately half of all drugs approved in the United States are later approved for supplemental indications, modifications to the initial indication, or expanded populations.9,10 Supplemental indications are typically identified on the basis of prospective clinical trials. To determine whether real-world data analyses can confirm a supplemental indication, we identified a supplemental approval amenable to study and applied the same inclusion and exclusion criteria and outcomes measurements that were used in the pivotal randomized clinical trial.
Our cohort study was conducted in commercially insured patients using the MarketScan health care database provided by Truven (January 1, 2003, through September 30, 2009). This nationwide database captures anonymized longitudinal, individual-level data on health care use, patient demographics, inpatient and outpatient diagnostic and procedural codes, and pharmacy dispensing of prescription drugs for more than 60 million commercially insured people in the United States. The study was approved by the institutional review board at Brigham and Women’s Hospital, including a waiver for informed consent, and a valid data licensing agreement was in place. All data were anonymized and deidentified.
To identify an experimental setting, we reviewed all supplemental applications to the FDA from 2005 to 2014 and their accompanying clinical trials.9 The supplemental indications were classified into 3 mutually exclusive categories: new indication (n = 138), modification (n = 86), and expansion (n = 66) (eAppendix in the Supplement).9 Of the 138 new indications, 108 (78.3%) of the pivotal clinical trials had a primary outcome that was not identifiable in US longitudinal health care databases (eg, pathology results, change in clinical scores, and radiologic tumor response), 12 (8.7%) did not have an active comparator, 4 (2.9%) were based on in-hospital medication administration (eg, postoperative nausea medication, anesthetic medications), and 14 (10.1%) were potentially replicable with the claims data available to us. Of the 14, we selected telmisartan a priori and did not analyze data for the other 13 (eAppendix in the Supplement).
The angiotensin receptor blocker (ARB) telmisartan (Micardis) was approved as an antihypertensive in 1998. In October 2009, it was approved supplementarily for cardiovascular risk reduction in patients 55 years or older who are unable to take angiotensin-converting enzyme inhibitors (ACE-Is) and have a high risk of major cardiovascular events. Telmisartan was an optimal case study for 3 reasons. First, the primary outcome in the pivotal supplemental indication trial could be accurately identified in health care use data. Second, the randomized clinical trial used an active comparator, the ACE-I ramipril (Altace), which would minimize confounding in cohort studies.3,7 Third, the inclusion criteria, exclusion criteria, and baseline patient characteristics were identifiable in claims data.
The trial that identified the supplemental indication for telmisartan for cardiovascular risk reduction, Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial (ONTARGET), was published in April 2008.11 ONTARGET’s primary objectives were to determine whether telmisartan was at least as effective as ramipril at reducing cardiovascular risk and to assess whether the combination of telmisartan and ramipril was more effective than ramipril alone. The trial was conducted across 733 centers in 40 countries between 2001 and 2008.11
Potentially eligible patients must have had at least 6 months of continuous enrollment in a participating health plan before the date of cohort entry. Our inclusion and exclusion criteria mirrored those of ONTARGET.11 We included patients 55 years or older who filled a new prescription for telmisartan or ramipril (no fills for either drug or any other ACE-I or ARB during the prior 180 days). Cohort entry date was the first day of a prescription fill. As in ONTARGET, we included patients with a diagnosis of coronary artery disease, peripheral artery disease, cerebrovascular disease, or diabetes mellitus during the 180 days before cohort entry.
As in ONTARGET, we excluded patients with a limited life expectancy (ie, living in a hospice, palliative care facility, or a nursing home and those with cancer), liver disease, syncope or a recent myocardial infarction (within 2 days of cohort entry), transient ischemic attack (within 7 days of cohort entry), percutaneous transluminal coronary angiography (within 30 days of cohort entry), or hospitalization for congestive heart failure during the 180 days before cohort entry. Other exclusion criteria used in ONTARGET were not applied because they were not readily identifiable (known allergy to study medication, unable to tolerate study medication, hemodynamically significant primary valvular or outflow tract obstruction, uncorrected volume or sodium depletion, planned cardiac procedure, blood pressure >160/100 mm Hg despite treatment, significant renal artery stenosis, and angina in the absence of multivessel coronary artery disease) or rare (hereditary fructose intolerance, complex congenital heart disease, primary hyperaldosteronism, and heart transplant). We also excluded patients who previously received any ACE-I or ARB.
Our primary outcome was a composite of myocardial infarction, stroke, or hospitalization for congestive heart failure using the primary discharge diagnosis code for an inpatient visit (see eTable 1 in the Supplement for International Classification of Diseases, Ninth Revision codes). These definitions have satisfactory measurement characteristics; the positive predictive value for myocardial infarction was 93% or higher; stroke, 81% or higher; and congestive heart failure, 87% or higher.12-14 Cardiovascular deaths were included in the composite outcome if they occurred during a hospitalization for myocardial infarction, stroke, or heart failure but not outside the hospital.
Our primary analysis compared the rates of the composite end point among patients initiating treatment with telmisartan vs ramipril. Data were censored for patients when they discontinued use of their initial medication, switched to the comparator medication, experienced a study outcome, disenrolled from their health plan, or died, or on September 30, 2009.15 To address confounding, we adjusted for 74 patient characteristics, including demographics, comorbid conditions, concurrent medications, and health care use measures, using PS methods (Table 1 and eTable 2 in the Supplement). To balance patient characteristics, we used 1:1 PS matching with a caliper of 0.05 and did not perform further variable selection. We compared standardized differences to evaluate the level of balance achieved in patient characteristics after PS matching16 and used unstratified Cox proportional hazards regression to compute hazard ratios (HRs) and 95% CIs. We then performed a predefined secondary analysis that carried forward the exposure to the first-used medication for 365 days.6
To assess the robustness of our results, we also sought to confirm the well-established increased risk of angioedema for ramipril, expecting that rates of angioedema would be lower for telmisartan, as also demonstrated in ONTARGET. To further assess robustness, we replicated all study end points using a larger cohort derived from less stringent exclusion criteria by creating a cohort that allowed for past ACE-I or ARB use other than telmisartan or ramipril in the preceding 180 days. All analyses were conducted using the Aetion platform and R, version 18.104.22.168 (R Foundation for Statistical Computing), which has been previously validated for a range of studies17,18 and for predicting clinical trial findings.19
We identified 640 951 patients who filled a prescription for ramipril or telmisartan from January 1, 2003, through September 30, 2009, and had a sufficient baseline enrollment period of at least 180 days. After applying study inclusion and exclusion criteria, 52 739 patients were included (Figure), of whom 48 053 were newly prescribed ramipril (mean [SD] age, 68.29 [9.52] years; 31 940 male [66.5%]) and 4665 were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 male [51.7%]) (a total of 21 patients did not begin follow-up). Patients prescribed ramipril were more likely to be male and have cardiac disease, whereas patients prescribed telmisartan were more likely to have hypertension, kidney disease, and previous transient ischemic attack or stroke and be prescribed a calcium channel blocker (Table 1). After PS matching 4665 telmisartan users (mean [SD] age, 69.43 [9.60] years; 2413 [51.7%]) to 4665 ramipril users (mean [SD] age, 69.36 [9.67] years; 2343 male [50.2%]), these differences were well balanced with standardized differences less than 0.1 (Table 1). Most frequencies of baseline characteristics were consistent with ONTARGET (eg, similar age, rates of hypertension, coronary artery disease, diabetes, and stroke), whereas some were not (ie, lower rates of angina, lower rates of smoking, less documented antiplatelet use, and more women included in our study) (eTable 2 in the Supplement). In the unmatched cohort, mean follow-up time was 232 days (interquartile range, 113-454 days) for the ramipril group and 188 days (interquartile range, 108-427 days) for the telmisartan group. The most common reason for censoring was treatment discontinuation, in 32 135 ramipril users (66.9%) and 3483 telmisartan users (74.7%).
In ONTARGET, the relative risk of the composite outcome of death from cardiovascular causes, myocardial infarction, stroke, or hospitalization for congestive heart failure was 1.01 (95% CI, 0.94-1.09), indicating no significant difference between telmisartan and ramipril. In our study, the PS-matched relative risk of the composite of myocardial infarction, stroke, or hospitalization for congestive heart failure was almost identical (HR, 0.99; 95% CI, 0.85-1.14) (Table 2).
A sensitivity analysis using the last exposure to the first-used medication for 365 days without considering treatment discontinuation found that the primary end point occurred in 402 ramipril users (86 events per 1000 patients) and 363 telmisartan users (78 events per 1000 patients). This resulted in no significant difference in risk after PS matching (HR, 0.90; 95% CI, 0.77-1.04).
Among PS-matched individuals, there were 18 angioedema events in new users of ramipril (3.1 events per 1000 person-years) and 2 events in new users of telmisartan (0.4 events per 1000 person-years). A decreased risk (HR, 0.13; 95% CI, 0.03-0.56) of angioedema with telmisartan was also observed in ONTARGET (HR, 0.40; P = .01).
In the cohort with less stringent exclusion criteria to allow for past ACE-I or ARB use apart from ramipril or telmisartan, we identified 8656 PS-matched new users of telmisartan and 8656 PS-matched new users of ramipril. In this cohort, there was a similar PS-matched relative risk of the composite of myocardial infarction, stroke, or hospitalization for congestive heart failure (HR, 0.97; 95% CI, 0.88-1.08) (Table 3). A decreased PS-matched risk of angioedema with telmisartan compared with ramipril (HR, 0.35; 95% CI, 0.17-0.71) was also revealed.
Among patients newly prescribed telmisartan and ramipril before the FDA’s decision to approve a supplemental indication for telmisartan, we found results that were almost identical to those of the randomized clinical trial that led to telmisartan’s supplemental indication. We further identified and quantified the known causal association between ramipril and angioedema. This finding suggests that our data and analysis plan were sufficiently valid to detect known causal associations first identified in a prospective trial.20
This study is one of the largest to analyze real-world data to mirror a large randomized clinical trial that had established the clinical basis for a supplemental indication for a medication. In contrast to ONTARGET, which took approximately 7 years to complete and cost tens of millions of dollars, our study took approximately 12 weeks to implement for less than a hundredth of the cost. The fact that our case study bolstered the conclusions of a trial designed to identify a supplemental indication for a marketed medication and was done relatively efficiently using available data sets, rigorous epidemiologic methods, and modern software platforms supports the concept of conducting similar database analyses as part of routine practice for manufacturers submitting applications for supplemental indications to the FDA.21
Results concordant with the pivotal clinical trial can provide regulators with greater confidence in approving the indication, whereas discordant results could warrant deeper reexamination of the clinical trial or nonrandomized data. When results are discordant with the pivotal trial, an in-depth analysis of the trial and the nonrandomized study will be necessary to identify reasons for this discordance. These reasons can include issues related to study design, statistical analysis, and patient population. Additional research will be necessary to help navigate this scenario.22 Eventually, the FDA can develop empirically based guidance on when database analyses are useful in this context and when they are less reliable as a confirmatory source.
There have been examples of real-world data providing results before the randomized clinical trial was completed23-25 and nonrandomized real-world studies that changed prescribing practices for which there will likely never be randomized clinical trial findings.26-29 A common signal of quality among these studies and our current study was the use of a new-user, active-comparator design. This approach compares 2 groups of patients who newly start taking a medication and avoids comparing 2 groups with intrinsically discrepant risk profiles as would be found using a nonuser comparator or comparing new users with ongoing users. The new-user design with an active comparator allows a more homogeneous baseline population and was one of the main reasons why the observed baseline characteristics for our patients were similar even before matching. By design, approximately 80% of the 74 baseline characteristics were well balanced before PS matching, suggesting that unmeasured factors may be equally balanced. Similar results were observed in the recent new-user, active-comparator study by Graham et al26 that compared the safety and effectiveness of rivaroxaban with those of dabigatran.
By contrast, some nonrandomized real-world studies30,31 found results that differed from those in subsequent randomized clinical trials.32-34 This difference can occur for many reasons, including incorrect study design implementation, reverse causation,35 immortal time bias,8 depletion of susceptibles,36 failure to identify important unmeasured confounding factors, or the inclusion of a different study population than was used in the clinical trial. In particular, comparators that use patients defined as those who did not fill a prescription (nonusers) may introduce treatment selection bias that may not be controllable with any statistical method.37,38
Studies such as ours require that inclusion and exclusion criteria and end points be adequately defined in a randomized clinical trial report and subsequently identifiable in the health care data set being studied. Many trials include study end points that are not recorded in claims data or electronic health care records (eg, rating scales used in trials of psychiatric medications) or not easily identifiable (eg, progression-free survival used in oncology trials) without requiring challenging natural language processing of free-text information. It would also be difficult to replicate results from randomized clinical trials that include different treatment modalities with substantially different risk-benefit profiles (eg, implantable cardioverter defibrillators compared with medical therapy) because of fundamental differences in risk profiles between the 2 populations.7,38
Pharmacoepidemiology analysis of data from nonrandomized, real-world health care databases can be used to support supplemental indications established in prospective randomized clinical trials of marketed medications. This is powerful because they represent outcomes in settings of typical care, rather than the highly controlled research environments of RCTs, and can be accomplished quickly and inexpensively. The analyses can also include subgroups of patients who are underrepresented in clinical trials, including elderly individuals, patients with many comorbidities, pregnant women, and other at-risk groups. In our study, for example, 50% of patients were women compared with approximately 26% in ONTARGET. Finally, such studies can evaluate a larger population of patients and can assess end points that trials are often underpowered to detect, such as rare adverse events.
Our observed null finding might reflect limitations within our data set (eg, lack of out-of-hospital death data), duration of follow-up, or study design rather than a true observation. It is well established that noninferiority can appear to be present because of inadequate rigor or scale in any study, whether a randomized clinical trial or an observational analysis.39,40 However, this does not explain the increased risk of angioedema that we observed with ramipril but not telmisartan. Some authors41,42 have questioned the value of PS matching over traditional risk-adjusted regression analysis, neither of which guarantee full account for unmeasured confounding. However, our unadjusted primary, secondary, and sensitivity analyses did not change meaningfully after PS matching. Another limitation of our study was an inability to assess medication adherence beyond prescription filling, although this is generally seen as a valid measure of actual use.15
The FDA is currently considering how it will use nonrandomized, real-world data as part of supplemental indication applications.42,43 In the absence of large-scale empirical comparative analyses that identify the reasons for failure and success to replicate randomized controlled findings with real-world data analyses, we performed a case study that highlights some important considerations. Many context-specific questions about study design, confounding control, data quality, and outcome validity will need to be considered.4,6 Preregistering study designs and analysis plans and providing a publicly available summary of the results when available, similar to the current practice of randomized clinical trials, promotes ethical conduct of these studies.
Even well-designed analyses sometimes result in incorrect conclusions, and some randomized clinical trials may be inaccurate.44 Retrospective reviews of the literature34,45-48 provide single summarizations of the differences between these 2 approaches but provide few insights on the validity of individual real-world data analyses. To establish a meaningful baseline, the FDA will need many sets of randomized clinical trials with prospectively designed, nonrandomized analyses to match the populations included in randomized clinical trials across a range of clinical questions, each investigated with a set of designs and methods following rigorous epidemiologic principles.
Regulators have a difficult task in providing specific rules for decision making in this maturing yet still developing and highly context-specific field. However, if done selectively and with principled methods, it might be feasible to use nonrandomized, real-world data to provide supportive evidence in establishing supplemental drug indications.
Corresponding Author: Michael Fralick, MD, The Program on Regulation, Therapeutics, and Law, Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, 1620 Tremont St, Ste 3030, Boston, MA 02120 (firstname.lastname@example.org).
Accepted for Publication: August 15, 2017.
Published Online: November 20, 2017. doi:10.1001/jamainternmed.2017.3919
Open Access: This article is published under the JN-OA license and is free to read on the day of publication.
Author Contributions: Dr Schneeweiss had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Fralick.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Fralick, Schneeweiss.
Obtained funding: Kesselheim, Avorn, Schneeweiss.
Administrative, technical, or material support: Fralick, Avorn.
Study supervision: Kesselheim, Avorn, Schneeweiss.
Conflict of Interest Disclosure: Dr Kesselheim reported receiving grants from the US Food and Drug Administration Office of Generic Drugs and Division of Health Communication (2013-2016) unrelated to the topic of this article. Dr Schneeweiss reported receiving support from grants from the US Food and Drug Administration and the Patient Centered Outcomes Research Institute; serving as a consultant to WHISCON, LLC, and Aetion Inc, a software manufacturer of which he also owns equity; and being a principal investigator of investigator-initiated grants to the Brigham and Women’s Hospital from Genentech, Bayer, and Boehringer Ingelheim (not directly related to the topic of this article). No other disclosures were reported.
Funding/Support: This study was funded by the Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women’s Hospital, and the Laura and John Arnold Foundation. Dr Fralick was funded by the University of Toronto Clinician Scientist Training Program, Toronto, Ontario, Canada. Dr Kesselheim’s work is additionally supported by the Engelberg Foundation and the Harvard Program in Therapeutic Science.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: