Utility of 90-Day Mortality vs 30-Day Mortality as a Quality Metric for Transcatheter and Surgical Aortic Valve Replacement Outcomes | Valvular Heart Disease | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Figure 1.  Changes in Hospital Performance Rankings at 90 Days and 1 Year Stratified by 30-Day Mortality Performance Groups for Transcatheter Aortic Valve Replacement (TAVR) and Surgical Aortic Valve Replacement (SAVR)
Changes in Hospital Performance Rankings at 90 Days and 1 Year Stratified by 30-Day Mortality Performance Groups for Transcatheter Aortic Valve Replacement (TAVR) and Surgical Aortic Valve Replacement (SAVR)

Hospitals are classified into 3 groups according to their mean 30-day mortality rate during the 4-year period: (1) top-performing group (lowest 10% mortality rate), (2) middle-performing group (middle 80% mortality rate), or (3) bottom-performing group (highest 10% mortality). Hospital performance rankings vary significantly over time for both procedures such that a significant proportion of hospitals changed performance rankings from 30 to 90 days.

Figure 2.  Kaplan-Meier Survival Curves Stratified by 30-Day Mortality Hospital Performance Groups for Transcatheter Aortic Valve Replacement (TAVR) and Surgical Aortic Valve Replacement (SAVR)
Kaplan-Meier Survival Curves Stratified by 30-Day Mortality Hospital Performance Groups for Transcatheter Aortic Valve Replacement (TAVR) and Surgical Aortic Valve Replacement (SAVR)

The SAVR group was used as a reference cohort. Separate Kaplan-Meier survival analyses are run for each procedure with pairwise and overall log-rank tests. Both 90-day and 1-year survival remained statistically significant between groups based on 30-day mortality performance, with a heavy penalty on 1-year survival for the bottom-performing hospitals for both procedures.

Figure 3.  Instantaneous Hazard of Procedure-Related Mortality, First Readmission, and Comparison of the Respective Time-Interval Hazard Ratios During the First Postoperative Year in the Overall Cohort
Instantaneous Hazard of Procedure-Related Mortality, First Readmission, and Comparison of the Respective Time-Interval Hazard Ratios During the First Postoperative Year in the Overall Cohort

Time zero is the day of the procedure. The instantaneous hazard diminishes rapidly during the first 30-day period but remained elevated through 90 days and then plateaued to fairly constant levels thereafter during the first year for both procedural-related mortality and first readmission. Correspondingly, only marginal change in hazard ratios was observed beyond 90 days. Dashed lines in A and B and whiskers in C indicate 95% CIs.

Table 1.  Characteristics and Outcomes of Patients Undergoing Transcatheter Aortic Valve Replacement Stratified by 30-Day Mortality Hospital Performance Ranking
Characteristics and Outcomes of Patients Undergoing Transcatheter Aortic Valve Replacement Stratified by 30-Day Mortality Hospital Performance Ranking
Table 2.  Changes in Hospital Performance Rankings Based on Unadjusted 30-Day Mortality Hospital Performance Groups for TAVR and SAVR
Changes in Hospital Performance Rankings Based on Unadjusted 30-Day Mortality Hospital Performance Groups for TAVR and SAVR
1.
Bryant  AS, Rudemiller  K, Cerfolio  RJ.  The 30- versus 90-day operative mortality after pulmonary resection.  Ann Thorac Surg. 2010;89(6):1717-1722. doi:10.1016/j.athoracsur.2010.01.069PubMedGoogle ScholarCrossref
2.
Adam  MA, Turner  MC, Sun  Z,  et al.  The appropriateness of 30-day mortality as a quality metric in colorectal cancer surgery.  Am J Surg. 2018;215(1):66-70. doi:10.1016/j.amjsurg.2017.04.018PubMedGoogle ScholarCrossref
3.
Mizushima  T, Yamamoto  H, Marubashi  S,  et al.  Validity and significance of 30-day mortality rate as a quality indicator for gastrointestinal cancer surgeries.  Ann Gastroenterol Surg. 2018;2(3):231-240. doi:10.1002/ags3.12070PubMedGoogle ScholarCrossref
4.
Koeckert  MS, Grossi  EA, Vining  PF,  et al.  Ninety-day readmissions of bundled valve patients: implications for healthcare policy.  Semin Thorac Cardiovasc Surg. 2019;31(1):32-37. doi:10.1053/j.semtcvs.2018.07.017PubMedGoogle ScholarCrossref
5.
McMillan  RR, Berger  A, Sima  CS,  et al.  Thirty-day mortality underestimates the risk of early death after major resections for thoracic malignancies.  Ann Thorac Surg. 2014;98(5):1769-1774. doi:10.1016/j.athoracsur.2014.06.024PubMedGoogle ScholarCrossref
6.
Pezzi  CM, Mallin  K, Mendez  AS, Greer Gay  E, Putnam  JB  Jr.  Ninety-day mortality after resection for lung cancer is nearly double 30-day mortality.  J Thorac Cardiovasc Surg. 2014;148(5):2269-2277. doi:10.1016/j.jtcvs.2014.07.077PubMedGoogle ScholarCrossref
7.
Damhuis  RA, Wijnhoven  BP, Plaisier  PW, Kirkels  WJ, Kranse  R, van Lanschot  JJ.  Comparison of 30-day, 90-day and in-hospital postoperative mortality for eight different cancer types.  Br J Surg. 2012;99(8):1149-1154. doi:10.1002/bjs.8813PubMedGoogle ScholarCrossref
8.
Mise  Y, Vauthey  JN, Zimmitti  G,  et al.  Ninety-day postoperative mortality is a legitimate measure of hepatopancreatobiliary surgical quality.  Ann Surg. 2015;262(6):1071-1078. doi:10.1097/SLA.0000000000001048PubMedGoogle ScholarCrossref
9.
Swanson  RS, Pezzi  CM, Mallin  K, Loomis  AM, Winchester  DP.  The 90-day mortality after pancreatectomy for cancer is double the 30-day mortality: more than 20,000 resections from the national cancer data base.  Ann Surg Oncol. 2014;21(13):4059-4067. doi:10.1245/s10434-014-4036-4PubMedGoogle ScholarCrossref
10.
Centers for Medicare & Medicaid Services. Proposed Decision Memo for Transcatheter Aortic Valve Replacement (TAVR). https://www.cms.gov/medicare-coverage-database/details/nca-proposed-decision-memo.aspx?NCAId=293. Accessed April 12, 2019.
11.
Mack  MJ, Leon  MB, Thourani  VH,  et al; PARTNER 3 Investigators.  Transcatheter aortic-valve replacement with a balloon-expandable valve in low-risk patients.  N Engl J Med. 2019;380(18):1695-1705. doi:10.1056/NEJMoa1814052PubMedGoogle ScholarCrossref
12.
Popma  JJ, Deeb  GM, Yakubov  SJ,  et al; Evolut Low Risk Trial Investigators.  Transcatheter aortic-valve replacement with a self-expanding valve in low-risk patients.  N Engl J Med. 2019;380(18):1706-1715. doi:10.1056/NEJMoa1816885PubMedGoogle ScholarCrossref
13.
Button  KS, Ioannidis  JP, Mokrysz  C,  et al.  Power failure: why small sample size undermines the reliability of neuroscience.  Nat Rev Neurosci. 2013;14(5):365-376. doi:10.1038/nrn3475PubMedGoogle ScholarCrossref
14.
Mack  MJ, Leon  MB, Smith  CR,  et al; PARTNER 1 trial investigators.  5-year outcomes of transcatheter aortic valve replacement or surgical aortic valve replacement for high surgical risk patients with aortic stenosis (PARTNER 1): a randomised controlled trial.  Lancet. 2015;385(9986):2477-2484. doi:10.1016/S0140-6736(15)60308-7PubMedGoogle ScholarCrossref
15.
Leon  MB, Smith  CR, Mack  M,  et al; PARTNER Trial Investigators.  Transcatheter aortic-valve implantation for aortic stenosis in patients who cannot undergo surgery.  N Engl J Med. 2010;363(17):1597-1607. doi:10.1056/NEJMoa1008232PubMedGoogle ScholarCrossref
16.
Kodali  SK, Williams  MR, Smith  CR,  et al; PARTNER Trial Investigators.  Two-year outcomes after transcatheter or surgical aortic-valve replacement.  N Engl J Med. 2012;366(18):1686-1695. doi:10.1056/NEJMoa1200384PubMedGoogle ScholarCrossref
17.
Adams  DH, Popma  JJ, Reardon  MJ,  et al; U.S. CoreValve Clinical Investigators.  Transcatheter aortic-valve replacement with a self-expanding prosthesis.  N Engl J Med. 2014;370(19):1790-1798. doi:10.1056/NEJMoa1400590PubMedGoogle ScholarCrossref
18.
Gleason  TG, Reardon  MJ, Popma  JJ,  et al; CoreValve U.S. Pivotal High Risk Trial Clinical Investigators.  5-Year outcomes of self-expanding transcatheter versus surgical aortic valve replacement in high-risk patients.  J Am Coll Cardiol. 2018;72(22):2687-2696. doi:10.1016/j.jacc.2018.08.2146PubMedGoogle ScholarCrossref
19.
O’Brien  SM, Shahian  DM, Filardo  G,  et al; Society of Thoracic Surgeons Quality Measurement Task Force.  The Society of Thoracic Surgeons 2008 cardiac surgery risk models, part 2: isolated valve surgery.  Ann Thorac Surg. 2009;88(1)(suppl):S23-S42. doi:10.1016/j.athoracsur.2009.05.056PubMedGoogle ScholarCrossref
20.
Quan  H, Li  B, Couris  CM,  et al.  Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.  Am J Epidemiol. 2011;173(6):676-682. doi:10.1093/aje/kwq433PubMedGoogle ScholarCrossref
21.
Teixeira-Pinto  A, Normand  SL.  Statistical methodology for classifying units on the basis of multiple-related measures.  Stat Med. 2008;27(9):1329-1350. doi:10.1002/sim.3187PubMedGoogle ScholarCrossref
22.
Talsma  AK, Lingsma  HF, Steyerberg  EW, Wijnhoven  BP, Van Lanschot  JJ.  The 30-day versus in-hospital and 90-day mortality after esophagectomy as indicators for quality of care.  Ann Surg. 2014;260(2):267-273. doi:10.1097/SLA.0000000000000482PubMedGoogle ScholarCrossref
23.
In  H, Palis  BE, Merkow  RP,  et al.  Doubling of 30-day mortality by 90 days after esophagectomy: a critical measure of outcomes for quality improvement.  Ann Surg. 2016;263(2):286-291. doi:10.1097/SLA.0000000000001215PubMedGoogle ScholarCrossref
24.
Gaudiani  V, Deeb  GM, Popma  JJ,  et al.  Causes of death from the randomized CoreValve US Pivotal High-Risk Trial.  J Thorac Cardiovasc Surg. 2017;153(6):1293-1301.e1. doi:10.1016/j.jtcvs.2016.11.069PubMedGoogle ScholarCrossref
25.
Amrane  H, Deeb  GM, Popma  JJ,  et al; SURTAVI Trial Causes of Death Working Group.  Causes of death in intermediate-risk patients: The Randomized Surgical Replacement and Transcatheter Aortic Valve Implantation Trial.  J Thorac Cardiovasc Surg. 2019;158(3):718-728.e3. doi:10.1016/j.jtcvs.2018.11.129PubMedGoogle ScholarCrossref
26.
Siregar  S, Groenwold  RH, de Mol  BA,  et al.  Evaluation of cardiac surgery mortality rates: 30-day mortality or longer follow-up?  Eur J Cardiothorac Surg. 2013;44(5):875-883. doi:10.1093/ejcts/ezt119PubMedGoogle ScholarCrossref
Original Investigation
December 18, 2019

Utility of 90-Day Mortality vs 30-Day Mortality as a Quality Metric for Transcatheter and Surgical Aortic Valve Replacement Outcomes

Author Affiliations
  • 1Division of Cardiac Surgery, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
  • 2Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
JAMA Cardiol. 2020;5(2):156-165. doi:10.1001/jamacardio.2019.4657
Key Points

Question  What is the validity of 90-day mortality as a hospital performance metric for transcatheter aortic valve replacement and surgical aortic valve replacement?

Findings  This cohort study of Medicare beneficiaries found that evaluation of hospital performance based on 30-day mortality may underestimate outcomes and therefore substantially misrepresent institutional performance after transcatheter aortic valve replacement and surgical aortic valve replacement compared with 90-day mortality, even after risk adjustment. Capturing 90-day events was also more robustly informative regarding expected 1-year outcomes.

Meaning  Although 30-day mortality has been validated, 90-day mortality may be a more reliable outcome metric for measuring hospital performance and capturing procedure-related mortality after transcatheter aortic valve replacement and surgical aortic valve replacement.

Abstract

Importance  Questions have recently arisen as to whether 30-day mortality is a reasonable metric for understanding institutional practice differences after transcatheter aortic valve replacement (TAVR) and surgical aortic valve replacement (SAVR).

Objective  To examine the utility of 30-day vs 90-day mortality after TAVR and SAVR as a mortality quality metric.

Design, Setting, and Participants  This nationally representative, multicenter, cohort study analyzed data from Medicare beneficiaries undergoing TAVR and SAVR procedures from January 1, 2012, to December 31, 2015. Concomitant coronary artery bypass grafting and other heart valve or other major open-heart procedures were excluded. Hospitals that performed fewer than 50 TAVR or 70 SAVR procedures per year were excluded to ensure reliable estimates and to reduce the risks of inflated results because of small institutional sample sizes. Data were analyzed from October 2018 to August 2019.

Exposures  Hospitals were ranked into top- (10%), middle- (80%), and bottom-performing (10%) groups based on their 4-year mean 30-day mortality.

Main Outcomes and Measures  Changes in hospital performance rankings at 90 days and 1 year and correlation of 30- and 90-day mortality with 1-year mortality were examined.

Results  A total of 30 329 TAVR admissions at 184 hospitals and 26 021 SAVR admissions at 191 hospitals were evaluated. For TAVR, 40 hospitals (21.7%) changed performance rankings at 90 days: 13 (48.1%) in the top-performing group and 8 (29.6%) in the bottom-performing group. At 1 year, 56 hospitals (30.4%), which included 21 (77.8%) in the top-performing group and 12 (44.4%) in the bottom-performing group, changed rankings. Similar findings were observed for SAVR, with an overall 90-day conversion rate of 17.3% and a 1-year rate of 30.3%. These findings persisted after adjusting for the differences in patient risk profiles among the 3 groups. Capturing 90-day events was also more robustly informative regarding expected 1-year outcomes after both TAVR and SAVR, largely owing to the observed plateau in the instantaneous hazard observed beyond this point.

Conclusions and Relevance  The findings suggest that evaluation of hospital performance based on 30-day mortality may underestimate outcomes and therefore substantially misrepresent institutional performance after TAVR and SAVR compared with 90-day mortality, even after risk adjustment. Although 30-day mortality has been validated, 90-day mortality may be a more reliable outcome metric for measuring hospital performance and capturing procedure-related mortality.

Introduction

Hospital performance metrics, such as 30-day mortality, are increasingly used in the context of value-based health care assessment and public reporting.1-3 These metrics provide useful benchmarking data to various stakeholders, including surgeons, patients, policy makers, and payers. For all surgical procedures, monitoring of 30-day outcomes is also mandated for reimbursements by insurance payers and is relatively cheaper and easier to implement from a hospital efficiency standpoint. However, with increasing scrutiny of hospital outcomes in the era of episode-based or bundled payment models implemented by the Centers for Medicare & Medicaid Services (CMS),4 there has been interest in tracking longer-term patient outcomes for a few reasons. First is that 30-day mortality may not capture all surgery- and non–surgery-related deaths.1-3 Second, there is a possibility that limiting follow-up to only 30 days may be associated with an increased risk of patients being unavailable for follow-up. Third, there is also some concern for potential perverse incentive to prolong the life of a dying patient past the 30-day barrier and to transfer the patient to another facility before the 30-day cutoff.5 All these instances can lead to underreporting of true mortality rates from the index procedures.6,7

Previous studies1-3,6,7 in noncardiac surgery have further challenged the validity and significance of 30-day mortality as a quality indicator of hospital outcomes. In particular, these studies1-3,6,7 have found that tracking only 30-day outcomes can lead to misclassification of the quality of a hospital’s postoperative care. A few studies3,6,8,9 have also found that 90-day mortality more accurately represented postoperative mortality because it captured more surgery-related deaths and was a better indicator of 1-year mortality.

In the context of the transcatheter aortic valve replacement (TAVR) procedure, data on the validity of 90-day outcomes is lacking. Questions have arisen as to whether 30-day mortality is a reliable metric for understanding institutional practice differences. In addition, the lower volume requirement in national coverage determination by CMS10 and the results of low-risk trials11,12 are expected to increase TAVR volume substantially. Therefore, establishing accurate quality-control metrics, especially for the newer adopters of TAVR, will be crucial to maintain the outcomes as the use of this procedure continues to expand. As such, we sought to assess the validity of 30-day vs 90-day mortality after TAVR. We used the surgical aortic valve replacement (SAVR) procedure as a reference. We hypothesized that 90-day postoperative mortality would be reliable, would capture a higher proportion of procedure-related deaths than 30-day mortality, and may be a superior measure of surgical quality in patients undergoing TAVR and SAVR.

Methods
Study Population

For this cohort study, we analyzed the Medicare Provider Analysis Review and Master Beneficiary Summary File data from January 1, 2012, to December 31, 2015, for all aortic valve replacement procedures. This study was approved by the Harvard Medical School Institutional Review Board. The Partners Health Care Institutional Review Board provided a waiver of informed consent. The CME data were not deidentified, but there are strict guidelines on data management.

Relevant International Classification of Diseases codes for SAVR and TAVR procedures are presented in eTable 1 in the Supplement. Records in which these procedures were associated with diagnosis related groups other than 216 to 221 or 306 and 307 were excluded from this study. The master file included a total of 250 877 unique claims. To facilitate longitudinal follow-up, individuals who had more than 1 month of health maintenance organization coverage were excluded, resulting in 208 400 unique claims of beneficiaries not enrolled in a health maintenance organization at any point during the study years. We also used the chronic conditions files, which provide comorbidities beyond those that may be coded for cardiac surgery billing, along with the date(s) first documented by CMS. This approach ensured that we would have full continuity of inpatient events for the patient set. Concomitant coronary artery bypass grafting and other heart valve or other major open-heart procedures were excluded, as were secondary and subsequent admissions for aortic valve replacement (eFigure 1 in the Supplement). To ensure reliable estimates and to reduce the risks of inflation of results because of small institutional sample sizes, a priori power calculations for repeated-measures McNemar tests were run to determine appropriate volume cutoffs as previously described,13 using assumptions from the observed mortality rates from prior TAVR studies.14-18 For the power calculations, we assumed baseline mortality of 3% for SAVR and 5% for TAVR to detect differences of a moderate effect size. Thus, patients from hospitals that performed fewer than 50 TAVR or 70 SAVR procedures per year were excluded on the basis of having too few patients to have a reliable positive predictive value.13

Data Collection and Definitions

Preoperative comorbidities and chronic conditions were derived from the chronic conditions file and by searching the coded diagnoses and procedures in the year before surgery. These coded variables had date of onset before the index admission and included acute myocardial infarction (if coded within the previous year), Alzheimer disease, atrial fibrillation, chronic kidney disease, chronic obstructive pulmonary disease, congestive heart failure, depression, diabetes, dyslipidemia, hip fracture (if coded within the previous year), hypertension, ischemic heart disease, osteoporosis, stroke, or transient ischemic attack. These diagnoses and procedures were chosen based on their comparability to the Society of Thoracic Surgeon’s (STS) risk factors,19 inclusion in the Charlson Comorbidity Index score,20 potential for confounding with our outcomes of interest, and clinical judgment (eTable 1 in the Supplement). All 25 diagnosis and procedure code fields were searched, with the switch field for present on admission indicating no or the date field indicating later than the date of index procedure. Bleeding complications included postprocedural bleeding or the presence of any of the following not listed as present on admission: hemorrhage, hematemesis, gastrointestinal bleeding, acute posthemorrhage anemia, adverse anticoagulation reaction (excluding heparin-induced thrombocytopenia), or hemorrhage due to anticoagulation, epistaxis, cardiovascular bleeding, or pulmonary bleeding or hemoptysis.

Outcomes of Interest

Our primary outcomes of interest included 30-day, 90-day, and 1-year mortality. Survival was calculated in days from the procedure date until death date or through December 31, 2017, if alive. Mortality data were derived from the National Death Index. Secondary outcomes included bleeding complications, acute kidney injury, permanent stroke, permanent pacemaker implantation, hospital length of stay, discharge to skilled nursing facility, and 30-day readmission.

Statistical Analysis

For comparative analysis, hospitals were ranked based on the mean 30-day mortality rate during the 4-year period into 3 groups: (1) top-performing (lowest 10% mortality rate), (2) middle-performing (middle 80% mortality rate), or (3) bottom-performing (highest 10% mortality) groups. The rationale for using these 3 hospital performance groups was based on existing rankings used by CMS and consumer reports and previous health care quality studies.2,21 The 4-year period was chosen to account for cumulative hospital experience. Changes in hospital performance rankings (categorized as improved, maintained, or declined) at 90 days and 1 year were evaluated. The SAVR group was used as a reference cohort. Separate Kaplan-Meier survival analyses were run for each procedure with pairwise and overall log-rank tests. To account for the differences in patient risk profile among the 3 groups, we built a hierarchical regression model that used CMS variables that were roughly analogous to many STS-predicted risk of mortality variables (except for creatinine level, ejection fraction, and degree of aortic stenosis) and included other variables that the STS does not account for, such as depression and rheumatoid arthritis. This model performed tolerably well, predicting 30-day mortality (area under curve, 0.720; adjusted R2 with observed mortality within each institution, 0.32).

To address the issue of whether 30- or 90-day mortality is a better indicator of procedure-related mortality vs mortality attributable to other factors and whether 1-year mortality is the most appropriate benchmark, we first examined the risk hazard function over time with time zero as the day of the procedure. Second, we examined causes of death for patients undergoing SAVR and TAVR at 30 days, 90 days, and 1 year. Cardiac procedure–related death was defined as heart failure, myocardial infarction, cardiac arrest, or death from cardiac surgery or malfunction of the valve. Cardiac non–procedure-related death was defined as anything the STS would code as a postoperative complication: renal or multisystem organ failure deaths in a person without preexisting diagnosis of major organ disease; stroke, coma, or thromboembolic events; sepsis or postoperative surgery–related infections; or unexpected bleeding or coagulopathy. Noncardiac causes of death included all other causes, such as trauma, unintentional death, gunshot wound, suicide, death attributable to preexisting noncardiac disease, and death after noncardiac procedure.

In addition, the partial correlations (controlling for institutional factors) between 30-day or 90-day mortality and 1-year mortality were examined. One-year survival was also examined using separate generalized linear models for each of 30-day and 90-day mortality (because they are not independent) with institution as a random effect, controlling for the covariates listed in Table 1. Shifting of hospitals within the 3 performance groups when moving from 30 to 90 days, especially shifting into and out of the smaller top- and bottom-performing categories, was further quantified through median shift in a continuous rank position. Continuous variables were tested for distribution and compared using analysis of variance for normally distributed variables or Kruskal-Wallis 1-way analysis of variance if nonnormally distributed and presented as mean (SD) or median (interquartile range) as appropriate. Binary variables are presented as number (percentage) and were compared using χ2 tests or Fisher exact test, depending on cell sizes. Hospitals with missing data were excluded from analysis. A 2-sided P ≤ .01 was the criterion of significance. All analyses were conducted using SPSS statistical software, version 23.0 (IBM Corp) or R, version 3.4.1 (R Foundation for Statistical Computing). Data were analyzed from October 2018 to August 2019.

Results
Characteristics and Outcomes of Hospital Performance Groups

A total of 30 329 TAVR admissions from 184 hospitals and 26 021 SAVR admissions from 191 hospitals were evaluated. For TAVR, there were 27 hospitals in the top-performing group, 130 in the middle-performing group, and 27 in the bottom-performing group. Baseline characteristics and outcomes of patients in the 3 performance groups are listed in Table 1. Compared with the bottom-performing group, patients undergoing TAVR in the top-performing group were significantly younger (mean [SD] age, 82.9 [7.5] years vs 83.4 [7.4] years; P < .001) with less burden of comorbidities, such as hypertension (2615 [95.9%] vs 2933 [97.2%]; P = .01) and atrial fibrillation (1008 [37.0%] vs 1226 [40.6%]; P = .005). Rates of cardiac arrest (56 [2.1%] in the top-performing hospitals, 584 [2.4%] in the middle-performing hospitals, and 118 [3.9%] in the bottom-performing hospitals; P = .001), acute kidney injury (273 [10.0%] in the top-performing hospitals, 2635 [10.7%] in the middle-performing hospitals, and 387 [12.8%] in the bottom-performing hospitals; P = .001), and transfusion requirements (321 [11.8%] in the top-performing hospitals, 4365 [17.8%] in the middle-performing hospitals, and 519 [17.2%] in the bottom-performing hospitals; P = .001) except for permanent pacemaker implantation (220 [81.%] in the top-performing hospitals, 1745 [7.1%] in the middle-performing hospitals, and 190 [6.3%] in the bottom-performing hospitals; P = .01) were significantly lower in the top-performing group. However, no significant differences were found in the rates of permanent stroke (89 [3.3%] in the top-performing hospitals, 736 [3.0%] in the middle-performing hospitals, and 115 [3.8%] in the bottom-performing hospitals; P = .25), bleeding complications (456 [16.7%] in the top-performing hospitals, 4326 [17.6%] in the middle-performing hospitals, and 525 [17.4%] in the bottom-performing hospitals; P = .505), or complete heart block (10 [0.4%] in the top-performing hospitals, 133 [0.5%] in the middle-performing hospitals, and 18 [0.6%] in the bottom-performing hospitals; P = .26). The bottom-performing group also had the highest rates of 30-day readmission (628 [20.8%]; P = .001) and 90-day (383 [12.7%]; P = .001) and 1-year (722 [23.9%]; P < .001) mortality. For SAVR (comparison cohort), a similar pattern of patient characteristics was observed. In terms of postoperative outcomes, rates in the bottom- vs top-performing groups were significantly higher for acute kidney injury (374 [13.9%] vs 318 [11.2%]; P = .003), permanent pacemaker implantation (140 [5.2%] vs 108 [3.8%]; P = .01), and cardiac arrest (72 [2.7%] vs 36 [1.3%]; P = .001) except for bleeding complications (635 [23.5%] vs 812 [28.5%]; P = .001) and permanent stroke (121 [4.5%] vs 126 [4.4%]; P = .95) (eTable 2 in the Supplement).

Changes in Hospital Performance Rankings at 90 Days and 1 Year

For TAVR, 40 hospitals (21.7%) changed performance rankings at 90 days: 13 (48.1%) in the top-performing group declined in their performance ranking, whereas 8 (29.6%) of the hospitals in the bottom-performing group improved their performance ranking (Figure 1A and C and Table 2). At 1 year, 56 hospitals (30.4%) changed performance rankings, including 21 (77.8%) top-performing and 12 (44.4%) bottom-performing hospitals. For SAVR, similar findings were observed with a 90-day conversion rate of 17.3% and a 1-year conversion rate of 30.3% (Figure 1B and D). After adjusting for patient risk profiles, these findings persisted for both TAVR and SAVR (eFigure 2 in the Supplement). Patient characteristics of hospitals that changed performance ranking at 90 days are highlighted in eTable 3 and eTable 4 in the Supplement.

When assessing median shift in continuous rank, we observed similar trends when examining the top- and bottom-performing groups. Specifically, 142 programs (37.9%) were within ±6 percentiles of their 30-day rankings at 90 days, 136 (36.3%) were within ±18 percentiles (ie, institutions at the 50th percentile at 30 days fell between the 32th and 68th percentile at 90 days), 59 (15.7%) decreased rank by more than 18 percentiles, and 38 (10.2%) increased rank by more than 18 percentiles. Thus, not only were 97 institutions in a clinically meaningful different rank at 90 days from their 30-day baseline, their mortality rankings were more likely to be substantially worse at 90 days than improved by a ratio of 3:2 (eTable 5 in the Supplement).

On Kaplan-Meier analyses, both 90-day survival (94% vs 91% vs 86% for TAVR and 99% vs 97% vs 93% for SAVR in the top-, middle-, and bottom-performing groups) and 1-year survival (87% vs 83% vs 76% for TAVR and 96% vs 94% vs 90% for SAVR in the top-, middle-, and bottom-performing groups) remained statistically significant among groups based on 30-day mortality performance (P < .001), with a heavy penalty on 1-year survival for the bottom-performing hospitals in both TAVR and SAVR (Figure 2A and B). The random effects from institutions was small in our generalized linear mixed models, and our findings remained robust.

Validity of 90-Day Mortality

For TAVR and SAVR, cardiac procedure–related deaths accounted for most deaths at 30 days, 90 days, and 1-year (eFigure 3 in the Supplement). However, hospitals in the top-performing group demonstrated a higher increase in mortality from 30 to 90 days compared with those in the middle- and bottom-performing groups (eTable 6 in the Supplement). For example, for TAVR, mean mortality at 90 days was 5.7% (relative increase, 5.9-fold) in the top-performing group, 7.4% (relative increase, 0.9-fold) in the middle-performing group, and 13.4% (relative increase, 0.5-fold) in the bottom-performing group. A similar pattern was observed for hospitals that performed SAVR.

An analysis of the instantaneous hazard found that, although the hazard (risk of mortality) diminished rapidly during the first 30-day period, it remained elevated through 90 days for the overall cohort (Figure 3A) and then plateaued to fairly constant levels thereafter during the first year for TAVR or SAVR (eFigure 4 in the Supplement). Thus, capturing 90-day events was more robustly informative regarding expected 1-year outcomes. In further support of the validity of the 90-day threshold, we examined readmission rates during the first postoperative year, with similar findings. The instantaneous hazard of a first readmission for the overall cohort persisted beyond the 30-day period but plateaued or diminished slowly beyond 90 days (Figure 3B). As a result, the 90-day metric captured an additional 23.5% of first readmissions such that 63.5% of first readmissions happened within 90 days. There was also only minimal change in time interval hazard ratios beyond 90 days (Figure 3C).

Discussion

Understanding the utility of 30-day vs 90-day mortality as a quality metric in the context of TAVR and SAVR appears to be essential given the close interaction between quality of patient care and hospital reimbursements.1-3 This nationally representative study had several key findings. First, it found a persistent hazard (risk of mortality) after TAVR and SAVR that remained elevated through 90 days but then plateaued to fairly constant levels in the first year after the procedure. Second, and most importantly, the 30-day mortality outcome metric underestimated procedure-related outcomes and therefore substantially misrepresented institutional performance after TAVR and SAVR compared with 90 days, even after risk adjustment. As a consequence, hospital performance rankings varied significantly over time for TAVR and SAVR, such that a significant proportion of hospitals changed performance rankings from 30 to 90 days. Third, capturing 90-day events was more robustly informative regarding expected 1-year outcomes for TAVR and SAVR. Together, these findings emphasize the need to reexamine the utility of 90-day mortality as a potentially valuable additional performance metric for TAVR and SAVR.

Existing studies1-3,6,7,22,23 examining the validity of 90-day mortality have been largely in noncardiac surgery. For instance, Talsma et al22 examined their single institution 30- and 90-day outcomes of patients undergoing esophagectomy between 1991 and 2011 and found that 30-day mortality had a 33% sensitivity and 100% specificity, whereas 90-day mortality had 74% sensitivity and 96% specificity in detecting surgery-related deaths. Adam et al2 further used the National Cancer Database to examine outcomes in more than 180 000 patients undergoing colorectal surgery and found that 90-day mortality was nearly double the 30-day mortality, and 30-day evaluation of hospital mortality misclassified at least 15% of the hospitals. In comparison, we also observed that mortality more than doubled within each performance group at 90 days, and 30-day evaluation of hospital mortality misclassified approximately 1 in 5 hospitals for TAVR and SAVR. For the top-performing group for instance, almost half of the hospitals declined in ranking for TAVR compared with 21% in the SAVR group at 90 days. At 1 year, more than 75% of hospitals that performed TAVR and more than 50% of hospitals that performed SAVR had declined in their performance rankings. Given the nature of our database, we could not determine the exact reasons that contributed to the decline in their performance rankings, although this warrants further research.

Our findings raise the question of whether 90-day mortality accurately captures hospital performance specific to TAVR and SAVR. We suspect that 90 days is a reliable metric for several reasons. First, the longer follow-up. Because TAVR and SAVR can impose significant physiologic stresses that may persist beyond the 30-day period in a population that is often also frail at baseline,5,24,25 assessing procedural mortality until the risk from the procedure itself is negligible may be the most relevant metric. Procedural mortality encompasses patient risk, case complexity, and technical skills of the treating team, including postprocedural care, whereas survival at 1 year is more dependent on how well you correct the aortic stenosis and the patient’s intrinsic risk level minus procedural mortality. Thus, intuitively, 90-day mortality may more closely reflect 1-year results than 30-day mortality because the extended period yields more information on what is likely to happen in 1 year. Although the same can be argued for 180 or 270 days, in our analysis, we found that little predictive information about 1-year mortality was gained beyond the 90-day period, likely because of the observed plateau in the instantaneous hazard observed beyond this point. Second, 90-day mortality may account for hospital-wide practice variations in terms of transferring patients to other health care facilities or back to referring hospitals. Thus, it is possible that hospitals with early discharge or transfer policies will have lower in-hospital mortality.26

Third, 90-day mortality may account for the persistent hazard of death that exists after TAVR and SAVR. This finding has been reported in several previous studies.24,25 For instance, Gaudiani et al24 analyzed the causes of death from the randomized CoreValve US Pivotal High-Risk trial and found that although the instantaneous risk of death for TAVR and SAVR decreased from the procedure to day 30, the SAVR risk increased between days 31 and 90, whereas the TAVR risk plateaued. This increased risk largely contributed to the superiority of 1-year TAVR outcomes and was likely associated with higher rates of technical failure, surgical complications, and lack of recovery from the surgical procedure.24 Similarly, in the randomized surgical replacement and transcatheter aortic valve implantation intermediate-risk trial, the hazard of death for both procedures decreased from procedure to day 30, but both remained flat thereafter such that most deaths occurring during the 30 days were attributable to TAVR complications and deaths occurring during 90 days were attributable to SAVR complications.25 The demographics and the observed persistent hazard of death after TAVR and SAVR in this current study were similar to those in the CoreValve high-risk trial, supporting the utility of 90-day mortality. Furthermore, because we still treat high-risk patients in both groups, 90-day mortality would help capture the true procedural mortality for comparison. As highlighted in our study, for both procedures, the 90-day metric captured an additional 40% of the baseline 30-day mortality, the predominance of which was cardiac procedure related. A similar pattern was observed when accounting for first readmission rates during the first postoperative year such that the 90-day metric captured an additional 23.5% of the baseline 30-day readmission.

Our findings have important implications for physicians, patients, and policy makers as we enter an era during which TAVR will become the preferred treatment of choice for symptomatic aortic stenosis. With the promising results of low-risk TAVR trials and subsequent approval by the US Food and Drug Administration,11,12 as well as the fewer volume requirements in the recent national coverage determination for TAVR,10 we expect a large increase in the number of TAVR procedures and number of new institutions adopting TAVR. This increase leads most to believe in the need to establish extensive quality control frameworks to measure hospital performance. The American College of Cardiology (ACC) had initiated certification this year, and the STS/ACC through the Transcatheter Valve Therapy (TVT) registry is planning to start public reporting of TAVR outcomes based on STS Star ratings in 2020. Since most newly eligible programs will likely be small-volume centers, ensuring accountability of TAVR outcomes will be pivotal.

Currently, the TVT registry, which captures TAVR data, mandates follow-up for 30 days and 1 year. However, the STS Adult Cardiac Surgery Database, which captures SAVR data, is limited to 30-day follow-up. Although there may be interest in tracking or benchmarking 1-year mortality, it may be limited for the following reasons. First, it would create additional cost and administrative burden to attempt 1-year follow-up of patients. Second, 1-year mortality may not provide timely enough data to allow hospitals to evaluate and develop quality improvement strategies. A 1-year lag in outcomes reporting may pose a challenge to registries trying to perform quality control, especially for the newer adopters of TAVR. Third, the higher rate of losing these patients during 1-year follow-up will not provide adequate analysis for the quality control. In light of the above limitations, the added value of tracking 90-day outcomes for TAVR may be justifiable. Of note, CMS is also trialing 90-day bundles for certain procedures, and it appears to be moving in the direction of tracking 90-day outcomes for most common procedures, which TAVR will soon be.

Limitations

This study has limitations. The CMS database is a hospital claims database without access to individual medical records and is subject to the shortcomings of other administrative data sets. Inconsistencies related to coding may overestimate or underestimate our findings, although we adjusted for several confounders in our generalized linear models. We were unable to perform root cause analysis or model predictors of hospitals for changing rankings from 30 to 90 days because of low numbers of institutions. We were also unable to account for patients who switched hospitals for their care either as transfers or as new readmissions. The CMS database also precludes detailed assessment of patient presentation, procedural and echocardiographic details, and STS risk scores. Given the nature of the database, formal assessment of frailty and quality-of-life variables was not possible. Surrogate measures, such as Charlson Comorbidity Index scores, were used. Furthermore, although our analysis was based on a large number of patients, the results cannot be extrapolated to those who did not meet our inclusion criteria (eg, health maintenance organization patients, those undergoing concomitant procedures, and younger or privately insured patients). Given that the study period ended in 2015 and that TAVR is a dynamically changing field, it is possible that our findings may not adequately reflect current clinical practice and outcomes. Thus, our findings must be interpreted with context.

Conclusions

The findings suggest that evaluation of hospital performance based on 30-day mortality may underestimate outcomes and therefore substantially misrepresent institutional performance after TAVR and SAVR compared with 90-day mortality, even after risk adjustment. Although 30-day mortality was validated in our study, 90-day mortality may be a reliable reporting outcome metric for measuring hospital performance and capturing procedure-related mortality in the future when there is emphasis on quality control.

Back to top
Article Information

Accepted for Publication: September 22, 2019.

Corresponding Author: Tsuyoshi Kaneko, MD, Division of Cardiac Surgery, Brigham and Women’s Hospital, Harvard Medical School, 15 Francis St, Boston, MA 02115 (tkaneko2@partners.org).

Published Online: December 18, 2019. doi:10.1001/jamacardio.2019.4657

Author Contributions: Ms McGurk and Dr Kaneko had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Hirji, Kiehm, Ejiofor, Ramirez-Del Val, Berry, Pelletier, Shah, O'Gara, Kaneko.

Acquisition, analysis, or interpretation of data: Hirji, McGurk, Ejiofor, Kolkailah, Sobieszczyk, O'Gara, Kaneko.

Drafting of the manuscript: Hirji, Kiehm, Ejiofor, Kolkailah, Kaneko.

Critical revision of the manuscript for important intellectual content: Hirji, McGurk, Ejiofor, Ramirez-Del Val, Berry, Sobieszczyk, Pelletier, Shah, O'Gara, Kaneko.

Statistical analysis: Hirji, McGurk, Ramirez-Del Val, Kolkailah.

Administrative, technical, or material support: McGurk, Ejiofor, Berry, Sobieszczyk, Pelletier, Kaneko.

Supervision: Ramirez-Del Val, Sobieszczyk, Pelletier, O'Gara, Kaneko.

Conflict of Interest Disclosures: Dr Kaneko reports being a speaker for Edwards Life Sciences and Medtronic. None other disclosures were reported.

Funding/Support: This work was supported by Levinger gift funds from Brigham and Women’s Hospital, Harvard Medical School.

Role of the Funder/Sponsor: The study sponsor provided support with acquisition of data and was notified of this project. They approved the work conducted.

Meeting Presentation: This work was presented as the Adult Cardiac Plenary Presentation at the 99th Annual Meeting of The American Association of Thoracic Surgery; May 5, 2019; Toronto, Ontario, Canada.

Additional Contributions: The following individuals provided statistical guidance and assistance during the revision phase: E. Francis Cook, ScD, Harvard School of Public Health, and Cheryl Zogg, MSPH, MHS, Yale School of Medicine. They were not compensated for their work.

Additional Information: This project received the C. Walton Lillehei Award in Adult Cardiac Surgery.

References
1.
Bryant  AS, Rudemiller  K, Cerfolio  RJ.  The 30- versus 90-day operative mortality after pulmonary resection.  Ann Thorac Surg. 2010;89(6):1717-1722. doi:10.1016/j.athoracsur.2010.01.069PubMedGoogle ScholarCrossref
2.
Adam  MA, Turner  MC, Sun  Z,  et al.  The appropriateness of 30-day mortality as a quality metric in colorectal cancer surgery.  Am J Surg. 2018;215(1):66-70. doi:10.1016/j.amjsurg.2017.04.018PubMedGoogle ScholarCrossref
3.
Mizushima  T, Yamamoto  H, Marubashi  S,  et al.  Validity and significance of 30-day mortality rate as a quality indicator for gastrointestinal cancer surgeries.  Ann Gastroenterol Surg. 2018;2(3):231-240. doi:10.1002/ags3.12070PubMedGoogle ScholarCrossref
4.
Koeckert  MS, Grossi  EA, Vining  PF,  et al.  Ninety-day readmissions of bundled valve patients: implications for healthcare policy.  Semin Thorac Cardiovasc Surg. 2019;31(1):32-37. doi:10.1053/j.semtcvs.2018.07.017PubMedGoogle ScholarCrossref
5.
McMillan  RR, Berger  A, Sima  CS,  et al.  Thirty-day mortality underestimates the risk of early death after major resections for thoracic malignancies.  Ann Thorac Surg. 2014;98(5):1769-1774. doi:10.1016/j.athoracsur.2014.06.024PubMedGoogle ScholarCrossref
6.
Pezzi  CM, Mallin  K, Mendez  AS, Greer Gay  E, Putnam  JB  Jr.  Ninety-day mortality after resection for lung cancer is nearly double 30-day mortality.  J Thorac Cardiovasc Surg. 2014;148(5):2269-2277. doi:10.1016/j.jtcvs.2014.07.077PubMedGoogle ScholarCrossref
7.
Damhuis  RA, Wijnhoven  BP, Plaisier  PW, Kirkels  WJ, Kranse  R, van Lanschot  JJ.  Comparison of 30-day, 90-day and in-hospital postoperative mortality for eight different cancer types.  Br J Surg. 2012;99(8):1149-1154. doi:10.1002/bjs.8813PubMedGoogle ScholarCrossref
8.
Mise  Y, Vauthey  JN, Zimmitti  G,  et al.  Ninety-day postoperative mortality is a legitimate measure of hepatopancreatobiliary surgical quality.  Ann Surg. 2015;262(6):1071-1078. doi:10.1097/SLA.0000000000001048PubMedGoogle ScholarCrossref
9.
Swanson  RS, Pezzi  CM, Mallin  K, Loomis  AM, Winchester  DP.  The 90-day mortality after pancreatectomy for cancer is double the 30-day mortality: more than 20,000 resections from the national cancer data base.  Ann Surg Oncol. 2014;21(13):4059-4067. doi:10.1245/s10434-014-4036-4PubMedGoogle ScholarCrossref
10.
Centers for Medicare & Medicaid Services. Proposed Decision Memo for Transcatheter Aortic Valve Replacement (TAVR). https://www.cms.gov/medicare-coverage-database/details/nca-proposed-decision-memo.aspx?NCAId=293. Accessed April 12, 2019.
11.
Mack  MJ, Leon  MB, Thourani  VH,  et al; PARTNER 3 Investigators.  Transcatheter aortic-valve replacement with a balloon-expandable valve in low-risk patients.  N Engl J Med. 2019;380(18):1695-1705. doi:10.1056/NEJMoa1814052PubMedGoogle ScholarCrossref
12.
Popma  JJ, Deeb  GM, Yakubov  SJ,  et al; Evolut Low Risk Trial Investigators.  Transcatheter aortic-valve replacement with a self-expanding valve in low-risk patients.  N Engl J Med. 2019;380(18):1706-1715. doi:10.1056/NEJMoa1816885PubMedGoogle ScholarCrossref
13.
Button  KS, Ioannidis  JP, Mokrysz  C,  et al.  Power failure: why small sample size undermines the reliability of neuroscience.  Nat Rev Neurosci. 2013;14(5):365-376. doi:10.1038/nrn3475PubMedGoogle ScholarCrossref
14.
Mack  MJ, Leon  MB, Smith  CR,  et al; PARTNER 1 trial investigators.  5-year outcomes of transcatheter aortic valve replacement or surgical aortic valve replacement for high surgical risk patients with aortic stenosis (PARTNER 1): a randomised controlled trial.  Lancet. 2015;385(9986):2477-2484. doi:10.1016/S0140-6736(15)60308-7PubMedGoogle ScholarCrossref
15.
Leon  MB, Smith  CR, Mack  M,  et al; PARTNER Trial Investigators.  Transcatheter aortic-valve implantation for aortic stenosis in patients who cannot undergo surgery.  N Engl J Med. 2010;363(17):1597-1607. doi:10.1056/NEJMoa1008232PubMedGoogle ScholarCrossref
16.
Kodali  SK, Williams  MR, Smith  CR,  et al; PARTNER Trial Investigators.  Two-year outcomes after transcatheter or surgical aortic-valve replacement.  N Engl J Med. 2012;366(18):1686-1695. doi:10.1056/NEJMoa1200384PubMedGoogle ScholarCrossref
17.
Adams  DH, Popma  JJ, Reardon  MJ,  et al; U.S. CoreValve Clinical Investigators.  Transcatheter aortic-valve replacement with a self-expanding prosthesis.  N Engl J Med. 2014;370(19):1790-1798. doi:10.1056/NEJMoa1400590PubMedGoogle ScholarCrossref
18.
Gleason  TG, Reardon  MJ, Popma  JJ,  et al; CoreValve U.S. Pivotal High Risk Trial Clinical Investigators.  5-Year outcomes of self-expanding transcatheter versus surgical aortic valve replacement in high-risk patients.  J Am Coll Cardiol. 2018;72(22):2687-2696. doi:10.1016/j.jacc.2018.08.2146PubMedGoogle ScholarCrossref
19.
O’Brien  SM, Shahian  DM, Filardo  G,  et al; Society of Thoracic Surgeons Quality Measurement Task Force.  The Society of Thoracic Surgeons 2008 cardiac surgery risk models, part 2: isolated valve surgery.  Ann Thorac Surg. 2009;88(1)(suppl):S23-S42. doi:10.1016/j.athoracsur.2009.05.056PubMedGoogle ScholarCrossref
20.
Quan  H, Li  B, Couris  CM,  et al.  Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.  Am J Epidemiol. 2011;173(6):676-682. doi:10.1093/aje/kwq433PubMedGoogle ScholarCrossref
21.
Teixeira-Pinto  A, Normand  SL.  Statistical methodology for classifying units on the basis of multiple-related measures.  Stat Med. 2008;27(9):1329-1350. doi:10.1002/sim.3187PubMedGoogle ScholarCrossref
22.
Talsma  AK, Lingsma  HF, Steyerberg  EW, Wijnhoven  BP, Van Lanschot  JJ.  The 30-day versus in-hospital and 90-day mortality after esophagectomy as indicators for quality of care.  Ann Surg. 2014;260(2):267-273. doi:10.1097/SLA.0000000000000482PubMedGoogle ScholarCrossref
23.
In  H, Palis  BE, Merkow  RP,  et al.  Doubling of 30-day mortality by 90 days after esophagectomy: a critical measure of outcomes for quality improvement.  Ann Surg. 2016;263(2):286-291. doi:10.1097/SLA.0000000000001215PubMedGoogle ScholarCrossref
24.
Gaudiani  V, Deeb  GM, Popma  JJ,  et al.  Causes of death from the randomized CoreValve US Pivotal High-Risk Trial.  J Thorac Cardiovasc Surg. 2017;153(6):1293-1301.e1. doi:10.1016/j.jtcvs.2016.11.069PubMedGoogle ScholarCrossref
25.
Amrane  H, Deeb  GM, Popma  JJ,  et al; SURTAVI Trial Causes of Death Working Group.  Causes of death in intermediate-risk patients: The Randomized Surgical Replacement and Transcatheter Aortic Valve Implantation Trial.  J Thorac Cardiovasc Surg. 2019;158(3):718-728.e3. doi:10.1016/j.jtcvs.2018.11.129PubMedGoogle ScholarCrossref
26.
Siregar  S, Groenwold  RH, de Mol  BA,  et al.  Evaluation of cardiac surgery mortality rates: 30-day mortality or longer follow-up?  Eur J Cardiothorac Surg. 2013;44(5):875-883. doi:10.1093/ejcts/ezt119PubMedGoogle ScholarCrossref
×