Performance of Hospitals When Assessing Disease-Based Mortality Compared With Procedural Mortality for Patients With Acute Myocardial Infarction | Acute Coronary Syndromes | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Funnel Plots of Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)
Funnel Plots of Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)

A, Funnel plot of hospital-level EMR-D from the National Cardiovascular Data Registry (NCDR) Chest Pain–MI Registry for all participating hospitals. B, Funnel plot of hospital-level EMR-P from the NCDR CathPCI Registry for all participating hospitals. Each dot represents a unique hospital. The central dotted light blue line is the mean, the gray lines represent 1.96 SDs around the mean, and the orange lines represent 3 SDs around the mean.

Figure 2.  Correlation Between Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)
Correlation Between Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)

Plot of EMR-D and EMR-P among 625 hospitals with linked data available. Each dot represents a unique hospital. Spearman rank correlation coefficient between EMR-P and EMR-D produced a ρ of 0.53 (95% CI, 0.47-0.58).

Figure 3.  Mean Differences Between Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)
Mean Differences Between Excess Mortality Ratio for Acute Myocardial Infarction as a Disease (EMR-D) and Excess Mortality Ratio for Percutaneous Coronary Intervention (EMR-P)

A, All patients (1-sample t test comparing the mean with zero; P < .001). B, Patients with ST-segment elevation myocardial infarction (STEMI) (1-sample t test comparing the mean with zero; P = .01). C, Patients with all acute myocardial infarction (AMI) complicated by cardiogenic shock or cardiac arrest (1-sample t test comparing the mean with zero, P < .001). The mean is solid blue line in the center, and the 95% CIs are delineated by the top and bottom dotted blue lines.

Table 1.  Baseline Characteristics for the Overall Population in Each NCDR Registrya
Baseline Characteristics for the Overall Population in Each NCDR Registrya
Table 2.  Tertile Ranking of Hospitals by Risk-Adjusted Mortality to Identify Reclassificationa
Tertile Ranking of Hospitals by Risk-Adjusted Mortality to Identify Reclassificationa
1.
Hannan  EL.  Using mortality data for profiling hospital quality of care and targeting substandard care.   J Soc Health Syst. 1989;1(1):31-48.PubMedGoogle Scholar
2.
Shahian  DM, Normand  SL.  Comparison of “risk-adjusted” hospital outcomes.   Circulation. 2008;117(15):1955-1963. doi:10.1161/CIRCULATIONAHA.107.747873 PubMedGoogle ScholarCrossref
3.
Wasfy  JH, Borden  WB, Secemsky  EA, McCabe  JM, Yeh  RW.  Public reporting in cardiovascular medicine: accountability, unintended consequences, and promise for improvement.   Circulation. 2015;131(17):1518-1527. doi:10.1161/CIRCULATIONAHA.114.014118 PubMedGoogle ScholarCrossref
4.
Wadhera  RK, Bhatt  DL.  Taking the “public” out of public reporting of percutaneous coronary intervention.   JAMA. 2017;318(15):1439-1440. doi:10.1001/jama.2017.12087 PubMedGoogle ScholarCrossref
5.
Wadhera  RK, Joynt Maddox  KE, Yeh  RW, Bhatt  DL.  Public reporting of percutaneous coronary intervention outcomes: moving beyond the status quo.   JAMA Cardiol. 2018;3(7):635-640. doi:10.1001/jamacardio.2018.0947 PubMedGoogle ScholarCrossref
6.
Wadhera  RK, Bhatt  DL.  Toward precision policy—the case of cardiovascular care.   N Engl J Med. 2018;379(23):2193-2195. doi:10.1056/NEJMp1806260 PubMedGoogle ScholarCrossref
7.
Harder  B, Comarow  A.  Hospital quality reporting by US News & World Report: why, how, and what’s ahead.   JAMA. 2015;313(19):1903-1904. doi:10.1001/jama.2015.4566 PubMedGoogle ScholarCrossref
8.
Duffy  PL, Naidu  SS, Kurian  KC, Marshall  JJ.  Rationale for ST-elevation myocardial infarction call compensation by healthcare systems.   Catheter Cardiovasc Interv. 2013;81(6):909-911. doi:10.1002/ccd.24921 PubMedGoogle ScholarCrossref
9.
Peterson  ED, Roe  MT, Rumsfeld  JS,  et al.  A call to ACTION (Acute Coronary Treatment and Intervention Outcomes Network): a national effort to promote timely clinical feedback and support continuous quality improvement for acute myocardial infarction.   Circ Cardiovasc Qual Outcomes. 2009;2(5):491-499. doi:10.1161/CIRCOUTCOMES.108.847145 PubMedGoogle ScholarCrossref
10.
Brindis  RG, Fitzgerald  S, Anderson  HV, Shaw  RE, Weintraub  WS, Williams  JF.  The American College of Cardiology–National Cardiovascular Data Registry (ACC-NCDR): building a national clinical data repository.   J Am Coll Cardiol. 2001;37(8):2240-2245. doi:10.1016/S0735-1097(01)01372-9 PubMedGoogle ScholarCrossref
11.
Chin  CT, Chen  AY, Wang  TY,  et al.  Risk adjustment for in-hospital mortality of contemporary patients with acute myocardial infarction: the Acute Coronary Treatment and Intervention Outcomes Network (ACTION) registry–Get With The Guidelines (GWTG) acute myocardial infarction mortality model and risk score.   Am Heart J. 2011;161(1):113-122.e2. doi:10.1016/j.ahj.2010.10.004 PubMedGoogle ScholarCrossref
12.
Brennan  JM, Curtis  JP, Dai  D,  et al; National Cardiovascular Data Registry.  Enhanced mortality risk prediction with a focus on high-risk percutaneous coronary intervention: results from 1,208,137 procedures in the NCDR (National Cardiovascular Data Registry).   JACC Cardiovasc Interv. 2013;6(8):790-799. doi:10.1016/j.jcin.2013.03.020 PubMedGoogle ScholarCrossref
13.
Doll  JA, Dai  D, Roe  MT,  et al.  Assessment of operator variability in risk-standardized mortality following percutaneous coronary intervention: a report from the NCDR.   JACC Cardiovasc Interv. 2017;10(7):672-682. doi:10.1016/j.jcin.2016.12.019 PubMedGoogle ScholarCrossref
14.
Bland  JM, Altman  DG.  Statistical methods for assessing agreement between two methods of clinical measurement.   Lancet. 1986;1(8476):307-310. doi:10.1016/S0140-6736(86)90837-8 PubMedGoogle ScholarCrossref
15.
Fernandez  G, Narins  CR, Bruckel  J, Ayers  B, Ling  FS.  Patient and physician perspectives on public reporting of mortality ratings for percutaneous coronary intervention in New York state.   Circ Cardiovasc Qual Outcomes. 2017;10(9):e003511. doi:10.1161/CIRCOUTCOMES.116.003511 PubMedGoogle Scholar
16.
Blumenthal  DM, Valsdottir  LR, Zhao  Y,  et al.  A survey of interventional cardiologists’ attitudes and beliefs about public reporting of percutaneous coronary intervention.   JAMA Cardiol. 2018;3(7):629-634. doi:10.1001/jamacardio.2018.1095 PubMedGoogle ScholarCrossref
17.
Hochman  JS, Sleeper  LA, White  HD,  et al; SHOCK Investigators. Should We Emergently Revascularize Occluded Coronaries for Cardiogenic Shock.  One-year survival following early revascularization for cardiogenic shock.   JAMA. 2001;285(2):190-192. doi:10.1001/jama.285.2.190 PubMedGoogle ScholarCrossref
18.
Werner  RM, Asch  DA, Polsky  D.  Racial profiling: the unintended consequences of coronary artery bypass graft report cards.   Circulation. 2005;111(10):1257-1263. doi:10.1161/01.CIR.0000157729.59754.09 PubMedGoogle ScholarCrossref
19.
Nathan  AS, Khatana  SAM, Yeh  RW, Groeneveld  PW, Giri  J.  Hospital-specific mortality for acute myocardial infarction versus emergency percutaneous coronary intervention in New York state.   JACC Cardiovasc Interv. 2019;12(9):898-899. doi:10.1016/j.jcin.2019.02.045 PubMedGoogle ScholarCrossref
20.
Altman  DG, Bland  JM.  Measurement in medicine: the analysis of method comparison studies.   Statistician. 1983;32:307-317. doi:10.2307/2987937 Google ScholarCrossref
21.
Lemkes  JS, Janssens  GN, van der Hoeven  NW,  et al.  Coronary angiography after cardiac arrest without ST-segment elevation.   N Engl J Med. 2019;380(15):1397-1407. doi:10.1056/NEJMoa1816897 PubMedGoogle ScholarCrossref
22.
McCabe  JM, Waldo  SW, Kennedy  KF, Yeh  RW.  Treatment and outcomes of acute myocardial infarction complicated by shock after public reporting policy changes in New York.   JAMA Cardiol. 2016;1(6):648-654. doi:10.1001/jamacardio.2016.1806 PubMedGoogle ScholarCrossref
23.
Nathan  AS, Shah  RM, Khatana  SA,  et al.  Effect of public reporting on the utilization of coronary angiography after out-of-hospital cardiac arrest.   Circ Cardiovasc Interv. 2019;12(4):e007564. doi:10.1161/CIRCINTERVENTIONS.118.007564 PubMedGoogle Scholar
Original Investigation
April 29, 2020

Performance of Hospitals When Assessing Disease-Based Mortality Compared With Procedural Mortality for Patients With Acute Myocardial Infarction

Author Affiliations
  • 1Cardiovascular Division, Hospital of the University of Pennsylvania, Philadelphia
  • 2Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia
  • 3Penn Cardiovascular Outcomes, Quality, and Evaluative Research Center, University of Pennsylvania, Philadelphia
  • 4Duke Clinical Research Institute, Duke University Medical Center, Durham, North Carolina
  • 5Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
  • 6Cardiovascular Division, Brigham and Women’s Hospital, Boston, Massachusetts
  • 7Richard and Susan Smith Center for Outcomes Research in Cardiology, Beth Israel Deaconess Medical Center, Boston, Massachusetts
  • 8Cardiovascular Division, Columbia–New York Presbyterian Hospital, New York, New York
  • 9Division of General Internal Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
JAMA Cardiol. 2020;5(7):765-772. doi:10.1001/jamacardio.2020.0753
Key Points

Question  How does hospital performance for patients with acute myocardial infarction differ when assessed by disease-based mortality compared with procedural mortality for percutaneous coronary intervention?

Findings  This cross-sectional study of 776 890 patients found moderate correlation between hospital-level disease-based mortality and procedural mortality, with considerable reclassification of hospital performance between the 2 measures. Based on observed disease-based mortality and procedural mortality statistics, potential risk-avoidant behavior was identified among treatment decisions for the highest-risk patients with cardiogenic shock or cardiac arrest.

Meaning  Percutaneous coronary intervention procedural mortality in isolation may not accurately reflect the quality of cardiovascular care delivered by hospitals for patients with acute myocardial infarction.

Abstract

Importance  Quality of percutaneous coronary intervention (PCI) is commonly assessed by risk-adjusted mortality. However, this metric may result in procedural risk aversion, especially for high-risk patients.

Objective  To determine correlation and reclassification between hospital-level disease-specific mortality and PCI procedural mortality among patients with acute myocardial infarction (AMI).

Design, Setting, and Participants  This hospital-level observational cross-sectional multicenter analysis included hospitals participating in the Chest Pain–MI Registry, which enrolled consecutive adult patients admitted with a diagnosis of type I non–ST-segment elevation myocardial infarction (NSTEMI) or ST-segment elevation myocardial infarction (STEMI), and hospitals in the CathPCI Registry, which enrolled consecutive adult patients treated with PCI with an indication of NSTEMI or STEMI, between April 1, 2011, and December 31, 2017.

Exposures  Inclusion into the National Cardiovascular Data Registry Chest Pain–MI and CathPCI registries.

Main Outcomes and Measures  For each hospital in each registry, a disease-based excess mortality ratio (EMR-D) for AMI was calculated, which represents a risk-adjusted observed to expected rate of mortality for AMI as a disease using the Chest Pain–MI Registry, and a procedure-based excess mortality ratio (EMR-P) for PCI was calculated using the CathPCI Registry.

Results  A subset of 625 sites participated in both registries, with a final count of 776 890 patients from the Chest Pain–MI Registry (509 576 men [65.6%]; 620 981 white [80.0%]; and median age, 64 years [interquartile range, 55-74 years]) and 853 386 patients from the CathPCI Registry (582 701 men [68.3%]; 691 236 white [81.0%]; and median age, 63 years [interquartile range, 54-73 years]). Among the 625 linked hospitals, the Spearman rank correlation coefficient between EMR-D and EMR-P produced a ρ of 0.53 (95% CI, 0.47-0.58), suggesting moderate correlation. Among the highest-performing tertile for disease-based risk-adjusted mortality, 90 of 208 sites (43.3%) were classified into a lower category for procedural risk-adjusted mortality. Among the lowest-performing tertile for disease-based risk-adjusted mortality, 92 of 208 sites (44.2%) were classified into a higher category for procedural risk-adjusted mortality. Bland-Altman plots for the overall linked cohort demonstrate a mean difference between EMR-P and EMR-D of 0.49% (95% CI, –1.61% to 2.58%; P < .001), with procedural mortality higher than disease-based mortality. However, among patients with AMI complicated by cardiogenic shock or cardiac arrest, the mean difference between EMR-P and EMR-D was –0.64% (95% CI, –4.41% to 3.12%; P < .001), with procedural mortality lower than disease-based mortality.

Conclusions and Relevance  This study suggests that, for hospitals treating patients with AMI, there is only a moderate correlation between procedural outcomes and disease-based outcomes. Nearly half of hospitals in the highest tertile of performance for PCI performance were reclassified into a lower performance tertile when judged by disease-based metrics. Higher rates of mortality were observed when using disease-based metrics compared with procedural metrics when assessing patients with cardiogenic shock and/or cardiac arrest, signifying what appears to be potential risk avoidance among this highest-risk subset of patients.

Introduction

Several state and national registries collect and publicly disclose information on percutaneous coronary intervention (PCI) performance measures for individual interventional cardiologists and hospitals. One of the most widely used metrics to assess quality is the risk-adjusted mortality rate after PCI.1

However, interventional cardiologists and hospitals have voiced concerns that risk-adjustment models for PCI mortality may not adequately account for high-risk clinical features, and as a result, they think that treating high-risk patients may result in worse calculated PCI-associated mortality rates.2 Recently, there have been proposals to reduce operator risk aversion toward critically ill patients with acute myocardial infarction (AMI) by assessing disease-specific mortality as opposed to procedural mortality.3-6 Although PCI is an important tool in the treatment of AMI, the care of patients with AMI requires a multidisciplinary approach, including timely and accurate diagnosis by emergency department professionals and cardiologists, initiation of evidence-based treatments, and proper transition of care to the outpatient setting. Therefore, a disease-based mortality metric may better reflect the quality of care delivered across a care episode for patients with AMI.

Efforts to publicly report outcomes appear to be gaining traction. As these initiatives have focused primarily on mortality after PCI, there is growing concern that they may impede access to care for critically ill patients.7 Empirical data supporting the use of disease-based measures instead may help mitigate these unintended consequences. Both procedure-based and disease-based metrics are being used to assess the quality of a cardiovascular program by the National Cardiovascular Data Registry (NCDR), which provides data reports to hospitals and health systems, as well as health insurance payers.8 As reimbursement is increasingly tied to outcomes under value-based payment models, it will be important for stakeholders to understand which of these metrics best reflects the quality of care delivered by clinicians and hospitals.

In this study, we used the NCDR Chest Pain MI Registry, which captures admissions for ST-segment elevation myocardial infarction (STEMI) and non–ST-elevation myocardial infarction (NSTEMI), and the NCDR CathPCI Registry, which captures all patients undergoing PCI, including those with STEMI and NSTEMI. This study addressed 2 questions. First, what is the hospital-level correlation between disease-specific mortality and PCI procedural mortality among patients with AMI at sites participating in both registries? Second, is there evidence of reclassification when assessing hospital-level disease-specific mortality compared with procedural mortality in patients with AMI?

Methods

The data, analytical methods, and study materials will not be made available to other researchers. Informed consent was waived for this study by the Advarra Institutional Review Board. All human participant research was conducted in compliance with the Common Rule (45 CFR 46). The protocol and details are available in the eAppendix in the Supplement.

Study Sample

The NCDR Chest Pain–MI Registry is a national quality improvement database that collects standardized information on all patients who present to participating hospitals with NSTEMI or STEMI, and the NCDR CathPCI Registry collects data on all patients who undergo PCI at participating hospitals. Both registries capture patient and hospital characteristics, including clinical features at presentation, procedural details, medications, and in-hospital outcomes.9,10

We identified all hospitals capable of performing PCI that participated in the NCDR Chest Pain–MI and CathPCI registries between April 1, 2011, and December 31, 2017. Hospitals in the Chest Pain–MI Registry included consecutive adult patients (age, >18 years) admitted to that hospital with a diagnosis of type I NSTEMI or STEMI, regardless of PCI status; hospitals in the CathPCI Registry included consecutive adult patients (age, >18 years) treated with PCI with an indication of NSTEMI or STEMI (on or during hospital admission) for the procedure. We excluded hospitals that contributed data for fewer than 8 consecutive quarters and that had a mean of fewer than 40 patients per year of participation in either registry. Furthermore, we excluded any patients who were transferred from one acute care facility to another to accurately assess mortality outcomes.

Outcome Measures

The primary outcomes of interest were the excess mortality ratios of patients with NSTEMI or STEMI in each hospital that participated in both the NCDR Chest Pain–MI and CathPCI registries, calculated using previously validated in-hospital mortality models developed for each registry. For patients in the Chest Pain–MI Registry, the variables used for adjustment in the model were age, serum creatinine level, systolic blood pressure, troponin ratio, presentation with heart failure or cardiogenic shock, ST-segment changes, heart rate, and peripheral artery disease.11 For patients in the CathPCI Registry, the variables used for adjustment in the model were presence of STEMI, age, body mass index, cerebrovascular disease, peripheral artery disease, chronic lung disease, prior PCI, diabetes, glomerular filtration rate, renal failure, ejection fraction, cardiogenic shock, PCI urgency, New York Heart Association functional classification within 2 weeks of PCI, cardiac arrest within 24 hours, at least 1 previously treated lesion within 1 month with in-stent thrombosis, left main or proximal left anterior descending coronary artery involvement, number of diseased vessels, and chronic total occlusion.12 The expected mortality models are expected to be colinear for individual patients given the significant overlap in variables.

For each registry, the hospital-level estimated mortality rate was calculated using a hierarchical logistic regression model with fixed effects and a random hospital-specific intercept for included patients. A hospital’s risk-adjusted mortality rate was calculated as the ratio of the observed death rate in the given hospital and the estimated mortality rate for the given hospital multiplied by the overall observed mortality rate in the registry. This calculation resulted in an excess mortality ratio, which represents a risk-adjusted observed to expected rate of mortality for AMI as a disease using the Chest Pain–MI Registry (EMR-D), and an excess mortality ratio for PCI using the CathPCI Registry (EMR-P).

Statistical Analysis

Summary statistics for patient characteristics from each registry are presented as means with SDs for continuous data and total number and percentages for categorical data. The patients in the 2 registries are not exclusive to each other and direct comparisons were not made between the 2 groups at the patient level.

Funnel plots were generated to assess the stability of the estimates of EMR-D and EMR-P and understand the volume-outcome association by plotting EMR-D and EMR-P against the total number of patients at the participating hospitals in each registry. These plots have been previously used to assess the stability of estimates for operator-level PCI mortality.13

Spearman rank correlation coefficient was used to assess the correlation between EMR-D and EMR-P for each hospital. We divided the ranks of hospitals by disease-based risk-adjusted mortality and procedural risk-adjusted mortality into tertiles and, given the ordinal nature of the rows and columns, compared hospital categories for procedural risk-adjusted mortality with disease-based risk-adjusted mortality using the nonzero correlation test.

Finally, we assessed agreement between EMR-P and EMR-D by generating Bland-Altman plots.14 For each hospital, we plotted the mean of EMR-D and EMR-P along the x-axis and the difference between EMR-P and EMR-D along the y-axis. We then calculated the 95% limits of agreement and performed a 1-sample t test to identify evidence of fixed bias. Predefined subgroups for analysis included STEMI alone and NSTEMI or STEMI complicated by cardiogenic shock or cardiac arrest.

All analyses were performed by the Duke Clinical Research Institute using SAS, version 9.3 (SAS Institute Inc). All statistical testing was 2-tailed, with P < .05 designated statistically significant.

Results

Between April 1, 2011, and December 31, 2017, there were 831 178 patients with STEMI or NSTEMI from 876 sites initially identified from the Chest Pain–MI Registry, and 1 852 734 patients treated with PCI for STEMI or NSTEMI from 1758 sites from the CathPCI Registry (eTable 1 in the Supplement). There were 32 569 patients in the Chest Pain–MI Registry and 35 002 patients in the CathPCI Registry excluded owing to interhospital transfer. After restricting the cohort to hospitals that contributed at least 8 consecutive quarters of data and had at least 40 patients per year on average in each registry, 802 438 patients from 679 sites remained from the Chest Pain–MI Registry and 1 824 837 patients from 1451 sites remained from the CathPCI Registry.

We focused on the subset of 625 sites that participated in both registries, with a final count of 776 890 patients from the Chest Pain–MI Registry (509 576 men [65.6%]; 620 981 white [80.0%]; median age, 64 years [interquartile range, 55-74 years]) and 853 386 patients from the CathPCI Registry (582 701 men [68.3%]; 691 236 white [81.0%]; median age, 63 years [interquartile range, 54-73 years]) (eTable 1 in the Supplement). Summary statistics are presented for the patients at the 625 linked hospitals from each registry in Table 1. Among patients from the CathPCI Registry, 484 446 patients (56.8%) underwent PCI for NSTEMI and 368 940 patients (43.2%) underwent PCI for STEMI. From the Chest Pain–MI Registry, 484 233 patients (62.3%) presented with NSTEMI and 292 657 patients (37.7%) presented with STEMI. A total of 258 371 of 484 233 patients (53.4%) who presented with NSTEMI and 263 727 of 292 657 patients (90.1%) who presented with STEMI underwent PCI.

Funnel plots of EMR-D and EMR-P for the 625 linked hospitals in the Chest Pain–MI and CathPCI registries are presented in Figure 1. Visually, there is more variation around the mean among lower-volume sites, even after restricting to sites with more than 40 patients per year. The rates of mortality exceeded 10% for some low-volume sites, with much less variation around the mean for the highest-volume sites. There were 202 of 625 hospitals (32.3%) with an EMR-D greater than 1.96 SDs from the mean and 86 of 625 hospitals (13.8%) with an EMR-D greater than 3 SDs from the mean. In addition, there were 191 of 625 hospitals (30.6%) with an EMR-P greater than 1.96 SDs from the mean and 90 of 625 hospitals (14.4%) with an EMR-P greater than 3 SDs from the mean.

Among linked hospitals, the Spearman rank correlation coefficient between EMR-D and EMR-P produced a ρ of 0.53 (95% CI, 0.47-0.58), suggesting a moderate correlation between the 2 measures (Figure 2). Among the highest-performing tertile for disease-based risk-adjusted mortality, 118 of the 208 sites (56.7%) were also in the highest tertile for procedural risk-adjusted mortality and 90 sites (43.3%) were classified into a lower category for procedural risk-adjusted mortality, with 25 sites (12.0%) classified into the lowest-performing tertile for procedural risk-adjusted mortality. Conversely, among the lowest-performing tertile for disease-based risk-adjusted mortality, 116 of 208 sites (55.8%) were in the lowest tertile for procedural risk-adjusted mortality and 92 sites (44.2%) were classified into a higher category for procedural risk-adjusted mortality, with 24 sites (11.5%) reclassified into the highest-performing tertile for procedural risk-adjusted mortality (Table 2). Results of the nonzero correlation test were significant, consistent with correlation between the rankings (P < .001). Reclassification rates for high-volume and low-volume sites in each registry were similar to the findings in the overall cohort (eTable 2 in the Supplement).

Bland-Altman plots demonstrated higher procedural mortality compared with disease-based mortality for the overall linked cohort, with a mean difference between EMR-P and EMR-D of 0.49% (95% CI, –1.61% to 2.58%) (Figure 3A). There were 17 hospitals with a difference between EMR-P and EMR-D greater than 1.96 SDs above the mean, and 15 hospitals with a difference between EMR-P and EMR-D greater than 1.96 SDs (in absolute value) below the mean. Results of a 1-sample t test comparing the mean difference with zero were significant (P < .001), consistent with the presence of a systematic bias. Repeating the analysis for patients with STEMI only (Figure 3B) demonstrated a smaller absolute difference between EMR-P and EMR-D of 0.12% (95% CI, –2.22% to 2.47%; P = .01). However, restricting the cohort to patients with AMI complicated by cardiogenic shock or cardiac arrest reversed the association between procedural mortality compared with disease-based mortality, with lower procedural mortality compared with disease-based mortality (Figure 3C). There was a mean difference between EMR-P and EMR-D of –0.64% (95% CI, –4.41% to 3.12%; P < .001).

Discussion

Among hospitals treating patients with AMI, we found a moderate correlation between disease-based outcomes from the Chest Pain–MI Registry and procedural-based outcomes from the CathPCI Registry. In addition, when grouping hospitals into tertiles of performance, nearly half of the hospitals that were either high performing or low performing based on procedural metrics were reclassified when judged by a disease-based metric. When assessing the difference between hospital-level PCI procedural mortality and disease-based mortality in sites participating in both registries, we found that, among all patients, reported procedural mortality was significantly higher than disease-based mortality. Although still statistically significant, this finding was greatly mitigated among the subgroup of patients with STEMI, with the 2 metrics more closely mirroring one another. However, among the subgroup of patients with AMI complicated by cardiogenic shock or cardiac arrest, the association changed, and disease-based mortality was significantly higher than procedure-based mortality, potentially demonstrating procedural risk avoidance among this highest-risk cohort.

There is a growing emphasis on quality of care assessment for patients with AMI, with key measures that include mortality, procedural complications, discharge medications, the use of cardiac rehabilitation, and adherence rates to consensus-appropriate use criteria. Hospital reimbursement is increasingly being tied to performance.8 In addition, public reporting programs have been gaining traction nationally but have focused primarily on procedure-based mortality. However, individual interventional cardiologists may not trust the adequacy of risk-adjustment methods for procedural-based metrics and, as a result, may eschew high-risk patients in an effort to maintain good reported outcomes.15,16 Unfortunately, these may be the patients who would benefit most from a life-saving procedure. More important, risk aversion is disproportionately associated with racial/ethnic minorities, who often present for medical care at later stages.17,18 Transitioning from a focus on PCI outcomes to disease-based summary measures has been proposed as a remedy for risk avoidance by interventional cardiologists.4-6 Metrics focused on the outcomes of all patients with AMI, not just those undergoing procedures, might incentivize the most optimal decision-making at all points of care during a hospitalization.

In this study, we sought to evaluate whether the use of disease-based metrics would result in the reclassification of hospitals ranked according to procedure-based metrics, and furthermore, to assess whether the measured performance according to procedural mortality was significantly different from disease-based mortality. Overall, we found only moderate correlation between risk-adjusted disease-based outcomes and procedure-based outcomes. More important, we found a considerable rate of reclassification among hospitals using procedural metrics. High performance for disease-based mortality did not guarantee high performance when measuring procedural mortality. In fact, nearly half of hospitals that were high performers for disease-based mortality were not in the top tertile for procedural mortality. Among hospitals with the best performance for procedural mortality, 24 of 208 would have been in the lowest tertile of hospitals for performance according to overall AMI mortality, and among hospitals with the worst performance for procedural mortality, 25 of 208 would have been in the highest tertile of hospitals for performance according to overall AMI mortality. These findings are consistent with results previously reported in New York state, suggesting that procedural outcomes alone are inadequate in the assessment of the care delivered for patients with AMI.19

We next evaluated the association between disease-based mortality and procedural-based mortality for each hospital using Bland-Altman plots, as these plots have been previously used to assess the degree of agreement between 2 clinical measurements and can identify the presence of fixed biases.20 Among the hospitals that participate in both the Chest Pain–MI and CathPCI registries, we discovered evidence of systematic bias between measurements of disease-based mortality and procedural-based mortality. For all patients with AMI, procedural mortality was systematically higher than disease-based mortality, which may be owing to treatment selection. However, the difference between disease-based mortality and procedural mortality for patients with STEMI was far smaller, perhaps because the management of these patients is generally more uniform, with PCI being a key, routine piece of their care. This outcome was supported by the finding that more than 90% of patients in the overall Chest Pain–MI Registry with STEMI underwent PCI. However, among the high-risk subgroup of patients with AMI with cardiogenic shock or cardiac arrest, the association between disease-based mortality and procedural mortality was reversed, with significantly higher disease-based mortality compared with procedural mortality. This finding suggests that some of the highest-risk patients with AMI are not undergoing PCI, which may be owing to clinical assessment of futility and appropriate deferral of PCI owing to a low likelihood of meaningful benefit, which may be the case for patients with an extreme risk of mortality independent of their coronary artery disease.21 Alternatively, this finding may represent procedural risk avoidance with limited access for patients who could potentially benefit most from this life-saving procedure. Disease-based mortality may be higher for these patients as interventional cardiologists exhibit reluctance to perform diagnostic angiography and PCI in these patients, resulting in fewer recorded deaths within the CathPCI Registry, although they are still captured in the Chest Pain–MI Registry.22,23

These findings highlight the inadequacy of procedural metrics. Because they can identify only the patients who undergo treatment for the disease, they are prone to underestimating the mortality of the subsets of the highest-risk patients, in this case those with cardiogenic shock or cardiac arrest. Physicians and hospitals that practice systematic risk avoidance among high-risk patients may have low measured rates of PCI mortality, although they risk higher rates of disease-based mortality because they are less likely to perform a potentially life-saving, but high-risk, PCI. Given the inability of procedural metrics to account for these factors, our findings suggest that procedural metrics are insufficient to assess the quality of care delivered for patients with AMI and, if used at all, should be interpreted in the context of a disease-based metric, which has the ability to capture the care of patients not offered procedures.

Disease-based metrics distribute risk across the spectrum of health care services provided at a hospital, thus incentivizing the hospital to provide the optimal care for an admitted patient. However, disease-based metrics should not be viewed as a panacea to the problem of risk avoidance, as in response to a focus on disease-based outcomes, hospitals themselves may practice risk avoidance by avoiding high-risk patients through interhospital transfers. This potential should be monitored in a prospective manner. Furthermore, the overall focus on mortality precludes a nuanced assessment for opportunities for improvement. As a supplement to mortality metrics, disease-based process metrics such as use of optimal medication therapy and cardiac rehabilitation referral may provide tangible areas for improvement for a hospital.

We expected there to be near complete overlap in the number of patients in both registries who underwent PCI for AMI given that the analysis was performed on a linked data set that included only hospitals actively participating in both registries and mandated consecutive patient data capture. The wide discrepancy in enrollment of patients undergoing PCI into the registries has several possible explanations. First, hospitals do not capture all types of patients with AMI in the Chest Pain–MI Registry. This may be particularly true among patients with NSTEMI, as only type 1 myocardial infarctions on admission mandate consecutive data capture in the Chest Pain–MI Registry. All sites are instructed to enter patients with elevated cardiac biomarkers and symptoms on admissions typical for AMI. However, both the overall cohort and the STEMI-only subset demonstrated a higher procedural mortality compared with disease-based mortality, which was reversed for patients with AMI with cardiac arrest or cardiogenic shock.

Our analysis has important implications for contemporary care despite the imperfect overlap of patients within the registries. The data management processes observed here are representative of how data are collected and used in the real world, with the performance metrics from both registries used for individual hospital quality improvement efforts and, in some cases, submitted to insurance companies and payers.

Limitations

Our study has several limitations. We selected for hospitals that participated in both the Chest Pain–MI and CathPCI registries, although not all hospitals participate in both. Hospitals and health systems who elect to submit data to both registries may be fundamentally different from those who do not, and as a result, our findings may not be generalizable to all hospitals nationwide. However, among the large sample of hospitals and patients included in this study, our findings highlight the challenges of interpreting and relying on procedural metrics when assessing the quality of care delivered for patients with AMI.

Conclusions

For hospitals treating patients with AMI, we found only moderate correlation between procedural outcomes and disease-based outcomes, and nearly half of hospitals in the highest tertile for PCI performance were reclassified into a lower performance tertile when judged by disease-based metrics. Although we observed higher rates of procedural mortality compared with disease-based mortality in the overall AMI cohort, we found higher rates of mortality using disease-based metrics compared with procedural metrics when assessing patients with cardiogenic shock or cardiac arrest, which may represent potential risk avoidance among this highest-risk subset of patients.

Back to top
Article Information

Accepted for Publication: February 21, 2020.

Corresponding Author: Jay Giri, MD, MPH, Cardiovascular Division, Hospital of the University of Pennsylvania, Perelman Center, South Tower, 11th Floor, 3400 Civic Center Blvd, Philadelphia, PA 19104 (jay.giri@pennmedicine.upenn.edu).

Published Online: April 29, 2020. doi:10.1001/jamacardio.2020.0753

Author Contributions: Ms Xiang and Mr Wojdyla had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Nathan, Wadhera, Giri.

Acquisition, analysis, or interpretation of data: Nathan, Xiang, Wojdyla, Khatana, Dayoub, Bhatt, Kolansky, Kirtane, Rao, Yeh, Groeneveld, Wang, Giri.

Drafting of the manuscript: Nathan, Giri.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Xiang, Wojdyla, Groeneveld.

Obtained funding: Nathan, Giri.

Administrative, technical, or material support: Dayoub, Wang, Giri.

Supervision: Kolansky, Groeneveld, Giri.

Other: Wadhera.

Conflict of Interest Disclosures: Dr Wadhera reported receiving grants from National Heart, Lung, and Blood Institute; research support from the American College of Cardiology/Merck Cardiovascular Research Fellowship; and prior personal fees from Regeneron outside the submitted work. Dr Bhatt reported serving on the advisory board for Cardax, Cereno Scientific, Elsevier Practice Update Cardiology, Medscape Cardiology, PhaseBio, and Regado Biosciences; serving on the board of directors for Boston VA Research Institute, Society of Cardiovascular Patient Care, and TobeSoft; serving as the chair of the American Heart Association Quality Oversight Committee; serving as the chair of the NCDR-ACTION Registry Steering Committee; serving as the chair of the VA CART Research and Publications Committee; serving on data monitoring committees for Baim Institute for Clinical Research (formerly Harvard Clinical Research Institute, for the PORTICO trial, funded by St Jude Medical, now Abbott), Cleveland Clinic (including for the ExCEED trial, funded by Edwards), Duke Clinical Research Institute, Mayo Clinic, Mount Sinai School of Medicine (for the ENVISAGE trial, funded by Daiichi Sankyo), and Population Health Research Institute; receiving honoraria from the American College of Cardiology (senior associate editor, Clinical Trials and News, ACC.org; vice-chair, ACC Accreditation Committee), Baim Institute for Clinical Research (formerly Harvard Clinical Research Institute; RE-DUAL PCI clinical trial steering committee funded by Boehringer Ingelheim; AEGIS-II executive committee funded by CSL Behring), Belvoir Publications (editor in chief, Harvard Heart Letter), Duke Clinical Research Institute (clinical trial steering committees), HMP Global (editor in chief, Journal of Invasive Cardiology), Journal of the American College of Cardiology (guest editor; associate editor), Medtelligence/ReachMD (CME steering committees), Population Health Research Institute (for the COMPASS operations committee, publications committee, steering committee, and USA national coleader, funded by Bayer), Slack Publications (chief medical editor, Cardiology Today’s Intervention), Society of Cardiovascular Patient Care (secretary/treasurer), and WebMD (CME steering committees); serving as deputy editor for Clinical Cardiology; receiving research funding from Abbott, Amarin, Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol-Myers Squibb, Chiesi, CSL Behring, Eisai, Ethicon, Ferring Pharmaceuticals, Forest Laboratories, Idorsia, Ironwood, Ischemix, Lilly, Medtronic, PhaseBio, Pfizer, Regeneron, Roche, Sanofi Aventis, Synaptic, and The Medicines Company; receiving royalties from Elsevier (editor, Cardiovascular Intervention: A Companion to Braunwald’s Heart Disease); serving as site coinvestigator: for Biotronik, Boston Scientific, St Jude Medical (now Abbott), and Svelte; serving as trustee for the American College of Cardiology; and performing unfunded research for FlowCo, Fractyl, Merck, Novo Nordisk, PLx Pharma, and Takeda. Dr Yeh reported receiving grants frorm AstraZeneca; consulting fees from Teleflex; serving on advisory boards for Abbott, Boston Scientific, and Medtronic; and receiving grants and personal fees from Abbott Vascular, Boston Scientific, and Medtronic outside the submitted work. Dr Wang reported research grants to the Duke Clinical Research Institute from AstraZeneca, Bristol-Myers Squibb, Cryolife, Portola, and Regeneron; receiving grants and personal fees from AstraZeneca; grants from Bristol-Myers Squibb, Cryolife, Regeneron, Merck, and Pfizer; personal fees from Sanofi; and consulting honoraria from AstraZeneca during the conduct of the study. Dr Giri reported serving on an advisory board for and receiving personal fees from AstraZeneca; grants from Recor Medical and St Jude Medical; and personal fees from Partners Healthcare outside the submitted work. Dr Kirtane reported institutional funding to Columbia University and/or the Cardiovascular Research Foundation from Medtronic, Boston Scientific, Abbott Vascular, Abiomed, CSI, CathWorks, Siemens, Philips, and ReCor Medical. No other disclosures were reported.

Funding/Support: This project was funded by the American College of Cardiology/National Cardiovascular Data Registry CathPCI Registry.

Role of the Funder/Sponsor: The funding source was involved in the design and conduct of the study, collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank Dadi Dai, PhD, for his assistance with the statistical analysis. He was not compensated beyond his employment at the Duke Clinical Research Institute.

References
1.
Hannan  EL.  Using mortality data for profiling hospital quality of care and targeting substandard care.   J Soc Health Syst. 1989;1(1):31-48.PubMedGoogle Scholar
2.
Shahian  DM, Normand  SL.  Comparison of “risk-adjusted” hospital outcomes.   Circulation. 2008;117(15):1955-1963. doi:10.1161/CIRCULATIONAHA.107.747873 PubMedGoogle ScholarCrossref
3.
Wasfy  JH, Borden  WB, Secemsky  EA, McCabe  JM, Yeh  RW.  Public reporting in cardiovascular medicine: accountability, unintended consequences, and promise for improvement.   Circulation. 2015;131(17):1518-1527. doi:10.1161/CIRCULATIONAHA.114.014118 PubMedGoogle ScholarCrossref
4.
Wadhera  RK, Bhatt  DL.  Taking the “public” out of public reporting of percutaneous coronary intervention.   JAMA. 2017;318(15):1439-1440. doi:10.1001/jama.2017.12087 PubMedGoogle ScholarCrossref
5.
Wadhera  RK, Joynt Maddox  KE, Yeh  RW, Bhatt  DL.  Public reporting of percutaneous coronary intervention outcomes: moving beyond the status quo.   JAMA Cardiol. 2018;3(7):635-640. doi:10.1001/jamacardio.2018.0947 PubMedGoogle ScholarCrossref
6.
Wadhera  RK, Bhatt  DL.  Toward precision policy—the case of cardiovascular care.   N Engl J Med. 2018;379(23):2193-2195. doi:10.1056/NEJMp1806260 PubMedGoogle ScholarCrossref
7.
Harder  B, Comarow  A.  Hospital quality reporting by US News & World Report: why, how, and what’s ahead.   JAMA. 2015;313(19):1903-1904. doi:10.1001/jama.2015.4566 PubMedGoogle ScholarCrossref
8.
Duffy  PL, Naidu  SS, Kurian  KC, Marshall  JJ.  Rationale for ST-elevation myocardial infarction call compensation by healthcare systems.   Catheter Cardiovasc Interv. 2013;81(6):909-911. doi:10.1002/ccd.24921 PubMedGoogle ScholarCrossref
9.
Peterson  ED, Roe  MT, Rumsfeld  JS,  et al.  A call to ACTION (Acute Coronary Treatment and Intervention Outcomes Network): a national effort to promote timely clinical feedback and support continuous quality improvement for acute myocardial infarction.   Circ Cardiovasc Qual Outcomes. 2009;2(5):491-499. doi:10.1161/CIRCOUTCOMES.108.847145 PubMedGoogle ScholarCrossref
10.
Brindis  RG, Fitzgerald  S, Anderson  HV, Shaw  RE, Weintraub  WS, Williams  JF.  The American College of Cardiology–National Cardiovascular Data Registry (ACC-NCDR): building a national clinical data repository.   J Am Coll Cardiol. 2001;37(8):2240-2245. doi:10.1016/S0735-1097(01)01372-9 PubMedGoogle ScholarCrossref
11.
Chin  CT, Chen  AY, Wang  TY,  et al.  Risk adjustment for in-hospital mortality of contemporary patients with acute myocardial infarction: the Acute Coronary Treatment and Intervention Outcomes Network (ACTION) registry–Get With The Guidelines (GWTG) acute myocardial infarction mortality model and risk score.   Am Heart J. 2011;161(1):113-122.e2. doi:10.1016/j.ahj.2010.10.004 PubMedGoogle ScholarCrossref
12.
Brennan  JM, Curtis  JP, Dai  D,  et al; National Cardiovascular Data Registry.  Enhanced mortality risk prediction with a focus on high-risk percutaneous coronary intervention: results from 1,208,137 procedures in the NCDR (National Cardiovascular Data Registry).   JACC Cardiovasc Interv. 2013;6(8):790-799. doi:10.1016/j.jcin.2013.03.020 PubMedGoogle ScholarCrossref
13.
Doll  JA, Dai  D, Roe  MT,  et al.  Assessment of operator variability in risk-standardized mortality following percutaneous coronary intervention: a report from the NCDR.   JACC Cardiovasc Interv. 2017;10(7):672-682. doi:10.1016/j.jcin.2016.12.019 PubMedGoogle ScholarCrossref
14.
Bland  JM, Altman  DG.  Statistical methods for assessing agreement between two methods of clinical measurement.   Lancet. 1986;1(8476):307-310. doi:10.1016/S0140-6736(86)90837-8 PubMedGoogle ScholarCrossref
15.
Fernandez  G, Narins  CR, Bruckel  J, Ayers  B, Ling  FS.  Patient and physician perspectives on public reporting of mortality ratings for percutaneous coronary intervention in New York state.   Circ Cardiovasc Qual Outcomes. 2017;10(9):e003511. doi:10.1161/CIRCOUTCOMES.116.003511 PubMedGoogle Scholar
16.
Blumenthal  DM, Valsdottir  LR, Zhao  Y,  et al.  A survey of interventional cardiologists’ attitudes and beliefs about public reporting of percutaneous coronary intervention.   JAMA Cardiol. 2018;3(7):629-634. doi:10.1001/jamacardio.2018.1095 PubMedGoogle ScholarCrossref
17.
Hochman  JS, Sleeper  LA, White  HD,  et al; SHOCK Investigators. Should We Emergently Revascularize Occluded Coronaries for Cardiogenic Shock.  One-year survival following early revascularization for cardiogenic shock.   JAMA. 2001;285(2):190-192. doi:10.1001/jama.285.2.190 PubMedGoogle ScholarCrossref
18.
Werner  RM, Asch  DA, Polsky  D.  Racial profiling: the unintended consequences of coronary artery bypass graft report cards.   Circulation. 2005;111(10):1257-1263. doi:10.1161/01.CIR.0000157729.59754.09 PubMedGoogle ScholarCrossref
19.
Nathan  AS, Khatana  SAM, Yeh  RW, Groeneveld  PW, Giri  J.  Hospital-specific mortality for acute myocardial infarction versus emergency percutaneous coronary intervention in New York state.   JACC Cardiovasc Interv. 2019;12(9):898-899. doi:10.1016/j.jcin.2019.02.045 PubMedGoogle ScholarCrossref
20.
Altman  DG, Bland  JM.  Measurement in medicine: the analysis of method comparison studies.   Statistician. 1983;32:307-317. doi:10.2307/2987937 Google ScholarCrossref
21.
Lemkes  JS, Janssens  GN, van der Hoeven  NW,  et al.  Coronary angiography after cardiac arrest without ST-segment elevation.   N Engl J Med. 2019;380(15):1397-1407. doi:10.1056/NEJMoa1816897 PubMedGoogle ScholarCrossref
22.
McCabe  JM, Waldo  SW, Kennedy  KF, Yeh  RW.  Treatment and outcomes of acute myocardial infarction complicated by shock after public reporting policy changes in New York.   JAMA Cardiol. 2016;1(6):648-654. doi:10.1001/jamacardio.2016.1806 PubMedGoogle ScholarCrossref
23.
Nathan  AS, Shah  RM, Khatana  SA,  et al.  Effect of public reporting on the utilization of coronary angiography after out-of-hospital cardiac arrest.   Circ Cardiovasc Interv. 2019;12(4):e007564. doi:10.1161/CIRCINTERVENTIONS.118.007564 PubMedGoogle Scholar
×