Error bars indicate 95% CIs.
eTable. Agency for Healthcare Research & Quality (AHRQ) Risk Adjustment Model Components
eFigure 1. Rates of Revascularization and In-hospital Death for Patients With Myocardial Infarction and Cardiogenic Shock in New York and Non–New York States Prepolicy Change (2002-2005) Compared With Postpolicy Change (2006-2012)
eFigure 2. Sensitivity Analysis Comparing Adjusted Risk of Receiving PCI in 2006-2012 Compared With 2002-2005 in New York vs Control States Performed Serially Excluding Each of the Control States in Turn
eFigure 3. Sensitivity Analysis Comparing Adjusted Risk of In-hospital Mortality in 2006-2012 Compared With 2002-2005 in New York vs Control States Performed Serially Excluding Each of the Control States in Turn
Customize your JAMA Network experience by selecting one or more topics from the list below.
McCabe JM, Waldo SW, Kennedy KF, Yeh RW. Treatment and Outcomes of Acute Myocardial Infarction Complicated by Shock After Public Reporting Policy Changes in New York. JAMA Cardiol. 2016;1(6):648–654. doi:10.1001/jamacardio.2016.1806
In 2006, New York began excluding patients with cardiogenic shock from the publicly reported percutaneous coronary intervention (PCI) risk-adjusted mortality analyses.
To examine the effects of the New York shock-exclusion policy change on rates of revascularization and mortality for patients with acute myocardial infarction (AMI) complicated by cardiogenic shock.
Design, Setting, and Participants
This study used several comprehensive statewide hospitalization databases to identify patients with AMI and shock from January 1, 2002, through December 31, 2012, in New York and a series of comparator states (Massachusetts, Michigan, and New Jersey from January 1, 2002, through December 31, 2012, and California from January 1, 2003, through December 31, 2011). Data analysis was performed from October 1, 2015, to March 15, 2016.
Main Outcomes and Measures
A difference-in-differences approach was used to evaluate whether the likelihood of receiving PCI and surviving to discharge differed after the policy change in New York in 2006 compared with comparator states that did not enact such a change.
Among 45 977 patients with AMI and cardiogenic shock (11 298 in New York), 21 974 (47.8%) underwent PCI. The mean (SD) age of the patients was 69.7 (13.2) years, and 18 139 (39.5%) were female. After adjusting for patient factors, patients in New York were significantly more likely to undergo PCI after the public reporting policy changes than they were previously (adjusted relative risk [aRR], 1.28; 95% CI, 1.19-1.37; P < .001) compared with a 9% increase in comparator states during the same period (aRR, 1.09; 95% CI, 1.05-1.13; P < .001; interaction P < .001). Nevertheless, rates of PCI remained lower in New York compared with comparator states throughout the study period. The adjusted risk of in-hospital death among patients in New York with AMI and shock decreased significantly faster after the policy change (aRR, 0.76; 95% CI, 0.72-0.81; P < .001) compared with comparator states (aRR, 0.91; 95% CI, 0.87-0.94; P < .001; interaction P < .001).
Conclusions and Relevance
The exclusion of patients with ongoing cardiogenic shock from New York PCI public reports in 2006 was associated with a significant increase in the use of PCI for cardiogenic shock and a concomitant decrease in in-hospital mortality, exceeding simultaneously observed trends in the comparator states. However, rates of PCI for AMI and shock were lower in New York throughout the study. Alterations in policies related to reporting mortality outcomes after cardiovascular procedures may have significant implications for physician behavior and the public health.
Public reporting of mortality outcomes after cardiovascular procedures, specifically percutaneous coronary intervention (PCI) and coronary artery bypass grafting (CABG), is an increasing mandate from government agencies, payer groups, media outlets, and consumer advocates.1,2 New York State was the first to organize and publicly report in-hospital risk-adjusted mortality after PCI in 1992. Other states, including Massachusetts, Pennsylvania, and, more recently, Washington, have followed suit.3
Although public reporting is intended to build public trust and improve the quality of care, there is concern that it has also inadvertently led to inappropriate risk avoidance of higher-risk patients among PCI operators.4-6 This concern has been controversial primarily because of an inability to track or record data on patients who may have been considered for PCI but did not receive it. As such, the evidence of risk aversion has been gleaned indirectly by comparing practice patterns and mortality rates between public-reporting and non–public-reporting states.4,7-10 On the basis of increasing concern about risk avoidance as a public health hazard,11 the New York State Department of Public Health decided to exclude patients with refractory cardiogenic shock from their publicly reported analysis of PCI as of January 1, 2006.12
To date, no comprehensive assessment has been performed on the effects of this policy change has had on quality of care for patients with acute myocardial infarction (AMI) complicated by cardiogenic shock. We therefore assessed whether rates of coronary revascularization and in-hospital mortality changed among patients with AMI with shock after the exclusion of such patients from public reporting in 2006. We further compared the observed changes in revascularization and outcomes in New York with those observed during the same period in several comparator states as a control for secular changes in practice.
Question Does exclusion of certain high-risk patients from public reporting of outcomes after percutaneous coronary intervention affect operators’ willingness to treat such patients?
Findings In this study, after a policy change in New York in 2006, operators were 28% more likely to perform high-risk percutaneous coronary intervention, a significantly greater change than the 9% increase seen in comparator states, although the total percentage of patients treated in New York remained lower.
Meaning Physician risk aversion based on public outcome transparency can be partially mitigated through targeted policy change.
The State Inpatient Databases comprise a family of databases that include comprehensive all-payer inpatient discharge records from hospitals within a given state. For this analysis, we used the state hospitalization databases of New York, Massachusetts, Michigan, and New Jersey from January 1, 2002, through December 31, 2012, and California from January 1, 2003, through December 31, 2011. Data analysis was performed from October 1, 2015, to March 15, 2016.
Patients hospitalized in New York constituted the primary analytic group, whereas those hospitalized in California, Massachusetts, Michigan, and New Jersey were selected to serve as comparator states that do not publicly report risk-adjusted mortality (California, Michigan, and New Jersey) or publicly report mortality outcomes but did not institute cardiogenic shock exclusion policies (Massachusetts, which altered its risk adjustment scheme to adjust for compassionate use in 200513 but does not exclude patients with shock from reporting). The study did not qualify as human subjects research because of the deidentified nature of the public databases used; therefore, institutional review board approval was not obtained.
All patients admitted with a primary and secondary discharge diagnosis of AMI and shock (in either order) using the International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) codes were initially included. Consistent with previously validated codes used in other data sets, AMI was defined as a primary discharge diagnosis of non–ST-segment elevation MI (codes 410.71 and 410.91) or ST-segment elevation MI (codes 410.11-410.61 and 410.81); shock was defined as cardiogenic shock (code 785.51).9 Patients admitted to hospitals without PCI capabilities were excluded. Patients transferred to another acute care facility were also excluded to avoid double counting of hospitalizations for the same presentation and to ensure an accurate assessment of in-hospital procedural management and outcomes.
Demographic characteristics, including patient age, sex, and race, were derived from the data set. To evaluate the procedural management of patients with these diagnoses, the data set was queried for procedural codes for PCI (ICD-9-CM codes 00.66, 17.55, 36.01, 36.02, 36.05, 36.06, and 36.07). These codes have been previously used to identify patients receiving these procedures in this and other data sets.14 The primary outcomes in this analysis were rates of invasive management, including coronary angiography, PCI, and all revascularization (PCI or CABG surgery), as well as in-hospital death.
We first stratified the cohort into the period before the date of change in the New York public reporting policy vs the period after this change. New York State removed patients presenting with cardiogenic shock and AMI from the public report on a trial basis starting at the beginning of 2006 and made this policy permanent in 2008. As such, all patients with adjudicated shock have been excluded since January 1, 2006, and we used this time point to stratify the cohort into prepolicy (2002-2005) and postpolicy (2006-2012) periods.
Patient characteristics and outcomes were compared during prepolicy vs postpolicy change periods in New York and in comparator states separately using the Fisher exact test, χ2 test, Mantel-Haenszel trend test, and linear trend test for comparisons of categorical and continuous variables, as appropriate. To examine the association between the policy change and changes in rates of coronary angiography, PCI, all revascularization, and in-hospital death for patients with AMI and shock, we used a difference-in-differences approach. In this approach, we used a modified Poisson regression model15 to examine whether the relative risk of all outcomes differed in the postpolicy vs prepolicy change periods in New York and compared these changes with those observed in comparator states using a period × state interaction term. This model included hospital site as a random effect to account for clustering at the hospital level. In addition, the model included covariates for age, sex, ST-segment elevation MI presentation, and 29 comorbid medical conditions identified by the risk adjustment model developed by the Agency for Healthcare Research and Quality (eTable in the Supplement).
To examine any temporal delays between the policy change and changes in treatment and outcomes, we additionally examined the primary outcomes on a yearly basis. Specifically, we constructed similar hierarchical Poisson regression models adjusted for patient characteristics in which calendar year was considered a categorical variable.
A variety of sensitivity analyses were performed to evaluate the validity of our results. To ensure that no one comparator state unduly influenced the observed differences between New York and the comparator group, we repeated the primary analysis serially removing each state from the comparator group in turn.
Finally, the influence of potential changes in coding was examined. First, we compared rates of cardiogenic shock diagnoses among patients with AMI in New York and the comparator states before and after 2006. Second, we performed sensitivity analyses in which all regression models were further adjusted for a hospital’s rate of shock among AMI admissions. Such an adjustment would serve to mitigate the effects of up-coding of shock over time as an explanation of any observed differences that may have occurred over time between New York and the comparator states.
Dichotomous data are presented as percentages. Continuous variables are presented as means (SDs) or medians (interquartile ranges) for parametric and nonparametric data, respectively. All statistical analyses were performed using SAS statistical software, version 9.3 (SAS Institute Inc).
A total of 45 977 patients diagnosed as having AMI and cardiogenic shock between 2002 and 2012 were included in this analysis, of whom 11 298 (24.6%) were in New York. The mean (SD) age of the patients was 69.7 (13.2) years, and 18 139 (39.5%) were female. Among this cohort, 15 043 AMIs complicated by shock occurred during the baseline reference period of 2002 through 2005 and 30 934 during the postpolicy change period from 2006 through 2012. Differences in patient-level risk factors between the 2 periods are listed in Table 1. Across all states, rates of coronary angiography for cardiogenic shock during the reference and postpolicy change years were not statistically different (P = .53) (Table 2). However, the number of patients receiving PCI for shock increased from 6753 (44.9%) in the reference period to 15 221 (49.2%) in the postpolicy change period (P < .001), whereas the number of patients receiving CABG decreased from 2966 (19.7%) in the reference period to 5591 (18.1%) in the postpolicy change period (P < .001). Numbers of in-hospital death also decreased between the 2 periods in question from 6725 (44.7%) to 11 714 (37.9%), respectively (P < .001) (Table 2). In New York specifically, comparing the reference period and the postpolicy period revealed an increase in the use of coronary angiography (2233 [63.6%] vs 5289 [67.9%], P < .001), an increase in the use of PCI (1070 [30.5%] vs 3093 [39.7%], P < .001), unchanged rates of CABG (759 [21.6%] vs 1682 [21.6%], P > .99), and a decrease in the rates of in-hospital death (1654 [47.1%] vs 2760 [35.5%], P < .001) (Table 3 and eFigure 1 in the Supplement). Comparing the same periods in the reference states, the number of PCIs increased from 5683 (49.3%) to 12 128 (52.4%) (P < .001), the number of CABGs decreased from 2207 (19.1%) to 3909 (16.9%) (P < .001), and the number of in-hospital deaths decreased from 5071 (44.0%) to 8954 (38.7%) (P < .001) (Table 3 and eFigure 1 in the Supplement).
After patient demographic and presentation risk factors were adjusted for, operators in New York State were 28% more likely to perform PCI on patients with AMI and cardiogenic shock in the era after the public reporting policy changes than they were previously (adjusted relative risk [aRR], 1.28; 95% CI, 1.19-1.37; P < .001). By comparison, operators in California, Massachusetts, Michigan, and New Jersey were 9% more likely to perform PCI in the later era relative to the former (aRR, 1.09; 95% CI, 1.05-1.13; P < .001). When comparing the changes in periods in New York with those in the comparator states, the difference in differences was also statistically significant (P < .001 for interaction). A sensitivity analysis serially dropping each comparator state in turn demonstrated consistent results (eFigure 2 in the Supplement). Examination of trends in the use of PCI for cardiogenic shock by individual year (compared with non–New York states in 2002) demonstrates that the public reporting policy changes in New York State coincided with an immediate and sustained increase in the use of PCI for patients with cardiogenic shock in that state that was not seen in the comparator states (Figure 1).
Broadening the adjusted analysis to include all forms of revascularization also demonstrated a 15% increased rate of revascularization for shock in New York after 2006 (aRR, 1.15; 95% CI, 1.09-1.22; P < .001) compared with a nonsignificant change in the comparator states (aRR, 1.03; 95% CI, 1.00-1.06; P = .72). Again, the differences between the in-state comparisons were statistically significant (P = .001 for interaction). Conversely, the adjusted risk of in-hospital death for all patients with cardiogenic shock and AMI in New York decreased by 24% compared with the era before the policy change (aRR, 0.76; 95% CI, 0.72-0.81; P < .001), whereas the adjusted risk of in-hospital death for all patients with shock and AMI in the comparator states decreased by 9% during the same periods (aRR, 0.91; 95% CI, 0.87-0.94; P < .001). The differences between the changes in mortality in New York and the comparator states were also significantly different (P < .001 for interaction). A sensitivity analysis serially removing each comparator state in turn demonstrated consistent results (eFigure 3 in the Supplement).
As demonstrated in Figure 2, when evaluated per year relative to non–New York states in 2002, the significant improvements in mortality for patients with AMI in cardiogenic shock from New York State also began after the onset of the 2006 public reporting exclusion.
Shock among patients with AMI was diagnosed more frequently over time in all states but to a greater degree in New York. From 2002 to 2005, the prevalence of shock among patients with an AMI was 3.2% in New York and 3.5% in the control cohort. From 2006 to 2012, the rates were 3.9% and 3.9%, respectively. Thus, the risk ratio for a diagnosis of shock in the setting of AMI in New York in 2006 to 2012 compared with 2002 to 2005 was 1.22 (95% CI, 1.18-1.25), whereas it was 1.11 (95% CI, 1.08-1.14) in the control states (P < .001 for interaction).
We therefore performed a sensitivity analysis incorporating shock rate per hospital per year to adjust for the increase in shock diagnoses coded. The aRR for PCI in shock in New York and the comparator states did not differ significantly from our primary findings after controlling for rates of shock (aRR for New York, 1.27; 95% CI, 1.18-1.37; aRR for comparator states, 1.09; 95% CI, 1.05-1.12). Similarly, improvements in in-hospital death did not significantly differ from the primary analysis (aRR for New York, 0.79; 95% CI, 0.74-0.84; and aRR for comparator states, 0.92; 95% CI, 0.89-0.95).
In this study, we found that rates of revascularization, specifically with PCI, for patients with AMI and cardiogenic shock increased significantly in New York after policy changes in the New York public reporting process that excluded patients with refractory cardiogenic shock from publicly disclosed risk-adjusted mortality rates. This increase in the use of PCI for cardiogenic shock was significantly greater than that observed during the same period in several comparator states. These data may suggest that the policy changes implemented in the New York public reporting process had the desired effect of reducing risk aversion. Concomitantly, the in-hospital mortality of all patients with AMI and cardiogenic shock decreased to a significantly greater degree in New York than in the comparator states during the intervals analyzed, suggesting that the public reporting policy change may have also improved public health by facilitating more revascularization, a guideline-directed therapy for cardiogenic shock.11
However, compared with the comparator states, New York started with far lower rates of coronary revascularization and worse rates of in-hospital death for AMI complicated by cardiogenic shock. Thus, despite a significantly greater increase in the use of revascularization, New York continued to have lower overall rates of revascularization for AMI with cardiogenic shock even at the end of the study period. These findings, in addition to lower observed rates of coronary revascularization and higher rates of CABG in New York compared with the comparator states before and after the policy change, could be indicative of continued risk aversion on the part of PCI operators in a public reporting environment.
Although rates of revascularization for AMI complicated by shock appear to coincide directly with the institution of the public reporting policy change in 2006, this change actually excluded only 0.25% of all PCI cases from public disclosure during the first 3 years it was in place.16 Given that 99.75% of all PCIs continued to be analyzed and reported publicly, the magnitude of effect the policy change may have had on physician behavior appears to have far surpassed the effect it actually had on the population of patients considered in the public reports. This discrepancy may be related to strong messaging associated with the policy change or with a belief that excluding high-risk patients is fundamentally more fair to clinicians than risk adjustment alone, which has previously been perceived by cardiologists in New York as inadequate.17 Alternatively, the observed increase in rates of revascularization could be related strictly to secular changes, although this was not observed in the comparator states. Similarly, the improvement in mortality observed in New York could be related to the change in policy and other quality improvement initiatives occurring in the state during the period studied.
A frequent concern in the public reporting literature is the possibility that much of the observed effects over time could be the result of coding migration, up-coding, or gaming the system such that patients’ risks are more thoroughly documented or even exaggerated in an effort to influence the association between observed and expected outcomes in a risk-adjusted model.18 However, New York had been participating in risk-adjusted mortality outcome reporting for nearly 15 years before the policy change in question. It is therefore likely that any emphasis on coding and reporting was well established by 2006. Interestingly, however, we found a significant increase in shock diagnoses after the public reporting policy change that was greater than that seen in the control states at the same time. However, this difference between New York and the control states was actually a product of New York having a lower prevalence of shock during the reference period of 2002 through 2005. New York and the comparator states had the same prevalence of shock among patients with AMI during the subsequent postpolicy change period (3.9%). Furthermore, a sensitivity analysis adjusting for the prevalence of the shock diagnoses per hospital per year suggested that the change in prevalence, which may have been the result of coding practices, did not play an important role in the increased use of PCI or in the decrease in rates of in-hospital death observed in New York.
The present analysis should be interpreted in the context of several limitations. Patients were identified in administrative data sets, a practice that has inherent limitations. Our analysis, however, incorporates 100% of relevant discharges at all nonfederal hospitals in each of the states analyzed rather than relying on sampling that may differ from year to year and bias temporal analyses. Furthermore, multivariable adjustment for the severity of illness could only be performed based on claims-based data. Because of this limitation, it is possible that there are comorbidities that differ between the various groupings that are not captured by the data set but could potentially explain some differences in procedural management and outcomes. Thus, it is possible that the differences before and after the policy change reflect differences in illness severity among patients coded as having shock rather than a true effect of the policy change on physician behavior. However, the criteria for exclusion from the New York State public report were not based on billing data but required substantiated medical record documentation. The criteria set forth for exclusion from the public report in New York included acute hypotension with a systolic blood pressure less than 80 mm Hg or a cardiac index less than 2.0 L · min−1 · m−2 despite pharmacologic or mechanical support at the time of PCI.16 Last, although we used a difference-in-differences approach with several control states, which mitigates confounding attributable to unobserved secular trends, we cannot establish causality based on the data presented.
Public reporting of risk-adjusted mortality after cardiovascular procedures is an increasingly common practice. Concerns of risk aversion among operators practicing in public reporting areas have led to some evidence suggesting public reporting of mortality may be associated with a decline in public health for the sickest patients. After the New York State policy change in 2006, in which very high-risk patients were censored from the public report, there was an immediate increase in the use of PCI therapy for patients with AMI and cardiogenic shock.
There was also an improvement in the in-hospital survival of such patients that surpassed secular changes seen in comparator states. Nevertheless, New York continues to lag significantly behind the comparator states in terms of rates of PCI for patients with AMI and cardiogenic shock. Among institutions actively reporting in-hospital mortality after cardiovascular procedures, there is no uniform methodologic approach to address concerns of risk aversion. This lack of uniformity is increasingly important as the National Cardiovascular Data Registry and other national agencies move toward public transparency. The present analysis suggests that the censoring of adjudicated, extreme-risk cases may have been effective at facilitating guideline-directed revascularization and improving outcomes. Further research is required to better understand how to balance the desires for health care transparency with a system that encourages appropriate care for the highest-risk patients.
Correction: This article was corrected on August 31, 2016, for a typographical error in the byline.
Accepted for Publication: May 13, 2016.
Corresponding Author: Robert W. Yeh, MD, MSc, Smith Center for Outcomes Research in Cardiology, Beth Israel Deaconess Medical Center, 375 Longwood Ave, Fourth Floor, Boston, MA 02215 (email@example.com).
Published Online: July 27, 2016. doi:10.1001/jamacardio.2016.1806
Author Contributions: Drs McCabe and Yeh had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: McCabe, Waldo, Yeh.
Acquisition, analysis, or interpretation of data: McCabe, Waldo, Yeh
Drafting of the manuscript: McCabe.
Critical revision of the manuscript for important intellectual content: Waldo, Kennedy, Yeh.
Statistical analysis: McCabe, Kennedy, Yeh.
Obtained funding: Yeh.
Administrative, technical, or material support: Yeh.
Study supervision: McCabe, Yeh.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: Funding for this project was provided by Massachusetts General Hospital’s Hassenfeld Scholar Award and the Richard and Susan Smith Center for Outcomes Research (Dr Yeh).
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.