Ferraris VA, Bolanos M, Martin JT, Mahan A, Saha SP. Identification of Patients With Postoperative Complications Who Are at Risk for Failure to Rescue. JAMA Surg. 2014;149(11):1103-1108. doi:10.1001/jamasurg.2014.1338
A minority of patients who experience postoperative complications die (failure to rescue). Understanding the preoperative factors that lead to failure to rescue helps surgeons predict and avoid operative mortality.
To provide a mechanism for identifying a high-risk group of patients with postoperative complications who are at a substantially increased risk for failure to rescue.
Design, Setting, and Patients
Observational study evaluating failure to rescue in patients entered into the American College of Surgeons National Surgical Quality Improvement Program database. The large sample of surgical patients included in this study underwent a wide range of operations during a 5-year period in more than 200 acute care hospitals. We examined and identified patients at high risk for failure to rescue using propensity stratification. We also developed a risk-scoring system that allowed preoperative identification of patients at the highest risk for failure to rescue.
Main Outcomes and Measures
Risk-scoring system that predicts failure to rescue.
Of the 1 956 002 database patients, there were 207 236 patients who developed serious postoperative complications. Deaths occurred in 21 731 patients with serious complications (10.5% failure to rescue). Stratification of patients into quintiles, according to their propensity for developing serious complications, found that 90% of operative deaths occurred in the highest-risk quintile, usually within a week of developing the initial complication. A risk-scoring system for failure to rescue, based on regression-derived variable odds ratios, predicted patients in the highest-risk quintile with good predictive accuracy. Only 31.8% of failure-to-rescue patients had a single postoperative complication. Perioperative deaths increased exponentially as the number of complications per patient increased. Patients with complications who had surgical residents involved in their care had reduced rates of failure to rescue compared with patients without resident involvement.
Conclusions and Relevance
Twenty percent of high-risk patients account for 90% of failure to rescue (Pareto principle). More than two-thirds of patients with failure to rescue have multiple complications. On average, a few days elapse before death following a complication. A risk-scoring system based on preoperative variables predicts patients in the highest-risk category of failure to rescue with good accuracy. In high-risk patients who develop complications, our results suggest that early intervention, preferably in a high-level intensive care facility with a surgical training program, offers the best chance to reduce failure-to-rescue rates.
Traditional measurements of either surgical complication rates or operative mortality rates are not ideal indicators of surgical quality.1- 4 More than 30 years ago, Donabedian5 recognized that surgical quality is a complex process that is described by an amalgam of outcome, process, and structural measures rather than a single outcome measure. An additional measure of quality is the ability of surgeons and institutions to rescue patients who develop postoperative complications. This metric shows wide variation among surgical programs and correlates with operative mortality.4,6- 10 Inclusion of the inability to rescue patients with postoperative complications (failure to rescue) as part of quality assessment may allow discrimination among high- and low-performing surgical programs and provide opportunities for quality improvement.10- 13
Although many reports identify failure to rescue as a metric of surgical quality, to our knowledge, few reports outline methods to identify and remedy these failures. To use failure to rescue as a quality-improvement tool, an understanding of the factors that predispose to this problem is necessary. The American College of Surgeons (ACS) National Quality Improvement Program (NSQIP) database is a large well-validated surgical database that offers a logical means of identifying patients with failure to rescue.14,15 Furthermore, the robustness of the database provides a preoperative picture of patients having a wide range of surgical procedures and who develop postoperative complications. We used the ACS-NSQIP database to study failure to rescue.
Because most postoperative complications do not result in operative mortality, we hypothesized that the Pareto principle applies to the complex process of failure to rescue. Management consultant Joseph Juran championed the principle that 20% of individuals account for 80% of the outcomes of a complex process. He named this the Pareto principle after Italian economist Vilfredo Pareto, who noticed that only 20% of the pea pods in his garden produced 80% of the peas. Because only about 10% to 20% of patients with complications die of their complications, we hypothesized that most operative mortality following perioperative complications (perhaps as much as 80%) occurs in a small subset of patients with complications (perhaps as little as 10%-20%). A corollary is that preoperative identification of the minority of patients who are likely to develop failure to rescue is possible using the power of the ACS-NSQIP database. Our aim was to create and validate a profile of the subset of patients with complications who will develop failure to rescue. Identification of this cohort provides a focus for quality-improvement efforts to reduce failure-to-rescue rates.
We used the ACS-NSQIP database to identify patients with failure to rescue and to identify clinical predictors of death following development of postoperative complications (failure to rescue).1 The study group consisted of all patients entered into the database between 2008 and 2012. The ACS-NSQIP database contains patient deidentified information available to participants who sign and comply with the ACS-NSQIP Data Use Agreement. The Data Use Agreement uses the data protections of the Health Insurance Portability and Accountability Act of 1996. The University of Kentucky institutional review board determined that this study met federal criteria to qualify as an exempt study. We analyzed the ACS-NSQIP participant use file containing surgical cases submitted by more than 200 acute care hospitals throughout the United States. This database excludes trauma and pediatric patients. We excluded database patients with Current Procedural Terminology codes listed as “procedure not otherwise specified” because of uncertainty in the type of procedures performed and because of the lack of associated work relative value units with the unspecified procedures. For database variables with more than 2% missing values, we used maximization expectation methods to replace missing values.
The ACS-NSQIP database includes more than 50 demographic and preoperative clinical risk variables, as well as 30-day morbidity and mortality variables. We used demographic and preoperative clinical variables to identify risk predictors. To minimize confounding when estimating the failure-to-rescue rates among patients with differing risks for developing postoperative complications, we performed a stratified propensity analysis.16,17 Propensity scores, corrected for all measured preoperative variables, identified each patient’s probability (ie, propensity) of developing a serious postoperative complication. We calculated the propensity scores using logistic regression with all preoperative variables entered into the model. We then divided the study group into 5 strata based on increasing propensity scores to assess the failure-to-rescue rate as the probability of developing serious postoperative complications increased. Each strata contained equal numbers of patients and differed only in the risk for developing serious postoperative complications. Version 22 of SPSS statistical software (IBM) performed these calculations.
Our analysis used outcomes recorded in the ACS-NSQIP database as follows: (1) mortality within 30 days of operation or within the same hospitalization; (2) individual morbidities (1 of 7 serious adverse postoperative events uniformly defined by the ACS-NSQIP18); and (3) composite morbidity consisting of any combination of the 7 serious individual complications. Serious postoperative complications included the following: (1) wound complications (deep organ space surgical site infection, deep surgical wound infection, and wound dehiscence); (2) pulmonary complications including pneumonia, unplanned intubation, pulmonary embolism or deep venous thrombosis, and mechanical ventilation for longer than 48 hours; (3) renal complications including acute kidney injury or dialysis; (4) central nervous system complications including new postoperative stroke or coma; (5) cardiac complications including postoperative myocardial infarction or cardiac arrest; (6) sepsis including postoperative septic shock or bloodborne sepsis; and (7) postoperative bleeding or return to the operating room. Patients with serious postoperative complications who died comprised the failure-to-rescue cohort. The analysis also included the time from the operation to the development of individual complications and the time from the development of complications until death as indices of the time sensitivity of failure to rescue. We used the logistic regression coefficients of the propensity model to create a risk-scoring index. Calculated risk scores were the linear sum of individual regression coefficient variable values. We created a training set of risk scores from a 60% random sample of the study group. The coefficients used to create the training set were then applied to the remaining 40% of the study group to check the accuracy of the prediction model. Comparison of the training set and the validation set of risk scores assessed the adequacy of the risk prediction model.
Of the 1 956 002 patients in the study group, there were 207 236 patients (10.6%) who developed serious postoperative complications. Operative deaths occurred in 21 731 patients with serious complications (10.5% failure to rescue).
The separation of patients into approximately 5 equal-sized groups differing only by their propensity for developing serious postoperative complications resulted in an average group size of 380 869 patients per quintile. Table 1 lists the failure-to-rescue rates in each quintile risk group. Eighty-eight percent of patients (19 117 of 21 731 patients) with failure to rescue were in the highest-risk quintile. Ninety-five percent of patients with failure to rescue (20 613 of 21 731 patients) were in the 2 highest-risk quintiles.
The type of serious postoperative complication affects the failure-to-rescue rate. Cardiac arrest, or perioperative myocardial infarction, had the highest failure-to-rescue rate (45%) (Table 2).
The ACS-NSQIP database records the days from operation until the development of a serious postoperative complication, as well as the days from operation until death. Table 3 lists the average difference between the development of a selected individual complication and eventual death in patients with failure to rescue. With the exception of cardiac arrest, most serious complications occurred up to more than 1 week before operative mortality. A frequent observation from the database was the intervening development of an additional complication following the sentinel problem.
A single postoperative complication often precedes the development of multiple complications. For an individual patient, as the number of complications increases, the failure-to rescue rate increases significantly (Figure 1). The development of a single sentinel postoperative complication is often the primary warning sign of eventual multiple complications and failure to rescue.
Resident involvement in perioperative care in the ACS-NSQIP was only recorded for the first 3 years of the 5-year study. Resident involvement was known for 119 259 patients in the study group. Of these, 79 904 patients had surgical residents involved in their care.
Improvement in failure to rescue occurred with resident involvement, despite significantly increased operative morbidity (11.4% vs 7.8% with attending only; P < .001) and prolonged operative time (127 minutes vs 93 minutes for attending only; P < .001). Resident involvement was associated with reduced failure-to-rescue rates compared with procedures performed by attending surgeons alone (9.4% vs 12.4% for attending alone; P < .001).
We developed a risk-scoring system for the prediction of failure to rescue. The regression coefficients of the predictor variables used to generate the propensity stratification outlined in Table 1 allowed calculation of a risk score for each patient based on the sum of regression coefficients. We used this method to calculate a risk score for a 60% random sample of patients in the database (training data set). We then calculated a risk score in the remaining 40% of patients (experimental data set) using the risk algorithm generated in the training data set. Figure 2 compares the training risk scores with the experimental scores for each complication risk quintile.
Management consultant Joseph Juran championed the principle that 20% of individuals account for 80% of the outcomes of a complex process, calling it the Pareto principle after an Italian economist who made similar observations. We found that 20% of patients with the greatest risk for developing postoperative complications account for roughly 90% of failure to rescue. This is an example of the Pareto principle as it applies to the complex process of postoperative death following the development of complications. These results suggest that considering failure to rescue as a single metric of surgical quality is simplistic. Failure to rescue is a complex process with uncertain causes and undefined solutions. The value of considering and applying the Pareto principle to failure to rescue is that the 20% of patients who account for 80% of the outcomes can be defined with some accuracy. Focusing on this 20% of high-risk patients is likely to have the biggest impact on reducing the failure-to-rescue rate. The Pareto principle concept provides a starting point with the greatest chance of accomplishing positive results.
A survey of the literature suggests that multiple factors predict failure to rescue. We found that failure to rescue depends on multiple preoperative risk factors, as well as the type and number of postoperative complications. Other authors showed that predictors of failure to rescue are different than those for mortality or morbidity.19
Table 3 suggests that aggressive and timely treatment of the initial sentinel complication is the best opportunity to intervene and limit failure to rescue. Most serious complications occur 5 to 10 days before death, suggesting that the window for intervention to rescue patients is limited but potentially amenable to early aggressive treatments.
We found that certain complications portend much higher failure-to-rescue rates. Cardiac events were particularly concerning postoperative complications that had the highest failure-to-rescue rates. Others have found similar results.20 Stroke, renal failure, and pulmonary failure had nearly as great a failure-to-rescue risk as did cardiac events. All of these serious complications often occur sequentially and in combination, making a particularly high-risk set of circumstances.
Measuring failure to rescue combines all 3 of the Donabedian5,21 dimensions of quality: outcome, process, and structural measures. Our results suggest that each of these components plays a role in rescuing patients with postoperative complications. Published reports suggest that structural measures, such as nurse to patient ratios, higher-level intensive care units, and hospital safety culture, play a role in improving failure to rescue.7,22- 29 Furthermore, the use of aggressive and extreme interventions, typically found in higher-acuity institutions, seems to reduce failure-to-rescue rates.25- 29
We found that resident involvement in patients with serious postoperative complications predicted improved failure-to-rescue rates. This suggested a structural contribution to failure to rescue. Resident involvement is likely a surrogate for structural components of hospitals that can improve failure to rescue. Hospitals with improved structural components of quality usually have surgical residency programs. Failure to rescue is lower in hospitals with higher teaching intensity.22,30,31 One study suggested that improved failure-to-rescue rates in resident training programs were not attributable to the presence of residents alone.32 It is interesting that availability of hospital resources does not improve failure to rescue33 and that resident duty-hour limitations did not lead to better surgical outcomes.34 However, it is likely that better intensive care unit care (ie, higher-level intensive care unit availability) provides improved failure-to-rescue rates.35 The exact hospital structural components responsible for improved failure-to-rescue rates require more study.
There is some evidence that subspecialty surgeons may be better at rescuing patients with postoperative complications. A review of the National Inpatient Survey found that general surgeons and thoracic surgeons had equivalent complication rates following esophagectomy; however, thoracic surgeons had decreased mortality and decreased failure-to-rescue rates.22,36
Extreme measures may be necessary to rescue patients with severe complications, for example, the use of extracorporeal circulation membrane oxygenation for refractory cardiopulmonary failure in trauma.37 Aggressive treatment styles predict decreased failure to rescue.38,39 Both aggressive process interventions and high-quality structural components likely improve failure to rescue.
An important and obvious contribution to failure to rescue is actual development of serious postoperative complications. Avoidance of adverse outcomes is an important means of avoiding failure to rescue. Patients without serious postoperative complications have a very low mortality rate (0.4% in our series). This emphasizes the importance of avoidance of complications, something that encompasses a large amount of surgeon focus. Our results suggest that structural and process components of quality deserve equal attention to surgical morbidity. The interesting fact is that failure to rescue encompasses all 3 of these quality metrics.
We found that a minority of high-risk patients accounted for as much as 90% of failure to rescue. This observation is an example of the Pareto principle as it applies to health care. The important consequence of this Pareto phenomena is that the members of the high-risk group who account for the bulk of failure to rescue are targets for making an inroad into reducing failure-to-rescue rates. We developed a risk-scoring system that allows identification of this high-risk cohort. Future studies using preoperative risk identification and a scoring system like the one described in our study can formalize interventions to reduce failure to rescue.
Corresponding Author: Victor A. Ferraris, MD, PhD, University of Kentucky, A301 Kentucky Clinic, 740 S Limestone, Lexington, KY 40536-0284 (firstname.lastname@example.org).
Accepted for Publication: May 6, 2014.
Published Online: September 3, 2014. doi:10.1001/jamasurg.2014.1338.
Author Contributions: Dr Ferraris had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Ferraris, Bolanos, Mahan, Saha.
Acquisition, analysis, or interpretation of data: Ferraris, Bolanos, Martin.
Drafting of the manuscript: Ferraris, Bolanos, Martin.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Ferraris, Martin.
Administrative, technical, or material support: Martin, Mahan, Saha.
Study supervision: Ferraris.
Conflict of Interest Disclosures: Dr Ferraris has served on an advisory board for Haemonetics, participated in a continuing medical education event for CMEology/Baxter Healthcare, and has served as an unpaid consultant for Novo Nordisk. Dr Saha has participated in research trials for CVRx and Baxter HealthCare. No other disclosures were reported.
Disclaimer: The American College of Surgeons National Surgical Quality Improvement Program and the hospitals participating in the database are the source of the data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.
Previous Presentation: This study was presented at the 2014 Association of VA Surgeons Annual Meeting; April 6, 2014; New Haven, Connecticut.