[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.163.94.5. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure.
Plots of Observed and Universal Model–Predicted Rates of Postacute Care Discharge for Current Procedural Terminology Codes
Plots of Observed and Universal Model–Predicted Rates of Postacute Care Discharge for Current Procedural Terminology Codes

Plots of observed and universal model–predicted rates of postacute care discharge for Current Procedural Terminology codes. The dashed limit lines on either side of the diagonal representing perfect agreement are set at ±25%. A, Scatterplot. B, Interquartile ranges around the predicted values for each Current Procedural Terminology code. This represents the theoretical differences in patient factors within each Current Procedural Terminology code.

Table 1.  
Rates of Postacute Care Discharge Overall and by Specialty
Rates of Postacute Care Discharge Overall and by Specialty
Table 2.  
Selected Characteristics of Derivation and Validation Cohorts
Selected Characteristics of Derivation and Validation Cohorts
Table 3.  
Factors Associated With Postacute Care Discharge
Factors Associated With Postacute Care Discharge
1.
Bilimoria  KY, Liu  Y, Paruch  JL,  et al.  Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833-42.e1, 3.
PubMedArticle
2.
Khuri  SF.  The NSQIP: a new frontier in surgery. Surgery. 2005;138(5):837-843.
PubMedArticle
3.
Khuri  SF, Henderson  WG, Daley  J,  et al; Principal Investigators of the Patient Safety in Surgery Study.  Successful implementation of the Department of Veterans Affairs’ National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery study. Ann Surg. 2008;248(2):329-336.
PubMedArticle
4.
Khuri  SF, Henderson  WG, Daley  J,  et al; Principal Site Investigators of the Patient Safety in Surgery Study.  The patient safety in surgery study: background, study design, and patient populations. J Am Coll Surg. 2007;204(6):1089-1102.
PubMedArticle
5.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-346, e1.
PubMedArticle
6.
Merkow  RP, Bentrem  DJ, Cohen  ME,  et al.  Effect of cancer surgery complexity on short-term outcomes, risk predictions, and hospital comparisons. J Am Coll Surg. 2013;217(4):685-693.
PubMedArticle
7.
Hyder  JA, Wakeam  E, Habermann  EB, Hess  EP, Cima  RR, Nguyen  LL.  Derivation and validation of a simple calculator to predict home discharge after surgery. J Am Coll Surg. 2014;218(2):226-236.
PubMedArticle
8.
Oldmeadow  LB, McBurney  H, Robertson  VJ.  Hospital stay and discharge outcomes after knee arthroplasty: implications for physiotherapy practice. Aust J Physiother. 2002;48(2):117-121.
PubMedArticle
9.
Oldmeadow  LB, McBurney  H, Robertson  VJ.  Predicting risk of extended inpatient rehabilitation after hip or knee arthroplasty. J Arthroplasty. 2003;18(6):775-779.
PubMedArticle
10.
Munin  MC, Kwoh  CK, Glynn  N, Crossett  L, Rubash  HE.  Predicting discharge outcome after elective hip and knee arthroplasty. Am J Phys Med Rehabil. 1995;74(4):294-301.
PubMedArticle
11.
de Pablo  P, Losina  E, Phillips  CB,  et al.  Determinants of discharge destination following elective total hip replacement. Arthritis Rheum. 2004;51(6):1009-1017.
PubMedArticle
12.
Baztán  JJ, Suárez-García  FM, López-Arrieta  J, Rodríguez-Mañas  L, Rodríguez-Artalejo  F.  Effectiveness of acute geriatric units on functional decline, living at home, and case fatality among older patients admitted to hospital for acute medical disorders: meta-analysis. BMJ. 2009;338:b50.
PubMedArticle
13.
Inouye  SK, Bogardus  ST  Jr, Baker  DI, Leo-Summers  L, Cooney  LM  Jr; Hospital Elder Life Program.  The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients. J Am Geriatr Soc. 2000;48(12):1697-1706.
PubMedArticle
14.
Chen  CC, Lin  MT, Tien  YW, Yen  CJ, Huang  GH, Inouye  SK.  Modified hospital elder life program: effects on abdominal surgery patients. J Am Coll Surg. 2011;213(2):245-252.
PubMedArticle
15.
Balentine  CJ, Naik  AD, Robinson  CN,  et al.  Association of high-volume hospitals with greater likelihood of discharge to home following colorectal surgery. JAMA Surg. 2014;149(3):244-251.
PubMedArticle
Original Investigation
May 2015

Risk of Discharge to Postacute CareA Patient-Centered Outcome for the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator

Author Affiliations
  • 1Department of Surgery, Henry Ford Hospital, Detroit, Michigan
  • 2Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois
  • 3Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
  • 4Department of Surgery, University of California, Los Angeles
  • 5VA Greater Los Angeles Healthcare System, Los Angeles, California
JAMA Surg. 2015;150(5):480-484. doi:10.1001/jamasurg.2014.3176
Abstract

Importance  Individualized risk prediction tools have an important role as decision aids for use by patients and surgeons before surgery. Patient-centered outcomes should be incorporated into such tools to widen their appeal and improve their usability.

Objective  To develop a patient-centered outcome for the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator, a web-based, individualized risk prediction tool.

Design, Setting, and Participants  Retrospective cohort study using data from the ACS NSQIP, a national clinical data registry. A total of 973 211 patients from July 2010 to June 2012, encompassing 392 hospitals, were used in this analysis.

Main Outcomes and Measures  Risk of discharge to a postacute care setting.

Results  The overall rate of discharge to postacute care was 8.8%. Significant predictors of discharge to postacute care included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.20-2.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). The final model predicted risk of discharge to postacute care with excellent accuracy (C statistic = 0.924) and calibration (Brier score = 0.05).

Conclusions and Relevance  Individualized risk of discharge to postacute care can be predicted with excellent accuracy. This outcome will be incorporated into the ACS NSQIP Surgical Risk Calculator.

Introduction

Individualized surgical risk prediction tools have become an important opportunity for shared decision making in surgical patients. These are tools that allow input of the procedure type as well as individual patient factors, such as demographic characteristics and comorbidities, after which an underlying prediction algorithm outputs predicted risk for a number of postoperative outcomes.1 Currently, most of these tools offer traditional outcomes such as postoperative cardiac events, urinary tract infections, or surgical site infections. Although understanding the risk of these outcomes is important for decision making, there has been recent focus on complementary outcomes that represent the concerns and preferences of patients and are perhaps easier to conceptualize. These patient-centered outcomes allow more informed decision making, particularly when it comes to choosing one of several treatment or intervention options. Their incorporation into prediction tools is essential for widening their appeal and use by surgeons and patients. Discharge to postacute care settings such as acute rehabilitation facilities is an example of one such outcome. It is patient centered, reflecting in part a functional outcome following surgery, and predicting patients at high risk for such a discharge can be important for discharge planning, health system resource allocation, and cost containment.

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator currently allows for entry of 21 preoperative patient factors in addition to the type of procedure and uses regression models to output the individualized risk of 8 surgical outcomes. This universal risk prediction tool leverages a large amount of clinical data and has been shown to provide predictions that are nearly as accurate as those from specialized calculators.1 Since 2011, patient discharge destination has been collected from all ACS NSQIP hospitals. This encompasses both home and postacute care destinations such as skilled nursing facilities, acute rehabilitation facilities, and unskilled nursing homes. We sought to use these data to develop a new outcome for the Surgical Risk Calculator: risk of discharge to a postacute care facility.

Methods
Data Source and Patients

Data were obtained from ACS NSQIP Semiannual Report data sets from July 2010 to June 2012. This program, including its history, purpose, and sampling method, has been detailed extensively in the literature.24 Briefly, ACS NSQIP is a nationally validated, risk-adjusted, outcomes-based program designed to measure and improve the quality of surgical care. It has collected data from more than 500 hospitals in a standardized, reliable fashion on a variety of patient factors, including demographic characteristics and comorbidities, operations, and postoperative outcomes. These data are collected by trained and audited surgical clinical reviewers at each participating hospital. Following the index operation, patients are followed up for 30-day postoperative outcomes. In this study, all patients undergoing operations between July 2010 and June 2012 were included. Because this study did not meet the definition of human research, institutional review board approval and informed consent were not required.

Preoperative Risk Factors and Outcome

Twenty-one preoperative risk factors were used in predicting patient-specific risk of postacute care discharge. These preoperative factors were selected based on predictive value, availability, and face validity. All preoperative factors were made categorical before inclusion into regression models. Risk of postacute care discharge coming purely from the procedure was determined from preliminary models in which Current Procedural Terminology (CPT) code was used as a random effect in a hierarchical model. This generated a continuous variable—a logit-transformed predicted probability of the outcome—unique for each procedure. An additional variable for work relative value units was included to account for procedure complexity.1

The outcome of interest was discharge to postacute care. All nonhome destinations, which included skilled nursing facilities, unskilled nursing facilities, and all acute rehabilitation facilities, were grouped together. Patients who died were excluded, and those who were discharged from the hospital to facilities at which they were living prior to their operation were grouped with patients discharged home.

Statistical Analysis

For this study, two-thirds of the total cohort, randomly sampled without replacement, was used to develop random-intercept, fixed-slope hierarchical models. This accounts for clustering of cases in hospitals and uses an empirical Bayes-type shrinkage adjustment.5 Twenty-one patient factors as well as the operation CPT linear risk score and a work relative value units variable were used for risk prediction.5,6 The final model was validated on the remaining one-third of the cohort. Model performance was evaluated using the C statistic and the Brier score. The C statistic, or area under the receiver operating characteristic curve, is a measure of discrimination, with 0.5 corresponding to discrimination that is no better than chance and 1.0 corresponding to perfect prediction. Finally, the Brier score is both a measure of discrimination and calibration and is defined as the means squared difference between predicted and actual outcomes, with 0.0 indicating perfect model performance.5 After model validation, a final model was developed using the entire cohort for incorporation into the risk calculator. All data manipulation and statistical analysis were performed using SAS version 9.4 statistical software (SAS Institute, Inc).

Results

There were 973 211 cases from 392 hospitals in the final analysis. These encompassed all surgical specialties except transplant and trauma. The overall event rate of discharge to postacute care was 8.8%. This varied by specialty, with the highest rates in patients undergoing orthopedic (24.5%), cardiac (20.3%), and vascular (16.3%) procedures (Table 1).

In aggregate, patients discharged home were younger and healthier, with fewer comorbidities and better functional status. The derivation (n = 643 211) and validation (n = 330 000) cohorts were not significantly different in terms of their demographic characteristics or comorbidities (Table 2).

The most significant predictors associated with continuing care discharge, adjusted for procedure type and complexity, included being 85 years or older (odds ratio [OR] = 9.17; 95% CI, 8.84-9.50), the presence of septic shock (OR = 2.43; 95% CI, 2.20-2.69) or ventilator dependence (OR = 2.81; 95% CI, 2.56-3.09) preoperatively, American Society of Anesthesiologists class of 4 or 5 (OR = 3.59; 95% CI, 3.46-3.71), and totally dependent functional status (OR = 2.27; 95% CI, 2.11-2.44). All predictors are shown in Table 3. In addition to these patient factors, the final model included emergent case status, the presence of dyspnea, renal failure, chronic obstructive pulmonary disease, congestive heart failure, body mass index, steroid use, smoking status, hypertension, diabetes mellitus, recent cardiac event, wound class, the presence of locally invasive cancer, and whether the patient was on dialysis preoperatively. Model fit and calibration, as measured with the C statistic and Brier score, were excellent (C statistic = 0.924; Brier score = 0.05).

The relationship between the observed and expected (predicted) rates of postacute care discharge for 498 CPT codes for the continuing care discharge outcome in the validation cohort is shown in the Figure both as a scatterplot and with interquartile ranges. This illustrates that the model for predicting continuing care discharge is reasonably accurate, with each point representing a CPT clustering around the diagonal representing perfect agreement. In theory, the interquartile ranges represent the perceived differences among patients undergoing the same procedure.

Discussion

In this study, we developed a tool that focuses on an outcome easily conceptualized by patients—the risk of discharge to postacute care—and incorporated it into the universal, patient-specific ACS NSQIP Surgical Risk Calculator. This calculator allows the entry of 21 patient factors as well as the procedure and outputs the risk of several adverse postoperative outcomes, allowing a frank discussion between surgeon and patient about the risks of surgery. Importantly, it may also allow hospitals to predict resource utilization more accurately.

Recent work by Hyder et al7 has demonstrated the feasibility of an individual risk calculator designed to predict home discharge after surgery. Their calculator focused on usability and questions that can be answered relatively quickly and easily by the patient; excluded all joint replacements; and showed good discrimination and calibration. In contrast, the ACS NSQIP Surgical Risk Calculator is designed to be used by the surgeon and has a preexisting user interface that facilitates the consideration and use of numerous patient factors. In addition, we also incorporated information about the procedure into our risk prediction, and we included all joint replacements, which account for the largest proportion of nonhome discharges in the ACS NSQIP.

Predicting discharge to postacute care settings is particularly important in orthopedic surgery. Several investigators have examined patient factors that predict discharge to either a rehabilitation facility or skilled nursing facility. Most studies are single-institution or small multicenter studies using claims data that have identified age, increased American Society of Anesthesiologists class, poor functional status and/or use of a gait aid, obesity, and increased comorbidity burden as important predictors.811 Our study similarly found that total joint replacements and hip fracture operations had the highest risk of discharge to postacute care and that the same patient factors were the most significant risk factors.

Discharge to a nursing or acute rehabilitation facility is easily understood as a kind of functional outcome. Some authors have suggested that, in the absence of a medical reason, patient preferences play an important role in discharge to rehabilitation following joint replacement.9 Ideally, such preferences are guided by objective risk information, which this tool provides, based on the procedure and patient comorbidity burden. This allows an honest framing of expectations for recovery and return to independence. In addition, this information could give confidence to decision making during a preoperative appointment or during the informed consent process.

At the hospital or health system level, knowledge of this information would allow for a number of strategies that could potentially improve the quality and efficiency of recovery pathways and discharge planning in elective surgery. For example, preoperative risk stratification could identify a high-risk cohort for whom early, evidence-based interventions designed to prevent functional decline could be implemented.1214 This cohort might warrant earlier or more aggressive involvement of physical and occupational therapy. Case managers and social workers might also be able to complete earlier needs assessment and skilled nursing or acute rehabilitation eligibility evaluation.

The results of this study should be considered in light of several limitations. First, our study was limited to ACS NSQIP hospitals, which are typically larger and have a focus on quality improvement. Thus, these hospitals may be different from the rest of the hospitals in the United States regarding use of postacute care facilities. Second, this risk prediction does not incorporate information about postoperative adverse outcomes that likely affect the decision to send a patient to postacute care. This information is not available prospectively, however, and it would be important to communicate this in any discussions with patients. Finally, this outcome only takes into account patient factors and the procedure and does not account for practice variation, socioeconomic factors, home support, and hospital characteristics with regard to use of postacute care. Recent work has shown that inpatient surgical volume or other proxies for hospital size and resource availability affect the use of postacute care.15 With that said, when using the ACS NSQIP Surgical Risk Calculator, the surgeon is free to modify risk estimates in accordance with local conditions through use of the surgeon adjustment score.1

Conclusions

Using a large clinical data set, we developed a model with excellent discriminatory ability and calibration that allows determination of the risk of discharge to postacute care as a new outcome for the ACS NSQIP Surgical Risk Calculator. This will undoubtedly prove useful to patients and hospitals when planning for surgery and postoperative care.

Back to top
Article Information

Corresponding Author: Sanjay Mohanty, MD, Division of Research and Optimal Patient Care, American College of Surgeons, 633 N St Clair, 22nd Floor, Chicago, IL 60611 (smohanty@facs.org).

Accepted for Publication: October 1, 2014.

Published Online: March 25, 2015. doi:10.1001/jamasurg.2014.3176.

Author Contributions: Dr Mohanty had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Mohanty, Paruch, Ko, Bilimoria.

Acquisition, analysis, or interpretation of data: Mohanty, Liu, Kmiecik, Cohen, Bilimoria.

Drafting of the manuscript: Mohanty, Bilimoria.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Mohanty, Liu, Paruch, Cohen.

Administrative, technical, or material support: Mohanty, Kmiecik.

Study supervision: Ko, Bilimoria.

Conflict of Interest Disclosures: None reported.

Funding/Support: Dr Mohanty is supported by the Clinical Scholar in Residence Program from the American College of Surgeons and by the American Geriatric Society and the Hartford Foundation.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References
1.
Bilimoria  KY, Liu  Y, Paruch  JL,  et al.  Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833-42.e1, 3.
PubMedArticle
2.
Khuri  SF.  The NSQIP: a new frontier in surgery. Surgery. 2005;138(5):837-843.
PubMedArticle
3.
Khuri  SF, Henderson  WG, Daley  J,  et al; Principal Investigators of the Patient Safety in Surgery Study.  Successful implementation of the Department of Veterans Affairs’ National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery study. Ann Surg. 2008;248(2):329-336.
PubMedArticle
4.
Khuri  SF, Henderson  WG, Daley  J,  et al; Principal Site Investigators of the Patient Safety in Surgery Study.  The patient safety in surgery study: background, study design, and patient populations. J Am Coll Surg. 2007;204(6):1089-1102.
PubMedArticle
5.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-346, e1.
PubMedArticle
6.
Merkow  RP, Bentrem  DJ, Cohen  ME,  et al.  Effect of cancer surgery complexity on short-term outcomes, risk predictions, and hospital comparisons. J Am Coll Surg. 2013;217(4):685-693.
PubMedArticle
7.
Hyder  JA, Wakeam  E, Habermann  EB, Hess  EP, Cima  RR, Nguyen  LL.  Derivation and validation of a simple calculator to predict home discharge after surgery. J Am Coll Surg. 2014;218(2):226-236.
PubMedArticle
8.
Oldmeadow  LB, McBurney  H, Robertson  VJ.  Hospital stay and discharge outcomes after knee arthroplasty: implications for physiotherapy practice. Aust J Physiother. 2002;48(2):117-121.
PubMedArticle
9.
Oldmeadow  LB, McBurney  H, Robertson  VJ.  Predicting risk of extended inpatient rehabilitation after hip or knee arthroplasty. J Arthroplasty. 2003;18(6):775-779.
PubMedArticle
10.
Munin  MC, Kwoh  CK, Glynn  N, Crossett  L, Rubash  HE.  Predicting discharge outcome after elective hip and knee arthroplasty. Am J Phys Med Rehabil. 1995;74(4):294-301.
PubMedArticle
11.
de Pablo  P, Losina  E, Phillips  CB,  et al.  Determinants of discharge destination following elective total hip replacement. Arthritis Rheum. 2004;51(6):1009-1017.
PubMedArticle
12.
Baztán  JJ, Suárez-García  FM, López-Arrieta  J, Rodríguez-Mañas  L, Rodríguez-Artalejo  F.  Effectiveness of acute geriatric units on functional decline, living at home, and case fatality among older patients admitted to hospital for acute medical disorders: meta-analysis. BMJ. 2009;338:b50.
PubMedArticle
13.
Inouye  SK, Bogardus  ST  Jr, Baker  DI, Leo-Summers  L, Cooney  LM  Jr; Hospital Elder Life Program.  The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients. J Am Geriatr Soc. 2000;48(12):1697-1706.
PubMedArticle
14.
Chen  CC, Lin  MT, Tien  YW, Yen  CJ, Huang  GH, Inouye  SK.  Modified hospital elder life program: effects on abdominal surgery patients. J Am Coll Surg. 2011;213(2):245-252.
PubMedArticle
15.
Balentine  CJ, Naik  AD, Robinson  CN,  et al.  Association of high-volume hospitals with greater likelihood of discharge to home following colorectal surgery. JAMA Surg. 2014;149(3):244-251.
PubMedArticle
×