Role of Operative Complexity Variables in Risk Adjustment for Patients With Cancer | Targeted and Immune Cancer Therapy | JAMA Surgery | JAMA Network
[Skip to Navigation]
Sign In
Figure.  Change in Akaike Information Criterion (AIC) During Stepwise Selection of Variables
Change in Akaike Information Criterion (AIC) During Stepwise Selection of Variables

The Oncology NSQIP National Cancer Center Collaborative (ONNCC) variables were either available or not available for selection; the lower values are better. Among all models evaluated, only a single ONNCC variable (Previous Radiation Therapy) was selected: at step 7 for readmission (Readmit [blue arrow]) and at step 10 for death or serious morbidity (DSM [orange arrow]). NSQIP indicates National Surgical Quality Improvement Program.

Table.  Outcomes by Oncology NSQIP National Cancer Center Collaborative Variablesa
Outcomes by Oncology NSQIP National Cancer Center Collaborative Variablesa
1.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.  J Am Coll Surg. 2013;217(2):336-346.e1. PubMedGoogle ScholarCrossref
2.
ACS NSQIP. Collaboratives. American College of Surgeons website. https://www.facs.org/quality-programs/acs-nsqip/participants/collaboratives. Accessed March 12, 2016.
3.
American College of Surgeons.  ACS NSQIP Semiannual Report. Chicago, IL: American College of Surgeons; 2015.
4.
Merkow  RP, Kmiecik  TE, Bentrem  DJ,  et al.  Effect of including cancer-specific variables on models examining short-term outcomes.  Cancer. 2013;119(7):1412-1419.PubMedGoogle ScholarCrossref
5.
Meguid  RA, Bronsert  MR, Juarez-Colunga  E, Hammermeister  KE, Henderson  WG.  Surgical Risk Preoperative Assessment System (SURPAS): II, parsimonious risk models for postoperative adverse outcomes addressing need for laboratory variables and surgeon specialty-specific models.  Ann Surg. 2016;264(1):10-22. PubMedGoogle ScholarCrossref
Research Letter
November 2016

Role of Operative Complexity Variables in Risk Adjustment for Patients With Cancer

Author Affiliations
  • 1Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois
  • 2Department of Surgery, University of Chicago Hospitals, Chicago, Illinois
  • 3Department of Surgery, University of Wisconsin Carbone Cancer Center, Madison
  • 4Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles
  • 5Veterans Affairs Greater Los Angeles Healthcare System, Los Angeles, California
  • 6Department of Surgery, Surgical Outcomes and Quality Improvement Center, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
JAMA Surg. 2016;151(11):1084-1086. doi:10.1001/jamasurg.2016.2253

The comprehensive capture of relevant details of patients is essential to accurately predict outcomes and benchmark hospital performance.1 Members of the Oncology NSQIP National Cancer Center Collaborative (ONNCC), established in 2011, were concerned that important oncology-related variables were not captured by the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The ONNCC began collecting 3 additional case characteristics representing greater operative complexity and risk with the aim to improve patient risk prediction and case-mix adjustment.2

Methods

Trained surgical clinical reviewers from 16 ONNCC hospitals abstracted 3 novel variables not routinely collected in ACS NSQIP (“Previous Surgery in the Operative Region,” “Previous Radiotherapy to the Operative Region,” and “Previous Chemotherapy”). All variables were stratified by time from surgery (≥90 days vs <90 days). For example, consider a patient with a distant history of prostatectomy and adjuvant pelvic radiotherapy now undergoing low anterior resection for rectal cancer. The operative region (the pelvis) will be reentered; thus, the data coded for this patient is “Previous Surgery: ≥90 days,” “Previous Radiotherapy: ≥90 days,” and “Previous Chemotherapy: None.”

Five outcomes were modeled (death or serious morbidity [DSM], any surgical site infection, organ-space surgical site infection, reoperation, and readmission).3 Unadjusted odds ratios were calculated for each ONNCC variable. Based on standard ACS NSQIP modeling, multivariable analyses were then conducted for each outcome using forward selection.1,4 If forward selection identified an ONNCC variable among the 45 standard ACS NSQIP variables, modeling was repeated without the ONNCC variable. Changes in the C statistic and Akaike information criterion (AIC) were assessed at each step to quantify the degree to which each variable added explanatory power to the model, paying particular attention to the contribution of the ONNCC variables. This study was deemed exempt by the institutional review board of Northwestern University.

Results

In unadjusted analyses, “Previous Surgery” was not associated with any of the studied outcomes. “Previous Chemotherapy” was associated with greater odds of DSM (unadjusted odds ratio [OR], 1.34 [95% CI, 1.21-1.49]), reoperation (unadjusted OR, 1.75 [95% CI, 1.30-2.36]), and readmission (unadjusted OR, 1.40 [95% CI, 1.13-1.71]) but only greater than or equal to 90 days from the current operation. “Previous Radiotherapy” was associated with greater odds of DSM (unadjusted OR, 1.12 [95% CI, 0.91-1.37] for <90 days and 1.44 [95% CI, 1.23-1.69] for ≥90 days), reoperation (unadjusted OR, 1.71 [95% CI, 1.19-2.45] for <90 days and 1.96 [95% CI, 1.42-2.72] for ≥90 days), or readmission (unadjusted OR, 1.32 [95% CI, 1.09-1.60] for <90 days and 1.52 [95% CI, 1.28-1.81] for ≥90 days) regardless of when the patients had previous radiation therapy (Table).

The “Previous Surgery” and “Previous Chemotherapy” variables were not selected for inclusion in any of the models during multivariable analyses. The “Previous Radiotherapy” variable was selected for inclusion only in the DSM and reoperation models at steps 10 and 7, respectively. Inclusion of the ONNCC variables increased the AIC for DSM by 0.06% (AIC of 7077.69 with; AIC of 7073.31 without), while the AIC for reoperation decreased by 0.11% (AIC of 5939.10 with; AIC of 5945.50 without) (Figure). The C statistic for every model correspondingly increased as the AIC decreased for each step.

Discussion

To accurately predict outcomes and risk adjust hospital comparisons, it is important to consider and evaluate new clinically relevant variables. We found that complex cancer-related variables were unable to improve ACS NSQIP modeling.1 Prior studies combining oncologic data from the National Cancer Data Base with ACS NSQIP data did show some ability to improve patient-level risk prediction, but there was no effect on hospital-level comparisons.4 Despite some bivariate associations being significant, we found that the new ONNCC variables did not affect patient-level comparisons and would correspondingly have no effect on hospital-level comparisons. Thus, currently abstracted ACS NSQIP variables provide sufficient adjustment for short-term outcomes of surgical patients undergoing complex cancer operations. Although the ONNCC variables represent clinical scenarios with the potential for poor postoperative outcomes, they do not provide additional statistical explanatory power to justify the added effort required to abstract them given the lack of modeling benefit. The desire for clinical specificity when building predictive models must be weighed against statistical parsimony in the face of data collection burden.5

Back to top
Article Information

Corresponding Author: Jason B. Liu, MD, Division of Research and Optimal Patient Care, American College of Surgeons, 633 N St Clair, 22nd Floor, Chicago, IL 60611 (jliu@facs.org).

Published Online: August 3, 2016. doi:10.1001/jamasurg.2016.2253.

Author Contributions: Dr Liu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Liu, Weber, Berian, Ko, Bilimoria.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Liu, Weber, Ko.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Liu, Berian, Chen, Cohen, Ko.

Obtained funding: Bilimoria.

Administrative, technical, or material support: Berian.

Study supervision: Berian, Cohen, Bilimoria.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was funded by the Northwestern Institute for Comparative Effectiveness Research in Oncology. Dr Liu’s position is supported by the American College of Surgeons Clinical Scholars in Residence Program and by a research fellowship from the Department of Surgery, University of Chicago. Dr Berian is the John A. Hartford Foundation James C. Thompson Geriatric Surgery Research Fellow whose position is supported by the John A. Hartford Foundation and by the American College of Surgeons Clinical Scholars in Residence Program.

Role of the Funder/Sponsor: The American College of Surgeons had a role in the collection and management of the data but had no role in the design and conduct of the study; analysis or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge and thank the 16 centers participating in this pilot study, the individual surgeons representing their institutions in the ONNCC, and especially the surgical clinical reviewers for data collection.

References
1.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.  J Am Coll Surg. 2013;217(2):336-346.e1. PubMedGoogle ScholarCrossref
2.
ACS NSQIP. Collaboratives. American College of Surgeons website. https://www.facs.org/quality-programs/acs-nsqip/participants/collaboratives. Accessed March 12, 2016.
3.
American College of Surgeons.  ACS NSQIP Semiannual Report. Chicago, IL: American College of Surgeons; 2015.
4.
Merkow  RP, Kmiecik  TE, Bentrem  DJ,  et al.  Effect of including cancer-specific variables on models examining short-term outcomes.  Cancer. 2013;119(7):1412-1419.PubMedGoogle ScholarCrossref
5.
Meguid  RA, Bronsert  MR, Juarez-Colunga  E, Hammermeister  KE, Henderson  WG.  Surgical Risk Preoperative Assessment System (SURPAS): II, parsimonious risk models for postoperative adverse outcomes addressing need for laboratory variables and surgeon specialty-specific models.  Ann Surg. 2016;264(1):10-22. PubMedGoogle ScholarCrossref
×