A Novel Risk-Adjusted Nomogram for Rectal Cancer Surgery Outcomes

Importance  The circumferential resection margin is the primary determinant of local recurrence and a major factor in survival in rectal cancer. Neither chemotherapy nor chemoradiation compensates for a margin positive for cancer.

Objective  To identify treatment-related factors associated with hospital margin-positive resection and to develop a tool that could be used by individual hospitals to assess their outcomes based on their unique mix of patient and tumor characteristics.

Design  Retrospective review of the National Cancer Data Base, 1998-2007.

Settings  Community and academic/research hospitals.

Participants  Individuals with histologically confirmed localized rectal/rectosigmoid adenocarcinoma.

Exposure  All individuals underwent radical resection for rectal cancer with or without neoadjuvant therapy.

Main Outcomes and Measures  Rate of margin positivity determined and adjusted for patient- and tumor-related factors to calculate expected margin positivity per hospital. An observed to expected ratio was calculated based on patient- and tumor-related factors to identify treatment-associated variation.

Results  The overall margin-positive resection rate was 5.2%. Patients with margins positive for cancer were more likely to be older, male, and African American; not have private insurance; and have their cancer diagnosed later in the study period. Associated tumor-related factors include rectal location, higher American Joint Committee on Cancer stage, signet/mucinous histology, and poor/undifferentiated grade. Among hospitals that were significantly low outliers, 47% were comprehensive community hospitals, and 43.9% were academic/research hospitals; of those that were significantly high outliers, 52.3% were comprehensive community hospitals, and 17.8% were academic/research hospitals. High-volume centers made up 80% of significantly low outlier hospitals and 17% of significantly high outlier hospitals. The rates of chemotherapy and radiation were similar, but low outlier hospitals gave more neoadjuvant radiation (26.3% vs 17%).

Conclusions and Relevance  After adjustment for patient- and tumor-related factors, we identified both low and high outlier hospitals for margin positivity at resection, as well as potentially modifiable risk factors. The nomogram created in this model allows for the evaluation of observed and expected event rates for individual hospitals, providing a hospital self-assessment tool for identifying targets for improvement.

In 2012, there were an estimated 40 290 cases of rectal cancer diagnosed in the United States. Colorectal cancer is the second leading cause of US cancer death, with rectal cancer accounting for approximately one-third of colorectal cancer deaths.¹ For cases without metastases, the primary treatment is radical resection with proctectomy and en bloc radical lymphadenectomy; for locally advanced cases, primary treatment is perioperative radiation and chemotherapy. The widespread adoption of sharp total mesorectal excision has improved local control and survival by improving radical lymph node clearance and decreasing the risk for a surgical margin positive for cancer (hereafter referred to as a positive margin). Accordingly, local recurrence rates have decreased to 5% to 10% from as high as 50%.^2-4 Circumferential resection margin status has been identified as one of the most important determinants of local recurrence risk. Positive resection margins are associated with metastatic disease and decreased survival.^5,6 Unfortunately, neither radiation nor chemotherapy can compensate for a positive margin in rectal cancer.^7,8 Therefore, the achievement of a negative resection margin is a primary goal of surgery for rectal cancer.

A number of tumor-related factors are associated with the risk for margin positivity at rectal cancer resection. Tumor location in the distal third of the rectum, T4 status, advanced local nodal stage, need for abdominoperineal resection, and anteriorly located tumors are associated with a higher risk of margin positivity. Patient- and treatment-related factors have also been identified, including American Joint Committee on Cancer (AJCC) stage and advanced age.^7,9-14

Factors influencing the risk for margin positivity at resection can be categorized as patient-, tumor-, and treatment-related factors. To further investigate these relationships, we used a large hospital-based national database to examine risk factors for margin-positive resection of rectal cancer. We specifically sought to identify treatment-related factors associated with margin-positive resection and to develop a tool that could be used by individual hospitals to assess their outcomes based on their unique mix of patient and tumor characteristics.

Data from the National Cancer Data Base (NCDB) were used for our study. The NCDB is a joint program of the Commission on Cancer of the American College of Surgeons and the American Cancer Society and is used as a surveillance tool to assess patterns of care for patients with cancer.¹⁵ More than 1500 cancer programs in the United States are accredited by the Commission on Cancer of the American College of Surgeons; the NCDB captures approximately 76% of newly diagnosed cancer cases. Hospitals contributing to the NCDB are classified as academic/research hospital cancer programs, comprehensive cancer programs, or community hospital cancer programs. Data collected for each cancer case include patient characteristics, cancer staging, tumor characteristics, types of treatment administered, and outcomes.

Rectal or rectosigmoid cancers (International Classification of Disease for Oncology codes C199 and C209) were identified within the NCDB between 1998 and 2007 (Figure 1). All patients with histologically confirmed adenocarcinoma who had undergone a partial or complete proctectomy (excluding cases of local tumor excision/destruction), were 18 to 90 years of age, and had received no prior cancer diagnosis were included. Patients who died within 30 days of cancer diagnosis, had stage IV disease, or had an unknown margin status were excluded. Finally, hospitals reporting a margin-positive rate of 0 were excluded because they may have been subject to either very low procedure volume or coding discrepancies that precluded analysis.

We analyzed data on the basis of patient-, tumor-, and treatment/hospital-related variables. The primary outcome measure was the individual hospital rate of margin-positive resection adjusted for patient- and tumor-related risk factors. Demographic data included patient age (categorized in groups as 18-49, 50-75, and 76-90 years), sex, race, year of diagnosis, insurance status, median quartile income, percentage of residents in the patient’s neighborhood without a high school diploma, and area-based measures of residence as metropolitan, urban, or rural. The patients’ educational and income levels were assessed by matching patients’ zip codes to files derived from 2000 US Census data coded as quartiles. Tumor characteristics included anatomic location (rectosigmoid vs rectum), AJCC (sixth edition) tumor stage, tumor size, histology, and grade. Treatment-related variables included type of facility (community cancer program, comprehensive community cancer program, or academic/research program), yearly hospital surgical volumes categorized by quartiles, receipt of chemotherapy or radiotherapy and the sequence of treatment, and specific type of surgery (low anterior resection, coloanal anastomosis, or abdominoperineal resection).

We sought to ascertain the hospital rate of margin positivity after adjustment for patient- and tumor-related risk factors in order to identify potentially modifiable treatment-related factors. We first developed a patient-level multivariable risk model for margin positivity using multiple logistic regression analysis incorporating important patient- and tumor-related clinical variables, leaving the treatment-related factors unadjusted in the model for post hoc comparison.

This risk model was used to calculate the expected number of patients with a positive margin for each hospital by summing all patients’ risk estimates by hospital. The observed number of margin-positive cases was then divided by the expected number of events (based on patient- and tumor-related factors) to create an observed to expected (O/E) ratio for each hospital. To estimate the significance of the O/E ratio, the probability (P) that a hospital had exactly the observed number of events (X = k) was determined based on the binomial function P(X = k), where k denotes the actual number of events observed within the hospital, n denotes the hospital volume, and P denotes the model-derived expected probability of the event.¹⁶ Finally, hospital- and treatment-related factors were compared on the basis of statistically significant outlier status.

A nomogram was constructed using multivariate logistic regression analysis and internally validated for model discrimination and calibration by bootstrapping with 200 resamples. Model discrimination was first quantified using the concordance index to measure the predictive accuracy of the model by analyzing all possible pairs of patients. After quantifying the model discrimination, the model calibration was graphically assessed using a calibration plot.¹⁷

All statistical analyses were performed using Stata MP software version 11.0 (release 2010), and the R software (www.r-project.org/) with the RMS package was used to construct the nomogram. Because the analysis used preexisting data with no personal identifiers, it was exempt from review by our institutional review board.

A total of 85 190 patients (treated in 1064 hospitals) met the study criteria and were eligible for analysis. The identification of cases and a flowchart of the analysis are shown in Figure 1. Demographic and tumor data for all patients are summarized in our Table. The overall rate of margin-positive resections for the entire cohort was 5.2%. Patients with a margin-positive resection were more likely to be older, male, and African American; to have their disease diagnosed in the more recent years of the study period (2004-2007); and to not have private insurance. Other demographic factors, including median income quartile, educational level, and residence (rural vs urban), were not associated with risk of margin positivity.

Baseline patient and tumor characteristics and margin status are summarized in our Table. Tumors located in the rectum with a higher AJCC stage (final pathology stage), signet or mucinous histology, or classified as poorly differentiated or undifferentiated were associated with a higher risk for positive margin. Tumor size was not associated with margin status.

An adjusted risk model for margin positivity was developed based on patient and tumor characteristics. Nearly 64% of patients in the patient sample were between 40 and 75 years of age. The margin-positive rate was lowest in this group. On multivariate analysis, patient-related factors associated with a small to moderate increased risk for margin positivity (adjusted odds ratio [AOR] 1.0-1.50) included age older than 75 years, male sex, African American race, and more recent year of diagnosis (2004-2005 and 2006-2007). A stronger association was observed with lack of private insurance with a 63% relative increase in margin positivity (8.5%; AOR, 1.54 [95% CI, 1.29-1.84]).

Tumor-specific factors were evaluated on the final pathologic specimen. Factors significantly associated with a moderately increased risk for margin positivity included poorly differentiated grade and tumor location in the rectum. Stronger associations were observed for signet ring cell or mucinous histologic type, and more advanced T or N category. The highest rates of margin-positive resection were associated with AJCC stage IIB tumors (margin-positive rate, 23.6%; AOR, 21.8 [95% CI, 18.74-25.36] vs stage I tumors) and signet ring histology (22.4%; AOR, 2.49 [95% CI, 2.02-3.07] vs non–signet ring, nonmucinous adenocarcinomas) (Table). Notably, 24% of patients with stage IIB cancer received neoadjuvant radiation, and 20.3% of these patients had a positive margin at resection. As expected, the margin-positivity rate for the adjuvant group was higher at 31.5%.

The O/E ratio of margin-positive resection was determined for individual hospitals (Figure 2) and plotted logarithmically to compare the ratios of observed to expected margin-positive resection rates according to the model-derived expected probabilities. Hospitals with an O/E ratio of 1 had exactly the same number of observed margin-positive cases as expected according to patient and tumor characteristics. Those with an O/E ratio of more than 1 had a greater number of observed margin-positive tumors than expected; those with an O/E ratio of less than 1 had fewer observed margin-positive tumors than expected. These were further designated as significantly high (referred to as “high outliers”) and significantly low outliers (referred to as “low outliers”) on the basis of the probability of H₀: O = E, defined by P < .05. This allowed us to make comparisons among the hospitals on the basis of outlier status, taking into account hospital characteristics and treatment patterns.

Of the total 1064 hospitals examined, 173 were significant low or high outliers. Of the low outliers (significantly fewer than expected margin-positive resections), 9.1% were community cancer programs, 47% were comprehensive community hospitals, and 43.9% were academic/research hospitals (Figure 3). Of the high outlier hospitals (significantly greater than expected margin-positive resections), 29.9% were community cancer programs, 52.3% were comprehensive community hospitals, and 17.8% were academic/research institutions.

Hospitals were stratified by yearly surgical volume into quartiles, and differences in positive-margin rates between significantly high- and low-volume hospitals were assessed (Figure 3). Among the low outlier hospitals, more than 80% were the highest quartile for volume (10.7-98.9 cases per year). No hospitals from the lowest quartile for volume (1.0-4.4 cases per year) were low outliers. However, high volume did not preclude high outlier status because 17.8% of the high margin-positive outliers were high-volume hospitals, whereas the remaining hospitals were fairly evenly distributed among mid–high-volume (32.7%), mid–low-volume (23.4%), and low-volume (26.2%) quartiles.

The treatment-related variables assessed included receipt of chemotherapy or radiation therapy and their sequence. We also examined type of surgery (low anterior resection, coloanal anastomosis, or abdominoperineal resection) to examine differences between the significant outlier groups. Rates of chemotherapy (54.1% vs 54%) or radiation therapy (46.5% vs 45.8%) between the low and high outlier hospitals did not differ meaningfully. However, there were notable differences in the sequence of surgery and radiation. Among low outlier hospitals, 56.6% of the patients who received radiation received it prior to surgery, whereas among high outlier hospitals, only 37.3% of patients received radiation prior to surgery. Finally, there was little difference between the low and high outlier groups with regard to specific operation. The majority underwent a low anterior resection, whereas coloanal anastomosis was a more common procedure in the low outlier hospitals (8.5%) than in the high outlier hospitals (5.5%).

We used the logistic regression model (Table) to construct a nomogram for determining the expected rate of margin positivity for a given hospital (Figure 4). The nomogram has a bootstrap-corrected concordance index of 0.7474 (eFigure in Supplement), indicating good performance. The calibration plot demonstrates that the predicted probability derived from the nomogram corresponds well with the observed probability and is slightly overestimated only when greater than 0.3. This nomogram can thus be used to determine an individual hospital’s expected rate of margin-positive resection for comparison with its observed rate within the NCDB.

Resection margin status is a primary determinant of long-term oncologic outcome among patients with rectal cancer. In our study, we used the NCDB to investigate the relationship between patient-, tumor-, and treatment-related variables and the risk for margin positivity during resection for rectal cancer. The analysis demonstrates a number of patient-related risk factors for margin-positive resection, including older age, male sex, African American race, and public insurance. Tumor-related risk factors include more advanced AJCC stage (especially T category), signet or mucinous histology, and poor/undifferentiated grade.

Using the adjusted risk model, we determined the ratio of observed to expected rates of margin positivity and identified 173 of 1064 hospitals that were significant high or low outliers. High outlier status was associated with a lower rate of neoadjuvant radiotherapy, lower hospital volume, and nonacademic/research hospital type, whereas low outlier status was associated with a higher rate of neoadjuvant radiotherapy, high hospital volume, and academic/research hospital type. However, it was highly notable that neither academic/research hospital type nor high hospital volume precluded high outlier status.

The historically high rates of local failure following resection of rectal cancer were dramatically improved with the widespread adoption of the sharp total mesorectal excision technique.^4,7,18,19 Total mesorectal excision improves the clearance of regional lymph nodes and the achievement of negative circumferential resection margins. Previous studies^14,20-22 of risk factors for circumferential resection margin positivity have focused on tumor-related factors such as T and N categories, tumor size and location, vascular invasion, and the need for abdominoperineal resection. Other studies^23-25 have reported associations between higher provider and hospital volumes and improved outcomes for oncologic procedures. In the present analysis, we have identified additional patient- and tumor-related factors associated with an underlying risk for margin-positive resection. In addition, we have identified the use of preoperative radiotherapy and treating facility characteristics as important treatment-related factors accounting for outlier status and significant variance from risk-adjusted expected rates of margin positivity. We then developed a nomogram that individual hospitals can use to determine their own risk-adjusted expected margin-positivity rates. With this nomogram, individual hospital-based cancer programs can use their own previously abstracted data to determine their expected rate of margin-positive resection and compare it with the benchmark-expected rate based on their individual patient and tumor mix. If significant variance is identified, it can alert the cancer program to consider quality improvement initiatives to decrease their rates of margin positivity.

Resection margins have a significant effect on both local recurrence and disease-free survival.^5,6,26-28 Neither preoperative radiotherapy nor adjuvant chemotherapy can compensate for margin positivity; however, randomized trials^29-31 have demonstrated that neoadjuvant radiotherapy can improve local control. Furthermore, preoperative combined modality chemotherapy and radiation has been associated with improved local control, margin-negative resection, and the potential for sphincter preservation and disease-free survival compared with postoperative therapy.^31-33 Our analysis indicates that patients who receive preoperative radiation therapy are more likely to have been treated in a low outlier hospital and that those who receive postoperative therapy are more likely to have been treated in a high outlier hospital, thus identifying a simple, key, potentially modifiable treatment-related factor that may help hospitals improve their negative resection margin rates.

In this analysis, we have focused on comparing the ratio of the observed to expected rates of margin positivity within the NCDB. Outlier status was not exclusively predicted by hospital volume, nor by academic/research or nonacademic/research hospital type (ie, no hospital type was exempt from high outlier status, nor could a hospital be ensured low outlier status). Thus, the analysis is applicable and relevant for all types of individual hospitals that want to compare their observed rate to their risk-adjusted expected rate. All facility types may have factors in their practice that may be targeted to improve rectal cancer outcomes. High outlier hospitals can look for modifiable treatment-related factors to alter their clinical practice to improve rates and outcomes. Using the O/E ratio allows a hospital to evaluate itself while controlling for its mix of patient- and tumor-related risk factors. We have created a simple-to-use nomogram to facilitate this evaluation.

Our study has important limitations, as well as strengths. Analysis of a large database is subject to data constraints, incorrect or missing data, and an inability to account for pathologic or surgical variability. The Commission on Cancer does conduct abstraction training, as well as annual quality-control audits, to minimize these errors in the database. We also noted that the rate of margin positivity was higher during the later years of the study, suggesting a secular trend in reporting bias as the importance of margin status has become increasingly recognized. However, the analysis remains valid for individual hospitals during the reporting period because their individual patient and tumor characteristics served as the basis for the model. Moreover, our study includes only data from Commission on Cancer hospitals, so hospital selection bias may exist. The percentage of minorities in our study is smaller than expected (7% African American in our study vs 12.6% in the 2010 US Census), suggesting this potential for bias. Information regarding provider specialty or experience was not available, and the NCDB did not collect data on comorbid conditions until 2003; therefore, the effect of these variables is unknown. However, while less surgeon familiarity with rectal cancer surgery may have been associated with margin positivity, being an academic/research or higher-volume program did not preclude high outlier status. Finally, while the nomogram was not independently validated, the information is taken from the NCDB, which collects data on approximately 76% of incident cancer cases in the United States and thus represents the majority of the US population with rectal cancer. For internal validation, we did perform bootstrapping and identified good performance with a concordance index of 0.75. The calibration plot also demonstrates that the predicted probability derived from the nomogram corresponds well with the observed probability.

In conclusion, the rates of margin positivity have decreased since the introduction of the total mesorectal excision, but margin positivity remains a major determinant of local recurrence, metastases, and overall survival. Our study identified both patient- and tumor-related risk factors for margin positivity, as well as treatment-related factors such as the use of preoperative radiation therapy and facility characteristics, that are associated with rates of positive-margin resection. Although type of hospital and volume were associated with outcomes, no hospital was immune to being a high outlier for margin positivity. The nomogram for determining the risk-adjusted expected rate of margin positivity that has thus been developed in our study permits hospitals to evaluate their own performance and identify potential areas for process improvement.

Accepted for Publication: February 1, 2013.

Corresponding Author: George J. Chang, MD, MS, Department of Surgical Oncology; University of Texas MD Anderson Cancer Center, 1400 Pressler St, FCT 17.6000, Houston, TX 77030 (gchang@mdanderson.org).

Published Online: June 26, 2013. doi:10.1001/jamasurg.2013.2136.

Author Contributions:Study concept and design: Hu, Cormier, Skibber, Nelson, Chang.

Analysis and interpretation of data: Russell, You, Hu, Cormier, Feig, Rodriguez-Bigas, Chang.

Drafting of the manuscript: Russell, Hu, Cormier, Chang.

Critical revision of the manuscript for important intellectual content: You, Cormier, Feig, Skibber, Rodriguez-Bigas, Nelson, Chang.

Statistical analysis: Russell, Hu, Cormier, Chang.

Obtained funding: Chang.

Administrative, technical, and material support: Russell, Feig, Skibber.

Study supervision: You, Skibber, Nelson, Chang.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was supported by the National Institutes of Health/National Cancer Institute grants K07-CA133187 (to Dr Chang) and CA016672 (MD Anderson Cancer Center support grant).

Additional Contributions: We thank Kathryn Hale for her assistance with the manuscript editing and Christina Caraway for her technical assistance with manuscript preparation.