A, Actuarial OS and DFS plot for all patients in the training and validation cohorts. B, Actuarial OS and DFS grouped by adjuvant chemotherapy (CT). HR indicates hazard ratio. To use the nomogram, first draw a vertical line to the top points row to assign points for each variable; then, add the points from each variable together and drop a vertical line from the total points row to obtain the 1-year survival, 3-year survival, 5-year survival, and median survival time (in months).
For an individual patient, first use nomogram A to calculate the expected OS without adjuvant CT; then use nomogram B to calculate the expected OS with adjuvant CT. The difference between the 2 estimates is the expected net survival gain from adjuvant CT. CA19-9 indicates cancer antigen 19-9; CEA, carcinoembryonic antigen. To use the nomogram, first draw a vertical line to the top points row to assign points for each variable; then, add the points from each variable together and drop a vertical line from the total points row to obtain the 1-year survival, 3-year survival, 5-year survival, and median survival time (in months).
For an individual patient, first use nomogram A to calculate the expected DFS without adjuvant CT; then use nomogram B to calculate the expected DFS with adjuvant CT. The difference between the 2 estimates is the expected net survival gain from adjuvant CT. CA19-9 indicates cancer antigen 19-9; CEA, carcinoembryonic antigen. To use the nomogram, first draw a vertical line to the top points row to assign points for each variable; then, add the points from each variable together and drop a vertical line from the total points row to obtain the 1-year survival, 3-year survival, 5-year survival, and median survival time (in months).
eFigure 1. Recruitment Pathway for Patients in This Study
eFigure 2. Calculating Tool That Can Be Used to Help Make Clinical Decisions Regarding the Potential Benefit of Adjuvant Chemotherapy (CT) for an Individual Patient
eFigure 3. Calibration of the Nomograms in the Training and Validation Cohorts
eFigure 4. Decision Curve Analysis for the Nomograms in the Validation Cohort
eTable 1. Descriptive Statistics for Stage II and III Gastric Cancer in the Training and Validation Cohort
eTable 2. Univariate Regression Analysis of Factors Associated With DFS and OS
eTable 3. Patient and Tumor Characteristics in the Training and Validation Cohort After PSM
eTable 4. Univariate Regression Analysis of Factors Associated With DFS and OS in the Training Cohort After PSM
eTable 5. Multivariate Cox Regression Analysis of Factors Associated With DFS and OS in the Training Cohort After PSM
Customize your JAMA Network experience by selecting one or more topics from the list below.
Jiang Y, Li T, Liang X, et al. Association of Adjuvant Chemotherapy With Survival in Patients With Stage II or III Gastric Cancer. JAMA Surg. 2017;152(7):e171087. doi:10.1001/jamasurg.2017.1087
Who are more likely to benefit from adjuvant chemotherapy among patients with stage II or stage III gastric cancer?
In this multicenter retrospective analysis of medical records of 1719 patients with stage II or stage III gastric cancer, 2 nomograms and 1 calculating tool were built to estimate the net survival gain from adjuvant chemotherapy for stage II and stage III gastric cancer.
The survival prediction model can be used to make individualized predictions of the expected survival benefit of adjuvant chemotherapy, thus guiding clinical practice.
The current staging system of gastric cancer is not adequate for defining a prognosis and predicting the patients most likely to benefit from chemotherapy.
To construct a survival prediction model based on specific tumor and patient characteristics that enables individualized predictions of the net survival benefit of adjuvant chemotherapy for patients with stage II or stage III gastric cancer.
Design, Setting, and Participants
In this multicenter retrospective analysis, a survival prediction model was constructed using data from a training cohort of 746 patients with stage II or stage III gastric cancer who satisfied the study’s inclusion criteria and underwent surgery between January 1, 2004, and December 31, 2012, at Nanfang Hospital in Guangzhou, China. Patient and tumor characteristics were included as covariates, and their association with overall survival and disease-free survival with and without adjuvant chemotherapy was assessed. The model was internally validated for discrimination and calibration using bootstrap resampling. To externally validate the model, data were included from a validation cohort of 973 patients with stage II or stage III gastric cancer who met the inclusion criteria and underwent surgery at First Affiliated Hospital in Guangzhou, China, and at West China Hospital of Sichuan Hospital in Chendu, China, between January 1, 2000, and June 30, 2009. Data were analyzed from July 10, 2016, to September 1, 2016.
Main Outcomes and Measures
Concordance index and decision curve analysis for each measure associated with postoperative overall survival and disease-free survival.
Of the 1719 patients analyzed, 1183 (68.8%) were men and 536 (31.2%) were women and the median (interquartile range) age was 57 (49-66) years. Age, location, differentiation, carcinoembryonic antigen, cancer antigen 19-9, depth of invasion, lymph node metastasis, and adjuvant chemotherapy were significantly associated with overall survival and disease-free survival, with P < .05. The survival prediction model demonstrated good calibration and discrimination, with relatively high bootstrap-corrected concordance indexes in the training and validation cohorts. In the validation cohort, the concordance index for overall survival was 0.693 (95% CI, 0.671-0.715) and for disease-free survival was 0.704 (95% CI, 0.681-0.728). Two nomograms and a calculating tool were built on the basis of specific input variables to estimate an individual’s net survival gain attributable to adjuvant chemotherapy.
Conclusions and Relevance
The survival prediction model can be used to make individualized predictions of the expected survival benefit from the addition of adjuvant chemotherapy for patients with stage II or stage III gastric cancer.
Gastric cancer (GC) is the fourth most common human malignant disease and the second leading cause of cancer-related deaths worldwide.1 Surgical resection is the primary treatment for resectable GC; however, even after complete resection, a subset of patients will develop local recurrences and metachronous metastases.2 For proper postoperative surveillance and treatment, it is necessary to develop prognostic tools to characterize the heterogeneity of GC.
In recent years, significantly improved outcomes have been reported for patients with GC, mostly because of improvements in drug therapy.2-5 Adjuvant chemotherapy has been recommended as a standard postoperative chemotherapy regimen for patients with stage II or stage III GC.2-4,6 However, administration of adjuvant chemotherapy to all patients with stage II or stage III GC is unnecessary and may even be harmful for some patients. Consequently, there is considerable interest in exploring the potential individual benefit of adjuvant chemotherapy. Some markers have been identified that may be associated with chemotherapy benefits for GC; however, most proposed biomarkers are not clinically implemented because they lack reproducibility and/or standardization. Therefore, they cannot be used to quantify the individual net survival benefit of adding adjuvant chemotherapy. As a result, clinicians currently have little evidence to use when determining whether adjuvant chemotherapy will be beneficial to their patients.
The aim of this study was to construct a survival prediction model—a decision aid—to estimate the potential survival benefit of adjuvant chemotherapy after surgery for patients with stage II or stage III GC. To this end, we constructed a Cox proportional hazards multivariate regression model and validated it in a validation cohort.
The study included a retrospective medical records review of patients who had been enrolled in Nanfang Hospital of Southern Medical University (Guangzhou, China), First Affiliated Hospital of Sun Yat-sen University (Guangzhou, China), and West China Hospital of Sichuan University (Chendu, China). All the data were extracted by 2 experienced abstractors (L. Z. and L. H.), who were blinded to the study hypothesis. The interrater reliability testing was operated, and the interrater agreement was good (κ = 0.93). The quality of the study was ensured by following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. The institutional review boards at Nanfang Hospital of Southern Medical University, First Affiliated Hospital of Sun Yat-sen University, and West China Hospital of Sichuan University (hereinafter, West China Hospital) approved the retrospective analysis of anonymous data and waived the need to obtain patient informed consent.
We collected data in the training cohort of 746 patients who underwent GC surgery at the Department of Surgery at Nanfang Hospital between January 1, 2004, and December 31, 2012 (eFigure 1 in the Supplement). All these patients satisfied the following inclusion criteria: presence of primary stage II or III GC, no combined malignant neoplasm, no preoperative chemotherapy, no distant metastasis, R0 resection (no residual macroscopic or microscopic tumor), more than 15 examined lymph nodes, and no missing values.
We included in the independent validation cohort another 973 patients who met the inclusion criteria and underwent surgery at the First Affiliated Hospital and West China Hospital between January 1, 2000, and June 30, 2009 (eFigure 1 in the Supplement). Data were analyzed from July 10, 2016, to September 1, 2016.
This data set included patient demographics (age and sex), overall status (American Society of Anesthesiologists score, Eastern Cooperative Oncology performance status, and Charlson comorbidity index), postoperative complications, pathologic characteristics (location, size, differentiation, Lauren type,7,8 depth of invasion, and lymph node metastasis), adjuvant chemotherapy, and follow-up data (follow-up duration and survival). The severity of postoperative complications was assessed according to the Clavien-Dindo classification.9 The location of the tumor was categorized as cardia, body, antrum, or whole, and the size of the tumor was measured at the longest diameter. The histologic subtype was categorized as well differentiated, moderately differentiated, poorly differentiated, and undifferentiated. The TNM staging was reclassified according to the seventh edition of the AJCC Cancer Staging Manual of the American Joint Committee on Cancer (AJCC)/International Union Against Cancer.10 Adjuvant chemotherapy was categorized as received or not received. The characteristics of patients who received and who did not receive adjuvant chemotherapy were similar in our study (Table 1 and eFigure 1 in the Supplement). Propensity score matching analysis was performed for receiving the chemotherapy using 1:1 nearest matching based on the following covariates: age, sex, Eastern Cooperative Oncology performance status, American Society of Anesthesiologists score, Charlson index, postoperative complications, differentiation, carcinoembryonic antigen (CEA), cancer antigen 19-9 (CA19-9), location, depth of invasion, lymph node metastasis, size, and Lauren type (which was not used in the validation cohort) as a sensitivity analysis. Follow-up data were collected from hospital records for patients who were lost to follow-up. The follow-up duration was measured from the time of surgery to the last follow-up date, and information regarding the survival status at the last follow-up was collected. Disease-free survival (DFS) was not recorded in the data set of West China Hospital.
Multivariate regression analysis was performed using Cox proportional hazards modeling, which formed the basis for the survival prediction model. In the training cohort, survival curves for different variable values were generated using the Kaplan-Meier estimates and were compared using the log-rank test. Variables that achieved statistical significance at P < .05 were entered into the multivariate analyses via the Cox regression model. Covariates included in the prediction model were selected on the basis of known clinical prognostic factors and availability in the training cohort. Included covariates were age, location, CEA and CA19-9 levels, differentiation, depth of invasion, lymph nodes metastasis, and adjuvant chemotherapy (yes or no). The prediction model was implemented into nomograms to enable use on plain paper and implementation as a calculation tool.
The survival prediction model was validated by measuring both discrimination and calibration. Both discrimination and calibration were evaluated on the original study cohort using bootstrapping with 1000 resamples.18,19 Discrimination was evaluated using the concordance index (C index), which is similar in concept to the area under a receiver operating characteristic curve. The C index measures the probability that, given a pair of randomly selected patients, the survival prediction model correctly predicts which patient will experience an event first. The C index of the model can range from 0.5, which represents random chance, to 1.0, which represents a perfectly discriminating model. The other validation measure was calibration, which compares predicted survival with actual survival. Calibration was evaluated with a calibration curve, in which patients are grouped by predicted survival and then plotted as actual vs predicted survival.
Decision curve analysis was performed to determine the clinical usefulness of the nomograms by quantifying the net benefits at different threshold probabilities.20,21
Differences in distributions between the variables examined were assessed with the unpaired, 2-tailed χ2 test or the Fisher exact test, as appropriate. Survival curves were generated according to the Kaplan-Meier method and compared using the log-rank test. Univariate and multivariate analyses were performed with the Cox proportional hazards model. Nomograms and calibration plots were generated using the rms package of R version 3.0.1. All other statistical tests were conducted using SPSS version 19.0 (IBM) and R version 3.0.1 (http://www.r-project.org). Statistical significance was set at 2-sided P < .05.
The clinicopathologic characteristics for the training cohort (n = 746) and validation cohort (n = 973) are listed in Table 1 and eTable 1 in the Supplement. The number of patients with stage II or stage III GC who received adjuvant chemotherapy was 417 (55.9%) in the training cohort and 505 (51.9%) in the validation cohort. The characteristics of patients who received and who did not receive adjuvant chemotherapy were similar in our study (Table 1 and eTable 1 in the Supplement). Of the 1719 patients analyzed, 1183 (68.8%) were men and 536 (31.2%) were women and the median (interquartile range [IQR]) age was 57 (49-66) years. The actuarial overall survival (OS) and DFS grouped by receipt of adjuvant chemotherapy are shown in Figure 1. Adjuvant chemotherapy significantly prolonged the OS and DFS of patients with stage II or stage III GC in both the training and validation cohorts (P = .001 and P = .01, respectively, in the training cohort; P < .001 and P = .002, respectively, in the validation cohort).
Compared with patients without adjuvant chemotherapy, the median (IQR) survival time increased from 30 (10-63) months to 37 (13-61) months for DFS and from 36 (17-67) months to 43 (23-65) months for patients with adjuvant chemotherapy in the training cohort. In the validation cohort, the median (IQR) survival time increased from 33 (13-68) months to 60 (19-81) months for DFS and from 38.5 (16-69) months to 61 (24-78) months for OS.
Results of the multivariate regression model are listed in Table 2 and eTable 2 in the Supplement. Statistically significant covariates were age, location, CEA and CA19-9 levels, differentiation, depth of invasion, lymph nodes metastasis, and receipt of adjuvant chemotherapy. Nomograms were constructed from the coefficients of this model. To estimate the net survival benefit from adjuvant chemotherapy, the 2 nomograms were used together (Figure 2 and Figure 3). The first nomogram (Figures 2A and 3A) estimated survival time without adjuvant chemotherapy, and the second nomogram (Figures 2B and 3B) estimated survival time with adjuvant chemotherapy. The difference between the 2 estimates was the expected net survival benefit from the addition of adjuvant chemotherapy. To use the nomogram, first draw a vertical line to the top points row to assign points for each variable; then, add the points from each variable together and drop a vertical line from the total points row to obtain the 1-year survival, 3-year survival, 5-year survival, and median survival time (in months). A calculating tool (eFigure 2 in the Supplement) is also implemented that can calculate the estimated net survival benefit from the addition of adjuvant chemotherapy after the user enters the requested patient and tumor characteristics. For example, for a 70-year-old patient with GC and elevated levels of CEA and CA19-9 as well as T4aN1 cardia tumor that is poorly differentiated, the model predicts that the addition of adjuvant chemotherapy would increase the 1-year OS rate from 65% to 78%, improving the median survival time from 18 months to 23 months (eFigure 2 in the Supplement).
Model performance was validated for discrimination and calibration. Discrimination was measured using the bootstrap-corrected C index.
The calibration curve (eFigure 3A and B in the Supplement) showed good agreement between predicted and observed outcomes in the training cohort. The C indexes were 0.683 (95% CI, 0.655-0.711) for OS prediction and 0.686 (95% CI, 0.660-0.713) for DFS prediction.
Good calibration was observed for the 1-year, 3-year, and 5-year outcomes in the validation cohort (eFigure 3C and D in the Supplement). In the validation cohort, the C indexes were 0.693 (95% CI, 0.671-0.715) for OS prediction and 0.704 (95% CI, 0.681-0.728) for DFS prediction.
Furthermore, we compared the discrimination of our nomogram with that of the TNM classification in the seventh edition of the AJCC Cancer Staging Manual. The discrimination of our nomogram was superior to that of the AJCC Cancer Staging Manual (C index for the training cohort: OS, 0.595 (95% CI, 0.574-0.616) and DFS, 0.595 (95% CI, 0.575-0.615); validation cohort: OS, 0.621 (95% CI, 0.603-0.638) and DFS, 0.625 (95% CI, 0.605-0.644).
The patient and tumor characteristics in the training and validation cohort after propensity score matching are shown in eTable 3 in the Supplement, which were not changed. The results of univariate and multivariate analyses were similar after propensity score matching (eTables 4 and 5 in the Supplement).
The decision curve analysis for these nomograms in the validation cohort is presented in eFigure 4 in the Supplement. The decision curve showed that if the threshold probability of a patient or physician is greater than 10%, using the 2 nomograms to predict the 3-year and 5-year OS and DFS provides more benefit than either the treat-all-patients scheme or the treat-no-patients scheme.
This study makes an important contribution by developing a survival prediction model using a large cohort of patients with stage II or stage III GC who were treated in China between 2000 and 2012. The model is more predictive than the stage grouping in the seventh edition of the AJCC Cancer Staging Manual, with higher C indexes and good calibration. The model is useful for individualizing therapeutic recommendations.
In this study, 520 patients had stage II GC and 1199 patients had stage III GC according to the staging system of the seventh edition of the AJCC Cancer Staging Manual. The CLASSIC (Capecitabine and Oxaliplatin Adjuvant Study in Stomach Cancer) trial—a phase 3, randomized, open-label study conducted at 35 cancer centers, medical centers, and hospitals in China, South Korea, and Taiwan—compared adjuvant capecitabine plus oxaliplatin with observation after D2 gastrectomy for patients with stage II or III GC.2,4 The estimated 5-year DFS was 68% (95% CI, 63%-73%) in the adjuvant capecitabine and oxaliplatin group vs 53% (95% CI, 47%-58%) in the observation alone group. The estimated 5-year OS was 78% (95% CI, 74%-82%) in the adjuvant capecitabine and oxaliplatin group vs 69% (95% CI, 64%-73%) in the observation group.2 Compared with the 5-year OS rate of 73% after surgery alone for this patient population, adjuvant chemotherapy with uracil-tegafur also significantly improved 5-year OS up to 86% in a randomized controlled trial.22
The Adjuvant Chemotherapy Trial of S-1 (an oral dihydropyrimidine dehydrogenase inhibitory fluoropyrimidine preparation) for Gastric Cancer, another randomized phase 3 trial with similar eligibility criteria, confirmed the effectiveness of 1-year, postoperative fluoropyrimidine preparation treatment compared with surgery alone in patients with stage II or stage III GC who underwent D2 gastrectomy.23-25
The OS rate at 5 years was 71.7% in the fluoropyrimidine preparation group and 61.1% in the surgery-only group (hazard ratio [HR], 0.669; 95% CI, 0.540-0.828). The relapse-free survival rate at 5 years was 65.4% in the fluoropyrimidine preparation group and 53.1% in the surgery-only group (HR, 0.653; 95% CI, 0.537- 0.793).5 However, whether all patients at this stage require adjuvant chemotherapy is uncertain. Our model may be useful for selecting patients who are unlikely to benefit from adjuvant chemotherapy.
The C indexes of our nomograms were 0.68 to 0.71, a range slightly lower than those reported in previous studies (0.70-0.80 in different patient populations),26-28 and were similar to those of Hirabayashi and colleagues.29 It is unclear why the C indexes of our nomograms were lower than others, but a possible explanation is the difference in patient populations. Previous studies included many patients with stage I GC whose prognosis was excellent and/or patients with stage IV GC whose prognosis was poor.26,27 The present study included only patients with stage II or stage III GC whose prognoses varied widely. When patients with stage I or stage IV GC were included in this study, the C indexes of the nomograms were much higher. Our nomogram discrimination was superior to that of the seventh edition of the AJCC Cancer Staging Manual, with P < .001 in both OS and DFS in the training and validation cohorts.
This study accurately predicts survival. The calibration plots of the training cohort and validation cohort indicated that actual survival corresponded closely with predicted survival, suggesting that predictive performance of the nomograms was good. The model can be widely used because these data collected from 3 different cancer treatment centers in China may minimize the effect of patients’ historical backgrounds and institutional differences.
In some cases, the model predicts that the addition of adjuvant chemotherapy would result in either no added benefit or slight improvement; however, we did not specify a threshold at which adjuvant chemotherapy should be recommended; we believe that the final decision to administer adjuvant chemotherapy should be made after careful discussion between the clinician and the patient and after considering multiple factors, many of which cannot be accounted for in a prediction model. Although net potential survival benefit as predicted by this model is an important consideration, it should not be the sole basis for decision making. Quality of life and specific patient preferences are also important considerations in deciding treatment.
There has been growing interest in the development of cancer prediction models. A number of important cancer risk prediction models are being used today for prostate,30,31 breast,32,33 pancreatic,34 and gastric cancers.27-29 Such models are preferable to any individual clinician’s limited personal experience, and such models may be more accurate than extrapolating from other types of cancer. In addition, customized survival predictions are more relevant to individual patients than recommendations based on coarse groupings of large numbers of heterogeneous patients. Estimating survival probability on the basis of stage alone is not always accurate; our model aptly illustrates how prognosis changes markedly with variation in other factors, such as patient age, CEA and CA19-9 levels, and differentiation. As more specific patient and tumor information, such as genetic information and molecular tumor biomarkers, becomes routinely collected in the future, use of these types of predictive models will become increasingly important.
There are some limitations to this study. First, our nomogram was developed and validated using data from almost exclusively Chinese patients. Second, the study was conducted retrospectively, making it susceptible to the inherent biases of such a study format. Third, the use of adjuvant chemotherapy was not within a randomized comparison, and the decision to treat or not to treat patients after surgery was made by the patients and/or clinicians. Although the clinical characteristics of patients with and without adjuvant chemotherapy were not significantly different in the training and validation cohorts (Table 1), there are important clinicopathologic differences between patients who received adjuvant chemotherapy and patients who did not. Fourth, the C indexes of our model were only 0.68 to 0.71, which was not encouraging. Clearly, our results should be further validated by prospective studies in multicenter clinical trials. Finally, our model considered only the various survival benefits of adjuvant chemotherapy for a given patient but not the potential toxic effects of chemotherapy, which can vary from patient to patient. Only when the clinician considers both the potential risks and benefits of a given therapy can the clinician make an informed recommendation to a given patient.
In the future, we will seek to externally validate the performance of our model using other patient databases. We will also explore the possibility of including additional prognostic variables to further improve performance.35 Other regression modeling techniques will be used to determine whether predictive accuracy can be further improved.36,37
We present a survival prediction model that can make an individualized estimate of the net survival benefit of adding adjuvant chemotherapy for patients with stage II or stage III GC. This model can assist clinicians and patients in quantifying the benefit of adjuvant chemotherapy after surgical resection of GC and in making individualized therapeutic recommendations and treatment decisions.
Corresponding Author: Guoxin Li, MD, PhD, FRCS, Department of General Surgery, Nanfang Hospital, Southern Medical University, 1838 N Guangzhou Ave, Guangzhou 510515, China (firstname.lastname@example.org).
Accepted for Publication: March 8, 2017.
Published Online: May 24, 2017. doi:10.1001/jamasurg.2017.1087
Author Contributions: Drs Jiang, T. Li, and Hu and Ms Liang contributed equally to this study. Dr G. Li had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Jiang, Qi, H. Liu, G. Li.
Acquisition, analysis, or interpretation of data: Jiang, T. Li, Liang, Hu, Huang, Liao, Zhao, Han, Zhu, Wang, Xu, Yang, Yu, W. Liu, Cai.
Drafting of the manuscript: Jiang, T. Li, Hu, Huang, Zhao, Han, Zhu, Wang, Yu, W. Liu.
Critical revision of the manuscript for important intellectual content: Liang, Liao, Xu, Qi, H. Liu, Yang, Cai, G. Li.
Statistical analysis: Jiang, Liang, Huang, Han.
Obtained funding: Yang, W. Liu.
Administrative, technical, or material support: Jiang, T. Li, Hu, Liao, Zhao, Zhu, Wang, Xu, Qi, Yang, Yu, W. Liu, Cai, G. Li.
Study supervision: Jiang, Liang, H. Liu, G. Li.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was funded in part by grants 81672446, 81600510, 81370575, and 81570593 from the National Natural Science Foundation of China; grant 2014A030313131 from the Natural Science Foundation of Guangdong Province; grants 2014B020228003, 2014B030301041, and 2015A030312013 from the Science and Technology Planning Project of Guangzhou; grants 158100076 and 201400000001-3 from the Science and Technology Program of Guangzhou; grants 201402015 and 201502039 from the Public Welfare in Health Industry, National Health, and Family Planning Commission of China; and Key Clinical Specialty Discipline Construction Program.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.