Clu_shade indicates cluster shade; GGH indicates Guangdong General Hospital; GLCM, gray level co-occurrence matrix; IMC, informational measure of correlation; GLDM, gray level dependence matrix; DNN, dependence nonuniformity normalized; GLRLM, gray level run length matrix; GLSZM, gray level size zone matrix; LAHGLE, large area high gray level emphasis; LCHI, Liaoning Cancer Hospital and Institute; LALGLE, large area low gray level emphasis; LDE, large dependence emphasis; LDHGLE, large dependence high gray level emphasis; LDLGLE, large dependence low gray level emphasis; LRHGLE, long run high gray level emphasis; SDHGLE, small dependence high gray level emphasis; and SYSUCC, Sun Yat-sen University Cancer Center.
A and B, Feature selection using the least absolute shrinkage and selection operator binary logistic regression model. This method minimizes the sum of residues’ squares, with the sum of the absolute values of the selected features coefficients being less than a tuning parameter (λ). A, The area under the receiver operating characteristic curve (AUC) was plotted vs log(λ). The vertical blue line represents the chosen parameter. B, Each colored line represents the coefficient of each feature. A vertical blue line was drawn at the λ selected, where 20 features had nonzero coefficients. C and D, Receiver operating characteristic curves of the radiomics signature in the training cohort (C) and validation cohort (D). E and F, Plots depicting the calibration of the radiomics signature in terms of agreement between predicted and observed respondents in the training cohort (E) and validation cohort (F). G, Decision curves for predicting potential respondents for neoadjuvant chemotherapy. The orange line represents the assumption that all patients will respond to neoadjuvant treatment, and the horizontal blue line represents the assumption that no patient will respond to treatment.
A and B, The closer the curves, the higher the probability that the radiomics signature would identify positive responders from a total estimated number of responders. The threshold value represents the value after which the rate of misdiagnosis would be lowest, thereby providing an optimal treatment to benefit ratio for the patient. The blue line represents the total number of patients who would be considered as having a true-positive response (tumor regression grade 1a-1b) for each threshold. C and D, Probability assessment using the radiomics signature to differentiate between true-positive (tumor regression grade 1a-1b) and false-positive (tumor regression grade 2-3) responders. The farther apart the curves are, the greater the probability that the model would differentiate between the 2 types of responders.
eMethods. Supplementary Methods
eTable. Baseline Comparison of the Clinicopathological Characteristics Between the Training and Validation Cohorts
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Wang W, Peng Y, Feng X, et al. Development and Validation of a Computed Tomography–Based Radiomics Signature to Predict Response to Neoadjuvant Chemotherapy for Locally Advanced Gastric Cancer. JAMA Netw Open. 2021;4(8):e2121143. doi:10.1001/jamanetworkopen.2021.21143
Can computed tomography–based radiomics be used to predict patients’ response to neoadjuvant chemotherapy at diagnosis?
In this cohort study of 323 patients with locally advanced gastric cancer, no pretreatment clinicopathological characteristics were associated with response or nonresponse to treatment. However, 20 radiomic features with low intercorrelation were significantly associated with treatment response and were used to create a radiomics signature with promising clinical reliability for identifying potential responders to neoadjuvant chemotherapy.
Given that no patients respond equally to neoadjuvant chemotherapy, the radiomics signature proposed in this study could guide clinicians in selecting appropriate patients for neoadjuvant chemotherapy.
Neoadjuvant therapies have been shown to decrease tumor burden, increase resection rate, and improve the outcomes among patients with locally advanced gastric cancer (GC). However, not all patients are equally responsive; therefore, differentiating potential respondents from nonrespondents is clinically important.
To use pretreatment computed tomography (CT)–pixelated feature-difference extraction techniques to identify diagnostically relevant features that could predict patients’ response to neoadjuvant chemotherapy at diagnosis.
Design, Setting, and Participants
This multicenter cohort study included patients with locally advanced GC who were treated from January 2010 to July 2017 at 2 hospitals in southern China (training cohort) and 1 hospital in northern China (external validation cohort). Their clinicopathological data, pretreatment CT images, and pathological reports were retrieved and analyzed. Data analysis was conducted from December 2017 to May 2021.
All patients underwent 2 to 4 cycles of fluorouracil in combination with a platinum-based neoadjuvant chemotherapy regimen. All gastrectomies were performed according to the Japanese Classification of Gastric Carcinoma (14th edition) guidelines.
Main Outcomes and Measures
Reliability of clinicopathological and radiomics-based features were assessed with area under receiver operating characteristic curve (AUC) and Mann-Whitney U test.
A total of 323 patients (242 [74.9%] men; median [range] age, 58 [24-82] years) were included in the study, with 250 patients (77.4%) in the training cohort and 73 (22.6%) in the validation cohort. The baseline pretreatment characteristics of the training and validation cohorts were well-balanced. The number of respondents in the training and validation cohort was 122 (48.8%) and 40 (54.8%), respectively, and the number of nonrespondents was 128 (51.2%) and 33 (45.2%), respectively. No clinicopathological variables were significantly associated with treatment response. Using radiomics, 20 low-intercorrelated features from a total of 7477 features were used to construct a radiomics signature that demonstrated significant association with treatment response. Good discrimination performance of the radiomics signature for predicting treatment response in the training (AUC, 0.736; 95% CI, 0.675-0.798) and external validation (AUC, 0.679; 95% CI, 0.554-0.803) cohorts was observed. Decision curve analysis confirmed the clinical utility of the radiomics signature.
Conclusions and Relevance
In this study, the proposed radiomics signature showed potential as a clinical aid for predicting the response of patients with locally advanced GC before treatment, thereby allowing timely planning for effective treatments for potential nonrespondents.
Gastric cancer (GC) is a highly heterogeneous disease associated with a high mortality rate.1 Although multimodal treatments, such as gastrectomies followed by adjuvant chemotherapy (the standard of care especially in Asian countries),2 have improved patients’ survival outcomes,3 benefits for locally advanced cases have been modest. The addition of neoadjuvant therapies to standard treatment has led to an increase in pathological complete remission (pCR) and pathological partial remission (pPR) rates of the tumor, an increase in resection rates, and improvements in survival outcomes.4-7 Thus, neoadjuvant therapy is being increasingly used outside clinical trial settings.
However, not all patients are equally responsive to neoadjuvant therapies. Differentiating between potential responders and nonresponders is critical for timely treatments. Also, considering that pCR and pPR are confirmed through invasive histopathological interventions and mostly after surgery, creating an alternative, noninvasive method remains a major challenge. Conventional and multicombinative radiographic approaches have been explored, but their accuracies have been limited.8,9 On the other hand, molecular biomarkers, including serologic and radiographic markers, have shown promising results, but none have been prospectively validated for routine clinical use.10,11
Radiomics has shown promise for extracting noninvasive radiographic virtual biopsy biomarkers. Landmark discoveries have been observed in nasopharyngeal,12 lung,13 breast,14 and urinary bladder15 cancers, demonstrating improved predictive values of multiparametric imaging features over conventional imaging metrics. We previously developed and validated a radiomics-based nomogram to predict preoperative lymph node metastasis in patients with colorectal cancer using radiomics-derived signatures.16 Liu et al17 developed and validated an individualized radiomics model to noninvasively predict pCR in patients with advanced colorectal cancer who received neoadjuvant treatment and were able to identify patients who could avoid surgery after neoadjuvant therapy. Regarding GC, research on radiomics for predicting pretreatment responses to neoadjuvant therapy via multicenter studies with robust validation is yet to be reported. Here, using a large cohort of patients with locally advanced GC from 3 high-volume institutions, a radiomics signature was developed and externally validated for the individualized prediction of pretreatment response to neoadjuvant chemotherapy to differentiate potential respondents from nonrespondents.
The demographic data, clinical characteristics, enhanced pretreatment CT images, pathological findings of biopsies before treatment and gross specimen after gastrectomy, and operative records of patients with locally advanced GC who were treated with neoadjuvant chemotherapy from January 2010 to July 2017 at 3 high volume centers were retrospectively retrieved. All patients underwent CT examination within 1 week before initiation of neoadjuvant chemotherapy. The main inclusion criteria were (1) biopsy-proven gastric adenocarcinoma; (2) nonmetastatic locally advanced GC determined by pretreatment CT examination or laparoscopic laparotomy; and (3) gastrectomy procedure after completion of neoadjuvant chemotherapy, after which tumor response was confirmed by postoperative pathologic examination. Main exclusion criteria were (1) presence of other gastric malignant neoplasms; (2) inoperable or nonresected cases (ie, exploratory laparoscopy only without gastrectomy; gastrojejunal anastomosis without tumor resection); (3) other types of neoadjuvant therapies; and (4) insufficient CT imaging quality to perform measurements (eg, due to motion artifacts). All gastrectomies were performed according to the Japanese Classification of Gastric Carcinoma (14th edition) guidelines. The 8th edition of the American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) classification was used for tumor staging.18 Patients from the Sun Yat-sen University Cancer Center (SYSUCC; Guangzhou, China) and Guangdong General Hospital (GGH; Guangzhou, China) were grouped as the training cohort, and those from Liaoning Cancer Hospital and Institute (LCHI; Liaoning, China) were grouped as the external validation cohort. The detailed enrollment process is illustrated in Figure 1.
The study protocol adhered to the regulations of the Declaration of Helsinki19 and was also approved by the ethics committee at each institution. All patients provided written informed consent at the time of diagnosis to use their data for scientific purposes. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies.
The prescribed neoadjuvant chemotherapy regimen was unified as fluorouracil in combination with platinum-based chemotherapy for 2 to 4 cycles. Because the data were retrospectively retrieved, the dose prescribed to each patient depended on the treating oncologists but still adhered to the standard National Comprehensive Cancer Network guidelines for GC clinical practice at the time of treatment.
After neoadjuvant chemotherapy, all patients underwent gastrectomy. The surgically resected specimens were histopathologically assessed by expert pathologists with more than 10 years of experience in gastrointestinal cancer assessment and were blinded to the study data. The tumor regression grade (TRG) proposed by Becker et al20 was adopted to evaluate neoadjuvant treatment response. Briefly, tumor samples without residual tumor were defined as complete tumor regression (TRG 1a). When more than 50% of residual tumors were identified, the sample was classified as minimal or no tumor regression (TRG 3). The presence of less than 10% of residual tumor cells per tumor bed was classified as sub-total regression (TRG 1b) and 10% to 50% of residual tumor cells per tumor bed was defined as partial regression (TRG 2). Furthermore, patients with TRG 1a and 1b were considered respondents to neoadjuvant chemotherapy, while those with TRG 2 and 3 were classified as nonrespondents.
The CT features extraction process is described in the eMethods in the Supplement. The region of interest (ROI) was manually delineated using portal venous phase CT images. The radiomics features extracted included (1) first-order statistics, (2) shape-based features, (3) texture features, and (4) wavelet features.
Three experienced radiologists were randomly assigned 30 CT images (10 patients’ images for each reader with 8, 10, and 13 years of clinical experience in abdominal cancer CT interpretation) for ROI-based feature extraction to assess intra-observer and interobserver reproducibility. Intraclass and interclass correlation coefficients (ICC) were used to assess the stability, reproducibility, and robustness of the extracted features. Features with an ICC of greater than 0.75 were considered to have agreeable reproducibility and were chosen for further analysis.
For constructing the radiomics signature, similar features with high correlations were rejected using the Spearman correlation coefficients. Feature pairs with Spearman correlation coefficients of greater than 0.9 were considered highly correlated features, and only 1 of the 2 was accepted. The least absolute shrinkage and selection operator (LASSO) method was then used to select the most robust predictive radiomics features with good reproducibility and association with treatment response in the training cohort.21 This method minimizes the sum of residues’ squares, with the sum of the absolute values of the selected features’ coefficients being less than a tuning parameter (λ). As λ becomes smaller, the coefficients of more features were shrunk to 0. To avoid overfitting, the number of selected features should be less than one-tenth of cases in the training set. Features with nonzero coefficients were then subsequently used to build the radiomics signature and were normalized using the z scores of the training and validation cohort (eMethods in the Supplement). Finally, a radiomics signature of each patient with GC was calculated with a linear combination of the final selection of features and multiplied by their normalized coefficients to obtain a radiomics signature.
The performance, discrimination, calibration, and clinical usefulness of the radiomics signature are described in the eMethods in the Supplement. Briefly, the area under receiver operating characteristics curve (AUC) was used to assess the discriminative power of the radiomics signature and the Mann-Whitney U test to evaluate the correlation between the radiomics signature and treatment response status. Calibration curves were plotted to assess the calibration of the radiomics signature, accompanied with the Hosmer-Lemeshow test. To quantify the discrimination performance of the radiomics signature, Harrell’s C index was measured. Decision curve analysis (DCA) was performed to quantify the clinical utility of the radiomics signature from a simulated cohort data of the training and validation data set to assess the ability of the derived radiomics signature to differentiate potential respondents from nonrespondents. An ROC plot was used to illustrate net benefit from neoadjuvant therapy, defined as the proportion of true respondents minus the proportion of false respondents, displayed as functions of the risk threshold.
We used R version 3.3.1 (R Project for Statistical Computing) and SPSS statistical software version 20.0 (IBM Corp) for statistical analyses. A 2-sided P < .05 was considered statistically significant.
The entire study cohort comprised of 323 patients (242 [74.1%] men; median [range] age, 58 [24-82] years) with locally advanced GC, with 250 patients (77.4%) in the training cohort and 73 (22.6%) in the validation cohort. Most patients were diagnosed with stage III locally advanced GC at the time of diagnosis (246 [76.2%]), with 13 (4.0%) with stage IIA; 29 (9.0%) with stage IIB; and 35 (10.8%) with stage IVA.
The mean number of neoadjuvant chemotherapy cycles was 3 (range, 2-4) cycles. The median time to surgery of the training and validation cohort was 3 (range, 2-4) weeks. There was no significant difference in the pretreatment clinical characteristics between the training and validation cohort, except for the percentage of patients with calcium levels in the reference range (210 [84.0%] vs 71 [97.3%]; P = .003), AST levels in the reference range (171 [68.4%] vs 62 [84.9%]; P = .045), activated partial thromboplastin time in the reference range (182 [72.8%] vs 64 [87.7%]); P = .009), and histological grade (eg, high or moderate grade: 48 [19.2%] vs 28 [38.4%]; P = .001) (eTable in the Supplement).
The number of respondents (TRG 1a-1b) and nonrespondents (TRG 2-3) in the training cohort was 122 (48.8%) and 128 (51.2%), respectively; in the training cohort, the number of respondents and nonrespondents was 40 (54.8%) and 33 (45.2%), respectively (Table 1). Our analyses showed that no patient characteristics were associated with neoadjuvant chemotherapy response, including commonly used clinical, serological, and radiographic variables, such as age, carcinoembryonic antigen level, CA199 level, CA724 level, tumor location, clinical tumor stage, clinical node stage, and clinical TNM stage in both the training and validation cohorts.
Radiomics techniques were then used to identify features that could differentiate between respondents and nonrespondents for neoadjuvant therapy. Initially, a total of 7477 two-dimensional radiomics features were extracted from the volume of interest of each patient. Among them, 2233 features were considered to have satisfactory intra-observer and interobserver reproducibility (ie, ICC >0.75). After omitting feature pairs with high correlations, 378 features from each patient were used for further selection.
Using the LASSO logistic regression model in the training cohort (Figure 2A and B), 20 independent features (Table 2) from the 378 robust radiomics features were identified. They were used in the formula that appears in Figure 1 to build the radiomics signature.
The AUCs of the radiomics signature for discriminating respondents from nonrespondents was 0.736 (95% CI, 0.675-0.798) in the training cohort (Figure 2C) and 0.679 (95% CI, 0.554-0.803) in the validation cohort (Figure 2D). Mann-Whitney U test showed good correlations between radiomics signature and treatment response in the training and validation cohort. To determine the agreement between the predicted and observed pathological response, the radiomics signature was calibrated using calibration plots to illustrate the goodness of fit (Figure 2E) in the training cohort and validated in the external validation cohort (Figure 2F). The Hosmer-Lemeshow test yielded a nonsignificant statistic, which suggested a small difference from perfect fit between the predicted and observed treatment response.
The DCA plot (Figure 2G) showed that the radiomics signature could predict patients’ response for probability thresholds ranging between 0.20 and 0.75. To further assess the clinical utility of the radiomics signature, a simulated model comprised of 1000 locally advanced GC cases of the training and validation cohorts was built to more accurately identify patients with potential positive response. Our findings showed a probability threshold of 0.6 in both cohorts (Figure 3A and B) was optimal to identify patients who would have positive response from neoadjuvant chemotherapy. Furthermore, the probability threshold of 0.5 to 0.6 in both the training and validation cohorts (Figure 3C and D) was optimal for differentiating true-positive responders from false-positive responders. Thus, using the threshold value of the radiomics signature on the DCA and ROC plots, a reliable individualized prediction of the response of a patient with locally advanced GC patient to neoadjuvant chemotherapy could be obtained.
In this large multicentered study comprised of locally advanced GC patients, no clinicopathological characteristics were significantly associated with treatment response. Radiomics was able to bridge radiology and histopathology by using postoperative pathological findings as a reference standard to identify potential responders for neoadjuvant chemotherapy. The extracted 20 features were used to generate a radiomics signature that was then used to identify an individualized threshold value from DCA and ROC plots, based on which potential respondents to neoadjuvant therapy could be identified.
Previous studies have shown that clinicopathological characteristics (such as age, signet-ring cell type histology, and microsatellite status) were associated with neoadjuvant response.22-25 However, the characteristics identified were not consistent across all reports. Also, they were single-center studies with small sample sizes, and their findings were not externally validated. We hypothesize the lack of association between clinicopathological characteristics and treatment response in our present study could be because of intratumoral heterogeneity, such as underlying mutation characteristics or cell cycle regulating pathways, which could be more strongly associated with treatment response than pretreatment clinicopathological characteristics given that neoadjuvant therapy, being a systemic treatment, can reshape tumor-immune signaling and the tumor microenvironment.26-28 Thus, intratumoral heterogeneity could be reflected, to a certain extent, by the unique textural and spatial gray level patterns of the extracted radiomics features in the pretreatment in vivo CT images. However, these theoretical assumptions are yet to be fully elucidated via radiogenomics and multi-omics studies.
The data used for this study were derived from 3 institutions, and heterogeneity in clinical and pathological data or CT imaging collection and interpretation at the different institutions may have existed. To overcome this, 3 experienced radiologists (with 8, 10, and 13 years of experience) performed intra-observer and interobserver reproducibility reassessment, for which only features with good reproducibility (ie, ICC >0.75) were selected to build the signature. Furthermore, using the radiomics score combined with the threshold on the DCA (Figure 2G) and ROC plots (Figure 3), the net benefit for suggesting neoadjuvant chemotherapy to a patient (ie, the proportion of true positive minus the proportion of false positive weighted by the relative consequence of false-positive and false-negative results as compared with a treat-all or treat-none) could be estimated.
To our knowledge, this is the first study to develop and externally validate a radiomics signature for patients with locally advanced GC. It could be used to offer evidence-based guidance, rather than solely relying on clinical judgment, for multidisciplinary consultation between radiologists and oncologists to predict pretreatment response and offer justifiable individualized therapeutic options to these patients. Also, for those patients predicted as having a high probability of stable or progressive disease (TRG 2 or 3), timely alternative curative-intent treatment approaches could be offered to improve their treatment and survival outcomes. Thus, the strength of this study is using existing routine diagnostic CT imaging and a validated method, without additional cost to the patient, to predict their treatment response at diagnosis.
This study has limitations. First, the retrospective nature of the analyses bears incoherent biases that could have affected the study findings, but we believe that these were partially solved via intra-observer and interobserver reproducibility assessments and DCA. Second, CT images were retrieved from different hospitals that could have different acquisition protocols that could have, to a certain extent, affected the radiomics analyses. Despite selecting patients who were similar in the general process of CT examination to reduce such a limitation, different protocols are implemented in the clinical practice at different institutions, and this could in turn potentially improve the reproducibility of the multi-institutional study findings. Third, despite this being a multicenter validated study with a large cohort of patients with locally advanced GC and considering the heterogeneous nature of GC among different populations and ethnicities, additional validation in larger prospective cohorts of patients is still warranted to further confirm the clinical significance of the proposed radiomics signature.
Using a large cohort of patients with locally advanced GC who received neoadjuvant treatment, we proposed and externally validated a radiomics signature to differentiate between potential respondents and nonrespondents to neoadjuvant chemotherapy. This signature could be used as a clinical treatment decision aid for oncologists to predict neoadjuvant chemotherapy response, thereby allowing timely planning of effective personalized treatments to potential nonrespondents. We expect the findings of this study to be used as a baseline for larger cohorts or prospective studies to improve individualized predictions of which patients with GC will respond to neoadjuvant treatment.
Accepted for Publication: June 1, 2021.
Published: August 19, 2021. doi:10.1001/jamanetworkopen.2021.21143
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Wang W et al. JAMA Network Open.
Corresponding Author: Zhiwei Zhou, MD, PhD, Department of Gastric Surgery, Sun Yat-sen University Cancer Center, 651 Dongfeng Rd E, Guangzhou, 510060, P. R. China (email@example.com).
Author Contributions: Drs Zhou and Liu had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Wang, Peng, Feng, and Zhao contributed equally to this work. Drs Wang, Peng, and Feng contributed equally as co–first authors. Drs Liu and Li are co–senior authors.
Concept and design: Wang, Zhao, Seeruttun, Li, Liu, Zhou.
Acquisition, analysis, or interpretation of data: Wang, Peng, Feng, Seeruttun, Zhang, Cheng, Li, Zhou.
Drafting of the manuscript: Wang, Zhao, Seeruttun, Cheng, Li, Zhou.
Critical revision of the manuscript for important intellectual content: Wang, Peng, Feng, Seeruttun, Zhang, Li, Liu, Zhou.
Statistical analysis: Wang, Peng, Seeruttun, Cheng, Li, Liu.
Obtained funding: Wang, Zhou.
Administrative, technical, or material support: Wang, Feng, Zhao, Seeruttun, Cheng, Liu, Zhou.
Supervision: Wang, Li, Zhou.
Conflict of Interest Disclosures: None reported.
Funding/Support: This work was funded by grants 2018A030313631 and 2021A1515011327 from the Natural Science Foundation of Guangdong Province, grant 2012013 from the Sun Yat-sen University Clinical Research 5010 Program, grant 19ykpy172 from the Young Teacher Training Program of Sun Yat-sen University, and grant 320.6750.17513 from the Wu Jieping Medical Foundation.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank Rong Zhang, Jing Li, and Jing Zhao from the Sun Yat-sen University Cancer Center for their contributions as the 3 experienced radiologists who were randomly assigned computed tomography images for determining interobserver and intra-observer reproducibility.
Additional Information: The authenticity of this article has been validated by uploading the key raw data onto the Research Data Deposit (RDD) public platform (www.researchdata.org.cn) with approval RDD No. RDDA2021002051.