A Multi-institutional International Analysis of Textbook Outcomes Among Patients Undergoing Curative-Intent Resection of Intrahepatic Cholangiocarcinoma | Gastroenterology | JAMA Surgery | JAMA Network
[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.  Textbook Outcomes by Year and by Hospital
Textbook Outcomes by Year and by Hospital

A, Trends in rates of textbook outcome across years. Black dots indicate individual institutions; B, Hospital variation in risk-adjusted percentages of a textbook outcome. In A and B, black dots indicate individual institutions, and solid lines indicate the mean value per year. In B, the dotted lines indicate the 95% CIs.

Figure 2.  Textbook Outcome Distribution by Its Definition
Textbook Outcome Distribution by Its Definition

A, Total patients. Values are 584 (85.0%) for negative margins, 487 (70.9%) for absence of perioperative transfusion, 398 (57.9%) for absence of complications, 356 (51.8%) for no prolonged hospital stay, 644 (93.7%) for no 30-day readmission, and 657 (95.6%) for no 30-day mortality. The solid line indicates the mean. B, Patients categorized by Eastern and Western geographic regions; values for Eastern and Western hospitals, respectively, are 234 (94.0%) and 350 (79.9%) for negative margins, 176 (70.7%) and 311 (71.0%) for absence of perioperative transfusion, 170 (68.3%) and 228 (52.1%) for absence of complications, 64 (25.7%) and 292 (66.7%) for no prolonged hospital stay, 244 (98.0%) and 400 (91.3%) for no 30-day readmission, and 246 (98.8%) and 410 (93.8%) for no 30-day mortality. The solid line indicates the mean for Eastern hospitals and the dotted line, the mean for Western hospitals.

Figure 3.  Nomogram for the Chances of Achieving a Textbook Outcome After Curative-Intent Resection of Intrahepatic Cholangiocarcinoma
Nomogram for the Chances of Achieving a Textbook Outcome After Curative-Intent Resection of Intrahepatic Cholangiocarcinoma
Table 1.  Baseline Characteristics of Patients Who Underwent Curative-Intent Resection of Intrahepatic Cholangiocarcinoma
Baseline Characteristics of Patients Who Underwent Curative-Intent Resection of Intrahepatic Cholangiocarcinoma
Table 2.  Logistic Regression Analysis of Preoperative and Intraoperative Factors Associated With the Textbook Outcomes
Logistic Regression Analysis of Preoperative and Intraoperative Factors Associated With the Textbook Outcomes
1.
Dijs-Elsinga  J, Otten  W, Versluijs  MM,  et al.  Choosing a hospital for surgery: the importance of information on quality of care.  Med Decis Making. 2010;30(5):544-555. doi:10.1177/0272989X09357474PubMedGoogle ScholarCrossref
2.
Rössler  F, Sapisochin  G, Song  G,  et al.  Defining benchmarks for major liver surgery: a multicenter analysis of 5202 living liver donors.  Ann Surg. 2016;264(3):492-500. doi:10.1097/SLA.0000000000001849PubMedGoogle ScholarCrossref
3.
Donabedian  A.  Twenty years of research on the quality of medical care: 1964-1984.  Eval Health Prof. 1985;8(3):243-265. doi:10.1177/016327878500800301PubMedGoogle ScholarCrossref
4.
Parina  RP, Chang  DC, Rose  JA, Talamini  MA.  Is a low readmission rate indicative of a good hospital?  J Am Coll Surg. 2015;220(2):169-176. doi:10.1016/j.jamcollsurg.2014.10.020PubMedGoogle ScholarCrossref
5.
Merath  K, Bagante  F, Chen  Q,  et al.  The impact of discharge timing on readmission following hepatopancreatobiliary surgery: a nationwide readmission database analysis.  J Gastrointest Surg. 2018;22(9):1538-1548. doi:10.1007/s11605-018-3783-0PubMedGoogle ScholarCrossref
6.
Kneuertz  PJ, Pitt  HA, Bilimoria  KY,  et al.  Risk of morbidity and mortality following hepato-pancreato-biliary surgery.  J Gastrointest Surg. 2012;16(9):1727-1735. doi:10.1007/s11605-012-1938-yPubMedGoogle ScholarCrossref
7.
Gani  F, Lucas  DJ, Kim  Y, Schneider  EB, Pawlik  TM.  Understanding variation in 30-day surgical readmission in the era of accountable care: effect of the patient, surgeon, and surgical subspecialties.  JAMA Surg. 2015;150(11):1042-1049. doi:10.1001/jamasurg.2015.2215PubMedGoogle ScholarCrossref
8.
Lucas  DJ, Pawlik  TM.  Quality improvement in gastrointestinal surgical oncology with American College of Surgeons National Surgical Quality Improvement Program.  Surgery. 2014;155(4):593-601. doi:10.1016/j.surg.2013.12.001PubMedGoogle ScholarCrossref
9.
Dimick  JB, Staiger  DO, Baser  O, Birkmeyer  JD.  Composite measures for predicting surgical mortality in the hospital.  Health Aff (Millwood). 2009;28(4):1189-1198. doi:10.1377/hlthaff.28.4.1189PubMedGoogle ScholarCrossref
10.
Shahian  DM, Normand  SL, Torchiana  DF,  et al.  Cardiac surgery report cards: comprehensive review and statistical critique.  Ann Thorac Surg. 2001;72(6):2155-2168. doi:10.1016/S0003-4975(01)03222-2PubMedGoogle ScholarCrossref
11.
Dimick  JB, Welch  HG, Birkmeyer  JD.  Surgical mortality as an indicator of hospital quality: the problem with small sample size.  JAMA. 2004;292(7):847-851. doi:10.1001/jama.292.7.847PubMedGoogle ScholarCrossref
12.
Dimick  JB, Welch  HG.  The zero mortality paradox in surgery.  J Am Coll Surg. 2008;206(1):13-16. doi:10.1016/j.jamcollsurg.2007.07.032PubMedGoogle ScholarCrossref
13.
Dimick  JB, Birkmeyer  NJ, Finks  JF,  et al.  Composite measures for profiling hospitals on bariatric surgery performance.  JAMA Surg. 2014;149(1):10-16. doi:10.1001/jamasurg.2013.4109PubMedGoogle ScholarCrossref
14.
Dimick  JB, Staiger  DO, Osborne  NH, Nicholas  LH, Birkmeyer  JD.  Composite measures for rating hospital quality with major surgery.  Health Serv Res. 2012;47(5):1861-1879. doi:10.1111/j.1475-6773.2012.01407.xPubMedGoogle ScholarCrossref
15.
Kolfschoten  NE, Kievit  J, Gooiker  GA,  et al.  Focusing on desired outcomes of care after colon cancer resections; hospital variations in ‘textbook outcome’.  Eur J Surg Oncol. 2013;39(2):156-163. doi:10.1016/j.ejso.2012.10.007PubMedGoogle ScholarCrossref
16.
Marang-van de Mheen  PJ, Dijs-Elsinga  J, Otten  W,  et al.  The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment.  Med Decis Making. 2011;31(6):816-827. doi:10.1177/0272989X10386799PubMedGoogle ScholarCrossref
17.
Karthaus  EG, Lijftogt  N, Busweiler  LAD,  et al; Dutch Society of Vascular Surgery, the Steering Committee of the Dutch Surgical Aneurysm Audit, the Dutch Institute for Clinical Auditing.  Textbook outcome: a composite measure for quality of elective aneurysm surgery.  Ann Surg. 2017;266(5):898-904. doi:10.1097/SLA.0000000000002388PubMedGoogle ScholarCrossref
18.
Busweiler  LA, Schouwenburg  MG, van Berge Henegouwen  MI,  et al; Dutch Upper Gastrointestinal Cancer Audit (DUCA) group.  Textbook outcome as a composite measure in oesophagogastric cancer surgery.  Br J Surg. 2017;104(6):742-750. doi:10.1002/bjs.10486PubMedGoogle ScholarCrossref
19.
Salet  N, Bremmer  RH, Verhagen  MAMT,  et al.  Is textbook outcome a valuable composite measure for short-term outcomes of gastrointestinal treatments in the Netherlands using hospital information system data? a retrospective cohort study.  BMJ Open. 2018;8(2):e019405. doi:10.1136/bmjopen-2017-019405PubMedGoogle ScholarCrossref
20.
Davison  ACHD.  Bootstrap Methods and Their Application. Cambridge, United Kingdom: Cambridge University Press; 1997. doi:10.1017/CBO9780511802843
21.
Mayer  EK, Bottle  A, Aylin  P, Darzi  AW, Vale  JA, Athanasiou  T.  What is the role of risk-adjusted funnel plots in the analysis of radical cystectomy volume-outcome relationships?  BJU Int. 2011;108(6):844-850. doi:10.1111/j.1464-410X.2010.09896.xPubMedGoogle ScholarCrossref
22.
Lawson  EH, Zingmond  DS, Stey  AM, Hall  BL, Ko  CY.  Measuring risk-adjusted value using Medicare and ACS-NSQIP: is high-quality, low-cost surgical care achievable everywhere?  Ann Surg. 2014;260(4):668-677. doi:10.1097/SLA.0000000000000931PubMedGoogle ScholarCrossref
23.
Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. https://www.ncbi.nlm.nih.gov/pubmed/25057539. Published 2001. Accessed March 13, 2019.
24.
Ejaz  A, Spolverato  G, Kim  Y,  et al.  Impact of blood transfusions and transfusion practices on long-term outcome following hepatopancreaticobiliary surgery.  J Gastrointest Surg. 2015;19(5):887-896. doi:10.1007/s11605-015-2776-5PubMedGoogle ScholarCrossref
25.
Carson  JL, Grossman  BJ, Kleinman  S,  et al; Clinical Transfusion Medicine Committee of the AABB.  Red blood cell transfusion: a clinical practice guideline from the AABB*.  Ann Intern Med. 2012;157(1):49-58. doi:10.7326/0003-4819-157-1-201206190-00429PubMedGoogle ScholarCrossref
26.
Ferraris  VA, Brown  JR, Despotis  GJ,  et al; Society of Thoracic Surgeons Blood Conservation Guideline Task Force; Society of Cardiovascular Anesthesiologists Special Task Force on Blood Transfusion; International Consortium for Evidence Based Perfusion.  2011 update to the Society of Thoracic Surgeons and the Society of Cardiovascular Anesthesiologists blood conservation clinical practice guidelines.  Ann Thorac Surg. 2011;91(3):944-982. doi:10.1016/j.athoracsur.2010.11.078PubMedGoogle ScholarCrossref
27.
Schneider  EB, Hyder  O, Wolfgang  CL,  et al.  Provider versus patient factors impacting hospital length of stay after pancreaticoduodenectomy.  Surgery. 2013;154(2):152-161. doi:10.1016/j.surg.2013.03.013PubMedGoogle ScholarCrossref
28.
Hyder  O, Sachs  T, Ejaz  A, Spolverato  G, Pawlik  TM.  Impact of hospital teaching status on length of stay and mortality among patients undergoing complex hepatopancreaticobiliary surgery in the USA.  J Gastrointest Surg. 2013;17(12):2114-2122. doi:10.1007/s11605-013-2349-4PubMedGoogle ScholarCrossref
29.
Tiessen  J, Kambara  H, Sakai  T, Kato  K, Yamauchi  K, McMillan  C.  What causes international variations in length of stay: a comparative analysis for two inpatient conditions in Japanese and Canadian hospitals.  Health Serv Manage Res. 2013;26(2-3):86-94. doi:10.1177/0951484813512287PubMedGoogle ScholarCrossref
30.
Ishikawa  KB, Holt  M, Kaihara  S, Yoshikawa  A, Hopkins  DS, Mason  M.  Performance, characteristics, and case mix in Japanese and American teaching hospitals.  Med Care. 1993;31(6):542-551. doi:10.1097/00005650-199306000-00008PubMedGoogle ScholarCrossref
31.
Shariat  SF, Karakiewicz  PI, Suardi  N, Kattan  MW.  Comparison of nomograms with other methods for predicting outcomes in prostate cancer: a critical analysis of the literature.  Clin Cancer Res. 2008;14(14):4400-4407. doi:10.1158/1078-0432.CCR-07-4713PubMedGoogle ScholarCrossref
32.
Merath  K, Bagante  F, Beal  EW,  et al.  Nomogram predicting the risk of recurrence after curative-intent resection of primary non-metastatic gastrointestinal neuroendocrine tumors: an analysis of the U.S. Neuroendocrine Tumor Study Group.  J Surg Oncol. 2018;117(5):868-878. doi:10.1002/jso.24985PubMedGoogle ScholarCrossref
33.
Mavros  MN, Mayo  SC, Hyder  O, Pawlik  TM.  A systematic review: treatment and prognosis of patients with fibrolamellar hepatocellular carcinoma.  J Am Coll Surg. 2012;215(6):820-830. doi:10.1016/j.jamcollsurg.2012.08.001PubMedGoogle ScholarCrossref
34.
Shimada  H, Endo  I, Togo  S, Nakano  A, Izumi  T, Nakagawara  G.  The role of lymph node dissection in the treatment of gallbladder carcinoma.  Cancer. 1997;79(5):892-899. doi:10.1002/(SICI)1097-0142(19970301)79:5<892::AID-CNCR4>3.0.CO;2-EPubMedGoogle ScholarCrossref
35.
Bagante  F, Gani  F, Spolverato  G,  et al.  Intrahepatic cholangiocarcinoma: prognosis of patients who did not undergo lymphadenectomy.  J Am Coll Surg. 2015;221(6):1031-40.e1, 4. doi:10.1016/j.jamcollsurg.2015.09.012PubMedGoogle ScholarCrossref
36.
Zhang  XF, Chen  Q, Kimbrough  CW,  et al.  Lymphadenectomy for intrahepatic cholangiocarcinoma: has nodal evaluation been increasingly adopted by surgeons over time? a national database analysis.  J Gastrointest Surg. 2018;22(4):668-675. doi:10.1007/s11605-017-3652-2PubMedGoogle ScholarCrossref
37.
Zhang  XF, Chakedis  J, Bagante  F,  et al.  Trends in use of lymphadenectomy in surgery with curative intent for intrahepatic cholangiocarcinoma.  Br J Surg. 2018;105(7):857-866. doi:10.1002/bjs.10827PubMedGoogle ScholarCrossref
38.
Spolverato  G, Bagante  F, Weiss  M,  et al.  Comparative performances of the 7th and the 8th editions of the American Joint Committee on Cancer staging systems for intrahepatic cholangiocarcinoma.  J Surg Oncol. 2017;115(6):696-703. doi:10.1002/jso.24569Google ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    April 24, 2019

    A Multi-institutional International Analysis of Textbook Outcomes Among Patients Undergoing Curative-Intent Resection of Intrahepatic Cholangiocarcinoma

    Author Affiliations
    • 1The Ohio State University Wexner Medical Center, Columbus
    • 2University of Verona, Verona, Italy
    • 3Fundeni Clinical Institute, Bucharest, Romania
    • 4Curry Cabral Hospital, Lisbon, Portugal
    • 5Ospedale San Raffaele, Milan, Italy
    • 6Emory University, Atlanta, Georgia
    • 7University of Sydney, Sydney, Australia
    • 8Johns Hopkins Hospital, Baltimore, Maryland
    • 9University of Virginia, Charlottesville, Virginia
    • 10Eastern Hepatobiliary Surgery Hospital, Shanghai, China
    • 11Stanford University, Stanford, California
    • 12Beaujon Hospital, Clichy, France
    • 13University of Ottawa, Ontario, Canada
    • 14Erasmus University Medical Center, Rotterdam, the Netherlands
    • 15Yokohama City University, Yokohama, Japan
    JAMA Surg. 2019;154(6):e190571. doi:10.1001/jamasurg.2019.0571
    Key Points

    Question  To determine the incidence of so-called textbook outcomes, a composite measure of the quality of surgical care, among patients undergoing curative-intent resection of intrahepatic cholangiocarcinoma.

    Findings  In this cohort study of 687 patients undergoing the reference procedure, a textbook outcome was achieved in 175 patients (25.5%). A nomogram to assess the probability of textbook outcome was developed and had good accuracy in both the training data set (area under the curve, 0.755) and validation data set (area under the curve, 0.763).

    Meaning  While hepatic resection for intrahepatic cholangiocarcinoma was performed with less than 5% mortality in specialized centers, a textbook outcome appears to have been achieved in only one-quarter of patients.

    Abstract

    Importance  Composite measures may be superior to individual measures for the analysis of hospital performance and quality of surgical care.

    Objective  To determine the incidence of a so-called textbook outcome, a composite measure of the quality of surgical care, among patients undergoing curative-intent resection of intrahepatic cholangiocarcinoma.

    Design, Setting, and Participants  This cohort study involved an analysis of a multinational, multi-institutional cohort of patient from 15 major hepatobiliary centers in North America, Europe, Australia, and Asia who underwent curative-intent resection of intrahepatic cholangiocarcinoma between 1993 and 2015. Data analysis was conducted from April 2018 to May 2018.

    Main Outcomes and Measures  Hospital variation in the composite end point of textbook outcome, defined as negative margins, no perioperative transfusion, no postoperative surgical complications, no prolonged length of stay, no 30-day readmissions, and no 30-day mortality. Secondary end points were factors associated with achieving textbook outcomes.

    Results  Among 687 patients (of whom 370 [53.9%] were men; median patient age, 61 [range, 18-86] years) undergoing curative-intent resection of intrahepatic cholangiocarcinoma, a textbook outcome was achieved in 175 patients (25.5%). Being 60 years or younger (odds ratio [OR], 1.61 [95% CI, 1.04-2.49]; P = .03), absence of preoperative jaundice (OR, 4.40 [95% CI, 1.28-15.15]; P = .02), no neoadjuvant chemotherapy (OR, 2.57 [95% CI, 1.05-6.29]; P = .04), T1a/T1b-stage disease (OR, 1.58 [95% CI, 1.01-2.49]; P = .049), N0 status (OR, 3.89 [95% CI, 1.77-8.54]; P = .001), and no bile duct resection (OR, 2.46 [95% CI, 1.25-4.84]; P = .009) were independently associated with achieving a textbook outcome after resection. A prolonged length of stay had the greatest negative association with a textbook outcome. A nomogram to assess the probability of textbook outcome was developed and had good accuracy in both the training data set (area under the curve, 0.755) and validation data set (area under the curve, 0.763).

    Conclusions and Relevance  In this study, while hepatic resection for intrahepatic cholangiocarcinoma was performed with less than 5% mortality in specialized centers, a textbook outcome was achieved in only approximately 26% of patients. A textbook outcome may be useful for the reporting of patient-level hospital performance and hospital variation, leading to quality improvement efforts after resection of intrahepatic cholangiocarcinoma.

    Introduction

    Because surgical outcomes can vary widely among hospitals and surgeons, the quality of surgical care is an increasing area of interest. Specifically, there is a need for better hospital quality indicators associated with surgery, especially among patients undergoing complex operative procedures.1 Although the analysis of individual outcome measures, such as mortality, morbidity, length of stay (LOS), and readmission, has advantages, composite benchmarks of surgical quality may be more relevant and helpful.2-8 In particular, data on a single indicator do not reflect the whole surgical process and may not reliably measure overall hospital quality.9-12 Composite measures combine information from multiple domains into a single summary measure and therefore may be superior to individual measures for the analysis of hospital performance.9,13,14

    One composite measure is the textbook outcome (TO), which provides a comprehensive summary of hospital performance.15 Specifically, TO includes multiple desirable postoperative outcomes that, when achieved, represent the ideal (so-called textbook) hospitalization. In this manner, TO reflects overall quality across all domains of performance. In addition, TO may be easier for both patients and clinicians to interpret when analyzing quality trends, rather than trying to interpret different individual performance indicators.1,16 As a result, TO may be a more patient-centered benchmark to determine the ideal hospitalization from the patient perspective.

    Although TO has been examined for elective aneurysm repair and colorectal and esophagogastric cancer, the use of TO to define quality among patients diagnosed with intrahepatic cholangiocarcinoma (ICC) has not been evaluated.15,17-19 To that end, the aim of the current study was to define and assess TO among patients undergoing curative-intent resection of ICC. Specifically, using a multi-institutional database composed of high-volume specialized hepatobiliary centers, we assessed hospital variation associated with the TO composite measure. In addition, we developed and validated a nomogram to identify the chance that a patient would experience a TO after curative-intent resection of ICC.

    Methods
    Data Sources and Study Population

    A multi-institutional database representing 15 major tertiary hepatobiliary centers in the United States, Canada, Europe, Australia, and Asia was used to identify patients who underwent resection of ICC between 1993 and 2015. Patients with ICC who underwent hepatectomy with curative intent and for whom information was available on the type of surgery, surgical margins, perioperative transfusion, postoperative complications, LOS, 30-day readmission, and 30-day mortality were included. The study was approved by the institutional review board of each participating institution and was considered exempt from informed consent procedures.

    Textbook Outcome

    The TO for this analysis was defined based on a selection of 6 relevant outcome parameters representing the optimal patient outcome after surgery: negative margins, no perioperative transfusion, no postoperative surgical complications, no prolonged LOS (an LOS ≤50th percentile of the total cohort), no readmissions within 30 days after discharge, and no postoperative mortality within 30 days after surgery. Tumor margin was categorized as microscopically negative (R0) or microscopically positive (R1); a positive resection margin (R1) was defined as either a margin width less than 1 mm or involved margins. No patient in the study had macroscopically positive margins (R2). A perioperative transfusion was defined as the occurrence of an intraoperative or postoperative transfusion. Readmission was defined as admission to any hospital within 30 days after discharge. Mortality was defined as death within 30 days of the index operation. When all 6 desired outcomes were achieved, the patient was categorized as having experienced a TO.

    Statistical Analysis

    Categorical variables were presented as frequencies, and continuous variables were presented as median and interquartile ranges (IQR) or means and SDs. Patient, tumor, and treatment characteristics were compared among patients who did or did not have a TO via the Pearson χ2 test for categorical variables and the Kruskal-Wallis test for continuous variables. Multivariable logistic regression was used to investigate possible associations between patient, tumor, treatment characteristics, and TO. Variables in the models were selected based on factors established in the literature as well as on the basis of the lowest Akaike information criterion value. A sensitivity analysis was performed to examine the outcome that changes in preoperative jaundice, neoadjuvant chemotherapy, nodal metastasis, and bile duct resection had on the results of the multivariable analysis. The final multivariable model was used to develop a nomogram; a validation data set was constructed using the bootstrap method, and internal validation of the predictive model was performed by the bootstrap resampling method. Bootstrap validation is a method of randomly resampling with replacement from an original data set for conducting statistical inference.20 Model performance was assessed using receiver operating characteristic curves and the area under the curve (AUC).

    To analyze hospital variation in TO, risk-adjusted hospital results were calculated by adjusting for patient age, American Society of Anesthesiologists score, preoperative jaundice, perineural invasion, biliary invasion, liver capsule involvement, American Joint Committee on Cancer Eighth Edition N stages and T stages, type of surgical procedure (ie, minor vs major hepatectomy), neoadjuvant chemotherapy, and bile duct resection. Individual hospital results were displayed using funnel plots combining scatter plot and a sequence of 95.0% and 99.8% CIs.21 A subanalysis was performed to evaluate TO that included and excluded LOS to define potential differences in TO at Eastern centers (in Asia) vs Western centers (in North America, Europe, and Australia), given possible cultural differences in LOS practices. A P value less than .05 was considered statistically significant. No multiplicity adjustment was performed for the P values. All statistical analyses were performed with Stata version 14.0 MP statistical software (StataCorp).

    Results
    Patient, Tumor, and Operative Characteristics

    A total of 687 patients undergoing curative-intent resection of ICC were included in the analytic cohort. More than half of the patients were male (370 [53.9%]), and the median (range) patient age was 61 (18-86) years. Most patients had a solitary tumor (582 [84.7%]), and the median (range) size of the largest lesion was 6.0 (0.88-25.0) cm. At the time of surgical resection, the extent of hepatic resection was a minor hepatectomy in 29.0% of patients (199) and a major hepatectomy in 71.0% of patients (488). On final pathologic analysis, the margin status was microscopically positive (R1) in 103 patients (15.0%) and microscopically negative (R0) in 584 patients (85.0%). Moreover, 84 patients (12.2%) had vascular invasion, while 107 patients (15.6%) had biliary invasion.

    Textbook Outcome After Resection of ICC

    The unadjusted incidence of TO varied during the study, ranging from 9.6% to 53.9% (Figure 1A). Achievement of each outcome indicator was calculated separately to identify which indicators were met. Specifically, for each outcome indicator, the percentage of patients for whom that specific outcome and all previous outcome indicators were realized was calculated (Figure 2A). Overall, TO was ultimately achieved in only 175 patients (25.5%) (Table 1). For example, while most patients experienced each TO outcome, such as no 30-day mortality (657 [95.6%]), negative surgical margins (584 [85.0%]), and no perioperative transfusion (487 [70.9%]), other TO outcomes, including no complications (398 [57.9%]) and no prolonged LOS (356 [51.8%]), were achieved much less frequently. In fact, a prolonged LOS had the greatest negative association with attaining a TO (in that only 356 patients [51.8%] had no prolonged hospital stay; Figure 2A). Among patients undergoing minor liver resections, the mean (SD) LOS was 13.7 (8.5) days, while for patients undergoing major liver resections, the mean (SD) LOS was 16.1 (15.3) days.

    When hospitals were stratified by Eastern and Western locations, Eastern hospitals had lower rates of TO (64 [25.7%]) than Western hospitals (292 [66.7%]), which was almost entirely attributable to lower rates of achieving no prolonged LOS (Figure 2B). When a subanalysis was conducted excluding LOS from the definition of TO, Eastern hospitals demonstrated higher rates of TO compared with Western hospitals (eFigure 1 in the Supplement).

    The adjusted percentage of patients with a TO for each individual hospital relative to center volume was examined using a funnel plot. No overall association between ICC resection volume and TO was found (Figure 1B).

    Factors Associated With TO

    Factors that reduced the chances of a TO included preoperative jaundice (patients who did not have a TO, 488 of 512 [87.5%]; patients who had a TO, 172 of 175 [98.3%]; P < .001), a mixed periductal infiltrative or a mass-forming subtype (patients who did not have a TO, 66 of 512 [12.9%]; patients who had a TO, 5 of 175 [2.9%]; P < .001), major vascular invasion (patients who did not have a TO, 434 of 512 [84.8%]; patients who had a TO, 167 of 175 [95.4%]; P < .001), advanced T-stage disease (T2, T3, and T4 combined; patients who did not have a TO, 301 of 512 [58.8%]; patients who had a TO, 86 of 175 [49.1%]; P < .001), and nodal metastasis (N1; 109 of 512 [21.3%]; 11 of 175 [6.3%]; P < .001). In addition, patients who underwent a major hepatectomy (patients who did not have a TO, 378 of 512 [73.8%]; patients who had a TO, 110 of 175 [62.9%]; P = .006), as well as patients who required bile duct resection (patients who did not have a TO, 123 of 512 [24.0%]; patients who had a TO, 13 of 175 [7.4%]; P < .001), were also less likely to experience a TO after surgery. In contrast, patients who were more likely to experience a TO had less aggressive tumor characteristics. Specifically, patients in the TO group were less likely to present with preoperative jaundice and more likely to have a mass-forming ICC subtype (patients who did not have a TO, 401 of 512 [78.3%]; patients who had a TO, 146 of 175 [83.4%]; P < .001). In addition, patients who had a TO were less likely to have major vascular invasion (patients who did not have a TO, 76 of 512 [14.8%]; patients who had a TO, 8 of 175 [4.6%]; P < .001), as well as advanced T-stage or N-stage disease. Moreover, patients who experienced a TO were less likely to have received neoadjuvant chemotherapy (patients who did not have a TO, 60 of 512 [11.7%]; patients who had a TO, 6 of 175 [3.4%]; P = .001; Table 1).

    In multivariable analysis, after controlling for competing risk factors, an age younger than 60 years (odds ratio [OR], 1.61 [95% CI, 1.04-2.49]; P = .03), absence of preoperative jaundice (OR, 4.40 [95% CI, 1.28-15.15]; P = .02), no neoadjuvant chemotherapy (OR, 2.57 [95% CI, 1.05-6.29]; P = .04), no nodal metastasis (OR, 3.89 [95% CI, 1.77-8.54]; P = .001), T1a/T1b-stage disease (OR, 1.58 [95% CI, 1.01-2.49]; P = .049), and no bile duct resection (OR, 2.46 [95% CI, 1.25-4.84]; P = .009) were each associated with a higher likelihood of achieving a TO (Table 2). Using data from the multivariable analysis, a nomogram to assess the probability that a patient would experience a TO after resection of ICC was developed based on clinically relevant factors (Figure 3). Each factor in the nomogram was assigned a weighted number of points, and the sum of points for each patient was associated with a specific probability of a TO. For example, a patient who was 60 years or younger (3.5 points), had no preoperative jaundice (10 points), did not undergo neoadjuvant chemotherapy (7.5 points), underwent minor hepatectomy (1.0 point), had no major vascular invasion (6.0 points), had a T1a tumor (3.0 points), had N0-stage disease (9.5 points) and had no bile duct resection (6.0 points) scored 46 points, indicating a greater than 50% probability of a TO after surgery. In contrast, a patient who was older than 60 years, did not have preoperative jaundice (10 points), had neoadjuvant chemotherapy (7.5 points), underwent major hepatectomy (0 points), had major vascular invasion (0 points), had N1-stage disease (0 points), had a T1b tumor (3 points), and did not undergo bile duct resection (6 points) would score 26.5 points, indicating a probability of achieving a TO of only 8%. The nomogram had good accuracy in both the training data set (AUC, 0.755) and validation data set (AUC, 0.763).

    Discussion

    A TO is a composite measure that represents the most favorable (or textbook) postoperative course. To the best of our knowledge, this is the first study to examine TO as a composite quality measure in the assessment of short-term outcomes after curative-intent resection of ICC. The current study was important because the data demonstrated that a TO was achieved in only 25.5% of patients (n = 175) who underwent curative-intent resection of ICC. In addition, while TO rates ranged from 9.6% to 53.9% during the study period, there was no association between hospitals’ procedure volume and TO rates. Of note, the LOS outcome parameter had the greatest negative association with TO, followed by postoperative complications. In particular, while TO parameters such as attainment of a negative margin and avoidance of transfusion were readily accomplished among most patients, no complications and no prolonged LOS were more difficult to achieve. Using clinical relevant factors, a nomogram to prognosticate TO was proposed and internally validated. The nomogram had a good accuracy to assess the probability of achieving a TO after resection of ICC.

    Traditionally, the assessment of quality of care in surgery has relied on individual outcomes such as morbidity, mortality, hospital LOS, and readmission.4,11,22 The analysis of individual metrics as a global assessment of quality may be inherently flawed, especially from the patient perspective, because individual patients experience their hospital course as an all-or-none phenomena. According to the Institute of Medicine,23 health care should be safe, effective, patient-centered, timely, efficient, and equitable. The surgical process can be considered safe for patients undergoing potentially curative resection of ICC when no adverse outcomes, such as mortality and morbidity, have occurred and effective if complete tumor removal without perioperative transfusion has been achieved.24-26 These goals were included in the TO in the current study. In turn, while it may be interesting to assess individual quality metrics from a systems perspective, from the patient’s perspective only 25.5% of all individuals who underwent resection of ICC experienced a TO. Specifically, although no 30-day mortality was achieved in 657 patients (95.6%), other parameters such as no prolonged LOS (356 [51.8%]), no postoperative complication (398 [57.9%]) and no perioperative transfusion (487 [70.9%]) were achieved in fewer patients (Figure 2A). Collectively, these data have important implications as they suggest that only 1 in 4 patients will have a complete TO after hepatic resection of ICC.

    Hospital LOS is an important benchmark to assess health care quality in the United States.27,28 Similar to data in the current study, data on TO for other types of surgical procedures described prolonged LOS as the main obstacle for achieving a TO.15,17,18 In fact, only roughly one-half of patients undergoing resection of ICC in this study experienced no prolonged LOS. Of note, the incidence of no prolonged LOS was markedly different among Eastern hospitals (25.7%) and Western hospitals (66.7%; Figure 2B). The reason why LOS was longer among Eastern countries was undoubtedly multifactorial. For example, Japanese hospitals have a larger supply of acute care beds and smaller number of long-term care beds compared with hospitals worldwide, which could affect their LOS.29 In addition, differences in LOS can be attributed to major cultural and organizational differences between health care systems, as well as economic incentives or disincentives associated with ownership and funding mechanisms.30 Because of the recognized differences in LOS in the Eastern vs Western hospitals, additional subanalyses were performed. Of note, the prolonged LOS in the Eastern hospitals was the main reason for the lower rates of TO in Asian hospitals vs those in the United States and Europe. In fact, when LOS was removed from the definition of TO, Eastern hospitals had higher rates of TO compared with Western hospitals (eFigure 1 in the Supplement). These data illustrate that quality associated with surgical process for patients with ICC was multidimensional. In turn, relying solely on a single metric for quality assessment among patients undergoing complex surgical procedures is not ideal.

    Several patient and procedural factors also influenced the probability of achieving a TO after resection of ICC (Table 2). Specifically, factors that were independently associated with an increased probability of TO included an age of 60 years or younger, absence of preoperative jaundice, no neoadjuvant chemotherapy, T1a/T1b disease, N0 status, and absence of bile duct resection. Using factors relevant to clinical practice, a nomogram was developed to assess the probability of a patient achieving a TO after curative-intent resection of ICC (Figure 3). Recently, nomograms have been proposed as prognostic tools that may better estimate an individual risk or the chance of an outcome using specific clinical variables.31,32 In this study, the nomogram demonstrated a good prognosticative ability on both the development and test data sets. The use of the proposed nomogram as a preoperative risk assessment tool may help identify patients at increased risk of an unfavorable postoperative course. For example, such a nomogram may aid in decision making in preoperative patient selection, as well as postoperative targeted approaches to improve the care of patients undergoing resection of ICC.

    For some solid gastrointestinal and hepatobiliary cancers, performance of lymphadenectomy might be considered part of the TO from an oncological perspective, similar to achieving an R0 resection.33,34 Despite the American Joint Committee on Cancer 8th edition recommendations, the role of lymphadenectomy as a routine part of the surgical procedure for ICC remains controversial.35 While some surgeons consider it standard, others perform lymphadenectomy only in selective circumstances, with studies36,37 demonstrating that lymphadenectomy use varies by geographical region. Interestingly, in the current study, there was a lower incidence of lymphadenectomy use during the early years of the study period, while the rates of lymphadenectomy after 2010 increased, coinciding with the introduction of the first disease-specific staging for ICC in the 7th edition of the American Joint Committee on Cancer manual (eFigure 2 in the Supplement).38 While we believe that the collective data strongly support the routine inclusion of lymphadenectomy as part of the surgery for ICC, owing to its emerging and controversial role, lymphadenectomy was not included as a parameter in the definition of TO. Interestingly, 109 of 512 patients (21.3%) who did not experience a TO had pN1-stage disease, vs 11 of the 175 patients (6.3%) who experienced TO. Future studies will need to define whether these differences may be associated with more aggressive biology or the extent of surgery (ie, more extensive surgery leading to potentially more complications).

    Limitations

    There are several limitations that should be considered when interpreting the results of this study. The outcomes included in the definition of TO were limited to the variables available in the multi-institutional database, as well as events within 30 days of surgery. As with all retrospective studies of surgical procedures, the current cohort may have been subject to selection bias. In addition, other measures of quality in hepatic surgery, such as patient satisfaction, were not available and could not be evaluated. Moreover, hospital volume was calculated based on the number of curative-intent resections of ICC that each participating hospital performed. Given the relatively rarity of ICC and the high incidence of other types of liver tumors that require surgery, hospital hepatic procedural volumes were undoubtedly underestimated.

    Furthermore, the data were derived from a large number of hepatobiliary centers from the United States, Europe, and Asia; therefore, patients were heterogeneous in demographic, clinical, and tumor-associated characteristics. However, this heterogeneity can be viewed as a major strength, in that it allowed us to study a real-world cohort, thereby facilitating generalizability of the findings.

    Finally, it is important to note that the results of the current study do not imply that patients who did not meet all indicators for TO were treated incorrectly. Medical and surgical complexity and the specific needs or demands of individual patients may have been valid reasons to divert from TO. However, the data do provide important information for patients on how often a TO should be expected after resection of ICC.

    Conclusions

    In conclusion, while hepatic resection for ICC was performed with less than 5% mortality in specialized centers, rates of TO were only 25.5%. In addition, there was a wide variation in which TO indicators were achieved among patients undergoing surgical resection. Data from the current study demonstrated that TO is a simple and feasible composite measure of desired outcomes after surgical resection of ICC. Therefore, TO may be useful for the reporting of patient-level hospital performance and hospital variation, leading to quality improvement efforts. A TO may be a useful a tool for aiding shared decision making among patient and clinicians to guide preoperative patient selection, as well as postoperative quality improvement projects after resection of ICC.

    Back to top
    Article Information

    Accepted for Publication: January 15, 2019.

    Corresponding Author: Timothy M. Pawlik, MD, MPH, PhD, The Ohio State University, Wexner Medical Center, 395 W 12th Ave, Ste 670, Columbus, OH 43210 (tim.pawlik@osumc.edu).

    Published Online: April 24, 2019. doi:10.1001/jamasurg.2019.0571

    Author Contributions: Dr Merath and Ms Chen had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Merath, Chen, Pulitano, Weiss, Groot Koerkamp, Guglielmi, Pawlik.

    Acquisition, analysis, or interpretation of data: Merath, Chen, Bagante, Alexandrescu, Marques, Aldrighetti, Maithel, Pulitano, Bauer, Shen, Poultsides, Soubrane, Martel, Groot Koerkamp, Endo, Cloyd, Pawlik.

    Drafting of the manuscript: Merath, Chen, Groot Koerkamp, Pawlik.

    Critical revision of the manuscript for important intellectual content: Chen, Bagante, Alexandrescu, Marques, Aldrighetti, Maithel, Pulitano, Weiss, Bauer, Shen, Poultsides, Soubrane, Martel, Groot Koerkamp, Guglielmi, Endo, Cloyd, Pawlik.

    Statistical analysis: Chen, Bagante, Groot Koerkamp, Pawlik.

    Obtained funding: None.

    Administrative, technical, or material support: Chen, Pulitano, Bauer, Shen, Poultsides, Martel, Endo, Pawlik.

    Supervision: Merath, Marques, Aldrighetti, Weiss, Poultsides, Groot Koerkamp, Guglielmi, Pawlik.

    Conflict of Interest Disclosures: Dr Bauer reported equity ownership from iTi Health outside the submitted work. No other disclosures were reported.

    Disclaimer: Dr Pawlik is Deputy Editor of JAMA Surgery, but he was not involved in any of the decisions regarding review of the manuscript or its acceptance.

    References
    1.
    Dijs-Elsinga  J, Otten  W, Versluijs  MM,  et al.  Choosing a hospital for surgery: the importance of information on quality of care.  Med Decis Making. 2010;30(5):544-555. doi:10.1177/0272989X09357474PubMedGoogle ScholarCrossref
    2.
    Rössler  F, Sapisochin  G, Song  G,  et al.  Defining benchmarks for major liver surgery: a multicenter analysis of 5202 living liver donors.  Ann Surg. 2016;264(3):492-500. doi:10.1097/SLA.0000000000001849PubMedGoogle ScholarCrossref
    3.
    Donabedian  A.  Twenty years of research on the quality of medical care: 1964-1984.  Eval Health Prof. 1985;8(3):243-265. doi:10.1177/016327878500800301PubMedGoogle ScholarCrossref
    4.
    Parina  RP, Chang  DC, Rose  JA, Talamini  MA.  Is a low readmission rate indicative of a good hospital?  J Am Coll Surg. 2015;220(2):169-176. doi:10.1016/j.jamcollsurg.2014.10.020PubMedGoogle ScholarCrossref
    5.
    Merath  K, Bagante  F, Chen  Q,  et al.  The impact of discharge timing on readmission following hepatopancreatobiliary surgery: a nationwide readmission database analysis.  J Gastrointest Surg. 2018;22(9):1538-1548. doi:10.1007/s11605-018-3783-0PubMedGoogle ScholarCrossref
    6.
    Kneuertz  PJ, Pitt  HA, Bilimoria  KY,  et al.  Risk of morbidity and mortality following hepato-pancreato-biliary surgery.  J Gastrointest Surg. 2012;16(9):1727-1735. doi:10.1007/s11605-012-1938-yPubMedGoogle ScholarCrossref
    7.
    Gani  F, Lucas  DJ, Kim  Y, Schneider  EB, Pawlik  TM.  Understanding variation in 30-day surgical readmission in the era of accountable care: effect of the patient, surgeon, and surgical subspecialties.  JAMA Surg. 2015;150(11):1042-1049. doi:10.1001/jamasurg.2015.2215PubMedGoogle ScholarCrossref
    8.
    Lucas  DJ, Pawlik  TM.  Quality improvement in gastrointestinal surgical oncology with American College of Surgeons National Surgical Quality Improvement Program.  Surgery. 2014;155(4):593-601. doi:10.1016/j.surg.2013.12.001PubMedGoogle ScholarCrossref
    9.
    Dimick  JB, Staiger  DO, Baser  O, Birkmeyer  JD.  Composite measures for predicting surgical mortality in the hospital.  Health Aff (Millwood). 2009;28(4):1189-1198. doi:10.1377/hlthaff.28.4.1189PubMedGoogle ScholarCrossref
    10.
    Shahian  DM, Normand  SL, Torchiana  DF,  et al.  Cardiac surgery report cards: comprehensive review and statistical critique.  Ann Thorac Surg. 2001;72(6):2155-2168. doi:10.1016/S0003-4975(01)03222-2PubMedGoogle ScholarCrossref
    11.
    Dimick  JB, Welch  HG, Birkmeyer  JD.  Surgical mortality as an indicator of hospital quality: the problem with small sample size.  JAMA. 2004;292(7):847-851. doi:10.1001/jama.292.7.847PubMedGoogle ScholarCrossref
    12.
    Dimick  JB, Welch  HG.  The zero mortality paradox in surgery.  J Am Coll Surg. 2008;206(1):13-16. doi:10.1016/j.jamcollsurg.2007.07.032PubMedGoogle ScholarCrossref
    13.
    Dimick  JB, Birkmeyer  NJ, Finks  JF,  et al.  Composite measures for profiling hospitals on bariatric surgery performance.  JAMA Surg. 2014;149(1):10-16. doi:10.1001/jamasurg.2013.4109PubMedGoogle ScholarCrossref
    14.
    Dimick  JB, Staiger  DO, Osborne  NH, Nicholas  LH, Birkmeyer  JD.  Composite measures for rating hospital quality with major surgery.  Health Serv Res. 2012;47(5):1861-1879. doi:10.1111/j.1475-6773.2012.01407.xPubMedGoogle ScholarCrossref
    15.
    Kolfschoten  NE, Kievit  J, Gooiker  GA,  et al.  Focusing on desired outcomes of care after colon cancer resections; hospital variations in ‘textbook outcome’.  Eur J Surg Oncol. 2013;39(2):156-163. doi:10.1016/j.ejso.2012.10.007PubMedGoogle ScholarCrossref
    16.
    Marang-van de Mheen  PJ, Dijs-Elsinga  J, Otten  W,  et al.  The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment.  Med Decis Making. 2011;31(6):816-827. doi:10.1177/0272989X10386799PubMedGoogle ScholarCrossref
    17.
    Karthaus  EG, Lijftogt  N, Busweiler  LAD,  et al; Dutch Society of Vascular Surgery, the Steering Committee of the Dutch Surgical Aneurysm Audit, the Dutch Institute for Clinical Auditing.  Textbook outcome: a composite measure for quality of elective aneurysm surgery.  Ann Surg. 2017;266(5):898-904. doi:10.1097/SLA.0000000000002388PubMedGoogle ScholarCrossref
    18.
    Busweiler  LA, Schouwenburg  MG, van Berge Henegouwen  MI,  et al; Dutch Upper Gastrointestinal Cancer Audit (DUCA) group.  Textbook outcome as a composite measure in oesophagogastric cancer surgery.  Br J Surg. 2017;104(6):742-750. doi:10.1002/bjs.10486PubMedGoogle ScholarCrossref
    19.
    Salet  N, Bremmer  RH, Verhagen  MAMT,  et al.  Is textbook outcome a valuable composite measure for short-term outcomes of gastrointestinal treatments in the Netherlands using hospital information system data? a retrospective cohort study.  BMJ Open. 2018;8(2):e019405. doi:10.1136/bmjopen-2017-019405PubMedGoogle ScholarCrossref
    20.
    Davison  ACHD.  Bootstrap Methods and Their Application. Cambridge, United Kingdom: Cambridge University Press; 1997. doi:10.1017/CBO9780511802843
    21.
    Mayer  EK, Bottle  A, Aylin  P, Darzi  AW, Vale  JA, Athanasiou  T.  What is the role of risk-adjusted funnel plots in the analysis of radical cystectomy volume-outcome relationships?  BJU Int. 2011;108(6):844-850. doi:10.1111/j.1464-410X.2010.09896.xPubMedGoogle ScholarCrossref
    22.
    Lawson  EH, Zingmond  DS, Stey  AM, Hall  BL, Ko  CY.  Measuring risk-adjusted value using Medicare and ACS-NSQIP: is high-quality, low-cost surgical care achievable everywhere?  Ann Surg. 2014;260(4):668-677. doi:10.1097/SLA.0000000000000931PubMedGoogle ScholarCrossref
    23.
    Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. https://www.ncbi.nlm.nih.gov/pubmed/25057539. Published 2001. Accessed March 13, 2019.
    24.
    Ejaz  A, Spolverato  G, Kim  Y,  et al.  Impact of blood transfusions and transfusion practices on long-term outcome following hepatopancreaticobiliary surgery.  J Gastrointest Surg. 2015;19(5):887-896. doi:10.1007/s11605-015-2776-5PubMedGoogle ScholarCrossref
    25.
    Carson  JL, Grossman  BJ, Kleinman  S,  et al; Clinical Transfusion Medicine Committee of the AABB.  Red blood cell transfusion: a clinical practice guideline from the AABB*.  Ann Intern Med. 2012;157(1):49-58. doi:10.7326/0003-4819-157-1-201206190-00429PubMedGoogle ScholarCrossref
    26.
    Ferraris  VA, Brown  JR, Despotis  GJ,  et al; Society of Thoracic Surgeons Blood Conservation Guideline Task Force; Society of Cardiovascular Anesthesiologists Special Task Force on Blood Transfusion; International Consortium for Evidence Based Perfusion.  2011 update to the Society of Thoracic Surgeons and the Society of Cardiovascular Anesthesiologists blood conservation clinical practice guidelines.  Ann Thorac Surg. 2011;91(3):944-982. doi:10.1016/j.athoracsur.2010.11.078PubMedGoogle ScholarCrossref
    27.
    Schneider  EB, Hyder  O, Wolfgang  CL,  et al.  Provider versus patient factors impacting hospital length of stay after pancreaticoduodenectomy.  Surgery. 2013;154(2):152-161. doi:10.1016/j.surg.2013.03.013PubMedGoogle ScholarCrossref
    28.
    Hyder  O, Sachs  T, Ejaz  A, Spolverato  G, Pawlik  TM.  Impact of hospital teaching status on length of stay and mortality among patients undergoing complex hepatopancreaticobiliary surgery in the USA.  J Gastrointest Surg. 2013;17(12):2114-2122. doi:10.1007/s11605-013-2349-4PubMedGoogle ScholarCrossref
    29.
    Tiessen  J, Kambara  H, Sakai  T, Kato  K, Yamauchi  K, McMillan  C.  What causes international variations in length of stay: a comparative analysis for two inpatient conditions in Japanese and Canadian hospitals.  Health Serv Manage Res. 2013;26(2-3):86-94. doi:10.1177/0951484813512287PubMedGoogle ScholarCrossref
    30.
    Ishikawa  KB, Holt  M, Kaihara  S, Yoshikawa  A, Hopkins  DS, Mason  M.  Performance, characteristics, and case mix in Japanese and American teaching hospitals.  Med Care. 1993;31(6):542-551. doi:10.1097/00005650-199306000-00008PubMedGoogle ScholarCrossref
    31.
    Shariat  SF, Karakiewicz  PI, Suardi  N, Kattan  MW.  Comparison of nomograms with other methods for predicting outcomes in prostate cancer: a critical analysis of the literature.  Clin Cancer Res. 2008;14(14):4400-4407. doi:10.1158/1078-0432.CCR-07-4713PubMedGoogle ScholarCrossref
    32.
    Merath  K, Bagante  F, Beal  EW,  et al.  Nomogram predicting the risk of recurrence after curative-intent resection of primary non-metastatic gastrointestinal neuroendocrine tumors: an analysis of the U.S. Neuroendocrine Tumor Study Group.  J Surg Oncol. 2018;117(5):868-878. doi:10.1002/jso.24985PubMedGoogle ScholarCrossref
    33.
    Mavros  MN, Mayo  SC, Hyder  O, Pawlik  TM.  A systematic review: treatment and prognosis of patients with fibrolamellar hepatocellular carcinoma.  J Am Coll Surg. 2012;215(6):820-830. doi:10.1016/j.jamcollsurg.2012.08.001PubMedGoogle ScholarCrossref
    34.
    Shimada  H, Endo  I, Togo  S, Nakano  A, Izumi  T, Nakagawara  G.  The role of lymph node dissection in the treatment of gallbladder carcinoma.  Cancer. 1997;79(5):892-899. doi:10.1002/(SICI)1097-0142(19970301)79:5<892::AID-CNCR4>3.0.CO;2-EPubMedGoogle ScholarCrossref
    35.
    Bagante  F, Gani  F, Spolverato  G,  et al.  Intrahepatic cholangiocarcinoma: prognosis of patients who did not undergo lymphadenectomy.  J Am Coll Surg. 2015;221(6):1031-40.e1, 4. doi:10.1016/j.jamcollsurg.2015.09.012PubMedGoogle ScholarCrossref
    36.
    Zhang  XF, Chen  Q, Kimbrough  CW,  et al.  Lymphadenectomy for intrahepatic cholangiocarcinoma: has nodal evaluation been increasingly adopted by surgeons over time? a national database analysis.  J Gastrointest Surg. 2018;22(4):668-675. doi:10.1007/s11605-017-3652-2PubMedGoogle ScholarCrossref
    37.
    Zhang  XF, Chakedis  J, Bagante  F,  et al.  Trends in use of lymphadenectomy in surgery with curative intent for intrahepatic cholangiocarcinoma.  Br J Surg. 2018;105(7):857-866. doi:10.1002/bjs.10827PubMedGoogle ScholarCrossref
    38.
    Spolverato  G, Bagante  F, Weiss  M,  et al.  Comparative performances of the 7th and the 8th editions of the American Joint Committee on Cancer staging systems for intrahepatic cholangiocarcinoma.  J Surg Oncol. 2017;115(6):696-703. doi:10.1002/jso.24569Google ScholarCrossref
    ×