[Skip to Navigation]
Sign In
Figure.  Receiver Operator Curves of Multivariate Preoperative Surgical Risk Models for Adverse Surgical Outcomes
Receiver Operator Curves of Multivariate Preoperative Surgical Risk Models for Adverse Surgical Outcomes

eFI indicates Electronic Frailty Index; HFRS; Hospital Frailty Risk Score; mFI-5, 5-Factor Modified Frailty Index; RAI, Risk Analysis Index.

Table 1.  Characteristics of Abdominal Surgical Cases, 2010 to 2020
Characteristics of Abdominal Surgical Cases, 2010 to 2020
Table 2.  Correlation Matrix of Electronic Frailty Metrics
Correlation Matrix of Electronic Frailty Metrics
Table 3.  Predictive Performance of Individual Electronic Frailty Metrics and Preoperative Risk Factors for Adverse Surgical Outcomes
Predictive Performance of Individual Electronic Frailty Metrics and Preoperative Risk Factors for Adverse Surgical Outcomes
Table 4.  Stratified Predictive Discrimination of Multivariate Models for 30-Day Morbiditya
Stratified Predictive Discrimination of Multivariate Models for 30-Day Morbiditya
1.
Chow  WB, Rosenthal  RA, Merkow  RP, Ko  CY, Esnaola  NF; American College of Surgeons National Surgical Quality Improvement Program; American Geriatrics Society.  Optimal preoperative assessment of the geriatric surgical patient: a best practices guideline from the American College of Surgeons National Surgical Quality Improvement Program and the American Geriatrics Society.   J Am Coll Surg. 2012;215(4):453-466. doi:10.1016/j.jamcollsurg.2012.06.017PubMedGoogle ScholarCrossref
2.
Alvarez-Nebreda  ML, Bentov  N, Urman  RD,  et al.  Recommendations for Preoperative Management of Frailty from the Society for Perioperative Assessment and Quality Improvement (SPAQI).   J Clin Anesth. 2018;47:33-42. doi:10.1016/j.jclinane.2018.02.011PubMedGoogle ScholarCrossref
3.
Mohanty  S, Rosenthal  RA, Russell  MM, Neuman  MD, Ko  CY, Esnaola  NF.  Optimal perioperative management of the geriatric patient: a best practices guideline from the American College of Surgeons NSQIP and the American Geriatrics Society.   J Am Coll Surg. 2016;222(5):930-947. doi:10.1016/j.jamcollsurg.2015.12.026PubMedGoogle ScholarCrossref
4.
Dent  E, Morley  JE, Cruz-Jentoft  AJ,  et al.  Physical frailty: ICFSR international clinical practice guidelines for identification and management.   J Nutr Health Aging. 2019;23(9):771-787. doi:10.1007/s12603-019-1273-zPubMedGoogle ScholarCrossref
5.
Turner  G, Clegg  A; British Geriatrics Society; Age UK; Royal College of General Practioners.  Best practice guidelines for the management of frailty: a British Geriatrics Society, Age UK and Royal College of General Practitioners report.   Age Ageing. 2014;43(6):744-747. doi:10.1093/ageing/afu138PubMedGoogle ScholarCrossref
6.
Castillo-Angeles  M, Cooper  Z, Jarman  MP, Sturgeon  D, Salim  A, Havens  JM.  Association of frailty with morbidity and mortality in emergency general surgery by procedural risk level.   JAMA Surg. 2021;156(1):68-74. doi:10.1001/jamasurg.2020.5397PubMedGoogle ScholarCrossref
7.
Abellan van Kan  G, Rolland  Y, Houles  M, Gillette-Guyonnet  S, Soto  M, Vellas  B.  The assessment of frailty in older adults.   Clin Geriatr Med. 2010;26(2):275-286. doi:10.1016/j.cger.2010.02.002PubMedGoogle ScholarCrossref
8.
Farhat  JS, Velanovich  V, Falvo  AJ,  et al.  Are the frail destined to fail? frailty index as predictor of surgical morbidity and mortality in the elderly.   J Trauma Acute Care Surg. 2012;72(6):1526-1530. doi:10.1097/TA.0b013e3182542fabPubMedGoogle ScholarCrossref
9.
McIsaac  DI, Bryson  GL, van Walraven  C.  Association of frailty and 1-year postoperative mortality following major elective noncardiac surgery: a population-based cohort study.   JAMA Surg. 2016;151(6):538-545. doi:10.1001/jamasurg.2015.5085PubMedGoogle ScholarCrossref
10.
Suskind  AM, Walter  LC, Jin  C,  et al.  Impact of frailty on complications in patients undergoing common urological procedures: a study from the American College of Surgeons National Surgical Quality Improvement database.   BJU Int. 2016;117(5):836-842. doi:10.1111/bju.13399PubMedGoogle ScholarCrossref
11.
Eamer  G, Gibson  JA, Gillis  C,  et al.  Surgical frailty assessment: a missed opportunity.   BMC Anesthesiol. 2017;17(1):99. doi:10.1186/s12871-017-0390-7PubMedGoogle ScholarCrossref
12.
Santos-Eggimann  B, Sirven  N.  Screening for frailty: older populations and older individuals.   Public Health Rev. 2016;37(1):7. doi:10.1186/s40985-016-0021-8PubMedGoogle ScholarCrossref
13.
Walston  J, Buta  B, Xue  QL.  Frailty screening and interventions: considerations for clinical practice.   Clin Geriatr Med. 2018;34(1):25-38. doi:10.1016/j.cger.2017.09.004PubMedGoogle ScholarCrossref
14.
Shinall  MC  Jr, Arya  S, Youk  A,  et al.  Association of preoperative patient frailty and operative stress with postoperative mortality.   JAMA Surg. 2020;155(1):e194620. doi:10.1001/jamasurg.2019.4620PubMedGoogle ScholarCrossref
15.
Seib  CD, Rochefort  H, Chomsky-Higgins  K,  et al.  Association of patient frailty with increased morbidity after common ambulatory general surgery operations.   JAMA Surg. 2018;153(2):160-168. doi:10.1001/jamasurg.2017.4007PubMedGoogle ScholarCrossref
16.
Fowler  AJ, Abbott  TEF, Prowle  J, Pearse  RM.  Age of patients undergoing surgery.   Br J Surg. 2019;106(8):1012-1018. doi:10.1002/bjs.11148PubMedGoogle ScholarCrossref
17.
Kehler  DS, Ferguson  T, Stammers  AN,  et al.  Prevalence of frailty in Canadians 18-79 years old in the Canadian Health Measures Survey.   BMC Geriatr. 2017;17(1):28. doi:10.1186/s12877-017-0423-6PubMedGoogle ScholarCrossref
18.
Fried  LP, Tangen  CM, Walston  J,  et al; Cardiovascular Health Study Collaborative Research Group.  Frailty in older adults: evidence for a phenotype.   J Gerontol A Biol Sci Med Sci. 2001;56(3):M146-M156. doi:10.1093/gerona/56.3.M146PubMedGoogle ScholarCrossref
19.
Clegg  A, Young  J, Iliffe  S, Rikkert  MO, Rockwood  K.  Frailty in elderly people.   Lancet. 2013;381(9868):752-762. doi:10.1016/S0140-6736(12)62167-9PubMedGoogle ScholarCrossref
20.
Rolfson  DB, Majumdar  SR, Tsuyuki  RT, Tahir  A, Rockwood  K.  Validity and reliability of the Edmonton Frail Scale.   Age Ageing. 2006;35(5):526-529. doi:10.1093/ageing/afl041PubMedGoogle ScholarCrossref
21.
McIsaac  DI, Taljaard  M, Bryson  GL,  et al.  Frailty as a predictor of death or new disability after surgery: a prospective cohort study.   Ann Surg. 2020;271(2):283-289. doi:10.1097/SLA.0000000000002967PubMedGoogle ScholarCrossref
22.
Kovacs  J, Moraru  L, Antal  K, Cioc  A, Voidazan  S, Szabo  A.  Are frailty scales better than anesthesia or surgical scales to determine risk in cardiac surgery?   Korean J Anesthesiol. 2017;70(2):157-162. doi:10.4097/kjae.2017.70.2.157PubMedGoogle ScholarCrossref
23.
Kapoor  A, Matheos  T, Walz  M,  et al.  Self-reported function more informative than frailty phenotype in predicting adverse postoperative course in older adults.   J Am Geriatr Soc. 2017;65(11):2522-2528. doi:10.1111/jgs.15108PubMedGoogle ScholarCrossref
24.
Shah  R, Borrebach  JD, Hodges  JC,  et al.  Validation of the risk analysis index for evaluating frailty in ambulatory patients.   J Am Geriatr Soc. 2020;68(8):1818-1824. doi:10.1111/jgs.16453PubMedGoogle ScholarCrossref
25.
Khalafallah  AM, Huq  S, Jimenez  AE, Brem  H, Mukherjee  D. The 5-factor modified frailty index: an effective predictor of mortality in brain tumor patients. J Neurosurg. Published online August 14, 2020. doi:10.3171/2020.5.JNS20766
26.
Khalafallah  AM, Shah  PP, Huq  S,  et al.  The 5-Factor Modified Frailty Index predicts health burden following surgery for pituitary adenomas.   Pituitary. 2020;23(6):630-640. doi:10.1007/s11102-020-01069-5PubMedGoogle ScholarCrossref
27.
Gunnarsdottir  GM, Helgadottir  S, Einarsson  SG,  et al.  Validation of the Hospital Frailty Risk Score in older surgical patients: a population-based retrospective cohort study.   Acta Anaesthesiol Scand. 2021;65(8):1033-1042. doi:10.1111/aas.13837PubMedGoogle ScholarCrossref
28.
Hannah  TC, Neifert  SN, Caridi  JM,  et al.  Utility of the Hospital Frailty Risk Score for predicting adverse outcomes in degenerative spine surgery cohorts.   Neurosurgery. 2020;87(6):1223-1230. doi:10.1093/neuros/nyaa248PubMedGoogle ScholarCrossref
29.
Meyer  M, Parik  L, Leiß  F, Renkawitz  T, Grifka  J, Weber  M.  Hospital Frailty Risk Score predicts adverse events in primary total hip and knee arthroplasty.   J Arthroplasty. 2020;35(12):3498-3504.e3. doi:10.1016/j.arth.2020.06.087PubMedGoogle ScholarCrossref
30.
Aitken  SJ, Lujic  S, Randall  DA, Noguchi  N, Naganathan  V, Blyth  FM.  Predicting outcomes in older patients undergoing vascular surgery using the Hospital Frailty Risk Score.   Br J Surg. 2020.PubMedGoogle Scholar
31.
George  EL, Hall  DE, Youk  A,  et al.  Association between patient frailty and postoperative mortality across multiple noncardiac surgical specialties.   JAMA Surg. 2021;156(1):e205152-e205152. doi:10.1001/jamasurg.2020.5152PubMedGoogle ScholarCrossref
32.
Holzgrefe  RE, Wilson  JM, Staley  CA, Anderson  TL, Wagner  ER, Gottschalk  MB.  Modified frailty index is an effective risk-stratification tool for patients undergoing total shoulder arthroplasty.   J Shoulder Elbow Surg. 2019;28(7):1232-1240. doi:10.1016/j.jse.2018.12.004PubMedGoogle ScholarCrossref
33.
Chimukangara  M, Helm  MC, Frelich  MJ,  et al.  A 5-item frailty index based on NSQIP data correlates with outcomes following paraesophageal hernia repair.   Surg Endosc. 2017;31(6):2509-2519. doi:10.1007/s00464-016-5253-7PubMedGoogle ScholarCrossref
34.
Hall  DE, Arya  S, Schmid  KK,  et al.  Development and initial validation of the risk analysis index for measuring frailty in surgical populations.   JAMA Surg. 2017;152(2):175-182. doi:10.1001/jamasurg.2016.4202PubMedGoogle ScholarCrossref
35.
Gilbert  T, Neuburger  J, Kraindler  J,  et al.  Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study.   Lancet. 2018;391(10132):1775-1782. doi:10.1016/S0140-6736(18)30668-8PubMedGoogle ScholarCrossref
36.
Subramaniam  S, Aalberg  JJ, Soriano  RP, Divino  CM.  New 5-Factor Modified Frailty Index Using American College of Surgeons NSQIP Data.   J Am Coll Surg. 2018;226(2):173-181.e8. doi:10.1016/j.jamcollsurg.2017.11.005PubMedGoogle ScholarCrossref
37.
Callahan  KE, Clark  CJ, Edwards  AF,  et al.  Automated frailty screening at-scale for pre-operative risk stratification using the Electronic Frailty Index.   J Am Geriatr Soc. 2021;69(5):1357-1362. doi:10.1111/jgs.17027PubMedGoogle ScholarCrossref
38.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KG; TRIPOD Group.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.   Circulation. 2015;131(2):211-219. doi:10.1161/CIRCULATIONAHA.114.014508PubMedGoogle ScholarCrossref
39.
Massarweh  NN, Legner  VJ, Symons  RG, McCormick  WC, Flum  DR.  Impact of advancing age on abdominal surgical outcomes.   Arch Surg. 2009;144(12):1108-1114. doi:10.1001/archsurg.2009.204PubMedGoogle ScholarCrossref
40.
Jaber  S, Lescot  T, Futier  E,  et al; NIVAS Study Group.  Effect of noninvasive ventilation on tracheal reintubation among patients with hypoxemic respiratory failure following abdominal surgery: a randomized clinical trial.   JAMA. 2016;315(13):1345-1353. doi:10.1001/jama.2016.2706PubMedGoogle ScholarCrossref
41.
Futier  E, Constantin  J-M, Paugam-Burtz  C,  et al; IMPROVE Study Group.  A trial of intraoperative low-tidal-volume ventilation in abdominal surgery.   N Engl J Med. 2013;369(5):428-437. doi:10.1056/NEJMoa1301082PubMedGoogle ScholarCrossref
42.
Myles  PS, Bellomo  R, Corcoran  T,  et al; Australian and New Zealand College of Anaesthetists Clinical Trials Network and the Australian and New Zealand Intensive Care Society Clinical Trials Group.  Restrictive versus liberal fluid therapy for major abdominal surgery.   N Engl J Med. 2018;378(24):2263-2274. doi:10.1056/NEJMoa1801601PubMedGoogle ScholarCrossref
43.
Raval  MV, Cohen  ME, Ingraham  AM,  et al.  Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score.   J Am Coll Surg. 2010;211(6):715-723. doi:10.1016/j.jamcollsurg.2010.07.021PubMedGoogle ScholarCrossref
44.
Quan  H, Sundararajan  V, Halfon  P,  et al.  Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.   Med Care. 2005;43(11):1130-1139. doi:10.1097/01.mlr.0000182534.19832.83PubMedGoogle ScholarCrossref
45.
Charlson  ME, Pompei  P, Ales  KL, MacKenzie  CR.  A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.   J Chronic Dis. 1987;40(5):373-383. doi:10.1016/0021-9681(87)90171-8PubMedGoogle ScholarCrossref
46.
Escobar  GJ, Gardner  MN, Greene  JD, Draper  D, Kipnis  P.  Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system.   Med Care. 2013;51(5):446-453. doi:10.1097/MLR.0b013e3182881c8ePubMedGoogle ScholarCrossref
47.
Hall  BL, Hamilton  BH, Richards  K, Bilimoria  KY, Cohen  ME, Ko  CY.  Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals.   Ann Surg. 2009;250(3):363-376. doi:10.1097/SLA.0b013e3181b4148fPubMedGoogle ScholarCrossref
48.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.   J Am Coll Surg. 2013;217(2):336-346.e1. doi:10.1016/j.jamcollsurg.2013.02.027PubMedGoogle ScholarCrossref
49.
Rockwood  K, Song  X, MacKnight  C,  et al.  A global clinical measure of fitness and frailty in elderly people.   CMAJ. 2005;173(5):489-495. doi:10.1503/cmaj.050051PubMedGoogle ScholarCrossref
50.
Subramaniam  S, Aalberg  JJ, Soriano  RP, Divino  CM.  The 5-Factor Modified Frailty Index in the geriatric surgical population.   Am Surg. 2021;87(9):1420-1425. doi:10.1177/0003134820952438PubMedGoogle ScholarCrossref
51.
Arya  S, Varley  P, Youk  A,  et al.  Recalibration and external validation of the risk analysis index: a surgical frailty assessment tool.   Ann Surg. 2020;272(6):996-1005. doi:10.1097/SLA.0000000000003276PubMedGoogle ScholarCrossref
52.
Howlett  SE, Rockwood  MR, Mitnitski  A, Rockwood  K.  Standard laboratory tests to identify older adults at increased risk of death.   BMC Med. 2014;12(1):171. doi:10.1186/s12916-014-0171-9PubMedGoogle ScholarCrossref
53.
Gani  F, Canner  JK, Pawlik  TM.  Use of the Modified Frailty Index in the American College of Surgeons National Surgical Improvement Program database: highlighting the problem of missing data.   JAMA Surg. 2017;152(2):205-207. doi:10.1001/jamasurg.2016.3479PubMedGoogle ScholarCrossref
54.
Le  ST, Corbin  D, Myers  LC, Kipnis  P, Cohn  B, Liu  VX.  Development and validation of an electronic health record-based score for triage to perioperative medicine.   Ann Surg. 2021. Published online November 9, 2021. doi:10.1097/SLA.0000000000005284PubMedGoogle ScholarCrossref
55.
Evans  JD.  Straightforward Statistics for the Behavioral Sciences. Thomson Brooks/Cole Publishing Co; 1996.
56.
Hastie  T, Tibshirani  R, Friedman  J.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media; 2009. doi:10.1007/978-0-387-84858-7
57.
Fried  LP, Ferrucci  L, Darer  J, Williamson  JD, Anderson  G.  Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care.   J Gerontol A Biol Sci Med Sci. 2004;59(3):255-263. doi:10.1093/gerona/59.3.M255PubMedGoogle ScholarCrossref
58.
Austin  PC, Steyerberg  EW.  Predictive accuracy of risk factors and markers: a simulation study of the effect of novel markers on different performance measures for logistic regression models.   Stat Med. 2013;32(4):661-672. doi:10.1002/sim.5598PubMedGoogle ScholarCrossref
59.
Arruda-Olson  AM, Afzal  N, Priya Mallipeddi  V,  et al.  Leveraging the electronic health record to create an automated real-time prognostic tool for peripheral arterial disease.   J Am Heart Assoc. 2018;7(23):e009680. doi:10.1161/JAHA.118.009680PubMedGoogle ScholarCrossref
60.
Ridgway  JP, Robicsek  A, Shah  N,  et al.  A randomized controlled trial of an electronic clinical decision support tool for inpatient antimicrobial stewardship.   Clin Infect Dis. 2021;72(9):e265-e271. doi:10.1093/cid/ciaa1048PubMedGoogle ScholarCrossref
61.
Wu  CX, Suresh  E, Phng  FWL,  et al.  Effect of a real-time risk score on 30-day readmission reduction in Singapore.   Appl Clin Inform. 2021;12(2):372-382. doi:10.1055/s-0041-1726422PubMedGoogle ScholarCrossref
62.
Bergman  H, Ferrucci  L, Guralnik  J,  et al.  Frailty: an emerging research and clinical paradigm--issues and controversies.   J Gerontol A Biol Sci Med Sci. 2007;62(7):731-737. doi:10.1093/gerona/62.7.731PubMedGoogle ScholarCrossref
63.
Fisher  AL.  Just what defines frailty?   J Am Geriatr Soc. 2005;53(12):2229-2230. doi:10.1111/j.1532-5415.2005.00510.xPubMedGoogle ScholarCrossref
64.
Bilimoria  KY, Liu  Y, Paruch  JL,  et al.  Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons.   J Am Coll Surg. 2013;217(5):833-42.e1, 3. doi:10.1016/j.jamcollsurg.2013.07.385PubMedGoogle ScholarCrossref
65.
Bertges  DJ, Neal  D, Schanzer  A,  et al; Vascular Quality Initiative.  The Vascular Quality Initiative Cardiac Risk Index for prediction of myocardial infarction after vascular surgery.   J Vasc Surg. 2016;64(5):1411-1421.e4. doi:10.1016/j.jvs.2016.04.045PubMedGoogle ScholarCrossref
66.
Bihorac  A, Ozrazgat-Baslanti  T, Ebadi  A,  et al.  MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery.   Ann Surg. 2019;269(4):652-662. doi:10.1097/SLA.0000000000002706PubMedGoogle ScholarCrossref
67.
Brennan  M, Puri  S, Ozrazgat-Baslanti  T,  et al.  Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.   Surgery. 2019;165(5):1035-1045. doi:10.1016/j.surg.2019.01.002PubMedGoogle ScholarCrossref
68.
Elixhauser  A, Steiner  C, Harris  DR, Coffey  RM.  Comorbidity measures for use with administrative data.   Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004PubMedGoogle ScholarCrossref
69.
Escobar  GJ, Greene  JD, Scheirer  P, Gardner  MN, Draper  D, Kipnis  P.  Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases.   Med Care. 2008;46(3):232-239. doi:10.1097/MLR.0b013e3181589bb6PubMedGoogle ScholarCrossref
70.
Carli  F, Scheede-Bergdahl  C.  Prehabilitation to enhance perioperative care.   Anesthesiol Clin. 2015;33(1):17-33. doi:10.1016/j.anclin.2014.11.002PubMedGoogle ScholarCrossref
71.
Wynter-Blyth  V, Moorthy  K.  Prehabilitation: preparing patients for surgery.   BMJ. 2017;358:j3702. doi:10.1136/bmj.j3702PubMedGoogle ScholarCrossref
72.
Makary  MA, Segev  DL, Pronovost  PJ,  et al.  Frailty as a predictor of surgical outcomes in older patients.   J Am Coll Surg. 2010;210(6):901-908. doi:10.1016/j.jamcollsurg.2010.01.028PubMedGoogle ScholarCrossref
73.
Street  A, Maynou  L, Gilbert  T, Stone  T, Mason  S, Conroy  S.  The use of linked routine data to optimise calculation of the Hospital Frailty Risk Score on the basis of previous hospital admissions: a retrospective observational cohort study.   Lancet Healthy Longev. 2021;2(3):e154-e162. doi:10.1016/S2666-7568(21)00004-0PubMedGoogle ScholarCrossref
74.
Wan  MA, Clark  JM, Nuño  M, Cooke  DT, Brown  LM.  Can the risk analysis index for frailty predict morbidity and mortality in patients undergoing high-risk surgery?   Ann Surg. 2020. doi:10.1097/SLA.0000000000004626PubMedGoogle ScholarCrossref
Original Investigation
March 16, 2022

Comparison of Electronic Frailty Metrics for Prediction of Adverse Outcomes of Abdominal Surgery

Author Affiliations
  • 1Division of Research, Kaiser Permanente Northern California, Oakland
  • 2Department of Surgery, University of California San Francisco-East Bay, Oakland
  • 3The Permanente Medical Group, Oakland, California
JAMA Surg. 2022;157(5):e220172. doi:10.1001/jamasurg.2022.0172
Key Points

Question  How do electronic frailty metrics compare and what is their predictive value for adverse surgical outcomes?

Findings  In this cohort study among 37 186 patients who underwent abdominal surgery, the Hospital Frailty Risk Score (HFRS), adapted Electronic Frailty Index, 5-Factor Modified Frailty Index, and Risk Analysis Index demonstrated low to moderate correlation. The HFRS had the highest predictive value for mortality, readmission, and major complications, and augmented a multivariate preoperative risk model.

Meaning  The 4 electronic frailty metrics often disagree and exhibit heterogeneous discrimination for adverse surgical outcomes; the HFRS demonstrated the strongest overall predictive value, which suggests its potential utility for automated screening for high-risk or frail patients.

Abstract

Importance  Electronic frailty metrics have been developed for automated frailty assessment and include the Hospital Frailty Risk Score (HFRS), the Electronic Frailty Index (eFI), the 5-Factor Modified Frailty Index (mFI-5), and the Risk Analysis Index (RAI). Despite substantial differences in their construction, these 4 electronic frailty metrics have not been rigorously compared within a surgical population.

Objective  To characterize the associations between 4 electronic frailty metrics and to measure their predictive value for adverse surgical outcomes.

Design, Setting, and Participants  This retrospective cohort study used electronic health record data from patients who underwent abdominal surgery from January 1, 2010, to December 31, 2020, at 20 medical centers within Kaiser Permanente Northern California (KPNC). Participants included adults older than 50 years who underwent abdominal surgical procedures at KPNC from 2010 to 2020 that were sampled for reporting to the National Surgical Quality Improvement Program.

Main Outcomes and Measures  Pearson correlation coefficients between electronic frailty metrics and area under the receiver operating characteristic curve (AUROC) of univariate models and multivariate preoperative risk models for 30-day mortality, readmission, and morbidity, which was defined as a composite of mortality and major postoperative complications.

Results  Within the cohort of 37 186 patients, mean (SD) age, 67.9 (female, 19 127 [51.4%]), correlations between pairs of metrics ranged from 0.19 (95% CI, 0.18- 0.20) for mFI-5 and RAI 0.69 (95% CI, 0.68-0.70). Only 1085 of 37 186 (2.9%) were classified as frail based on all 4 metrics. In univariate models for morbidity, HFRS demonstrated higher predictive discrimination (AUROC, 0.71; 95% CI, 0.70-0.72) than eFI (AUROC, 0.64; 95% CI, 0.63-0.65), mFI-5 (AUROC, 0.58; 95% CI, 0.57-0.59), and RAI (AUROC, 0.57; 95% CI, 0.57-0.58). The predictive discrimination of multivariate models with age, sex, comorbidity burden, and procedure characteristics for all 3 adverse surgical outcomes improved by including HFRS into the models.

Conclusions and Relevance  In this cohort study, the 4 electronic frailty metrics demonstrated heterogeneous correlation and classified distinct groups of surgical patients as frail. However, HFRS demonstrated the highest predictive value for adverse surgical outcomes.

Introduction

Frailty is a state of decreased physiologic reserve and multiple system impairments that increases vulnerability to stressors, such as surgery. National institutions and specialty societies have strongly recommended preoperative assessment for frailty as a critical component of surgical care,1-5 as frailty is a powerful determinant of complications, nonhome discharge, and new disability.6-10 However, implementation of frailty assessments has been limited, in part owing to time-consuming frailty assessment methods that require direct clinician evaluation.11-13 Given the association between frailty and morbidity following low-risk procedures,14,15 the aging surgical population,16 and the prevalence of frailty among nonelderly adults,17 the number of patients for whom frailty assessment may be relevant is potentially large. Practical methods for assessing frailty are needed.

The most established frailty instruments require direct manual assessment by a clinician,18-20 which has raised concerns about the feasibility of using these tools across populations.21-23 This has led to the development of several electronic frailty metrics, which leverage readily available data within the electronic health record (EHR) to measure frailty. Therefore, these metrics have strong potential to streamline preoperative assessments. However, these metrics differ substantially in the data they incorporate, their construction, and their conceptual approach to frailty. Prior studies have demonstrated univariate associations between individual electronic frailty metrics and adverse outcomes within various surgical populations,24-33 but to our knowledge, no studies have systematically compared and characterized the predictive value of multiple metrics alongside other preoperative risk factors. This information is critical for clinicians and health systems seeking to improve surgical care and decision-making by implementing one of these tools into clinical workflows.

In this external validation of electronic frailty metrics, we used EHR data from surgical procedures that were performed at a large, integrated health care system to compare 4 existing electronic frailty metrics: the Hospital Frailty Risk Score (HFRS), the adapted Electronic Frailty Index (eFI), the 5-Factor Modified Frailty Index (mFI-5), and the Risk Analysis Index (RAI).34-37 We address the following questions: First, do these metrics agree in their measurement and classification of frailty? Second, what is the predictive value of each metric for adverse surgical outcomes, including mortality, readmission, and major postoperative complications? And additionally, do these metrics improve prediction of adverse surgical outcomes, independent of other, readily measurable preoperative risk factors?

Methods

This study was approved by the Kaiser Permanente Institutional Review Board. A waiver of informed consent was provided because the research complied with Exempt Research Category 4 requirements. We followed Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines for prediction model validation.38

Cohort Identification

Within Kaiser Permanente Northern California (KPNC), all hospitals and clinics use the same information systems and share a single EHR system. Our study population consisted of 37 186 adult patients older than 50 years who required inpatient admission for abdominal surgical procedure at KPNC facilities between January 1, 2010, and December 31, 2020. We excluded cases if the patient had undergone an abdominal surgery within 30 days prior. Within KPNC, procedure types are labeled based on an internal coding system. We identified abdominal surgical cases based on procedure codes that were reviewed by 2 surgeons (S.T.L. and P.D.P.). In the absence of an accepted definition of abdominal surgery and given the variety of different types of procedures included in prior investigations,39-42 we selected a broad range of procedures performed within the abdomen, excluding transplant, trauma, obstetrics, and endoscopic procedures. We limited our study sample to cases that were manually reviewed for reporting to the National Surgical Quality Improvement Program (NSQIP), to ensure standardized ascertainment of postoperative complications. The sampling strategy for NSQIP requires a dedicated surgical clinical nurse review to abstract data on specific risk factors and 30-day postoperative complications.43

Preoperative Risk Factors

We captured a set of characteristics within the EHR that are known to be associated with adverse surgical outcomes. These include age, sex, comorbidity burden, and procedure information. We quantified comorbid disease burden with the Charlson Comorbidity Index,44,45 based on patients’ medical diagnoses within the 12 months preceding the date of surgery.46 We considered a case to be urgent if the surgeon indicated it was an add-on case or needed to be performed within 48 hours, in the form that is submitted to schedule the case. We classified the operative approach as minimally invasive vs open if the primary procedure code indicated that the procedure was performed laparoscopically, robotically, or endoscopically. Additionally, to model the risks of adverse surgical outcomes specific to different surgical procedures, we grouped cases based on procedure codes, using a hierarchical scheme, similar to how NSQIP previously grouped surgical procedures to model procedure-specific risk.47,48 To ensure adequate sample size, similar groups were further aggregated into 24 groups based on the number of cases during the study period (eTable 1 in the Supplement).

Background and Calculation of Electronic Frailty Metrics

The developments of HFRS, eFI, mFI-5, and RAI have been previously reported in the literature.34-37,49-51 The HFRS is derived from a cluster analysis of older adult patients who were hospitalized and uses diagnosis codes to predict frailty.35 The eFI is conceptually based on the cumulative deficits model of frailty and incorporates diagnoses, laboratory values, vitals, medications, social history, and functional status.52 The mFI-5 is conceptually based on the cumulative deficits model of frailty and incorporates factors captured in NSQIP data.53 The RAI is based on a model to predict 30-day mortality among older adults surgical patients using NSQIP data.34,51 We calculated 4 electronic frailty metrics based on available EHR data prior to surgery, and details on our adaptation and calculation of these metrics are provided in eTables 2 through 5 in the Supplement. The eFI was deemed not calculable for 5237 patients (14.1%), based on previously published requirements to exclude patients with insufficient data to calculate eFI.

Surgical Adverse Outcomes

Our primary outcome was 30-day postoperative morbidity, which was defined as a composite of 30-day mortality and 30-day major postoperative complications. Mortality was identified using KPNC health system records, and 30-day major postoperative complications included cardiac arrest, myocardial infarction, pulmonary embolism, sepsis, septic shock, deep or organ space surgical site infections, unplanned intubation, deep vein thrombosis, progressive kidney insufficiency or kidney failure, or cerebrovascular accident. These complications were selected and outcome was constructed based on prior studies of major NSQIP complications.48,54 Secondary outcomes included 30-day mortality from the day of surgery and 30-day readmission from the day of discharge.

Modeling and Analysis

We assessed the correlation of the electronic frailty metrics with Pearson correlation coefficients. We considered correlation values of 0 to 0.19 as very weak, 0.20 to 0.39 as weak, 0.40 to 0.59 as moderate, 0.60 to 0.79 as strong, and 0.80 to 1.00 as very strong.55 P values associated with the Pearson correlation between pairs of metrics were computed from 2-tailed, single-hypothesis tests with the null hypothesis that these correlations are 0. We quantified the number of patients who might be considered frail based on each of the electronic frailty metrics. For patients in the HFRS, RAI, and mFI-5 groups, patients were classified as frail based on published thresholds, including 35 or higher for RAI, 15 or higher for HFRS, and 0.2 or higher for mFI-5. Because the availability of data within a health system can increase eFI, patients were classified as frail based on eFI if eFI was calculable and in the top quartile. We then compared the proportion of patients classified as frail based on the different metrics.

To characterize the predictive value of these metrics for adverse surgical events while avoiding the potential for overfitting, we prespecified models and compared their predictive discrimination. We first assessed the predictive discrimination of each individual metric for adverse surgical outcomes by using logistic regression models for each outcome with each frailty metric alone, as continuous predictors. For the models with eFI, patients with eFI not calculable owing to insufficient data were flagged, and not calculable eFI was entered into the model as an additional binary predictor. Model discrimination was evaluated based on the area under the receiver operator characteristic curve (AUROC).56 As a sensitivity analysis, we evaluated the predictive discrimination of the electronic frailty metrics for adverse surgical outcomes within a variety of elective surgical procedure types. We then assessed the AUROC of multivariate logistic models to measure the incremental predictive value of electronic frailty metrics along other risk factors for postoperative adverse outcomes. Of note, factors such as chronologic age and comorbidity are conceptualized to be associated but distinct constructs from the concept of frailty,18,57 and therefore we considered the contribution of frailty, as quantified by the metrices, to be independent of age or comorbidity for surgical outcomes. We evaluated the discrimination of base multivariate preoperative risk models that include age, sex, Charlson Comorbidity Index, procedure type, and urgency. Age was entered into the models as a restricted cubic spline with 3 knots. We examined the predictive discrimination of multivariate models including those base factors and individual frailty metrics, and reported the P value comparing the AUROCs of those models with the base models. Additionally, we stratified the predictive discrimination of these models in subgroups based on age, elective vs urgent, and open vs minimally invasive surgical approach.

All statistical tests were 2-tailed, and statistical significance was accepted at the P < .05 level. Statistical analyses were performed using SAS version 9.4 (SAS Institute), R version 4.0.2 (The R Foundation), and Stata version 16 (StataCorp).

Results

The analytic cohort consisted of 34 945 patients (male, 18 059 [48.6%]) who underwent 37 186 abdominal surgical procedures. Mean (SD) age was 67.9 (10.6) years (Table 1). Within this cohort, 21 283 (57.2%) were designated American Society of Anesthesiologists class III or greater, 18 427 (50.2%) of the surgical procedures were performed using an open approach, and 12 260 (32.9%) were urgent. The most common procedure types were colectomies without low-pelvic anastomosis (11 283 [30.3%]), cholecystectomy (4006 [10.8%]), and nephrectomy (2317 [6.2%]). The rates of 30-day morbidity, mortality, and readmission were 4573 (12.3%), 923 (2.5%), and 4125 (11.4%), respectively.

The RAI had weak to moderate correlation with the other 3 metrics, ranging from 0.19 (95% CI, 0.18-0.20) with mFI-5 0.33 with eFI (95% CI, 0.32-0.34) (Table 2). The mFI-5 and eFI were moderately correlated (0.44; 95% CI, 0.43-0.45; P < .001) while the efI and HFRS models had strong correlation and the highest correlation of any pair of metrics (0.69; 95% CI, 0.68-0.70; P < .001). The proportion of patients classified as frail by each individual metric was 7496 (20.2%) in the mFI-5, 9360 (25.2%) in the HFRS, 9664 (26.0%) in the eFI, and 10 788 (29.0%) in the RAI. The overlap between patients classified as frail based on pairs of metrics ranged from 2711 (7.3%) classified as frail by both mFI-5 and RAI, to 5899 (15.9%) classified as frail by both HFRS and eFI; 1085 (2.9%) of patients were classified as frail by all 4 metrics.

Of the electronic frailty metrics, HFRS had the highest individual predictive discrimination in univariate models for morbidity (AUROC, 0.71; 95% CI, 0.70-0.72), mortality (AUROC, 0.82, 95% CI, 0.81-0.84), and readmission (AUROC, 0.64; 95% CI, 0.63-0.65), followed by eFI for morbidity (AUROC, 0.64; 95% CI, 0.63-0.65), mortality (AUROC, 0.73; 95% CI, 0.71-0.75), and readmission (AUROC, 0.61; 95% CI, 0.60-0.62) (Table 3). The mFI-5 and RAI demonstrated lower predictive discrimination for all 3 outcomes; for example, the AUROC of RAI for morbidity was 0.57 (95% CI, 0.57-0.58). Among the preoperative surgical risk factors, procedure factors demonstrated the highest discrimination for morbidity (AUROC, 0.70; 95% CI, 0.69-0.70), mortality (AUROC, 0.77; 95% CI, 0.75-0.79), and readmission (AUROC, 0.61; 95% CI, 0.60-0.62). Within select elective surgical procedure types, HFRS demonstrated higher discrimination for adverse surgical outcomes than mFI-5 and RAI; for example, among pancreatectomies, the AUROC of HFRS for mortality was 0.77 (95% CI, 0.68-0.87) compared with 0.64 (95% CI, 0.49-0.78) for RAI and 0.56 (95% CI, 0.44-0.68) for mFI-5 (eTable 6 in the Supplement).

In multivariate models, models including HFRS consistently demonstrated the highest predictive discrimination for all 3 outcomes, and the addition of the HFRS significantly improved the discrimination of the base risk models with demographic, comorbidity, and procedure factors (Figure). For example, the AUROC of the base model was 0.72 (95% CI, 0.71-0.73), but including HFRS improved the AUROC to 0.76 (95% CI, 0.75-0.77; P < .001). In subgroup analyses, the incremental predictive value of HFRS for adverse outcomes was seen in the 50 to 74 years age group and the 75 years and older age group, elective and urgent surgical procedures, and open and minimally invasive surgical procedures. For example, including HFRS improved the AUROC of the base model for morbidity from 0.64 (95% CI, 0.63-0.65) to 0.70 (95% CI, 0.69-0.71) among elective surgical procedures and 0.72 (95% CI, 0.71-0.73) to 0.76 (95% CI, 0.75-0.77) among urgent surgical procedures (Table 4). The incremental predictive value of HFRS was seen for mortality (eTable 7 in the Supplement) and readmission (eTable 8 in the Supplement) across all subgroups, whereas the inclusion of eFI, mFI-5, or RAI as predictors resulted in little incremental predictive value.

Discussion

The tremendous effect that frailty has on surgical outcomes and decision-making has accentuated the need for objective and standardized assessments of frailty. The need to assess frailty at scale has led to the development of frailty metrics that are readily calculable using available EHR data. Within a large abdominal surgical population, the 4 electronic frailty metrics demonstrated low to moderate correlation with one another and classified different groups of patients as frail. However, of the 4 electronic frailty metrics, HFRS demonstrated the highest predictive discrimination across 3 adverse surgical outcomes, and HFRS augmented the performance of a multivariate preoperative risk model that included standard risk factors. The eFI demonstrated lower predictive discrimination and did not augment the discrimination of a standard preoperative-risk model, while the RAI and mFI-5 models demonstrated relatively poor predictive discrimination for adverse surgical outcomes.

Prior studies have characterized the predictive validity of electronic frailty metrics by measuring univariate associations between the metrics and adverse surgical outcomes, but this approach fails to adequately consider risk factors for adverse surgical outcomes that are conceptually distinct from frailty. By comparing the predictive discrimination of various multivariate models, we disentangled the degree to which HFRS, eFI, mFI-5, and RAI potentially measure frailty vs simply measuring age or comorbidity burden.58 We found that RAI and mFI-5 did not augment the predictive value of models that included age and the Charlson Comorbidity Index. Given the Charlson Comorbidity Index is readily calculable within many comprehensive EHR systems, the incremental utility of calculating RAI and mFI-5 is questionable.59-61

Ultimately, preoperative risk, comorbidity, and frailty assessments should be considered distinct objectives for surgical health systems.62,63 Preoperative risk assessment is concerned with predicting surgical outcomes, and several well-established tools have been developed to predict the risk of various adverse surgical outcomes,64,65 including automated tools embedded into the EHR.66,67 Comorbidity is defined as the presence of multiple medical conditions, and tools for the measurement of comorbidity44,68,69 can potentially identify patients with conditions that are amenable to specific medical treatments. Frailty assessments seek to identify patients with a syndrome that is amenable to frailty-specific interventions, such as exercise prehabilitation,70,71 nutritional optimization, or anxiety-relief strategies, or who may require additional support in surgical decision-making or postoperative recovery. Guidelines for preoperative assessment of older surgical patients from the American College of Surgeons and American Geriatrics Society1,3 strongly recommend the assessment of baseline frailty using the Fried frailty score,18,72 but this score requires manual examination of walking speed and grip strength, as well as additional patient input to determine the presence of exhaustion, low physical activity, and weight loss. Further research is needed to understand how appropriately and efficiently electronic frailty metrics can target additional frailty-specific evaluations or interventions.

Health systems that are considering implementing electronic frailty metrics into their workflows must consider the challenges of adapting these metrics for calculation and real-time deployment within their information systems. The mFI-5 and RAI models are derived from NSQIP data, which are limited to a sample of all surgical cases performed at participating institutions. Adapting these measures for the EHR would require the identification of sources or replacements for the relevant NSQIP variables within the EHR. Compared with the other metrics, eFI incorporates the largest breadth of patient data, which may not be readily available within all information systems, and a proportion of patients may have insufficient data to calculate eFI. The fragmentation of different health care environments and information systems may influence the calculation of these metrics.73

Limitations

Our study has several limitations. First, our study did not assess the association between these electronic frailty metrics and other frailty instruments based on in-person clinician assessment, such as the FRAIL Scale, the Clinical Frail Scale, or the Edmonton Frail Scale. Further study is needed to consider the correlation and to compare the predictive validity of electronic frailty metrics and clinician-based frailty assessments. Second, our analysis was performed within an integrated health system with a single EHR system. Health systems with more fragmented processes and data systems may have more difficulty with standardized calculation of scores. Third, our calculations of the electronic frailty metrics necessitated minor deviations from what was originally published. For example, 6 of the 109 International Classification of Diseases codes used to calculate HFRS are not used within our health system, we used diagnosis codes to replace several NSQIP variables that are no longer available after 2015 to calculate RAI,74 and we calculated a published version of the eFI without functional status deficits owing to lack of availability of structured data from Medicare Annual Wellness Visits. Furthermore, the accuracy of the diagnostic codes used by the electronic frailty metrics likely varies across health systems with different coding practices. We view potential issues with diagnostic coding accuracy as inherent challenges with deriving a standardized frailty metric that can be transported across health care delivery systems and time periods. Fourth, while we evaluated the association between electronic frailty metrics and major NSQIP complications, mortality, and readmissions, we did not evaluate the association between promising electronic frailty metrics and important procedure-specific complications, such as endoleaks after endovascular aortic aneurysm repair. Additionally, while we investigated various subpopulations based on specific procedure types, urgency, age, and operative approach, further investigation is needed into the predictive value of these frailty metrics and ultimately their utility in guiding care within specific populations and clinical contexts.

Conclusions

The focus on leveraging the EHR to generate a measure of frailty reflects the need for objective and standardized metrics to support preoperative assessments without creating excess work for frontline clinicians. Health systems that are considering how to embed electronic frailty metrics into clinical workflows must consider the performance characteristics of these tools for identifying frailty, the value of these metrics above and beyond other tools for preoperative risk stratification, and the feasibility of adapting and calculating these metrics within their information systems. Additional study is needed to develop and evaluate measures of frailty that can be implemented at scale and to develop electronic measures of conditions underlying frailty, such as sarcopenia, malnutrition, and functional limitation.

Back to top
Article Information

Accepted for Publication: December 31, 2021.

Published Online: March 16, 2022. doi:10.1001/jamasurg.2022.0172

Corresponding Author: Elizabeth M. Cespedes Feliciano, ScD, MSc, Division of Research, Kaiser Permanente Northern California, 2000 Broadway, 5th Floor, Oakland, CA 94612 (elizabeth.m.cespedes@kp.org).

Author Contributions: Drs Le and Zhang had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Le, Liu, Peng, Cespedes Feliciano.

Acquisition, analysis, or interpretation of data: Le, Liu, Kipnis, Zhang, Cespedes Feliciano.

Drafting of the manuscript: Le, Liu.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Le, Kipnis, Zhang.

Obtained funding: Cespedes Feliciano.

Administrative, technical, or material support: Liu, Peng, Cespedes Feliciano.

Supervision: Liu, Kipnis, Peng, Cespedes Feliciano.

Conflict of Interest Disclosures: Dr Liu reported grants from National Institutes of Health (G35GM128672) during the conduct of the study. Dr Cespedes Feliciano reported grants from the National Institute on Aging during the conduct of the study and grants from the National Cancer Institute outside the submitted work. No other disclosures were reported.

Funding/Support: This work was supported by National Institute on Aging/National Institutes of Health (R01AG065334). Dr Le received funding from The Permanente Medical Group Delivery Science Fellowship Program. Dr Liu was supported in part by National Institutes of Health (R35GM128672).

Role of the Funder/Sponsor: The National Institutes of Health and The Permanente Medical Group had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References
1.
Chow  WB, Rosenthal  RA, Merkow  RP, Ko  CY, Esnaola  NF; American College of Surgeons National Surgical Quality Improvement Program; American Geriatrics Society.  Optimal preoperative assessment of the geriatric surgical patient: a best practices guideline from the American College of Surgeons National Surgical Quality Improvement Program and the American Geriatrics Society.   J Am Coll Surg. 2012;215(4):453-466. doi:10.1016/j.jamcollsurg.2012.06.017PubMedGoogle ScholarCrossref
2.
Alvarez-Nebreda  ML, Bentov  N, Urman  RD,  et al.  Recommendations for Preoperative Management of Frailty from the Society for Perioperative Assessment and Quality Improvement (SPAQI).   J Clin Anesth. 2018;47:33-42. doi:10.1016/j.jclinane.2018.02.011PubMedGoogle ScholarCrossref
3.
Mohanty  S, Rosenthal  RA, Russell  MM, Neuman  MD, Ko  CY, Esnaola  NF.  Optimal perioperative management of the geriatric patient: a best practices guideline from the American College of Surgeons NSQIP and the American Geriatrics Society.   J Am Coll Surg. 2016;222(5):930-947. doi:10.1016/j.jamcollsurg.2015.12.026PubMedGoogle ScholarCrossref
4.
Dent  E, Morley  JE, Cruz-Jentoft  AJ,  et al.  Physical frailty: ICFSR international clinical practice guidelines for identification and management.   J Nutr Health Aging. 2019;23(9):771-787. doi:10.1007/s12603-019-1273-zPubMedGoogle ScholarCrossref
5.
Turner  G, Clegg  A; British Geriatrics Society; Age UK; Royal College of General Practioners.  Best practice guidelines for the management of frailty: a British Geriatrics Society, Age UK and Royal College of General Practitioners report.   Age Ageing. 2014;43(6):744-747. doi:10.1093/ageing/afu138PubMedGoogle ScholarCrossref
6.
Castillo-Angeles  M, Cooper  Z, Jarman  MP, Sturgeon  D, Salim  A, Havens  JM.  Association of frailty with morbidity and mortality in emergency general surgery by procedural risk level.   JAMA Surg. 2021;156(1):68-74. doi:10.1001/jamasurg.2020.5397PubMedGoogle ScholarCrossref
7.
Abellan van Kan  G, Rolland  Y, Houles  M, Gillette-Guyonnet  S, Soto  M, Vellas  B.  The assessment of frailty in older adults.   Clin Geriatr Med. 2010;26(2):275-286. doi:10.1016/j.cger.2010.02.002PubMedGoogle ScholarCrossref
8.
Farhat  JS, Velanovich  V, Falvo  AJ,  et al.  Are the frail destined to fail? frailty index as predictor of surgical morbidity and mortality in the elderly.   J Trauma Acute Care Surg. 2012;72(6):1526-1530. doi:10.1097/TA.0b013e3182542fabPubMedGoogle ScholarCrossref
9.
McIsaac  DI, Bryson  GL, van Walraven  C.  Association of frailty and 1-year postoperative mortality following major elective noncardiac surgery: a population-based cohort study.   JAMA Surg. 2016;151(6):538-545. doi:10.1001/jamasurg.2015.5085PubMedGoogle ScholarCrossref
10.
Suskind  AM, Walter  LC, Jin  C,  et al.  Impact of frailty on complications in patients undergoing common urological procedures: a study from the American College of Surgeons National Surgical Quality Improvement database.   BJU Int. 2016;117(5):836-842. doi:10.1111/bju.13399PubMedGoogle ScholarCrossref
11.
Eamer  G, Gibson  JA, Gillis  C,  et al.  Surgical frailty assessment: a missed opportunity.   BMC Anesthesiol. 2017;17(1):99. doi:10.1186/s12871-017-0390-7PubMedGoogle ScholarCrossref
12.
Santos-Eggimann  B, Sirven  N.  Screening for frailty: older populations and older individuals.   Public Health Rev. 2016;37(1):7. doi:10.1186/s40985-016-0021-8PubMedGoogle ScholarCrossref
13.
Walston  J, Buta  B, Xue  QL.  Frailty screening and interventions: considerations for clinical practice.   Clin Geriatr Med. 2018;34(1):25-38. doi:10.1016/j.cger.2017.09.004PubMedGoogle ScholarCrossref
14.
Shinall  MC  Jr, Arya  S, Youk  A,  et al.  Association of preoperative patient frailty and operative stress with postoperative mortality.   JAMA Surg. 2020;155(1):e194620. doi:10.1001/jamasurg.2019.4620PubMedGoogle ScholarCrossref
15.
Seib  CD, Rochefort  H, Chomsky-Higgins  K,  et al.  Association of patient frailty with increased morbidity after common ambulatory general surgery operations.   JAMA Surg. 2018;153(2):160-168. doi:10.1001/jamasurg.2017.4007PubMedGoogle ScholarCrossref
16.
Fowler  AJ, Abbott  TEF, Prowle  J, Pearse  RM.  Age of patients undergoing surgery.   Br J Surg. 2019;106(8):1012-1018. doi:10.1002/bjs.11148PubMedGoogle ScholarCrossref
17.
Kehler  DS, Ferguson  T, Stammers  AN,  et al.  Prevalence of frailty in Canadians 18-79 years old in the Canadian Health Measures Survey.   BMC Geriatr. 2017;17(1):28. doi:10.1186/s12877-017-0423-6PubMedGoogle ScholarCrossref
18.
Fried  LP, Tangen  CM, Walston  J,  et al; Cardiovascular Health Study Collaborative Research Group.  Frailty in older adults: evidence for a phenotype.   J Gerontol A Biol Sci Med Sci. 2001;56(3):M146-M156. doi:10.1093/gerona/56.3.M146PubMedGoogle ScholarCrossref
19.
Clegg  A, Young  J, Iliffe  S, Rikkert  MO, Rockwood  K.  Frailty in elderly people.   Lancet. 2013;381(9868):752-762. doi:10.1016/S0140-6736(12)62167-9PubMedGoogle ScholarCrossref
20.
Rolfson  DB, Majumdar  SR, Tsuyuki  RT, Tahir  A, Rockwood  K.  Validity and reliability of the Edmonton Frail Scale.   Age Ageing. 2006;35(5):526-529. doi:10.1093/ageing/afl041PubMedGoogle ScholarCrossref
21.
McIsaac  DI, Taljaard  M, Bryson  GL,  et al.  Frailty as a predictor of death or new disability after surgery: a prospective cohort study.   Ann Surg. 2020;271(2):283-289. doi:10.1097/SLA.0000000000002967PubMedGoogle ScholarCrossref
22.
Kovacs  J, Moraru  L, Antal  K, Cioc  A, Voidazan  S, Szabo  A.  Are frailty scales better than anesthesia or surgical scales to determine risk in cardiac surgery?   Korean J Anesthesiol. 2017;70(2):157-162. doi:10.4097/kjae.2017.70.2.157PubMedGoogle ScholarCrossref
23.
Kapoor  A, Matheos  T, Walz  M,  et al.  Self-reported function more informative than frailty phenotype in predicting adverse postoperative course in older adults.   J Am Geriatr Soc. 2017;65(11):2522-2528. doi:10.1111/jgs.15108PubMedGoogle ScholarCrossref
24.
Shah  R, Borrebach  JD, Hodges  JC,  et al.  Validation of the risk analysis index for evaluating frailty in ambulatory patients.   J Am Geriatr Soc. 2020;68(8):1818-1824. doi:10.1111/jgs.16453PubMedGoogle ScholarCrossref
25.
Khalafallah  AM, Huq  S, Jimenez  AE, Brem  H, Mukherjee  D. The 5-factor modified frailty index: an effective predictor of mortality in brain tumor patients. J Neurosurg. Published online August 14, 2020. doi:10.3171/2020.5.JNS20766
26.
Khalafallah  AM, Shah  PP, Huq  S,  et al.  The 5-Factor Modified Frailty Index predicts health burden following surgery for pituitary adenomas.   Pituitary. 2020;23(6):630-640. doi:10.1007/s11102-020-01069-5PubMedGoogle ScholarCrossref
27.
Gunnarsdottir  GM, Helgadottir  S, Einarsson  SG,  et al.  Validation of the Hospital Frailty Risk Score in older surgical patients: a population-based retrospective cohort study.   Acta Anaesthesiol Scand. 2021;65(8):1033-1042. doi:10.1111/aas.13837PubMedGoogle ScholarCrossref
28.
Hannah  TC, Neifert  SN, Caridi  JM,  et al.  Utility of the Hospital Frailty Risk Score for predicting adverse outcomes in degenerative spine surgery cohorts.   Neurosurgery. 2020;87(6):1223-1230. doi:10.1093/neuros/nyaa248PubMedGoogle ScholarCrossref
29.
Meyer  M, Parik  L, Leiß  F, Renkawitz  T, Grifka  J, Weber  M.  Hospital Frailty Risk Score predicts adverse events in primary total hip and knee arthroplasty.   J Arthroplasty. 2020;35(12):3498-3504.e3. doi:10.1016/j.arth.2020.06.087PubMedGoogle ScholarCrossref
30.
Aitken  SJ, Lujic  S, Randall  DA, Noguchi  N, Naganathan  V, Blyth  FM.  Predicting outcomes in older patients undergoing vascular surgery using the Hospital Frailty Risk Score.   Br J Surg. 2020.PubMedGoogle Scholar
31.
George  EL, Hall  DE, Youk  A,  et al.  Association between patient frailty and postoperative mortality across multiple noncardiac surgical specialties.   JAMA Surg. 2021;156(1):e205152-e205152. doi:10.1001/jamasurg.2020.5152PubMedGoogle ScholarCrossref
32.
Holzgrefe  RE, Wilson  JM, Staley  CA, Anderson  TL, Wagner  ER, Gottschalk  MB.  Modified frailty index is an effective risk-stratification tool for patients undergoing total shoulder arthroplasty.   J Shoulder Elbow Surg. 2019;28(7):1232-1240. doi:10.1016/j.jse.2018.12.004PubMedGoogle ScholarCrossref
33.
Chimukangara  M, Helm  MC, Frelich  MJ,  et al.  A 5-item frailty index based on NSQIP data correlates with outcomes following paraesophageal hernia repair.   Surg Endosc. 2017;31(6):2509-2519. doi:10.1007/s00464-016-5253-7PubMedGoogle ScholarCrossref
34.
Hall  DE, Arya  S, Schmid  KK,  et al.  Development and initial validation of the risk analysis index for measuring frailty in surgical populations.   JAMA Surg. 2017;152(2):175-182. doi:10.1001/jamasurg.2016.4202PubMedGoogle ScholarCrossref
35.
Gilbert  T, Neuburger  J, Kraindler  J,  et al.  Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study.   Lancet. 2018;391(10132):1775-1782. doi:10.1016/S0140-6736(18)30668-8PubMedGoogle ScholarCrossref
36.
Subramaniam  S, Aalberg  JJ, Soriano  RP, Divino  CM.  New 5-Factor Modified Frailty Index Using American College of Surgeons NSQIP Data.   J Am Coll Surg. 2018;226(2):173-181.e8. doi:10.1016/j.jamcollsurg.2017.11.005PubMedGoogle ScholarCrossref
37.
Callahan  KE, Clark  CJ, Edwards  AF,  et al.  Automated frailty screening at-scale for pre-operative risk stratification using the Electronic Frailty Index.   J Am Geriatr Soc. 2021;69(5):1357-1362. doi:10.1111/jgs.17027PubMedGoogle ScholarCrossref
38.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KG; TRIPOD Group.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.   Circulation. 2015;131(2):211-219. doi:10.1161/CIRCULATIONAHA.114.014508PubMedGoogle ScholarCrossref
39.
Massarweh  NN, Legner  VJ, Symons  RG, McCormick  WC, Flum  DR.  Impact of advancing age on abdominal surgical outcomes.   Arch Surg. 2009;144(12):1108-1114. doi:10.1001/archsurg.2009.204PubMedGoogle ScholarCrossref
40.
Jaber  S, Lescot  T, Futier  E,  et al; NIVAS Study Group.  Effect of noninvasive ventilation on tracheal reintubation among patients with hypoxemic respiratory failure following abdominal surgery: a randomized clinical trial.   JAMA. 2016;315(13):1345-1353. doi:10.1001/jama.2016.2706PubMedGoogle ScholarCrossref
41.
Futier  E, Constantin  J-M, Paugam-Burtz  C,  et al; IMPROVE Study Group.  A trial of intraoperative low-tidal-volume ventilation in abdominal surgery.   N Engl J Med. 2013;369(5):428-437. doi:10.1056/NEJMoa1301082PubMedGoogle ScholarCrossref
42.
Myles  PS, Bellomo  R, Corcoran  T,  et al; Australian and New Zealand College of Anaesthetists Clinical Trials Network and the Australian and New Zealand Intensive Care Society Clinical Trials Group.  Restrictive versus liberal fluid therapy for major abdominal surgery.   N Engl J Med. 2018;378(24):2263-2274. doi:10.1056/NEJMoa1801601PubMedGoogle ScholarCrossref
43.
Raval  MV, Cohen  ME, Ingraham  AM,  et al.  Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score.   J Am Coll Surg. 2010;211(6):715-723. doi:10.1016/j.jamcollsurg.2010.07.021PubMedGoogle ScholarCrossref
44.
Quan  H, Sundararajan  V, Halfon  P,  et al.  Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.   Med Care. 2005;43(11):1130-1139. doi:10.1097/01.mlr.0000182534.19832.83PubMedGoogle ScholarCrossref
45.
Charlson  ME, Pompei  P, Ales  KL, MacKenzie  CR.  A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.   J Chronic Dis. 1987;40(5):373-383. doi:10.1016/0021-9681(87)90171-8PubMedGoogle ScholarCrossref
46.
Escobar  GJ, Gardner  MN, Greene  JD, Draper  D, Kipnis  P.  Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system.   Med Care. 2013;51(5):446-453. doi:10.1097/MLR.0b013e3182881c8ePubMedGoogle ScholarCrossref
47.
Hall  BL, Hamilton  BH, Richards  K, Bilimoria  KY, Cohen  ME, Ko  CY.  Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals.   Ann Surg. 2009;250(3):363-376. doi:10.1097/SLA.0b013e3181b4148fPubMedGoogle ScholarCrossref
48.
Cohen  ME, Ko  CY, Bilimoria  KY,  et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.   J Am Coll Surg. 2013;217(2):336-346.e1. doi:10.1016/j.jamcollsurg.2013.02.027PubMedGoogle ScholarCrossref
49.
Rockwood  K, Song  X, MacKnight  C,  et al.  A global clinical measure of fitness and frailty in elderly people.   CMAJ. 2005;173(5):489-495. doi:10.1503/cmaj.050051PubMedGoogle ScholarCrossref
50.
Subramaniam  S, Aalberg  JJ, Soriano  RP, Divino  CM.  The 5-Factor Modified Frailty Index in the geriatric surgical population.   Am Surg. 2021;87(9):1420-1425. doi:10.1177/0003134820952438PubMedGoogle ScholarCrossref
51.
Arya  S, Varley  P, Youk  A,  et al.  Recalibration and external validation of the risk analysis index: a surgical frailty assessment tool.   Ann Surg. 2020;272(6):996-1005. doi:10.1097/SLA.0000000000003276PubMedGoogle ScholarCrossref
52.
Howlett  SE, Rockwood  MR, Mitnitski  A, Rockwood  K.  Standard laboratory tests to identify older adults at increased risk of death.   BMC Med. 2014;12(1):171. doi:10.1186/s12916-014-0171-9PubMedGoogle ScholarCrossref
53.
Gani  F, Canner  JK, Pawlik  TM.  Use of the Modified Frailty Index in the American College of Surgeons National Surgical Improvement Program database: highlighting the problem of missing data.   JAMA Surg. 2017;152(2):205-207. doi:10.1001/jamasurg.2016.3479PubMedGoogle ScholarCrossref
54.
Le  ST, Corbin  D, Myers  LC, Kipnis  P, Cohn  B, Liu  VX.  Development and validation of an electronic health record-based score for triage to perioperative medicine.   Ann Surg. 2021. Published online November 9, 2021. doi:10.1097/SLA.0000000000005284PubMedGoogle ScholarCrossref
55.
Evans  JD.  Straightforward Statistics for the Behavioral Sciences. Thomson Brooks/Cole Publishing Co; 1996.
56.
Hastie  T, Tibshirani  R, Friedman  J.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media; 2009. doi:10.1007/978-0-387-84858-7
57.
Fried  LP, Ferrucci  L, Darer  J, Williamson  JD, Anderson  G.  Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care.   J Gerontol A Biol Sci Med Sci. 2004;59(3):255-263. doi:10.1093/gerona/59.3.M255PubMedGoogle ScholarCrossref
58.
Austin  PC, Steyerberg  EW.  Predictive accuracy of risk factors and markers: a simulation study of the effect of novel markers on different performance measures for logistic regression models.   Stat Med. 2013;32(4):661-672. doi:10.1002/sim.5598PubMedGoogle ScholarCrossref
59.
Arruda-Olson  AM, Afzal  N, Priya Mallipeddi  V,  et al.  Leveraging the electronic health record to create an automated real-time prognostic tool for peripheral arterial disease.   J Am Heart Assoc. 2018;7(23):e009680. doi:10.1161/JAHA.118.009680PubMedGoogle ScholarCrossref
60.
Ridgway  JP, Robicsek  A, Shah  N,  et al.  A randomized controlled trial of an electronic clinical decision support tool for inpatient antimicrobial stewardship.   Clin Infect Dis. 2021;72(9):e265-e271. doi:10.1093/cid/ciaa1048PubMedGoogle ScholarCrossref
61.
Wu  CX, Suresh  E, Phng  FWL,  et al.  Effect of a real-time risk score on 30-day readmission reduction in Singapore.   Appl Clin Inform. 2021;12(2):372-382. doi:10.1055/s-0041-1726422PubMedGoogle ScholarCrossref
62.
Bergman  H, Ferrucci  L, Guralnik  J,  et al.  Frailty: an emerging research and clinical paradigm--issues and controversies.   J Gerontol A Biol Sci Med Sci. 2007;62(7):731-737. doi:10.1093/gerona/62.7.731PubMedGoogle ScholarCrossref
63.
Fisher  AL.  Just what defines frailty?   J Am Geriatr Soc. 2005;53(12):2229-2230. doi:10.1111/j.1532-5415.2005.00510.xPubMedGoogle ScholarCrossref
64.
Bilimoria  KY, Liu  Y, Paruch  JL,  et al.  Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons.   J Am Coll Surg. 2013;217(5):833-42.e1, 3. doi:10.1016/j.jamcollsurg.2013.07.385PubMedGoogle ScholarCrossref
65.
Bertges  DJ, Neal  D, Schanzer  A,  et al; Vascular Quality Initiative.  The Vascular Quality Initiative Cardiac Risk Index for prediction of myocardial infarction after vascular surgery.   J Vasc Surg. 2016;64(5):1411-1421.e4. doi:10.1016/j.jvs.2016.04.045PubMedGoogle ScholarCrossref
66.
Bihorac  A, Ozrazgat-Baslanti  T, Ebadi  A,  et al.  MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery.   Ann Surg. 2019;269(4):652-662. doi:10.1097/SLA.0000000000002706PubMedGoogle ScholarCrossref
67.
Brennan  M, Puri  S, Ozrazgat-Baslanti  T,  et al.  Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.   Surgery. 2019;165(5):1035-1045. doi:10.1016/j.surg.2019.01.002PubMedGoogle ScholarCrossref
68.
Elixhauser  A, Steiner  C, Harris  DR, Coffey  RM.  Comorbidity measures for use with administrative data.   Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004PubMedGoogle ScholarCrossref
69.
Escobar  GJ, Greene  JD, Scheirer  P, Gardner  MN, Draper  D, Kipnis  P.  Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases.   Med Care. 2008;46(3):232-239. doi:10.1097/MLR.0b013e3181589bb6PubMedGoogle ScholarCrossref
70.
Carli  F, Scheede-Bergdahl  C.  Prehabilitation to enhance perioperative care.   Anesthesiol Clin. 2015;33(1):17-33. doi:10.1016/j.anclin.2014.11.002PubMedGoogle ScholarCrossref
71.
Wynter-Blyth  V, Moorthy  K.  Prehabilitation: preparing patients for surgery.   BMJ. 2017;358:j3702. doi:10.1136/bmj.j3702PubMedGoogle ScholarCrossref
72.
Makary  MA, Segev  DL, Pronovost  PJ,  et al.  Frailty as a predictor of surgical outcomes in older patients.   J Am Coll Surg. 2010;210(6):901-908. doi:10.1016/j.jamcollsurg.2010.01.028PubMedGoogle ScholarCrossref
73.
Street  A, Maynou  L, Gilbert  T, Stone  T, Mason  S, Conroy  S.  The use of linked routine data to optimise calculation of the Hospital Frailty Risk Score on the basis of previous hospital admissions: a retrospective observational cohort study.   Lancet Healthy Longev. 2021;2(3):e154-e162. doi:10.1016/S2666-7568(21)00004-0PubMedGoogle ScholarCrossref
74.
Wan  MA, Clark  JM, Nuño  M, Cooke  DT, Brown  LM.  Can the risk analysis index for frailty predict morbidity and mortality in patients undergoing high-risk surgery?   Ann Surg. 2020. doi:10.1097/SLA.0000000000004626PubMedGoogle ScholarCrossref
×