[Skip to Navigation]
Sign In
Figure.  Course Design for a 4-Participant, 4-Instructor Expanded Advanced Surgical Skills for Exposures in Trauma (ASSET+) Course
Course Design for a 4-Participant, 4-Instructor Expanded Advanced Surgical Skills for Exposures in Trauma (ASSET+) Course

A, Day 1 is a formative experience during which participants perform trauma procedures guided by experienced trauma surgery faculty in a 1:1 participant:faculty ratio, with rotation of participants to another faculty after a lunch break. B, Day 2 is a summative experience, with participants repeating procedures from day 1 and being assessed by 2 faculty not seen on day 1.

Table 1.  Pretraining and Posttraining Procedural Confidence (5-Point Likert Scale), Knowledge (Number/% Correct), and Independence Outcomes (% of Procedures)
Pretraining and Posttraining Procedural Confidence (5-Point Likert Scale), Knowledge (Number/% Correct), and Independence Outcomes (% of Procedures)
Table 2.  Pretraining and Posttraining Performance Outcomes for 24 Trauma Proceduresa
Pretraining and Posttraining Performance Outcomes for 24 Trauma Proceduresa
Table 3.  Trauma Procedures Reported 3 Months After Traininga
Trauma Procedures Reported 3 Months After Traininga
1.
Haverkamp  FJC, Veen  H, Hoencamp  R,  et al.  Prepared for mission? a survey of medical personnel training needs within the International Committee of the Red Cross.   World J Surg. 2018;42(11):3493-3500. doi:10.1007/s00268-018-4651-5PubMedGoogle ScholarCrossref
2.
Tyler  JA, Ritchie  JD, Leas  ML,  et al.  Combat readiness for the modern military surgeon: data from a decade of combat operations.   J Trauma Acute Care Surg. 2012;73(2)(suppl 1):S64-S70. doi:10.1097/TA.0b013e3182625ebbPubMedGoogle ScholarCrossref
3.
Becker  HP, Gerngross  H, Schwab  R.  The challenge of military surgical education.   World J Surg. 2005;29(suppl 1):S17-S20. doi:10.1007/s00268-004-2054-2PubMedGoogle ScholarCrossref
4.
Drake  FT, Van Eaton  EG, Huntington  CR, Jurkovich  GJ, Aarabi  S, Gow  KW.  ACGME case logs: surgery resident experience in operative trauma for two decades.   J Trauma Acute Care Surg. 2012;73(6):1500-1506. doi:10.1097/TA.0b013e318270d983PubMedGoogle ScholarCrossref
5.
Bingmer  K, Ofshteyn  A, Stein  SL, Marks  JM, Steinhagen  E.  Decline of open surgical experience for general surgery residents.   Surg Endosc. 2020;34(2):967-972. doi:10.1007/s00464-019-06881-0PubMedGoogle ScholarCrossref
6.
Haag  A, Cone  EB, Wun  J,  et al.  Trends in surgical volume in the military health system: a potential threat to mission readiness.   Mil Med. 2020;186(7-8):646-650. doi:10.1093/milmed/usaa543Google ScholarCrossref
7.
Strumwasser  A, Grabo  D, Inaba  K,  et al.  Is your graduating general surgery resident qualified to take trauma call? a 15-year appraisal of the changes in general surgery education for trauma.   J Trauma Acute Care Surg. 2017;82(3):470-480. doi:10.1097/TA.0000000000001351PubMedGoogle ScholarCrossref
8.
Engels  PT, Versolatto  A, Shi  Q, Coates  A, Rice  TJ.  Cause for concern: resident experience in operative trauma during general surgery residency at a Canadian centre.   Can Med Educ J. 2020;11(6):e54-e59. doi:10.36834/cmej.69323PubMedGoogle Scholar
9.
Hall  AB, Davis  E, Vasquez  M,  et al.  Current challenges in military trauma readiness: insufficient relevant surgical case volumes in military treatment facilities.   J Trauma Acute Care Surg. 2020;89(6):1054-1060. doi:10.1097/TA.0000000000002871PubMedGoogle ScholarCrossref
10.
Edwards  MJ, White  CE, Remick  KN, Edwards  KD, Gross  KR.  Army general surgery’s crisis of conscience.   J Am Coll Surg. 2018;226(6):1190-1194. doi:10.1016/j.jamcollsurg.2018.03.001PubMedGoogle ScholarCrossref
11.
Hutter  PJ, Roski  J, Woodson  J,  et al.  Readiness of medical providers in the military health system: overview of operational and policy considerations.   Health Aff (Millwood). 2019;38(8):1274-1280. doi:10.1377/hlthaff.2019.00336PubMedGoogle ScholarCrossref
12.
Knudson  MM, Elster  EA, Bailey  JA, Woodson  J.  Military health system strategic partnership with the American College of Surgeons.   J Am Coll Surg. 2018;227(2):296-297. doi:10.1016/j.jamcollsurg.2018.06.001PubMedGoogle ScholarCrossref
13.
Edwards  MJ, Edwards  KD, White  C, Shepps  C, Shackelford  S.  Saving the military surgeon: maintaining critical clinical skills in a changing military and medical environment.   J Am Coll Surg. 2016;222(6):1258-1264. doi:10.1016/j.jamcollsurg.2016.03.031PubMedGoogle ScholarCrossref
14.
Mackenzie  CF, Tisherman  SA, Shackelford  S, Sevdalis  N, Elster  E, Bowyer  MW.  Efficacy of trauma surgery technical skills training courses.   J Surg Educ. 2019;76(3):832-843. doi:10.1016/j.jsurg.2018.10.004PubMedGoogle ScholarCrossref
15.
Mackenzie  CF, Elster  EA, Bowyer  MW, Sevdalis  N.  Scoping evidence review on training and skills assessment for open emergency surgery.   J Surg Educ. 2020;77(5):1211-1226. doi:10.1016/j.jsurg.2020.02.029PubMedGoogle ScholarCrossref
16.
Kuhls  DA, Risucci  DA, Bowyer  MW, Luchette  FA.  Advanced surgical skills for exposure in trauma: a new surgical skills cadaver course for surgery residents and fellows.   J Trauma Acute Care Surg. 2013;74(2):664-670. doi:10.1097/TA.0b013e31827d5e20PubMedGoogle ScholarCrossref
17.
Matthews  G, Wohleber  RW, Lin  J. Stress, skilled performance, and expertise: overload and beyond. In: Ward  P, Schraagen  JM, Gore  J, Roth  EM, eds.  Oxford Handbook of Expertise. Oxford University Press; 2019.
18.
Sweller  J. Cognitive load theory. In: Mestre JP, Ross BH.  Psychology of Learning and Motivation. Elsevier;2011:37-76.
19.
Andreatta  P. Cognitive neuroscience foundations of surgical and procedural expertise: focus on theory. In: Nestel  D, Reedy  G, McKenna  L, Gough  S, eds.  Clinical Education for the Health Professions. Springer;2020.
20.
Andreatta  PB, Dougherty  PJ. Supporting the development of psychomotor skills. In: Nestel  D, Dalrymple  K, Paige  JT, Aggarwal  R.  Advancing Surgical Education Theory, Evidence and Practice. Springer;2019:183-196.
21.
Kulik  C-LC, Kulik  JA, Bangert-Drowns  RL. 1990.  Effectiveness of mastery learning programs: a meta-analysis.   Rev Educ Res, 1990;60:265–299. doi:10.3102/00346543060002265Google ScholarCrossref
22.
Shackelford  S, Garofalo  E, Shalin  V,  et al.  Development and validation of trauma surgical skills metrics: preliminary assessment of performance after training.   J Trauma Acute Care Surg. 2015;79(1):105-110. doi:10.1097/TA.0000000000000685PubMedGoogle ScholarCrossref
23.
Cohen  J.  The statistical power of abnormal-social psychological research: a review.   J Abnorm Soc Psychol. 1962;65:145-153. doi:10.1037/h0045186PubMedGoogle ScholarCrossref
24.
Louras  N, Fortune  J, Osler  T, Hyman  N.  Nontrauma surgeons can safely take call at an academic, rural level I trauma center.   Am J Surg. 2016;211(1):129-132. doi:10.1016/j.amjsurg.2015.05.020PubMedGoogle ScholarCrossref
25.
Haut  ER, Chang  DC, Efron  DT, Cornwell  EE  III.  Injured patients have lower mortality when treated by “full-time” trauma surgeons vs. surgeons who cover trauma “part-time”.   J Trauma. 2006;61(2):272-278. doi:10.1097/01.ta.0000222939.51147.1cPubMedGoogle ScholarCrossref
26.
Haut  ER, Chang  DC, Hayanga  AJ, Efron  DT, Haider  AH, Cornwell  EE  III.  Surgeon- and system-based influences on trauma mortality.   Arch Surg. 2009;144(8):759-764. doi:10.1001/archsurg.2009.100PubMedGoogle ScholarCrossref
27.
Mackenzie  CF, Garofalo  E, Puche  A,  et al; Retention and Assessment of Surgical Performance (RASP) Group of Investigators.  Performance of vascular exposure and fasciotomy among surgical residents before and after training compared with experts.   JAMA Surg. 2017;152(6):581-588. doi:10.1001/jamasurg.2017.0092PubMedGoogle ScholarCrossref
28.
Mackenzie  CF, Bowyer  MW, Henry  S,  et al; Retention and Assessment of Surgical Performance Group of Investigators.  Cadaver-based trauma procedural skills training: skills retention 30 months after training among practicing surgeons in comparison to experts or more recently trained residents.   J Am Coll Surg. 2018;227(2):270-279. doi:10.1016/j.jamcollsurg.2018.04.028PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    September 15, 2021

    A Novel Paradigm for Surgical Skills Training and Assessment of Competency

    Author Affiliations
    • 1Department of Surgery, Uniformed Services University of the Health Sciences and the Walter Reed National Military Medical Center, Bethesda, Maryland
    • 2University of South Florida Morsani College of Medicine, Tampa
    JAMA Surg. 2021;156(12):1103-1109. doi:10.1001/jamasurg.2021.4412
    Key Points

    Question  Can general surgeons demonstrate improved trauma skills accuracy and independence following a standardized surgical skills curriculum?

    Findings  In this cohort study of 65 general surgeons who were assessed for expert criteria for 24 trauma skills during a standardized cadaveric course with repeated-measures design and real-time formative and summative assessment, accurate performance of skills improved from 1% to 99% with 78% of participants demonstrating independence.

    Meaning  Standardized, focused surgical skills training may improve performance and independence of procedural skills; this approach has broad applicability to all specialties with procedural competencies.

    Abstract

    Importance  Sustainment of comprehensive procedural skills in trauma surgery is a particular problem for surgeons in rural, global, and combat settings. Trauma care often requires open surgical procedures for low-frequency/high-risk injuries at a time when open surgical experience is declining in general and trauma surgery training.

    Objective  To determine whether general surgeons participating in a 2-day standardized trauma skills course demonstrate measurable improvement in accuracy and independent performance of specific trauma skills.

    Design, Setting, and Participants  General surgeons in active surgical practice were enrolled from a simulation center with anatomic laboratory from October 2019 to October 2020. Differences in pretraining/training and posttraining performance outcomes were examined using (1) pretraining/posttraining surveys, (2) pretraining/posttraining knowledge assessment, and (3) training/posttraining faculty assessment. Analysis took place in November 2020.

    Interventions  A 2-day standardized, immersive, cadaver-based skills course, developed with best practices in instructional design, that teaches and assesses 24 trauma surgical procedures was used.

    Main Outcomes and Measures  Trauma surgery capability, as measured by confidence, knowledge, abilities, and independent performance of specific trauma surgical procedures; 3-month posttraining skill transfer.

    Results  The study cohort included 65 active-duty general surgeons, of which 16 (25%) were women and 49 (75%) were men. The mean (SD) age was 38.5 (4.2) years. Before and during training, 1 of 65 participants (1%) were able to accurately perform all 24 procedures without guidance. After course training, 64 participants (99%) met the benchmark performance requirements for the 24 trauma procedures, and 51 (78%) were able to perform them without guidance. Procedural confidence and knowledge increased significantly from before to after the course. At 3 months after training, 37 participants (56%) reported skill transfer to trauma or other procedures.

    Conclusions and Relevance  In this study, direct measurement of procedural performance following standardized training demonstrated significant improvement in skill performance in a broad array of trauma procedures. This model may be useful for assessment of procedural competence in other specialties.

    Introduction

    Surgeons who care for trauma patients in rural, humanitarian, and military settings require competency in a broad-based skill set that transcends traditional training or practice.1-3 Modern surgical trainees have experienced a steady decline in open surgical experience, both in general and trauma surgery.4,5 This reality has led to concerns that graduating surgical resident physicians, trauma fellows, and many practicing general surgeons are inadequately prepared to treat traumatic injuries.6-8 This concern is magnified for the active-duty general surgeon, whose typical peacetime practice rarely, if ever, includes trauma and is limited in both volume and complexity.6 A 2020 review of the case logs of 57 trauma, general, and vascular surgeons at 4 major military hospitals with residency training programs found that only 0.3% of cases were relevant to combat casualty (trauma) care with the number of total mean cases per surgeon of only 92 per year.9

    To address the challenges of acquiring and maintaining relevant trauma knowledge, skills, and abilities, the US Department of Defense, in partnership with the American College of Surgeons, has initiated a data-driven Clinical Readiness Program.10-13 This approach sustains trauma currency and competency in military general surgeons through periodic assessment of knowledge and procedural skills; structured training and retraining; and measurement of readiness based on the predicted transferability of day-to-day surgical practice to the requirements for deployed casualty trauma care. Following review of existing trauma skills courses,14,15 we developed and implemented a comprehensive procedural skills course that uses evidence-based education principles in a standardized approach to teach and assess performance of 24 trauma surgical procedures. The course expands and reimagines a 1-day fresh cadaver-based trauma skills course, Advanced Surgical Skills for Exposures in Trauma (ASSET), developed by the American College of Surgeons Committee on Trauma.16 Requirements for this expanded course, titled ASSET+, were derived from identified gaps in the capabilities of deployed surgeons and determined by consensus agreement of military trauma surgeons who had deployed in support of combat missions. Over 2 days, participants engage in immersive hands-on training and assessment activities with 1-to-1 oversight by 4 experienced trauma surgeons. The ASSET+ course differs from the existing ASSET course, as well as the complimentary Advanced Trauma Operative Management course, by virtue of not only the 1:1 ratio, but more significantly by the inclusion of both a formative and summative assessment and the opportunity for each student to perform and be assessed on every procedure to a trauma surgeon standard. We hypothesized that this immersive cadaver-based skills course would measurably improve the accurate and independent performance of specific trauma surgical skills.

    Methods
    Study Design and Sample

    This study received expedited institutional review board approval from Uniformed Services University of the Health Sciences, School of Medicine without need for participant consent. The study cohort included active-duty general surgeons (without trauma surgery fellowship training or active practice of trauma) who completed the ASSET+ course. We used a repeated-measures study design to examine the differences between pretraining/training and posttraining performance outcomes. The training state of the participants was the only independent variable. We measured 4 dependent variables, each chosen owing to its known significance as a contributor to trauma surgery capability: confidence, knowledge, abilities, and independence.17-21 Participants were enrolled from October 2019 to October 2020.

    Throughout 18 months, 8 ASSET+ courses were conducted with 30 experienced trauma surgeon faculty, all of whom were current ASSET and Advanced Trauma Life Support instructors. All performance data were captured in real time using iPads (Apple) and the Ace Surgery App (Ace Metrics) and sent to a secure server. We controlled instructor variability by precourse 1-to-1 briefing of faculty by a professional psychometrician on how to use the instrumentation, 1-to-1 mentorship by master instructors, psychometrician coaching and oversight during the course, comparative analyses of new faculty scoring compared with the assigned mentor’s scoring (interrater reliability), and internal consistency of new faculty scores for learners who were also scored by other faculty on different procedures. Scores from all faculty are examined for internal consistency for each course. Outliers were identified and remediated until they were able to meet the requirements for assessors.

    Posttraining skill transfer was assessed by a survey sent to each participant by email at 3 months following course completion. A pilot examination of skills degradation was conducted in a small cohort (n = 8) of general surgeons who completed ASSET+ at least 6 months earlier.

    Course Design

    The ASSET+ courses were conducted throughout 2 days, during which participants rotated between tables to enable instruction and assessment of each participant by 4 separate experienced trauma surgeon instructors (Figure). The course provided immersive experiences in procedural skills deemed essential for deployed surgeons caring for trauma patients. Participants completed a pretraining procedural knowledge examination and procedural confidence survey prior to course start. The course manual was mailed to each participant 1 month before each course. During day 1 (training with formative assessment), participants were coached and assessed 1-to-1 by their assigned instructors while performing sequential trauma procedures in fresh cadavers within time limits consistent with actual situations. The procedural dissections were preceded by interactive, scenario-based PowerPoint (Microsoft) presentations that included indications, contraindications, and step-by-step static image and video demonstrations of standardized procedure performance. Immediately following each procedure, instructors completed a performance assessment on a tablet application to evaluate procedural ability and independence. Benchmarks for each procedure were defined as the completion of sequenced tasks to the quality standard of an experienced trauma surgeon. Participants were paired with a different instructor midday and received detailed feedback verbally (from both instructors) during session breaks and by emailed assessment reports for review before day 2.

    Day 2 focused on summative assessment without guidance from instructors, and 2 new instructors were assigned to each participant to assess their independent ability to perform the same sequential trauma procedures from day 1 using new fresh cadaveric specimens. All assessment reports were sent by email to the participants. The course concluded with participant completion of the posttraining procedural knowledge examination and procedural confidence survey.

    Assessment Instruments
    Procedural Confidence Survey

    Participants were asked to self-rate their precourse and postcourse procedural confidence for the 24 trauma procedures taught in the course on a 5-interval rating scale. Scores ranged from 1, very little confidence, to 5, a lot of confidence.

    Procedural Knowledge Assessment

    Two 40-item multiple choice examinations (pretest and posttest) with validated content covering the 24 trauma procedures were created. Factor analysis confirmed measurement validity for a single factor, with no crossloadings. Reliability was confirmed through test-retest and Chronbach α measure of internal consistency less than .70. We established the benchmark passing score as 28 (70%). The knowledge tests were administered before and after training through Google forms.

    Procedural Abilities Assessment

    Procedural abilities were assessed for each step required for completion of the procedures, with the standard defined as accurate performance. Critical tasks were derived from previously validated subject matter consensus task deconstruction for each of the procedures.21,22 Specific tasks included identification of anatomical structures, procedural sequencing, accurate approach and incisions, technique and instrument use, completeness of required procedural tasks, protection of neurovascular structures, avoidance of iatrogenic injury, and ability to manage complications. The scale for task assessment included not applicable, does not meet performance standards, partially meets performance standards, meets performance standards, and exceeds performance standards. Scores for task assessments were grounded on the performance accuracy and skill of experienced trauma surgeons performing each task. As an example, the assessment tool for cricothyroidotomy is included as the eTable in the Supplement. To account for cadaver habitus and anatomical variation, case difficulty was assessed for each procedure, and an evidence-based weighting factor was applied to specific procedure scores. Psychometric support for skills performance assessment instruments included content, convergent, criterion, and predictive validity evidence (P < .05) and test-retest reliability more than 0.90.

    The maximum score by procedure for all tasks performed accurately and independently was 100. The benchmark scores for all surgical skills were set to 90, which reflected accurate and independent performance of all critical tasks. Participants could score greater than 100 if they also performed efficiently.

    Procedural Independence Assessment

    Independent performance by procedure was assessed using a graded scale of ability to perform each step relative to guidance required: not applicable, required substantial guidance, required minimal guidance, no guidance required, and efficient/no guidance required. The standard was defined as independent performance.

    Skills Transfer Assessment and Retention Pilot

    Participants were surveyed at 3-month intervals after training to evaluate the use of course skills and abilities in the performance of trauma procedures or other operations. Further, 8 general surgeons who were at least 6 months from completion of the ASSET+ course repeated the summative experience from day 2 of the course. Results were compared with the postcourse performance benchmark recorded for each learner.

    Statistical Analyses

    Paired t tests were used to analyze the association of the training with procedural confidence, knowledge, and ability to perform each procedure accurately and independently. Statistical significance was set at P value less than .05. We calculated Cohen d as a measure of effect size for each procedure (with effect sizes categorized as medium, ≥0.5; large, ≥0.8; and very large, ≥1.4).23 Mean total procedural skills scores were calculated and compared with the performance benchmarks for accuracy and independence. Descriptive statistics are reported for transfer and retention outcomes. All statistics were performed using SPSS Statistics version 27 (IBM). Analysis took place in November 2020.

    Results
    Association of Training With Procedural Confidence, Knowledge, and Procedural Independence

    The study cohort included 65 active-duty general surgeons, of which 16 (25%) were women and 49 (75%) were men. Nine surgeons (14%) were African American, 9 (14%) were Asian American, 6 (9%) were Latin American, and 41 (63%) were White. There were significant improvements between pretraining and posttraining for procedural confidence (medium effect size) and knowledge (very large effect size) (Table 1). Procedural independence scores increased between formative and summative evaluation, with a very large effect size (Table 1). During training (day 1), 1 participant (1%) were able to perform all procedures to the defined standard without guidance. After training (day 2), 51 participants (78%) were able to perform all procedures to the defined standard without guidance, while the remaining 14 (22%) required minor guidance.

    Association of Training With Trauma Procedural Abilities

    There were significant improvements in performance scores for all procedures, with calculated effect sizes ranging from 0.59 to 1.56 (Table 2). During training, all participants reached the benchmark performance score for only 1 procedure (cricothyroidotomy). After training, 64 of 65 surgeons (98.5%) met the performance benchmark (score, ≥90) for all procedures, and benchmark aggregate performance was reached for 23 of 24 procedures (95.8%) (Table 2). Only thigh fasciotomy was slightly below benchmark at 88. The standard deviation of performance scores was significantly smaller after training for 23 of 24 procedures (95.8%), with the exception of trauma laparotomy.

    Transfer and Retention of Skills

    Of 41 surgeons who responded to the posttraining survey, 19 (46.3%) reported direct skill transfer to trauma procedures performed after training and 10 (24.4%) reported skill transfer to other operations (eg, nontrauma vascular exposures, intervention for intra-abdominal bleeding, and colon mobilization) (Table 3). Accounting for overlap, 23 individuals (56.1%) overall reported skill transfer to trauma or other procedures. Ten surgeons (24.4%) performed more than 2 trauma procedures in their subsequent practice.

    In the retention pilot study with 8 surgeons at 6 to 18 months (mean [SD], 10 [3.2] months) posttraining, knowledge remained above benchmark, while procedural abilities remained at or above the benchmark for 21 of 24 procedures (87.5%). Vascular exposures of popliteal and subclavian arteries fell slightly below the benchmark score.

    Discussion

    Prior to training, the majority of participants were not confident in their abilities to treat trauma patients, and during training, only 1% were able to perform all 24 procedures to standard without guidance. Following the ASSET+ course, 99% of participants met the benchmark performance requirements for 24 trauma procedures, and 78% were able to perform them without guidance. Confidence and knowledge scores also significantly increased. To our knowledge, this is the first report to demonstrate the association of trauma skills training through standardized training and direct measurement of procedural performance following training.

    Of the 24 trauma procedures performed in ASSET+, most are low-frequency, high-impact procedures that are used uncommonly by trauma surgeons and rarely by general surgeons. The primary objective of ASSET+ is to ensure the trauma readiness of military general surgeons who deploy to austere and hostile environments without other trauma support. The results of this study indicate that nontrauma-trained general surgeons can achieve benchmark standards of performance in these procedures, giving important input to the debate among trauma and military surgeons about whether military trauma surgery should be the exclusive domain of trauma specialists or should include general surgeons.24-26 Performance outcomes improved significantly for all participants across all procedures, and smaller standard deviations between pretraining and posttraining performance scores indicate less variation between participants after training (with the exception of trauma laparotomy). This exception may be the result of consistent general surgeon experience with elective laparotomy, enabling transfer of these skills to the trauma context at the outset of the course.

    Within 3 months of course completion, 56% of survey respondents had performed trauma procedures or applied skills from the ASSET+ course. The retention pilot suggested that knowledge and skills may be retained up to 18 months. Further work with a larger posttraining group is needed to define the retention interval further, yet these initial pilot findings, coupled with our previous work on skills retention,27,28 have prompted consideration of course renewal every 2 years for military surgeons.

    Strengths and Limitations

    Strengths of this educational intervention include the standardized content, instructional design that incorporates best practices in experiential learning, and rigorous performance assessment for each participant by multiple experts. The course appeals to multiple learning styles by using comprehensive multimodal learning activities and 1-to-1 direct table oversight with experienced trauma instructors. The assessment instruments are sufficiently precise to identify variance within individual procedural performance (how individuals varied in their performance across procedures), as well as procedural performance between individuals (how performance on 1 procedure varied between individuals). The detailed assessments provide a common basis for instructor feedback to participants. Over 2 days, each participant was the primary surgeon on all 24 procedures, and 4 different surgeons independently evaluated each participant. This method ensures comprehensive and objective assessment of participant abilities. This education model addresses weaknesses in existing trauma training skills courses, which include variable participant:faculty ratios (typically 2:1 or 4:1), inability of participants to perform all procedures because of limitations with a single cadaver, and single formative participant assessment.

    Study limitations include the short interval between formative and summative assessment (<24 hours) and the small size of the retention pilot. Thus, this study cannot demonstrate long-term retention of specific procedural skills in the entire cohort. Results may have been more favorable because the participants are particularly motivated, given the probability of deployment. Although concerns with assessment reliability between instructors were attenuated by rigorous training of faculty, data monitoring for outlier scores during the course, and 4-instructor evaluation of each participant, variations between the perceptions and judgments of individual instructors may be present. Study reproducibility is challenged by the expense (at least double the cost of a 1-day cadaver course) and opportunity costs for participants and faculty.

    Conclusions

    Previous research suggests that even experienced trauma surgeons may lack readiness to perform many of the 24 trauma procedures to benchmarked standards and may benefit from periodic refresher training to maintain competency.28 Thus, the ASSET+ model may have relevance for the introduction and sustainment of trauma operative skills for surgeons in civilian trauma, rural surgery, and global surgery. Further, this novel education and assessment paradigm may be applicable to procedural training for all medical specialties and may serve as a framework for higher stakes assessments toward board certification. Such a structured framework enables movement beyond formative to summative evaluation of procedural skills.

    Back to top
    Article Information

    Corresponding Author: Mark W. Bowyer, MD, Department of Surgery, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Rd, Bethesda, MD 20814 (mark.bowyer@usuhs.edu).

    Accepted for Publication: July 4, 2021.

    Published Online: September 15, 2021. doi:10.1001/jamasurg.2021.4412

    Author Contributions: Drs Bowyer and Andreatta had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Bowyer, Andreatta, Elster.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Bowyer, Andreatta, Armstrong, Elster.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Bowyer, Andreatta.

    Obtained funding: Elster.

    Administrative, technical, or material support: Bowyer, Andreatta, Armstrong, Elster.

    Supervision: Bowyer, Andreatta, Elster.

    Conflict of Interest Disclosures: None reported.

    Disclaimer: The contents of this presentation are the sole responsibility of the authors and do not necessarily reflect the views, opinions or policies of Uniformed Services University of the Health Sciences, the US Department of Defense, or the US Departments of the Army, Navy, or Air Force. Mention of trade names, commercial products, or organizations does not imply endorsement by the US government.

    References
    1.
    Haverkamp  FJC, Veen  H, Hoencamp  R,  et al.  Prepared for mission? a survey of medical personnel training needs within the International Committee of the Red Cross.   World J Surg. 2018;42(11):3493-3500. doi:10.1007/s00268-018-4651-5PubMedGoogle ScholarCrossref
    2.
    Tyler  JA, Ritchie  JD, Leas  ML,  et al.  Combat readiness for the modern military surgeon: data from a decade of combat operations.   J Trauma Acute Care Surg. 2012;73(2)(suppl 1):S64-S70. doi:10.1097/TA.0b013e3182625ebbPubMedGoogle ScholarCrossref
    3.
    Becker  HP, Gerngross  H, Schwab  R.  The challenge of military surgical education.   World J Surg. 2005;29(suppl 1):S17-S20. doi:10.1007/s00268-004-2054-2PubMedGoogle ScholarCrossref
    4.
    Drake  FT, Van Eaton  EG, Huntington  CR, Jurkovich  GJ, Aarabi  S, Gow  KW.  ACGME case logs: surgery resident experience in operative trauma for two decades.   J Trauma Acute Care Surg. 2012;73(6):1500-1506. doi:10.1097/TA.0b013e318270d983PubMedGoogle ScholarCrossref
    5.
    Bingmer  K, Ofshteyn  A, Stein  SL, Marks  JM, Steinhagen  E.  Decline of open surgical experience for general surgery residents.   Surg Endosc. 2020;34(2):967-972. doi:10.1007/s00464-019-06881-0PubMedGoogle ScholarCrossref
    6.
    Haag  A, Cone  EB, Wun  J,  et al.  Trends in surgical volume in the military health system: a potential threat to mission readiness.   Mil Med. 2020;186(7-8):646-650. doi:10.1093/milmed/usaa543Google ScholarCrossref
    7.
    Strumwasser  A, Grabo  D, Inaba  K,  et al.  Is your graduating general surgery resident qualified to take trauma call? a 15-year appraisal of the changes in general surgery education for trauma.   J Trauma Acute Care Surg. 2017;82(3):470-480. doi:10.1097/TA.0000000000001351PubMedGoogle ScholarCrossref
    8.
    Engels  PT, Versolatto  A, Shi  Q, Coates  A, Rice  TJ.  Cause for concern: resident experience in operative trauma during general surgery residency at a Canadian centre.   Can Med Educ J. 2020;11(6):e54-e59. doi:10.36834/cmej.69323PubMedGoogle Scholar
    9.
    Hall  AB, Davis  E, Vasquez  M,  et al.  Current challenges in military trauma readiness: insufficient relevant surgical case volumes in military treatment facilities.   J Trauma Acute Care Surg. 2020;89(6):1054-1060. doi:10.1097/TA.0000000000002871PubMedGoogle ScholarCrossref
    10.
    Edwards  MJ, White  CE, Remick  KN, Edwards  KD, Gross  KR.  Army general surgery’s crisis of conscience.   J Am Coll Surg. 2018;226(6):1190-1194. doi:10.1016/j.jamcollsurg.2018.03.001PubMedGoogle ScholarCrossref
    11.
    Hutter  PJ, Roski  J, Woodson  J,  et al.  Readiness of medical providers in the military health system: overview of operational and policy considerations.   Health Aff (Millwood). 2019;38(8):1274-1280. doi:10.1377/hlthaff.2019.00336PubMedGoogle ScholarCrossref
    12.
    Knudson  MM, Elster  EA, Bailey  JA, Woodson  J.  Military health system strategic partnership with the American College of Surgeons.   J Am Coll Surg. 2018;227(2):296-297. doi:10.1016/j.jamcollsurg.2018.06.001PubMedGoogle ScholarCrossref
    13.
    Edwards  MJ, Edwards  KD, White  C, Shepps  C, Shackelford  S.  Saving the military surgeon: maintaining critical clinical skills in a changing military and medical environment.   J Am Coll Surg. 2016;222(6):1258-1264. doi:10.1016/j.jamcollsurg.2016.03.031PubMedGoogle ScholarCrossref
    14.
    Mackenzie  CF, Tisherman  SA, Shackelford  S, Sevdalis  N, Elster  E, Bowyer  MW.  Efficacy of trauma surgery technical skills training courses.   J Surg Educ. 2019;76(3):832-843. doi:10.1016/j.jsurg.2018.10.004PubMedGoogle ScholarCrossref
    15.
    Mackenzie  CF, Elster  EA, Bowyer  MW, Sevdalis  N.  Scoping evidence review on training and skills assessment for open emergency surgery.   J Surg Educ. 2020;77(5):1211-1226. doi:10.1016/j.jsurg.2020.02.029PubMedGoogle ScholarCrossref
    16.
    Kuhls  DA, Risucci  DA, Bowyer  MW, Luchette  FA.  Advanced surgical skills for exposure in trauma: a new surgical skills cadaver course for surgery residents and fellows.   J Trauma Acute Care Surg. 2013;74(2):664-670. doi:10.1097/TA.0b013e31827d5e20PubMedGoogle ScholarCrossref
    17.
    Matthews  G, Wohleber  RW, Lin  J. Stress, skilled performance, and expertise: overload and beyond. In: Ward  P, Schraagen  JM, Gore  J, Roth  EM, eds.  Oxford Handbook of Expertise. Oxford University Press; 2019.
    18.
    Sweller  J. Cognitive load theory. In: Mestre JP, Ross BH.  Psychology of Learning and Motivation. Elsevier;2011:37-76.
    19.
    Andreatta  P. Cognitive neuroscience foundations of surgical and procedural expertise: focus on theory. In: Nestel  D, Reedy  G, McKenna  L, Gough  S, eds.  Clinical Education for the Health Professions. Springer;2020.
    20.
    Andreatta  PB, Dougherty  PJ. Supporting the development of psychomotor skills. In: Nestel  D, Dalrymple  K, Paige  JT, Aggarwal  R.  Advancing Surgical Education Theory, Evidence and Practice. Springer;2019:183-196.
    21.
    Kulik  C-LC, Kulik  JA, Bangert-Drowns  RL. 1990.  Effectiveness of mastery learning programs: a meta-analysis.   Rev Educ Res, 1990;60:265–299. doi:10.3102/00346543060002265Google ScholarCrossref
    22.
    Shackelford  S, Garofalo  E, Shalin  V,  et al.  Development and validation of trauma surgical skills metrics: preliminary assessment of performance after training.   J Trauma Acute Care Surg. 2015;79(1):105-110. doi:10.1097/TA.0000000000000685PubMedGoogle ScholarCrossref
    23.
    Cohen  J.  The statistical power of abnormal-social psychological research: a review.   J Abnorm Soc Psychol. 1962;65:145-153. doi:10.1037/h0045186PubMedGoogle ScholarCrossref
    24.
    Louras  N, Fortune  J, Osler  T, Hyman  N.  Nontrauma surgeons can safely take call at an academic, rural level I trauma center.   Am J Surg. 2016;211(1):129-132. doi:10.1016/j.amjsurg.2015.05.020PubMedGoogle ScholarCrossref
    25.
    Haut  ER, Chang  DC, Efron  DT, Cornwell  EE  III.  Injured patients have lower mortality when treated by “full-time” trauma surgeons vs. surgeons who cover trauma “part-time”.   J Trauma. 2006;61(2):272-278. doi:10.1097/01.ta.0000222939.51147.1cPubMedGoogle ScholarCrossref
    26.
    Haut  ER, Chang  DC, Hayanga  AJ, Efron  DT, Haider  AH, Cornwell  EE  III.  Surgeon- and system-based influences on trauma mortality.   Arch Surg. 2009;144(8):759-764. doi:10.1001/archsurg.2009.100PubMedGoogle ScholarCrossref
    27.
    Mackenzie  CF, Garofalo  E, Puche  A,  et al; Retention and Assessment of Surgical Performance (RASP) Group of Investigators.  Performance of vascular exposure and fasciotomy among surgical residents before and after training compared with experts.   JAMA Surg. 2017;152(6):581-588. doi:10.1001/jamasurg.2017.0092PubMedGoogle ScholarCrossref
    28.
    Mackenzie  CF, Bowyer  MW, Henry  S,  et al; Retention and Assessment of Surgical Performance Group of Investigators.  Cadaver-based trauma procedural skills training: skills retention 30 months after training among practicing surgeons in comparison to experts or more recently trained residents.   J Am Coll Surg. 2018;227(2):270-279. doi:10.1016/j.jamcollsurg.2018.04.028PubMedGoogle ScholarCrossref
    ×