GI indicates gastrointestinal.
The percentages in the top quartile were 0% (0 of 9) in 2009, 11% (1 of 9) in 2010, 10% (1 of 10) in 2011, and 50% (5 of 10) in 2012.
Krajewski A, Filippa D, Staff I, Singh R, Kirton OC. Implementation of an Intern Boot Camp Curriculum to Address Clinical Competencies Under the New Accreditation Council for Graduate Medical Education Supervision Requirements and Duty Hour Restrictions. JAMA Surg. 2013;148(8):727-732. doi:10.1001/jamasurg.2013.2350
Copyright 2013 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Today’s general surgery interns are faced with increased duty hour restrictions and stringent competency-based supervision milestone requirements (ie, from direct to indirect supervision). Working within these constraints, we instituted a unique 2-month intern curriculum (boot camp) incorporating knowledge-based, experiential, and practical components.
To describe our curriculum and the effect on resident performance and teaching faculty and nursing staff perceptions.
All interns underwent a 2-month (July and August 2011) boot camp curriculum consisting of two 2½-hour knowledge-based and procedural skills (SimMan) didactic sessions per week and completion of 25 core intensive introductory American College of Surgeons Fundamentals of Surgery web-based self-study modules, followed by a standardized patient clinical skills assessment.
Integrated general surgery residency program at the University of Connecticut School of Medicine, Farmington.
Postgraduate year 1 general surgery categorical and preliminary residents.
Main Outcomes and Measures
: We used several assessment tools, including an intern boot camp survey, clinical skills assessment scores, intern American Board of Surgeons In-Training Examination scores, and nursing staff and teaching faculty surveys of intern performance and aptitudes compared with the previous year’s interns. Data were analyzed by independent group t test, χ2 tests of proportions, and Fisher exact test for small sample cross tables.
In total, 84% (91 of 108) of intern respondents agreed or strongly agreed with the usefulness, relevance, and execution of the boot camp. Compared with the previous year’s interns, the nursing staff agreed or strongly agreed that the cohort interns were better at patient assessment, collaboration, and effective communication and provided compassionate and respectful patient care. More than 40% (7 of 17) of surveyed teaching faculty agreed or strongly agreed that the cohort interns demonstrated better patient care and procedural skills and self-confidence compared with the previous year’s interns. The clinical skills assessment scores after the 2-month boot camp paralleled the scores typically seen at the end of the previous 2 internship years (P > .25 for all). The proportion of nondesignated and categorical interns pursuing careers in general surgery scoring in the top quartile on the American Board of Surgery In-Training Examination increased from 7% (2 of 28) to 50% (5 of 10) compared with the previous 2 internship years (P = .01).
Conclusions and Relevance
Recent changes in intern duty hours and supervision rules mandate that residency training programs must institute a competency-oriented curriculum to provide interns with the necessary knowledge and practical skills to attain clinical competence.
Surgical education has come a long way since the advent of the first modern American surgical residency under William Halstead. Residents were required to live in the hospitals, work countless hours, and achieve competency in an apprenticelike atmosphere.1 Several key landmark events have slowly shaped surgical residency into the reality it is today. The publication of the Flexner report in 1910 brought national attention to the state of medical education and promoted the values of scientific rigor and discipline in medical teaching.1 The aftermath of the 1984 Libby Zion case in New York City forever turned attention to the structure of surgical education.2 The 2003 Accreditation Council for Graduate Medical Education (ACGME) common program requirements emphasized the importance of resident fatigue, mandated an 80-hour workweek, and ushered in competency-based curriculum and evaluation.1- 4 No longer were 120-hour workweeks acceptable, and the paramount importance of resident supervision took center stage.2,4,5 In 2011, the framework of resident education was further limited with the introduction of 16-hour maximum duty shifts for interns and more stringent oversight rules.3- 5 General surgery residency has evolved into a perfect storm, where the realities of duty hour restrictions, more stringent junior resident oversight, and meeting of competency-based education requirements are inherent components of an ACGME-accredited training program curriculum. In July 2011, we instituted a 2-month, twice-weekly protected-time intern boot camp using simulation and skills laboratories, interactive competency-based didactics, participation in a structured clinical skills assessment (CSA) laboratory, and completion of 25 core intensive introductory American College of Surgeons Fundamental of Surgery Web-based self-study modules. The objective of this article is to describe our curriculum and its effect on resident performance and teaching faculty and nursing staff perceptions.
Our study was reviewed by the institutional review board at the University of Connecticut, who determined that this activity did not meet the federal definition of research according to 45 CFR §46.102(d). All residents provided their consent for participation in the study. The University of Connecticut School of Medicine general surgery residency program comprises 46 residents, graduating 6 chief residents each year. The internship class consists of 18 residents, including 6 categorical residents, 7 designated subspecialty preliminary residents, and 5 nondesignated preliminary general surgery residents. Categorical and designated subspecialty preliminary residents are primarily from US ACGME-accredited medical schools; the nondesignated preliminary general surgery residents are primarily from international medical schools. The mean (SD) internship class United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores were 221 (18) and 226 (19), respectively, for 2009 through 2011. All surgical interns (n = 18) at the University of Connecticut School of Medicine during the 2011-2012 academic year participated in the intern boot camp.
The boot camp curriculum comprised the following 4 learning components: didactic sessions, an actor-based CSA laboratory, simulation and skills laboratories, and completion of 25 core intensive introductory American College of Surgeons Fundamentals of Surgery web-based self-study modules. Surgical teaching faculty (including O.C.K.) taught the didactic sessions. Topics included preoperative evaluation and postoperative management of clinical issues such as pain, fever, wounds, nutrition, and arrhythmias (Figure 1). The procedural skills (SimMan; Laerdal) scenarios occurred in the Center for Education, Simulation, and Innovation (CESI), a 1395.5-m2 dedicated simulation center (http://www.harthosp.org/simcenter/default.aspx). The CESI contains a simulated operating room, intensive care unit, delivery room, and trauma room. It also contains several laparoscopic, endoscopic, robotic, endovascular, and surgical techniques suites. Interns practice suturing, knot tying, airway intubation, and insertion of Foley catheters, chest tubes, and central venous lines, as well as managing simulated patients on high-fidelity mannequins for gastrointestinal tract bleeding, cardiac arrhythmias, and septic shock scenarios. The interns were tested for competency on each of the procedural skills scenarios. Interns were encouraged to practice skills during their free time. They were able to practice the skills after-hours in the CESI because each resident had 24/7 access. We do not know how many times a resident performed each of the skills; however, the resident had to pass the competency evaluation metric, which was administered by a teaching faculty member.
The American College of Surgeons Fundamentals of Surgery is a web-based interactive curriculum that addresses surgical topics essential to junior residents. Residents must assess symptoms and signs, order appropriate tests and procedures, and evaluate data and initiate appropriate actions in each clinical scenario. The interns were required to complete 25 core intensive introductory Fundamentals of Surgery web-based self-study modules. The modules were linked to the subject matter within the curriculum.
The 25 core intensive introductory Fundamentals of Surgery web-based self-study modules included the following: safety (module 1), nutrition (module 2), mental status (module 1), cardiac (modules 1 and 2), wounds (modules 2 and 4), pain management (module 1), postoperative fever (module 2), pulmonary (modules 1-3), nutrition introduction (module 1), preoperative evaluation (module 1), gastrointestinal (modules 3, 4, and 7), postoperative fever introduction (module 1), postoperative hypotension (modules 2 and 4), and fluids and electrolytes (modules 1, 2, 4, 7, and 8).
The CSA is a comprehensive assessment of a student’s encounter with an actor patient in a simulated environment. The intern participates in a patient encounter in which he or she needs to complete an appropriate history and physical examination, determine a differential diagnosis, and communicate with the patient the plan. Until the 2010-2011 academic year, the CSA was administered near the completion of the postgraduate year 1 (April and May). With the intern boot camp, the CSA exercise was shifted to immediately follow the completion of the boot camp didactic lecture series.
We used several measures to assess the effect of the boot camp curriculum. A perception survey was distributed to the interns at the conclusion of the boot camp, in which they were asked to assess curriculum topics and the overall value of the boot camp using a Likert-type scale (strongly disagree, disagree, neither disagree nor agree, agree, or strongly agree).
Nursing staff and teaching faculty were administered surveys at 3 months after the conclusion of the boot camp to assess their perception of intern performance. These surveys also used a Likert-type scale, in which the responder could strongly disagree, disagree, neither disagree nor agree, agree, or strongly agree. Six general surgery service nurse managers from the 5 hospitals that comprise the residency program rotation sites completed the surveys. The nurse managers responded to the survey after discussing the questions with nursing staff and reaching a consensus. The nursing staff were asked to compare the current academic year’s interns (who completed the boot camp curriculum) with the previous year’s interns (who did not complete the boot camp curriculum) in their ability to be respectful, actively listen, assess a patient, cope with stressful situations, and collaborate in a professional manner. The teaching faculty with the most exposure to the interns completed the teaching faculty survey (n = 17). The teaching faculty were similarly asked to compare the current academic year’s interns with the previous year’s interns. However, the faculty survey specifically addressed the core competencies of medical knowledge, patient care and procedural skills, and communication and interpersonal skills, as well as professionalism (specifically the perception of intern self-confidence, the intern’s ability to demonstrate adequate knowledge of pathophysiology, and the intern’s ability to perform accurate and thorough histories and physical examinations). The CSA was used as a performance measure. Intern performance was measured through history, physical examination, and communication checklists. Interns were evaluated using the Master Interview Rating Scale, which evaluates the interactions between the trainee and a standardized patient. The Master Interview Rating Scale includes 27 items rooted in the Arizona Clinical Interview Rating Scale and uses a 5-point Likert-type scale with anchoring statements to define in objective terms the behaviors of a clinician. These scores were analyzed between groups with independent group t tests.
The scores on the January American Board of Surgeons In-Training Examination of the current academic year’s interns were compared with the 2 years before instituting the boot camp curriculum. χ2 Tests of proportions and Fisher exact test for small sample cross tables were used for data analysis (P < .05 was considered significant).
All boot camp sessions were well attended; only 2 of 18 interns missed more than 1 session because of on-call status, with 11% (12 of 108 sessions) overall absenteeism if vacation and medical leave are counted. Each resident was assessed by a faculty member when performing each skill. Only 2 of 18 interns needed remedial skills training on suturing and knot tying, and both subsequently demonstrated competency.
Eighty-four percent (91 of 108) of intern responses agreed or strongly agreed with the usefulness, relevance, and execution of the boot camp. The intern boot camp survey revealed that the interns considered the instructors knowledgeable, the overall boot camp curriculum of value, and the Fundamentals of Surgery web-based self-study modules and didactic lectures relevant. The interns reported more self-confidence with procedural activities such as intubation, chest tube insertion, and Foley catheter insertion (Table 1). In the survey, interns also included their comments; some of the responses included “excellent experience,” “appreciated the skills sessions,” and “I believe it’s an essential tool to bring us up to speed.” Interns were also asked to comment on what they would like to see improved in the boot camp curriculum. These responses demonstrated that the interns would have liked to cover wound care and practical aspects of electrolyte management in more detail. In general, the interns wanted to cover the practical aspects of being an intern rather than theoretical topics. Interns considered the lectures provided by nonsurgeons (on geriatric medicine, acute mental status change, and cardiac chest pain and arrhythmias) too focused on obscure nonsurgical conditions, lacking immediate bedside applicability. Furthermore, the interns wanted better task coordination and stocking of supplies. For example, the CESI had limited chest tube mannequins for the skills station and a limited number of ultrasonography machines for central venous line placement.
Six general surgery service nurse floor managers from the 5 hospitals that comprise the residency completed the nursing surveys. Compared with the previous year’s interns, the general surgery nurse floor managers agreed or strongly agreed that the cohort interns were more respectful, communicated effectively, adhered to ethical principles, provided compassionate patient care, and were better able to assess the patient (Table 2). More than 40% (7 of 17) of surveyed teaching faculty agreed or strongly agreed that the cohort interns demonstrated better patient care and procedural skills and self-confidence compared with the previous year’s interns (Table 3).
The CSA performed at the completion of the 2-month boot camp did not reveal significant differences between the cohort interns and the interns in the 2 prior years, who received the CSA near the completion of their internship. The mean (SD) percentages of correct responses for the prior year interns and the cohort interns, respectively, were 75% (7%) and 73% (4%) for histories (P = .42), 45% (11%) and 46% (11%) for physical examinations (P = .66), 87% (6%) and 87% (5%) for patient encounter notes (P = .96), and 74% (5%) and 72% (6%) for the Master Interview Rating Scale (P = .25) (Figure 2).
We found that in the 3 years before initiation of the structured boot camp, only 2 nondesignated and categorical interns pursuing careers in general surgery (7.1%) were scoring in the top quartile on the American Board of Surgeons In-Training Examination, whereas 5 of 10 of the current 2011 cohort interns (50%) scored in the top quartile (P = .01) (Figure 3). There were no differences in USMLE Step 1 and Step 2 scores or demographics between the intern years before the implementation and actuation of the boot camp. The mean (SD) USMLE Step 1 scores were 219 (17) in 2009, 220 (11) in 2010, and 223 (19) in 2011 (P = .32). The mean (SD) USMLE Step 2 scores were 220 (20) in 2009, 228 (18) in 2010, and 229 (21) in 2011 (P = .37).
The uniqueness of our intensive 2½-month boot camp curriculum is that it was designed to enable the intern to maximize his or her learning in an era of duty hour restrictions and to achieve competency.4,6- 8 The Friday core curriculum includes a 2½-hour protected time, when residents are free of clinical responsibilities. In addition, the interns had another 2½-hour protected educational time during the week for the first 2 months. Web-based reading material and 24/7 access to the simulation laboratory were important learning requisites. Furthermore, the intern boot camp was designed to satisfy the ACGME requirement that the progression of interns from direct supervision to indirect supervision with direct supervision immediately available must follow the attainment of demonstrable clinical and performance competency. The intern must become credentialed to independently perform tasks such as history and physical examination, consults, and bedside procedures. Our credentialing process is embedded in the intern boot camp, including 10 didactic lectures, 9 procedural skills stations and high-fidelity simulations, and completion of 25 core intensive introductory Fundamentals of Surgery web-based self-study modules, as well as participation in the CSA, all with indirect supervision approved by the residency clinical competency committee.
This first year of our new intern boot camp curriculum received high resident satisfaction. The results of the nursing staff and faculty surveys also showed improvement in staff perceptions of intern performance, aptitudes, and interpersonal and communication skills. We also thought that the basic surgical science curriculum had a role in leading to improvement in the year’s American Board of Surgeons In-Training Examination scores. The 25 core intensive introductory Fundamentals of Surgery web-based self-study modules were linked to the subject matter within the curriculum. Although prior years’ interns were encouraged to complete the Fundamentals of Surgery modules at their leisure and interest, it was mandatory for the 2011 class to complete the 25 core intensive introductory modules. There was no significant between-year difference in USMLE scores for the preliminary residents and categorical residents to explain the variation in American Board of Surgeons In-Training Examination results. We believe that the difference could be explained by a strong start to the year with emphasis on preparation, adult learning, and basic science.
Although we found no differences in the CSA scores between the comparison groups of interns, we thought that incorporating the CSA into the intern boot camp significantly enhanced the intern core curriculum and reflected a more comprehensive and inclusive education for all the general surgery interns embarking on surgical training. Before implementation of the boot camp curriculum, the CSA was administered to interns nearing the completion of their intern year. By shifting the CSA to earlier in the academic year, the CSA became an important part of a more comprehensive boot camp intern curriculum, and the results demonstrated that an earlier introduction in the intern year may accelerate competency attainment. Whereas the didactic lectures emphasized basic science and clinical pathophysiology, the role of the CSA was to highlight clinical assessment, communication skills, and professional expectations of the resident. The interns received direct feedback, gained insight into their bedside clinical strengths and deficiencies, and were able to apply these competencies to real-life patient encounters. Integration of the CSA into the boot camp experience gave interns the opportunity to incorporate these principles into their daily patient interactions. Based on these results, we will no longer administer CSA in the late spring of the internship year.
Starting in 2003, residents were limited to 80-hour workweeks. Beginning in July 2011, the duty period for interns could not exceed 16 hours in duration within that 80-hour workweek.2,4,5 In addition, interns should have 10 hours (and must have 8 hours) free of duty between scheduled duty periods. This duty time limitation led many physicians to believe that adequate surgical training could not be accomplished within those constraints. Adding to the restrictions imposed by duty hours are those imposed by increasing oversight. This is particularly true of interns, among whom the level of appropriate supervision is more stringently defined. Progression from direct supervision to indirect supervision with direct supervision immediately available requires demonstrated clinical and performance competency. The intern must become credentialed to perform tasks, such as histories and physical examination, consults, and bedside procedures, without direct supervision.
Surgical training programs are expected to design a curriculum that addresses and fulfills the 6 ACGME competencies.3,9- 13 As part of the Outcome Project3 by the ACGME, 6 core competencies were formulated to ensure standardization of surgical training in a measurable form. All topics in our 2-year curriculum have an identified competency. Each rotation is designed to define how a competency is achieved and measured. We use the CESI and have a core number of procedures to which the residents are exposed (intubation, chest tubes, Foley catheters, and central venous pressure line placement). The terms competency and competency-based learning have been incorporated into the ACGME language of surgical training, defined as sufficient and capable performances in the domains of patient care, professionalism, medical knowledge, system-based practice, communication and interpersonal skills, and practice-based learning and improvement.14
The general surgery nursing staff and teaching faculty perceived an improvement in the bedside clinical performance of the interns. Nurse managers and teaching faculty completing the survey did not know when the interns completed the boot camp, nor did they know the actual boot camp curriculum or the performance of the individual resident.
Despite our success thus far with the intern boot camp curriculum, the study was limited by several factors. Because the intern boot camp is still in its infancy, one limitation is the small size of our study cohort interns and comparison populations. We plan to continue the curriculum in the coming years, expand and improve on it each year, and reevaluate our efforts. Another limitation of our study lies in the inherent difficulty of determining competence. Our goal in instituting the boot camp was to allow interns to achieve competence more quickly within the new work-hour restrictions and increasing oversight. Evaluating residents in satisfying the core competencies has proved challenging, and the optimal method of evaluation is still unknown.2,9,13 We plan to use standardized prepost testing as another method of evaluating interns’ experience with the boot camp, as well as improved 360°-evaluations that include patients, faculty, residents, nursing staff, and midlevel care providers.
Surgical training has undergone many changes to bring it into the perfect storm of workload restrictions, greater oversight, and increasing competency-based maturation. Programs now have to adopt these new tenets, creating programs that not only comply with new standards but also produce competent surgeons. Time will tell what the optimal solution will be to this new surgical training reality.
Accepted for Publication: October 13, 2012.
Corresponding Author: Orlando C. Kirton, MD, Department of Surgery, University of Connecticut School of Medicine, 263 Farmington Ave, Farmington, CT 06030 (email@example.com).
Published Online: June 12, 2013. doi:10.1001/jamasurg.2013.2350.
Author Contributions:Study concept and design: Krajewski, Filippa, Kirton.
Acquisition of data: Krajewski, Filippa, Singh.
Analysis and interpretation of data: Krajewski, Staff, Kirton.
Drafting of the manuscript: Krajewski, Singh, Kirton.
Critical revision of the manuscript for important intellectual content: Krajewski, Filippa, Staff, Kirton.
Statistical analysis: Krajewski, Staff.
Administrative, technical, and material support: Krajewski, Filippa, Singh.
Study supervision: Kirton.
Conflict of Interest Disclosures: None reported.
Previous Presentations: This paper was presented at the 93rd Annual Meeting of the New England Surgical Society; September 21, 2012; Rockport, Maine; and is published after peer review and revision.