Protected Block Curriculum Enhances Learning During General Surgery Residency Training

Background  Changes in medical education require a rethinking of our training paradigm. We implemented a protected block curriculum for postgraduate year (PGY)–1 and PGY-2 surgery residents.

Hypothesis  A protected block curriculum promotes adult learning consistent with the 6 competencies.

Design  Prospective static-group comparison with pretesting and posttesting.

Setting  Medical College of Wisconsin, Milwaukee.

Participants  Eight university-based surgical residents (curriculum group) and 8 residents who did not participate in the curriculum (control group).

Main Outcome Measures  The curriculum occurs during protected time away from clinical activity. Predefined learning objectives and competencies were identified for PGY-1 and PGY-2 residents. Multiple choice examinations were administered to assess knowledge. The first 3 tests of the year in the PGY-2 curriculum were also given to the PGY-3 and PGY-4 and -5 residents for comparison with curriculum residents. In-training examination scores of control and curriculum residents were compared. Surgical and communication skills were assessed using checklist assessment forms. Curriculum residents evaluated the content and delivery.

Results  Pretest and posttest results demonstrated acquisition of knowledge with improved aggregated mean scores from 57.5% to 71.4% for PGY-1 residents and 58.6% to 72.6% for PGY-2 residents. The average curriculum test results were 76.7% for curriculum residents, 56.9% for control residents, and 57.3% for all residents. The 2-year average in-training scores were 71.2% for curriculum and 60.3% for control residents. Assessments demonstrated improvements in communication and surgical skills.

Conclusions  A protected block curriculum enhanced surgical residents' learning compared with a traditional model. Improvement in medical knowledge was easiest to measure, but performance in other Accreditation Council for Graduate Medical Education competency areas also demonstrated improvement.

Surgical residency training programs face many challenges in producing competent surgeons. The Accreditation Council for Graduate Medical Education (ACGME) requires residents to receive training in 6 competencies, and programs must document the achievement of those competencies with great precision.¹ As these new training guidelines were introduced, new duty-hour restrictions were mandated limiting residents' time on service and in educational activities.² Numerous articles detail how training programs are dealing with the duty-hour restrictions.^3-9 Fewer articles discuss efforts to meet educational expectations in this changing environment.^10-12 Although the struggle to meet the work-hour restrictions and accreditation mandates can be all-consuming, these changes were intended to ensure both quality patient care and competent trainees. Our goal was to design a training curriculum that maximized residents' educational experience consistent with competency-based education within the confines of duty-hour limitations.^13-15

In 2002, our residency training program started a major curriculum change, developing a protected block curriculum (PBC).¹⁵ This curriculum model started with a needs assessment to identify learning objectives, guidance from the Office of Educational Services at the Medical College of Wisconsin, and development of the curriculum around adult learning principles. We established the curriculum using the following assumptions: (1) residents are adult learners;(2) clinical care operations can function without residents' constant presence; (3) specific competency-based objectives for each training year could be identified; (4) learning was best achieved by training-year cohorts; (5) graduated introduction of clinical knowledge and skills could be achieved in a nonclinical situation; and (6) ongoing evaluation of the program and trainees would be used to guide program revision.

Our first class completed 2.5 years of the curriculum. This study highlights key features of the longitudinal curriculum and educational outcomes from the first 2 years, which we believe are linked to our PBC. This study was classified as exempt by the Medical College of Wisconsin institutional review board.

The postgraduate year (PGY)–1 and PGY-2 curriculum group (n = 8) includes 2 nongeneral surgery residents. Their educational outcomes were compared with 8 residents (6 general surgery and 2 nongeneral surgery) in their PGY-3 year who served as the noncurricular or control group. The United States Medical Licensing Examination scores of both groups showed no significant difference between curriculum and noncurriculum residents.

A hallmark of our curricular design is protected time away from clinical responsibilities. Each curriculum week is preceded by a free weekend for resident preparation. For the period described, the PGY-1 week was 5.5 days and the PGY-2 curriculum 4.5 days, including the preceding weekend. During these blocks of time, they take no calls, do not report to their service, and are not allowed vacation, thus assuring 100% attendance and participation.

In the first year of implementation, PGY-1 residents participated in six 1-week curriculum blocks. The curriculum topics were based on educational objectives developed through the needs assessments and curriculum materials from the Association of Program Directors in Surgery and the American College of Surgeons Committee on Education.^16,17 Instruction during these blocks used adult-learning methods including out of class assignments, interactive lectures, case-based discussions and presentations, problem-based learning sessions, skills laboratories, cadaver laboratories, and simulator laboratories. A sample of a PGY-1 curriculum block day is provided (Table 1).

Curriculum topics for PGY-2 were again developed based on local needs assessment data. These topics were informed by available educational materials from textbooks, professional publications, and current literature. The structure and instructional formats for the PGY-2 curriculum were consistent with principles of adult learning and modified by feedback and evaluation data from year 1. For example, faculty and residents recommended that protected PGY-2 curriculum time be reduced and that lectures be eliminated in favor of more participatory learning. Each curriculum block was structured to address a specific organ system(s) and disease state(s). Topics covered included breast disease, trauma, critical care, diverticular disease, colorectal cancer, and vascular disease. Table 2 provides an example of a PGY-2 curriculum day. Surgical skill laboratories were integral to the curriculum and represented a continuum of learning. An example of the progression of surgical skills covered during the PGY-1 and PGY-2 years is provided in Table 3.

All curriculum sessions and topics, along with associated learner assessment tools, were explicitly linked to the 6 ACGME core competencies.¹ Evaluations were designed to assess resident performance and curriculum design.

Multiple assessment tools aligned with the instructional objectives and methods were used to evaluate the curriculum. Resident performance was assessed using multiple-choice question examinations, American Board of Surgery In-training Examination (ABSITE) performance, Objective Structured Assessment of Technical Skill (OSATS), performance checklists, and portfolio entries. Residents evaluated the teaching faculty and curriculum structure using structured curriculum evaluation forms.

Statistical analysis was performed on the results of the multiple-choice question examinations, the ABSITE examinations, and the OSATS. Independent t tests were used to determine significance using SPSS for Windows Version 15.0 (SPSS Inc, Chicago, Illinois).

Multiple-choice question examinations were developed and reviewed by experienced faculty based on identified learning objectives. These were administered to the residents prior to and after each curriculum block to provide formative assessment of knowledge gained. Multiple-choice examinations were analyzed for item difficulty, discrimination, and overall test reliability. Pretests and posttests for the PGY-1 and PGY-2 resident years were compared by pooling the mean pretest and posttest results for each curricular block by year. In addition, an examination covering the material from the first 3 weeks of the PGY-2 curriculum was given to the entire residency (PGY-1 to -5). Results of this examination were compared between the PGY-2 (n = 8) curriculum residents and the PGY-3 (n = 8) noncurriculum residents as controls as well as the entire group of noncurriculum residents (PGY-1 to -5; n = 32). Competencies assessed by multiple-choice question examinations included medical knowledge, patient care, professionalism, and systems-based practice.

Results of the first 2 years of ABSITE scores of the PGY-2 general surgery residents (n = 6) were compared with those of the PGY-3 noncurriculum general surgery residents (n = 6). Comparing the ABSITE scores over the same training period between these 2 groups was an attempt to identify medical knowledge gains using a standardized national examination.

Objective surgical skills assessment was performed for surgical skills at the PGY-1 and PGY-2 levels. These skills were selected based on learning objectives identified by our needs assessment and curriculum committee. Basic skills, including tying knots and performing simple interrupted, running, and mattress sutures were videotaped and assessed in an OSATS manner using an item analysis and global assessment form during the PGY-1 and -2 years. Competency in performing a bowel anastomosis using pig colon and a vascular anastomosis using a polytetrafluroethylene graft was assessed by trained faculty using a similarly structured form based on a previously validated assessment form.^18,19 Results of the basic suture skills evaluations from the PGY-1 year were compared with PGY-2 results and PGY-3 resident results using independent t tests. The overall competence of the PGY-2 class performing colonic and vascular anastomoses was analyzed based on criterion achievement as the percentage of residents who were competent after each skills session.

Communication skills and professionalism were emphasized across the curriculum with selected sessions. For example, the PGY-1 residents were given instruction in presentation skills and then videotaped giving a 10-minute presentation on a medical topic. Communication, presentation skills, and professionalism were directly assessed using a checklist competency form completed by 2 trained faculty observers (T.W. and P.R.). The videotapes were reviewed and evaluated by a single faculty evaluator (T.W.), and feedback was provided to each resident while reviewing highlights of the presentations. A similar presentation videotape debriefing exercise was repeated during the PGY-2 curriculum. Performance on the videotaped presentation as a PGY-2 resident was compared with their PGY-1 performance. These results were analyzed as the percentage who achieved successful completion of goals and using qualitative analysis of comments from the assessment forms.

The Surgical Learning and Instructional Portfolio (SLIP) is a learning portfolio that is used to improve the depth of knowledge of residents as they examine a case and describe specific learning points associated with their management of the case.²⁰ Residents are to complete their SLIP entry prior to participation in the curriculum block. Competencies assessed using the SLIP include medical knowledge, practice-based learning, systems-based practice, communication skills, and professionalism. Data analysis was limited to completion of the SLIP.

Curriculum design and content was evaluated using several methods and data sources. Likert scale evaluation forms were distributed to the residents after each session and at the end of the curriculum blocks. The scaled ratings were analyzed to yield a mean distribution of positive vs negative scores and qualitative analysis to itemize themes and frequency within the comments. Overall quantitative results across sessions by year are reported with detailed qualitative analysis of narrative comments.

Pretest and posttest results showed acquisition of knowledge with statistically significant improvement in mean (SD) pooled test scores from 57.5% (3.9%) to 71.4% (3.0%) (P < .001) for the PGY-1 residents and 58.6% (6.2%) to 72.6% (6.2%) for PGY-2 residents (P < .001), and are presented in Table 4. Mean (SD) curriculum test results were 76.7% (6.7%) for curriculum residents, 56.9% (7.0%) for noncurriculum control residents, and 57.3% (9.4%) for all residents, with a statistically significant difference between the curriculum and noncurriculum control residents (P < .001), and are presented in Table 5.

General surgery PGY-2 residents' ABSITE scores were combined for years 1 and 2 (n = 6) and compared with the scores for our noncurriculum PGY-3 general surgery residents (n = 6) taken during their second year. The mean (SD) 2-year average in-training test scores were 71.2% (7.5%) for curriculum residents and 60.3% (9.0%) for control residents, with the difference in mean scores being statistically significant (P = .004).

Surgical skills demonstrated improvement over the course of the PGY-1 and -2 years. Analysis of the time, item score, and global score (range, 1-5; 1, novice; 5, expert) all showed improvement. The PGY-1 global scores improved from a mean of 1.25 to a mean of 3.14 (P < .001). The mean global score for the curriculum PGY-2 residents was 3.5 compared with a mean score of 2.9 for the noncurriculum PGY-3 residents (P = .04). These assessments were done at the same time in the residents' training. At the end of the PGY-2 year, all (n = 8) residents met the minimum competency standards, performing vascular and bowel anastomoses based on checklist criteria and global assessment.

Curriculum residents' communication skills demonstrated improvement between the PGY-1 and PGY-2 years. Improvements were most notable in the ability of the residents to engage the audience with appropriate eye contact and body language and to use appropriately formatted PowerPoint (Microsoft, Redmond, Washington) slides. Specific faculty comments included, “The structure of the presentations have become much more focused and easy to follow” and, the presenter has “greatly improved his ability to engage the audience.” Specific resident evaluation comments included, “The presentations, although embarrassing, were very helpful for self analysis.”

All (n = 8) curriculum residents successfully completed this assignment. The importance of this activity as a component of lifelong learning was emphasized during the curriculum.

Longitudinal and session evaluations demonstrated high satisfaction with the quality of the curriculum design, objectives, teaching, and attainment of the targeted educational objectives. Table 6 shows evaluation data from the session evaluations for the most recent PGY-1 and PGY-2 curriculum. Specific comments included, “The curriculum is a huge positive about the program. It has provided me with a real sense of team with my colleagues.”

A major effort is under way to define a core surgical curriculum.²¹ This goal was set as part of the recommendations from the blue ribbon committee report on surgical education.²² A distinction should be made between a curriculum and an educational model. A core curriculum should be a constant for each discipline.^8,9,22-25 In addition to a core curriculum, a systematic, evidence-based education plan for how to provide this instruction in compliance with the ACGME Outcome Project and the 80-hour work week is necessary for the curriculum to succeed.

Our core curriculum was systematically developed based on current recommendations, with needs assessment data by our faculty and residents. Once core curriculum elements were determined, our PBC instructional model was developed. In designing our model, we borrowed from evidence suggesting that protected time away from clinical responsibilities would positively affect education and learning. Dedicated learning time on a given topic has been used to improve surgery in-training examination scores.²⁶ A dedicated 13-hour curriculum to enhance resident's teaching skills has been successful.²⁷ These models, like ours, enable aspects of a traditional surgical residency to be retained for most of the residency training program. A more complete restructuring of a surgical residency program has been provided by Schneider and colleagues.¹⁰ They conclude that blended apprenticeships, small teams, and a night float system can be integrated into a new model without reducing resident educational experience.

We chose a curriculum model that had protected or dedicated time for the adult learner. This model has distinct benefits in preparing residents, as adult learners and emerging professionals, to identify and develop learning plans and strategies that address identified performance needs. Consistent with the literature on adult learning, instructional methods blending independent and self-directed learning with interactive discussions and simulations were rated as highly successful by residents.^28-30 These principles are also consistent with the ACGME's Practice-based Learning and Improvement competency, and the emerging requirements for maintenance of certification. Additional principles include appropriate time for integrating new ideas and thoughts, ability to practice without distractions, and consistent attendance for small group discussion. Residents in our PBC emphasize the value of time away from clinical duties in facilitating learning.¹⁵ The PBC also produces a sense of collegiality among the group and ensures that the learner is rested and optimally receptive to be challenged to learn and apply new concepts and skills.^31,32

Objective data concerning acquisition of new knowledge is supported by test scores and attainment of surgical skills. We constructed a control group of residents who were matched for level of training but not exposed to the PBC. This noncurriculum resident control group participated in our traditional educational program consisting of weekly departmental and divisional educational meetings. Comparison revealed improved performance by curriculum residents compared with noncurriculum residents on internal tests and the in-training examination. These tests relate primarily to patient care and medical knowledge competencies. As baseline performance on standardized examinations of curriculum and control residents was similar, we attribute most performance differences to the new curriculum format. The ABSITE performance differences by year must be interpreted with caution, as scores across years are not necessarily equated. However, our residents' mean percentage of correct responses has been consistent with those of our control group over previous years; thus, the increase in the performance of our residents is heartening.

Similar results were apparent in communication skills, professionalism, practice-based learning, and systems-based practice. We demonstrated improved outcomes, but recognize that more objective comparisons are needed. Improvement in presentation skills was transparent when reviewing residents' PGY-1 presentations compared with their PGY-2 presentations. The importance of using an individual portfolio to demonstrate lifelong learning was a component of the curriculum.

The longitudinal evaluations clearly indicate the value of the curriculum from the learners' perspective. Faculty assessments of curriculum residents' skills in the operating room and the patient care setting were uniformly positive. These assessments reveal a trend toward stronger performance compared with similar-level control residents, but sample sizes and statistical analysis options are limited at this point. In spite of these limitations, the overall positive evaluations by residents and faculty are hard to ignore.

The Centers for Medicare and Medicaid Services' (CMS) request that residents not be taken away from clinical service for 24 hours straight presents a new challenge.³³ We sought to incorporate patient care sessions into each day of our curriculum to address this requirement; however, this stipulation is counterproductive to the educational goals and methods of our training program. We suggest that this CMS directive places educational goals in direct conflict with service activities and hope this request will be reconsidered. Our goal must be educational and service activities that comply with duty hours, include all ACGME competencies, enhance patient safety, and promote quality medical care.

We acknowledge limitations of this analysis. This program is from a single academic institution and spans 2 years of a 5-year program. Though we are able to accomplish this program within the 80-hour work week, some programs may find it difficult, and other educational models can increase the priority of educational activities. The PBC puts a strain on faculty and the residents who are not participating in the sessions. Faculty and resident buy-in is necessary to effect this type of dramatic culture change, although any major change in our educational system will face the same challenge. Faculty buy-in is necessary at 2 levels. The first is developing educational material and teaching time. The second is developing strategies for patient care as residents are rotated into their protected time. From the residents' perspective, there is additional work when a class of trainees is away at their curriculum week.³⁴

Finally, identifying curriculum content requires a significant amount of work and planning. The current national activity to define a core curriculum will help define core content, establishing consistency across programs.²¹ The remaining issue is the instructional model used to engage surgical trainees in learning the components of the core curriculum. Combining the core curriculum with systematically designed and evidence-based instructional models will allow us to produce and document the competence of our surgeons for the future.

We are satisfied that the gains in performance of the curricular residents support the continued use of PBC in our training program. Success with our protected block instruction model for surgery core curriculum has spurred us to develop a complementary core curriculum and an instructional model for our PGY-3 through -5 residents.

Correspondence: Travis P. Webb, MD, 9200 W Wisconsin Ave, Milwaukee, WI 53226 (trwebb@mcw.edu).

Accepted for Publication: January 24, 2008.

Author Contributions:Study concept and design: Weigelt, Anderson, and Simpson. Acquisition of data: Webb, Redlich, and Simpson. Analysis and interpretation of data: Webb, Weigelt, Redlich, and Brasel. Drafting of the manuscript: Webb, Weigelt, and Simpson. Critical revision of the manuscript for important intellectual content: Webb, Weigelt, Redlich, Anderson, Brasel, and Simpson. Statistical analysis: Simpson. Administrative, technical, and material support: Webb, Weigelt, Redlich, Brasel, and Simpson. Study supervision: Webb, Weigelt, and Brasel.

Financial Disclosure: None reported.

Previous Presentation: Presented at the 2007 Western Surgical Association meeting; November 7, 2007; Colorado Springs, Colorado.

Additional Contributions: We would like to acknowledge and thank all of the residents at the Medical College of Wisconsin, those participating in the curriculum and those who did not have the opportunity to do so. We also acknowledge Robert Treat, PhD, from the Office of Educational Services for data management and statistical analysis.