Yaghoubian A, Galante J, Kaji A, Reeves M, Melcher M, Salim A, Dolich M, de Virgilio C. General Surgery Resident Remediation and AttritionA Multi-institutional Study. Arch Surg. 2012;147(9):829-833. doi:10.1001/archsurg.2012.1676
Author Affiliations: Departments of Surgery (Drs Yaghoubian and de Virgilio) and Emergency Medicine (Dr Kaji), and Los Angeles Biomedical Institute (Drs Kaji and de Virgilio), Harbor-UCLA Medical Center; Department of Surgery, University of California Davis Medical Center, Davis (Dr Galante); Department of Surgery, Loma Linda University, Loma Linda (Dr Reeves); Department of Surgery, Stanford University, Stanford (Dr Melcher); Department of Surgery, Cedars Sinai Medical Center, Los Angeles (Dr Salim); and Department of Surgery, University of California, Irvine Medical Center, Irvine (Dr Dolich), California.
Objective To determine the rates and predictors of remediation and attrition among general surgery residents.
Design, Setting, and Participants Eleven-year retrospective analysis of 348 categorical general surgery residents at 6 West Coast programs.
Main Outcome Measures Rates and predictors of remediation and attrition.
Results Three hundred forty-eight categorical general surgery residents were included. One hundred seven residents (31%) required remediation, of which 27 were remediated more than once. Fifty-five residents (15.8%) left their programs, although only 2 were owing to failed remediation. Remediation was not a predictor of attrition (20% attrition for those remediated vs 15% who were not [P = .40]). Remediation was most frequently initiated owing to a deficiency in medical knowledge (74%). Remediation consisted of monthly meetings with faculty (79%), reading assignments (72%), required conferences (27%), therapy (12%), and repeating a clinical year (6.5%). On univariate analysis, predictors of remediation included receiving honors in the third-year surgery clerkship, United States Medical Licensing Examination (USMLE) step 1 and/or step 2, and American Board of Surgery In-Training Examination scores at postgraduate years 1 through 4. On multivariable regression analysis, remediation was associated with receiving honors in surgery (odds ratio, 1.9; P = .01) and USMLE step 1 score (odds ratio, 0.9; P = .02). On univariate analysis, the only predictor of attrition was the American Board of Surgery In-Training Examination score at the postgraduate year 3 level (P = .04).
Conclusions Almost one third of categorical general surgery residents required remediation during residency, which was most often owing to medical knowledge deficits. Lower USMLE step 1 scores were predictors of the need for remediation. Most remediated residents successfully completed the program. Given the high rates of remediation and the increased educational burden on clinical faculty, medical schools need to focus on better preparing students to enter surgical residency.
In 1999, the Accreditation Council for Graduate Medical Education (ACGME) residency review and institutional review committees announced a new model to measure resident performance. Six general competencies were endorsed including patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and system-based practice. The impetus behind this shift was to create an effective means of educating and evaluating physicians, with the ultimate goal of creating well-trained, educated, ethical, and compassionate physicians.
To our knowledge, to date, there is little data available regarding how successful surgical residency programs have been at achieving these competencies. With the advent of an 80-hour work week restriction and now the 16-hour work limit for interns, there is heightened concern among surgical educators that future surgical residents will be inadequately trained. One measure of the adequacy of the residents' education is whether they require any form of remediation during residency. Thus, the purposes of our study were to determine the frequency of resident remediation (ie, which of the 6 ACGME competencies most commonly needed remediation) and to identify factors predictive of the need for remediation. This may provide insight into how to more effectively modify the surgical curriculum in this new era of limited hours. Another purpose of the study was to determine the rate of attrition by surgical residents.
The study was approved by the human subjects committee of the Los Angeles Biomedical Research Institute at Harbor-University of California, Los Angeles Medical Center. Program directors from 6 residency programs in the Western United States were contacted and surveyed. Data were collected by program directors and/or program coordinators without using any resident identifiers. Variables for graduates from 1999 to 2010 were collected including sex; medical education (US vs non-US medical school); United States Medical Licensing Examination (USMLE) steps 1, 2, and 3 scores; and the American Board of Surgery In-Training Examination (ABSITE) scores (reported as national percentiles) for each postgraduate year (PGY). Remediation was defined as a specific intervention that was in addition to standard resident requirements and that was initiated by the program director owing to an identified deficiency in 1 or more of the 6 ACGME competencies (patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and/or system-based practice). A deficiency in an ACGME competency was identified based on a review of each resident's file, including formal evaluations, biannual reviews, and/or any adverse reports or letters. For residents who required remediation, the program director was asked to identify which competency was deficient. The PGY level at which remediation was initiated, the format/type of remediation, and whether the resident required repeat remediation was recorded. Resident attrition was also collected as was whether the resident left the program voluntarily owing to a career change or involuntarily owing to performance issues.
Resident data were collected in an Excel database (Microsoft Corporation) and translated into a native SAS format using DBMS/Copy (Dataflux Corporation). Analyses were conducted using SAS version 9.1 (SAS Institute). Descriptive statistics were calculated for all variables. When appropriate, numerical variables were compared using the nonparametric Wilcoxon rank sum test and are reported as medians with interquartile ranges. Categorical or nominal variables were compared using the χ2 test or Fisher exact test, as appropriate. P < .05 was considered significant. Univariate analysis was performed to determine factors predictive of the need for remediation and predictive of attrition. Factors found to be significant at a level less than .05 were entered into a multivariable analysis.
There were 348 categorical general surgery residents included in the study. Of these, 220 were men (63.2%) and 128 were women (36.8%). United States Medical Licensing Examination scores, ABSITE scores, and third-year medical school surgery clerkship performance are displayed in Table 1. Overall, 107 residents (31%) required remediation. The attrition rate was 15.8% (55 residents).
One hundred seven residents (31%) required remediation. The remediation was initiated at the PGY 1 level in 27 residents (25%), 37 (35%) at the PGY 2 level, 22 (21%) at the PGY 3 level, 17 (16%) at the PGY 4 level, and 4 (4%) at the PGY 5 level. In addition, 27 residents required remediation more than once. Remediation was owing to deficiencies in the following ACGME competencies: medical knowledge in 80 residents (74%), interpersonal and communication skills in 26 (24%), patient care in 24 (22%), professionalism in 23 (18%), system-based practice in 15 (14%), and practice-based learning in 9 (8%). Residents were remediated by monthly meetings with faculty (79%), specific reading assignments (72%), required attendance at review courses and/or conferences (27%), and evaluation by a therapist/psychologist/psychiatrist (12%). Seven of the residents who were remediated were required to repeat a year. On univariate analysis, predictors of needing remediation included receiving honors in the third-year surgery clerkship, USMLE step 1 and/or step 2, and ABSITE scores at PGYs 1 through 4 (Table 2). On multivariable regression analysis, factors predictive of remediation included receiving honors in surgery (odds ratio, 1.9; 95% CI, 1.13-3.00; P = .01) and USMLE step 1 score (odds ratio, 0.900; 95% CI, 0.985-0.998; P = .02).
Fifty-five residents (15.8%) left their surgical programs. In 53 of these instances, the attrition was voluntary. Only 2 residents (1.1%) were forced to leave their programs owing to failed remediation. On univariate analysis, the only predictor of attrition was the ABSITE score at the PGY 3 level (Table 3). The need for remediation was not a predictor of attrition (20% attrition for those who required remediation vs 15% for those who did not [P = .40]).
Our 11-year study of 348 categorical general surgery residents from 6 academic general surgery residency programs in California reports that 31% of residents required remediation owing to a deficiency in 1 of the 6 ACGME competencies at some point during residency. Remediation was by far most frequently initiated owing to a deficiency in medical knowledge competency (74% of those who required remediation). On multivariable analysis, the USMLE step 1 score was predictive of remediation. Although counterintuitive, having received honors in the third-year surgery clerkship was also predictive of the need for remediation.
The high rate of remediation observed in our study is concerning on several fronts. Resident remediation places additional strain on the program director and entire clinical faculty. As pointed out by Williams et al,1 problem residents may also complicate patient care, as they may increase the workload of other health care providers in the form of duplication of care and an increase in communication demands. They also increase the need for faculty supervision. In the 80-hour resident work week era, as reported by Coverdill and colleagues,2 teaching faculty have increased workloads, higher stress levels, and less satisfaction with work. The high remediation level noted is also surprising given that the resident cohort was a relatively high-achieving group as judged by the mean USMLE step 1 (229) and step 2 (230) scores, both of which were greater than the US national average. Thus, one would not have anticipated such a significant need for remediation.
The high rate of remediation reported in our study begs the question of whether we are falling short in the education of surgical residents. One thought is that medical students start residency underprepared for the rigors of surgical residency. The final year of medical school is often liberally structured, allowing students considerable leeway in choosing electives. Numerous surgical educators have advocated a restructuring of the final year of medical school. Naylor and colleagues3 instituted an integrated cognitive and proficiency-based skills curriculum based on American College of Surgeons Graduate Medical Education Committee competencies. The course included cadaver dissections, didactic sessions, team training, and training in clinical and technical skills. Similarly Esterl and colleagues4 instituted a 4-week senior medical student boot camp. We believe that more needs to be done in the final year of medical school to better prepare future general surgeons and that we as surgeons need to take the initiative.
Another potential explanation is that we as surgical educators are inadequately training our residents, a problem that is now possibly heightened by the 80-hour work week limit. To address this issue, national initiatives are underway to reform surgical education. Representatives from the American Surgical Association, the American College of Surgeons, the American Board of Surgery, the Residency Review Committee for Surgery, the Association of Program Directors in Surgery, and the Association for Surgical Education created a national consortium called the Surgical Council on Resident Education.5,6 It is still too early to determine what effect these efforts will have.
Although the duty-hour limits have reduced resident work hours in the hospital, the 80-hour work week limit may have resulted in other, less desirable effects. General surgery residency now requires more efficiency and a higher-functioning resident to complete more work within a shorter timeframe. This in turn may give the faculty a sense that the resident is not adequately performing. Additionally, although the work hour reduction has resulted in an improvement in the resident lifestyle, it may not have translated into more time for self-study at home.7
Several studies have evaluated remediation in surgical residency. Williams and colleagues1 noted in a single-institution study a 22% rate of serious performance problems among surgery residents. Although they did not categorize the problems by ACGME competency, they reported that the problems fell into 3 broad categories of relations with health care workers, including interpersonal conflict, insufficient knowledge, and failure to communicate effectively. A sobering finding of the study was that virtually all of the problems persisted throughout residency, and all problems appeared to be refractory to remediation attempts. In addition, 41% of the problem residents failed to achieve board certification compared with 100% board certification for residents who did not have problems. Bergen et al8 also identified in a single-institution study that 20.8% of surgical residents were high-risk or problem residents. Most were found to have deficiencies in interpersonal behavior, followed by cognitive deficiencies. Approximately one fifth of those problem residents withdrew from the program, and most did so involuntarily. We similarly found that the most common ACGME competency deficiency was in medical knowledge, followed by interpersonal and communication skills as well as inadequate patient care. Other studies have focused specifically on remediation of low ABSITE scores.
There is no uniformity on the best process for surgical resident remediation. In a national survey of surgery programs, Torbeck and Canal9 found that only 52% to 75% of programs had a specific policy for ACGME competency remediation. In about half of the programs, the remediation plan was developed by the program director alone. In nearly 75%, the program director alone was responsible for supervising and monitoring the program. For medical knowledge deficits, a study program was most often instituted, as in our study. For both patient care and professionalism as well as interpersonal communication skills deficiencies, most programs simply recommended an increase in direct resident observation. For system-based practice and practice-based learning and improvement, tutorial meetings were the most frequent remediation method. Hauer et al10 proposed a 4-step model for remediation applicable to both medical students and residents across all specialties. The model begins with competence assessment using validated tools, followed by diagnosis of deficiencies and development of individualized learning plans; instruction/remediation with deliberate practice, feedback, and reflection; and finally focused reassessment and certification of competence. Other medical specialties have created a national task force to address remediation.11 Given the significant need for remediation in our study and the variability in remediation methods, we feel that national, surgery-specific remediation guidelines should be developed.
Our study found a resident attrition rate of 15.8%, with most being voluntary. Remediation was not a predictor of attrition. Attrition within general surgery has been well studied and is an ongoing concern. A 20-year retrospective study at Yale University found that 30 of 99 surgery residents (30%) failed to complete the program. Of these, 21 (70%) withdrew, 5 (17%) transferred, and 4 (13%) were dismissed. Attrition occurred before entering the third clinical year in 23 of 30 (77%). The attrition rate was 40% (12 of 30) since the 2000 academic year.12 Similarly, a national study was performed by Yeo and colleagues13 of a survey performed of residents taking the ABSITE from 2007 to 2008. They found an overall attrition rate of 19.5%, highest in the PGY 1 and 2 levels. Women were no more likely to leave programs than men (2.1% vs 1.9%).13 Other researchers have demonstrated that the 80-hour work week has not resulted in a reduction in surgical resident attrition.14 Attrition has been attributed to several causes, including the desire for a less-stressful environment; marital, family, and personal issues; economic pressures necessitating rapid entry into a higher-wage workforce; health-related issues; and the perception that a resident is incompetent accompanied by the belief that firing is imminent. Interestingly, this latter cause may apply to our residents, as the only predictor of resident attrition in our study was a low ABSITE score at the PGY 3 level. Bergen et al8 found that deterioration of monthly evaluations, a counseling event, or a letter of complaint trigger close scrutiny and may predict residents at risk for attrition.
There are several limitations to this study. This was a retrospective study, and we were not able to fully evaluate the reasons why residents left surgical residency and the specific circumstances. Furthermore, we did not separate the attrition rate by year and consequently cannot determine whether the attrition rate has improved following the institution of the 80-hour work week. We also did not collect information on the American Board of Surgery board pass rates. Ultimately, the success of a program is measured by residents' success of the American Board of Surgery certifying and qualifying examinations. It is unclear whether remediated residents have a higher or lower board pass rate.
In conclusion, we identified a relatively high remediation rate among residents of 6 academic surgical residencies in California. A lower USMLE 1 score was predictive of the need for remediation. Most remediated residents successfully completed the surgical residency program, and remediation was not a predictor of attrition. The high rate of remediation should give surgical educators pause as we should closely examine the potential sources of these deficiencies. Remediation places an increased educational burden on clinical faculty. Surgical societies should take the initiative to encourage the restructuring of medical school education, such that future surgeons are better prepared to enter surgical residency. At the same time, residency programs need to determine whether current educational methods are adequate to prepare future surgeons.
Correspondence: Christian de Virgilio, MD, Harbor-UCLA Medical Center, 1000 W Carson St, Box 25, Bldg 1-E, Torrance, CA 90509 (email@example.com).
Accepted for Publication: April 30, 2012.
Author Contributions:Study concept and design: Yaghoubian, Galante, and de Virgilio. Acquisition of data: Galante, Reeves, Melcher, Salim, Dolich, and de Virgilio. Analysis and interpretation of data: Galante, Kaji, Reeves, and de Virgilio. Drafting of the manuscript: Yaghoubian and de Virgilio. Critical revision of the manuscript for important intellectual content: Yaghoubian, Galante, Kaji, Reeves, Melcher, Salim, Dolich, and de Virgilio. Statistical analysis: Kaji. Administrative, technical, and material support: Reeves, Salim, and Dolich. Study supervision: Yaghoubian and de Virgilio.
Financial Disclosure: None reported.
Previous Presentation: This paper was presented at the 83rd Annual Meeting of the Pacific Coast Surgical Association; February 20, 2012; Napa Valley, California, and is published after peer review and revision.