Reichel JL, Peirson RP, Berg D. Teaching and Evaluation of Surgical Skills in DermatologyResults of a Survey. Arch Dermatol. 2004;140(11):1365-1369. doi:10.1001/archderm.140.11.1365
Copyright 2004 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2004
To assess how the surgical skills of residents are taught and evaluated within dermatology residency programs in the United States; to assess which surgical techniques training directors and residents consider important for residents to perform or at least understand by the end of residency training.
A 126-question survey was sent to all 106 of the US dermatology residency programs accredited by the Accreditation Council for Graduate Medical Education. Contact was initially made via e-mail. Surveys were addressed to the program director, surgical training director, and chief resident of each program. A follow-up survey was mailed to nonresponders.
Ninety-five surveys were returned representing 71 (67%) of 106 programs. Eighty-nine percent of programs (n=63) reported having a formal curriculum in dermatologic surgery. Among programs represented, 97% (n=69) taught surgical skills in the procedure room, 84% (n=57) used pigs’ feet, and fewer than 10% (n=6) used human cadavers. Ninety-four percent of programs (n=61) scheduled surgical lectures; two thirds (n=41) formally assigned surgical reading, and over half (n=36) used Web-based lectures to teach skills. To assess training, most programs (86%; n=50) used subjective global evaluation at the end of a surgery rotation. Fewer than 30% (n=15) discussed specific objectives prior to the rotation. Only about 25% of programs (n=17) reported the use of written or oral examinations to assess resident surgery skills. Traditional biopsy and simple surgical procedures were reported as most important to know and perform. Interest by both faculty members and residents in more advanced surgical techniques was more limited and variable. Cosmetic surgery techniques were most likely to be viewed as unimportant.
Most dermatology programs teach surgical skills by traditional apprenticeship methods supplemented by work in pigs’ feet laboratory classes and regularly scheduled lectures. Skill assessment is mainly done through subjective means. Almost all respondents thought that basic biopsy and excisional skills were essential for residents to know and perform. More complex surgical techniques and the use of lasers were considered less important. Cosmetic techniques were those most frequently viewed as unimportant.
With increased demand for oncologic, cosmetic, and laser procedures, the task of teaching and assessing dermatologic surgical skills is now an important part of residency programs.1- 3 The Accreditation Council for Graduate Medical Education (ACGME) mandates that dermatology residents be competent in cutaneous surgery.4 The requirements include a list of procedures that residents must be competent to perform and other procedures that “trainees must gain an understanding of even though they may not personally perform them.”4 In addition, the American Board of Dermatology requires residents to track the number of procedures they perform, assist, and observe each year. Residents should perform a number of excisions, closures, flaps, and laser procedures, assist on grafts, nail surgery, and Mohs micrographic surgery (MMS), and “become familiar with” cosmetic procedures such as hair transplantation, sclerotherapy, liposuction, peels, and soft tissue augmentation. Despite these formal written accreditation mandates, recent reports indicate that the quality and amount of surgical and cosmetic training vary greatly among programs.5
Although recent studies have looked at surgical teaching and evaluation practices in other disciplines, few reports exist in the dermatology literature. In 1989, Anders et al6 surveyed US dermatology programs concerning the use of live animals, animal models, and other models to teach cutaneous surgery. At that time, 48% of programs used pigs’ feet laboratory classes, and fewer than 5% used live or other animal models. More recently, Todd et al5 looked at the breadth of surgical training and use of formal didactic teaching in dermatologic residencies. They found that over 90% of all responding programs offered training in surgical excisions, MMS, laser surgery, and at least 1 cosmetic procedure. Additionally, they reported that 83% of programs had regularly scheduled surgery didactics and that 72% reviewed dermatologic surgery literature. They concluded that surgical training at that time varied widely among residencies and that a trend toward more cosmetic procedures was evident.
To elucidate the current trends in surgical training of dermatology residents, we report here the results of a survey sent to all ACGME-accredited US dermatology residencies. We addressed 3 questions: (1) How do programs teach surgical skills (formal curricula, didactic sessions, “hands-on” training)? (2) How do they assess these skills (written and/or oral examinations, subjective procedure-room evaluations)? and (3) Which skills do program directors and residents consider important for the graduating resident to be able to perform vs which need only to be understood?
An electronic questionnaire was created using the University of Washington’s Catalyst Webtools software (Seattle, Wash) and Web site. An initial contact e-mail with a hyperlink directing the recipient to the questionnaire Web site was sent to 94 US dermatology programs with available e-mail addresses. The 6-page paper version was then sent to those programs that indicated a paper preference and to those for which no contact e-mail address could be located. In total, all 106 of the USACGME-accredited programs identified from the 2001 American Medical Association Fellows and Residents Educational Index Document7 received the survey. A follow-up e-mail and paper survey with a second letter were sent 4 months later to nonrespondents. To compare answers and to maximize return rate, we asked the chief resident, director of dermatologic surgery, and the program director for each institution to complete the questionnaire.
The questionnaire consisted of 19 objective questions detailing the quality and extent of the surgical education of dermatology residents during their training and 21 objective questions regarding assessment of surgical skills. Objective questions were defined as those considered to have only one true answer (for example, “Do you have regularly scheduled surgical lectures?”). Respondents were also asked to rate, subjectively, certain surgical procedures on a 4-point scale as essential, important, nice to know, or unimportant for residents to either understand or actually be able to perform competently by the time of graduation.
Results were tabulated, and descriptive data were analyzed using Microsoft Excel (Redmond, Wash). For questions with individual nonresponses, the nonresponses were not included in the tabulation of final results and were detailed when possible. When questionnaires were returned from more than 1 individual within an institution, results to the objective questions were compared for discrepancies thought to reflect possible question ambiguity, unequal knowledge of the true answer among respondents, or simple errors in filling out the forms. If more than 25% of respondents from 1 institution had a different response, questions were considered ambiguous or too subjective and were discarded. In cases where discrepancies were present but from fewer than 25% of respondents from a single institution, the response from the surgery director was used.
Ninety-five surveys were returned representing 71 (67%) of 106 programs. Forty percent of responses (n=38) were via electronic submission. Responses were received from a broad spectrum of participants, with 26% being from chief residents (n=24); 36% from training program directors (n=34); 11% from division chairs (n=11); and 27% from surgery directors (n=26). One fourth of programs (n=18) submitted responses from more than 1 individual. Eleven programs had responses only from the chief resident, and 60 (85%) had at least 1 faculty member respond.
Most programs (89%; n=63) had formal surgical curriculum (Table 1). Almost all endorsed using the procedure room (97%; n=69) and pigs’ feet laboratories (84%; n=57) as their major hands-on methods of skills teaching. Other hands-on techniques were used minimally; for example, fewer than 20% of programs (n=14) used synthetic models or cadavers. Very few programs used live animals, and almost no programs used virtual reality models.
The responses indicated the widespread use of formal didactic teaching techniques in dermatologic surgery. Ninety-four percent of programs (n=61) conducted regularly scheduled lectures; 72% (n=44) conducted lectures at least once a month; over half of all programs (n=36) used Web-based lectures; and two thirds of the programs (n=41) assigned readings—most assigned chapters and articles.
The following information is based on data from the 60 programs from which at least 1 faculty member responded (Table 2). Eighty-six percent of programs (n=50) used a formal global evaluation to assess residents’ surgical skills. Only 50% (n=26) evaluated a list of specific surgical skills as part of their global evaluation. Few programs used written or oral examinations (33% [n=17] and 23% [n=12], respectively), and almost no programs used computer-based testing. Most faculty members did not assess surgical skills when residents entered their programs. Ninety percent of programs (n=52) used resident-kept procedure lists to evaluate residents.
The following results are based on the answers given by all 95 respondents regardless of position or program. Respondents were asked to rate the importance of understanding and discussing certain procedures without necessarily being able to perform them by the time of program graduation. The respondents were then asked to rate the importance of performing those same procedures by program graduation.
Shave and punch biopsies were the only 2 procedures that 100% of respondents (n=95) considered essential to perform. Procedures considered essential to perform by at least 95% of respondents (n=90) are listed in Table 3. Table 4 lists procedures that at least 75% of respondents (n=71) considered essential or important to perform. All basic surgical procedures made the 75% cutoff. Advanced surgical procedures (MMS, flaps, and grafts) were not included. No cosmetic procedures were included in this group. Of cosmetic procedures, only sclerotherapy, chemical peels, and laser for vascular lesions were considered important or essential to perform by even half (n=47) of the respondents. Table 5 lists procedures rated as unimportant or only nice to perform by more than 50% (n=47) of respondents. Overall, no procedures were rated as unimportant by more than 50% of respondents.
As expected, all procedures rated as essential to perform were also considered essential to comprehend (Table 3, Table 4, and Table 5 ). In addition to the procedures listed in Table 3, undermining and performing wedge and/or incisional biopsies were considered by more than 95% of respondents as essential to comprehend without needing to perform. Flaps, MMS, and grafts fell below the 75% cutoff for essential to perform but were rated above the 75% cutoff for important or essential to comprehend. Of cosmetic procedures, only sclerotherapy, chemical peels, and laser treatment for vascular lesions were considered important or essential to comprehend (but not perform) by at least 75% of respondents.
Seventy-one faculty members and 24 chief residents responded to the survey. The percentages of residents and faculty members who ranked each procedure as either essential or important were calculated separately and then analyzed. Answers correlated well overall. The intergroup (resident vs faculty) variation was less than 5% for almost half of all questions. All faculty and resident respondents ranked simple biopsy techniques, wound management, cryosurgery, and local anesthesia as either essential or important. Table 6 lists procedures for which a disagreement of greater than 20% occurred. This happened for only 2 (5%) of the “to know” questions, and 8 (19%) of the “to do” questions. In general, chief residents rated more cosmetic skills as essential or important than did faculty respondents, and most residents and faculty members ranked cosmetic procedures as less important than noncosmetic procedures.
Traditionally, residents in all surgical specialties learn technical skills through the “apprenticeship model” of assisting attending surgeons during surgical procedures. Intellectual knowledge is commonly taught via didactic lectures, and evaluation of surgical competency is usually done by subjective faculty assessment.8- 10 Reports indicate that this model continues to be the mainstay in the fields of general surgery and obstetrics-gynecology.10,11
Dermatologic surgery is a relatively young procedural subspecialty. With the importance of dermatologic surgery codified by the ACGME and the American Board of Dermatology, our survey found that most dermatology programs (89%; n=63) have a formal surgical curriculum. Almost all programs use the procedure room and surgical lectures as the primary means to teach. Aside from the use of pigs’ feet laboratories, which has increased 40% in the past decade,6 very few programs use other teaching methods such as cadavers, live animals, or synthetic models.
Problems with traditional, apprenticeship-based teaching and evaluation methods have been highlighted recently and include the high cost of training, ethical constraints, variability in skills acquisition among residents, and unknown validity and/or poor reliability of subjective evaluations.10,12,13 Hence, other disciplines are investigating techniques such as bench simulations and the use of physical and virtual reality models.11,14,15 For example, Martin et al,11 Hamdorf and Hall,13 and Reznick et al16 have developed the Objective Structured Assessment of Technical Skill (OSATS) by which general surgery trainees are assessed as they rotate through bench-top simulation stations.11,13,16 The OSATS increased reliability and validity in surgical skills assessment over previous assessment tools.
According to our survey, most dermatology training programs (86%; n=50) formally assess residents’ surgical skills. Since the use of a subjective, end-of-rotation global evaluation form is common in other disciplines, it is no surprise that this is the most common form of evaluation reported by dermatology program respondents (86%; n=51). Despite its questionable reliability and validity,17 this form of evaluation is perpetuated because of its relative ease of use.18 Use of a structured checklist of procedures to assess skills has been shown to increase interrater reliability and validity.19 In dermatology programs, however, according to faculty survey respondents, only half of all programs included a list of specific skills in their global resident evaluations. Less than one third of programs used written or oral examinations to assess surgical skills. Future directions in dermatologic surgery training and evaluation may be the construction and implementation of more standardized, specific, and reliable testing of skills such as the OSATS program.
A second widespread method of evaluating skills in dermatology is the maintenance of quantitative procedure lists. Because lists are often used for granting hospital privileges and credentialing and are required by the American Board of Dermatology, we expected to find a larger number of programs using than we did. Skepticism as to the validity and reliability of lists remains; it has been asserted that the simple presence during or even participation in a certain number of procedures does not automatically make one an expert in those procedures.13
The rapidly widening scope of new oncologic and cosmetic procedures in dermatologic surgery prompted us to ask residents and faculty members which specific skills they considered valuable to have or comprehend. Most of the traditional practical skills mandated by the ACGME showed little variation in perceived importance. Long-established basic dermatology procedures such as shave and punch biopsies, local anesthesia administration, and cryosurgery were nearly universally considered essential or important. Procedures considered essential or important to perform by more than 75% of respondents are also relatively basic and included biopsy techniques, simple excision, electrosurgery, suture techniques, and management of excisional complications.
Most programs viewed the more recent developments in technology and practice as less important than the basics. The procedures most often ranked unimportant or only nice to perform primarily included cosmetic surgery techniques and more complicated surgical techniques such as intravenous conscious sedation and cartilage grafting. In parallel to the wording of the ACGME guidelines, considerably more respondents considered cosmetic procedures “essential or important to comprehend” as opposed to “essential or important to perform.” Nevertheless, only 3 cosmetic procedures (laser treatment for vascular lesions, chemical peels, and sclerotherapy) were rated essential or important to comprehend by more than 75% of respondents (n>71). Although liposuction and hair transplantation (2 of the more complex cosmetic surgeries) are listed by the ACGME as procedures that residents are to “become familiar with,” more than 80% of respondents (n=80) considered these either unimportant or only “nice to comprehend.” Although we may be moving toward practicing more cosmetic procedures, the pervasive attitude in residency programs still seems to be that these procedures are relatively unimportant.
As part of our analysis of the data for a comparative needs assessment, we compared residents’ identified educational priorities with those of the program and dermatologic surgery directors. Most residents and faculty members ranked cosmetic procedures as less important than noncosmetic procedures. Still, chief residents ranked more cosmetic procedures as essential or important. This is not surprising and perhaps indicates that young dermatologists intend to practice more cosmetic and surgical procedures than did the previous generation.
In conclusion, our survey indicates that most programs had a formal dermatologic surgery curriculum. There was broad agreement among programs that basic skills should be taught in dermatology residency. However, wide variation was present among programs as to how residents’ skills should be assessed. The development of a standardized test of surgical skills would likely increase the assurance that all residents are being taught, and are able to perform, the set of skills deemed necessary to graduate. Our survey also showed that newer technologies and procedures in the area of cosmetic surgery were more likely than the fundamental skills to be considered unimportant or to receive a broad range of responses. With the advent of new ACGME procedural fellowships in dermatology, it will be important for leadership in our specialty to better define the skills required to be taught during residency and fellowship.
Correspondence: Jennifer L. Reichel, MD, Division of Dermatology, University of Washington, Box 356524, 1959 NE Pacific, Seattle, WA 98195 (firstname.lastname@example.org).
Accepted for Publication: June 25, 2004.
Funding/Support: This study was supported by the National Institutes of Health, Bethesda, Md, grant T32 AR07019, and funds from the Carl J. Herzog Foundation Inc, Galveston, Tex.
Financial Disclosure: None.