Figure. Consolidated Standards of Reporting Trials diagram. SP indicates standardized patient.
Jain N, Anderson MJ, Patel P, et al. Melanoma simulation model: promoting opportunistic screening and patient counseling (published online March 20,2013). JAMA Dermatol.. doi:10.1001/jamadermatol.2013.2882.
eFigure. Method of Creating Pigmented Lesions
eAppendix. Threshold Rules of Pigmented Lesion Management
eTable. Attitudes About Opportunistic Surveillance
Jain N, Anderson MJ, Patel P, Blatt H, Davis L, Bierman J, McGaghie W, Brucker JB, Martini M, Robinson JK. Melanoma Simulation ModelPromoting Opportunistic Screening and Patient Counseling. JAMA Dermatol. 2013;149(6):710-716. doi:10.1001/jamadermatol.2013.2882
Author Affiliations: Departments of Dermatology (Mss Jain and Patel, Messrs Anderson and Blatt, and Drs Martini and Robinson) and Internal Medicine (Dr Bierman), Center for Simulation Technology and Immersive Learning (Ms Davis), Lurie Cancer Center (Drs Martini and Robinson), Center for Education in Medicine (Dr McGaghie), and Galter Health Sciences Library (Mr Brucker), Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Importance Lack of training hampers melanoma recognition by physicians.
Objective To evaluate a melanoma simulation model to teach visual assessment and counseling skills.
Design and Setting Simulation model study in an academic research setting.
Participants A convenience sample of third-year medical students was randomly assigned to receive the intervention before or after a standardized patient.
Intervention During the primary care clerkship, medical students participated in melanoma skills training using 2 simulation models replicating melanomas and abnormal or benign nevi. Scoring threshold rules for visual assessment and management of pigmented lesions and videos of patient counseling were provided.
Main Outcome Measures Identifying a melanoma moulage and counseling the standardized patient. Secondary measures were preintervention and 2-week postintervention knowledge, attitudes about and confidence in their ability to perform opportunistic surveillance and counseling, as well as identification on the model of clinically suspicious pigmented lesions, lesions needing a biopsy, and lesions to be monitored for change.
Results Among 74 students, confidence in their ability to perform opportunistic surveillance improved significantly after skills training (P < .05, χ2 test). Monitoring clinically suspicious lesions for change decreased from 16% (12 of 74) to 3% (2 of 74) and performing a biopsy increased from 80% (59 of 74) to 96% (71 of 74), monitoring benign lesions for change decreased from 43% (32 of 74) to 3% (2 of 74), and biopsying melanoma in situ increased from 10% (7 of 74) to 26% (20 of 74) (P < .05 for all, χ2 test). Detection of the melanoma moulage on the standardized patient occurred more often by trained students (P < .05, χ2 test).
Conclusion and Relevance A 1-hour melanoma simulation education and skills training experience improved performance of opportunistic surveillance, management, and patient counseling by third-year medical students.
Trial Registration clinicaltrials.gov Identifier: NCT01191294
In 2012, it was estimated that 76 250 individuals would develop invasive melanoma and another 53 390 would develop melanoma in situ in the United States.1 While melanoma is treatable when limited to the skin, it was estimated that 9180 deaths would occur in 2012. Patients seen with melanoma in situ (stage 0) have a 99% 5-year survival rate when treated with excision, while those seen with ulcerated lesions and a tumor thickness of 2.01 to 4.00 mm (stage IIB) have a 65% 5-year survival rate. The sizable difference in survival illustrates the need for early detection of melanoma.
Approximately 587 million physician office visits occur annually with primary care physicians, who have an unparalleled opportunity to screen for melanoma during physical examinations.2 Most patients with melanoma (63%) have extensive contact with their primary care physicians in the year before their diagnosis, and 16% of patients report that their melanoma was discovered by physicians.3 Ideally, screening may occur opportunistically during the physical examination. For example, the back, which is a common location for melanoma in men, may be viewed when listening to airflow in the lungs.4 Significant barriers, including lack of skills training for visual diagnosis when in medical school and infrequent clinical experience with melanoma, prevent physicians from performing opportunistic surveillance.5 Even in high-risk groups, such as men 60 years or older, only 1 in 33 000 pigmented lesions is malignant; therefore, visually distinguishing melanoma from benign lesions can be difficult for physicians, who can be expected to diagnose 1 melanoma for every 400 patients examined.6,7 In addition, the training required for visual diagnosis is not uniformly provided during medical education.8
Rates of opportunistic surveillance for melanoma during physical examinations of at-risk populations may increase through focused physician education with visual assessment of pigmented lesions and clinical experience. Opportunistic screening, which is defined as physician-performed visual inspection of parts of the body that are exposed during a customary physical examination, is warranted for at-risk patients. The melanoma skills training simulation model gave medical students the opportunity to interactively learn and apply criteria to visually assess for melanoma. The effectiveness of the educational experience in influencing the student's ability to implement the skill and provide counseling was evaluated through a standardized patient (SP) encounter.
The previously described method of creating silicone rubber skins9 was modified by elevating portions of the lesions, while maintaining the distinctive characteristics visible with magnification (original magnification ×4) (eFigure). Each simulation model presented the following lesions: model 1 (2 melanomas, 1 melanoma in situ, 4 benign nevi, and 1 seborrheic keratosis) and model 2 (3 melanomas, 1 melanoma in situ, and 5 benign nevi). Cancerous and noncancerous lesions were randomly distributed over the back to avoid any recognizable pattern in the location of the lesions. The lesions reproduced on the back models were obtained from the clinical archival images of 2 of us (M.M. and J.K.R.), and all diagnoses were confirmed by pathological examination. The archival images were modified to the correct scale using available software (AutoCAD 2011; Autodesk, Inc) in preparation for replication. The back was chosen because it is commonly encountered by the physician during a focused physical examination when listening to breath sounds and because 50% of melanomas in men and 34% in women occur on the back.4
Each model was independently reviewed by board-certified dermatologists (n = 25) during development. Participants were asked to indicate which lesions they believed were clinically suspicious of being a melanoma, which lesions they would biopsy, and which lesions they would monitor over time. Model 1 lesions failing to achieve 99% interrater agreement were replaced in the next iteration; however, model 2 was created with the intention of having a few lesions that were diagnostically challenging.
In the visual assessment, the border, color, and diameter of a pigmented lesion are each assessed and scored as 1, 2, or 3 (Table 1, left column). For border, 1 is a smooth, regular border; 2 is cannot decide, need to watch; and 3 is a jagged, irregular border. For color, 1 is 1 or 2 colors without blending, uniformly distributed color; 2 is cannot decide, need to watch; and 3 is various colors with blending and nonuniform distribution of color. For diameter, 1 is 4 mm or less; 2 is 5 mm, need to watch; and 3 is 6 mm or more.10
Evaluation of pigmented lesions has evolved from assessing change over time to include guidance about the lesion management decision.11 The threshold rules of lesion management were developed in the first year (July 1, 2010, through December 30, 2010) (eAppendix Because recognition of change in nevi requires comparison of lesions during 6 months to 1 year,11 the threshold rules allowed the physician to examine the lesion twice in 6 months to assess possible change in features (Table 1, right column). In addition, the recommendation to return for follow-up examination in 3 months was based on the common practice of dermatologists to monitor clinically suspicious nevi for 3 months rather than immediately performing a biopsy. Patients with stage IA or IB melanoma are routinely examined 2 to 4 times annually for several years after the initial treatment.12- 14
The malignant cumulative score of 9 for the border, color, and diameter was based on previously reported sensitivity and specificity of diagnosis of melanoma as 65.5% and 81.0%, respectively, if 3 criteria of the ABCD (border irregularity, color variation, and diameter ≥6 mm) were present.15 A benign cumulative score of 3 (smooth border, 1 or 2 colors distributed uniformly without blending, and a diameter of ≤4 mm) resulted in reassurance of the patient.
In the last half of the year, students implemented the threshold rules. They used simulation models9 with lesions that were not the same as the ones used to develop the threshold rules.
During the monthly primary care clerkship, third-year medical students at Northwestern University Feinberg School of Medicine were invited to participate in the research. Students were informed that participation would not influence their class standing. All students signed the honor code at the beginning of the orientation to the objective structured clinical examination (OSCE). This study was approved by the Northwestern University institutional review board, and all participants provided written informed consent before participation. Participants did not receive compensation.
During the clerkship month, 2 educational interventions (1 hour each) were provided. The first 1-hour educational skills training intervention consisted of a presentation, practice with the simulation model, and group discussion of findings. A 10-minute presentation provided information about identifying patients at risk to develop melanoma based on a history and a physical examination and the threshold rules for visual assessment and lesion management. The threshold rules were presented visually and verbally. Videos of patient counseling by one of us (J.B.) were shown when the group decided to reassure the patient that the lesion was benign, recommend monitoring the lesion for change in 3 months, or refer the patient to a dermatologist for assessment with possible biopsy.
Groups of 4 students assembled around 1 of 4 copies of model 1 and used the threshold rules to assess the lesions using a lighted magnifying lens (original magnification ×4) and a ruler to reach treatment recommendations. A group representative presented decisions about each lesion by relating the scores assigned to the border, color, and diameter, as well as the management decision for the lesion and counseling the patient. Discussion of appropriate counseling of the patient was facilitated by one of us (M.J.A.). Each student was able to verbalize and reinforce his or her understanding of the threshold rules through deliberate, focused practice.
Two weeks later, students assessed model 2. Just as before, one student from each group presented findings and recommended appropriate patient counseling. During the 30-minute session, the moderator provided feedback regarding the students' findings and reinforced the visual assessment and management criteria.
Students completed a 57-item questionnaire to assess the following domains: (1) color perception (n = 6)16; (2) knowledge of people at risk (n = 6, α = .92), early detection (n = 3, α = .95), and prognosis (n = 3, α = .96); (3) clinical reasoning with case examples of identifying a patient's risk of developing melanoma and making a management decision (n = 5, α = .87); (4) attitudes about the importance of assessing risk and performing opportunistic surveillance (n = 26, α = .85); and (5) self-efficacy in recognizing an at-risk patient and a melanoma and in performing counseling (n = 8, α = .92). Finally, their attitudes were assessed on a 5-point Likert-type scale as follows: (1) not concerned/important, (2) slightly concerned/important, (3) somewhat concerned/important, (4) very concerned/important, or (5) extremely concerned/important; alternative 5-point responses were (1) strongly disagree, (2) disagree, (3) uncertain, (4) agree, or (5) strongly agree (eTable). Nine items assessing attitudes were derived from or the same as measures of attitudes toward nutrition previously used with medical students17 (Table 2).
During the preintervention session, third-year medical students examined model 1 and indicated which lesions they believed were clinically suspicious of being a melanoma, which lesions they would biopsy, and which lesions they would monitor over time. Two weeks after the educational intervention, the students assessed model 2 in a similar manner.
In the primary care clerkship's OSCE, a subset of randomly selected student participants interacted with an SP who had a melanoma moulage (realistic melanoma mimic that adheres to the skin with adhesive, 0.7 mm in diameter) surreptitiously applied to his neck 1 cm posterior to the carotid artery. This SP presented with dizziness due to inadvertently doubling his thyroid medication. The SPs were instructed to avoid cuing the student about the moulage. If asked about the melanoma moulage, the SP was trained to deny knowing anything about the lesion. Therefore, detection of the melanoma moulage was a possible incidental finding that may have occurred during the physical examination of the thyroid and neck region. The SPs recorded whether the student detected the melanoma moulage and counseled them regarding the next steps to take. The student's encounter with the SP was videotaped, and the recording was subsequently assessed by one of us (P.P.) for the following items: the student detected the lesion, the student obtained a relevant lesion history, and the student correctly counseled the patient on how to proceed with treatment. The students performed the focused history and physical examination with the SP on the same day as the melanoma simulation training experience. Half of the students interacted with the SP before the melanoma simulation training experience, while the other half saw the SP immediately after the learning experience. The simulation training and the SP experiences occurred on different floors of the same building. When it was time to switch, the 2 groups of students moved between floors using different staircases.
Using the final iteration of the threshold rules, the management decisions made by 98% (n = 64) of the students in the first year were clinically appropriate for the known pathological diagnosis, and all melanomas were referred for biopsy. Then, 84 students used the threshold rules to make decisions about 75 pigmented lesions on the 2 simulation models. Students applied the threshold rules to make clinically appropriate decisions about the pigmented lesions (Cronbach α = .92). All melanomas were referred for biopsy. Because of student errors in assigning scores to the lesions, the melanoma in situ received a score of 6, and the decision was made for the patient to return for follow-up analysis in 3 months; 2% (2 of 84) chose to biopsy the Spitz nevus.
In the second year (July 1, 2011, through June 30, 2012), 111 students assessed model 1. To examine retention of the threshold rule, 75 students assessed model 2 at 2 weeks after the initial learning experience. In the first session, students applied the threshold rules to make clinically appropriate decisions about the pigmented lesions (Cronbach α = .97). In the second session, 85% (6 of 75) of students requested the threshold rules to make management decisions. Of the 15% (1 of 75) of students who were confident enough to make management decisions without being given the threshold rules, implementation was not reliably performed (Cronbach α = .45). One training session was inadequate for the students to retain knowledge and implement it 2 weeks later.
Seventy-five third-year medical students elected to participate in both sessions of this educational experience. One male student failed to meet eligibility criteria due to color blindness (Figure). Fifty-four percent (40 of 74) of the students were male, and 46% (34 of 74) were female. Among the students, the mean (SD) age was 24.9 (2.2) years (age range, 22-32 years). Fourteen students (19%) intended to become primary care physicians, no students intended to become dermatologists, and the remaining students indicated that they intend to pursue other medical specialties. All students attended at least 1 lecture about melanoma during the second year, and 40% (30 of 75) participated in a small-group discussion of melanoma during the required surgery clerkship.
The reliability of the assessment of pigmented lesions on model 1 was the same for board-certified dermatologists (n = 25) and third-year dermatology residents (n = 3) nearing the completion of their training. Lesions were identified as clinically suspicious for being a melanoma (Cronbach α = .99), melanomas were reliably biopsied (Cronbach α = .96), and abnormal pigmented lesions were monitored for change (Cronbach α = .86). Model 2 was intended to be more difficult in each of the 3 assessment categories (clinically suspicious of melanoma, biopsy, and monitor for change). On model 2, the assessment of lesions suspected of being a melanoma (Cronbach α = .87), melanomas biopsied (Cronbach α = .83), and abnormal lesions monitored for change (Cronbach α = .71) was less consistent among the dermatologists than on model 1. The variation in responses was limited to 2 diagnostically challenging lesions, a Spitz nevus and a melanoma (Table 1). In addition to providing valuable feedback regarding the fidelity of the model, the dermatologists affirmed its validity as an educational trainer.
Of 12 knowledge items on the student questionnaire, 1 risk item concerning the anatomic location where people with skin of color most commonly develop melanoma and 1 early detection question regarding opportunistic surveillance for melanoma had statistically significant improvement after the educational intervention (P < .05, χ2 test). The location where people with skin of color could develop melanoma and information about the other knowledge items were discussed in the lectures attended by second-year students; however, opportunistic surveillance was not included in the lectures.
Differences in attitudes before and after education were not meaningful because the perception of the importance of early detection of melanoma was already high before education (eTable). For 5 of 9 items, attitudes about the importance of melanoma opportunistic surveillance were statistically significantly more favorable than those about nutritional counseling (P < .05, χ2 test) (Table 2). All 8 self-efficacy items improved after 2 skills training sessions (Table 3).
Before the educational intervention, the accuracy of medical students' selecting 2 melanomas as being clinically suspicious lesions was 96%, and the accuracy of selecting the lesions for biopsy was 80%. There was no statistically significant difference after education in the ability to recognize the less difficult melanoma as clinically suspicious; however, there was a statistically significant difference in the management decision, with a shift from monitoring for change (from 16% [12 of 74] to 3% [2 of 74]) to performing a biopsy (from 80% [59 of 74] to 96% [71 of 74]) (P < .05, χ2 test) (n = 74). Other changes were a statistically significant reduction in monitoring benign lesions for change (from 43% [32 of 74] to 3% [2 of 74]) and biopsy of melanoma in situ (from 10% [7 of 74] to 26% [20 of 74]) (P < .05, χ2 test) (n = 74) (Table 4).
Using a split-group design, group A had the simulation learning experience before the SP encounter, and group B had the simulation learning experience after the SP encounter. During the physical examinations, the melanoma moulage was detected and mentioned by the student to the SP by 64% (14 of 22) of group A and by 31% (5 of 16) of group B (P = .008, χ2 test). Appropriate counseling was provided by 50% (11 of 22) of group A, while 19% (4 of 21) of group B counseled their patients (P = .03, χ2 test). Detection of the moulage and counseling were not associated with the sex of the student. In addition, 5 students in group A and 1 student in group B recorded the presence of the moulage in their written report of physical findings. These students did not mention the moulage to the SP because they decided it had nothing to do with the focused history and physical examination that they were asked to perform. There was complete agreement between the SPs' written evaluation of the students' counseling and the review of the videos, and there was 98% agreement for detection of lesions. No statistically significant difference was observed in the preintervention knowledge, attitudes, or self-efficacy between students randomized to group A or group B in the split-group design.
The medical students participating in this study recognized the value of screening for melanoma but lacked the self-confidence and skills to do so before the educational intervention. The self-efficacy of the medical students improved after formal instruction with skills training using a simulation model of melanoma combined with threshold rules and videos demonstrating patient counseling. Third-year medical students were able to implement their newly acquired visual assessment and counseling skills during a focused history and physical examination of an SP. The melanoma skills training experience provided medical students the unique opportunity to carefully examine various pigmented lesions that would not normally exist on a single patient. Each student participated in deliberate practice and received feedback that was intended to enhance performance of opportunistic surveillance.
Providing this small-group skills training experience within the primary care clerkship presented opportunistic surveillance for melanoma as a skill relevant to all physicians. The current curriculum at US medical schools does not ensure that each student will have such training. Among graduating medical students at 7 US medical schools, 23.0% had never observed a skin cancer examination, 26.7% had never been trained to perform a skin cancer examination, and 43.4% had never examined a patient for skin cancer.12 Furthermore, self-efficacy in performing skin cancer examination was very low for these students. Only 19.0% of students rated themselves as somewhat or very skilled in skin cancer examination. Students who had been trained at least once in skin cancer examination were 7 times more likely to rate themselves as being somewhat or very skilled in this area.18
Teaching students to recognize patients at risk to have melanoma19- 21 may enhance performance of opportunistic surveillance. A major facilitator of skin cancer screening was identifying a patient at high risk; therefore, risk assessment was part of this educational intervention.11 Medical students participating in the melanoma skills training experience achieved the goal of performing a biopsy of less difficult melanomas with greater than 90% accuracy. The other goal of not performing biopsies on more than 20% of benign lesions was also satisfied. The students' diagnostic accuracy and self-efficacy enabled 63% (45 of 74) of them to detect the moulage during their interaction with the SP. Appropriate counseling of the SP about further treatment was more likely to be performed by the medical students who participated in the melanoma skills training experience before their OSCE than those who had the OSCE before the skills training experience. Two weeks after the intervention, retention of knowledge was limited. Additional training and deliberate practice sessions (eg, training in dermoscopy) as fourth-year medical students will be needed for students to internalize the concepts.22
Some students who received skills training did not detect the melanoma moulage, and 23% (5 of 22) of students who saw the moulage discounted its importance because the OSCE instructions were to perform a focused history and physical examination related to the primary complaint. During feedback sessions with one of us (J.B.), some students related that they saw the moulage but thought it was the SP's own lesion and not related to the case. Students who made the decision not to pursue the unrelated finding failed to perform opportunistic surveillance. Our experience was consistent with those who have noted that simulation training in health professional education was associated with large effects for change in knowledge, skills, and behaviors and with moderate effects for patient-related outcomes.23 With 1 hour of melanoma skills training experience, students were able to acquire skills and apply the visual skills to assess the simulation model. Some implemented the skills into their clinical examination to positively affect patient outcomes with appropriate counseling.
A limitation of the study was the performance with a single class in a single institution, which may limit generalizability. Recognition of nodular melanoma was not included in the curriculum. Because the second model was created to be more difficult than the first model due to the diagnostic difficulty of its lesions, it was not possible to assess student improvement in diagnostic skills by comparing performance with the 2 models. Future iterations of the melanoma simulation models will have negligible variability in difficulty among them so visual assessment skills can be compared during the course of the educational intervention. While the threshold rules provided a minimum standard for education of students, they have not been used in longitudinal studies of clinical care.
The incidence of melanoma is increasing1; therefore, it is essential that physicians are able to incorporate opportunistic melanoma surveillance in their practice of medicine. Toward that end, physicians in training need to be able to differentiate benign pigmented lesions from malignant melanocytic neoplasms. The melanoma skills training experience allows assessment of a wide range of pigmented lesions that would not be found on an individual patient. The ability to compare pigmented lesions, commit to an appropriate management decision, and obtain faculty feedback is desirable for developing proficiency in identifying clinically suspicious nevi. By building self-confidence in their skills, one of the barriers to performing opportunistic surveillance may be obviated through simulation training. Learning to distinguish benign nevi from clinically suspicious lesions through realistic simulation would better prepare physicians to perform accurate opportunistic surveillance during routine physical examinations and potentially lower the mortality rate of melanoma in the United States.
Correspondence: June K. Robinson, MD, Department of Dermatology, Northwestern University Feinberg School of Medicine, 676 N St Clair St, Ste 1260, Chicago, IL 60611 (firstname.lastname@example.org).
Accepted for Publication: November 8, 2012.
Published Online: March 20, 2013. doi:10.1001/jamadermatol.2013.2882
Author Contributions: Dr Robinson had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Davis, Bierman, McGaghie, Brucker, Martini, and Robinson. Acquisition of data: Anderson and Brucker. Analysis and interpretation of data: Jain, Patel, Blatt, and McGaghie. Drafting of the manuscript: Jain. Critical revision of the manuscript for important intellectual content: Jain, Anderson, Patel, Blatt, Davis, Bierman, McGaghie, Brucker, Martini, and Robinson. Statistical analysis: Jain. Obtained funding: Robinson. Administrative, technical, and material support: Anderson, Blatt, Davis, Bierman, Brucker, and Martini. Study supervision: Blatt and Robinson.
Conflict of Interest Disclosures: Dr Robinson, who is the editor of JAMA Dermatology, was not involved in the editorial evaluation or editorial decision to accept this work for publication.
Funding/Support: This research was funded in part by the Northwestern Memorial Foundation Alvin H. Baum Clinical Simulation Research Grants Initiative and by grant P30AR057216 from the Northwestern University Skin Disease Research Center with support from the National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institutes of Health. Dr McGaghie's contribution was supported in part by the Jacob R. Suker, MD, professorship in medical education and by grant UL RR 025741 from the National Center for Research Resources, National Institutes of Health.
Role of the Sponsors: The National Institutes of Health had no role in the preparation, review, or approval of the manuscript.
Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the view of the Northwestern University Skin Disease Research Center or the National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institutes of Health.