In the control group, 6 participants did not return their pretests after the 3-day presentation, and 5 participants did not return their posttests before the small-group tutorial. In the intervention group, 7 participants did not return their pretests after the 3-day presentation, and 14 participants did not return their posttests before the small-group tutorial.
The percentage of 50 participants in the class of 2013 who were on track with their e-learning module according to the days since enrollment. Although participant engagement was globally high during the year, the lowest attendances occurred during vacation periods. Participants were considered to be on track if their last response was less than 2 days old.
Boespflug A, Guerra J, Dalle S, Thomas L. Enhancement of Customary Dermoscopy Education With Spaced Education e-LearningA Prospective Controlled Trial. JAMA Dermatol. 2015;151(8):847-853. doi:10.1001/jamadermatol.2015.0214
Dermoscopy permits the detection of early-stage melanomas but is difficult to learn. It is important to develop effective teaching methods. Spaced education is a methodology within the field of adaptive learning that uses online tools to reinforce long-term retention.
To determine whether a spaced education dermoscopy module improved dermoscopy skills in the continuing medical education setting and to evaluate participant satisfaction.
Design, Setting, and Participants
We designed a prospective controlled study with 2 sequential cohorts of participants enrolled between September 2010 and September 2013, in the continuing medical education dermoscopy program of the Claude Bernard–Lyon 1 University in Lyon, France. Participants enrolled in this program were either certified dermatologists or senior dermatology residents. The control group (n = 95) comprised all participants enrolled during the 2 first years of the study (49 participants in the class of 2010, 46 in the class of 2011). The intervention group (n = 96) comprised all participants enrolled during the third and fourth years of the study (46 in the class of 2012; 50 in the class of 2013).
All participants attended a 3-day lecture followed by small-group tutorials 4 months later. Each participant also attended a day of consultation with a dermoscopy specialist. In addition, participants in the intervention group were enrolled in an e-learning spaced education dermoscopy program.
Main Outcomes and Measures
The main outcome measure was mean participant scores at the posttest evaluation, which was conducted 4 months after course enrollment.
The intervention group had better results at the posttest, with a mean (SD) score (out of a possible 160.0 points) of 148.1 (5.8) (n = 82 participants) vs 145.7 (7.7) (n = 90 participants) in the control group (P = .02). Ninety-two percent of the participants (80 of 87) were extremely or very satisfied with the e-learning module. Participant engagement was high, with an average of 85% of participants (80 of 94) “on track” at any given time of the year.
Conclusions and Relevance
Our study shows that, in the context of continuing medical education, a spaced education Internet dermoscopy module combined with in-class training increases participant performances in dermoscopy. It is easy to use and adaptable to professional working schedules.
Malignant melanoma (MM) is one of the most rapidly growing cancers worldwide, with a consistent increase in incidence among white populations.1 Malignant melanoma is the ninth leading cancer in France2 and is an ideal candidate for early screening because most of these neoplasms have an initially slow radial growth3,4; the early detection of MM has been associated with thinner lesions and improved survival without the need for invasive and costly procedures.5
Because dermoscopy has proved that it allows the diagnosis of thinner MM,6 its use has considerably expanded. A recent study7 has shown that 97.5% of dermatologists in the dermatology departments of French hospitals and 94% of private practice dermatologists8 claim to use it. While dermoscopy improves the diagnostic accuracy for the detection of MM by trained users9—without considerably extending the duration of a standard complete skin examination10—it might reduce the diagnostic accuracy when used by nontrained users.11
One-fifth of French dermatologists who use dermoscopy have never received formal dermoscopic training,7 and, for most users, excessive training times are the strongest disadvantages of this technique.7 In this context, it is important to develop effective new ways of teaching dermoscopy that are easy, accessible, and adapted to the physician’s activity and workload.
Web-assisted courses are currently being developed in many fields of continuing medical education.12 Many different e-learning modules in general dermatology have been tested and have shown that they are acceptable and useful tools for initial medical training (undergraduates and postgraduates)13- 17 and for patient education.18 Few studies have been conducted to assess the benefits of an online educational training program for continuing medical education in dermatology. An uncontrolled study19 evaluating a web-based training program in dermoscopy has been published. To our knowledge, no controlled studies have evaluated a dermoscopy e-learning program.
Among the many different web-assisted, problem-based learning courses, the online educational form of learning called “spaced education” is based on 2 psychology research findings (the spacing20 and testing effects21) and has been shown in randomized trials to improve knowledge acquisition, boost learning retention for up to 2 years, and durably improve clinical behavior.22- 25
In this study, we performed a prospective controlled trial to estimate the efficacy of an online spaced education educational training for dermoscopy, and we studied the factors related to its efficacy, in the context of continuing medical education.
We designed a prospective controlled study with 2 sequential cohorts of participants enrolled in a continuing medical education dermoscopy program at the Claude Bernard–Lyon 1 University in Lyon, France (Attestation d’Etudes Universitaires en dermoscopie; ie, a Dermoscopy diploma), from September 2010 to September 2013. This study has not been registered in a public trial registry because it does not prospectively assign human participants to intervention or comparison groups to evaluate the cause and effect relationship between a medical intervention and a health outcome. This study was approved by the University Lyon 1 University ethics committee.
Cohort 1 (the control group) comprised all participants enrolled during the 2 first years of the study (the classes of 2010 and 2011). Cohort 2 (the intervention group) comprised all participants enrolled during the third and fourth years of the study (the classes of 2012 and 2013).
All participants in the 2 cohorts attended a 3-day lecture on dermoscopy with the same teacher in September of 2010, 2011, 2012, and 2013, followed by small-group tutorials 4 months later (each participant attended a 5-hour session with 10 other participants), and a 1-on-1 day of consultation with the same dermoscopy specialist. In addition, participants in cohort 2 were offered access to a free, spaced education e-learning module (Figure 1).
An Internet spaced education module was developed on the qstream platform, where we purchased a dedicated private website. Spaced education is a novel form of online education that combines the educational benefits of the spacing effect,20 the testing effect21 and compelling game mechanics. Spaced education involves spaced repetition of question-based educational content, but also includes an adaptive rescheduling algorithm combined with compelling game mechanics.
A total of 175 image-based questions were created, including 92 multiple-choice questions and 83 open questions that covered all the predefined learning objectives of the training program.
Our spaced education module used an automated, interactive email system developed at Harvard Medical School,22 which sent 3 questions daily to each participant. Once participants clicked on a hyperlink in an email, a web page opened, allowing them to submit their answer to a question. The answer was downloaded to a central server. Participants were immediately presented with a web page with the correct answer and with elements explaining why possible answers were correct or incorrect.
If the participant answered correctly, the question was repeated 40 days later. If the answer was not correct, the question was repeated 14 days later. Questions were eliminated when answered correctly twice.26 The module was completed once all questions were eliminated. The length of this adaptive spaced education course thus varied based on each participant’s baseline knowledge and his or her ability to learn and retain knowledge based on the spaced education’s question and answer presentation.
Inclusion criteria consisted of being enrolled in the Claude Bernard–Lyon 1 University Dermoscopy degree program from September 2010 to September 2014. The only exclusion criterion consisted of the absence of consent to participate in the study.
Participants who enrolled in this program were either board-certified dermatologists or senior dermatology residents. Participants were informed that their participation on the e-learning module would have no effect on their final grade, standing, or promotion. All participants in this study gave their voluntary, fully informed written consent to participate; they were not compensated.
Clinical photographs were taken using a Canon EOS 350D camera with a 60-mm macrolens and annular flashlight (Canon France). Dermoscopy photographs were taken using a Dermlite FOTO System (3gen).
Knowledge acquisition was evaluated through pretests and posttests, which took place immediately after the delivery of the lecture (pretest) and immediately before the small group tutorial 4 months after the lecture (posttest), respectively (Figure 1). Pretest and posttests consisted of 40 multiple-choice, image-based questions to be answered in less than 40 minutes (the maximum score at the pretest and posttests was 160.0). The same 40 questions were used for each pretest and posttest in both cohorts. The same person (A. B.) corrected all participants anonymously.
During the enrollment period, all participants provided information concerning their demographics, the region in which they were board certified, the region in which they practiced, and the year of their certification. Additional information was asked of participants who were in the intervention group in order to identify factors linked to the intervention’s compliance or efficacy. These questions were related to the participants’ type of dermatology practices, their previous dermoscopy qualifications, as well as general information concerning their use of a dermoscope and of a smartphone. During the year, every week, and for each participant in the intervention group, the following variables were automatically collected: their percentage of answers answered correctly on first occurrence and their “on-track” or “off-track” status.
Participant engagement with the e-learning module was evaluated weekly. Participants were considered on track if their last response was less than 2 days old, falling behind if their last response was less than 4 days old, and off track if the last response was more than 5 days old.
Six months after enrollment, participants in the intervention group were asked by email to complete an additional web-based 5-point Likert scale questionnaire with 35 items that assessed subjective feeling of the participants, their satisfaction, and their experience using the e-learning module. An email reminder was sent to participants who did not respond after 2 weeks.
To further evaluate user satisfaction, participants in the intervention group could pay €15 (US $16) to subscribe to an optional advanced spaced education module (as a point of comparison, the reimbursement of a dermoscopic examination of the skin by the French health insurance is of €54.10 (US $57). We calculated the percentage of people who subscribed to this second module.
We compared the characteristics of the 2 groups using 2-tailed univariate analysis with an α risk of 5%. The results of the posttest formed the primary judgment criterion for the efficacy of the spaced education Internet module. The secondary judgment criterion for the efficacy of the spaced education Internet module was the mean progression between the pretest and the posttest. The associations between the judgment criteria and the collected variables were studied using a 2-tailed univariate analysis with an α risk of 5%. χ2 Tests and Fisher tests were used with qualitative variables, t tests, and Mann-Whitney tests with quantitative variables. The spaced education module efficacy was estimated using an adjusted linear regression model (each judgment criterion was adjusted for the allocated group of the participant, the results of the pretest, the number of years of experience, the participant’s sex, and the variables associated with the judgment criterion in an univariate analysis with P < .25; the latter variables were removed 1 by 1 until all of them significantly improved significantly the model fit). Factors associated with the efficacy of the training module were analyzed using an adjusted linear regression model (in the intervention group, each judgment criterion adjusted for the results of the pretest, the number of years of experience, the participant’s sex, the amount of student engagement, the principal device used to answer to the spaced education Internet module, the principal moment when spaced education module was used, and the variables associated with the judgment criterion in an univariate analysis with P < .25. The latter variables were removed 1 by 1 until all of them improved significantly the model fit). All analysis were performed using the R statistical software (R Foundation for Statistical Computing).
A total of 191 participants were included in the study. The control group comprised all 95 participants enrolled during the 2 first years of the study (class of 2010, 49 participants; class of 2011, 46 participants). The intervention group comprised all 96 participants enrolled during the third and fourth years of the study (class of 2012, 46 participants; class of 2013, 50 participants) (Figure 1).
The median age of all 191 participants was 39 years (10th and 90th percentiles: 29 and 56 years), and 88% (169 of 191) were women. The median number of years since their dermatology certification was 9 years (10th and 90th percentiles: 2.0 and 26.8 years; 172 of 191), and most participants were certified dermatologists (173 of 191 [90.6%]).
The Table shows the demographics of cohort 1 (control group) and cohort 2 (intervention group). No characteristics were significantly different between control and intervention groups except for pretest scores (149.9 out of a possible 160.0 in the control group vs 148.2 of 160.0 in the intervention group; P = .047). To take this difference into account, all multivariate analyses were adjusted on the pretest score.
The intervention group had better results on the posttest, which took place 4 months after the in-class lecture, with a mean (SD) score of 148.1 (5.8) (82 of 96 participants) vs 145.7 (7.7) in the control group (90 of 95) (P = .02).
In the multivariate analysis, the posttest score was adjusted on the intervention group, the years since the certification, the participant’s sex, and the pretest score. Two variables were significantly associated with a higher posttest score in the multivariate analysis: having a higher pretest score (P ≤ .001) and being in the intervention group (P < .001).
When considering only the intervention group, the posttest score was adjusted in the multivariate analysis on the years since the certification, the sex, the pretest score, the amount of engagement, the main device used for e-learning, the principal moment for e-learning, the type of activity, and the percentage of answers answered correctly on the first occurrence. The variables that were significantly associated with a higher posttest score in the multivariate analysis were using a computer instead of a smartphone or tablet to visualize spaced education questions (P ≤ .03) and having a high percentage of correct answers at first occurrence on the spaced education module (P = .03).
The intervention group had a higher progression between pretest and posttest scores with a mean (SD) progression of 0.3 (7.1) points (76 of 95 participants) in the intervention group vs −3.6 (6.2) points (83 of 96) in the control group (P <.001) (Table).
The variables that were significantly associated to a higher progression between pretest and posttest scores in the multivariate analysis were having a lower pretest score (P < .001) and being in the intervention group (P = .01). If we considered only the intervention group, the sole fact of having a lower pretest score was significantly associated with a higher progression between pretest and posttest scores in the multivariate analysis (P < .001) (detailed results are available on request).
On average 85% of the participants (80 of 94) were on track with their e-learning questions at any given time of the year. The lowest attendances were observed during holidays, as seen in Figure 2.
Ninety percent of the intervention group (87 of 96) answered the user satisfaction questionnaire, which was sent 6 months after enrollment. Ninety-two percent of those participants (80 of 87) were extremely or very satisfied with the e-learning module, and 93% of them (81 of 87) considered it essential compared with a classroom lecture. Participants considered that the module was compatible with their professional activities, and 94% of them (82 of 85) confirmed that the time it took them to answer their daily questions was ideal (<5 minutes for 68% of the users [58 of 85] and less than 10 minutes for 28% [23 of 85]).
Participants used all possible media devices to answer their daily questions (personal, professional or family computer, or smartphone), although professional computers were most often used. While participants consulted their daily questions at all moments during the day, they mostly consulted them at the end of their workday (50% [42 of 87]) and between 2 consultations (13% [11 of 84]).
When asked if they would like to participate to another dermatology e-learning module using the adaptive spaced education format, 98% of them said “yes” (81 of 82). Eighty-nine percent of the class of 2012 (41 of 46) paid to subscribe to another advanced dermoscopy module in October 2013.
Spaced education has been shown in randomized trials to improve knowledge acquisition, boost learning retention for up to 2 years, and durably improve clinical behavior.22- 25 It is based on 2 psychology research findings: the spacing20 and the testing effects.21 The spacing effect is based on the hypothesis that, when you present and repeat information over intervals of time, you can increase the uptake of knowledge as opposed to learning a large amount of information in a single presentation. The testing effect is based on the hypothesis that, when you present information in a “test” format, rather than just reading it, long-term retention is improved.
Spaced education has especially been tested in urology22 and cardiology.25 Our study shows that, in the context of continuing medical education, an spaced education Internet dermoscopy module combined with an in-class training program increases participant performances in dermoscopy.
In this study we show that, 4 months after an in-class lecture, the intervention group with the e-learning module had significantly better mean (SD) scores (148.1 [5.8]) than the control group (1457 [7.7]). This difference was still present using a multivariate analysis. The mean difference between both groups was small (2.4 points out of 160.0), and further studies could be necessary to evaluate its relevance.
This module is easy to use, adaptable to professional working schedules, and acceptable in terms of time spent on the course. Ninety-four percent of participants considered that the time it to took them to answer their daily questions was ideal, and 68% of them needed less than 5 minutes to complete their daily course. On average, 85% of the participants were “on track” in their e-learning questions at any given time of the year.
User satisfaction was also qualitatively high, with 91% of participants being extremely satisfied or very satisfied with our spaced education dermoscopy e-learning module. User satisfaction was confirmed quantitatively when 89% of the participants who were asked to join a second dermoscopy module enlisted and paid a small nominal fee of €15 (US $16).
Despite the wide use of dermoscopy in France, few board-certified dermatologists have received formal training.8 Although 94% of senior dermatology residents in the United States use dermoscopy, only 48% of them were trained with a pigmented lesion specialist.27 It is therefore important to specifically target these 2 populations when creating dermoscopy teaching programs. Our study population was composed of senior dermatology residents and certified dermatologists and was comparable with the population of dermoscopy users described previously7,8 with most women (88%) working in the private practice setting.
e-Learning is undoubtedly becoming popular in the continuing medical education setting,28 but even if e-learning is well appreciated by participants, the efficacy of e-learning modules must be rigorously evaluated compared with a conventional in-class lecture.29 Most studies evaluating e-learning programs show important user satisfaction, but few studies show real increases in performances.29 In our study, we show that our space education Internet module significantly increased knowledge retention after 4 months when combined with intensive in-class training. Further studies are necessary to evaluate its benefits on long-term memory and, more important, on patient outcomes.
e-Learning is fashionable.30 Our study evaluated a combination of in-class lectures and an e-learning spaced education module. More studies are necessary to evaluate the impact of e-learning modules used alone or in combination with in-class lectures. However, a recent study shows that although e-learning modules are appreciated by general practitioners for their continuing medical education, their favorite learning setting still remains face-to-face lecture-based formats.31 This supports the idea of combining both.
Finally, we show that when our spaced education e-learning questions were mainly visualized on smartphones or tablets, this was associated with worse posttest scores in multivariate analysis. This might be because participants who use their smartphones concentrate less than participants who use their computers, but visualizing questions at the end of the day and during leisure time was not associated with better posttest scores. This might be explained by the poorer quality of dermoscopy photographs when seen on smartphone screens and, to a lesser extent, on tablet screens. Dermoscopy is intrinsically visual, and the temptation to use smartphones for diagnosis and training is logical; new software applications and technologies have therefore been created.32 Our results tend to recommend caution in the use of smartphones to teach dermoscopy.
We developed a spaced education Internet dermoscopy module that showed its efficacy and user satisfaction in the continuing medical education setting when combined with a formal in-class dermoscopy program. Further studies are necessary to evaluate the impact of our module on clinical behavior and to evaluate the place of smartphones in training modules.
Corresponding Author: Luc Thomas, MD, PhD, Service de Dermatologie, CH Lyon Sud, 165 Chemin du Grand Revoyet, 69495 Pierre Bénite CEDEX, France (firstname.lastname@example.org).
Accepted for Publication: January 29, 2015.
Published Online: April 22, 2015. doi:10.1001/jamadermatol.2015.0214.
Author Contributions: Dr Boespflug had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: Boespflug, Guerra, Thomas.
Drafting of the manuscript: Boespflug, Guerra, Thomas.
Critical revision of the manuscript for important intellectual content: Guerra, Dalle, Thomas.
Statistical analysis: Guerra.
Obtained funding: Thomas.
Administrative, technical, or material support: Thomas.
Study supervision: Dalle, Thomas.
Conflict of Interest Disclosures: Dr Dalle received basic science research grants from Roche. Dr Thomas has been, is, and will be principal investigator for the BRIM-3, coBRIM, BRIM-8, and COMBI-AD clinical trials, including trials of vemurafenib in either 1 or both treatment arms, but does not receive any personal honorarium. He is also the author or coauthor of several dermatology and dermoscopy textbooks for which he receives royalties. No other disclosures are reported.
Funding/Support: This study was supported in part by a grant from the Claude Bernard–Lyon 1 University. Dr Dalle received academic grants from Claude Bernard–Lyon 1 University. Dr Thomas receives grants for academic work and research activity from Claude Bernard–Lyon 1 University, Direction de la Recherche Clinique des Hospices Civils de Lyon, and la Ligue Contre le Cancer du Rhône.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.