During weeks 1 through 7, 3 residents used the simulator for 4 sessions, 28 exercises, and 2.7 hours. During weeks 8 through 14, 21 residents used the simulator for 70 sessions, 1632 exercises, and 83.9 hours. The residents, sessions, exercises, and use duration increased by 7-fold, 17-fold, 58-fold, and 32-fold, respectively. The dashed line indicates the time when the tournament was announced.
Kerfoot BP, Kissane N. The Use of Gamification to Boost Residents’ Engagement in Simulation Training. JAMA Surg. 2014;149(11):1208-1209. doi:10.1001/jamasurg.2014.1779
Interval deliberate practice of surgical skills is required for surgical residents to develop and maintain their technical skills. Even 2 weeks without practice on a minimally invasive surgical simulator can lead to a substantial decline in skills.1 In spite of the value of interval simulation training, incentivizing residents to engage regularly in simulation training can be a challenge for many institutions. Can gamification (the introduction of game mechanics to engage users, including but not limited to competition and leaderboards)2 boost residents’ engagement in simulation training? Although game-based learning is cited as an emerging technology likely to have a large impact on education in the next 2 to 3 years,3 little research has been conducted to demonstrate its efficacy for health professional training.4 We investigated whether the introduction of competitive game mechanics into simulator education among residents could significantly boost the use and cost-effectiveness of the da Vinci Skills Simulator (Intuitive Surgical Inc).
Our study was conducted over 14 weeks from January to May 2012 at the Veterans Affairs (VA) Boston Healthcare System in West Roxbury, Massachusetts. All urology and general surgery residents at Boston University (13.8 km [8.6 miles] away) and Brigham and Women’s Hospital in Boston, Massachusetts ( 11.5 km [7.2 miles] away), were eligible to participate. During weeks 1 through 7, residents were invited to use the simulator. A single elimination tournament was then announced via e-mail at the end of week 7. The 16 residents with the highest aggregate scores on 9 exercises on the simulator during weeks 8 through 14 would qualify. Leaderboards were posted via e-mail every 1 to 2 weeks. Tournament prizes ranged from $50 to an iPad (Apple Inc) for the winner. Over all 14 weeks, a $30 travel reimbursement was given to residents not on a VA rotation for each trip to use the simulator. The tournament compared ring-walk simulator and dry-laboratory performance among qualifying residents. Cost estimates include the price of the simulator ($85 000 amortized over 5 years), travel reimbursements, and prizes but not the salary expenses of a simulation coordinator.
Of 141 eligible residents, 22 (16%) used the simulator over the 14 weeks of our study (6 of 11 urology residents at Brigham and Women’s Hospital [55%], 5 of 8 urology residents at Boston University [63%], 4 of 68 general surgery residents at Brigham and Women’s Hospital [6%], and 7 of 54 general surgery residents at Boston University [13%]). Use varied widely by specialty: 11 of 19 urology residents (58%) and 11 of 122 general surgery residents (9%) used the simulator. During weeks 1 through 7, 3 residents used the simulator for 4 sessions, 28 exercises, and 2.7 hours (Figure). During weeks 8 through 14, 21 residents used the simulator for 70 sessions, 1632 exercises, and 83.9 hours. Residents, sessions, exercises, and use duration increased by 7-fold, 17-fold, 58-fold, and 32-fold, respectively. After the announcement of the tournament, the estimated cost per hour of simulator use decreased 12-fold from $864 to $74. In the tournament, qualifying ranks and scores correlated with simulator ring-walk scores (P < .01 for both), but none of these 3 metrics correlated significantly with dry-laboratory ring-walk scores.
The introduction of game mechanics into simulator education significantly boosted the use of our da Vinci Skills Simulator, especially among urology residents. The discrepancy in use by specialty is likely due in part to the importance attributed to robotic surgical training by urology residents given the relatively high frequency of robotic surgical cases in their field. By increasing the use of the simulator, gamification substantially reduced the estimated cost per hour of simulator use. Given the great expense of many simulation systems, cost is an important outcome measure that, to date, has only been reported in approximately 6% of simulation-based medical education research studies.5 Further work is needed to determine if the boost generated by the tournament can be replicated longitudinally and in the absence of monetary prizes.
Corresponding Author: B. Price Kerfoot, MD, EdM, VA Boston Healthcare System, 150 S Huntington Ave, 151DIA, Boston, MA 02130 (email@example.com).
Published Online: September 17, 2014. doi:10.1001/jamasurg.2014.1779.
Author Contributions: Dr Kerfoot had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Both authors.
Acquisition, analysis, or interpretation of data: Both authors.
Drafting of the manuscript: Kerfoot
Critical revision of the manuscript for important intellectual content: Kissane.
Obtained funding: Kerfoot.
Administrative, technical, or material support: Both authors.
Study supervision: Both authors.
Conflict of Interest Disclosures: Dr Kerfoot was a paid educational consultant to the legal team defending Intuitive Surgical Inc in a 2013 malpractice case. No other disclosures are reported.
Funding/Support: This research was funded by Intuitive Surgical Inc.
Role of the Funder/Sponsor: Intuitive Surgical Inc had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentation: The paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 8, 2014; New Haven, Connecticut.