The Benefits and Limitations of Targeted Training in Flexible Transnasal Laryngoscopy Diagnosis | Laryngology | JAMA Otolaryngology–Head & Neck Surgery | JAMA Network
[Skip to Navigation]
Figure.  Standardized Study Questionnaire
Standardized Study Questionnaire

Each participant completed a questionnaire for each video viewed (2 sets of 25 videos). PGY indicates postgraduate year.

Table 1.  Preintervention ICCs of Each Training Group
Preintervention ICCs of Each Training Group
Table 2.  Postintervention ICCs of Each Training Group
Postintervention ICCs of Each Training Group
1.
Brook  CD, Platt  MP, Russell  K, Grillone  GA, Aliphas  A, Noordzij  JP.  Time to competency, reliability of flexible transnasal laryngoscopy by training level: a pilot study.  Otolaryngol Head Neck Surg. 2015;152(5):843-850.PubMedGoogle ScholarCrossref
2.
Zaiontz  C. Real statistical analysis using Excel. Real statistical resource pack. http://www.real-statistics.com. Accessed May 2015.
3.
Fung  K.  Otolaryngology–head and neck surgery in undergraduate medical education: advances and innovations.  Laryngoscope. 2015;125(suppl 2):S1-S14.PubMedGoogle ScholarCrossref
4.
Smith  ME, Leung  BC, Sharma  R, Nazeer  S, McFerran  DJ.  A randomized controlled trial of nasolaryngoscopy training techniques.  Laryngoscope. 2014;124(9):2034-2038.PubMedGoogle ScholarCrossref
5.
Nagendran  M, Gurusamy  KS, Aggarwal  R, Loizidou  M, Davidson  BR.  Virtual reality training for surgical trainees in laparoscopic surgery.  Cochrane Database Syst Rev. 2013;8(8):CD006575.PubMedGoogle Scholar
6.
Zendejas  B, Brydges  R, Hamstra  SJ, Cook  DA.  State of the evidence on simulation-based training for laparoscopic surgery: a systematic review.  Ann Surg. 2013;257(4):586-593.PubMedGoogle ScholarCrossref
7.
Okuda  Y, Bryson  EO, DeMaria  S  Jr,  et al.  The utility of simulation in medical education: what is the evidence?  Mt Sinai J Med. 2009;76(4):330-343.PubMedGoogle ScholarCrossref
8.
Kamine  TH, Gondek  S, Kent  TS.  Decrease in junior resident case volume after 2011 ACGME work hours.  J Surg Educ. 2014;71(6):e59-e63.PubMedGoogle ScholarCrossref
9.
Schwartz  SI, Galante  J, Kaji  A,  et al.  Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.  JAMA Surg. 2013;148(9):829-833.PubMedGoogle ScholarCrossref
10.
Scally  CP, Reames  BN, Teman  NR, Fritze  DM, Minter  RM, Gauger  PG.  Preserving operative volume in the setting of the 2011 ACGME duty hour regulations.  J Surg Educ. 2014;71(4):580-586.PubMedGoogle ScholarCrossref
11.
Allak  A, Liu  YE, Oliynyk  MS, Weng  KH, Jameson  MJ, Shonka  DC  Jr.  Development and evaluation of rigid esophagoscopy simulator for residency training.  Laryngoscope. 2016;126(3):616-619.PubMedGoogle ScholarCrossref
12.
Patel  NR, Makai  GE, Sloan  NL, Della Badia  CR.  Traditional vs simulation resident surgical laparoscopic salpingectomy training: a randomized controlled trial.  J Minim Invasive Gynecol. 2016;23(3):372-377.Google ScholarCrossref
13.
York  SL, Maizels  M, Cohen  E,  et al.  Development and evaluation of cesarean section surgical training using computer-enhanced visual learning.  Med Teach. 2014;36(11):958-964.PubMedGoogle ScholarCrossref
14.
Abdelsattar  JM, Pandian  TK, Finnesgard  EJ,  et al.  Do you see what I see? how we use video as an adjunct to general surgery resident education.  J Surg Educ. 2015;72(6):e145-e150.PubMedGoogle ScholarCrossref
15.
van Det  MJ, Meijerink  WJHJ, Hoff  C, Middel  LJ, Koopal  SA, Pierie  JPEN.  The learning effect of intraoperative video-enhanced surgical procedure training.  Surg Endosc. 2011;25(7):2261-2267.PubMedGoogle ScholarCrossref
16.
Hayden  EL, Seagull  FJ, Reddy  RM.  Developing an educational video on lung lobectomy for the general surgery resident.  J Surg Res. 2015;196(2):216-220.PubMedGoogle ScholarCrossref
17.
Levitt  MA, Dawkins  R, Williams  V, Bullock  S.  Abbreviated educational session improves cranial computed tomography scan interpretations by emergency physicians.  Ann Emerg Med. 1997;30(5):616-621.PubMedGoogle ScholarCrossref
18.
Rowse  PG, Ruparel  RK, AlJamal  YN, Abdelsattar  JM, Heller  SF, Farley  DR.  Catering to millennial learners: assessing and improving fine-needle aspiration performance.  J Surg Educ. 2014;71(6):e53-e58.PubMedGoogle ScholarCrossref
Original Investigation
July 2017

The Benefits and Limitations of Targeted Training in Flexible Transnasal Laryngoscopy Diagnosis

Author Affiliations
  • 1Department of Otolaryngology–Head and Neck Surgery, Boston University Medical Center, Boston, Massachusetts
JAMA Otolaryngol Head Neck Surg. 2017;143(7):707-711. doi:10.1001/jamaoto.2017.0120
Key Points

Question  Can targeted laryngoscopy teaching improve diagnostic abilities in de novo learners?

Findings  In this prospective study of fiberoptic laryngoscopy interpretations in interns and attending physicians, significant improvement occurred after targeted teaching about vocal cord abnormalities.

Meaning  Learning of flexible laryngoscopy can be improved with the use of a teaching video.

Abstract

Importance  Targeted laryngoscopy training can be used successfully in de novo learners.

Objective  To determine the value of targeted laryngoscopy education in interns.

Design, Setting, and Participants  This prospective study of fiberoptic laryngoscopy interpretations enrolled 13 participants in an academic hospital setting from August 1 to December 31, 2015. Participants included 10 postgraduate year 1 emergency and otolaryngology interns and 3 board-certified otolaryngology attending physicians.

Interventions  Participants viewed 25 selected and digitally recorded fiberoptic laryngoscopies and were asked to rate 13 items relating to abnormalities in the pharynx, hypopharynx, larynx, and subglottis; the level of concern; and confidence with the diagnosis. A laryngoscopy teaching video was then administered to the interns before rating a second set of 25 videos. Improvement in diagnosis and intraclass correlation coefficients (ICC) were calculated for each question and compared between the first and second administration.

Main Outcomes and Measures  Improvement in correct diagnosis of abnormalities in recorded laryngoscopies.

Results  All 13 participants completed the interventions. The ICCs for all questions were generally low for the intern groups and higher for the attending group. For vocal cord mobility, a preintervention ICC of 0.25 (95% CI, 0.16-0.37) improved to 0.47 (95% CI, 0.36-0.59) among interns after the intervention. The ICCs for vocal cord mobility were higher among attendings for the preintervention (0.89; 95% CI, 0.84-0.93) and postintervention (0.89; 95% CI, 0.83-0.93) assessments. Minimal improvement was observed in intern scores for base of tongue abnormalities, subglottic stenosis, vocal cord abnormalities, level of comfort, level of concern, pharyngeal abnormalities, or laryngeal, pharyngeal, and hypopharyngeal masses.

Conclusions and Relevance  Learning of flexible laryngoscopy can be improved with the use of a teaching video; however, additional interventions are needed to attain competence in accurately diagnosing upper airway lesions. Clinicians who seek to perform flexible laryngoscopy require robust training.

Introduction

Flexible transnasal laryngoscopy (FTNL) is a commonly used diagnostic procedure for evaluation of the upper aerodigestive tract. Findings on these procedures can range from normal to minor abnormalities to critical diagnoses that necessitate immediate intervention. No standardized method for teaching this important diagnostic skill currently exists, and trainees learn by repetition of the procedure on patients in the inpatient and outpatient settings. In a previous study,1 we concluded that otolaryngology residents reached competency in FTNL compared with otolaryngology attendings by postgraduate year (PGY) 3. By that time, the PGY-3 residents had undergone approximately 15 months of dedicated otolaryngology training and had performed numerous FTNLs without the administration of any targeted training or additional educational tools for the procedure. The purpose of this study was to determine whether administering an educational tool in the form of an educational narrated video could improve novice clinicians’ ability to diagnose various upper aerodigestive diagnoses and confidence in identifying worrisome airways using FTNL.

Methods

This prospective study of fiberoptic FTNL interpretations used archived endoscopy videos taken during routine clinical evaluations in the otolaryngology outpatient clinic of Boston University Medical Center, Boston, Massachusetts, from January 1, 2006, through December 31, 2013. This study was approved by the institutional review board at Boston University Medical Center, which waived the need for informed consent.

We created 2 sets of 25 videos. Participants were asked to view 1 set of 25 videos, followed by a teaching video, and then asked to view a second set of 25 videos. A total of 50 videos were administered to all participants in 2 groups of 25 videos. Of the videos in each set of 25, 21 were unique and 4 were repeated to calculate intrarater reliability.

A standardized questionnaire was given to each participant immediately after each video that they viewed, totaling 2 sets of 25 questionnaires for each participant (Figure). We designed the questionnaire to be general and to address each subsite in the larynx to reduce priming. Participants were aware of the possibility of viewing normal and abnormal findings. Approximately 30 minutes elapsed between viewing the first set of videos and the second set of videos. Participants were asked to rate 13 items relating to abnormalities in the pharynx, hypopharynx, larynx, and subglottis; the level of concern; and confidence with the diagnosis, with the level of abnormalities or concern based on an attending otolaryngology rating (ie, criterion standard). All videos excluding the narrated instructional video were administered without sound to eliminate bias by recognition of the examiner’s voice (eg, a specific otolaryngologist who specializes in cancer).

Participants for the study were chosen to represent de novo learners of fiberoptic FTNL based on having limited to no previous training in the area but motivation to improve their current diagnostic skills. Emergency department (ED) and otolaryngology interns were given the option to participate in the study. Seven ED and 3 otolaryngology interns participated for a total of 10 interns. Three attending otolaryngologists at our institution also rated each set of videos. The mean of these scores was calculated and used as the criterion standard. Of the attending otolaryngologists, 1 was a fellowship-trained rhinologist and 2 were general otolaryngologists. The PGY1 interns were administered the instructional video between the first and second sets of videos, and the attending otolaryngologists were not administered the teaching video. The information from the questionnaires was then recorded in a spreadsheet for analysis.

All endoscopies were performed on the same type of digital video laryngoscopes (type ENF-V2; Olympus) and accessed from digital archives. All endoscopies used in the study were of adult patients. We specifically selected the videos from the archives for the presence of certain abnormalities or findings. The videos represented roughly the same number of normal and distinct pathologic conditions in each set of 25. Pathologic conditions represented in videos included vocal cord paralysis, subglottic stenosis, and laryngeal and pharyngeal masses. The videos were deidentified and edited to representative examinations of 20 to 50 seconds by one of us who is a fellowship-trained laryngologist (J.P.N.). None of us who participated in the selection or editing process reviewed the examinations as participants. Videos included selections from the nasopharynx, oropharynx, hypopharynx, and larynx. None of the endoscopies passed beyond the glottis, although the subglottis was represented in some of the videos. The purpose of editing was to eliminate nondiagnostic portions of the examination at the beginning and the end of the videos that could bias the results. The videos were contiguous examinations except for 1 video that had a long, redundant segment removed. Videos were labeled and sorted in 2 folders with each having 4 videos repeated to determine intrarater reliability. A narrated, educational, 8-minute 17-second video was created using selections from the first set of 21 videos. The narration was coauthored and approved by one of us (J.P.N.) to discuss relevant anatomy, pathologic changes present, and the level of concern for that airway.

Videos were administered from August 1 to December 31, 2015. Data analysis was performed using Excel with the Real-Statistics Resource Pack (Microsoft Corp).2 Interrater reliability was assessed with intraclass correlation coefficients (ICCs) for all interns compared with attendings, ED interns compared with attendings, otolaryngology interns compared with attendings, and among the attendings. In addition, interrater ICCs were calculated for pooled vocal cord mobility (left and right vocal cord mobility) because we believed that this represented a single diagnostic skill rather than 2 independent areas of observation.

Results

We generated ICCs to compare the degree of agreement within each training group (ED and otolaryngology interns and attending otolaryngologists) and the degree of agreement between each group of interns and the attending otolaryngologist criterion standard. A total of 13 participants were included. For interpretation, we used reference ICCs of 0.80 to 1.00 for very good agreement, 0.60 to 0.79 for good agreement, 0.40 to 0.59 for moderate agreement, 0.20 to 0.39 for fair agreement, and 0 to 0.19 for poor agreement.

The ICCs for all questions before and after the intervention were generally low for the interns of both groups and higher for the attending group (Table 1 and Table 2). The ICCs for ED interns compared with attendings were generally slightly lower than for the otolaryngology interns. Improvement was noted between the prevideo and postvideo pooled right and left vocal cord mobility scores. The ED interns’ preintervention ICC was 0.25 (95% CI, 0.16-0.39), and the postintervention ICC was 0.44 (95% CI,0.33-0.57). The otolaryngology interns’ preintervention ICC was 0.34 (95% CI, 0.08-0.56), and the postintervention ICC was 0.50 (95% CI, 0.26-0.68). These findings moved both groups from a preintervention agreement of fair to moderate agreement, suggesting a small but real improvement in diagnostic ability for vocal cord mobility. We found minimal improvement in intern scores for base of tongue abnormalities, subglottic stenosis, vocal cord abnormalities, level of comfort, level of concern, pharyngeal abnormalities, or laryngeal, pharyngeal, and hypopharyngeal masses.

Finally, when assessing ICCs after the second viewing of the FTNL videos, we found that all raters demonstrated good to very good agreement. For ED interns, ICCs were 0.78 (95% CI, 0.74-0.81) before and 0.81 (95% CI, 0.77-0.84) after the intervention. For otolaryngology interns, ICCs were 0.83 (95% CI, 0.76-0.88) before and 0.89 (95% CI, 0.84-0.92) after the intervention. For attendings, ICCs were 0.93 (95% CI, 0.91-0.95) before and 0.95 (95% CI, 0.93-0.96) after the intervention.

Discussion

The use of educational tools, including videos, simulation exercises, and online modules, has been increasing for medical education in recent years.3 Many medical specialties, including otolaryngology, general surgery, obstetrics and gynecology, anesthesiology, pediatrics, and emergency medicine, have shown improvements in accuracy and skill and decreases in procedure times with the use of these educational tools.4-7 Previous studies using simulation exercises for fiberoptic FTNL have shown improved skills in trainees performing the procedure on patients, allowing them to focus on other aspects of the examination.4 The relatively new restrictions on resident work hours necessitate more efficient training for surgical residents because of concern for decreased numbers of operative cases by residents after the implementation of the 80-hour work week.8-10 Educational tools designed and administered appropriately may help a resident achieve competency faster, leaving time for other, more time-intensive learning opportunities, such as working in the operating room.

Many additional examples of using simulation or educational tools for resident learning have had positive results. Allak et al11 developed a simulator for performing rigid esophagoscopy. Otolaryngology residents with varied levels of experience showed improved completion of procedural steps, higher Objective Structured Assessment of Technical Skills scores, and decreased force applied to tissue by the esophagoscope after the simulation exercise. Patel et al12 used simulation exercises that included a didactic lecture, procedural video, and simulation of laparoscopic salpingectomy on a porcine cadaver performed by the intervention group compared with a control group who underwent traditional training per routine residency rotations. The intervention group demonstrated improvement in surgical technique Objective Structured Assessment of Technical Skills scores compared with the traditional training group. First-year obstetric residents learned cesarean deliveries through an online multimedia module as detailed in a study by York et al.13 Abdelsattar et al14 had residents and fellows view a set of selected surgical procedure videos offering points for identifying a checklist of items, including visible anatomy, visualizing poor surgical technique, and correctly identifying surgical instruments on the screen. They believed that this tool was a way to identify trainees ready for hands-on training and those who may require additional preemptive education. Other studies have also found that the administration of video tools enhances surgical resident education and skill.13,15

Prior studies16 have also shown that trainees find videos to be useful educational tools. These videos can be designed to be short, specific, and easily accessed to provide succinct, quick, and useful educational material. Levitt et al17 demonstrated that ED attending physicians’ misinterpretation rate of cranial computed tomographic scans was decreased by 4% after a 1-hour educational session on interpreting these types of scans. Rowse et al18 discussed how the current generation of learners, including residents, favor the use of technology to obtain new information. They incorporated that idea into a 90-second video clip administered to residents demonstrating proper fine-needle aspiration biopsy techniques. The viewing of this video increased their residents’ performances of this skill when viewed within 2 weeks before evaluation.

Our data suggest that a short, narrated educational video can be useful in accelerating learning. Overall, the ICCs for the intern group were low, as expected given their lack of training. Improvement was noted between preintervention and postintervention ICCs in regard to pooled right and left vocal fold mobility (overall interns improved from 0.25 [95% CI, 0.16-0.37] to 0.47 [95% CI, 0.36-0.59]), although these results were still lower than attendings’ ICCs for vocal cord immobility (0.89 [95% CI, 0.83-0.93]). Although this specific diagnostic area improved, the intervention group did not reach the diagnostic ability of otolaryngology attendings with many years of experience, and they showed minimal improvement in other areas of the examination. As we have previously shown,1 reliability is poorer in some aspects of FTNL despite completion of training, but this finding also suggests that a simple training video is not a substitute for years of clinical experience.

Clinicians with limited FTNL experience could benefit from viewing educational videos such as this one. An ED physician who does not perform many flexible laryngoscopies or has extended periods between performances could quickly watch this type of video before performing FTNL. The video would provide a quick refresher on common aerodigestive abnormalities for the clinician to keep in mind during his or her examination. The video could also assist the clinician in determining whether the patient has a concerning airway that may necessitate an emergent intervention vs referring for an outpatient evaluation by an otolaryngologist with more experience.

The possibility of expanding the video tool could lead to further improvement. Learning tools focusing on specific upper aerodigestive subsite abnormalities that are rare or more difficult for the novice to identify could increase the trainees’ ability to recognize abnormalities in those areas and is an area open to further research. Additional studies involving emergency medicine attendings and multiple institutions may be useful for further investigation.

Limitations

Limitations of the study include a small sample size and short interval between the participants’ viewing of the first and second sets of videos. A longer interval between administering the sets of videos could help to further investigate learning of the participants rather than risking the possibility of measuring simple recall, although the videos administered in the preintervention and postintervention sessions were different. The authors were surprised, however, at the limited improvement seen overall in ICCs, even with the short interval between administrations. The ICCs for level of comfort with diagnosis by participants were low and even decreased after intervention. This finding could reflect an overall new appreciation for the vast complexity of the upper airway and endoscopic diagnosis with which the trainees were previously unfamiliar. It suggests that a relatively simple training tool, such as the video, does not replace the experience acquired through mentorship and guided clinical learning. Some of these early trainees may also lack the foundational knowledge to assess laryngeal anatomy at a level high enough to recognize normal and abnormal, and they may have benefitted from more extensive formal didactic training on upper airway anatomy.

Conclusions

Diagnostic interpretation of FTNL can be improved with the use of educational tools, including a teaching video of laryngoscopies, especially for vocal cord mobility. Additional experience, however, is needed to attain competence in accurately diagnosing upper airway lesions. This tool is not intended to replace appropriate supervision of trainees by experienced clinicians in the area of flexible laryngoscopy. Trainees and clinicians with limited experience in FTNL may benefit from a tool like the one presented herein in the appropriate setting.

Back to top
Article Information

Corresponding Author: Michael P. Platt, MD, MSc, Department of Otolaryngology–Head and Neck Surgery, Boston University Medical Center, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA 02118 (michael.platt@bmc.org).

Accepted for Publication: January 14, 2017.

Published Online: May 4, 2017. doi:10.1001/jamaoto.2017.0120

Author Contributions: Dr Brook had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Russell, Brook, Platt, Noordzij.

Acquisition, analysis, or interpretation of data: Russell, Brook, Platt, Grillone, Aliphas.

Drafting of the manuscript: Russell, Brook, Platt.

Critical revision of the manuscript for important intellectual content: Brook, Platt, Grillone, Aliphas, Noordzij.

Statistical analysis: Brook.

Administrative, technical, or material support: Russell, Platt, Aliphas.

Study supervision: Brook, Platt, Grillone, Noordzij.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

References
1.
Brook  CD, Platt  MP, Russell  K, Grillone  GA, Aliphas  A, Noordzij  JP.  Time to competency, reliability of flexible transnasal laryngoscopy by training level: a pilot study.  Otolaryngol Head Neck Surg. 2015;152(5):843-850.PubMedGoogle ScholarCrossref
2.
Zaiontz  C. Real statistical analysis using Excel. Real statistical resource pack. http://www.real-statistics.com. Accessed May 2015.
3.
Fung  K.  Otolaryngology–head and neck surgery in undergraduate medical education: advances and innovations.  Laryngoscope. 2015;125(suppl 2):S1-S14.PubMedGoogle ScholarCrossref
4.
Smith  ME, Leung  BC, Sharma  R, Nazeer  S, McFerran  DJ.  A randomized controlled trial of nasolaryngoscopy training techniques.  Laryngoscope. 2014;124(9):2034-2038.PubMedGoogle ScholarCrossref
5.
Nagendran  M, Gurusamy  KS, Aggarwal  R, Loizidou  M, Davidson  BR.  Virtual reality training for surgical trainees in laparoscopic surgery.  Cochrane Database Syst Rev. 2013;8(8):CD006575.PubMedGoogle Scholar
6.
Zendejas  B, Brydges  R, Hamstra  SJ, Cook  DA.  State of the evidence on simulation-based training for laparoscopic surgery: a systematic review.  Ann Surg. 2013;257(4):586-593.PubMedGoogle ScholarCrossref
7.
Okuda  Y, Bryson  EO, DeMaria  S  Jr,  et al.  The utility of simulation in medical education: what is the evidence?  Mt Sinai J Med. 2009;76(4):330-343.PubMedGoogle ScholarCrossref
8.
Kamine  TH, Gondek  S, Kent  TS.  Decrease in junior resident case volume after 2011 ACGME work hours.  J Surg Educ. 2014;71(6):e59-e63.PubMedGoogle ScholarCrossref
9.
Schwartz  SI, Galante  J, Kaji  A,  et al.  Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.  JAMA Surg. 2013;148(9):829-833.PubMedGoogle ScholarCrossref
10.
Scally  CP, Reames  BN, Teman  NR, Fritze  DM, Minter  RM, Gauger  PG.  Preserving operative volume in the setting of the 2011 ACGME duty hour regulations.  J Surg Educ. 2014;71(4):580-586.PubMedGoogle ScholarCrossref
11.
Allak  A, Liu  YE, Oliynyk  MS, Weng  KH, Jameson  MJ, Shonka  DC  Jr.  Development and evaluation of rigid esophagoscopy simulator for residency training.  Laryngoscope. 2016;126(3):616-619.PubMedGoogle ScholarCrossref
12.
Patel  NR, Makai  GE, Sloan  NL, Della Badia  CR.  Traditional vs simulation resident surgical laparoscopic salpingectomy training: a randomized controlled trial.  J Minim Invasive Gynecol. 2016;23(3):372-377.Google ScholarCrossref
13.
York  SL, Maizels  M, Cohen  E,  et al.  Development and evaluation of cesarean section surgical training using computer-enhanced visual learning.  Med Teach. 2014;36(11):958-964.PubMedGoogle ScholarCrossref
14.
Abdelsattar  JM, Pandian  TK, Finnesgard  EJ,  et al.  Do you see what I see? how we use video as an adjunct to general surgery resident education.  J Surg Educ. 2015;72(6):e145-e150.PubMedGoogle ScholarCrossref
15.
van Det  MJ, Meijerink  WJHJ, Hoff  C, Middel  LJ, Koopal  SA, Pierie  JPEN.  The learning effect of intraoperative video-enhanced surgical procedure training.  Surg Endosc. 2011;25(7):2261-2267.PubMedGoogle ScholarCrossref
16.
Hayden  EL, Seagull  FJ, Reddy  RM.  Developing an educational video on lung lobectomy for the general surgery resident.  J Surg Res. 2015;196(2):216-220.PubMedGoogle ScholarCrossref
17.
Levitt  MA, Dawkins  R, Williams  V, Bullock  S.  Abbreviated educational session improves cranial computed tomography scan interpretations by emergency physicians.  Ann Emerg Med. 1997;30(5):616-621.PubMedGoogle ScholarCrossref
18.
Rowse  PG, Ruparel  RK, AlJamal  YN, Abdelsattar  JM, Heller  SF, Farley  DR.  Catering to millennial learners: assessing and improving fine-needle aspiration performance.  J Surg Educ. 2014;71(6):e53-e58.PubMedGoogle ScholarCrossref
×