Utility of Combining a Simulation-Based Method With a Lecture-Based Method for Fundoscopy Training in Neurology Residency

Key Points

Question  Is a simulation-based teaching method of assessment and teaching of the fundoscopic examination valuable in neurology residency training?

Findings  In this education research study of 48 neurology residents, the intervention group, who received additional simulation-based training, had significantly greater increases in fundoscopy skills and practice confidence compared with the control group, who only received lecture-based training.

Meaning  Use of a simulation-based method may be an effective adjunct to the conventional didactics-based method in fundoscopy education in neurology.

Importance  Fundoscopic examination is an essential component of the neurologic examination. Competence in its performance is mandated as a required clinical skill for neurology residents by the American Council of Graduate Medical Education. Government and private insurance agencies require its performance and documentation for moderate- and high-level neurologic evaluations. Traditionally, assessment and teaching of this key clinical examination technique have been difficult in neurology residency training.

Objective  To evaluate the utility of a simulation-based method and the traditional lecture-based method for assessment and teaching of fundoscopy to neurology residents.

Design, Setting, and Participants  This study was a prospective, single-blinded, education research study of 48 neurology residents recruited from July 1, 2015, through June 30, 2016, at a large neurology residency training program. Participants were equally divided into control and intervention groups after stratification by training year. Baseline and postintervention assessments were performed using questionnaire, survey, and fundoscopy simulators.

Interventions  After baseline assessment, both groups initially received lecture-based training, which covered fundamental knowledge on the components of fundoscopy and key neurologic findings observed on fundoscopic examination. The intervention group additionally received simulation-based training, which consisted of an instructor-led, hands-on workshop that covered practical skills of performing fundoscopic examination and identifying neurologically relevant findings on another fundoscopy simulator.

Main Outcomes and Measures  The primary outcome measures were the postintervention changes in fundoscopy knowledge, skills, and total scores.

Results  A total of 30 men and 18 women were equally distributed between the 2 groups. The intervention group had significantly higher mean (SD) increases in skills (2.5 [2.3] vs 0.8 [1.8], P = .01) and total (9.3 [4.3] vs 5.3 [5.8], P = .02) scores compared with the control group. Knowledge scores (6.8 [3.3] vs 4.5 [4.9], P = .11) increased nonsignificantly in both groups.

Conclusions and Relevance  This study supports the use of a simulation-based method as a supplementary tool to the lecture-based method in the assessment and teaching of fundoscopic examination in neurology residency.

Fundoscopic examination is a physical examination technique that allows visualization of the retina, a direct extension of central nervous system, by using only a fundoscope (ophthalmoscope) and the naked eye.¹ Although this clinical skill involves only pure inspection, it can help differentiate and diagnose a variety of emergency and nonemergency neurologic diseases.² Every medical student is expected to learn fundoscopy during their neurology rotation as part of the neurologic examination of patients.³ Neurology residents are expected to be competent in accurately visualizing papilledema during fundoscopy as a level 3 milestone requirement for graduation⁴ under the new evaluation system of the Accreditation Council for Graduate Medical Education (ACGME). Of importance, fundoscopic examination is an essential component of a complete neurologic examination from the standpoints of comprehensive patient care and required documentation for billing⁵ in clinical practice.

Historically, fundoscopy has been difficult to teach and learn because of the inherent limitation of not having direct and simultaneous confirmation of the student’s findings with the teacher’s assessment, leading to its underuse in clinical medicine.⁶ This problem has been further compounded in clinical neurology owing to the changing landscape of residency training, namely, the increased patient load and restricted duty hour requirements.⁷ For these reasons, the assessment and teaching of this important clinical skill have proven to be difficult for many neurology training programs and thus have largely been left to the trainee to practice and master.

There is an unmet need of developing and implementing novel training methods to fulfill this educational gap in neurologic training. There is ample literature that supports the notion that simulation can help build and reinforce competence in a variety of physical examination skills in medical education,⁸ including fundoscopic examination.^9-11 However, the implementation and utility of such simulation-based methods in this context have not been reported, to our knowledge, in the literature.

We postulated that the fundoscopy training needs of neurology residents are not fully met with the current paradigm of lecture-based teaching followed by unguided practice directly on patients. We hypothesized that the fundoscopic examination knowledge and skills of neurology trainees can be objectively assessed and improved with the use of a nontraditional, simulation-based teaching method in addition to lecture-based teaching. We tested our hypothesis by conducting an education research study of these complementary methods of teaching fundoscopic examination in a large neurology residency training program in the United States.

The study protocol and informed consent form were approved by the University Hospitals Cleveland Medical Center Institutional Review Board of our hospital. All residents were invited to participate in the study from July 1, 2015, through June 30, 2016, on a completely voluntary basis and given a written description of study aims and structure. All participants provided written informed consent and were deidentified using study identification numbers, which were generated and maintained by an independent departmental office, for masking, data collection, and analysis and to protect the anonymity of study participants.

We conducted a prospective, single-blind education research study by enrolling a cohort of neurology and pediatric neurology residents at a large, tertiary academic medical center. For the assessment of residents’ fundoscopic examination knowledge and skills, we designed a questionnaire based on the principle of relevance to the clinical practice of neurology and several years of clinical and teaching experience of the principal investigator (D.C.P.) as a professor and program director in neurology. This questionnaire (eAppendixes 1, 2, and 3 in the Supplement) consisted of 50 questions: 35 knowledge-based questions, which included 15 multiple-choice questions on the basic neuroanatomy, neurophysiology, and components of an ophthalmoscope and 20 image-based questions on findings observed on fundoscopic examination in healthy individuals and neurologic diseases, followed by a skills test that included identification of 15 neurologically relevant fundoscopic findings, including papilledema, optic atrophy, and central retinal artery occlusion, on a fundoscopy simulator using an actual ophthalmoscope. For image-based questions, a resident could receive 0.5 point if the answer was partially correct or 1 point if the answer was fully correct. Total score (range, 0-50, with 0 being the lowest score and 50 being the highest) was calculated by the sum of the knowledge score (range, 0-35, with 0 being the lowest score and 35 being the highest) and skills score (range, 0-15, with 0 being the lowest score and 15 being the highest). Primary outcome measures were defined as postintervention changes in the 3 different test scores, namely, knowledge score, skills score, and total scores. We also designed a 5-question survey (eAppendix 4 in the Supplement) for recording a resident’s self-reported attitude and practice patterns of fundoscopy in clinical practice on a scale of 1 to 10, with 0 being the lowest score and 10 being the highest. Secondary outcome measures were defined as the postintervention changes in the score for each of the 5 items on the survey.

To select the fundoscopy simulators to be used in the study, we tested different products available from 3 different companies (Adam,Rouilly Limited, OpthaSim, and Kyoto). The ease of use and lower cost were the primary factors that led us to choose the Eye Retinopathy Trainer (Adam,Rouilly Limited). The Eye Retinopathy Trainer had previously been applied in teaching fundoscopy in ophthalmology and was found to be an effective teaching tool for improvement and practice of fundoscopy among fourth-year medical students.¹² In addition, the availability of 2 different models, namely, AR303 (which had physical slides) (Video 1) and AR403 (which had electronic slides) (Video 2), that contained different actual images for the same clinical findings was a factor in the context of study design of the present study.

The control and test groups were given clear instructions on different components of the test procedure, including how to use the fundoscopy simulator before it was administered. After the baseline assessment, participants were equally allocated after stratification by postgraduation year (PGY) of training to control and intervention groups based on whether they received additional simulation-based training or not, using a free and validated web-based allocation tool.¹³

The control and intervention groups received lecture-based training in the form of 2 video lectures that covered basic principles of fundoscopy and fundoscopic examination findings important in neurologic practice through an easy-to-access online platform (ie, YouTube; http://bit.ly/fundoscopyinneurology). All residents were requested to notify the investigators in writing of the completion of these didactics and understanding the material. Subsequently, control group participants were free to continue the unguided practice of fundoscopy in routine clinical care. As part of the study intervention, intervention group residents participated in simulation-based training, which was an instructor-led, 1-hour, hands-on workshop session (teacher to student ratio of 1:4) that covered practical skills and aspects of performing the fundoscopic examination and identifying important fundoscopic examination findings in neurologic practice on the AR403 eye simulator (video recording of a typical session is available at the aforementioned URL).

After a minimum gap of 3 months from completion of the intervention, each resident took a postintervention test, which was identical to the one used in preintervention assessment. All test items were scored by 2 of us (D.K.G., N.K.) masked to maintain consistency and reduce errors.

Data were managed with Microsoft Excel (Microsoft Inc) and SPSS software (SPSS Inc). Sample size was predetermined because the study population consisted of a convenient sample of a fixed number of neurology residents in the residency program. A 2-sample, 2-sided t test and 1-way analysis of variance were used to check for the baseline differences in test scores between the 2 groups and by the PGY level, respectively. The Mann-Whitney test and Kruskal-Wallis test were used to check for baseline differences in survey scores between the 2 groups and by the PGY level, respectively. The nonparametric Mann-Whitney test was used to test for significant differences in primary and secondary outcome measures between the control and intervention groups. Significance testing was performed at P<.05.

We recruited a total of 48 of 50 available participants; 2 of the residents were the study investigators (D.K.G., N.K.). There were equal numbers of participants in the control (n = 24) and intervention (n = 24) groups. There were no missing data. The control and intervention groups had similar characteristics at baseline, with no significant differences in knowledge, skills, or total scores (Table 1). When compared by the PGY level, there was an expected trend of progressive increase in all 3 test scores from PGY1 to PGY4 (Table 2). Similar characteristics and trend in baseline scores were observed for each of the 5 survey scores in the 2 groups and by PGY level analyses, respectively (eTable 1 and eTable 2 in the Supplement).

There was an increase in all 3 primary outcome measures in the control and intervention groups (Figure 1). However, compared with the control group, the intervention group had significantly higher mean (SD) increases in skills score (2.5 [2.3] vs 0.8 [1.8], P = .01) and total score (9.3 [4.3] vs 5.3 [5.8], P = .02), whereas the increase in knowledge score was not significantly different (6.8 [3.3] vs 4.5 [4.9], P = .11).

A similar overall increase in secondary outcome measures (Figure 2) was observed in the control and intervention groups. However, compared with the control group, the intervention group had significantly higher increases in reported comfort level in attempting fundoscopy (3.0 vs 0.8, P < .001), self-reported success in visualizing the retina (2.0 vs 0.6, P = .03), and confidence in describing findings (3.1 vs 0.6, P < .001) after the study intervention. The self-reported frequency of attempting fundoscopy and opinion on the usefulness of the fundoscopy simulator were not different between the 2 groups (eTable 1 and eTable 2 in the Supplement).

In the present study, we provide novel evidence in support of using a simulation-based method, in addition to the conventional lecture-based method, for teaching fundoscopic examination to trainees in neurology residency. Our study also provides evidence that such an approach may improve the attitudes and comfort level of neurology residents in the use of this physical examination technique in clinical practice. Our study has potential relevance for improving clinical outcomes because key fundoscopic findings can enable early detection and appropriate management of certain neurologic disorders. Neurology residents and practicing neurologists need to be able to recognize critical fundoscopic findings, such as papilledema, central retinal artery occlusion, Hollenhorst plaques, and subhyaloid hemorrhages, that may necessitate appropriate emergency interventions for management of stroke, bleeding, and increased risk of brain herniation. However, fundoscopic examination performed by a well-trained neurologist is not as comprehensive as that of an ophthalmologist. Accordingly, ophthalmologic consultations will continue to provide more detailed and specific fundoscopic evaluation as part of standard and complete neurologic care to confirm abnormal findings and diagnose other primary ophthalmologic conditions beyond the expertise of a neurologist.

We also demonstrated the feasibility of assessing fundoscopic examination knowledge and skills of neurology residents in an objective manner by using a simulation-based method, which may be useful in the context of milestone-based assessment of residents’ competency under the new ACGME evaluation system. Finally, we developed a semiformal curriculum and open access multimedia resource for teaching fundoscopic examination in neurology that can be used by medical schools and residency training programs in other institutions.

A key strength of our study was the use of a relatively large sample size of participants from a large neurology residency training program for testing of a specific hypothesis in a prospective, blinded fashion. We also used 2 similar models of fundoscopy simulator and a minimum gap of 3 months between assessments and intervention to minimize the potential learning effect in the intervention group.

One limitation of our study was the use of a nonvalidated instrument questionnaire and survey for development of the outcome measures. The use of this questionnaire was unavoidable because of the lack of any previously existing instruments for the present context. However, these instruments seemed to capture the intended effects, as demonstrated by the expected findings of increasing test and survey scores from PGY1 to PGY4 (eTable 1 and eTable 2 in the Supplement). Another limitation was the relatively low absolute test scores, especially the skills score, which might be reflective of an overly difficult level of test questions.

Future directions for this educational research work include the assessment of the potential effect of the study’s intervention in improving patient outcomes and implementation of the curriculum and assessment methods in different neurology training programs of diverse sizes and from different locations in the United States. Development of validated instruments seems to be unlikely to occur because there is little equipoise that simulation-based training can be helpful in teaching fundoscopy; however, availability of these instruments will be valuable in future studies.

Corresponding Author: David C. Preston, MD, Department of Neurology, University Hospitals Cleveland Medical Center, Case Western Reserve University, 11100 Euclid Ave, Bolwell Fifth Floor, Room 5120, Cleveland, OH 44106 (david.preston@uhhospitals.org).

Accepted for Publication: May 30, 2017.

Published Online: September 11, 2017. doi:10.1001/jamaneurol.2017.2073

Author Contributions: Dr Preston had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Gupta, Khandker, Tatsuoka, Preston.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Gupta, Khandker, Tatsuoka, Preston.

Statistical analysis: Gupta, Tatsuoka.

Obtained funding: Gupta, Khandker, Tatsuoka.

Administrative, technical, or material support: Gupta, Khandker, Stacy.

Study supervision: Khandker, Preston.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was funded by an educational research grant from the American Academy of Neurology.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

Additional Contributions: Rebecca Kahl, MBA, Department of Neurology, University Hospitals Cleveland Medical Center, Cleveland, Ohio, supported the management of the grant, and Massimo Marano, MD, IRCCS Casa Sollievo della Sofferenza Hospital, Mendel Institute of Human Genetics, Rome, Italy, provided valuable input in the review of the manuscript. They were not compensated for their work.