For more detailed goals and objectives, see the eAppendix in the Supplement.
eAppendix. Massachusetts Eye and Ear Infirmary (MEEI) PGY-4 Resident Glaucoma Rotation Goals and Objectives
eTable 1. Modified GRASIS for Resident Self-Evaluation
eTable 2. Modified GRASIS for Chief Resident Evaluation
Shen LQ, Kloek CE, Turalba AV. Assessing the Effect of a Glaucoma Surgical Curriculum in Resident Physicians. JAMA Ophthalmol. 2015;133(9):1077-1080. doi:10.1001/jamaophthalmol.2015.1846
Subspecialty surgical training is an important part of resident education. We changed the glaucoma rotation in which postgraduate year 4 residents worked with multiple attending physicians with varying teaching styles to a structured surgical curriculum led by 2 dedicated preceptors, and we evaluated the effect on residents’ surgical performance prospectively.
A curriculum consisting of preoperative training, intraoperative teaching, postoperative feedback, and repetition was implemented for postgraduate year 4 residents between July 2, 2012, and June 30, 2014. In a class of 8 residents per year, the mean (SD) glaucoma surgical volume increased from 8.9 (0.8) cases in the prior year to 13.6 (2.5) in 2013 (mean difference, 4.8 cases; 95% CI, 2.4-7.1; P = .001) and 14.8 (4.2) in 2014 (mean difference, 5.9 cases; 95% CI, 2.1-9.6; P = .007). A self-assessment survey showed improvement in suturing (scores for each section range from 1 [worst] to 5 [best]; mean rating, 3.9 in the prior year vs 4.4 in 2013 [P = .04] and 4.5 in 2014 [P = .02]). A validated survey assessing overall surgical competency revealed improvement in handling adverse events (mean rating, 4.1 in the prior year vs 5.0 for both 2013 and 2014; both P < .001).
Conclusions and Relevance
Despite the small sample size and nonrandomized study design, these data suggest that a structured surgical curriculum has advantages in teaching subspecialty surgery and might be considered by other ophthalmology training programs.
Prior to July 2012, postgraduate year 4 (PGY-4) resident physicians in the Harvard Medical School Residency Program in Ophthalmology had a 7-week glaucoma rotation when they worked with multiple glaucoma specialists in the operating room performing trabeculectomies and aqueous shunt procedures according to the individual styles of the attending physician. Residents often were unable to participate in perioperative evaluations of the patients because of their clinical assignment with other physicians. There was also concurrent teaching of a glaucoma fellow alongside the resident in the operating room. The lack of a cohesive resident curriculum and the inconsistent experience during the glaucoma rotation impaired the attending physician’s ability to follow residents’ surgical progress and led to dissatisfaction from both the trainees and the faculty.
In response to these concerns, a structured glaucoma surgical curriculum (Figure) was developed by 2 attending physicians and implemented on July 2, 2012. In addition to surgical training, the residents worked with the same 2 physicians in clinic to participate in the perioperative care of patients with glaucoma to fulfill broader teaching objectives (eAppendix in the Supplement).
We conducted a study to assess the impact of this new curriculum. We hypothesized that the structured curriculum would improve the resident’s training experience by increasing the number of primary glaucoma cases, improving knowledge of glaucoma procedures, and enhancing ophthalmic surgical skills.
A prospective, nonrandomized study to assess the impact of this new curriculum was initiated on June 4, 2012, and received exemption from the institutional review board at Massachusetts Eye and Ear Infirmary (MEEI). The surgical numbers were collected via self-reported Accreditation Council for Graduate Medical Education surgical logs. To assess resident surgical skills, we modified the validated survey Global Rating Assessment of Skills in Intraocular Surgery1 for glaucoma operations (eTable 1 in the Supplement). The PGY-4 residents completed this self-assessment survey at graduation. In addition, the PGY-4 residents from July 2, 2012, to June 30, 2014, were surveyed immediately prior to their glaucoma rotation to assess their baseline knowledge. To evaluate the resident’s overall surgical ability at graduation, a similar survey (eTable 2 in the Supplement) was administered to the 3 MEEI chief residents from June 4, 2012, to June 30, 2014. The MEEI chief residents are full-time faculty members who just completed residency, direct the eye trauma service, and are responsible for supervising PGY-4 residents in the operating room for open globe repairs and cataract operations. Data were summarized as mean (standard deviation) unless described otherwise. The comparisons between the class prior to the curriculum change and the 2 classes afterward were performed with 2-sided t test of unequal variance, while the self-assessments before and after the glaucoma rotation were analyzed with paired t test.
In a class of 8 residents per year, the mean glaucoma surgical volume increased after the curriculum change from a mean (SD) of 8.9 (0.8) trabeculectomies and aqueous shunt procedures to 13.6 (2.5) in 2013 (mean difference, 4.8 cases; 95% CI, 2.4-7.1; P = .001) and 14.8 (4.2) in 2014 (mean difference, 5.9 cases; 95% CI, 2.1-9.6; P = .007) (Table 1). The mean (SD) t distribution–based percentage of increase of surgical volume was 53.5% (11.3%) in 2013 (95% CI, 29.2%-77.8%) and 66.2% (18.1%) in 2014 (95% CI, 27.4%-105.0%). The breakdown showed an increase in aqueous shunt procedures in 2013 (mean difference, 2.3 aqueous shunt procedures; 95% CI, 0.1-4.4; P = .04) and in trabeculectomies in 2014 (mean difference, 4.4 trabeculectomies; 95% CI, 1.8-7.0; P = .003). Of the 24 residents surveyed, 23 (95.8%) completed the self-evaluation at graduation (eTable 1 in the Supplement). After the curriculum change, there was sustained improvement in knowledge of aqueous shunt procedures (scores for each section range from 1 [worst] to 5 [best]; mean rating, 3.9 in the prior year vs 4.6 in 2013 [P = .04] and 4.6 in 2014 [P = .02]) and suturing skills (mean rating, 3.9 in the prior year vs 4.4 in 2013 [P = .04] and 4.5 in 2014 [P = .02]). The self-evaluations performed by the 2013 and 2014 residents before and after the glaucoma rotation showed progress of all categories, except for use of the nondominant hand in the 2013 class.
For nonglaucoma cases, the surveys from chief residents showed improvement after the curriculum change in the following categories (Table 2): preoperative planning (mean rating, 4.4 in the prior year vs 4.9 in 2013 [P = .04] and 5.0 in 2014 [P = .01]), knowledge of procedure (mean rating, 4.3 in the prior year vs 4.9 in 2013 [P = .01] and 5.0 in 2014 [P = .003]), handling of unexpected or adverse events (mean rating, 4.1 in the prior year vs 5.0 in 2013 [P < .001] and 5.0 in 2014 [P < .001]), and use of the nondominant hand (mean rating, 4.0 in the prior year vs 4.8 in 2013 [P = .01]).
Subspecialty surgical training is a significant element in resident education. Challenges in developing a glaucoma curriculum include training resident physicians to perform subspecialty operations in a limited time and exposing them to longitudinal patient care. The attending physicians dedicated to resident education designed a schedule that facilitated training of residents in the surgical and perioperative care of the patient with glaucoma, while allowing time for feedback, repetition, and reinforcement. This structure encouraged open communication between the attending physician and trainee as well as consistent effort from the resident physicians.
Structured surgical curricula and assessment tools have been proposed to help standardize and improve ophthalmology resident education.2- 4 Using survey methods to assess educational outcomes, one study demonstrated benefits of a stepwise approach to teaching cataract surgery.3 In subspecialty areas, one group reported advantages of a specific checklist tool used in teaching strabismus surgery to residents.4 In glaucoma, resident-performed operations were mainly assessed by patient outcomes rather than performance evaluation.5,6 We did not explore patient outcomes in this study because the postoperative care, which influences success in glaucoma surgery, is actively managed by the attending physician at MEEI. Instead, we compared surgical volume, self-evaluation by resident physicians, and standardized evaluation by nonglaucoma surgeons to demonstrate a favorable effect of a structured surgery curriculum on resident education.
In addition to increased glaucoma surgical numbers, the resident self-evaluations confirmed other positive effects of the curriculum such as improved suturing skills. Although the timing of the rotation and the short follow-up might have prevented an impact on residents’ subspecialty choice, the glaucoma faculty observed that the residents performed consistently regardless of their subspecialty interest or their natural surgical talent, suggesting that this curriculum can decrease variability in trainee experiences.7
A survey validated for evaluating a resident’s surgical competencies at our institution, the Global Rating Assessment of Skills in Intraocular Surgery,1 was used to assess the effects on overall surgical training. As shown in eTable 2 in the Supplement, the survey has descriptions for each level of surgical competency, which helps to compare trainees more objectively. The response from the chief residents showed improvement in surgical preparedness, knowledge, ability to handle adverse events, and operative skills with the nondominant hand, consistent with increased experience of the resident surgeons.
There are several limitations to the study. The small sample size makes it challenging to draw conclusions about the ultimate impact of this curriculum. The survey method is associated with subjectivity and bias, although this effect was lessened with a validated survey and confirmation of findings from 2013 with those from 2014. The consecutive, nonrandomized data are prone to confounding factors such as other simultaneous teaching, which can account for differences in resident educational outcomes. Furthermore, this study represents the experience at our institution only; additional research at other institutions is needed to validate our findings.
In summary, these data suggest that a structured surgical curriculum has advantages in teaching subspecialty surgery and might be considered by other ophthalmology training programs, especially if the results can be confirmed confidently in future studies.
Corresponding Author: Lucy Q. Shen, MD, Department of Ophthalmology, Harvard Medical School, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA 02114 (email@example.com).
Submitted for Publication: February 15, 2015; final revision received April 19, 2015; accepted April 23, 2015.
Published Online: June 18, 2015. doi:10.1001/jamaophthalmol.2015.1846.
Author Contributions: Dr Shen had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Shen, Turalba.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Shen.
Obtained funding: Shen.
Administrative, technical, or material support: All authors.
Study supervision: Shen, Kloek.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: This study was supported by the Harvard Glaucoma Center of Excellence, the Glaucoma Service, and the Residency Committee at Massachusetts Eye and Ear Infirmary.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentation: This work was presented in part at the 24th Annual Meeting of the American Glaucoma Society; March 1, 2014; Washington, DC.
Additional Contributions: Louis R. Pasquale, MD, Department of Ophthalmology, Harvard Medical School, Massachusetts Eye and Ear Infirmary, Boston, contributed to review of the manuscript; he did not receive any compensation. Hang Lee, PhD, Massachusetts General Hospital, Biostatistics Center, Boston, provided assistance in statistical analysis through Harvard Catalyst, The Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health grant UL1 TR001102); he did not receive any compensation.