Intense Simulation-Based Surgical Education for Manual Small-Incision Cataract Surgery: The Ophthalmic Learning and Improvement Initiative in Cataract Surgery Randomized Clinical Trial in Kenya, Tanzania, Uganda, and Zimbabwe | Cataract and Other Lens Disorders | JAMA Ophthalmology | JAMA Network
[Skip to Navigation]
Figure 1.  Trial Flowchart
Trial Flowchart

The control group received the allocated intervention after an initial follow-up period of 1 year.

Figure 2.  Surgical Competency Scores by Group
Surgical Competency Scores by Group

Arrows indicate training intervention for intervention and control groups. Sample sizes for groups at baseline are intervention, 25 and control, 24; at 3 months are intervention, 25 and control, 24; at 1 year are intervention, 22 and control, 23; and at 15 months are intervention, 10 and control, 16. Scores are of a possible total of 40. Sim-OSSCAR indicates Ophthalmic Simulation Surgical Competency Assessment Rubric.

Figure 3.  Linear Regression of Number of Posterior Capsule Ruptures and Number of Cataract Surgical Procedures by Group
Linear Regression of Number of Posterior Capsule Ruptures and Number of Cataract Surgical Procedures by Group

Data from 21 intervention and 16 control trainees. Shaded areas indicate 95% CIs.

Table 1.  Baseline Characteristics of the Intention-to-Treat Population
Baseline Characteristics of the Intention-to-Treat Population
Table 2.  Objective Evaluation of SICS Sim-OSSCAR Scores at Baseline and 3 Months
Objective Evaluation of SICS Sim-OSSCAR Scores at Baseline and 3 Months
1.
Flaxman  SR, Bourne  RRA, Resnikoff  S,  et al; Vision Loss Expert Group of the Global Burden of Disease Study.  Global causes of blindness and distance vision impairment 1990-2020: a systematic review and meta-analysis.   Lancet Glob Health. 2017;5(12):e1221-e1234. doi:10.1016/S2214-109X(17)30393-5 PubMedGoogle ScholarCrossref
2.
Wang  W, Yan  W, Fotis  K,  et al.  Cataract surgical rate and socioeconomics: a global study.   Invest Ophthalmol Vis Sci. 2016;57(14):5872-5881. doi:10.1167/iovs.16-19894 PubMedGoogle ScholarCrossref
3.
Weiser  TG, Haynes  AB, Molina  G,  et al.  Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes.   Lancet. 2015;385(suppl 2):S11. doi:10.1016/S0140-6736(15)60806-6 PubMedGoogle ScholarCrossref
4.
Brown  MM, Brown  GC, Lieske  HB, Lieske  PA.  Financial return-on-investment of ophthalmic interventions: a new paradigm.   Curr Opin Ophthalmol. 2014;25(3):171-176. doi:10.1097/ICU.0000000000000040 PubMedGoogle ScholarCrossref
5.
Horton  S, Gelband  H, Jamison  D, Levin  C, Nugent  R, Watkins  D.  Ranking 93 health interventions for low- and middle-income countries by cost-effectiveness.   PLoS One. 2017;12(8):e0182951. doi:10.1371/journal.pone.0182951 PubMedGoogle Scholar
6.
Resnikoff  S, Lansingh  VC, Washburn  L,  et al.  Estimated number of ophthalmologists worldwide (International Council of Ophthalmology update): will we meet the needs?   Br J Ophthalmol. 2020;104(4):588-592. doi:10.1136/bjophthalmol-2019-314336PubMedGoogle ScholarCrossref
7.
Dean  W, Gichuhi  S, Buchan  J,  et al.  Survey of ophthalmologists-in-training in Eastern, Central and Southern Africa: a regional focus on ophthalmic surgical education.   Wellcome Open Res. 2019;4:187. doi:10.12688/wellcomeopenres.15580.1 PubMedGoogle ScholarCrossref
8.
Young  AL, Jhanji  V, Liang  Y,  et al.  A survey of perceived training differences between ophthalmology residents in Hong Kong and China.   BMC Med Educ. 2015;15:158. doi:10.1186/s12909-015-0440-0 PubMedGoogle ScholarCrossref
9.
Martinerie  L, Rasoaherinomenjanahary  F, Ronot  M,  et al.  Health care simulation in developing countries and low-resource situations.   J Contin Educ Health Prof. 2018;38(3):205-212. doi:10.1097/CEH.0000000000000211 PubMedGoogle ScholarCrossref
10.
Issenberg  SB, McGaghie  WC, Petrusa  ER, Lee Gordon  D, Scalese  RJ.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.   Med Teach. 2005;27(1):10-28. doi:10.1080/01421590500046924 PubMedGoogle ScholarCrossref
11.
Alaker  M, Wynn  GR, Arulampalam  T.  Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis.   Int J Surg. 2016;29:85-94. doi:10.1016/j.ijsu.2016.03.034 PubMedGoogle ScholarCrossref
12.
Thomsen  AS, Subhi  Y, Kiilgaard  JF, la Cour  M, Konge  L.  Update on simulation-based surgical training and assessment in ophthalmology: a systematic review.   Ophthalmology. 2015;122(6):1111-1130.e1. doi:10.1016/j.ophtha.2015.02.028PubMedGoogle ScholarCrossref
13.
Feudner  EM, Engel  C, Neuhann  IM, Petermeier  K, Bartz-Schmidt  KU, Szurman  P.  Virtual reality training improves wet-lab performance of capsulorhexis: results of a randomized, controlled study.   Graefes Arch Clin Exp Ophthalmol. 2009;247(7):955-963. doi:10.1007/s00417-008-1029-7 PubMedGoogle ScholarCrossref
14.
Daly  MK, Gonzalez  E, Siracuse-Lee  D, Legutko  PA.  Efficacy of surgical simulator training versus traditional wet-lab training on operating room performance of ophthalmology residents during the capsulorhexis in cataract surgery.   J Cataract Refract Surg. 2013;39(11):1734-1741. doi:10.1016/j.jcrs.2013.05.044 PubMedGoogle ScholarCrossref
15.
McCannel  CA, Reed  DC, Goldman  DR.  Ophthalmic surgery simulator training improves resident performance of capsulorhexis in the operating room.   Ophthalmology. 2013;120(12):2456-2461. doi:10.1016/j.ophtha.2013.05.003 PubMedGoogle ScholarCrossref
16.
Ferris  JD, Donachie  PH, Johnston  RL, Barnes  B, Olaitan  M, Sparrow  JM.  Royal College of Ophthalmologists’ National Ophthalmology Database study of cataract surgery: report 6: the impact of EyeSi virtual reality training on complications rates of cataract surgery performed by first and second year trainees.   Br J Ophthalmol. 2020;104(3):324-329. doi:10.1136/bjophthalmol-2018-313817PubMedGoogle ScholarCrossref
17.
Staropoli  PC, Gregori  NZ, Junk  AK,  et al.  Surgical simulation training reduces intraoperative cataract surgery complications among residents.   Simul Healthc. 2018;13(1):11-15. doi:10.1097/SIH.0000000000000255PubMedGoogle Scholar
18.
Rothschild  P, Richardson  A, Franzco  JB, Franzco  RC.  Does virtual reality simulation training result in fewer real-life cataract surgery complications? A systematic literature review.   J Cataract Refract Surg. 2020. Published online July 13, 2020. doi:10.1097/j.jcrs.0000000000000323PubMedGoogle Scholar
19.
Dean  WH, Murray  NL, Buchan  JC, Golnik  K, Kim  MJ, Burton  MJ.  Ophthalmic Simulated Surgical Competency Assessment Rubric for manual small-incision cataract surgery.   J Cataract Refract Surg. 2019;45(9):1252-1257. doi:10.1016/j.jcrs.2019.04.010 PubMedGoogle ScholarCrossref
20.
Peyton  JW.  Teaching and Learning in Medical Practice. Manticore Europe 1998:174-177.
21.
Ericsson  KA, Krampe  RT, Tesch-Romer  C.  The role of deliberate practice in the acquisition of expert performance.   Psychological Rev. 1993;100(3):363-406. doi:10.1037/0033-295X.100.3.363 Google ScholarCrossref
22.
Phillips Studio. Ophthalmic simulated surgery. Accessed February 1, 2020. http://www.phillipsstudio.co.uk/
23.
Arora  S, Aggarwal  R, Sirimanna  P,  et al.  Mental practice enhances surgical technical skills: a randomized controlled study.   Ann Surg. 2011;253(2):265-270. doi:10.1097/SLA.0b013e318207a789 PubMedGoogle ScholarCrossref
24.
Schön  DA.  The Reflective Practitioner: How Professionals Think in Action. Temple Smith; 1983.
25.
Golnik  KC, Beaver  H, Gauba  V,  et al.  Cataract surgical skill assessment.   Ophthalmology. 2011;118(2):427.e1-427.e5. doi:10.1016/j.ophtha.2010.09.023PubMedGoogle ScholarCrossref
26.
Landis  JR, Koch  GG.  The measurement of observer agreement for categorical data.   Biometrics. 1977;33(1):159-174. doi:10.2307/2529310 PubMedGoogle ScholarCrossref
27.
International Council of Ophthalmology. ICO residency curriculum. Accessed August 20, 2020. http://www.icoph.org/refocusing_education/curricula.html
28.
Ramani  S, Pradeep  TG, Sundaresh  DD.  Effect of wet-laboratory training on resident performed manual small-incision cataract surgery.   Indian J Ophthalmol. 2018;66(6):793-797. doi:10.4103/ijo.IJO_1041_17 PubMedGoogle ScholarCrossref
29.
McCambridge  J, Witton  J, Elbourne  DR.  Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects.   J Clin Epidemiol. 2014;67(3):267-277. doi:10.1016/j.jclinepi.2013.08.015 PubMedGoogle ScholarCrossref
30.
McGaghie  WC, Issenberg  SB, Petrusa  ER, Scalese  RJ.  A critical review of simulation-based medical education research: 2003-2009.   Med Educ. 2010;44(1):50-63. doi:10.1111/j.1365-2923.2009.03547.x PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    November 5, 2020

    Intense Simulation-Based Surgical Education for Manual Small-Incision Cataract Surgery: The Ophthalmic Learning and Improvement Initiative in Cataract Surgery Randomized Clinical Trial in Kenya, Tanzania, Uganda, and Zimbabwe

    Author Affiliations
    • 1International Centre for Eye Health, Clinical Research Department, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom
    • 2Division of Ophthalmology, Groote Schuur Hospital, University of Cape Town, Cape Town, South Africa
    • 3Department of Ophthalmology, University of Nairobi, Kenyatta National Hospital, Nairobi, Kenya
    • 4Kilimanjaro Christian Medical Centre, Moshi, Tanzania
    • 5Department of Ophthalmology, Parirenyatwa Hospitals, University of Zimbabwe, Harare, Zimbabwe
    • 6Department of Ophthalmology, Makerere University School of Medicine, Kampala, Uganda
    • 7Mbarara University and Referral Hospital Eye Centre, Mbarara University of Science and Technology, Mbarara, Uganda
    • 8Bristol Eye Hospital, Bristol, United Kingdom
    • 9Tropical Epidemiology Group, Faculty of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, United Kingdom
    • 10Makeni Regional Hospital, Bombali, Sierra Leone
    • 11Moorfields Eye Hospital, London, United Kingdom
    JAMA Ophthalmol. 2021;139(1):9-15. doi:10.1001/jamaophthalmol.2020.4718
    Key Points

    Question  Does the addition of simulation-based surgical education to conventional training improve cataract surgical competence among trainees?

    Findings  In this randomized clinical trial, a simulation-based training intervention resulted in an almost 3-fold increase in objectively assessed surgical competence of trainees.

    Meaning  These results support pursuing simulation-based surgical training units, which may lead to safer, more effective, and more efficient surgical skills before trainees progress to conventional live surgical training.

    Abstract

    Importance  Cataracts account for 40% of cases of blindness globally, with surgery the only treatment.

    Objective  To determine whether adding simulation-based cataract surgical training to conventional training results in improved acquisition of surgical skills among trainees.

    Design, Setting, and Participants  A multicenter, investigator-masked, parallel-group, randomized clinical educational-intervention trial was conducted at 5 university hospital training institutions in Kenya, Tanzania, Uganda, and Zimbabwe from October 1, 2017, to September 30, 2019, with a follow-up of 15 months. Fifty-two trainee ophthalmologists were assessed for eligibility (required no prior cataract surgery as primary surgeon); 50 were recruited and randomized. Those assessing outcomes of surgical competency were masked to group assignment. Analysis was performed on an intention-to-treat basis.

    Interventions  The intervention group received a 5-day simulation-based cataract surgical training course, in addition to standard surgical training. The control group received standard training only, without a placebo intervention; however, those in the control group received the intervention training after the initial 12-month follow-up period.

    Main Outcomes and Measures  The primary outcome measure was overall surgical competency at 3 months, which was assessed with a validated competency assessment rubric. Secondary outcomes included surgical competence at 1 year and quantity and outcomes (including visual acuity and posterior capsule rupture) of cataract surgical procedures performed during a 1-year period.

    Results  Among the 50 participants (26 women [52.0%]; mean [SD] age, 32.3 [4.6] years), 25 were randomized to the intervention group, and 25 were randomized to the control group, with 1 dropout. Forty-nine participants were included in the final intention-to-treat analysis. Baseline characteristics were balanced. The participants in the intervention group had higher scores at 3 months compared with the participants in the control group, after adjusting for baseline assessment rubric score. The participants in the intervention group were estimated to have scores 16.6 points (out of 40) higher (95% CI, 14.4-18.7; P < .001) at 3 months than the participants in the control group. The participants in the intervention group performed a mean of 21.5 cataract surgical procedures in the year after the training, while the participants in the control group performed a mean of 8.5 cataract surgical procedures (mean difference, 13.0; 95% CI, 3.9-22.2; P < .001). Posterior capsule rupture rates (an important complication) were 7.8% (42 of 537) for the intervention group and 26.6% (54 of 203) for the control group (difference, 18.8%; 95% CI, 12.3%-25.3%; P < .001).

    Conclusions and Relevance  This randomized clinical trial provides evidence that intense simulation-based cataract surgical education facilitates the rapid acquisition of surgical competence and maximizes patient safety.

    Trial Registration  Pan-African Clinical Trial Registry, number PACTR201803002159198

    Introduction

    Of the 36 million people globally who are blind, more than one-third have blindness due to cataracts.1 Surgery remains the only treatment option for cataracts. An estimated 14 million cataract operations are performed globally annually.2,3 Cataract surgery can effectively restore vision, is one of the safest and most cost-effective of all health care interventions, and confers a large financial return on investment.4,5 However, in many regions, the rate of cataract surgery is insufficient to address the burden of avoidable blindness.

    Of the more than 230 000 ophthalmologists worldwide, the lowest mean number of ophthalmologists per million population is found in sub-Saharan Africa, at 2.5.6 The global estimated mean number of ophthalmologists per million population is 31.7; however, less than half perform cataract surgery (mean number, 14.1 ophthalmologists per million population).6 There is an urgent need to train and equip more ophthalmic surgeons to address the burden of surgically treatable blindness.

    In sub-Saharan Africa, the median number of cataract surgical procedures performed by trainee ophthalmologists in the first 2 years of training was zero.7 In mainland China, the median number of cataract surgical procedures performed by senior trainees by the end of 3 years of training was zero.8 Traditional surgical education is resource intensive. Slowly building surgical competence through trial and error by practicing solely on patients is unethical, and maximizing patient safety and reducing surgical errors must be priorities. Simulation-based education can help address this training need, especially in low-income settings where the disease magnitude is greatest.9

    Intensive simulation-based surgical education has been shown to increase surgical skills and decrease complication rates.10 During the past 10 years, randomized clinical trials (RCTs) have been conducted for surgical education, predominantly in laparoscopic surgery.11 The literature on simulation-based surgical education in eye care, however, is inadequate, despite widespread adoption and large expenditure.12

    Many animal, cadaver, artificial, and virtual reality models have been used in ophthalmic surgical education, including for cataracts.12-15 Retrospective studies have shown a reduction in complication rates with access to, and mandatory training using, a virtual reality simulator for cataract surgery training.16,17 Recent systematic reviews of trials involving simulation-based training or assessment of ophthalmic surgical skills concluded that studies are heterogeneous and that methodological rigor is inadequate.12,18

    We therefore designed and conducted the Ophthalmic Learning and Improvement Initiative in Cataract Surgery (OLIMPICS) Trial. The aim of the trial was to evaluate the effect of intense simulation-based surgical education in cataract surgery on surgical competence, as well as subsequent live surgery outputs and outcomes compared with conventional training alone.

    Methods
    Study Design

    We designed a multicenter, multicountry, investigator-masked, parallel-group RCT conducted from October 1, 2017, to September 30, 2019. Competency was assessed at baseline and in follow-up assessments over the course of 15 months. Trainee ophthalmologists from 5 ophthalmology training program institutions in Nairobi, Kenya; Moshi, Tanzania; Kampala and Mbarara, Uganda; and Harare, Zimbabwe were assessed for eligibility. Written informed consent was obtained from all participants. Participants were given no incentives or compensation. No changes to methods were made after trial commencement (trial protocol in Supplement 1). Ethical approval was attained from 10 separate research ethics committees. Full details are in Supplement 1.

    Participants and Prerandomization Baseline Assessment

    Inclusion criteria included having performed zero complete manual small-incision cataract surgery (SICS) procedures as primary surgeon and having performed parts of (or assisted in) fewer than 10 separate SICS procedures. After consent, participant trainees were evaluated in country. Baseline assessment included recorded performance of 3 surgical simulation procedures each. These assessments were anonymized and remotely graded in a masked fashion using the Ophthalmic Simulation Surgical Competency Assessment Rubric (Sim-OSSCAR).19 A standardized knowledge assessment was also administered, providing further baseline data. Participants were assured of confidentiality and anonymity of individual outcome assessments.

    Randomization

    The randomization sequences were computer generated centrally by a statistician (M.J.K.) based at the London School of Hygiene & Tropical Medicine who was independent of all other aspects of the trial. We randomly allocated candidates at the site level into batches of 2 or 4 trainees, with equal numbers of intervention and control allocations in each batch. Preprinted allocation cards that specified the center, batch group, unique identifier, and allocation (intervention or control) were concealed inside opaque sealed envelopes. This ensured that the principal investigator, coinvestigator, and participants had no prior knowledge of the allocation until the envelopes were opened. All the envelopes in the batch had an identical external appearance and batch label code. All trainees in the batch were each invited to simultaneously select and open one of the envelopes and to reveal their allocation card. If an odd number of participants were identified in a center, the final participant was invited to select 1 of 2 identical envelopes in a batch of 2. This ensured randomization, as all candidates had an equal chance of being in either group.

    Intervention

    The simulation-based training was conducted at the purpose-built Surgery Training Unit, University of Cape Town, South Africa. The SICS procedure was deconstructed and instruction on individual steps was achieved using the Peyton 4-stage approach to teaching a practical skill.20 Feedback was given to participants while they engaged in sustained deliberate practice of a particular step.21 Once all parts of the SICS procedure were covered, the full procedure was performed on high-fidelity synthetic simulation eyes,22 after a round of mental rehearsal (the cognitive rehearsal of a task before practice).23 Participants were able to record their surgical performance and engage in reflective learning by watching their performance on an iPad.24 This was enhanced by formative assessment and outcome measurement as they graded their performance against the Sim-OSSCAR.19 All training was conducted by one of us (W.H.D.). The study protocol and standard operating procedures, including a detailed description of the intervention, are available in Supplement 1.

    Control participants were offered the same training in Cape Town, South Africa, after 1 year. Both the intervention and control groups continued to undergo conventional postgraduate ophthalmology training.

    Outcomes

    Participants were followed up at 3 months after the intervention and at 1 year. Assessments included 3 sequential simulation SICS procedures recorded in the same manner as the baseline assessment. There was no time limit on the surgical procedure recordings. Further assessments included a supervised live SICS procedure at 12 months and a summary report of cataract surgery numbers and outcomes over 1 year. No changes to study outcomes were made after trial commencement.

    The primary outcome measure was the difference in Sim-OSSCAR19 scores between groups at 3 months. Each of the 20 items in the matrix was graded on a modified Dreyfus score (novice, advanced beginner, and competent). The minimum score was 0 points and the total possible score was 40 for each procedure. Masked assessments were performed remotely by 2 independent expert SICS surgeons (S.M. and L.H.-W.).

    Secondary outcome measures included assessment of surgical competence at 12 months (live and simulation), number of live SICS procedures performed, and surgery outcomes for a period of 12 months. Number and outcomes of live SICS procedures performed were self-reported retrospectively in a summary report after 12 months.

    Statistical Analysis

    Based on data from a pilot study, we anticipated a difference in Sim-OSSCAR scores between groups of 9 of 40 points, and an estimated variability of 0.9 SD. We therefore calculated that a sample of 23 individuals in each group would have 80% power and 95% confidence to detect a significant difference in scores. We aimed to recruit 25 individuals per group, to provide 2 extra participants per group for any loss to follow-up.

    The distributions of baseline variables by treatment group were compared. The primary outcome measure was the mean score of 3 masked assessments at 3 months of simulation surgical performance using the Sim-OSSCAR.

    Intention-to-treat analysis was used for all outcome measures. Primary analysis included a linear regression model with mean Sim-OSSCAR scores at 3 months as the outcome and trial group as the exposure, adjusting for baseline mean Sim-OSSCAR score taking training center as a random effect. A similar approach was used for secondary outcome measures of competence. Mean live SICS procedure ICO (International Council of Ophthalmology) Ophthalmology Surgical Competency Assessment Rubric (ICO-OSCAR)25 score at 1 year was analyzed by a t test. The number of surgical procedures performed in 1 year was analyzed using a Poisson regression, with trial group as the exposure of interest, adjusting for training center. Patient-specific outcomes for all surgical procedures performed during the 12-month period included the number of patients with poor postoperative visual acuity per surgeon, analyzed using the Wilcoxon rank-sum test. Further assessment included percentage rates of operative complications of posterior capsule rupture (PCR), analyzed using linear regression.

    An α level of P < .05 was considered statistically significant for the primary outcome. P values were 2-sided. A κ coefficient of 0.75 or more for interassessor agreement of video grading scores was considered to be excellent.26

    Data were initially entered into Microsoft Excel, version 15.31 (Microsoft Corp). Statistical analysis was performed using Stata, version 15.1 (StataCorp). A data monitoring and trial advisory committee oversaw the study.

    Prevention of Bias

    It is accepted that there will be variability in individual participants’ inherent or natural surgical aptitude. All efforts were made to standardize the training offered to the intervention participants (as well as to the control participants after the 1-year period). The intense simulation course was held in the same standardized surgical training unit, and all training was conducted by one of us (W.H.D.). Recordings of live and simulation surgical procedures were anonymized. Every effort was made to reduce contamination bias. Numerous standard risk-of-bias criteria may be used to evaluate RCTs. These criteria are further illustrated in the trial protocol (Supplement 1).

    Results

    A total of 52 potential participants were assessed for eligibility between October 1, 2017, and May 21, 2018. Fifty participants were recruited, and 49 participants were included in the final intention-to-treat analysis (Figure 1). Two potential participants were excluded before randomization owing to prior surgical experience. One trainee in the control group completed baseline assessments but suddenly left the training program, and was not contactable. All 5 ophthalmology training programs contributed participants (4 from the Kilimanjaro Christian Medical Centre, 8 from Mbarara University, 10 from Makerere University, 17 from the University of Nairobi, and 10 from the University of Zimbabwe). There were no unintended effects in each group.

    Table 1 shows participants’ baseline characteristics. There was good balance between groups. A total of 757 videos from across the different time points were independently graded in a masked fashion, each by 2 graders, of which 297 baseline and 3-month recordings contributed to the primary outcome measure. Interobserver reliability correlation showed a κ coefficient of 0.86 for total scores. Intraobserver agreement was 0.87.

    The mean (SD) Sim-OSSCAR scores at 3 months were 33.7 (3.0) (84.3% of points) for the intervention group and 17.9 (5.9) (44.8% of points) for the control group (P < .001) (Table 2). Linear regression analysis of Sim-OSSCAR scores at 3 months, taking into account center clustering, illustrated a large effect of the intervention. Those who received the training were estimated to have unadjusted scores 15.8 points higher (95% CI, 13.2-18.5) (P < .001) than those who did not receive the training. The difference in Sim-OSSCAR scores was 16.6 points higher (95% CI, 14.4-18.7) with adjustment for baseline scores (P < .001) (Table 2 and Figure 2).

    The mean (SD) Sim-OSSCAR scores at 1 year were 32.9 (3.6) (82.3%) for the intervention group and 24.2 (5.5) (61.1%) for the control group (Table 2). Scores at 1 year were 8.5 points (95% CI, 6.7-10.9; P < .001) higher in the intervention group compared with the controls, adjusting for baseline scores, supporting a continued benefit from the training intervention.

    Live surgical performance on patients was recorded anonymously at the 1-year mark for both groups, before the training course intervention began for the control participants. The mean surgical competency score using the ICO-OSCAR was 62.3 of 95 (65.6%) for the intervention group and 45.0 of 95 (47.4%) for the control group (difference, 17.3 points; 95% CI, 5.2-29.3; P = .006).

    The total number of live SICS procedures performed in 1 year (from 0 to 12 months) was recorded for each participant. Intervention group participants performed a mean of 21.5 surgical procedures as the primary surgeon and assisted in 24.6 cataract surgical procedures. Control group participants performed 8.5 surgical procedures as the primary surgeon and assisted in 10.9 cataract surgical procedures during the same period. The mean difference was 13.0 surgical procedures (95% CI, 3.9-22.2; P < .001). Poisson regression analysis, with trial group as the exposure of interest, adjusting for training center, showed strong evidence that those who received the intervention training performed more live surgical procedures (as primary surgeon or assistant) than did those in the control group; those receiving the intervention performed 2.5 times (95% CI, 2.2-3.0) as many surgical procedures as those who did not.

    The proportion of good outcomes (day 1 presenting visual acuity, ≥6/18) was 36.8% (138 of 375) and of poor outcomes (presenting visual acuity, <6/60) was 10.1% (38 of 375) for the intervention group; for the control participants, the proportion of good outcomes was 25.6% (30 of 117) and of poor outcomes was 12.8% (15/117). There was no significant difference in the proportion of good or poor outcomes between groups (Wilcoxon rank-sum P = .90 for the intervention group and P = .95 for the control group).

    The mean PCR proportion during the 1 year after the training intervention was 70.7% lower at 7.8% (42 of 537) for intervention trainees, compared with 26.6% (54 of 203) for the control participants for the same 12-month period (difference, 18.8%; 95% CI, 12.3%-25.3%; P < .001). Figure 3 illustrates the regression plot of the number of cataract surgical procedures performed and number of PCRs by group. For those who had performed surgery, logistic regression (where the unit is surgery, outcome is PCR, and intervention is the only difference) illustrated a strong effect of the intervention. Intervention participants had a higher chance of having no PCR (odds ratio, 4.27; 95% CI, 2.74-6.65; P < .001).

    Discussion

    The OLIMPICS trial has demonstrated that an intense 5-day simulation-based cataract surgical education course successfully improved the main outcome of cataract surgical competence at 3 months, and that the benefits persist over 1 year. There is evidence from secondary outcomes that live surgical performance was improved and patient safety benefited from reduced surgical complication rates. This multicenter RCT supports the use of intense simulation training for cataract surgery.

    Although the trainees in the intervention group performed and assisted in more live cataract surgical procedures in the year after the intervention training, it is unlikely that the better competency scores in the intervention group at 3 months are a result of having performed more SICS procedures. This is because most cataract surgery cases performed as primary surgeon were after the first 3 months of the study.

    The implications for real-world training programs are compelling. Trainee eye surgeons should be afforded the opportunity to participate in focused, intense simulation training courses. We believe that supervised live surgical training on patients should begin only after engaging in adequate deliberate practice with feedback, reflective learning, and a competency outcome assessment benchmarked to appropriate standards. The International Council of Ophthalmology has developed a comprehensive residency curriculum and standards for graduates to have basic competence before performing cataract surgery.27

    The implications for patient safety are ethically imperative. We illustrated a dramatic 70.7% reduction in surgical complication rates in the cases performed as primary surgeon in the first year of conventional training. Retrospective studies have shown that access to a virtual reality simulator for cataract surgery training (Eyesi; VR Magic) resulted in a 38.1% reduction in PCR rates for cataract surgical procedures performed by junior trainees in the UK, from 4.2% to 2.6%.16 Mandatory simulator training for novice residents in the US showed a retrospective comparative reduction in PCR rates from 4.8% to 2.2%.17 A retrospective study in India of wet-laboratory cataract surgery training using goat eyes showed PCR rates of 14.3% vs 6.9%.28

    Limitations and Strengths

    This study has some limitations. A potential limitation of the OLIMPICS trial is the use of the Sim-OSSCAR19 rather than live surgical competency assessment with the ICO-OSCAR25 as the primary outcome measure. We argue, however, that this is a strength. The simulation environment and use of the validated Sim-OSSCAR affords participants the chance to complete as much of the cataract surgery procedure as they can without potential harm to patients, whereas live surgery is prone to greater variation that impairs its use for comparative purpose with small samples. All live surgery performed at the 12-month assessment was supervised by a local senior surgeon. At their professional discretion, they could take over surgery at any time, and for that part of the procedure the trainee would score zero on the live ICO-OSCAR rubric. The live surgical competency scores are therefore more complex to interpret. They are based on the variable takeover threshold of different senior surgeons; the comorbidity, risk-stratification, and complexity of a particular case; the confidence level of an individual trainee; and other factors. The use of the simulation artificial eye afforded a standardization that would not have otherwise been achievable in the live surgical setting. Furthermore, it would have been unsafe and unethical for untrained surgeons to be evaluated on surgical procedures performed on patients at a very early stage. Limitations of the study also include variability in training opportunities and training environments. To mitigate against this variability, the randomization was stratified by institution, resulting in equal numbers of intervention and control participants within an institution. This may, to a large extent, compensate for the inter-institutional variability, leading to balance between trial groups in factors such as cataract case mix (number and complexity). Another potential limitation, which is impossible to quantify, is the Hawthorne effect, whereby the behavior of participants of a study is altered owing to their awareness of being observed.29

    This study also has some strengths. The strengths of the OLIMPICS trial are its RCT methodology, standardized intervention training for all participants, investigator masking, and double marking of all 757 surgical videos (each video was marked by 2 independent graders).

    A critical review of simulation-based medical education suggested 12 areas or features of best practice,30 many of which had been identified by other educational theorists. Of these, skill acquisition and maintenance, feedback, sustained deliberate practice, curriculum integration, outcome measurement, and simulation fidelity are key.10 These findings suggest that simulation-based surgical education should not be perceived as merely having access to a wet laboratory, dry laboratory, or computerized or full-immersion virtual reality simulator. For greatest impact, simulation-based surgical education should be seen and used as a comprehensive educational package. Part of this included the digital classroom, where procedures are recordable so that the trainee gets feedback on the whole process and can also review it themselves, engaging in critical reflective learning.

    Conclusions

    The OLIMPICS trial illustrated a positive effect on patient safety. Not only are trainees and trainers afforded a safe, calm, and effective environment to teach and learn away from patients but the result appears to be a substantial reduction in the rates of surgical complications. With RCT-level evidence of the utility of intense simulation-based surgical education for cataract surgery, the opportunity is presented for us to protect the patients we and our trainees serve, to collectively and collaboratively work together to have this approach to surgical education implemented and mandated.

    Back to top
    Article Information

    Accepted for Publication: September 21, 2020.

    Published Online: November 5, 2020. doi:10.1001/jamaophthalmol.2020.4718

    Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2020 Dean WH et al. JAMA Ophthalmology.

    Corresponding Author: William H. Dean, PhD, MEd, MBChB, BSc, International Centre for Eye Health, Clinical Research Department, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom (will.dean@lshtm.ac.uk).

    Author Contributions: Dr Dean had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Dean, Buchan, Otiti-Sengeri, Arunga, Mukherjee, Macleod, Cook, Burton.

    Acquisition, analysis, or interpretation of data: Dean, Gichuhi, Buchan, Makupa, Mukome, Arunga, Mukherjee, Kim, Harrison-Williams, Burton.

    Drafting of the manuscript: Dean, Otiti-Sengeri, Arunga, Mukherjee, Harrison-Williams.

    Critical revision of the manuscript for important intellectual content: Dean, Gichuhi, Buchan, Makupa, Mukome, Mukherjee, Kim, Harrison-Williams, Macleod, Cook, Burton.

    Statistical analysis: Dean, Kim, Macleod.

    Obtained funding: Dean, Burton.

    Administrative, technical, or material support: Dean, Gichuhi, Makupa, Arunga, Harrison-Williams, Cook.

    Supervision: Dean, Gichuhi, Buchan, Mukome, Otiti-Sengeri, Arunga, Burton.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: The Trial was funded by the British Council for the Prevention of Blindness, Ulverscroft Foundation, Christian Blind Mission (CBM), L’Occitane Foundation, Orbis International, and Queen Elizabeth Diamond Jubilee Trust. Dr Burton is supported by grant 207472/Z/17/Z from the Wellcome Trust. The trial sponsor was the London School of Hygiene & Tropical Medicine.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Data Sharing Statement: See Supplement 2.

    Additional Contributions: Nathan Congdon, MD, MPH, Queen’s University Belfast, reviewed the manuscript. He was not compensated for his contribution.

    References
    1.
    Flaxman  SR, Bourne  RRA, Resnikoff  S,  et al; Vision Loss Expert Group of the Global Burden of Disease Study.  Global causes of blindness and distance vision impairment 1990-2020: a systematic review and meta-analysis.   Lancet Glob Health. 2017;5(12):e1221-e1234. doi:10.1016/S2214-109X(17)30393-5 PubMedGoogle ScholarCrossref
    2.
    Wang  W, Yan  W, Fotis  K,  et al.  Cataract surgical rate and socioeconomics: a global study.   Invest Ophthalmol Vis Sci. 2016;57(14):5872-5881. doi:10.1167/iovs.16-19894 PubMedGoogle ScholarCrossref
    3.
    Weiser  TG, Haynes  AB, Molina  G,  et al.  Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes.   Lancet. 2015;385(suppl 2):S11. doi:10.1016/S0140-6736(15)60806-6 PubMedGoogle ScholarCrossref
    4.
    Brown  MM, Brown  GC, Lieske  HB, Lieske  PA.  Financial return-on-investment of ophthalmic interventions: a new paradigm.   Curr Opin Ophthalmol. 2014;25(3):171-176. doi:10.1097/ICU.0000000000000040 PubMedGoogle ScholarCrossref
    5.
    Horton  S, Gelband  H, Jamison  D, Levin  C, Nugent  R, Watkins  D.  Ranking 93 health interventions for low- and middle-income countries by cost-effectiveness.   PLoS One. 2017;12(8):e0182951. doi:10.1371/journal.pone.0182951 PubMedGoogle Scholar
    6.
    Resnikoff  S, Lansingh  VC, Washburn  L,  et al.  Estimated number of ophthalmologists worldwide (International Council of Ophthalmology update): will we meet the needs?   Br J Ophthalmol. 2020;104(4):588-592. doi:10.1136/bjophthalmol-2019-314336PubMedGoogle ScholarCrossref
    7.
    Dean  W, Gichuhi  S, Buchan  J,  et al.  Survey of ophthalmologists-in-training in Eastern, Central and Southern Africa: a regional focus on ophthalmic surgical education.   Wellcome Open Res. 2019;4:187. doi:10.12688/wellcomeopenres.15580.1 PubMedGoogle ScholarCrossref
    8.
    Young  AL, Jhanji  V, Liang  Y,  et al.  A survey of perceived training differences between ophthalmology residents in Hong Kong and China.   BMC Med Educ. 2015;15:158. doi:10.1186/s12909-015-0440-0 PubMedGoogle ScholarCrossref
    9.
    Martinerie  L, Rasoaherinomenjanahary  F, Ronot  M,  et al.  Health care simulation in developing countries and low-resource situations.   J Contin Educ Health Prof. 2018;38(3):205-212. doi:10.1097/CEH.0000000000000211 PubMedGoogle ScholarCrossref
    10.
    Issenberg  SB, McGaghie  WC, Petrusa  ER, Lee Gordon  D, Scalese  RJ.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.   Med Teach. 2005;27(1):10-28. doi:10.1080/01421590500046924 PubMedGoogle ScholarCrossref
    11.
    Alaker  M, Wynn  GR, Arulampalam  T.  Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis.   Int J Surg. 2016;29:85-94. doi:10.1016/j.ijsu.2016.03.034 PubMedGoogle ScholarCrossref
    12.
    Thomsen  AS, Subhi  Y, Kiilgaard  JF, la Cour  M, Konge  L.  Update on simulation-based surgical training and assessment in ophthalmology: a systematic review.   Ophthalmology. 2015;122(6):1111-1130.e1. doi:10.1016/j.ophtha.2015.02.028PubMedGoogle ScholarCrossref
    13.
    Feudner  EM, Engel  C, Neuhann  IM, Petermeier  K, Bartz-Schmidt  KU, Szurman  P.  Virtual reality training improves wet-lab performance of capsulorhexis: results of a randomized, controlled study.   Graefes Arch Clin Exp Ophthalmol. 2009;247(7):955-963. doi:10.1007/s00417-008-1029-7 PubMedGoogle ScholarCrossref
    14.
    Daly  MK, Gonzalez  E, Siracuse-Lee  D, Legutko  PA.  Efficacy of surgical simulator training versus traditional wet-lab training on operating room performance of ophthalmology residents during the capsulorhexis in cataract surgery.   J Cataract Refract Surg. 2013;39(11):1734-1741. doi:10.1016/j.jcrs.2013.05.044 PubMedGoogle ScholarCrossref
    15.
    McCannel  CA, Reed  DC, Goldman  DR.  Ophthalmic surgery simulator training improves resident performance of capsulorhexis in the operating room.   Ophthalmology. 2013;120(12):2456-2461. doi:10.1016/j.ophtha.2013.05.003 PubMedGoogle ScholarCrossref
    16.
    Ferris  JD, Donachie  PH, Johnston  RL, Barnes  B, Olaitan  M, Sparrow  JM.  Royal College of Ophthalmologists’ National Ophthalmology Database study of cataract surgery: report 6: the impact of EyeSi virtual reality training on complications rates of cataract surgery performed by first and second year trainees.   Br J Ophthalmol. 2020;104(3):324-329. doi:10.1136/bjophthalmol-2018-313817PubMedGoogle ScholarCrossref
    17.
    Staropoli  PC, Gregori  NZ, Junk  AK,  et al.  Surgical simulation training reduces intraoperative cataract surgery complications among residents.   Simul Healthc. 2018;13(1):11-15. doi:10.1097/SIH.0000000000000255PubMedGoogle Scholar
    18.
    Rothschild  P, Richardson  A, Franzco  JB, Franzco  RC.  Does virtual reality simulation training result in fewer real-life cataract surgery complications? A systematic literature review.   J Cataract Refract Surg. 2020. Published online July 13, 2020. doi:10.1097/j.jcrs.0000000000000323PubMedGoogle Scholar
    19.
    Dean  WH, Murray  NL, Buchan  JC, Golnik  K, Kim  MJ, Burton  MJ.  Ophthalmic Simulated Surgical Competency Assessment Rubric for manual small-incision cataract surgery.   J Cataract Refract Surg. 2019;45(9):1252-1257. doi:10.1016/j.jcrs.2019.04.010 PubMedGoogle ScholarCrossref
    20.
    Peyton  JW.  Teaching and Learning in Medical Practice. Manticore Europe 1998:174-177.
    21.
    Ericsson  KA, Krampe  RT, Tesch-Romer  C.  The role of deliberate practice in the acquisition of expert performance.   Psychological Rev. 1993;100(3):363-406. doi:10.1037/0033-295X.100.3.363 Google ScholarCrossref
    22.
    Phillips Studio. Ophthalmic simulated surgery. Accessed February 1, 2020. http://www.phillipsstudio.co.uk/
    23.
    Arora  S, Aggarwal  R, Sirimanna  P,  et al.  Mental practice enhances surgical technical skills: a randomized controlled study.   Ann Surg. 2011;253(2):265-270. doi:10.1097/SLA.0b013e318207a789 PubMedGoogle ScholarCrossref
    24.
    Schön  DA.  The Reflective Practitioner: How Professionals Think in Action. Temple Smith; 1983.
    25.
    Golnik  KC, Beaver  H, Gauba  V,  et al.  Cataract surgical skill assessment.   Ophthalmology. 2011;118(2):427.e1-427.e5. doi:10.1016/j.ophtha.2010.09.023PubMedGoogle ScholarCrossref
    26.
    Landis  JR, Koch  GG.  The measurement of observer agreement for categorical data.   Biometrics. 1977;33(1):159-174. doi:10.2307/2529310 PubMedGoogle ScholarCrossref
    27.
    International Council of Ophthalmology. ICO residency curriculum. Accessed August 20, 2020. http://www.icoph.org/refocusing_education/curricula.html
    28.
    Ramani  S, Pradeep  TG, Sundaresh  DD.  Effect of wet-laboratory training on resident performed manual small-incision cataract surgery.   Indian J Ophthalmol. 2018;66(6):793-797. doi:10.4103/ijo.IJO_1041_17 PubMedGoogle ScholarCrossref
    29.
    McCambridge  J, Witton  J, Elbourne  DR.  Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects.   J Clin Epidemiol. 2014;67(3):267-277. doi:10.1016/j.jclinepi.2013.08.015 PubMedGoogle ScholarCrossref
    30.
    McGaghie  WC, Issenberg  SB, Petrusa  ER, Scalese  RJ.  A critical review of simulation-based medical education research: 2003-2009.   Med Educ. 2010;44(1):50-63. doi:10.1111/j.1365-2923.2009.03547.x PubMedGoogle ScholarCrossref
    ×