[Skip to Content]
[Skip to Content Landing]
Download PDF
Figure.
Surgical skills board and the Qualisys motion capture instrument and software (Qualisys Medical AB, Gothenburg, Sweden). A, The Royal College of Ophthalmologists (London, England) surgical skills board in use, illustrating retroreflective skin markers. The skills board comprises several modules. The artificial skin with the preformed incision (used for the skin suture) can be seen at the back. The metallic hook with the surrounding plastic cylinder (used for the deep suture) can be seen on the right. B, Infrared motion capture camera system. C, Qualisys Track Manager software screenshot showing simulated hand movements. D, Qualisys Track Manager software showing tracking of reflective markers and their respective vector paths. E, Qualisys Track Manager software showing the surgical study setup with relative positions of both the motion capture units and markers.

Surgical skills board and the Qualisys motion capture instrument and software (Qualisys Medical AB, Gothenburg, Sweden). A, The Royal College of Ophthalmologists (London, England) surgical skills board in use, illustrating retroreflective skin markers. The skills board comprises several modules. The artificial skin with the preformed incision (used for the skin suture) can be seen at the back. The metallic hook with the surrounding plastic cylinder (used for the deep suture) can be seen on the right. B, Infrared motion capture camera system. C, Qualisys Track Manager software screenshot showing simulated hand movements. D, Qualisys Track Manager software showing tracking of reflective markers and their respective vector paths. E, Qualisys Track Manager software showing the surgical study setup with relative positions of both the motion capture units and markers.

Table. 
Summary of Results for Deep Suture and Skin Closure Tasks
Summary of Results for Deep Suture and Skin Closure Tasks
1.
Barnes  RW Surgical handicraft: teaching and learning surgical skills. Am J Surg 1987;153 (5) 422- 427
PubMedArticle
2.
Darzi  ATaffinder  N Assessing operative skill: needs to become more objective. BMJ 1999;318 (7188) 887- 888
PubMedArticle
3.
Datta  VMandalia  MMackay  SChang  ACheshire  NDarzi  A Relationship between skill and outcome in the laboratory-based model. Surgery 2002;131 (3) 318- 323
PubMedArticle
4.
Bann  SDKhan  MSDarzi  AW Measurement of surgical dexterity using motion analysis of simple bench tasks. World J Surg 2003;27 (4) 390- 394
PubMedArticle
5.
McBeth  PBHodgson  AJNagy  AGQayumi  K Quantitative methodology of evaluating surgeon performance in laparoscopic surgery. Stud Health Technol Inform 2002;85280- 286
PubMed
6.
Saleh  GMVoyatzis  YHance  JRatnasothy  JDarzi  A Evaluating surgical dexterity during corneal suturing. Arch Ophthalmol 2006;124 (9) 1263- 1266[published correction appear in Arch Ophthalmol. 2006;124(12):1801].
PubMedArticle
7.
Whittle  MW Gait Analysis: An Introduction. 3rd ed. Oxford, England Butterworth-Heinemann2002;
8.
Perry  J Gait Analysis: Normal and Pathological Function.  Thorofare, NJ Slack1992;
9.
Hingtgen  BMcGuire  JRWang  MHarris  GF An upper extremity kinematic model for evaluation of hemiparetic stroke. J Biomech 2006;39 (4) 681- 688
PubMedArticle
10.
O’Day  DM Assessing surgical competence in ophthalmology training programs. Arch Ophthalmol 2007;125 (3) 395- 396
PubMedArticle
11.
Martin  JAReznick  RMacRae  HMurnaghan  JHutchison  CBrown  M Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84 (2) 273- 278
PubMedArticle
12.
Reznick  RMacRae  HMartin  JMcCulloch  W Testing technical skill via an innovative “bench station” examination. Am J Surg 1997;173 (3) 226- 230
PubMedArticle
13.
Cremers  SLFerrufino-Ponce  ZKHenderson  BA Objective Assessment of Skills in Intraocular Surgery (OASIS). Ophthalmology 2005;112 (7) 1236- 1241
PubMedArticle
14.
Cremers  SLFerrufino-Ponce  ZK Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology 2005;112 (10) 1655- 1660
PubMedArticle
15.
Mills  RP American Board of Ophthalmology Program Directors' Task Force on Competencies: report of the American Board of Ophthalmology Task Force on the Competencies. Ophthalmology 2004;111 (7) 1267- 1268
PubMedArticle
16.
Anastakis  DJRegehr  GReznick  RK Assessment of technical skills transfer from the bench-training model to the human model. Am J Surg 1999;177 (2) 167- 170
PubMedArticle
17.
Saleh  GMGauda  VMitra  ALitwin  ASChung  AKBenjamin  L Objective structured assessment of cataract surgical skill. Arch Ophthalmol 2007;125 (3) 363- 366
PubMedArticle
18.
 Accreditation Council for Graduate Medical Education Web site. www.acgme.org. Accessed January 2007
19.
Spencer  F Teaching and measuring surgical techniques: the technical evaluation of competence. Bull Am Coll Surg 1978;63 (3) 9- 12
20.
Annett  J Acquisition of skill. Br Med Bull 1971;27 (3) 266- 271
PubMed
21.
Wade  MG Developmental motor learning. Exerc Sport Sci Rev 1976;4375- 394
PubMedArticle
Clinical Sciences
February 01, 2008

Motion Analysis as a Tool for the Evaluation of Oculoplastic Surgical SkillEvaluation of Oculoplastic Surgical Skill

Author Affiliations

Author Affiliations: Frimley Park Hospital (Drs Saleh, Sim, and Lindfield) and Department of Biomedical Engineering, University of Surrey (Dr Ghoussayni and Ms Borhani), Surrey, Moorfields Eye Hospital, London (Drs Saleh and Sim), and St James Hospital, Leeds (Dr Gauba), England.

Arch Ophthalmol. 2008;126(2):213-216. doi:10.1001/archophthalmol.2007.62
Abstract

Objective  To evaluate motion analysis as a discriminator of ophthalmic plastic surgical skill between surgeons of varying experience.

Methods  Thirty subjects were divided into 3 groups based on surgical experience: novice (< 5 performed procedures; n = 10), intermediate (5-100 procedures; n = 10), and expert (> 100 procedures; n = 10). Detailed 3-dimensional motion data from surgeons performing 2 oculoplastic surgical tasks on a wet laboratory skills board were obtained using the Qualisys motion capture system. The first task was a deep 3-1-1 suture. The second was skin closure with a continuous suture. The main outcome measures were time, overall path length, and total number of movements. Kruskal-Wallis analysis was performed to evaluate statistical significance.

Results  Highly significant differences were found during the skin closure task between all groups for mean time (P = .002), overall path length (P = .002), and number of movements (P = .001). For the deep stitch, highly significant differences were also found for time (P < .001), path length (P < .001), and number of movements (P < .001).

Conclusions  Motion analysis, using this technology, was able to differentiate between surgeons of varying experience performing oculoplastic tasks, thus demonstrating construct validity. This technique may be useful in the objective quantitative measurement of oculoplastic skill, with potential applications for training and research.

Training in ophthalmic surgery has been traditionally based on the apprenticeship system. With the innovation of video technology in the past few decades, recorded playback analysis with the supervising surgeon has become a popular method of training. However, this technique has a large interobserver variation1 and lacks quantifiable measures with which changes of surgical skill can be monitored over time.

The assessment of surgical technical skill has become more important in recent years2 and it has been shown that there is a significant correlation between objective measures of manual dexterity and surgical skill with the outcome of a procedure.3 Motion analysis, pioneered by Lord Ara Darzi, MD, KBE, is an emerging validated technique of surgical skill evaluation.46 To our knowledge, this method has not previously been applied to oculoplastic surgery.

The Qualisys motion capture instrument (Qualisys Medical AB, Gothenburg, Sweden) is a passive optoelectronic kinematic analysis system, most commonly used for the measurement of body motion, and has been validated and extensively used for gait analysis,7 motor control assessments,8 and upper limb function.9 Such technology is used for Mocap (optical motion capture) in entertainment and virtual reality applications. We describe a new adaptation of this technology to ophthalmic plastic surgery and discuss future potential of this technique for surgical skill evaluation.

METHODS
MOTION ANALYSIS TECHNIQUE AND TECHNOLOGY

The Qualisys ProReflex 500 (Qualisys Medical AB) motion analyzer incorporates a multiple camera system arranged in a 360° fashion. Each motion capture camera unit comprises infrared light-emitting diodes around the lens; a low-noise, high-speed sensor; and a built-in microprocessor (Figure, B). The motion capture camera unit uses the reflected infrared light to detect the position of each marker worn by subjects. These markers are coated with retroreflective tape to amplify their brightness as compared with the skin, clothing, and background (Figure, A). A 3-dimensional image of the markers is then formed by the Qualisys Track Manager software (Figure, C, D, and E). The motion data were captured at a sampling rate of 100 Hz, which is sufficiently high for the movements seen during ophthalmic microsurgery. Simultaneous video recordings were carried out using a digital camcorder, allowing motion tracking data to be synchronized and simultaneously processed with the video recording. Data filtering was performed using a zero-lag, second-order, digital Butterworth filter with a cutoff frequency of 10 Hz to eliminate high-frequency noise.

Following extensive pilot laboratory work calibrating the instrument for ophthalmic use, a distribution that most sensitively and accurately detected motion in the oculoplastic surgical setting was determined. Four 10-mm–diameter retroreflective markers were attached using double-sided adhesive tape to each hand at the second metacarpal head, fifth metacarpal head, midpoint between the base of the second and third metacarpal base, and dorsally on the midpoint of the middle phalanx of the second phalange (Figure, A). The main outcome measures were time, overall path length, and total number of movements because these have been previously validated.6

PARTICIPANTS

Thirty subjects were divided into 3 groups (n = 10 each) based on surgical experience: novice (< 5 performed procedures), intermediate (5-100 performed procedures), and expert (> 100 performed procedures). All subjects were given standardized instruction prior to the tasks, and an independent expert observer was present to ensure correct task completion in all cases.

SIMULATED SURGICAL TASKS

The first task was the insertion of a deep 3-1-1 suture using 6-0 polyglactin 910 (Vicryl; Ethicon, Somerville, New Jersey). It had to be placed around a metallic hook surrounded by a plastic cylinder (Figure, A). The suture needle was passed through the hook and then tied to the metallic frame (rim) with 3 throws on the first knot with 2 subsequent single throws to lock it. The loose ends of the suture were cut. The location of this deep stitch made the manipulation of the instruments more challenging. The second task involved the insertion of a continuous skin suture using 6-0 polypropylene (Prolene; Ethicon) in a preformed skin wound (Figure, A). The task commenced with a 3-1-1 knot placed beyond the skin wound to anchor the suture. The needle was then passed subcutaneously and regrasped inside the wound edges after which 3 subcutaneous bites were performed bringing the skin edges together. The needle was then passed from inside the wound edges and brought up through the skin (parallel to the wound), allowing a final 3-1-1 knot to be applied to secure the distal end of the continuous skin closure task.

For both tasks, a standardized wet laboratory environment was used for all surgeons undergoing testing, with the same instruments, surgical skills board (Royal College of Ophthalmologists, London, England) (Figure, A), and unmounted sutures being provided to all subjects. Subjects were allowed time to familiarize themselves with the environment, but once testing commenced, they were required to complete each task a single time without stopping or restarting.

Statistical analysis was performed using the Kruskal-Wallis test on SPSS (version 14; SPSS Inc, Chicago, Illinois). Statistical significance was set at 0.05. A nonparametric test was chosen because of the sample sizes in each cohort.

RESULTS

Summaries of path length, number of movements, and time are presented in the Table. The results demonstrate significant differences in path length, number of movements, and time taken to complete both surgical tasks, with more experienced surgeons demonstrating greater efficiency in completing the given tasks.

Highly statistically significant differences were found between the 3 grades of surgeons for both tasks. For the placement of the deep suture (task 1) (Table), the greater the degree of experience, the shorter the path length (Kruskal-Wallis, P < .001), the lesser the number of hand movements (Kruskal-Wallis, P < .001), and the shorter the time taken (Kruskal-Wallis, P < .001). For the placement of the subcutaneous skin closure (task 2) (Table), the greater the degree of experience, the shorter the path length (Kruskal-Wallis, P = .002), the lesser the number of hand movements (Kruskal-Wallis, P = .001), and the shorter the time taken (Kruskal-Wallis, P = .002).

COMMENT

The recent drive to develop more objective and standardized systems of evaluation for surgical trainees has been propagated by current changes in both the content and delivery of medical education.10 Techniques that have evolved in ophthalmology so far, though useful, have retained an assessor-dependent element of subjectivity. Kinematic and motion analysis is a purely objective tool in both its acquisition and analysis of data.3,9,1115 The data presented in this study show the ability of motion analysis to objectively discriminate between surgeons of differing levels of experience. For both oculoplastic tasks assessed, the more experienced the surgeon, the more efficiently they completed the tasks (with shorter path lengths, fewer hand movements, and less time). These results show construct validity, the ability to discriminate between surgeons of different experience, when applied to these oculoplastic tasks.

As shown in the Table, the spread, range, and standard deviation of the parameters narrow as the skill level progresses. The results suggest that with greater experience there is a conflation of surgical skills and that this effect is more pronounced with the more complicated the task (skin closure).

Poor clinical outcomes can result from inadequate technical skill, but despite this, there have been few formal attempts to evaluate it.6,12,14,16,17 There has recently been a drive toward an expanded set of tools for surgical evaluation beyond those currently tested in written, clinical, and oral examinations. The Accreditation Council of Graduate Medical Education has endorsed the inclusion of surgical competence as one of the competencies for ophthalmology residents.18 Similar changes are taking place in Europe. Motion analysis may have a role in helping formulate an objective and standardized system for appraising technical skill.1921

The surgical tasks selected evaluated specific components of technical competence in a wet laboratory environment and thus have inherent limitations. These tasks, however, were selected to represent core oculoplastic skills that residents should be familiar with and to be of different complexity. This tool has the potential for providing structured objective feedback on surgical performance that may be used to monitor progress and target further tuition and thus be a useful adjunct to current systems of evaluation.

Further research on the practical implementation of this method is required, including its potential to evaluate live surgery. Motion analysis, using a different technology, has been successfully applied to corneal suturing under the microscope.6 It is encouraging that this form of motion tracking technology, which to our knowledge has not been used for surgical evaluation previously and is more sensitive than other similar tools, was successfully adapted to oculoplastic surgery. You can escape bad teaching but not bad assessment. Good assessment procedures are fundamental for promotion, certification, and licensure. No single method can comprehensively assess the surgical skills of residents in training. Our results offer encouragement that as further research takes place in this field, motion analysis will prove to be a useful modality in accomplishing the current goal of more objective surgical evaluation.

Back to top
Article Information

Correspondence: George M. Saleh, MRCSEd, MRCOphth, Moorfields Eye Hospital, 162 City Rd, London EC1V 2PV, England (drgmsaleh@yahoo.co.uk).

Submitted for Publication: April 1, 2007; final revision received May 21, 2007; accepted June 22, 2007.

Financial Disclosure: None reported.

References
1.
Barnes  RW Surgical handicraft: teaching and learning surgical skills. Am J Surg 1987;153 (5) 422- 427
PubMedArticle
2.
Darzi  ATaffinder  N Assessing operative skill: needs to become more objective. BMJ 1999;318 (7188) 887- 888
PubMedArticle
3.
Datta  VMandalia  MMackay  SChang  ACheshire  NDarzi  A Relationship between skill and outcome in the laboratory-based model. Surgery 2002;131 (3) 318- 323
PubMedArticle
4.
Bann  SDKhan  MSDarzi  AW Measurement of surgical dexterity using motion analysis of simple bench tasks. World J Surg 2003;27 (4) 390- 394
PubMedArticle
5.
McBeth  PBHodgson  AJNagy  AGQayumi  K Quantitative methodology of evaluating surgeon performance in laparoscopic surgery. Stud Health Technol Inform 2002;85280- 286
PubMed
6.
Saleh  GMVoyatzis  YHance  JRatnasothy  JDarzi  A Evaluating surgical dexterity during corneal suturing. Arch Ophthalmol 2006;124 (9) 1263- 1266[published correction appear in Arch Ophthalmol. 2006;124(12):1801].
PubMedArticle
7.
Whittle  MW Gait Analysis: An Introduction. 3rd ed. Oxford, England Butterworth-Heinemann2002;
8.
Perry  J Gait Analysis: Normal and Pathological Function.  Thorofare, NJ Slack1992;
9.
Hingtgen  BMcGuire  JRWang  MHarris  GF An upper extremity kinematic model for evaluation of hemiparetic stroke. J Biomech 2006;39 (4) 681- 688
PubMedArticle
10.
O’Day  DM Assessing surgical competence in ophthalmology training programs. Arch Ophthalmol 2007;125 (3) 395- 396
PubMedArticle
11.
Martin  JAReznick  RMacRae  HMurnaghan  JHutchison  CBrown  M Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84 (2) 273- 278
PubMedArticle
12.
Reznick  RMacRae  HMartin  JMcCulloch  W Testing technical skill via an innovative “bench station” examination. Am J Surg 1997;173 (3) 226- 230
PubMedArticle
13.
Cremers  SLFerrufino-Ponce  ZKHenderson  BA Objective Assessment of Skills in Intraocular Surgery (OASIS). Ophthalmology 2005;112 (7) 1236- 1241
PubMedArticle
14.
Cremers  SLFerrufino-Ponce  ZK Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology 2005;112 (10) 1655- 1660
PubMedArticle
15.
Mills  RP American Board of Ophthalmology Program Directors' Task Force on Competencies: report of the American Board of Ophthalmology Task Force on the Competencies. Ophthalmology 2004;111 (7) 1267- 1268
PubMedArticle
16.
Anastakis  DJRegehr  GReznick  RK Assessment of technical skills transfer from the bench-training model to the human model. Am J Surg 1999;177 (2) 167- 170
PubMedArticle
17.
Saleh  GMGauda  VMitra  ALitwin  ASChung  AKBenjamin  L Objective structured assessment of cataract surgical skill. Arch Ophthalmol 2007;125 (3) 363- 366
PubMedArticle
18.
 Accreditation Council for Graduate Medical Education Web site. www.acgme.org. Accessed January 2007
19.
Spencer  F Teaching and measuring surgical techniques: the technical evaluation of competence. Bull Am Coll Surg 1978;63 (3) 9- 12
20.
Annett  J Acquisition of skill. Br Med Bull 1971;27 (3) 266- 271
PubMed
21.
Wade  MG Developmental motor learning. Exerc Sport Sci Rev 1976;4375- 394
PubMedArticle
×