Figure 1. Illustrations of kinematic analysis as a tool for elucidating patterns of movement. A, The motion capture units (MCU; ProReflex 500; Qualisys Medical AB). B, A screen display showing the relationship between the MCU and retroreflective markers. C, The surgical task being performed under standardized test conditions. D, A screen display of the retroreflective markers' position in 3-dimensional space. E, Location of the 4 retroreflective markers on the hand of one test subject (described in the “Motion Analysis Technique and Technology” subsection of the “Methods” section). F, Kinematic hand model based on the 3 metacarpal surface markers in part E.
Figure 2. Probability density function distribution in relation to the magnitude of position changes for each of the 3 groups of subjects with different levels of surgical experience.
Figure 3. Screen display of the video capture mode, synchronized with the motion tracker.
Saleh GM, Litwin A, Collin JRO, Rose GE, Gauba V, Ghoussayni S, Hussain B. Kinematic Analysis in Oculoplastic Reconstructive SurgeryMeasuring Manual Control and Fluidity of Movement. Arch Ophthalmol. 2012;130(12):1570-1574. doi:10.1001/archophthalmol.2012.2721
Author Affiliations: Orbital and Oculoplastic Service, Moorfields Eye Hospital, (Drs Saleh, Litwin, Collin, Rose, Gauba, and Hussain), and National Institute of Health Research Biomedical Research Centre, Moorfields Eye Hospital and UCL Institute of Ophthalmology (Dr Saleh), London, England; and Faculty of Engineering and Physical Sciences, Centre for Biomedical Engineering, University of Surrey, Surrey, England (Drs Saleh and Ghoussayni).
Objective To evaluate higher-order kinematic analysis, a technique not previously applied to surgical skills assessment, as a tool for elucidating patterns of movement.
Methods An observational cohort study of 27 subjects, divided into 3 equal groups based on surgical experience consisting of novice (performed <5 prior procedures), intermediate (performed 5-100 prior procedures), and expert (performed >100 prior procedures) subjects. The subjects placed a deep 3-1-1 suture onto a shielded hook on a standardized surgical skills practice board. Detailed 3-dimensional motion data were obtained using a motion capture system. Two novel parameters were used to analyze movement patterns: the frequency distribution (cumulative histogram), describing the distribution of movement sizes used, and the probability density function (normalization of frequency distribution data), evaluating the distribution of motion against the magnitude of movement. The α risk for statistical significance was set at .05.
Results We found significant differences among the 3 groups for frequency distribution (P = .02; Kruskal-Wallis test) and probability density function (P = .03).
Conclusions These 2 indices, derived from kinematic analysis, appear to distinguish between groups of test subjects with known differences in surgical experience. The evaluation of higher-order motion patterns appears to be of value in the objective evaluation of surgical skills. This method for assessment of manual skills is likely to provide a better guide as to which patterns of movement have the greatest efficiency for specific tasks.
The formal assessment of surgical skills has grown in importance during recent years1 owing to increasing evidence that unstructured systems of assessment have poor reproducibility, large interobserver variation,2 and lack of quantifiable measures. A paradigm shift has therefore begun, with the emergence of more objective and quantitative tools devised to complement current practice. One such example is motion analysis.3- 6
Motion analysis has been validated as a technique for surgical skill evaluation in general surgery,3,6 microsurgery,4 and oculoplastic surgery.5 Using this method, junior and senior surgeons demonstrated significant differences in the efficiency of hand movements, that is, in the time taken, the path length pursued, and the number of movements used to complete test tasks.3- 6 In this study, new and previously undescribed variables (drawn from a previously validated motion capture technique5) are examined in relation to tests of a surgeon's manual fluidity and fluency while performing a reconstructive surgical task. Assessments included hand/finger movements and overall motion control. These variables represent a higher-order analysis of movements and build on the linear variables previously reported in this field.3- 6
We used a marker-based optical motion analysis system to record hand and finger movements. The motion capture system (ProReflex 500; Qualisys Medical AB) consists of 6 motion capture units (MCUs) and the 3-dimensional data-tracking software (Qualisys Track Manager; Qualisys Medical AB) (Figure 1). This infrared-based system detects the position of small (10-mm-diameter), lightweight retroreflective markers attached to the skin using double-sided adhesive tape. Each MCU provides a 2-dimensional image of the measurement volume, which is combined with other MCU images to generate a 3-dimensional model of movements. The MCUs were positioned to minimize any masking of the markers by surgical instruments or other equipment. To minimize extraneous noise and interference within the measurement volume, meticulous and reproducible methods were followed for calibration and setup.
To capture the spectrum of movements typically made during reconstructive surgery, the sampling frequency was set at 100 Hz, and a digital camcorder was also used; the simultaneous video recording provided valuable insight in the interpretation of the MCU data. To minimize high-frequency noise and its effect on calculating velocity measures, the tracked data from 3-dimensional markers were filtered using a zero-lag, second-order digital Butterworth filter with a cutoff frequency of 10 Hz. A previously validated array for the marker placement was used5 with 4 retroreflective markers attached to each hand at the second metacarpal head, the fifth metacarpal head, and the midpoint between the base of the second and third metacarpals and dorsally on the midpoint of the middle phalanx of the index finger (Figure 1E).
We derived 2 novel parameters, frequency distribution (FD) and probability density function (PDF), to evaluate higher-order patterns of motion (ie, control and fluidity). These variables were related to the surgical experience of the test subjects while performing a standardized knotting procedure.
The absolute path length, Pi, between the i th and (i − 1)th sampling frame is derived as follows:
where X, Y, and Z are the Cartesian coordinates for a specificmarker (X being the anterior-posterior axis, Y the transverse axis, and Z the vertical axis). Based on the magnitude of position changes (pi) during the test, the values of pi for each subject are stratified into discrete, equally sized groups; the number of movements per group is then used to derive a cumulative frequency histogram (the FD). The FD reflects the absolute magnitude and the range of movements shown by the test subject and might, therefore, be expected to decrease with increasing surgical experience owing to better efficiency and finer movement control.
The PDF is a derivative from the FD for pi and provides an estimate of variation in movements of a single marker in relation to the mean value. This derivative is relevant because previous work has shown that, with greater experience, significantly fewer movements are required for completion of a given test task.4,5 The PDF pi for the i th and (i − 1)th sampling frame is calculated as follows:
where μ is the mean for all values of pi and σ is the standard deviation. The PDF pi provides a variable that represents the probability distribution in integral form. Analysis of PDF pi shows variation of movement magnitude (pattern and speed) used by the test subject and, from the distribution of PDF pi, we can deduce the movement magnitude of highest probability (PDF pai); a further variable, PDFr, was derived from the ratio of PDF pai divided by the magnitude of that movement of greatest probability (pi). A higher PDFr indicates that a narrower range of movement speed was used and/or that finely controlled movements are of a relatively lower speed.
Based on surgical experience, 27 participants were divided into 3 equal groups consisting of novice (performed <5 prior procedures), intermediate (performed 5-100 prior procedures), and expert (performed >100 prior procedures) subjects. All test subjects underwent assessment in a standardized environment (surgical training wet laboratory) with the same instruments, a surgical skills board (obtained from the Royal College of Ophthalmologists) (Figure 1C), and unmounted sutures. All subjects received a standardized set of instructions and were allowed equal time to familiarize themselves with the environment. An independent observer (G.M.S.) ensured correct completion of the test and, once testing had commenced, the subject was required to finish the task without pausing or restarting.
The task involved formation of a 3-1-1 throw knot using a 6-0 polyglactin 910 suture (Vicryl; Ethicon, Johnson & Johnson), the suture having to be placed around a metallic hook surrounded by a plastic cylinder (Figure 1C). The suture needle was passed through the hook and then tied to the metallic frame with 3 throws on the first knot, and 2 subsequent single throws to lock it; the loose ends of the suture were then trimmed. Placement of the stitch within the depths of a confined space was designed to make this test of dexterity more challenging.
We compared the variables using the Kruskal-Wallis test and with commercially available software (SPSS statistical package, version 14; SPSS Inc). An α risk of .05 was considered clinically significant.
All 27 test subjects completed the surgical task according to the protocol, with none having to restart the test. A significant difference (P < .05) for each of the 2 variables across the 3 experience levels emerged, with the expert group having a significantly lower mean FD and a higher mean derivative PDFr (Table). The expert test subjects had a smaller pai, a higher PDF pai, and thus a higher PDFr, suggesting a higher proportion of finely controlled and slower movements than we observed in the novice group (Figure 2).
The arbitrary stratification of experience into 3 classes, previously validated for microsurgery4 and oculoplastics,5 results in a broad range of values within each group. This range is reflected in high standard deviations (Table). The broad range is probably attributable to wide variations in surgical experience within each given group; for example, we would expect the measures to be different when comparing one test subject who performed 5 prior procedures and another who performed 99, although both would be classed as having intermediate experience.
The FD and PDFr variables were able to discriminate between surgeons with different levels of experience performing a simulated plastic surgical task (Table). In this test situation, the expert subjects had a significantly lower mean FD and a higher mean PDFr than the intermediate and novice test subjects (Table). The lower FD for experts suggests that those with greater experience displayed a smaller range of movements during testing and that, with increased experience, the most common magnitude of movement for an experienced test subject is smaller and less movement variation occurs (ie, the peak probability is higher with increased skill [Figure 2]). Because the magnitude of movement is calculated from consecutive data frames (at constant time intervals), a lower estimate of magnitude indicates a slower speed for that time segment. Together, these results for FD and PDFr imply that experts use a narrower range and slower movements.
Performing a task with a higher precision generally requires more time and, conversely, the faster a task is performed, the more likely errors are to occur; this trade-off between speed and accuracy is described by Fitts' law.7 At first inspection, the findings of this study appear to contravene Fitts' law; experts have been shown to complete a task more quickly,4,5 but the present data show them to display greater precision (with greater control and fluidity). The enigma is explained by the greater precision (slower and more accurate movements) of the experts, resulting in a reduced time to completion, rather than the reduced time being a manifestation of rushing and inaccuracy.
These results, in conjunction with other motion analysis studies,5 can also be used to inform wet laboratory and operating room instruction. Experts appear to use a more efficient pattern of preplanned experiential movements that trainees should aim to emulate. A “think before you act” approach leads to slower, more purposeful movements, allowing for the deployment of fewer actions. Therefore, the task is completed more quickly and with less tissue handling or collateral damage. Inefficiencies in task performance can thus be targeted in a more objective fashion and improved in simulated wet laboratory environments, before such tasks are undertaken on patients. Trainees may also be able to learn from this paradox: performing tasks hurriedly does not necessarily mean that the task is performed more quickly. In wet laboratories and actual surgery, selecting the correct movement and undertaking it correctly is the key to completing the task more quickly and safely.
Current changes in the content and delivery of medical education are driving the development of more objective and standardized systems of evaluation.8 Objective structured tools, such as the Objective Structured Assessment of Technical Skill,9 Ophthalmic Plastic Surgical Skills Assessment Tool,10 and others, are proving very useful in helping accomplish this goal, but they retain a degree of assessor-dependent subjectivity.8- 18 Kinematic and motion analysis, however, is purely objective in its data acquisition and analysis.4,5,13 The results of this study support previous work suggesting that motion and kinematic analysis may have a role in helping formulate a more objective system of technical skill appraisal.3- 6,13 The process may be augmented further by combining the motion evaluation technique with the available objective structured tools. This combination would be simple to accomplish because the motion capture technology has a video capture mode synchronized with the motion tracker (Figure 3).
Motion tracking and kinematic analysis could have several useful roles in surgical training. Indirectly, kinematic analysis may aid the development of parameters in formative feedback tools15,16,19 by generating metrics used in virtual reality simulation and general skills feedback as ascertained herein. The technique also has more direct applications, such as benchmarking skills acquisition in wet laboratories and quantitative feedback in surgical tasks or procedures. Ideally, this technique should be used at the earliest stages of training, where it has been shown to have the greatest ability to discriminate skills, and any anonymized comparisons would be made with the expert category, where the skills convergence produces a much narrower normative pool. The present study and others have shown construct validation, but as yet, no studies examining predictive validity have been published, so its usefulness as a marker of progression has not been established clearly.
The information presented on the techniques and variables evaluated already have useful applications in wet laboratory and operating room settings. However, the described technology is expensive and occupies a large working area, so further development of technological modalities are required before broader use of motion tracking can be used in training programs. Whichever instrument ultimately proves most suitable, motion analysis is useful as an adjunct to, but certainly not a substitute for, all the other tools presently used in residency training.
Experienced trainers often appear to show an intuitive ability to recognize differences in manual control in students at varying levels of training, but the kinematic analysis used in this investigation provides an objective, quantitative, and validated method for recording these differences in surgical control and fluidity, thereby adding a further valuable dimension to previously validated linear variables.4,5 Indeed, the significant difference between a subject with intermediate skills and a novice might be used as an index of completion of surgical training; likewise, the difference between a subject with intermediate skills and an expert might provide a measure of suitability for the latter group to adopt a practical teaching role in medicine.
The surgical task has inherent limitations because it is a limited test of technical competence in a wet laboratory environment; however, this task represents a core plastic surgical skill with which residents should be familiar. This tool provides a useful adjunct to current systems for evaluation of manual dexterity. The higher-order derivatives provide objective feedback on dexterity during this basic surgical procedure; these variables could, therefore, be used to monitor progress and direct future training. Furthermore, if a large normative pool of data were collected, this method could be used for comparative analysis with outcomes.
Correspondence: George M. Saleh, FRCSEd, FRCOphth, Orbital and Oculoplastic Service, Moorfields Eye Hospital, National Health Service Foundation Trust, National Institute of Health Research Biomedical Research Centre and Institute of Ophthalmology, 162 City Rd, London EC1V 2PD, England (firstname.lastname@example.org).
Submitted for Publication: April 11, 2012; final revision received June 21, 2012; accepted June 26, 2012.
Conflict of Interest Disclosures: None reported.