Human Error Identification: An Analysis of Myringotomy and Ventilation Tube Insertion

Objectives  To use a human reliability assessment tool to identify commonly occurring errors during myringotomy and ventilation tube (VT) insertion and to quantify the likelihood of error occurrence.

Methods  Error-free task analysis for myringotomy and VT insertion was defined at the outset. Fifty-five consecutive myringotomy and VT insertion procedures were videotaped. The operator was either the senior author (S.S.M.H.) or a trainee in the specialist registrar or senior house officer grade. Three assessors (M.-L.M., M.S.W.L, and S.S.M.H.) blinded to operator identity independently evaluated each procedure. Interobserver agreement was calculated (κ values).

Results  Twelve potential error types were identified. A total of 87 errors were observed in 55 procedures. In 53% of procedures (n = 29) multiple errors were identified. Seven percent of procedures (n = 4) were error free. The 4 most frequent errors identified were (1) failure to perform a unidirectional myringotomy incision (n = 37; 43%); (2) multiple attempts to place VT (n = 14; 16%); (3) multiple attempts to complete the myringotomy (n = 11; 13%); and (4) magnification setting too high (n = 11; 13%). The human error probability was 0.13. Interobserver agreement as expressed by κ statistics was high.

Conclusions  Human error identification in this most common of otologic procedures is crucial to future error avoidance. Eliminating the 2 most common errors in this model will halve the human error probability. Extending the role of error analysis to error-based teaching as an educational tool has potential.

Operative procedures of all types are potentially error prone. Unfortunately, much negative publicity is generated by newspaper headlines publicizing medical errors. These reports are rarely accompanied by informed comment on likelihood or cause. The causes of adverse events in health care relate to errors of operative procedures, drugs, medical procedures, and diagnoses.¹ Many of these errors are preventable.

Accidents such as that at Chernobyl have demonstrated the need to consider human error in high-risk systems. Such accidents have boosted research and development in the field of human reliability assessment (HRA). Human error identification is critical to assessing risk. Human error contribution to risk is assessed through the use of HRA tools. The role of HRA extends to many industries, including health care—and in particular surgery—wherever human errors propagate within systems.

The first ventilation tube (VT) was inserted by Armstrong in 1954,² but it was Sir Astley Cooper³ who popularized the procedure of myringotomy in the early 1800s as "an operation for the removal of a particular species of deafness." This refers to the very common condition of otitis media with effusion, more often known as glue ear. The procedure of myringotomy and insertion of a VT or "grommet" is usually recommended only for persistent disease with significant hearing loss causing symptomatic morbidity.⁴

We chose to analyze myringotomy and VT insertion for several reasons: (1) It is the most commonly performed otologic operation in the United Kingdom: 37 544 procedures were performed during 2001-2002.⁵ (2) It is usually the first ear operation learned by surgical trainees. (3) It has a number of discernible surgical stages, and each stage is potentially error prone, and (4) the entire procedure can be easily videotaped to provide copies for repeated viewing and analysis.

The procedure is performed with the aid of an operating microscope, which allows visualization of the tympanic membrane through a handheld speculum in the external auditory canal. The speculum is usually placed and maintained in position with the operator's nondominant hand leaving the dominant hand free for surgical maneuvers. A certain amount of dexterity is required, and this can only be acquired with practice and experience.

The errors that potentially occur during other otologic procedures, such as ossiculoplasty and stapedectomy, are in theory associated with greater morbidity or poorer long-term hearing outcomes. It is for this reason that we chose to analyze myringotomy and VT insertion in preference to other otologic procedures. In addition, these other procedures, although also performed with the aid of an operating microscope and having discernible stages that can be videotaped, tend to be performed by consultants with an established otologic practice or by senior trainees under direct consultant supervision, and these procedures are performed much less frequently than myringotomy and VT insertion.

Previous studies have applied HRA tools in the assessment of surgical errors occurring during laparoscopic⁶ and nasendoscopic⁷ surgery. To our knowledge, microscopic surgery has never been assessed in this way. The aims of the present study were to identify commonly occurring errors during myringotomy and VT insertion and to quantify the probability of error occurrence by applying a well-known HRA tool. Throughout the study, no moral connotation of personal failure or blame has been ascribed to the operator because this would be counterproductive to the HRA process. The term error in the context of this study refers to an undesirable event occurring during the procedure. Therefore, the occurrence and identification of a particular error does not equate to an injurious or harmful patient event but rather to an undesirable surgical event, the long-term implications of which are somewhat controversial at present and will require further prospective study.

This study applies an HRA technique well known in the field of industrial ergonomics—the Technique for Human Error Rate Prediction (THERP).⁸ This particular model considers external error modes at each step in the procedure defined in the task analysis (Table 1).

We defined error-free sequential task analysis for myringotomy and VT insertion at the outset according to a description of the procedure accepted to be standard practice in the United Kingdom.⁹ A sequence of 10 steps had to be completed appropriately for the entire procedure to be deemed error free (Table 2). We subsequently specified all conceivable errors that could occur during the procedure by identifying potential external error modes at each step in the sequence. The significance of each error type for immediate and long-term functional outcomes was not the subject of this study; therefore, no attempt was made during the study to assign clinical importance to each possible error type.

Fifty-five consecutive myringotomy and VT insertion procedures were videotaped under normal operating room conditions. A standard myringotomy tray with a unidirectional myringotome was opened for each procedure. The operator was either a consultant (S.S.M.H.) or a trainee in the specialist registrar or senior house officer grade. A conventional short-stay Shah VT was inserted in all cases. Subsequent real-time video playback allowed each procedure to be independently evaluated by 3 assessors blinded to operator identity (M.L.M., M.S.W.L., and S.S.M.H.) Evaluation for each procedure comprised identification and recording of all errors according to the agreed prespecified error types. Notes were also made by each assessor to document possible reasons for error occurrence and to record possible error pathways or cascades. Interobserver agreement was calculated using κ statistics. Human error probability (HEP), defined as an estimate of how often an error will occur in the task under analysis, was calculated using the formula HEP = number of errors observed/number of opportunities for error.

Twelve potential error types were identified after application of the THERP method to external error modes at each step in the task analysis (Table 3). A total of 87 instances of an error were observed in the 55 videotaped performances of the procedure. Twenty-eight procedures (51%) were performed on the right ear, 27 procedures (49%) on the left ear.

The frequency of each error type is shown in Figure 1. The most frequent error identified was failure to perform a unidirectional myringotomy incision (n = 37; 43%). The next most frequent errors were multiple attempts to place the VT (n = 14; 16%), multiple attempts to complete the myringotomy (n = 11; 13%), and microscope magnification set too high (n = 11; 13%). Four of the potential error types specified—incorrect myringotomy site, myringotomy incision too large, use of a circumferential incision, and failure to aspirate middle-ear fluid prior to VT placement—did not occur in the 55 procedures analyzed.

The number of attempts required to place the VT ranged from 1 (n = 38; 69%) to 7 (n = 1; 2%). Two attempts were required in 9 cases (16%); 3 attempts in 2 cases (4%); 4 attempts in 2 cases (4%); and more than 4 attempts in 4 cases (7%).

The numbers of error-free, single-error, and multiple-error procedures performed in the study are shown in Figure 2. In 53% of procedures (n = 29) multiple errors were identified. Only 7% of procedures (n = 4) were completely error free. The overall HEP was calculated to be 0.13: 87/(55 × 12). The HEP for the most frequently occurring error—failure to perform a unidirectional myringotomy incision—was 0.67: 37/55. Interobserver agreement, as expressed by the κ statistic, was high (Figure 3).

A number of approaches, ranging from very simple error classification schemes to analyses that require highly sophisticated software, have been developed to identify human error in human-machine systems. The THERP system is the most well-known HRA model, and it is simple to apply.⁸ In industrial settings, THERP comprises 4 phases (Table 4).¹⁰ The present study uses a simplified clinical modification of this HRA approach. Provided that it is administered by assessors with a sound knowledge of the task under analysis, the THERP system has the advantage of being systematic and comprehensive and thus provides an accurate reflection of HEP. It can be applied to any human task situation to derive error modes. To date it has been largely used in the context of nuclear power plant scenarios.

Defining what constitutes error in the context of a surgical procedure can be difficult because a range of approaches to a given task might be considered acceptable surgical practice. For example, whether the aspiration of middle-ear effusions prior to VT placement is really necessary is somewhat contentious. Standard practice favors this step because when it is taken, patient hearing is restored instantaneously and there is a lower chance that viscid secretions might block the VT.⁹ In theory, there is a risk that noise generated at the suction tip during aspiration may result in acoustic trauma or noise-induced hearing loss.¹¹ Operationally, error can be defined as "the failure to perform an action (including inaction if required) within the tolerance limits necessary for adequate and safe system performance."⁸ In surgical practice, the relative importance of the errors must be decided by a group of experts on the subject.

While some errors carry no seqelae, some may have serious or even fatal consequences and so deserve serious consideration. Having specified what constitutes error in this study, we believe it is important to determine the implications of such errors with respect to long-term outcomes, including effects on hearing, development of tympanosclerosis, VT-extrusion rates, and recurrence of otitis media with effusion. This will be the subject of a later report.

It is already known, however, that hemorrhage into the middle layer of the tympanic membrane is associated with the development of tympanosclerosis.¹² Excessive damage to the fibrous layer may also be associated with scar formation at the incision site. It would seem prudent therefore to minimize surgical trauma to the tympanic membrane during the procedure. Multiple attempts to complete the myringotomy and multiple attempts to place the VT were commonly found by the assessors to be associated with hemorrhage that obscured the operative field. The most frequent error identified—use of a bidirectional incision—was also frequently accompanied by hemorrhage. This can be caused by an inappropriate bidirectional "sweep" action performed with a unidirectional-blade myringotome (Figure 4A), the blunt side of the blade causing the trauma on the return sweep. The same trauma does not result from this action when a bidirectional myringotome (Figure 4B) is used.

Wax clearance was deemed to be traumatic by the assessors in only 1 case. This resulted in hemorrhage from the external auditory canal wall skin even before the myringotomy was begun. As a result of this error, multiple attempts were needed to complete the myringotomy and place the VT—an example of an error pathway or cascade. Error recovery could have been achieved with hemostatic control using a topical adrenaline pledget prior to proceeding with the myringotomy. The identification of error cascades and error recovery paths is largely dependent on the assessors' judgment.¹³

A disadvantage of the THERP model applied in this study is that it does not explain why the identified errors occurred. The relative effects of specific performance-shaping factors relating to personnel (eg, training and experience) and ergonomics (eg, equipment, instrumentation, workspace environment and work demands, and personnel communication) were not considered in this study. Considerable variation between how different assessors identify errors is also possible. In this study, interobserver agreement as measured by κ statistics was high (0.8-1.0).

Once the range of errors that can occur and their frequency have been identified, the predominant errors can be targeted to ensure future error reduction. In this model, if the 2 most frequently occurring errors were eliminated from practice, the HEP would be halved (Figure 5). A similar approach could be used to determine how surgeon grade or experience influences error probability. Indeed, the prospective analysis of error probability as specialty-specific training progresses could be used as a competency-based assessment tool. Other areas open to investigation using this model include the relationship between HEP and surgeon handedness and a comparative analysis of the HEP associated with the placement of the wide range of VT types. Above all, this clinical application of an HRA tool offers substantial audit potential.¹⁴

In conclusion, medical errors cannot be ignored. Identifying human error using this "black box recorder" approach will improve error recognition and is crucial to future error avoidance. Elimination of common errors can reduce error probability. Enormous potential exists for the application of human error detection models to error-based teaching in all surgical specialties.

Correspondence: S. S. M. Hussain, FRCS, Department of Otolaryngology (Ward 26), Ninewells Hospital and Medical School, Dundee, DD1 9SY Scotland (s.s.musheer.hussain@tuht.scot.nhs.uk).

Submitted for publication December 15, 2003; final revision received April 6, 2004; accepted April 30, 2004.

This article was presented at the 11th British Academic Conference in Otolaryngology meeting; July 3, 2003; Birmingham, England.