Identification and Categorization of Technical Errors by Observational Clinical Human Reliability Assessment (OCHRA) During Laparoscopic Cholecystectomy

Hypothesis  Surgical operative performance benefits from analysis of the mechanisms underlying technical errors committed during surgery.

Design  Prospective study using the Observational Clinical Human Reliability Assessment (OCHRA) system and complete unedited videotapes of the operations.

Setting  Three National Health Service hospitals within the United Kingdom.

Patients  Two hundred consecutive patients with symptomatic gallstone disease.

Interventions  Elective laparoscopic cholecystectomy for symptomatic gallstone disease by surgeons, who were blind to the nature and objectives of the study, using their usual operative technique.

Main Outcome Measures  Surgical consequential and inconsequential operative errors.

Results  The analysis of 38 062 steps of the 200 laparoscopic cholecystectomies performed by 26 surgeons identified 2242 errors. The mean ± SD total, inconsequential, and consequential errors per surgical procedure were 11.0 ± 8.0, 8.0 ± 6.0, and 4.0 ± 3.0, respectively. Dissection of the Calot triangle (second task zone of the operation) incurred more total errors (6.5 ± 5.4) compared with the first (2.9 ± 2.8, P<.001) and third (5.1 ± 3.9, P<.05) task zones. This translated to a higher error probability (6.9% vs 3.5% for the first and 5.5% for third task zones). The combined sharp and blunt dissection method had fewer errors than the blunt/teasing dissection technique (9.45 ± 7.6 vs 13.9 ± 7.3, P<.001) although different surgeons were involved. The most serious consequences were encountered during dissection with the electrosurgical hook knife.

Conclusion  This study has confirmed that the Observational Clinical Human Reliability Assessment system provides a comprehensive objective assessment of the quality of surgical operative performance by documenting the errors, the stage of the operation in which errors are enacted most frequently, and where these errors have serious consequences (hazard zones).

Surgery contributes to almost 50% of all adverse events and to 13% of all hospital deaths.^1-3 If the Institute of Medicine is right,³ 100 patients die from iatrogenic injuries in US hospitals each day.⁴ Some 40% of these injuries are committed in the operating room.^1,2 Population-based studies have indicated that most of these adverse events are preventable.^1-7 Therefore, prescriptive error-reduction systems are important in ensuring a good surgical outcome. However, these error-reduction systems have to be based on objective information about the nature of intraoperative technical errors and their mechanisms.

Morbidity and mortality data (MMD) cannot provide this information. Apart from the problems in reporting complications,^8,9 there are other limitations to the use of MMD as the sole indicators of surgical performance. Morbidity and mortality data identify the consequences of rather than the specifics of the adverse events. Not surprisingly, MMD studies overlook 65% to 90% of the adverse events.^10,11 In addition, they do not differentiate the exact role of technical skills from other components of clinical competence and factors related to teamwork and dynamics.

For all of these reasons, MMD cannot provide prescriptive information that specifies how the execution of an operation can be improved. This requires analysis of the mechanisms underlying technical errors and human factors that shape the performance of surgeons. The human factors approach has been used with considerable success by high-risk industries to study and to enhance human performance of complex, dynamic, and interactive tasks. In a pilot study, we demonstrated that it is possible to use such an approach based on industrial human reliability assessment to record and to identify intraoperative errors committed during endoscopic surgery.¹² Since then, the system has been developed further as the Observational Clinical Human Reliability Assessment (OCHRA) and is being applied to other more complex laparoscopic and open cancer operations. This study sought primarily to investigate the patterns of failure underlying both consequential and inconsequential errors and was based on the analysis of 200 laparoscopic cholecystectomies (LCs) using the OCHRA system. The study also investigated the error rates of the 2 techniques of endoscopic tissue dissection used by the participating surgeons.

Laparoscopic cholecystectomy was selected as an index common laparoscopic operation.¹³ Observational data capture of all errors was obtained from unedited videotape recordings of 200 consecutive LCs included in the OCHRA system investigation. The participating surgeons used “their usual” technique for carrying out the operations.

The definition of error agreed on at the Bellagio Conference on Human Error was used for the present study, that is, “ . . . something that has been done which was: (i) not intended by the actor, (ii) not desired by a set of rules or an external observer, or (iii) that led the task or system outside acceptable limits.”¹⁴ The external expression of an error is its consequence which may be neutral (inconsequential) or negative (consequential). This definition is similar to but more comprehensive than that proposed earlier by Swain and Guttmann.¹⁵

We considered any action or omission that resulted in a negative consequence or increased the time of the surgical procedure by necessitating a corrective action, that fell outside of the “acceptable limits” and was, therefore, registered as a consequential error. We defined inconsequential error as action or omission that increased the likelihood of negative consequence and under slightly different circumstances could have had a consequential effect.

The details of the error classification system were described in the pilot study.¹² In brief, 10 generic forms of error can be predicted for the execution of a surgical task.¹⁶ These 10 generic types or external error modes represent observed patterns of failure (Table1) and fall in 2 categories in relation to the underlying causative mechanism. External error modes 1 through 6 correspond to the ability of the surgeon to execute the component steps in the correct order and, hence, these are collectively grouped as “procedural error modes.” In contrast, external error modes 7 through 10 reflect manipulations with endoscopic instruments by the surgeon to execute a specific component step of the operation and are categorized as “execution error modes.” This distinction is of practical importance because it determines the nature of the prescriptive error-reduction system specific to the operation. Execution errors can be reduced by better training of operative skills and by improving instrument design; whereas, procedural errors can be minimized by improving the knowledge (perhaps aided by drop-down menus) that ensures the correct choreography of execution, that is, the surgeon performs the component tasks and steps of the operation in the correct order.

The hierarchal task analysis involved the division of LC into 3 component task zones.^12,13 The first task zone was the division of adhesions involving the gallbladder and adjacent organs/omentum in the right upper quadrant. The second task zone involved dissection of cystic pedicle and division of the cystic artery and duct. The third task zone consisted of separation of the gallbladder from the liver bed followed by extraction. These tasks were standardized by preset criteria for levels of difficulty (I [easy]-III [most difficult]) for each operation.¹⁷

As it happened, 2 dissection techniques were used by surgeons who participated in the study. The first involved a combination of sharp and blunt dissection with the electrosurgical hook knife, scissors, and pledget; whereas, the second consisted of blunt/teasing dissection of tissue with endoscopic graspers as the only dissecting instrument.

All videotapes were coded for the surgeon and participating National Health Service hospital prior to analysis. A single clinical research fellow (B.T.), who had completed 10 years of surgical training to chief-resident level, carried out direct observational methods of unedited videotapes. Before the study, the clinical research fellow had 8 months’ training in human factors research by an accredited human factors specialist and 2 academic surgeons with interest in surgical ergonomics and human factors and who were not involved in the operations. The interrater consistency of the OCHRA system had been assessed by the clinical research fellow and human factors specialist in the initial pilot study and was found to be 85%. The expert panel provided consultation for the clinical research fellow throughout the study and checked the accuracy of the videotape analysis process. All cases that resulted in postoperative complications or conversion were reviewed by the expert panel.

Each step of the operation was observed to record committed errors and their external modes. The consequence of each error and any corrective action required were identified. Errors were analyzed for each task and instrument used. Postoperative complications were registered from patients’ hospital records. The error probability for each task was calculated as:

(Total No. of Enacted Errors/No. of Steps Performed to Complete the Task) × 100%.

Similarly, the instrument error probability was calculated as:

(Total No. of Observed Errors/No. of Steps Using the Instrument) × 100%.

Dissection techniques were compared using the t test. Statistical significance level was set at P≤.05. Quantitative error data are expressed as mean ± SD.

Each surgeon had carried out at least 20 LCs before participating in the study. The combined sharp and blunt dissection method was used in 112 operations, whereas the endoscopic grasper was used as the only instrument for blunt/teasing dissection in 88 operations. Analysis of 38 062 constituent steps of the 200 LCs identified 2242 errors. Of these, 684 (30%) were errors with consequence (consequential) while 1558 (70%) errors had no consequences (inconsequential). The mean total, inconsequential, and consequential errors per procedure were 11.0 ± 8.0, 8.0 ± 6.0, and 4.0 ± 3.0, respectively. The nature of errors committed during the component tasks of the operation are shown in Table 2. The second task of the operation (dissection of the structures in the Calot triangle) was associated with more total errors (6.5 ± 5.4) compared with the first (2.9 ± 2.8, P<.001) and third tasks (5.1 ± 3.9, P<.05). This translated to a higher error probability (6.9%) compared with the first and third tasks (3.5% and 5.5%, respectively). As given in Table 3, the higher total errors and error probability during the second task were observed for both combined and blunt dissection techniques.

Two cases were converted to open surgery (small-bowel injury and uncontrollable bleeding from a cystic artery). Another 2 patients had postoperative complications. The first patient was readmitted with rigors and resolved jaundice. Abdominal ultrasonography showed no significant fluid collection and findings from endoscopic retrograde cholangiography were normal. The patient responded to conservative treatment with antibiotics and was discharged home with no long-term consequences. Analysis of the videotape revealed 40 errors (3 in task 1, 18 in task 2, and 17 in task 3) including injury of the cystic duct, perforation of the gallbladder, and bleeding from small vessels and the liver bed. No common bile duct injury was observed in the videotapes, which was confirmed by normal postoperative endoscopic retrograde cholangiography. We concluded that this patient may have had a small infected postoperative collection of fluid that was missed by ultrasonography. Mild postoperative pancreatitis (hyperamylasemia with no systemic effects), which settled conservatively, developed in the second patient. Analysis of this patient’s videotape revealed 13 errors (8 in task 2 and 5 in task 3); of which 6 were inconsequential, 4 were burns to the liver surface, and 1 caused minor liver bleeding.

The errors needing corrective action totaled 281. Perforation of the gallbladder (n = 103) was treated by closing the hole, suctioning the bile, and retrieving the stones from the peritoneal cavity. Injury of the cystic duct (n = 7) was closed with a clip and by applying an extracorporeal knot or by suturing. Bleeding from injury to the liver (n = 54), cystic artery (n = 7), small vessel (n = 91), and omentum (n = 5) was stopped using electrocoagulation, irrigation and/or suction, and/ or pressure with a pledget.

The combined dissection method had fewer total errors than the blunt/teasing dissection technique (9.4 ± 7.6 vs 13.9 ± 7.3, P<.001). The lower error rate associated with combined dissection was observed for both consequential errors (2.8 ± 2.5 vs 5.1 ± 3.2, P<.001) and inconsequential errors (6.9 ± 6.5 vs 8.8 ± 5.7, P<.0001) and during individual tasks (Table 3). In the combined dissection method, more errors and a higher error probability were noted when the surgeon was using the electrosurgical hook knife compared with sharp scissors dissection. The pledget had the lowest number of errors (Table 4). A high error probability was observed with holding of the endoscopic grasper although most errors were inconsequential (slippage on holding the gallbladder).

Table 5 outlines the consequential and inconsequential errors committed using the dissecting endoscopic grasper and the electrosurgical hook knife. Failure to visualize the instrument tip during dissection, use of excessive force, and wrong instrument direction/spatial orientation, respectively, accounted for 14.6%, 11.3%, and 73.8% of consequential errors committed with the dissecting endoscopic grasper and 12.9%, 31.6%, and 39.7% of consequential errors encountered on using the electrosurgical hook knife. Perforation of the gallbladder and bleeding from a liver injury were mainly caused by the use of excessive force or dissection in the wrong tissue planes. Electrosurgical burn to the liver resulted usually from wrong instrument direction and spatial orientation. Bleeding from small vessels was caused by nonvisualization of instrument tip and excessive force during dissection. The use of electrosurgical hook knife resulted in serious injuries such as perforation of the small bowel and uncontrolled bleeding from a cystic artery requiring conversion. A higher incidence of cystic artery and ductal injuries as well as electrosurgical burn to the duodenum and diaphragm was noted. These injuries were caused either by nonvisualization of the tip of the hook during dissection or by the use of excessive force resulting in the “follow through” effect. Failure to visualize the scissors blade tips accounted for 61% of consequential errors associated with this instrument, whereas the use of excessive force accounted for all consequential errors committed with the pledget.

Analysis of the error modes for underlying inconsequential errors (Table 5) indicates that failure to visualize the instrument tip during dissection, use of excessive force, and wrong instrument direction/spatial orientation, respectively, accounted for 73.0%, 19.0%, and 5.5% of the errors committed with the dissecting endoscopic grasper, and 6.0%, 53.0%, and 42.0% of the errors committed using the electrosurgical hook knife. In addition, failure to visualize the instrument tip during dissection was causative in 68% of the errors with the electrosurgical hook knife and all errors enacted with the pledget.

Procedure-based errors accounted for about 20% of consequential and 40% of inconsequential errors. Steps done with excessive or little force/distance and failure to visualize the instrument tip during dissection were the external modes associated with 86% of the inconsequential errors. Movement in wrong direction/point in space, with excessive force/distance, and failure to visualize the instrument tip during dissection were the external modes associated with 95% of the consequential errors. 

There is a growing interest in medical errors and adverse events.^18-22 Most published work has investigated errors at a system level^1-7 whereas the current study focused at the “coal face,” that is, technical performance of an operation. Research at both system and distal (coal face) levels is necessary and complementary.

The use of the OCHRA system to assess the quality of surgical operative performance has many advantages over retrospective population-based research. In the first instance, the system provides objective, complete tracking of errors related to the performance of a specific operation. Second, it identifies hazard zones of an operation where technical errors occur most commonly and are likely to jeopardize clinical outcome. The present study confirmed that the dissection of the Calot triangle was associated with the highest error probability with both dissection techniques and with the most serious consequential errors, for example, injury to the cystic pedicle, perforation of the small bowel, and others. All conversions and postoperative complications resulted from errors committed during the dissection of the Calot triangle. These observations identify the dissection of the Calot triangle as the hazard zone of LC. Defense systems to minimize errors or reduce their effect would have to be concentrated to this zone as far as LC is concerned.

The effect of an error mode varied with the zone in which it occurred—the co-incidental external circumstances of the error. For instance, excessive force with the electrosurgical hook knife can cause overshooting with no consequence, innocuous burn to the abdomen, or perforation of the small bowel. In this respect, the occurrence of serious consequence of an error is a chance phenomenon. Hence, operating systems that minimize all errors in hazard zones are essential for safe execution of an operation.

The study of inconsequential errors captured by the OCHRA system is equally important as it serves as a human factors research tool providing data of adequate size necessary for the formulation of error-reduction mechanisms. High-risk industries have used these near-miss errors to develop and to implement error-reduction/tolerant systems.^22,23

The study also demonstrated that the combined sharp and blunt dissection method had fewer errors than the sole blunt/teasing dissection technique although the study was not randomized and each surgeon used the technique to which he or she was accustomed. As the most commonly used dissecting instrument for LC, the electrosurgical-powered dissection instrument, such as the electrosurgical hook knife, carried a higher error probability than nonenergized dissection by pledget or scissors. The poor design of the electrosurgical hook knife is largely responsible for the error modes observed on usage of this instrument. Alternative methods of tissue dissection with ultrasonic dissectors and impedance-controlled electrosurgical^24,25 systems have not been subjected to ergonomic evaluation.

Better cognitive training should reduce procedural errors. Practical training modules in surgical skills laboratories should be directed to overcome wrong movementdirection in tissue planes, use of excessive force with inappropriate movement distance, and nonvisualization of the instrument tip during dissection as collectively these error modes accounted for 95% of the consequential errors.

Finally, we acknowledge that the OCHRA system is labor intensive and needs considerable expertise and resource. It cannot, thus, be used as a Surgical Error Reduction System and has to be regarded as a human factors research tool by which the performance of specific operations (open and endoscopic) can be assessed. The results of these studies can form the basis for an effective Surgical Error Reduction System specific to individual operations. These systems would incorporate defenses against common errors within hazard zones and nonpunitive incident reporting with built-in “root cause analysis”²⁶ or systems analysis²⁷ that collectively ensure quality execution of surgical operations which thereby guarantee the clinical outcome. We can never of course abolish surgical error completely, but we can reduce it to the as-low-as-reasonably-possible region. As Berwick⁴ indicates, we are, at present, well short of this target.

Correspondence: A. Cuschieri, MD, FRSE, Surgical Skills Unit, Ninewells Hospital and Medical School, University of Dundee, Dundee DD1 9SY, Scotland (a.cuschieri@dundee.ac.uk).

Accepted for Publication: May 31, 2004.

Funding/Support: This study was supported by grant K/OPR/15/10/F16 from the Chief Scientist Office of Scotland, Edinburgh (Drs Hanna and Cuschieri).

Previous Presentations: Preliminary results of this study were presented at the following meetings: Society of Academic and Research Surgery (Patey Prize session), Leeds, England; January 9, 2003; and Society of University Surgeons, Houston, Tex; February 16, 2003.