Patient Record Review of the Incidence, Consequences, and Causes of Diagnostic Adverse Events

Background  Diagnostic errors often result in patient harm. Previous studies have shown that there is large variability in results in different medical specialties. The present study explored diagnostic adverse events (DAEs) across all medical specialties to determine their incidence and to gain insight into their causes and consequences by comparing them with other AE types.

Methods  A structured review study of 7926 patient records was conducted. Randomly selected records were reviewed by trained physicians in 21 hospitals across the Netherlands. The method used in this study was based on the well-known protocol developed by the Harvard Medical Practice Study. All AEs with diagnostic error as the main category were selected for analysis and were compared with other AE types.

Results  Diagnostic AEs occurred in 0.4% of hospital admissions and represented 6.4% of all AEs. Of the DAEs, 83.3% were judged to be preventable. Human failure was identified as the main cause (96.3%), although organizational- and patient-related factors also contributed (25.0% and 30.0%, respectively). The consequences of DAEs were more severe (higher mortality rate) than for other AEs (29.1% vs 7.4%).

Conclusions  Diagnostic AEs represent an important error type, and the consequences of DAEs are severe. The causes of DAEs were mostly human, with the main causes being knowledge-based mistakes and information transfer problems. Prevention strategies should focus on training physicians and on the organization of knowledge and information transfer.

Diagnostic error in medicine is an important error type. Of all error types, diagnostic error is the type that patients are most concerned about,¹ and in malpractice claims, many cases (≥30%) are classified as diagnostic errors.^2,3 Diagnostic errors occur in many cases^4-6 and can have major consequences for the patient.⁷ Commonly used methods to study diagnostic errors are voluntary reports of diagnostic errors by physicians,^6,8 analysis of closed malpractice claims,⁹ and autopsy reports.¹⁰ The incidence rates of diagnostic error in these studies vary widely depending on the research method. Different research methods lead to a focus on different subsets of patients or criteria for selecting diagnostic errors.¹¹ For example, autopsy studies identify incorrect diagnoses in patients who are deceased, and prospective studies usually examine diagnostic errors in a specific medical specialty. To obtain a complete overview of diagnostic errors, a sample that includes diverse hospital admissions is needed, which makes the use of autopsy reports or malpractice claims as a research method unsuitable.

Patient record review studies are used to study the occurrence of adverse events (AEs) and are suitable for selecting a diverse sample of hospital admissions. Some of these AEs are diagnostic AEs (DAEs), that is, diagnostic errors that lead to patient harm. Adverse events embody an important group of errors because they are the more severe errors.^7,8 So far, it is not known how often DAEs occur and how their frequency and severity compare with those of other AE types.

To gain more insight into the specific features of diagnostic error and to develop interventions to reduce diagnostic errors, it is important to understand their causes. Because diagnosis is a cognitive task that involves many decision-making skills, the causes of diagnostic error are generally considered to be due to human failure. Insufficient knowledge and decision biases are often given as the causes of diagnostic errors.¹² Other causes, such as poor management and equipment design, may also play a role.⁶ There are no significant data about the different causes underlying DAEs or the severity of the consequences of DAEs compared with other AEs. Classifying the causes of AEs in retrospect and based on the information in a patient record has the advantage that a diverse sample of cases can be investigated and can provide valuable information about the causes. However, the cause classification is limited compared with that in prospective studies.

Internationally, many studies^5,13-18 have been conducted using the AE criteria to assess the occurrence of medical errors involving patient harm. The study protocol and the definitions are clear and well known. However, diagnostic error has not been studied extensively using these criteria. Therefore, we conducted a large patient record review study using a diverse cross-sectional sample of all hospital admissions in the Netherlands. The DAEs were identified and examined to gain more insight into the incidence, consequences, and causes of DAEs compared with other AE types.

A systematic 3-stage retrospective patient record review study was conducted including 7926 patient records of discharged and deceased patients admitted to Dutch acute care hospitals in 2004. Of a nationwide stratified random sample of 40 invited hospitals, 21 (4 [of 8] university hospitals, 6 [of 19] tertiary teaching hospitals, and 11 [of 74] general hospitals) participated in the study; 19 hospitals refused to participate for various reasons (eg, already participating in other patient safety studies and insufficient personnel available). From each hospital, a stratified random sample was selected of 200 admissions of patients discharged from the hospital and 200 admissions of patients deceased in the hospital (or less if the total number of patients who died in 2004 was lower). The index hospital admission was the admission that was selected to be reviewed in the study. The AEs were included if they occurred during this index admission and were detected during or within 12 months of the index admission. The AEs were also included if they were related to hospital admissions in the same hospital in the 12 months preceding the index admission but were not detected until the index admission.¹⁹ The method used in this study was based on the protocol of the Canadian Adverse Events Study,⁵ which was originally developed by the Harvard Medical Practice Study in New York in 1984.¹³

Between August 1, 2005, and October 31, 2006, trained nurses and physicians reviewed the patient records during a 3-stage process. During the first review stage, 66 nurses reviewed the randomly selected patient records in a consecutive order (1 nurse per record) and determined whether any of the 18 triggers (clues) for a potential AE were present (eg, readmission or unexpected death). Patient records with 1 or more triggers were selected for further review (4317 patient records). Fifty-five physician reviewers from the medical specialties of internal medicine, surgery, neurology, and pediatrics participated in the second review stage. During the previous stage, nurses indicated which medical specialty would be most suitable for reviewing each particular record. Two physician reviewers from the indicated specialty reviewed each record independently and determined whether an AE had occurred. The determination of an AE was based on 3 criteria: (1) an unintended (physical or mental) injury that (2) resulted in prolongation of the hospital stay, temporary or permanent disability, or death and (3) was caused by health care management rather than the patient's disease.²⁰ If an AE was identified, a variety of questions about the AE were asked, such as AE category (diagnostic, surgical, drug/fluid, medical procedure, other clinical management, discharge, and other), causes, consequences, preventability, and most responsible hospital department.

The third stage of the review involved reaching consensus. If the 2 physician reviewers did not agree on fundamental questions (ie, whether there was an AE or the level of preventability), they discussed the case. If the 2 physicians could not reach a consensus, a third physician reviewer with access to all the information determined the final judgment.

The reviewers selected all causes that contributed to the occurrence of the AE using the taxonomy of the Eindhoven Classification Model.²¹ The reviewers were asked to distinguish the main categories of the model: human, organizational, technical, patient related (ie, comorbidity and treatment adherence), and other causes. Subsequently, if the information was available in the patient record, more detailed subcategories were selected by the reviewers. These categories are based on the Rasmussen SRK (skill-based, rule-based, knowledge-based) model²² and on the theory of active and latent errors by Reason²³ and distinguish different types of human, organizational, and technical factors, such as knowledge-based mistakes and organization of protocols. It is difficult to distinguish between specific causes when using a patient record review, and not all contributing causes can be detected in the patient record. The causes that were identified in the record review were examined, and the causes of DAEs were compared with the causes of other AEs. Preventability was defined as care that fell below the current level of expected performance for practitioners or systems. Preventability was measured on a 6-point scale (1 = no preventability and 6 = definite preventability). Consistent with most previous international studies, the cutoff value was 4 because scores of 1 to 3 were considered to be AEs that were not preventable and scores of 4 to 6 were considered to be AEs that were preventable.

The reliability of the determination of AEs was assessed using a reliability study of 119 patient records. Each record was reviewed by 2 medical specialists. The reliability study between 2 pairs of reviewers on the assessment of whether an AE (all different types) occurred was fair (κ = 0.25; 95% confidence interval [CI], 0.05-0.45; 76% agreement). For determination of the preventability of the AEs, the reliability was moderate (κ = 0.40; 95% CI, 0.07-0.73; 70% agreement). More details about the reliability study can be found elsewhere.²⁰

All DAEs were selected for further analysis. For each AE, the physician reviewers indicated the main category of the AE (diagnostic, surgical, drug/fluid, medical procedure, other clinical management, discharge, or other). Although AEs sometimes fit into more than 1 category, only the AEs with “diagnostic” as a main category were selected for further analysis because DAEs were indicated as a secondary type in only a few cases. Because there were 2 reviewers per patient record, both reviewers could select a main AE category. If the 2 reviewers disagreed on the category, 2 other independent physicians decided on the final category, which was used in the subsequent analyses.

The physician reviewers both indicated all causes and consequences of the AEs, and the reviewers were not requested to reach consensus on the causes and consequences. As in the study by Smits et al,²⁴ all causes and consequences selected by both physician reviewers were reported to provide a broader perspective for improvement strategies. When both reviewers selected the same cause, it was counted once.

In the present study, a large sample of deceased patients was included to study this particular patient group in more detail. There was also a small overrepresentation of university hospitals. To convert the study sample into results representative of the Netherlands, a weighting factor was used to correct for overrepresentation of deceased patients and university hospitals in the study sample. The weighting factor was the inverse of the probability of being included in the sample and was calculated by dividing the representation of a group in the population by the representation of this same group in the study sample.¹⁹ This weighting factor was used in all the analyses involving a representation of the incidence of AEs.

For all the comparisons of incidence rates and consequences, descriptive statistics and frequency tables were used, as were tests for the comparison of proportions in 2 independent groups, corrected for binomial distribution.²⁵

The causes of DAEs were compared with the causes of other AE types using tests for comparison of proportions in 2 independent groups corrected for binomial distribution. The weighting factor was not used in the analysis of the causes because confounding by discharge status (death or alive) and hospital type were not present. We tested for confounding by assessing the change in the β level for each determinant after introducing the potential confounders into a logistic regression model. The change in the β level of interest was less than 10% in all instances for human and organizational causes and 11.8% for patient-related causes.

Of 8415 patient records selected for review, 383 could not be reviewed because they were unavailable or inadequate. Of the 8032 records that were eligible for review in the first stage of record review, 66 were not reviewed owing to hospitalization of the patient, records not meeting the inclusion criteria, or incomplete records.¹⁹ Another 40 records were excluded in the second stage. In total, 7926 patient records were reviewed.

The characteristics of the patients with a DAE are given in Table 1. Patients with DAEs were more often admitted to a general hospital than were patients with other AE types or patients without AEs. The hospital department was more often the internal medicine department. The mean age of patients with DAEs was higher than that for patients without an AE but similar to that for patients with another AE type. The autopsy report was available and used in 2 of 68 deceased patients with DAEs.

In total, 744 AEs were identified in this study, of which 80 were DAEs. Using the weighting factor to correct for overrepresentation of deceased patients and hospital type, the results showed that DAEs occurred in 0.4% (95% CI, 0.21%-0.59%) of hospital admissions and accounted for 6.4% (95% CI, 4.3%-8.5%) of all AEs. However, of all the AEs that were judged to be preventable, DAEs accounted for 13.4% (95% CI, 8.4%-18.4%) (Table 2). When selecting all DAEs, many were judged to be preventable (83.3%; 95% CI, 70.4%-96.2%), which was significantly higher than in other AE types (z = 5.02, P < .001).

Diagnostic AEs were more often discovered during a subsequent hospital admission (77.4%) than were other AE types (34.5%) (z = 4.60, P < .001) and occurred more often in an emergency setting vs other AEs (83.9% vs 41.9%, z = 4.37, P < .001). Diagnostic AEs more frequently occurred in the nonsurgical departments (75.0%) as opposed to the other AE types (33.2%, z = 4.58, P < .001), in particular in the departments of internal medicine (28.6%), cardiology (10.7%), and pulmonology (10.7%).

Diagnostic AEs covered a variety of cases (eTable). Pulmonary embolism was missed frequently (9 of 80 DAEs), as were different types of cancer (8 of 80 cases). Five DAEs involved missed diagnoses of sepsis. Diagnostic AEs involving coronary syndromes and appendicitis each occurred in 4 cases.

Diagnostic AEs more often contributed to patient death than did the other AE types (29.1% [95% CI, 13.0%-45.0%] vs 7.4% [5.0%-9.8%]; z = 3.98; P < .001) and resulted in significantly more readmissions to the hospital (38.7% [21.6%-55.8%] vs 22.7% [18.8%-26.4%]; z = 1.97; P = .05) (Figure 1). Additional interventions and treatment occurred less frequently for DAEs than for the other AE types (38.8% [95% CI, 21.7%-56.0%] vs 87.9% [85.0%-90.8%]; z = 7.03; P < .001), as did a prolonged hospital stay (28.1% [12.5%-43.7%] vs 46.7% [42.2%-51.2%]; P = .10).

More causes were reported for DAEs vs other AE types (mean, 2.7 vs 1.6). In nearly all DAEs, a human cause was present (96.3% [95% CI, 92.0%-100%]), which is significantly more often than in the other AE types (50.5% [46.7%-54.3%]; z = 7.67; P < .001) (Figure 2). Compared with the other AE types, organizational factors were also significantly more often involved in the occurrence of DAEs (25.0% [95% CI, 15.5%-34.5%] vs 12.7% [10.1%-15.2%]; z = 2.83; P = .005). In 30 cases, there was a combination of a human cause with other causes. Patient-related causes (ie, comorbidity and treatment adherence) occurred significantly less in DAEs vs other AE types (30.0% [95% CI, 20.0%-40.0%] vs 44.9% [41.1%-48.7%]; z = 2.42; P = .02), which was mainly due to a lower comorbidity rate.

In additional analyses, classifications into the subcategories of the Eindhoven Classification Model were compared (Table 3).²¹ Most subcategories of human causes occurred significantly more often in DAEs. In the main category of organizational factors, transfer of knowledge and cultural aspects were significantly more often involved in the occurrence of DAEs than of other AE types.

Diagnostic AEs occurred in 0.4% of all hospital admissions and accounted for 6.4% of all AEs in the Netherlands. The DAEs led to more severe consequences and were considered to be preventable in most cases. In DAEs, more human and organizational causes related to lack of knowledge or problems with transfer of knowledge were identified compared with other AE types.

The incidence of DAEs is comparable to findings in other AE studies^5,18 but lower than has been found in prospective studies.¹¹ The present study used a diverse sample that included all medical specialties, whereas many prospective studies on diagnostic error focused on a high-risk medical specialty, such as internal medicine or the emergency department. Furthermore, we used relatively strict criteria, that is, only cases with patient harm that originated from diagnostic errors, explaining the lower rate of diagnostic error compared with other studies. Because not all errors in diagnostic process lead to an incorrectly established diagnosis or patient harm, the rate of diagnostic error in this study is lower than that in the other studies.¹¹

The reviewers identified more causes in the patient record for DAEs than for other AEs. The common denominator in the causes of DAEs seemed to be the lack of knowledge. The reviewers indicated that several knowledge-related factors co-occurred. First, it appeared that the physicians either did not possess sufficient knowledge or did not apply their knowledge correctly (knowledge-based mistake). Second, physicians did not verify whether the intended treatment was applicable to the patient's current situation (verification error). Third, the physicians did not receive the latest updates about the patient or about new hospital procedures (lack of information transfer).

Multiple causes leading to DAEs occurred in different combinations and did not involve only knowledge-based causes. Combinations with patient-related causes (such as comorbidity and treatment adherence), the failure to monitor a patient's status, and violations also occurred frequently. Violations were a severe group of causes because they involved deliberate deviations from standard protocols or rules. Because it is unlikely that health care professionals deliberately jeopardized the patient's health, the violations probably occurred during circumstances such as high workload. Diagnostic AEs were more likely to occur when a patient was admitted to the hospital in an emergency setting. It is possible that the physicians could not obtain all necessary information in a setting where time is limited and information resources are not always available. A poorly executed examination of the patient in the emergency department might affect the continuation of the diagnostic process or treatment during the hospital admission. It is important to note that in the Netherlands, unlike in other countries, emergency medicine is not a separate medical specialty; instead, according to their main problem, acutely ill patients are seen by physicians from several medical specialties, such as surgery and internal medicine. Thus, the latter specialties are medically responsible and formally regarded as the treating specialty.

Future studies should focus on studying the diagnostic reasoning process prospectively and not only on adverse outcomes in retrospect. Using a prospective study, the causes of DAEs can be determined more precisely. Interventions to reduce DAEs should include additional training for physicians. Prevention strategies should first focus on making physicians aware of the problem and subsequently should use multiple strategies to improve physicians' clinical reasoning.²⁶ The focus should be on training physicians using in-depth and context-specific courses that not only expand physicians' knowledge but also teach them how to apply their knowledge in practice, that is, provide feedback on their performance.^27,28 Further interventions should also focus on the organization of knowledge and information transfer to make sure that health care professionals receive all the information they need. Especially in the emergency department, transfer of information should be more organized. Supervision and well-structured information transfer from preceding health care professionals is advisable, for example, letters from general physicians, information from the ambulance crew, and easy access to patient records of earlier hospital admissions.

A strong aspect of the study is the large number of patient records that were reviewed systematically. The use of a random (stratified) sample of diverse patient admissions and a systematic screening process led to results representative of Dutch hospitals. Also, the AE criteria are well known, which allows comparison with other studies.

Although the results should be interpreted with caution, the analyses of the causes provided interesting information about the causes of DAEs. It was possible to define the differences in causes between DAEs compared with other AE types, thereby providing an indication of problems related to the occurrence of DAEs and possibilities for interventions.

There are several methodological limitations relating to this study. First, not all invited hospitals agreed to participate, which could have caused a selection bias. Second, because the study examines DAEs, retrospective hindsight bias is possible. Also, the physician reviewers depended on the information available in the patient records. Information in patient records might sometimes be limited, and information relevant to revealing AEs and to classifying their causes might be lacking. This can lead to underestimation of the occurrence of DAEs. Third, the method we used involved 2 reviewers reaching a consensus on whether an AE occurred and the preventability of the AE. It is likely that 2 reviewers would find more AEs than would 1 reviewer.²⁹ This could have led to overestimation of the AE rate compared with studies with 1 reviewer but not to an overestimation per se. Fourth, this study shows that DAEs are often detected during a subsequent hospital admission. Not all patients with a DAE will be readmitted to the hospital. Some patients will receive care elsewhere or might die outside the hospital, resulting in DAEs possibly remaining undetected.

In conclusion, DAEs are an important AE type because they are considered to be highly preventable and their consequences are severe. The causes of human and organizational categories suggest a chain of events that lead to DAEs, the main causes being knowledge-based mistakes combined with verification and information transfer problems. Prevention strategies should focus on better training of physicians in a context-specific manner and on organization of knowledge and information transfer.

Correspondence: Laura Zwaan, MSc, Department of Public and Occupational Health, EMGO Institute for Health and Care Research, VU University Medical Center, Van der Boechorststraat 7, 1081 BT Amsterdam, the Netherlands (Laura.zwaan@vumc.nl).

Accepted for Publication: January 14, 2010.

Author Contributions: All authors had full access to all the data in the study and gave their approval of the final version of the manuscript. Study concept and design: de Bruijne, Wagner, van der Wal, and Timmermans. Acquisition of data: Zwaan and Smits. Analysis and interpretation of data: Zwaan, de Bruijne, Thijs, Smits, and Timmermans. Drafting of the manuscript: Zwaan and Timmermans. Critical revision of the manuscript for important intellectual content: de Bruijne, Wagner, Thijs, Smits, van der Wal, and Timmermans. Statistical analysis: de Bruijne. Obtained funding: Wagner, van der Wal, and Timmermans. Study supervision: Thijs, van der Wal, and Timmermans.

Financial Disclosure: None reported.

Funding/Support: The Dutch Patient Safety Research Program has been initiated and supported by the Dutch Society of Medical Specialists with financial support from the Dutch Ministry of Health, Welfare, and Sport. This program is carried out by EMGO Institute/VU University Medical Center and NIVEL.

Additional Contributions: We thank the nurses and physicians who reviewed the patient records, the 21 hospitals and their staff who facilitated access to the patient records, and Marieke Zegers, PhD, for her advice.

Online-Only Material: The eTable is available at www.archinternmed.com.