Singh H, Thomas EJ, Petersen LA, Studdert DM. Medical Errors Involving TraineesA Study of Closed Malpractice Claims From 5 Insurers. Arch Intern Med. 2007;167(19):2030-2036. doi:10.1001/archinte.167.19.2030
Copyright 2007 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2007
Despite wide recognition that the delivery of medical care by trainees involves special risks, information about the types and causes of medical errors involving trainees is limited. To describe the characteristics of and factors contributing to trainee errors, we analyzed malpractice claims in which trainees were judged to have played an important role in harmful errors.
The claims were closed between 1984 and 2004, and the errors occurred between 1979 and 2001. Specialist physicians reviewed random samples of closed malpractice claim files at 5 liability insurers from 2002 to 2004 and determined whether injuries had occurred, and if so, whether they were due to error. We described the clinical circumstances and contributing factors associated with harmful errors involving trainees (“cases”). We also compared the characteristics of cases with their nontrainee counterparts and probed trainee errors attributed to teamwork problems and lack of technical competence or knowledge.
Among 240 cases, errors in judgment (173 of 240 [72%]), teamwork breakdowns (167 of 240 [70%]), and lack of technical competence (139 of 240 [58%]) were the most prevalent contributing factors. Lack of supervision and handoff problems were most prevalent types of teamwork problems, and both were disproportionately more common among errors that involved trainees than those that did not (respectively, 54% vs 7% [P < .001] and 20% vs 12% [P = .009]). The most common task during which failures of technical competence occurred were diagnostic decision making and monitoring of the patient or situation. Trainee errors appeared more complex than nontrainee errors (mean of 3.8 contributing factors vs 2.5 [P < .001]).
In addition to problems with handoffs, house staff are particularly vulnerable to medical errors owing to teamwork failures, especially lack of supervision. Graduate medical education reform should focus on strengthening these aspects of training.
Graduate medical education poses unique challenges for the delivery of safe patient care,1- 3 and medical trainees face special risks of involvement in medical errors.4- 10 This is predictable: trainees are inexperienced, often fatigued, and occasionally unsupervised, and the academic medical centers in which they work are typically large and complex facilities charged with treating the sickest patients. Despite recognition of these risk factors, information about the types and causes of trainee errors is limited.8 This knowledge gap inhibits the design of effective prevention strategies, such as targeted educational programs and system changes to reduce trainee errors and advance patient safety.11 An improved understanding of the causes of trainee errors could help guide the implementation of the Accreditation Council for Graduate Medical Education's (ACGME) core competencies into residency curricula in directions that advance patient safety.2
The Malpractice Insurers Medical Error Prevention Study (MIMEPS), a review of 1452 malpractice claims from 5 insurers, provided us with a valuable opportunity to investigate trainee involvement in medical errors.12 We conducted a subanalysis of the MIMEPS claims in which study reviewers detected harmful errors and judged that 1 or more interns, residents, or fellows played an important causal role. Our main goals were to describe the characteristics of these “trainee errors” and identify their contributing factors. We anticipated that problems of technical competence, handoff,13 and other aspects of teamwork14 would figure prominently, so we explored these areas in depth.
Five malpractice insurance companies in 4 regions (northeastern, mid-Atlantic, southwestern, and western United States) participated in the study. The companies covered approximately 33 000 physicians, 61 acute care hospitals (35 academic and 26 nonacademic), and 428 outpatient facilities. The study was approved by ethics review boards at the investigators' institutions and by the review sites.
Data were extracted from random samples of closed claim files at each insurer in on-site reviews conducted between 2002 and 2004. The claim file is the repository of information accumulated by the insurer during the life of a claim. While the claim is open, it includes medical records pertaining to the episode of care at issue. For all sampled claims, we reacquired the relevant medical records from insured institutions.
We defined a claim as a written demand for compensation for medical injury.15,16 We focused on 4 clinical categories—(1) obstetric, (2) surgical, (3) missed and delayed diagnoses, and (4) medication—and applied a uniform definition of each across sites. These categories cover approximately 80% of all medical malpractice claims filed in the United States.17,18 Insurers contributed to the study sample in proportion to their annual claims volume. The number of claims by site varied from 84 to 662 (median, 294).
Reviewers were board-certified attending physicians, fellows, or final-year residents in surgery (surgical claims), obstetrics (obstetric claims), and internal medicine (diagnosis and medication claims). Physician investigators from the relevant specialties trained the reviewers in the content of claims files, use of the study instruments, and confidentiality procedures in 1-day sessions at each site. Reviews took a mean of 1.6 hours per file and were conducted by 1 reviewer.
A sequence of 4 instruments guided the review. The review process has been described in detail in previous publications.12,19 In summary, reviewers made an initial judgment about whether an adverse outcome had occurred, and if it did, they scored the outcome on a severity scale ranging from emotional injury to death.20 For claims with identifiable adverse outcomes, reviewers proceeded to consider the potential contributory role of 17 possible contributory factors in causing the adverse outcome. Next, they judged whether the adverse outcome was due to medical error, defined as “the failure of a planned action to be completed as intended (i.e., error of execution) or the use of a wrong plan to achieve an aim (ie, error of planning).”21 Reviewers' confidence in the error judgments was recorded on a 6-point scale ranging from 1 (little or no evidence that adverse outcome resulted from error/errors) to 6 (virtually certain evidence that adverse outcome resulted from error/errors).22,23 Claims that scored 4 (more likely than not that an adverse outcome resulted from error/errors; more than 50-50 but a close call) or higher were classified as involving an error. Finally, for the subset of claims judged to involve an adverse outcome due to error, reviewers gathered details of the clinical circumstances, including the specialty of involved clinicians and their contributory role as rated on a 5-point scale (1, somewhat important, to 5, highly important).
To test the reliability of the claims file review, 10% of the files were rereviewed by a second physician from the relevant specialty who was unaware of the first review. On the basis of 148 pairs of reviews, κ scores were 0.78 (95% confidence interval, 0.65-0.90) for the determination of injury and 0.63 (95% confidence interval, 0.12-0.74) for the judgment that error occurred. More detailed results of the reliability testing are reported elsewhere.12,19,24
The unit of analysis in MIMEPS was the episode of care in claims judged to involve errors that led to an adverse outcome. For ease of exposition, we henceforth refer to such episodes as “cases.” For this study, we drew a subsample of cases from the full sample of cases identified in MIMEPS (n = 889). The subsample consisted of cases in which the reviewer had rated the contributory role of a medical student, intern, resident, or fellow at 4 or 5 or rated it 3 and no other involved clinician had a higher rating. Reliability testing for the determination that 1 or more trainees were involved, which was based on 47 pairs of original reviews, showed good agreement (89% agreement; κ = 0.64 [95% confidence interval, 0.34-0.94]).
Analyses were conducted using the Stata/SE 8.0 (StataCorp, College Station, Texas) software package. We generated descriptive statistics to examine the characteristics of the trainees, patients, and adverse outcomes in the study sample. We compared the frequency with which contributing factors were associated with trainee errors vs nontrainee errors using Fisher exact tests.
We conducted an in-depth analysis of cases involving problems of teamwork and technical competence, respectively. With respect to teamwork problems, we investigated the personnel relationships in which they occurred. Cases with teamwork problems were defined as those in which the original reviewer had judged that 1 or more of the following contributory factors played a role in the error: communication breakdowns, supervision problems, handoff problems, failures to establish clear lines of responsibility, and conflict among clinical staff. Defining teamwork according to such behaviors is consistent with previous conceptualizations of teamwork in medicine25 and the World Health Organization's definition of the term.26
For problems of technical competence or knowledge, we identified the task being performed by the physician when the error occurred. The task options were drawn from a list of categories of general practitioner tasks provided by the Occupational Information Network,27 which are general enough to adequately capture the work of clinicians in a range of specialties. We made several modifications to the Occupational Information Network list. Specifically, we split 1 of the categories (“specialized medical care to treat or prevent illness, disease, or injury”) into nonprocedural and procedural work, and then further divided the latter into procedures that were related to obstetrical deliveries and those that were not.
Of 889 cases (claims with both error and injury) identified in the parent study, 240 (27%) involved trainees whose role in the error was judged to be at least moderately important. The claims were closed between 1984 and 2004 and the errors occurred between 1979 and 2001; 72% of the claims were closed in 1990 or later.
Table 1 gives the characteristics of trainees, patients, and adverse outcomes in the study sample. The mean age of injured patients was 30 years, and 51% were female. Residents were involved in 87% of the cases. The adverse outcomes were generally severe: one-third resulted in significant physical injury, one-fifth in major physical injury, and an additional third in death. Nearly a third (30%) of the cases occurred in the outpatient setting.
One-third of the cases involved trainees in obstetrics and gynecology (Table 2). The next most prevalent specialties were general surgery, adult primary care, orthopedic surgery, and pediatrics. Collectively, 78% of the cases involved trainees from 1 or more of these 5 specialties.
Characteristics of the parent study sample have been reported elsewhere.12 The 240 trainee errors differed from their 649 nontrainee counterparts on several measures. A larger proportion of trainee errors occurred during inpatient care (70% vs 52% [P < .001]). Patients injured in trainee errors were younger on average (30 years vs 38 years [P < .001]) and more likely to sustain fatal injury (33% vs 24% [P = .008]). In addition, a larger proportion of trainee errors involved obstetrics events (30% vs 21% [P = .003]) and fewer involved missed or delayed diagnosis (21% vs 32% [P = .002]).
Cognitive factors contributed to nearly all of the trainee errors, with 72% of cases involving judgment errors and 57% involving failures of vigilance or memory (Table 3). Fifty-eight percent of the cases involved lack of technical competence or knowledge and 70% involved teamwork-related factors. The most prevalent types of teamwork factors were lack of supervision and handoffs.
Eight contributing factors were significantly more prevalent among trainee cases than nontrainee cases (Table 3). Lack of technical competence (58% vs 42% [P < .001]), lack of supervision (54% vs 7% [P < .001]), handoff problems (19% vs 13% [P = .02]), and excessive workload (19% vs 5% [P < .001]) were particularly noteworthy contributing factors in the comparison, both because the differences were large and because these contributing factors were prevalent within the trainee group. At the aggregate level, teamwork factors contributed to 70% of trainee errors, more than twice the frequency with which they contributed to errors (P < .001).
The trainee errors involved a mean of 3.9 contributing factors, compared with a mean of 2.7 factors among nontrainee errors. The difference was significant (P < .001) and suggested greater causal complexity associated with the errors in the trainee group.
Because our sample covered a 22-year period, we also tested for secular trends in the frequency of the contributing factors among trainee errors. When events that occurred in 1993 (the median occurrence year) or earlier were compared with those that occurred in later years, one significant difference emerged: a larger proportion of events from the earlier period involved supervision problems (80 of 128 [61%] vs 51 of 112 [47%]; P = .01).
In 82% (106 of 129) of the cases involving lack of supervision, attending physicians' failure to supervise residents was at issue; in 12% (16 of 129) of the cases, supervision failures by both senior residents and attending physicians were apparent (Table 4). Handoff problems occurred most commonly in handoffs between trainees (19 of 56 [34%]) and between trainees and attending physicians (18 of 56 [32%]). Four cases were related to poor communication over the telephone between the trainee and the attending physician. In general, the chains of communication within which these breakdowns occurred were complex. Approximately a fifth (12 of 56) of handoff problems involved more than 2 entities, and a quarter (14 of 56) of the chains extended to interactions with nurses, pharmacy and laboratory personnel, and entities external to the trainee's home institution.
Diagnostic decision making was the primary task at hand in nearly half (67 of 139 [48%]) of cases in which technical competence or knowledge problems occurred (Table 5). For example, in one case, a surgical resident missed the diagnosis of a bile leak following abdominal surgery; in another case, an obstetric resident misdiagnosed a breech presentation.
Forty-three percent of cases (60 of 139) had secondary tasks associated with the technical competence problem and 17% (24 of 139) had tertiary tasks (mean of 1.6 tasks per problem). For example, one resident's failure to diagnose a high-risk pregnancy was accompanied by inadequate fetal heart rate monitoring; in this case, diagnostic decision making was classified as the primary task and monitoring, as a secondary task. Considering all tasks (n = 223) associated with technical competence breakdowns and not just the primary one, diagnosis remained the most common task (36%), followed by monitoring (17%).
This study of malpractice claims data identified several distinctive features of errors involving trainees. Trainee errors were characterized by frequent teamwork and communication breakdowns, especially failures of supervision and handoffs. They were also disproportionately likely to involve technical failures and problems of excessive workload. Diagnostic decision making and monitoring activities were the most common tasks at hand when trainee errors occurred.
The sample of errors we examined is noteworthy for its scope. Malpractice claims data from multiple insurers are a powerful repository of information on care breakdowns from a diverse range of institutions and physicians. Other studies of trainee error have focused on a single discipline or setting and relied on surveys and interviews.4,8,9,28 The chief causes of error identified in this previous work include lack of supervision, handoffs, inexperience and lack of competence in a surgical task, and excessive work hours leading to sleep deprivation and/or fatigue.4,8,13,29- 31 Our study corroborates and extends these findings. For example, in addition to finding that handoffs between house staff are an important risk factor for preventable adverse events,13 we found that similar information transfer problems occur between trainees and other agents in the delivery system, including attending physicians, nurses, pharmacists, laboratories, and institutions external to the home institution of the trainee.
Communication failures among residents may stem from several tensions in teamwork, such as medical hierarchies, role ambiguity, and interpersonal dynamics.28 Our study confirms the relationship of poor teamwork to preventable errors32 and quality of care.33 Despite ACGME systems-based practice competencies and attention to implementation of a team-based system of care in graduate medical education,34 the development of teamwork and other specific communication skills is probably underemphasized in residency. For instance, we found telephone communication to be problematic in several cases, but this skill has not been evaluated or specifically taught.35 One reason why uptake of teamwork training has been slow may be the current lack of effectiveness of formal teaching programs such as Medical Team Training.36,37 Information transfer problems are likely to grow. Implementation of the 80-hour-per-week work limits has increased both the volume of information transfers and the pressures on clinicians, making error prevention strategies in this area38 more critical than ever. (The errors we studied occurred prior to the introduction of ACGME duty-hour regulation in 2003.)
Better supervision of residents has been flagged as one of the more remediable contributors to substandard care.39 Our data underscore the importance of appropriate supervision. The program requirements of the ACGME place supervision responsibilities squarely on the shoulders of the attending physician of record.40 Although existing guidelines state that residents should be supervised, best practices in this area have received little theoretical or experiential evaluation.41,42 Moreover, explicit statements about what constitutes adequate supervision are lacking within some specialty areas.43 Nonprocedural work is especially impoverished in this regard, which is problematic because we found supervision breakdowns were no less common there. In addition, 1 in 8 of our cases linked errors by junior residents to poor supervision by senior residents, yet no clear guidelines for resident-to-resident supervision are broadly available. The design of curricula to help residents become better teachers and leaders44 should incorporate specific strategies geared to improve supervision skills of residents. The 80-hour-week requirement may create additional challenges to adequate supervision, making the case to advance the field of trainee supervision even more compelling.42
Recognition of the types of tasks most commonly associated with failures of technical competence may also help to shape ACGME competencies in ways that improve patient safety. For example, errors during diagnostic work were prevalent, and educational interventions to reduce these errors could focus on competencies such as medical knowledge and patient care.45 Good diagnostic decision making depends on a mix of system factors,46 including communication of information between the treating physician, laboratory personnel, and radiologists47 and other consultants. Thus, the much needed improvements in this area48 could be viewed within the spectrum of 4 different competencies including the ones that relate to systems-based practice and communication skills.
Technical competence problems arose relatively frequently during monitoring activities, a finding that reinforces existing evidence that this type of failure is a troubling source of medical error.49,50 Although team training emphasizes skills such as situational awareness to improve monitoring,51 there is a need for specific instructional strategies to improve technical skills. Both simulation52 and use of information technology53 hold promise, as do interventions to bolster and support cognitive skills.54
The methodological approach we used to identify trainee errors has a number of advantages over previously used approaches, such as surveys. In particular, claims data provide a valuable triage point with rich information on a large number of errors that caused serious harm. However, it also has limitations. First, litigated claims are the “tip of the iceberg” of all errors.55 Whether that tip is representative of what lies beneath depends on the characteristic in question. Severe injuries are certainly overrepresented, factors involving communication breakdowns may be also, as might procedural breakdowns that are easily observed by patients. Despite these biases, we know of no reason why the causal patterns associated with errors that lead to litigation would differ from the causal patterns associated with their nonlitigated counterparts. Nonetheless, generalizability beyond the serious injuries observed in malpractice claims is uncertain.
Second, certain contributing factors may not have been detectable through claims file review even though they played a role; fatigue and workload are particularly likely to have gone undocumented, unless they formed part of the plaintiff's allegation. Consequently, the prevalence estimates for such factors represent lower bounds, and the multifactorial causality we observed probably understates the true complexity. In addition, our review could not prioritize the relative contributions of contributing factors to the adverse events nor disentangle causal relationships among factors. Third, because we examined events that predated the 2003 ACGME duty-hour regulation, we cannot evaluate its effect on trainee errors.
Finally, the reviewers' judgments about the appropriateness of care are likely to have been influenced by hindsight bias. One possible version of this bias is the knowledge of the litigation outcome, which may have encouraged findings of errors in paid claims and vice versa. Another version relates to the presence of adverse outcomes, especially severe ones, which may have prompted inferences that care was inappropriate.
Aside from some attention to handoff errors among house staff, trainee errors have gone largely unstudied. The causal characteristics we detected in malpractice claims data suggest special vulnerabilities around teamwork, multiple levels of supervision, and diagnostic decision making. Our findings should help leaders of residency programs and the ACGME to orient training interventions toward these problem areas and also stimulate further research into why and how trainee errors occur.
Correspondence: David M. Studdert, LLB, ScD, MPH, School of Population Health, University of Melbourne, 207 Bouverie St, Carlton, Victoria 3052, Australia (email@example.com).
Accepted for Publication: March 29, 2007.
Author Contributions: Drs Studdert and Singh had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Singh, Thomas, Petersen, and Studdert. Acquisition of data: Singh, Thomas, and Studdert. Analysis and interpretation of data: Singh, Thomas, Petersen, and Studdert. Drafting of the manuscript: Singh and Studdert. Critical revision of the manuscript for important intellectual content: Singh, Thomas, and Petersen. Obtained funding: Thomas, Petersen, and Studdert. Administrative, technical, and material support: Singh, Petersen, and Studdert. Study supervision: Thomas, Petersen, and Studdert.
Financial Disclosure: None reported.
Funding/Support: This study was supported by grants from the Agency for Healthcare Research and Quality (grant HS011886-03) and the Harvard Risk Management Foundation. Dr Studdert was supported by the Agency for Healthcare Research and Quality (grant KO2HS11285). Dr Singh is a recipient of grant K23CA125585 from the National Cancer Institute and was also supported in part by resources and facilities at the 2 centers with which he is affiliated. Dr Thomas is supported by grant 1PO1HS1154401 from the Agency for Healthcare Research and Quality. Dr Petersen was a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar (grant 045444) at the time this work was completed and is a recipient of the American Heart Association Established Investigator Award (grant 0540043N).
Role of the Sponsors: The sponsors of the study had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript or the decision to submit the manuscript for publication.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily represent the views of the National Cancer Institute, the National Institutes of Health, or the Department of Veterans Affairs.
Previous Presentation: A part of the manuscript results were presented as an abstract at the annual meeting of the Society of General Internal Medicine; April 27, 2007; Toronto, Ontario, Canada.