Types and Origins of Diagnostic Errors in Primary Care Settings

Importance Diagnostic errors are an understudied aspect of ambulatory patient safety.

Objectives To determine the types of diseases missed and the diagnostic processes involved in cases of confirmed diagnostic errors in primary care settings and to determine whether record reviews could shed light on potential contributory factors to inform future interventions.

Design We reviewed medical records of diagnostic errors detected at 2 sites through electronic health record–based triggers. Triggers were based on patterns of patients' unexpected return visits after an initial primary care index visit.

Setting A large urban Veterans Affairs facility and a large integrated private health care system.

Participants Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006, and September 30, 2007.

Main Outcome Measures Through medical record reviews, we collected data on presenting symptoms at the index visit, types of diagnoses missed, process breakdowns, potential contributory factors, and potential for harm from errors.

Results In 190 cases, a total of 68 unique diagnoses were missed. Most missed diagnoses were common conditions in primary care, with pneumonia (6.7%), decompensated congestive heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) being most common. Process breakdowns most frequently involved the patient-practitioner clinical encounter (78.9%) but were also related to referrals (19.5%), patient-related factors (16.3%), follow-up and tracking of diagnostic information (14.7%), and performance and interpretation of diagnostic tests (13.7%). A total of 43.7% of cases involved more than one of these processes. Patient-practitioner encounter breakdowns were primarily related to problems with history-taking (56.3%), examination (47.4%), and/or ordering diagnostic tests for further workup (57.4%). Most errors were associated with potential for moderate to severe harm.

Conclusions and Relevance Diagnostic errors identified in our study involved a large variety of common diseases and had significant potential for harm. Most errors were related to process breakdowns in the patient-practitioner clinical encounter. Preventive interventions should target common contributory factors across diagnoses, especially those that involve data gathering and synthesis in the patient-practitioner encounter.

Primary care practitioners (PCPs) manage a wide range of increasingly complex and severe conditions through one or more relatively brief encounters. Thus, it is not surprising that the primary care setting is vulnerable to medical errors.^1-6 Diagnostic errors (missed, delayed, or wrong diagnoses)⁷ are of increasing concern in this setting.^8-14 However, data about the most frequent misdiagnosed conditions are scarce, and little is known about which diagnostic processes are most vulnerable to breakdown. Most current data about diagnostic errors in primary care are derived from studies of malpractice claims or self-report surveys.^10,15-17These methods introduce significant biases that limit the generalizability of findings to routine clinical practice.A recent report by the American Medical Association^18,19 recommends that efforts be made to “dramatically” strengthen the research base for outpatient safety, especially in the area of outpatient diagnostic errors. Understanding the circumstances in which these errors occur in typical practice is a necessary step toward generating preventive strategies.

In prior studies, we used a set of electronic health record (EHR)–based triggers (automated database queries) to identify primary care visits that were likely to be associated with diagnostic error.²⁰ Our triggers were composed of algorithms to detect unusual patterns of care, namely, unplanned hospitalizations, return visits, or emergency department visits within a short time after an initial primary care encounter. Physicians performed record reviews of triggered visits and nontriggered control visits to identify diagnostic errors. Our primary objectives in the present study were to determine the types of diseases missed and the diagnostic processes involved in cases of confirmed diagnostic errors in primary care settings. This exploration could advance knowledge about conditions that are vulnerable to being missed in primary care and help prioritize error prevention strategies. Our secondary objective was to determine whether record reviews could shed light on potential contributory factors to inform future interventions.

Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006, and September 30, 2007, in 2 large health systems. Both sites had integrated and well-established EHRs and large clinic networks through which they provided longitudinal care. Both sites provided care to ethnically and socioeconomically diverse patients from rural and urban areas.

Site A was a large urban Veterans Affairs facility with approximately 35 full-time PCPs, including physicians, physician assistants, and nurse practitioners, who provided comprehensive care to approximately 50 000 patients. Most PCPs were physicians, some of whom supervised residents. Primary care encounters included both scheduled follow-up visits and “drop-in” unscheduled visits.

Site B was a large integrated private health care system with 34 family medicine PCPs who provided primary and urgent care to almost 50 000 patients in 4 community-based clinics. More than half of the PCPs supervised residents.

Details about diagnostic error detection techniques used in this study have been published previously.²⁰ Briefly, our trigger queries were (1) a primary care index visit followed by an unplanned hospitalization within 14 days and (2) a primary care index visit followed by 1 or more primary care, emergency department, or urgent care visit(s) within 14 days. Trained physicians then reviewed all “triggered” records for evidence of diagnostic error. Reviewers were fellows from medicine subspecialty training programs or chief residents in medicine and were selected based on recommendations from faculty and interviews by our research team. They were instructed to judge diagnostic performance based only on data already available or easily available to the index visit practitioner to either make or pursue the correct diagnosis. Within these constraints, reviewers evaluated several aspects of EHR documentation (notes, tests, referrals, and case evolution over time) to ascertain the presence of a diagnostic error. An error was judged to have occurred if adequate data to suggest the final, correct diagnosis were already present at the index visit or if documented abnormal findings at the index visit should have prompted additional evaluation that would have revealed the correct, ultimate diagnosis. Thus, errors occurred only when missed opportunities to make an earlier diagnosis occurred based on retrospective review.^21-23 In diagnostic error cases, reviewers recorded the disease condition that was missed. A sample of randomly selected control visits (ie, visits that did not meet either trigger criterion) were reviewed for errors using the same procedure. In 212 165 visits at both sites, we found 190 diagnostic errors; 141 of 674 trigger 1 records (20.9%), 36 of 669 trigger 2 records (5.4%), and 13 of 614 controls (2.1%) contained diagnostic errors. An independent second reviewer confirmed each error case. Elsewhere we have described additional details regarding our methods for determining errors.²⁰

For each confirmed case of diagnostic error, one reviewer performed an additional level of review to determine process breakdowns, contributory factors involved, and potential for harm. We designed a data collection instrument after reviewing the previous literature about diagnostic errors, which included studies based on malpractice claims,¹⁰ physician surveys,^24,25 and medical record reviews.^{13,21,22,26-28} We categorized cases using a 5-dimension model of ambulatory diagnostic processes¹³ to indicate the point in the diagnostic process at which errors occurred: patient-practitioner clinical encounter, performance and/or interpretation of diagnostic tests, follow-up and tracking of diagnostic information, subspecialty, and referral-related and patient-specific processes. To assess the reliability of judgments related to process breakdowns, a second reviewer independently evaluated 10% of the error records; interrater reliability was quantified by computing the Cohen κ. Process breakdowns in the patient-practitioner clinical encounter could involve problems with history taking, physical examination, ordering of diagnostic tests for further workup, and review of previous documentation. Additional contributing factors were collected for the remaining 4 dimensions using a comprehensive list of factors based on previous literature.^{10,13,21,22,24,26-28} We used an 8-point scale to collect data about each error's potential for harm (with 1 indicating no harm/no inconvenience and 8 indicating immediate or inevitable death).²⁹ Finally, we collected data about the patient's age, sex, race/ethnicity, and the type of PCP at the index visit (attending physician, trainee, physician assistant, or nurse practitioner).

We generated descriptive statistics to quantify the frequency of clinical conditions associated with diagnostic errors and process breakdowns, contributory factors, and harm. We also compared patient and practitioner variables between primary care visits with and without diagnostic errors. We performed a t test to compare the mean age of patients involved in visits with and without diagnostic errors. We compared all other patient characteristics as proportions, using the Fisher exact test for categorical variables when the assumptions for the χ² test were not met (2-tailed).

Table 1 summarizes the characteristics of patients involved in visits with and without diagnostic errors at each site. At both sites, the mean age of patients involved in visits with diagnostic errors was slightly older than in cases not involving errors (66.5 vs 62.7 years, P = .002, at site A; 53.8 vs 45.6 years, P = .003, at site B). Diagnostic errors did not seem to be significantly associated with the type of practitioner at either site.

We found 68 unique diagnoses that were missed in both the trigger and control groups. When cases from both sites were combined, pneumonia (6.7%), decompensated congestive heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) were the most commonly missed diagnoses in primary care. However, the most frequently missed or delayed diagnoses differed by site (Table 2). In some cases, more than one diagnosis was missed. In most cases (85.8%), a different practitioner, either from the same specialty (43.1%) or a different specialty (42.7%), saw the patient on the return visit. In more than half of the cases (51.6%), errors were discovered because of the failure of the original symptom or sign to resolve. Other errors were detected on evolution of original symptoms and signs (34.8%) or the development of new symptoms or signs (22.6%). Only approximately one-fifth of the errors were discovered as part of planned follow-up, such as when practitioners asked the patient to return within a certain period (usually few days) for reevaluation. In 96% of triggered error cases, there was a clear relationship between the patient's admission or second outpatient visit and the presentation on index visit.

Table 3 lists the chief presenting symptoms, as documented by the index practitioners, that were present in 2 or more cases of diagnostic error. Of these, cough (sometimes associated with additional presenting symptoms) was the most common. Notably, in 22 cases patients did not have any specific chief presenting symptom; this occurred in instances such as when established patients were following up on chronic medical issues or when new patients were visiting to establish care within the system. The chief presenting symptom was directly related to the missed diagnosis in approximately two-thirds (67.4%) of cases (data not shown in table). However, documentation of adequate exploration and investigation of the chief presenting symptom was lacking in 93 cases (48.9%).

Breakdowns were found to occur in all 5 dimensions of the diagnostic process, and in 43.7% of cases more than one dimension was involved. The interrater reliability of process breakdown ratings was 0.56 (95% CI, 0.38-0.74). Most commonly, breakdowns occurred during the patient-practitioner clinical encounter (78.9%), and this finding was consistent across both sites (76.7% vs 83.6% for sites A and B, respectively). Breakdowns involving the patient-practitioner clinical encounter were most often judged to be due to data-gathering and synthesis problems (ie, cognitive errors) related to the medical history (56.3%), physical examination (47.4%), ordering of diagnostic tests for further workup (57.4%), and failure to review previous documentation (15.3%) (Table 4). Two additional documentation-related problems were notable. First, no differential diagnosis was documented at the index visit in 81.1% of cases. Second, practitioners copied and pasted previous progress notes into the index visit note in 7.4% of cases; of these cases, copying and pasting mistakes were determined to contribute to more than one-third (35.7%) of errors.

Outside the patient-practitioner encounter, process breakdowns also occurred in the areas of referrals, patient actions or inaction, follow-up and tracking of diagnostic information, and performance and interpretation of diagnostic tests (Table 4). The most common referral-related breakdowns were problems initiating a needed referral; in 1 of 10 error cases, an appropriate expert was not contacted when indicated. Contributory factors for these processes are summarized in Table 4. No one factor was attributed to most errors in these categories, and no single factor in these categories contributed to 10% or more of all error cases.

The potential severity of injury associated with the delayed or missed diagnosis was classified as moderate to severe in 86.8% of cases (ratings of 4-8 on the 8-point scale), with a mode of 5 (considerable harm) (Table 5). When we further broke down cases according to which diagnostic process was implicated in the error, we found a modal severity rating of 5 across all 5 processes.

We analyzed 190 primary care diagnostic errors, most of which were detected through electronic triggers. These errors involved a large variety of conditions that are seen commonly in primary care. Most diagnostic errors had potential for moderate to severe harm. Presenting symptoms for these conditions were highly variable and sometimes did not bear any obvious direct relationship to the condition that was missed. Most errors involved breakdowns of processes related to the patient-practitioner clinical encounter. Lower, although still meaningful, proportions of breakdowns occurred in the domains of referrals, patient factors, test ordering and interpretation, and follow-up and tracking of diagnostic information.

Our study is among the largest to address diagnostic error in routine outpatient practice and among the first to empirically evaluate the types of diagnostic errors that occur in primary care. Misdiagnosis of cancer has been considered among the most common diagnostic errors in primary care, mostly because of overrepresentation in studies of malpractice claims.^10,11,27 However, a previous systematic review of diagnostic error in primary care found a wide range of conditions that were commonly misdiagnosed, including not only cancer but also myocardial infarction, meningitis, dementia, iron deficiency anemia, asthma, tremor in elderly patients, and human immunodeficiency virus.³⁰ Most of the errors identified in our study involved missed diagnosis of a large variety of common conditions as opposed to either a few selected conditions or rare or unusual diseases. Pneumonia and decompensated congestive heart failure were most commonly missed, although they accounted for less than 13% of all errors. Furthermore, there were marked differences in the most common missed diagnoses across the 2 sites, largely because their local contexts and patient and practitioner populations were markedly different. For instance, practitioners at site A were predominantly internists who cared for older veterans (who generally have more comorbidities), whereas at site B, family practitioners cared for an overall younger population. However, at both sites, the most common process breakdowns arose within the patient-practitioner encounter. Because diagnostic errors in primary care involve a large number of heterogeneous conditions, future error reduction strategies should account for their common contributory factors and not just attempt to augment knowledge or clinical skills related to specific diseases because such interventions may not generalize across diseases or care settings.

We were also able to report patients' chief presenting symptoms associated with the commonly missed diagnoses, a topic that has been essentially unexplored.¹⁶ Knowledge about commonly implicated or “high-risk” chief presenting symptoms could potentially lead to targeted interventions to decrease the likelihood of error, although many found in our cohort are fairly common in outpatient practice. Given the myriad of symptoms that PCPs encounter in their daily practices, focusing on specific presentations is unlikely to form the sole basis for preventive strategies. Moreover, in approximately one-third of cases patients presented with symptoms that appeared to be unrelated to the missed diagnosis, which could easily divert the practitioner's attention during the short span of the primary care visit. Another notable finding was the absence of documentation of differential diagnosis, a fundamental step in the diagnostic reasoning process. How patients present their chief symptoms to the PCP and how this and other factors influence a PCP's clinical reasoning within the context of a busy clinic are areas ripe for further investigation.^17,23

Our findings highlight the need to focus on basic clinical skills and related cognitive processes (eg, data gathering within the medical history and physical examination and synthesis of data) in the age of increasing reliance on technology and team-based care to improve the health care system.^31-33 Most process breakdowns were related to the clinical encounter, wherein practitioners are almost always pressed for time to make decisions.¹⁵ With the current emphasis on patient-centered medical homes that facilitate team-based care, patients might be able to access or interact with their practitioners more effectively. However, these new models of care might not produce the level of cognitive support needed for gathering and/or interpreting a patient's key signs and symptoms effectively and safely. Furthermore, current forms of technology, including EHRs, are inadequately positioned to meet the needs of complex decision making.³⁴ Newer models of understanding how best to leverage both technological and nontechnological strategies to improve practitioner and team situational awareness are needed. In addition, more research is needed to determine why practitioners may not adequately search for data or synthesize findings and how to best improve clinical skills and cognitive processes in the complex primary care environment. Although the current literature highlights isolated cognitive difficulties among practitioners (eg, biases) and various interventions have been suggested to improve diagnostic decision making (eg, the use of checklists³⁵ or second opinions), few cognitive obstacles have been sufficiently examined in the complex “real-world”primary care environment, and few interventions have been satisfactorily tested.³⁶ Using the lens of missed opportunities in care rather than errors, institutions could create a new focus on discovering, learning from, and reducing diagnostic errors.³⁷ Our methods can be one way to “proactively” discover such missed opportunities in real practice.²⁰ Patient empowerment and engagement in the diagnostic process could add greatly to these strategies.

Our study has several limitations. Our methods may not apply to primary care practices that are not part of integrated health care systems. Moreover, ours was a retrospective study, so hindsight bias remains a concern. Because of our study design, we did not have the additional benefit of practitioner debriefing, which if performed soon after error discovery could provide additional useful data. The triggers we used are more likely to select for misdiagnosis of acute conditions and exacerbations of chronic conditions. These are important subsets of diagnostic errors³⁸ but are not inclusive of all diagnostic errors; for instance, the errors discovered through our methods were likely to have underrepresented diagnoses of conditions, such as cancer, that are less likely to emerge during an urgent presentation within a short time frame. We were able to achieve only moderate interrater reliability in determinations of process breakdowns, although it was similar to that reported in the landmark study of diagnostic error process breakdowns by Schiff et al.¹⁵ Finally, errors in our study were likely rated as more harmful because most of them were detected in the context of an unexpected hospitalization or return visit, and this may not generalize to the universe of diagnostic errors in primary care. Nevertheless, the field of diagnostic error is fairly nascent, and it is important to focus on any types of errors that cause harm or create the need for further care even though they might not be representative of all errors.

In conclusion, diagnostic errors in primary care include a heterogeneous group of common conditions, and most have potential to lead to moderate to severe harm. Most errors were related to patient-practitioner clinical encounter–related processes, such as taking medical histories, performing physical examinations, and ordering tests. Given the range of conditions associated with diagnostic errors in this setting, disease-specific efforts to reduce these types of diagnostic errors are unlikely to be sufficient. Thus, preventive interventions must focus on common contributory factors, particularly those that influence the effectiveness of data gathering and synthesis in the patient-practitioner encounter.

Correspondence: Hardeep Singh, MD, MPH, Houston VA Health Services Research and Development Center of Excellence, Michael E. DeBakey Veterans Affairs Medical Center (152), 2002 Holcombe Blvd, Houston, TX 77030 (hardeeps@bcm.edu).

Accepted for Publication: October 21, 2012.

Published Online: February 25, 2013. doi:10.1001/jamainternmed.2013.2777

Author Contributions:Study concept and design: Singh and Thomas. Acquisition of data: Singh, Forjuoh, Reis, and Thomas. Analysis and interpretation of data: Singh, Giardina, Meyer, Forjuoh, and Thomas. Drafting of the manuscript: Singh. Critical revision of the manuscript for important intellectual content: Singh, Giardina, Meyer, Forjuoh, Reis, and Thomas. Statistical analysis: Meyer. Obtained funding: Singh, Forjuoh, and Thomas. Administrative, technical, and material support: Giardina, Meyer, Reis, and Thomas. Study supervision: Singh, Forjuoh, Reis, and Thomas.

Conflict of Interest Disclosures: None reported.

Funding/Support: The study was supported by National Institutes of Health K23 Career Development Award K23CA125585 to Dr Singh, by Agency for Health Care Research and Quality Health Services Research Demonstration and Dissemination grant R18HS17244-02 to Dr Thomas, and in part by Houston VA HSR&D Center of Excellence grant HFP90-020 and the VA Office of Academic Affiliations Fellowship Program to Dr Meyer.

Role of the Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Disclaimer: The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or any other funding agency.