[Skip to Content]
[Skip to Content Landing]
Figure.  Flow Diagram of Study Population Selection
Flow Diagram of Study Population Selection

ED indicates emergency department.

Table 1.  Demographic and Clinical Characteristics of Patients
Demographic and Clinical Characteristics of Patients
Table 2.  Abdominal Pain and Its Combinations With Other Symptoms of Potentially Missed Appendicitisa
Abdominal Pain and Its Combinations With Other Symptoms of Potentially Missed Appendicitisa
Table 3.  Multivariable Modeling of Potentially Missed Appendicitis Among Adults, Stratified by Presentation of Single Symptoma
Multivariable Modeling of Potentially Missed Appendicitis Among Adults, Stratified by Presentation of Single Symptoma
Table 4.  Multivariable Modeling of Potentially Missed Appendicitis Among Children, Stratified by Presentation of Single Symptoma
Multivariable Modeling of Potentially Missed Appendicitis Among Children, Stratified by Presentation of Single Symptoma
1.
Galai  T, Beloosesky  OZ, Scolnik  D, Rimon  A, Glatstein  M.  Misdiagnosis of acute appendicitis in children attending the emergency department: the experience of a large, tertiary care pediatric hospital.  Eur J Pediatr Surg. 2017;27(2):138-141. doi:10.1055/s-0035-1570757PubMedGoogle ScholarCrossref
2.
Naiditch  JA, Lautz  TB, Daley  S, Pierce  MC, Reynolds  M.  The implications of missed opportunities to diagnose appendicitis in children.  Acad Emerg Med. 2013;20(6):592-596. doi:10.1111/acem.12144PubMedGoogle ScholarCrossref
3.
Chang  YJ, Chao  HC, Kong  MS, Hsia  SH, Yan  DC.  Misdiagnosed acute appendicitis in children in the emergency department.  Chang Gung Med J. 2010;33(5):551-557.PubMedGoogle Scholar
4.
Graff  L, Russell  J, Seashore  J,  et al.  False-negative and false-positive errors in abdominal pain evaluation: failure to diagnose acute appendicitis and unnecessary surgery.  Acad Emerg Med. 2000;7(11):1244-1255. doi:10.1111/j.1553-2712.2000.tb00470.xPubMedGoogle ScholarCrossref
5.
Leung  YK, Chan  CP, Graham  CA, Rainer  TH.  Acute appendicitis in adults: diagnostic accuracy of emergency doctors in a university hospital in Hong Kong.  Emerg Med Australas. 2017;29(1):48-55. doi:10.1111/1742-6723.12738PubMedGoogle ScholarCrossref
6.
Brown  TW, McCarthy  ML, Kelen  GD, Levy  F.  An epidemiologic study of closed emergency department malpractice claims in a national database of physician malpractice insurers.  Acad Emerg Med. 2010;17(5):553-560. doi:10.1111/j.1553-2712.2010.00729.xPubMedGoogle ScholarCrossref
7.
Selbst  SM, Friedman  MJ, Singh  SB.  Epidemiology and etiology of malpractice lawsuits involving children in US emergency departments and urgent care centers.  Pediatr Emerg Care. 2005;21(3):165-169. PubMedGoogle Scholar
8.
Medford-Davis  LN, Singh  H, Mahajan  P.  Diagnostic decision-making in the emergency department.  Pediatr Clin North Am. 2018;65(6):1097-1105. doi:10.1016/j.pcl.2018.07.003PubMedGoogle ScholarCrossref
9.
Mahajan  P, Mollen  C, Alpern  ER,  et al.  An operational framework to study diagnostic errors in emergency departments: findings from a consensus panel  [published online November 25, 2019].  J Patient Saf. doi:10.1097/PTS.0000000000000624PubMedGoogle Scholar
10.
National Academies of Sciences, Engineering, and Medicine.  Improving Diagnosis in Health Care. Washington, DC: National Academies Press; 2015.
11.
Singh  H, Meyer  AN, Thomas  EJ.  The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.  BMJ Qual Saf. 2014;23(9):727-731. doi:10.1136/bmjqs-2013-002627PubMedGoogle ScholarCrossref
12.
Graber  ML.  The incidence of diagnostic error in medicine.  BMJ Qual Saf. 2013;22(suppl 2):ii21-ii27. doi:10.1136/bmjqs-2012-001615PubMedGoogle ScholarCrossref
13.
Liberman  AL, Newman-Toker  DE.  Symptom-disease pair analysis of diagnostic error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data.  BMJ Qual Saf. 2018;27(7):557-566. doi:10.1136/bmjqs-2017-007032PubMedGoogle ScholarCrossref
14.
Moy  E, Barrett  M, Coffey  R, Hines  AL, Newman-Toker  DE.  Missed diagnoses of acute myocardial infarction in the emergency department: variation by patient and facility characteristics.  Diagnosis (Berl). 2015;2(1):29-40. doi:10.1515/dx-2014-0053PubMedGoogle ScholarCrossref
15.
Newman-Toker  DE, Moy  E, Valente  E, Coffey  R, Hines  AL.  Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample.  Diagnosis (Berl). 2014;1(2):155-166. doi:10.1515/dx-2013-0038PubMedGoogle ScholarCrossref
16.
Carberry  AR, Hanson  K, Flannery  A,  et al.  Diagnostic error in pediatric cancer.  Clin Pediatr (Phila). 2018;57(1):11-18. doi:10.1177/0009922816687325PubMedGoogle ScholarCrossref
17.
Freedman  SB, Rodean  J, Hall  M,  et al.  Delayed diagnoses in children with constipation: multicenter retrospective cohort study.  J Pediatr. 2017;186:87-94. doi:10.1016/j.jpeds.2017.03.061PubMedGoogle ScholarCrossref
18.
Kraemer  M, Franke  C, Ohmann  C, Yang  Q; Acute Abdominal Pain Study Group.  Acute appendicitis in late adulthood: incidence, presentation, and outcome: results of a prospective multicenter acute abdominal pain study and a review of the literature.  Langenbecks Arch Surg. 2000;385(7):470-481. doi:10.1007/s004230000165PubMedGoogle ScholarCrossref
19.
Benabbas  R, Hanna  M, Shah  J, Sinert  R.  Diagnostic accuracy of history, physical examination, laboratory tests, and point-of-care ultrasound for pediatric acute appendicitis in the emergency department: a systematic review and meta-analysis.  Acad Emerg Med. 2017;24(5):523-551. doi:10.1111/acem.13181PubMedGoogle ScholarCrossref
20.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008PubMedGoogle ScholarCrossref
21.
Alore  EA, Ward  JL, Todd  SR,  et al.  Population-level outcomes of early versus delayed appendectomy for acute appendicitis using the American College of Surgeons National Surgical Quality Improvement Program.  J Surg Res. 2018;229:234-242. doi:10.1016/j.jss.2018.04.011PubMedGoogle ScholarCrossref
22.
Seudeal  K, Abidi  H, Shebrain  S.  Early versus delayed appendectomy: a comparison of outcomes.  Am J Surg. 2018;215(3):483-486. doi:10.1016/j.amjsurg.2017.10.057PubMedGoogle ScholarCrossref
23.
Horn  CB, Tian  D, Bochicchio  GV, Turnbull  IR.  Incidence, demographics, and outcomes of nonoperative management of appendicitis in the United States.  J Surg Res. 2018;223:251-258. doi:10.1016/j.jss.2017.10.007PubMedGoogle ScholarCrossref
24.
Venkatesh  AK, Mei  H, Kocher  KE,  et al.  Identification of emergency department visits in Medicare administrative claims: approaches and implications.  Acad Emerg Med. 2017;24(4):422-431. doi:10.1111/acem.13140PubMedGoogle ScholarCrossref
25.
Kliegman  RM, Stanton  BMD, St Geme  J, Schor  NF. Acute appendicitis. In:  Nelson Textbook of Pediatrics. 20th ed. Amsterdam, the Netherlands: Elsevier; 2016:1887-1894.
26.
Martin  RF. Acute appendicitis in adults: clinical manifestations and differential diagnosis. https://www.uptodate.com/contents/acute-appendicitis-in-adults-clinical-manifestations-and-differential-diagnosis/print#!. Updated April 27, 2018. Accessed January 16, 2020.
27.
US Census Bureau. Census regions and divisions of the United States. https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf. Updated April 17, 2013. Accessed January 14, 2020.
28.
Moore  BJ, White  S, Washington  R, Coenen  N, Elixhauser  A.  Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index.  Med Care. 2017;55(7):698-705. doi:10.1097/MLR.0000000000000735PubMedGoogle ScholarCrossref
29.
Medford-Davis  L, Park  E, Shlamovitz  G, Suliburk  J, Meyer  AN, Singh  H.  Diagnostic errors related to acute abdominal pain in the emergency department.  Emerg Med J. 2016;33(4):253-259. doi:10.1136/emermed-2015-204754PubMedGoogle ScholarCrossref
30.
Shy  BD, Loo  GT, Lowry  T,  et al.  Bouncing back elsewhere: multilevel analysis of return visits to the same or a different hospital after initial emergency department presentation.  Ann Emerg Med. 2018;71(5):555-563. doi:10.1016/j.annemergmed.2017.08.023PubMedGoogle ScholarCrossref
31.
Ko  M, Lee  Y, Chen  C, Chou  P, Chu  D.  Incidence of and predictors for early return visits to the emergency department: a population-based survey.  Medicine (Baltimore). 2015;94(43):e1770. doi:10.1097/MD.0000000000001770PubMedGoogle Scholar
32.
Flum  DR, Koepsell  T.  The clinical and economic correlates of misdiagnosed appendicitis: nationwide analysis.  Arch Surg. 2002;137(7):799-804. doi:10.1001/archsurg.137.7.799PubMedGoogle ScholarCrossref
33.
Raja  AS, Wright  C, Sodickson  AD,  et al.  Negative appendectomy rate in the era of CT: an 18-year perspective.  Radiology. 2010;256(2):460-465. doi:10.1148/radiol.10091570PubMedGoogle ScholarCrossref
34.
Reuchlin-Vroklage  LM, Bierma-Zeinstra  S, Benninga  MA, Berger  MY.  Diagnostic value of abdominal radiography in constipated children: a systematic review.  Arch Pediatr Adolesc Med. 2005;159(7):671-678. doi:10.1001/archpedi.159.7.671PubMedGoogle ScholarCrossref
35.
Pensabene  L, Buonomo  C, Fishman  L, Chitkara  D, Nurko  S.  Lack of utility of abdominal x-rays in the evaluation of children with constipation: comparison of different scoring methods.  J Pediatr Gastroenterol Nutr. 2010;51(2):155-159. doi:10.1097/MPG.0b013e3181cb4309PubMedGoogle ScholarCrossref
36.
Qiao  WP, Powell  ES, Witte  MP, Zelder  MR.  Relationship between racial disparities in ED wait times and illness severity.  Am J Emerg Med. 2016;34(1):10-15. doi:10.1016/j.ajem.2015.08.052PubMedGoogle ScholarCrossref
37.
Schrader  CD, Lewis  LM.  Racial disparity in emergency department triage.  J Emerg Med. 2013;44(2):511-518. doi:10.1016/j.jemermed.2012.05.010PubMedGoogle ScholarCrossref
38.
Smith  MP, Katz  DS, Lalani  T,  et al.  ACR appropriateness criteria right lower quadrant pain—suspected appendicitis.  Ultrasound Q. 2015;31(2):85-91. doi:10.1097/RUQ.0000000000000118PubMedGoogle ScholarCrossref
39.
Drake  FT, Flum  DR.  Improvement in the diagnosis of appendicitis.  Adv Surg. 2013;47:299-328. doi:10.1016/j.yasu.2013.03.003PubMedGoogle ScholarCrossref
40.
Shogilev  DJ, Duus  N, Odom  SR, Shapiro  NI.  Diagnosing appendicitis: evidence-based review of the diagnostic approach in 2014.  West J Emerg Med. 2014;15(7):859-871. doi:10.5811/westjem.2014.9.21568PubMedGoogle ScholarCrossref
41.
van Randen  A, Lameris  W, Luitse  JS,  et al; OPTIMA Study Group.  The role of plain radiographs in patients with acute abdominal pain at the ED.  Am J Emerg Med. 2011;29(6):582-589. doi:10.1016/j.ajem.2009.12.020PubMedGoogle ScholarCrossref
42.
Ceresoli  M, Zucchi  A, Allievi  N,  et al.  Acute appendicitis: epidemiology, treatment and outcomes—analysis of 16544 consecutive cases.  World J Gastrointest Surg. 2016;8(10):693-699. doi:10.4240/wjgs.v8.i10.693PubMedGoogle ScholarCrossref
43.
Sartelli  M, Baiocchi  GL, Di Saverio  S,  et al.  Prospective observational study on acute appendicitis worldwide (POSAW).  World J Emerg Surg. 2018;13:19. doi:10.1186/s13017-018-0179-0PubMedGoogle ScholarCrossref
44.
Sceats  LA, Trickey  AW, Morris  AM, Kin  C, Staudenmayer  KL.  Nonoperative management of uncomplicated appendicitis among privately insured patients.  JAMA Surg. 2019;154(2):141-149. doi:10.1001/jamasurg.2018.4282PubMedGoogle ScholarCrossref
45.
Okafor  N, Payne  VL, Chathampally  Y, Miller  S, Doshi  P, Singh  H.  Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine.  Emerg Med J. 2016;33(4):245-252. doi:10.1136/emermed-2014-204604PubMedGoogle ScholarCrossref
46.
Singh  H, Khanna  A, Spitzmueller  C, Meyer  AND.  Recommendations for using the revised safer Dx instrument to help measure and improve diagnostic safety.  Diagnosis (Berl). 2019;6(4):315-323. doi:10.1515/dx-2019-0012PubMedGoogle ScholarCrossref
47.
Kim  DW, Yoon  HM, Lee  JY,  et al.  Diagnostic performance of CT for pediatric patients with suspected appendicitis in various clinical settings: a systematic review and meta-analysis.  Emerg Radiol. 2018;25(6):627-637. doi:10.1007/s10140-018-1624-9PubMedGoogle ScholarCrossref
48.
Rud  B, Vejborg  TS, Rappeport  ED, Reitsma  JB, Wille-Jørgensen  P.  Computed tomography for diagnosis of acute appendicitis in adults.  Cochrane Database Syst Rev. 2019;2019(11). doi:10.1002/14651858.CD009977.pub2PubMedGoogle Scholar
49.
Murphy  DR, Meyer  AN, Sittig  DF, Meeks  DW, Thomas  EJ, Singh  H.  Application of electronic trigger tools to identify targets for improving diagnostic safety.  BMJ Qual Saf. 2019;28(2):151-159. doi:10.1136/bmjqs-2018-008086PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Family History As A Clue To Appendicitis?
    Harald Aanning, MD | Retired general surgeon
    A positive family history may be helpful in establishing a diagnosis of appendicitis - especially if that history points to an unusual presentation...
    CONFLICT OF INTEREST: None Reported
    Original Investigation
    Emergency Medicine
    March 9, 2020

    Factors Associated With Potentially Missed Diagnosis of Appendicitis in the Emergency Department

    Author Affiliations
    • 1Department of Emergency Medicine, University of Michigan, Ann Arbor
    • 2Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor
    • 3Department of Health Services Research, Baylor College of Medicine, Houston, Texas
    • 4Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
    • 5Department of Emergency Medicine, Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota
    • 6Department of Surgery, University of Michigan, Ann Arbor
    JAMA Netw Open. 2020;3(3):e200612. doi:10.1001/jamanetworkopen.2020.0612
    Key Points español 中文 (chinese)

    Question  What factors are associated with a potentially missed diagnosis of appendicitis in the emergency department among adults and children?

    Findings  In this cohort study of 123 711 patients diagnosed with appendicitis, insurance claims data indicated that appendicitis was potentially missed in 6.0% of adults and 4.4% of children during the initial emergency department visit. Factors associated with potentially missed appendicitis included female sex, the coexistence of abdominal pain and constipation, and the presence of comorbidities.

    Meaning  Population-based estimates of the rates of potentially missed appendicitis reveal opportunities for improvement and identify factors that may alert clinicians and mitigate the risk of missed diagnosis.

    Abstract

    Importance  Appendicitis may be missed during initial emergency department (ED) presentation.

    Objective  To compare patients with a potentially missed diagnosis of appendicitis (ie, patients with symptoms associated with appendicitis, including abdominal pain, constipation, nausea and/or vomiting, fever, and diarrhea diagnosed within 1-30 days after initial ED presentation) with patients diagnosed with appendicitis on the same day of ED presentation to identify factors associated with potentially missed appendicitis.

    Design, Setting, and Participants  In this cohort study, a retrospective analysis of commercially insured claims data was conducted from January 1 to December 15, 2019. Patients who presented to the ED with undifferentiated symptoms associated with appendicitis between January 1, 2010, and December 31, 2017, were identified using the Clinformatics Data Mart administrative database (Optum Insights). The study sample comprised eligible adults (aged ≥18 years) and children (aged <18 years) who had previous ED visits within 30 days of an appendicitis diagnosis.

    Main Outcomes and Measures  Potentially missed diagnosis of appendicitis. Adjusted odds ratios (AORs) for abdominal pain and its combinations with other symptoms associated with appendicitis were compared between patients with a same-day diagnosis of appendicitis and patients with a potentially missed diagnosis of appendicitis.

    Results  Of 187 461 patients with a diagnosis of appendicitis, a total of 123 711 (66%; 101 375 adults [81.9%] and 22 336 children [18.1%]) were eligible for analysis. Among adults, 51 923 (51.2%) were women, with a mean (SD) age of 44.3 (18.2) years; among children, 9631 (43.1%) were girls, with a mean (SD) age of 12.2 (18.2) years. The frequency of potentially missed appendicitis was 6060 of 101 375 adults (6.0%) and 973 of 22 336 children (4.4%). Patients with isolated abdominal pain (adults, AOR, 0.65; 95% CI, 0.62-0.69; P < .001; children, AOR, 0.79; 95% CI, 0.69-0.90; P < .001) or with abdominal pain and nausea and/or vomiting (adults, AOR, 0.90; 95% CI, 0.84-0.97; P = .003; children, AOR, 0.84; 95% CI, 0.71-0.98; P = .03) were less likely to have missed appendicitis. Patients with abdominal pain and constipation (adults, AOR, 1.51; 95% CI, 1.31-1.75; P < .001; children, AOR, 2.43; 95% CI, 1.86-3.17; P < .001) were more likely to have missed appendicitis. Stratified by the presence of undifferentiated symptoms, women (abdominal pain, AOR, 1.68; 95% CI, 1.58-1.78; nausea and/or vomiting, AOR, 1.68; 95% CI, 1.52-1.85; fever, AOR, 1.32; 95% CI, 1.10-1.59; diarrhea, AOR, 1.19; 95% CI, 1.01-1.40; and constipation, AOR, 1.50; 95% CI, 1.24-1.82) and girls (abdominal pain, AOR, 1.64; 95% CI, 1.43-1.88; nausea and/or vomiting, AOR, 1.74; 95% CI, 1.42-2.13; fever, AOR, 1.55; 95% CI, 1.14-2.11; diarrhea, AOR, 1.80; 95% CI, 1.19-2.74; and constipation, AOR, 1.25; 95% CI, 0.88-1.78) as well as patients with a comorbidity index of 2 or greater (adults, abdominal pain, AOR, 3.33; 95% CI, 3.09-3.60; nausea and/or vomiting, AOR, 3.66; 95% CI, 3.23-4.14; fever, AOR, 5.00; 95% CI, 3.79-6.60; diarrhea, AOR, 4.27; 95% CI, 3.39-5.38; and constipation, AOR, 4.17; 95% CI, 3.08-5.65; children, abdominal pain, AOR, 2.42; 95% CI, 1.93-3.05; nausea and/or vomiting, AOR, 2.55; 95% CI, 1.89-3.45; fever, AOR, 4.12; 95% CI, 2.71-6.25; diarrhea, AOR, 2.17; 95% CI, 1.18-3.97; and constipation, AOR, 2.19; 95% CI, 1.30-3.70) were more likely to have missed appendicitis. Adult patients who received computed tomographic scans at the initial ED visit (abdominal pain, AOR, 0.58; 95% CI, 0.52-0.65; nausea and/or vomiting, AOR, 0.63; 95% CI, 0.52-0.75; fever, AOR, 0.41; 95% CI, 0.29-0.58; diarrhea, AOR, 0.83; 95% CI, 0.58-1.20; and constipation, AOR, 0.60; 95% CI, 0.39-0.94) were less likely to have missed appendicitis.

    Conclusions and Relevance  Regardless of age, a missed diagnosis of appendicitis was more likely to occur in women, patients with comorbidities, and patients who experienced abdominal pain accompanied by constipation. Population-based estimates of the rates of potentially missed appendicitis reveal opportunities for improvement and identify factors that may mitigate the risk of a missed diagnosis.

    Introduction

    Appendicitis is one of the most common surgical emergencies in the United States. However, the diagnosis of appendicitis is missed in 3.8% to 15.0% of children and in 5.9% to 23.5% of adults during emergency department (ED) visits.1-5 Appendicitis is the second most common condition among pediatric patients and the third most common condition cited in adult malpractice insurance claims.6,7

    The ED is a high-risk setting for diagnostic errors.8,9 With approximately 80% of errors potentially preventable10 and approximately 50% of all diagnostic errors having the potential for patient harm,11 the National Academies of Sciences, Engineering, and Medicine emphasizes that improving the diagnostic process is a public health imperative.10,11 A variety of approaches has been used to investigate diagnostic errors,12 including a recently proposed conceptual model that uses a symptom-disease dyad approach for analysis.13 This approach is particularly useful when there is a biologically plausible association between a symptom or a combination of symptoms and the eventual disease, which, when applied to the clinical context, assumes that the symptom should have directed the practitioner to make a timely and accurate diagnosis.13 Symptom-disease dyad analyses have been applied to large data sets to identify potential errors, including claims data errors, and have been used to estimate the probable missed diagnosis of various conditions, including chest pain and myocardial infarction,14 dizziness or headache and stroke,15 and oncology-related symptoms and pediatric cancer,16 to obtain population-level estimates of diagnostic errors.

    Previous studies examining patients with appendicitis that was missed at ED visits have tested several factors, including the association of a missed appendicitis diagnosis with an individual symptom or a combination of presenting symptoms, physical examination findings, clinical decision rules, and imaging, specifically abdominal radiography and ultrasonographic imaging in children.1-3 In other studies, constipation in children, female sex, and patient age (aged <5 years or >50 years) were also associated with a delayed or missed appendicitis diagnosis.17-19 However, the factors associated with missed appendicitis have been inconsistent, likely owing to small cohort sizes, single-site studies, and loss of patient follow-up, all of which can be mitigated by investigating large administrative data sets. In this study, we analyzed administrative claims data from a large private health insurance provider to examine factors associated with a potentially missed diagnosis of appendicitis at initial ED presentation.

    Methods

    We analyzed patients who presented to the ED with undifferentiated symptoms associated with appendicitis between 2010 and 2017 based on a symptom-disease pair analysis of diagnostic error (SPADE) look-back approach.13 Patients were identified using Clinformatics Data Mart (Optum Insights), a deidentified administrative claims database that captures all ED, outpatient, and inpatient encounters of more than 75 million individuals who are commercially insured by a single large US private health insurance provider. The database includes member enrollment data, demographic characteristics, individual-level insurance claims, a subset of laboratory test results, hospital discharge information, and pharmacy claims. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies.20 The study was approved by the institutional review board of the University of Michigan, with a waiver of consent granted because data were secondary and deidentified.

    Study Population

    We identified adults (aged ≥18 years) and children (aged <18 years) who were diagnosed with appendicitis using codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and the International Statistical Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM; eTable 1 in the Supplement).21-23 We then identified ED visits24 among patients who presented with isolated or combinations of undifferentiated symptoms1-3 that are commonly associated with appendicitis (abdominal pain, constipation, diarrhea, fever, and nausea and/or vomiting) in the 0 to 30 days before an appendicitis diagnosis.25,26 If a patient had more than 1 ED visit in a 30-day period with undifferentiated symptoms, we selected the ED visit closest to the appendicitis diagnosis. We excluded patients who had a diagnosis date of appendicitis after the appendectomy date (Figure).

    We defined a potentially missed diagnosis of appendicitis as an initial (or index) ED visit at which a patient presented with any single undifferentiated symptom or combination of undifferentiated symptoms associated with appendicitis for which the patient did not receive a diagnosis of appendicitis on the same day of symptom presentation but received a subsequent diagnosis of appendicitis within 30 days after the index ED visit (the potentially missed appendicitis group). We treated more than 1 ED visit by the same patient on the same day as a single ED visit. We defined a same-day diagnosis (ie, no missed diagnosis) of appendicitis as an initial ED visit at which a patient presented with any undifferentiated symptom or combination of undifferentiated symptoms associated with appendicitis for which the patient received a diagnosis of appendicitis on the same day of symptom presentation (the same-day diagnosis group).

    Outcome Measure and Covariates

    The main outcome of interest was the potentially missed diagnosis of appendicitis. We collected data on covariates from initial ED visits, including patient demographic characteristics (age, sex, race, insurance plan, and US census region)27 from health insurance membership files, clinical presentation of symptoms and comorbidities, laboratory tests performed (urinalysis and complete blood cell count), and types of abdominal radiographic imaging performed (radiography, ultrasonography, and computed tomography [CT]; eTable 1 in the Supplement). We calculated the Elixhauser Comorbidity Index to assess the association between potentially missed appendicitis and existing comorbidities for all ED visits.28

    Statistical Analysis

    We compared the same-day diagnosis and potentially missed appendicitis groups for all categorical variables using χ2 tests. Owing to the ordinal nature of the Elixhauser Comorbidity Index, we performed an ordered logistic regression analysis with comorbidity index as the dependent variable and potentially missed appendicitis status as only 1 covariate, and we reported the results of an overall type 3 analysis of fixed effects significance test for the overall effect of potentially missed appendicitis. For continuous variables, such as age, we calculated means and SDs and examined the distribution with the interquartile range (IQR) to ensure robustness of the performance of parametric t tests between the 2 groups. Continuous variables were examined and verified for normality using histograms and quantile-quantile plots for visual inspection; therefore, informed by distributional considerations, clinically relevant cutoffs were established. We used Kaplan-Meier survival curves to profile children and adults regardless of the undifferentiated symptom at the index ED visit and the length of time between the index visit and the return visit at which an appendicitis diagnosis was made. We calculated confidence intervals using Hall-Wellner 95% confidence bands and implemented a log-rank test to compare significance between the survival curves of adults and children.

    We performed bivariate logistic regression analyses to estimate unadjusted odds ratios (ORs) of potentially missed appendicitis in 6 different models based on the following undifferentiated symptoms as exposure variables: (1) abdominal pain only; (2) abdominal pain and constipation; (3) abdominal pain and nausea and/or vomiting; (4) abdominal pain, nausea and/or vomiting, and fever; (5) abdominal pain, nausea and/or vomiting, fever, and constipation; and (6) no abdominal pain but any of the other undifferentiated symptoms could be present (eMethods in the Supplement). To examine the adjusted odds ratios (AORs) of potentially missed appendicitis for each of the undifferentiated symptom combinations, we developed a family of multivariable logistic regression models with a common set of clinically relevant covariates (demographic characteristics, comorbidities, laboratory tests, and abdominal imaging) and with the aforementioned exposure variable representing a symptom combination covariate.

    Diagnostic evaluation for appendicitis was constructed as 1 categorical variable with 4 mutually exclusive groupings: (1) any CT (with or without additional imaging), (2) any ultrasonography (no evidence of CT), (3) radiography only (no evidence of CT or ultrasonography), and (4) no imaging. Further, we used multivariable logistic regression models with the same common set of clinically relevant covariates among 5 subpopulations, each consisting of patients with the presence of 1 single undifferentiated symptom regardless of the presence of other symptoms. Model concordance was examined using a C statistic. We conducted a complete case analysis to examine bias by fitting models with the effective sample size reduction and comparing unadjusted ORs and AORs for each of the included covariates.

    Statistical analysis was completed between January 1 and September 15, 2019. All data analyses were conducted using SAS statistical software, version 9.4 (SAS Institute Inc), and statistical testing was 2-tailed and unpaired, with a significance threshold of P = .05. All results were reported separately for adults and children.

    Results

    Of 187 461 patients with appendicitis diagnosed between January 1, 2010, and December 31, 2017, a total of 123 711 patients (66%; 101 375 adults [81.9%] and 22 336 children [18.1%]) were eligible for analysis. Among adults, 51 923 (51.2%) were women, with a mean (SD) age of 44.3 (18.2) years. Among children, 9631 (43.1%) were girls, with a mean (SD) age of 12.2 (18.2) years. A total of 7033 patients (5.7%) met the criteria for a potentially missed diagnosis of appendicitis, including 6060 adults (6.0%) and 973 children (4.4%) (Figure). The same-day diagnosis group comprised 95 315 adults (94.0%) and 21 363 children (95.6%).

    Adults in the potentially missed appendicitis group were older (mean [SD] age, 50.2 [20.0] vs 43.9 [17.9] years, respectively; P < .001) and more likely to be women (3884 patients [64.1%] vs 48 039 patients [50.4%], respectively; P < .001) (Table 1) compared with adults in the same-day diagnosis group. The 2 adult groups also had different racial compositions, with 506 patients (8.3%) in the potentially missed appendicitis group compared with 5929 patients (6.2%) in the same-day diagnosis group having black ancestry (P < .001). Similar differences in sex and race were observed in children. Patients in the potentially missed appendicitis group were also more likely to have 2 or more comorbidities compared with those in the same-day diagnosis group (among adults, 3514 patients [58.0%] vs 29 050 patients [30.5%], respectively; P < .001; among children, 121 patients [12.4%] vs 1078 patients [5.0%], respectively; P < .001). The appendectomy rate was higher in the same-day diagnosis group compared with the potentially missed appendicitis group among both adults (80 143 patients [84.1%] vs 3097 patients [51.1%], respectively; P < .001) and children (17 134 patients [80.2%] vs 667 patients [68.6%], respectively; P < .001). Based on the same data shown in Table 1, we calculated the proportion of patients with potentially missed appendicitis for each of the demographic and ED visit characteristics (eTable 2 in the Supplement).

    The frequency of undifferentiated symptoms is presented in eTable 3 in the Supplement. Abdominal pain was the most prevalent symptom in both adults (93 285 patients [97.9%] in the same-day diagnosis group vs 5487 patients [90.5%] in the potentially missed appendicitis group; P < .001) and children (20 948 patients [98.1%] the same-day diagnosis group vs 906 patients [93.1%] in the potentially missed appendicitis group; P < .001), followed by nausea and/or vomiting. Abdominal pain was more frequently absent in the potentially missed appendicitis group compared with the same-day diagnosis group among both adults (573 patients [9.5%] vs 2020 patients [2.1%], respectively; P < .001) and children (67 patients [6.9%] vs 415 patients [1.9%], respectively; P < .001). Constipation in combination with abdominal pain was more frequent in the potentially missed appendicitis group compared with the same-day diagnosis group among adults (228 patients [3.8%] vs 1953 patients [2.0%], respectively; P < .001) and children (67 patients [6.9%] vs 636 patients [3.0%], respectively; P < .001).

    Computed tomography was the most common diagnostic radiologic test performed among patients diagnosed with appendicitis regardless of the patient’s age and the timing of the diagnosis (same-day vs return visit); adults were more likely to receive a CT scan than children, with 93 424 adults (92.2%; 88 241 patients [92.6%] in the same-day diagnosis group vs 5183 patients [85.5%] in the potentially missed appendicitis group; P < .001) and 13 399 children (60.0%; 12 769 patients [59.8%] in the same-day diagnosis group vs 630 patients [64.7%] in the potentially missed appendicitis group; P < .001) receiving a CT scan (Table 1).

    Ultrasonography was the second most common imaging modality, performed in 10 032 adults (9.9%; 8678 patients [9.1%] in the same-day diagnosis group vs 1354 patients [22.3%] in the potentially missed appendicitis group; P < .001) and 9764 children (43.7%; 9296 patients [43.5%] in the same-day diagnosis group vs 468 patients [48.1%] in the potentially missed appendicitis group; P = .005). Unlike the same-day diagnosis group, in which a CT scan was performed at the index ED visit, only 105 of 5183 adults (2.0%) and 13 of 630 children (2.1%) in the potentially missed appendicitis group received a CT scan at the index visit. Indeed, most of the radiologic tests, including CT, ultrasonography, radiography, and laboratory investigations (including complete blood cell count and urinalysis), were performed in the potentially missed appendicitis group at return ED visits and not at the index ED visit (eTable 6 in the Supplement).

    Kaplan-Meier survival curves stratified by children and adults show the probability of a patient receiving an appendicitis diagnosis at a return visit after the initial ED visit (eFigure in the Supplement). Children had a significantly shorter median time in the number of days between the index visit and return visit (2 days; IQR, 1-7 days) compared with adults (4 days; IQR, 1-13 days) (log rank, P < .001).

    The results from logistic regression models are reported in Table 2, which shows unadjusted ORs and AORs for exposure to abdominal pain alone and in combination with other symptoms. Patients who did not have abdominal pain at the index ED presentation were more likely to have missed appendicitis, both in adults (AOR, 3.57; 95% CI, 3.22-3.95; P < .001) and children (AOR, 2.99; 95% CI, 2.25-3.96; P < .001). Patients who had both abdominal pain and constipation at the index ED visit (in adults, AOR, 1.51; 95% CI, 1.31-1.75; in children, AOR, 2.43; 95% CI, 1.86-3.17) were also more likely to have missed appendicitis. In the 6 adult models, the AORs for black patients compared with white patients consistently ranged from 1.14 to 1.16 (abdominal pain, AOR, 1.14; 95% CI, 1.02-1.27; nausea and/or vomiting, AOR, 1.09; 95% CI, 0.92-1.29; fever, AOR, 1.17; 95% CI, 0.84-1.65; diarrhea, AOR, 0.80; 95% CI, 0.58-1.11; and constipation, AOR, 0.78; 95% CI, 0.55-1.09) (Table 3). This finding was not observed in the 6 pediatric models. The detailed results for all covariates included in the models are reported in eTable 4 and eTable 5 in the Supplement.

    After stratifying for symptom presentation, women (abdominal pain, AOR, 1.68; 95% CI, 1.58-1.78; nausea and/or vomiting, AOR, 1.68; 95% CI, 1.52-1.85; fever, AOR, 1.32; 95% CI, 1.10-1.59; diarrhea, AOR, 1.19; 95% CI, 1.01-1.40; and constipation, AOR, 1.50; 95% CI, 1.24-1.82) and adults with 2 or more comorbidities (abdominal pain, AOR, 3.33; 95% CI, 3.09-3.60; nausea and/or vomiting, AOR, 3.66; 95% CI, 3.23-4.14; fever, AOR, 5.00; 95% CI, 3.79-6.60; diarrhea, AOR, 4.27; 95% CI, 3.39-5.38; and constipation, AOR, 4.17; 95% CI, 3.08-5.65) were more likely to have missed appendicitis (Table 3). Furthermore, adults who received a CT scan (abdominal pain, AOR, 0.58; 95% CI, 0.52-0.65; nausea and/or vomiting, AOR, 0.63; 95% CI, 0.52-0.75; fever, AOR, 0.41; 95% CI, 0.29-0.58; diarrhea, AOR, 0.83; 95% CI, 0.58-1.20; and constipation, AOR, 0.60; 95% CI, 0.39-0.94) were less likely to have missed appendicitis. However, adults who underwent radiography as the only radiologic test (abdominal pain, AOR, 2.73; 95% CI, 2.12-3.51; nausea and/or vomiting, AOR, 2.83; 95% CI, 1.93-4.16; fever, AOR, 1.49; 95% CI, 0.63-3.52; diarrhea, AOR, 3.65; 95% CI, 1.84-7.23; and constipation, AOR, 4.07; 95% CI, 2.11-7.85) were more likely to have missed appendicitis.

    Similar results were observed among children, with girls (abdominal pain, AOR, 1.64; 95% CI, 1.43-1.88; nausea and/or vomiting, AOR, 1.74; 95% CI, 1.42-2.13; fever, AOR, 1.55; 95% CI, 1.14-2.11; diarrhea, AOR, 1.80; 95% CI, 1.19-2.74; and constipation, AOR, 1.25; 95% CI, 0.88-1.78) and children with 2 or more comorbidities (abdominal pain, AOR, 2.42; 95% CI, 1.93-3.05; nausea and/or vomiting, AOR, 2.55; 95% CI, 1.89-3.45; fever, AOR, 4.12; 95% CI, 2.71-6.25; diarrhea, AOR, 2.17; 95% CI, 1.18-3.97; and constipation, AOR, 2.19; 95% CI, 1.30-3.70) more likely to have missed appendicitis (Table 4).

    Discussion

    We analyzed 8 years of insurance claims from a large private health insurance provider to estimate the frequency of a potentially missed diagnosis of appendicitis and to describe the associated factors. We found that female sex and a higher comorbidity index were associated with potentially missed appendicitis in both children and adults. Patients with a potentially missed diagnosis of appendicitis were more likely to be examined using only abdominal radiography during the initial ED visit. Among adults, black race was associated with potentially missed appendicitis in models that included isolated symptoms or symptom combinations, but this association was not seen in children.

    The analysis of administrative data sets allowed us to draw estimates of missed appendicitis diagnoses in large cohorts of patients. Abdominal pain, the most common presenting symptom with or without associated symptoms (constipation, diarrhea, fever, and nausea and/or vomiting), is closely associated with appendicitis. However, abdominal pain is also a major factor associated with return ED visits. One ED-based study examining return visits among patients with abdominal pain revealed that nearly one-third of cases had a diagnostic error.29 Our approach of using large administrative data sets enabled us to overcome sample size limitations and could account for the loss of patients seeking care outside the original health care system after the initial ED visit; thus, this approach offered additional insights into the epidemiology of diagnostic errors.30,31

    Our study findings of missed appendicitis rates for adults (6.0%) and children (4.4%) along with the increased risk in women, older adults, and individuals with comorbid conditions are consistent with previous studies, thus suggesting the validity of the approach used in this study.32,33 In addition, only 1 previous study revealed appendicitis as the most common missed diagnosis among children with constipation,17 which is consistent with our results. Constipation at index ED presentation may be a factor in the decision-making of the ED practitioner and may add to the risk of false-negative test results for appendicitis. However, no such data exist in adults with a missed diagnosis of appendicitis. Constipation might be the reason that practitioners perform radiography, which may lead to confirmation bias and early closure in the medical decision-making process, even though the utility of radiography for constipation is unclear.34,35

    The increased risk of potentially missed appendicitis associated with race is a finding requiring further exploration. In 6 separate multivariable models involving abdominal pain or its combination with other symptoms along with other variables, black race compared with white race was consistently associated with a higher rate of potentially missed appendicitis among adults, while adult Asian and pediatric Hispanic patients were less likely to have missed appendicitis (eTable 4 and eTable 5 in the Supplement). Although observations of racial and ethnic disparities in health care in general and in ED care in specific are not new,36,37 only 1 previous study of missed appendicitis included race in the analysis, and it did not find race to be associated with missed appendicitis.2 Our results identified racial disparity in a group of privately insured patients in a single-payer claims data set. Future work should involve validating the disparities in care associated with race and/or ethnicity in additional data sets, including Medicaid and Medicare claims data, and other types of prospectively conducted studies designed to investigate these differences.

    Our analyses of radiologic investigations performed at the initial ED visit revealed discordance between guidelines and appropriate use. Ultrasonographic imaging is inexpensive, does not expose the patient to ionizing radiation, and is the recommended first-line imaging modality in children and pregnant women.38,39 However, ultrasonographic imaging is operator dependent, and the appendix is often difficult to visualize in pregnant women, obese patients, and patients with retrocecal appendices.38,40 Computed tomography exposes patients to ionizing radiation and is often considered a second-line imaging modality, especially in children. In our analyses, adult patients with undifferentiated symptoms were more likely to be diagnosed with appendicitis at initial presentation if they had CT performed. For adults, the ultrasonography rate of 22.3% in the potentially missed appendicitis group and 9.1% in the same-day diagnosis group may reflect the lower test performance of ultrasonography compared with CT as an imaging modality. In children, the overall lower CT rates in both the same-day diagnosis and potentially missed appendicitis groups and the higher ultrasonography rates are consistent with the emphasis on less exposure to ionizing radiation in children.38,39 Patients who received only plain abdominal radiography were more likely to be in the potentially missed appendicitis group regardless of age. This finding is consistent with the suboptimal test characteristics of plain radiography in the diagnosis of appendicitis41 and likely reflects the possibility that the clinicians were not considering appendicitis as the primary diagnosis.

    With additional validation, our results may be able to assist clinicians by identifying phenotypes in patients for whom missed appendicitis is likely. Notably, our analysis could help further improve clinical decision-making by defining populations at risk of potentially missed appendicitis, and it could lead to a more careful discussion with the family and patient with regard to the further need for imaging, the type and timing of imaging, and follow-up. For instance, if validated, our results suggest that the probability of missed appendicitis for an Asian man aged 41 to 60 years who presents to the ED with abdominal pain and no comorbid conditions is 2.8%. On the other hand, a black woman aged 18 to 25 years with abdominal pain and 2 or more comorbid conditions would have a probability of missed appendicitis of 22.9%. Such estimates, if validated, are actionable and can be incorporated during clinical evaluation to enhance shared decision-making and ensure access to follow-up health care services as well as safe and timely diagnosis.10

    Limitations

    Our analysis had several limitations. Administrative claims data are limited by variability in the assignment of diagnosis codes. For instance, the appendectomy rates in our analysis were lower than the reported rates (>90%) of other studies,41-44 especially among the potentially missed appendicitis group, which may be reflective of incomplete coding or billing. A low appendectomy rate may also reflect a mislabeling in the claims data of patients with an appendicitis diagnosis. In this study, we took a more liberal approach to defining appendicitis with the aim of overcoming possible coding errors in claims data. Future studies may develop an optimal definition for the diagnosis of appendicitis in administrative data. The use of claims data without robust patient-level clinical data and context did not allow us to draw conclusions about the frontline decision-making or system factors involved in the diagnostic process.45 However, this approach allowed us to identify signals in large data sets to screen for patients at a higher risk of potentially missed appendicitis.

    We used a simplistic symptom-disease dyad approach and added a multisymptom-disease nuance to draw our conclusions. Although we used the look-back approach, we cannot comment on whether our findings would be different if both approaches (ie, look-back and look-forward) were simultaneously applied. Although similar symptom-disease dyad methods have been used in analyses of administrative data sets to gauge estimates about diagnostic errors,13-15 future studies using chart abstraction on patient-level data, such as the presence of right lower-quadrant pain or rebound tenderness as a criterion standard,46 are still needed to validate the claims-based definition for missed appendicitis. To better understand the frequency of potentially missed appendicitis among patients who present with uncommon and atypical symptoms as well as the role of practitioner characteristics, we suggest that our results should be validated using a study design that incorporates medical record review. Laboratory tests, especially complete blood cell counts, are often used in the diagnostic process for appendicitis, but very few laboratory claims (21%) were available, which was likely owing to incomplete laboratory claims data, bundled payments, and missing incorporation of laboratory coding.

    Among patients who received a CT scan at the index ED visit, 5.5% of adults and 4.7% of children were in the potentially missed appendicitis group. It is difficult to determine whether the radiologist or ED physician missed the signs of appendicitis at the index ED visit or whether the patient had normal CT results at the index ED visit followed by an appendicitis diagnosis at a return visit. Although a CT scan of the abdomen of patients with appendicitis has been reported to have excellent performance characteristics,47,48 2% of patients (105 adults and 13 children) with potentially missed appendicitis in our study had received a CT scan at the index visit. We are unable to comment on whether these results represent patients with unrelated ED visits (ie, the CT indicated true-negative results and the patient did not have appendicitis vs the patient had potentially missed appendicitis owing to false-negative CT results). All of these limitations can be overcome by future studies incorporating patient-level record reviews.49 Our data were limited to 1 private insurance provider; therefore, the results may not be representative of the entire US population. However, the advantage of using a single-payer claims data set included a lower likelihood of missing data, especially among patients who may not have had a return visit to the same ED.

    Conclusions

    Regardless of age, a missed diagnosis of appendicitis was more likely to occur in female patients, patients with comorbidities, and patients with abdominal pain accompanied by constipation. Population-based estimates of the rates of potentially missed appendicitis reveal opportunities for improvement and identify factors that may alert clinicians and mitigate the risk of missed diagnosis.

    Back to top
    Article Information

    Accepted for Publication: January 20, 2020.

    Published: March 9, 2020. doi:10.1001/jamanetworkopen.2020.0612

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Mahajan P et al. JAMA Network Open.

    Corresponding Author: Prashant Mahajan, MD, MPH, MBA, Department of Emergency Medicine, University of Michigan, 1540 E Hospital Dr, Room 2-737, SPC 4260, Ann Arbor, MI 48109-4260 (pmahajan@med.umich.edu).

    Author Contributions: Dr Mahajan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Mahajan, Basu, Pai, Singh, Kamdar.

    Acquisition, analysis, or interpretation of data: Mahajan, Basu, Pai, Petersen, Bellolio, Gadepalli, Kamdar.

    Drafting of the manuscript: Mahajan, Pai, Kamdar.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Basu, Pai, Petersen, Gadepalli, Kamdar.

    Administrative, technical, or material support: Mahajan, Basu, Pai.

    Supervision: Mahajan, Bellolio, Kamdar.

    Conflict of Interest Disclosures: Dr Singh reported receiving grants from the US Department of Veterans Affairs during the conduct of the study. Mr Kamdar reported receiving personal fees from Lucent Surgical Support Systems and Stanford University outside the submitted work. No other disclosures were reported.

    Funding/Support: This project was supported by financial funding from the Department of Emergency Medicine at the University of Michigan. Dr Singh is partially funded by the Houston VA Health Services Research and Development Service and the Center for Innovations in Quality, Effectiveness and Safety (CIN 13–413).

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Meeting Presentations: Preliminary results of this study were presented at the Society for Academic Emergency Medicine Annual Meeting; May 16, 2018; Indianapolis, Indiana; and at the Pediatric Academic Societies 2018 Meeting; May 7, 2018; Toronto, Ontario, Canada (poster presentation).

    Additional Contributions: Patrick Brady, MPH, of the Institute for Healthcare Policy and Innovation at the University of Michigan, provided administrative support and oversight for Messrs Kamdar and Basu. Elizabeth Duffy, MA, of the Department of Emergency Medicine at the University of Michigan, provided administrative and editorial support for Drs Mahajan and Pai. No compensation was received outside of usual salary.

    References
    1.
    Galai  T, Beloosesky  OZ, Scolnik  D, Rimon  A, Glatstein  M.  Misdiagnosis of acute appendicitis in children attending the emergency department: the experience of a large, tertiary care pediatric hospital.  Eur J Pediatr Surg. 2017;27(2):138-141. doi:10.1055/s-0035-1570757PubMedGoogle ScholarCrossref
    2.
    Naiditch  JA, Lautz  TB, Daley  S, Pierce  MC, Reynolds  M.  The implications of missed opportunities to diagnose appendicitis in children.  Acad Emerg Med. 2013;20(6):592-596. doi:10.1111/acem.12144PubMedGoogle ScholarCrossref
    3.
    Chang  YJ, Chao  HC, Kong  MS, Hsia  SH, Yan  DC.  Misdiagnosed acute appendicitis in children in the emergency department.  Chang Gung Med J. 2010;33(5):551-557.PubMedGoogle Scholar
    4.
    Graff  L, Russell  J, Seashore  J,  et al.  False-negative and false-positive errors in abdominal pain evaluation: failure to diagnose acute appendicitis and unnecessary surgery.  Acad Emerg Med. 2000;7(11):1244-1255. doi:10.1111/j.1553-2712.2000.tb00470.xPubMedGoogle ScholarCrossref
    5.
    Leung  YK, Chan  CP, Graham  CA, Rainer  TH.  Acute appendicitis in adults: diagnostic accuracy of emergency doctors in a university hospital in Hong Kong.  Emerg Med Australas. 2017;29(1):48-55. doi:10.1111/1742-6723.12738PubMedGoogle ScholarCrossref
    6.
    Brown  TW, McCarthy  ML, Kelen  GD, Levy  F.  An epidemiologic study of closed emergency department malpractice claims in a national database of physician malpractice insurers.  Acad Emerg Med. 2010;17(5):553-560. doi:10.1111/j.1553-2712.2010.00729.xPubMedGoogle ScholarCrossref
    7.
    Selbst  SM, Friedman  MJ, Singh  SB.  Epidemiology and etiology of malpractice lawsuits involving children in US emergency departments and urgent care centers.  Pediatr Emerg Care. 2005;21(3):165-169. PubMedGoogle Scholar
    8.
    Medford-Davis  LN, Singh  H, Mahajan  P.  Diagnostic decision-making in the emergency department.  Pediatr Clin North Am. 2018;65(6):1097-1105. doi:10.1016/j.pcl.2018.07.003PubMedGoogle ScholarCrossref
    9.
    Mahajan  P, Mollen  C, Alpern  ER,  et al.  An operational framework to study diagnostic errors in emergency departments: findings from a consensus panel  [published online November 25, 2019].  J Patient Saf. doi:10.1097/PTS.0000000000000624PubMedGoogle Scholar
    10.
    National Academies of Sciences, Engineering, and Medicine.  Improving Diagnosis in Health Care. Washington, DC: National Academies Press; 2015.
    11.
    Singh  H, Meyer  AN, Thomas  EJ.  The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.  BMJ Qual Saf. 2014;23(9):727-731. doi:10.1136/bmjqs-2013-002627PubMedGoogle ScholarCrossref
    12.
    Graber  ML.  The incidence of diagnostic error in medicine.  BMJ Qual Saf. 2013;22(suppl 2):ii21-ii27. doi:10.1136/bmjqs-2012-001615PubMedGoogle ScholarCrossref
    13.
    Liberman  AL, Newman-Toker  DE.  Symptom-disease pair analysis of diagnostic error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data.  BMJ Qual Saf. 2018;27(7):557-566. doi:10.1136/bmjqs-2017-007032PubMedGoogle ScholarCrossref
    14.
    Moy  E, Barrett  M, Coffey  R, Hines  AL, Newman-Toker  DE.  Missed diagnoses of acute myocardial infarction in the emergency department: variation by patient and facility characteristics.  Diagnosis (Berl). 2015;2(1):29-40. doi:10.1515/dx-2014-0053PubMedGoogle ScholarCrossref
    15.
    Newman-Toker  DE, Moy  E, Valente  E, Coffey  R, Hines  AL.  Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample.  Diagnosis (Berl). 2014;1(2):155-166. doi:10.1515/dx-2013-0038PubMedGoogle ScholarCrossref
    16.
    Carberry  AR, Hanson  K, Flannery  A,  et al.  Diagnostic error in pediatric cancer.  Clin Pediatr (Phila). 2018;57(1):11-18. doi:10.1177/0009922816687325PubMedGoogle ScholarCrossref
    17.
    Freedman  SB, Rodean  J, Hall  M,  et al.  Delayed diagnoses in children with constipation: multicenter retrospective cohort study.  J Pediatr. 2017;186:87-94. doi:10.1016/j.jpeds.2017.03.061PubMedGoogle ScholarCrossref
    18.
    Kraemer  M, Franke  C, Ohmann  C, Yang  Q; Acute Abdominal Pain Study Group.  Acute appendicitis in late adulthood: incidence, presentation, and outcome: results of a prospective multicenter acute abdominal pain study and a review of the literature.  Langenbecks Arch Surg. 2000;385(7):470-481. doi:10.1007/s004230000165PubMedGoogle ScholarCrossref
    19.
    Benabbas  R, Hanna  M, Shah  J, Sinert  R.  Diagnostic accuracy of history, physical examination, laboratory tests, and point-of-care ultrasound for pediatric acute appendicitis in the emergency department: a systematic review and meta-analysis.  Acad Emerg Med. 2017;24(5):523-551. doi:10.1111/acem.13181PubMedGoogle ScholarCrossref
    20.
    von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008PubMedGoogle ScholarCrossref
    21.
    Alore  EA, Ward  JL, Todd  SR,  et al.  Population-level outcomes of early versus delayed appendectomy for acute appendicitis using the American College of Surgeons National Surgical Quality Improvement Program.  J Surg Res. 2018;229:234-242. doi:10.1016/j.jss.2018.04.011PubMedGoogle ScholarCrossref
    22.
    Seudeal  K, Abidi  H, Shebrain  S.  Early versus delayed appendectomy: a comparison of outcomes.  Am J Surg. 2018;215(3):483-486. doi:10.1016/j.amjsurg.2017.10.057PubMedGoogle ScholarCrossref
    23.
    Horn  CB, Tian  D, Bochicchio  GV, Turnbull  IR.  Incidence, demographics, and outcomes of nonoperative management of appendicitis in the United States.  J Surg Res. 2018;223:251-258. doi:10.1016/j.jss.2017.10.007PubMedGoogle ScholarCrossref
    24.
    Venkatesh  AK, Mei  H, Kocher  KE,  et al.  Identification of emergency department visits in Medicare administrative claims: approaches and implications.  Acad Emerg Med. 2017;24(4):422-431. doi:10.1111/acem.13140PubMedGoogle ScholarCrossref
    25.
    Kliegman  RM, Stanton  BMD, St Geme  J, Schor  NF. Acute appendicitis. In:  Nelson Textbook of Pediatrics. 20th ed. Amsterdam, the Netherlands: Elsevier; 2016:1887-1894.
    26.
    Martin  RF. Acute appendicitis in adults: clinical manifestations and differential diagnosis. https://www.uptodate.com/contents/acute-appendicitis-in-adults-clinical-manifestations-and-differential-diagnosis/print#!. Updated April 27, 2018. Accessed January 16, 2020.
    27.
    US Census Bureau. Census regions and divisions of the United States. https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf. Updated April 17, 2013. Accessed January 14, 2020.
    28.
    Moore  BJ, White  S, Washington  R, Coenen  N, Elixhauser  A.  Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index.  Med Care. 2017;55(7):698-705. doi:10.1097/MLR.0000000000000735PubMedGoogle ScholarCrossref
    29.
    Medford-Davis  L, Park  E, Shlamovitz  G, Suliburk  J, Meyer  AN, Singh  H.  Diagnostic errors related to acute abdominal pain in the emergency department.  Emerg Med J. 2016;33(4):253-259. doi:10.1136/emermed-2015-204754PubMedGoogle ScholarCrossref
    30.
    Shy  BD, Loo  GT, Lowry  T,  et al.  Bouncing back elsewhere: multilevel analysis of return visits to the same or a different hospital after initial emergency department presentation.  Ann Emerg Med. 2018;71(5):555-563. doi:10.1016/j.annemergmed.2017.08.023PubMedGoogle ScholarCrossref
    31.
    Ko  M, Lee  Y, Chen  C, Chou  P, Chu  D.  Incidence of and predictors for early return visits to the emergency department: a population-based survey.  Medicine (Baltimore). 2015;94(43):e1770. doi:10.1097/MD.0000000000001770PubMedGoogle Scholar
    32.
    Flum  DR, Koepsell  T.  The clinical and economic correlates of misdiagnosed appendicitis: nationwide analysis.  Arch Surg. 2002;137(7):799-804. doi:10.1001/archsurg.137.7.799PubMedGoogle ScholarCrossref
    33.
    Raja  AS, Wright  C, Sodickson  AD,  et al.  Negative appendectomy rate in the era of CT: an 18-year perspective.  Radiology. 2010;256(2):460-465. doi:10.1148/radiol.10091570PubMedGoogle ScholarCrossref
    34.
    Reuchlin-Vroklage  LM, Bierma-Zeinstra  S, Benninga  MA, Berger  MY.  Diagnostic value of abdominal radiography in constipated children: a systematic review.  Arch Pediatr Adolesc Med. 2005;159(7):671-678. doi:10.1001/archpedi.159.7.671PubMedGoogle ScholarCrossref
    35.
    Pensabene  L, Buonomo  C, Fishman  L, Chitkara  D, Nurko  S.  Lack of utility of abdominal x-rays in the evaluation of children with constipation: comparison of different scoring methods.  J Pediatr Gastroenterol Nutr. 2010;51(2):155-159. doi:10.1097/MPG.0b013e3181cb4309PubMedGoogle ScholarCrossref
    36.
    Qiao  WP, Powell  ES, Witte  MP, Zelder  MR.  Relationship between racial disparities in ED wait times and illness severity.  Am J Emerg Med. 2016;34(1):10-15. doi:10.1016/j.ajem.2015.08.052PubMedGoogle ScholarCrossref
    37.
    Schrader  CD, Lewis  LM.  Racial disparity in emergency department triage.  J Emerg Med. 2013;44(2):511-518. doi:10.1016/j.jemermed.2012.05.010PubMedGoogle ScholarCrossref
    38.
    Smith  MP, Katz  DS, Lalani  T,  et al.  ACR appropriateness criteria right lower quadrant pain—suspected appendicitis.  Ultrasound Q. 2015;31(2):85-91. doi:10.1097/RUQ.0000000000000118PubMedGoogle ScholarCrossref
    39.
    Drake  FT, Flum  DR.  Improvement in the diagnosis of appendicitis.  Adv Surg. 2013;47:299-328. doi:10.1016/j.yasu.2013.03.003PubMedGoogle ScholarCrossref
    40.
    Shogilev  DJ, Duus  N, Odom  SR, Shapiro  NI.  Diagnosing appendicitis: evidence-based review of the diagnostic approach in 2014.  West J Emerg Med. 2014;15(7):859-871. doi:10.5811/westjem.2014.9.21568PubMedGoogle ScholarCrossref
    41.
    van Randen  A, Lameris  W, Luitse  JS,  et al; OPTIMA Study Group.  The role of plain radiographs in patients with acute abdominal pain at the ED.  Am J Emerg Med. 2011;29(6):582-589. doi:10.1016/j.ajem.2009.12.020PubMedGoogle ScholarCrossref
    42.
    Ceresoli  M, Zucchi  A, Allievi  N,  et al.  Acute appendicitis: epidemiology, treatment and outcomes—analysis of 16544 consecutive cases.  World J Gastrointest Surg. 2016;8(10):693-699. doi:10.4240/wjgs.v8.i10.693PubMedGoogle ScholarCrossref
    43.
    Sartelli  M, Baiocchi  GL, Di Saverio  S,  et al.  Prospective observational study on acute appendicitis worldwide (POSAW).  World J Emerg Surg. 2018;13:19. doi:10.1186/s13017-018-0179-0PubMedGoogle ScholarCrossref
    44.
    Sceats  LA, Trickey  AW, Morris  AM, Kin  C, Staudenmayer  KL.  Nonoperative management of uncomplicated appendicitis among privately insured patients.  JAMA Surg. 2019;154(2):141-149. doi:10.1001/jamasurg.2018.4282PubMedGoogle ScholarCrossref
    45.
    Okafor  N, Payne  VL, Chathampally  Y, Miller  S, Doshi  P, Singh  H.  Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine.  Emerg Med J. 2016;33(4):245-252. doi:10.1136/emermed-2014-204604PubMedGoogle ScholarCrossref
    46.
    Singh  H, Khanna  A, Spitzmueller  C, Meyer  AND.  Recommendations for using the revised safer Dx instrument to help measure and improve diagnostic safety.  Diagnosis (Berl). 2019;6(4):315-323. doi:10.1515/dx-2019-0012PubMedGoogle ScholarCrossref
    47.
    Kim  DW, Yoon  HM, Lee  JY,  et al.  Diagnostic performance of CT for pediatric patients with suspected appendicitis in various clinical settings: a systematic review and meta-analysis.  Emerg Radiol. 2018;25(6):627-637. doi:10.1007/s10140-018-1624-9PubMedGoogle ScholarCrossref
    48.
    Rud  B, Vejborg  TS, Rappeport  ED, Reitsma  JB, Wille-Jørgensen  P.  Computed tomography for diagnosis of acute appendicitis in adults.  Cochrane Database Syst Rev. 2019;2019(11). doi:10.1002/14651858.CD009977.pub2PubMedGoogle Scholar
    49.
    Murphy  DR, Meyer  AN, Sittig  DF, Meeks  DW, Thomas  EJ, Singh  H.  Application of electronic trigger tools to identify targets for improving diagnostic safety.  BMJ Qual Saf. 2019;28(2):151-159. doi:10.1136/bmjqs-2018-008086PubMedGoogle ScholarCrossref
    ×