[Skip to Navigation]
Sign In
Figure 1.  Pediatric Appendicitis Score and Pediatric Appendicitis Risk Calculator Score Among Children With and Without a Delay in Diagnosis From the Index (First) Encounter
Pediatric Appendicitis Score and Pediatric Appendicitis Risk Calculator Score Among Children With and Without a Delay in Diagnosis From the Index (First) Encounter

Variability in scores is depicted using standard box plots with the boxes depicting the median, 25th, and 75th percentile scale scores, the standard whisker lengths, and dots representing outliers. Those with a delay were further stratified by the likelihood of a missed opportunity to improve diagnosis. Scores are presented depending on the handling of missing peripheral white blood cell (WBC) counts. MOID indicates missed opportunity to improve diagnosis.

Figure 2.  Forest Plot of Sign or Symptom Being Present at Second vs First Encounter
Forest Plot of Sign or Symptom Being Present at Second vs First Encounter

Progression of symptoms and signs of appendicitis is shown as the relative likelihood of being present at the delay vs index encounter among children experiencing a delayed diagnosis. OR indicates odds ratio; RLQ, right lower quadrant.

Table 1.  Demographic Features of Cases and Controls
Demographic Features of Cases and Controls
Table 2.  Models Revealing Clinical Features Associated With Delayed Diagnosis of Appendicitis Compared With Those Without Delay
Models Revealing Clinical Features Associated With Delayed Diagnosis of Appendicitis Compared With Those Without Delay
Table 3.  Outcomes of Delayed Diagnosis of Appendicitis
Outcomes of Delayed Diagnosis of Appendicitis
1.
Addiss  DG, Shaffer  N, Fowler  BS, Tauxe  RV.  The epidemiology of appendicitis and appendectomy in the United States.   Am J Epidemiol. 1990;132(5):910-925. doi:10.1093/oxfordjournals.aje.a115734 PubMedGoogle ScholarCrossref
2.
Serres  SK, Cameron  DB, Glass  CC,  et al.  Time to appendectomy and risk of complicated appendicitis and adverse outcomes in children.   JAMA Pediatr. 2017;171(8):740-746. doi:10.1001/jamapediatrics.2017.0885 PubMedGoogle ScholarCrossref
3.
Baxter  KJ, Nguyen  HTMH, Wulkan  ML, Raval  MV.  Association of health care utilization with rates of perforated appendicitis in children 18 years or younger.   JAMA Surg. 2018;153(6):544-550. doi:10.1001/jamasurg.2017.5316 PubMedGoogle ScholarCrossref
4.
Papandria  D, Goldstein  SD, Rhee  D,  et al.  Risk of perforation increases with delay in recognition and surgery for acute appendicitis.   J Surg Res. 2013;184(2):723-729. doi:10.1016/j.jss.2012.12.008 PubMedGoogle ScholarCrossref
5.
Anandalwar  SP, Cameron  DB, Graham  DA,  et al.  Association of intraoperative findings with outcomes and resource use in children with complicated appendicitis.   JAMA Surg. 2018;153(11):1021-1027. doi:10.1001/jamasurg.2018.2085 PubMedGoogle ScholarCrossref
6.
Michelson  KA, Dart  AH, Bachur  RG, Mahajan  P, Finkelstein  JA.  Measuring complications of serious pediatric emergencies using ICD-10.   Health Serv Res. 2021;56(2):225-234. doi:10.1111/1475-6773.13615PubMedGoogle ScholarCrossref
7.
Sawin  RS. Appendix and Meckel’s diverticulum. In: Oldham KT, Colombani PM, Foglia RP, Skinner MA, eds. Principles and Practice of Pediatric Surgery. Lippincott Williams & Wilkins; 2005:1271-1282.
8.
Young  KA, Neuhaus  NM, Fluck  M,  et al.  Outcomes of complicated appendicitis: is conservative management as smooth as it seems?   Am J Surg. 2018;215(4):586-592. doi:10.1016/j.amjsurg.2017.10.032 PubMedGoogle ScholarCrossref
9.
Glerum  KM, Selbst  SM, Parikh  PD, Zonfrillo  MR. Pediatric malpractice claims in the emergency department and urgent care settings from 2001 to 2015. Pediatr Emerg Care. 2021;37(7):e376-e379. doi:10.1097/PEC.0000000000001602PubMed
10.
Rusnak  RA, Borer  JM, Fastow  JS.  Misdiagnosis of acute appendicitis: common features discovered in cases after litigation.   Am J Emerg Med. 1994;12(4):397-402. doi:10.1016/0735-6757(94)90047-7 PubMedGoogle ScholarCrossref
11.
Mahajan  P, Basu  T, Pai  CW,  et al.  Factors associated with potentially missed diagnosis of appendicitis in the emergency department.   JAMA Netw Open. 2020;3(3):e200612. doi:10.1001/jamanetworkopen.2020.0612 PubMedGoogle Scholar
12.
Freedman  SB, Rodean  J, Hall  M,  et al.  Delayed diagnoses in children with constipation: multicenter retrospective cohort study.   J Pediatr. 2017;186:87-94.e16. doi:10.1016/j.jpeds.2017.03.061 PubMedGoogle ScholarCrossref
13.
Freedman  SB, Thull-Freedman  J, Manson  D,  et al.  Pediatric abdominal radiograph use, constipation, and significant misdiagnoses.   J Pediatr. 2014;164(1):83-88.e2. doi:10.1016/j.jpeds.2013.08.074 PubMedGoogle ScholarCrossref
14.
Naiditch  JA, Lautz  TB, Daley  S, Pierce  MC, Reynolds  M.  The implications of missed opportunities to diagnose appendicitis in children.   Acad Emerg Med. 2013;20(6):592-596. doi:10.1111/acem.12144 PubMedGoogle ScholarCrossref
15.
Naoum  JJ, Mileski  WJ, Daller  JA,  et al.  The use of abdominal computed tomography scan decreases the frequency of misdiagnosis in cases of suspected appendicitis.   Am J Surg. 2002;184(6):587-589. doi:10.1016/S0002-9610(02)01086-3 PubMedGoogle ScholarCrossref
16.
Anderson  KT, Bartz-Kurycki  M, Austin  MT,  et al.  Approaching zero: implications of a computed tomography reduction program for pediatric appendicitis evaluation.   J Pediatr Surg. 2017;52(12):1909-1915. doi:10.1016/j.jpedsurg.2017.08.050 PubMedGoogle ScholarCrossref
17.
Bachur  RG, Callahan  MJ, Monuteaux  MC, Rangel  SJ.  Integration of ultrasound findings and a clinical score in the diagnostic evaluation of pediatric appendicitis.   J Pediatr. 2015;166(5):1134-1139. doi:10.1016/j.jpeds.2015.01.034 PubMedGoogle ScholarCrossref
18.
Staab  S, Black  T, Leonard  J, Bruny  J, Bajaj  L, Grubenhoff  JA.  Diagnostic accuracy of suspected appendicitis: a comparative analysis of misdiagnosed appendicitis in children.   Pediatr Emerg Care. 2020. doi:10.1097/PEC.0000000000002323PubMedGoogle Scholar
19.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.   J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008 PubMedGoogle ScholarCrossref
20.
Michelson  KA, Williams  DN, Dart  AH,  et al.  Development of a rubric for assessing delayed diagnosis of appendicitis, diabetic ketoacidosis and sepsis.   Diagnosis (Berl). 2020;8(2):219-225. doi:10.1515/dx-2020-0035 PubMedGoogle ScholarCrossref
21.
Michelson  KA, Buchhalter  LC, Bachur  RG, Mahajan  P, Monuteaux  MC, Finkelstein  JA.  Accuracy of automated identification of delayed diagnosis of pediatric appendicitis and sepsis in the ED.   Emerg Med J. 2019;36(12):736-740. doi:10.1136/emermed-2019-208841 PubMedGoogle Scholar
22.
Goldman  RD, Carter  S, Stephens  D, Antoon  R, Mounstephen  W, Langer  JC.  Prospective validation of the pediatric appendicitis score.   J Pediatr. 2008;153(2):278-282. doi:10.1016/j.jpeds.2008.01.033 PubMedGoogle ScholarCrossref
23.
Kharbanda  AB, Vazquez-Benitez  G, Ballard  DW,  et al.  Development and validation of a novel pediatric appendicitis risk calculator (pARC).   Pediatrics. 2018;141(4):e20172699. doi:10.1542/peds.2017-2699 PubMedGoogle Scholar
24.
Singh  H, Sittig  DF.  Advancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework.   BMJ Qual Saf. 2015;24(2):103-110. doi:10.1136/bmjqs-2014-003675 PubMedGoogle ScholarCrossref
25.
Singh  H, Khanna  A, Spitzmueller  C, Meyer  AND.  Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety.   Diagnosis (Berl). 2019;6(4):315-323. doi:10.1515/dx-2019-0012 PubMedGoogle ScholarCrossref
26.
Dribin  TE, Michelson  KA, Monuteaux  MC,  et al.  Identification of children with anaphylaxis at low risk of receiving acute inpatient therapies.   PLoS One. 2019;14(2):e0211949. doi:10.1371/journal.pone.0211949 PubMedGoogle Scholar
27.
Feudtner  C, Feinstein  JA, Zhong  W, Hall  M, Dai  D.  Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation.   BMC Pediatr. 2014;14(1):199. doi:10.1186/1471-2431-14-199 PubMedGoogle ScholarCrossref
28.
Jennings  R, Guo  H, Goldin  A, Wright  DR.  Cost-effectiveness of imaging protocols for suspected appendicitis.   Pediatrics. 2020;145(2):e20191352. doi:10.1542/peds.2019-1352 PubMedGoogle Scholar
29.
Papa  FJ, Stone  RC, Aldrich  DG.  Further evidence of the relationship between case typicality and diagnostic performance: implications for medical education.   Acad Med. 1996;71(1)(suppl):S10-S12. doi:10.1097/00001888-199601000-00028 PubMedGoogle ScholarCrossref
30.
Sittig  DF, Singh  H.  Electronic health records and national patient-safety goals.   N Engl J Med. 2012;367(19):1854-1860. doi:10.1056/NEJMsb1205420 PubMedGoogle ScholarCrossref
31.
Goyal  MK, Kuppermann  N, Cleary  SD, Teach  SJ, Chamberlain  JM.  Racial disparities in pain management of children with appendicitis in emergency departments.   JAMA Pediatr. 2015;169(11):996-1002. doi:10.1001/jamapediatrics.2015.1915 PubMedGoogle ScholarCrossref
32.
Ponsky  TA, Huang  ZJ, Kittle  K,  et al.  Hospital- and patient-level characteristics and the risk of appendiceal rupture and negative appendectomy in children.   JAMA. 2004;292(16):1977-1982. doi:10.1001/jama.292.16.1977 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Family History in Suspected Appendicitis
    Harald Aanning, MD | Retired general surgeon
    In my experience (unpublished), a positive family history was very helpful in diagnosing suspected appendicitis - especially in those pediatric patients presenting with minimal, unusual, or doubtful clinical features.
    CONFLICT OF INTEREST: None Reported
    Original Investigation
    Pediatrics
    August 31, 2021

    Clinical Features and Preventability of Delayed Diagnosis of Pediatric Appendicitis

    Author Affiliations
    • 1Division of Emergency Medicine, Boston Children’s Hospital, Boston, Massachusetts
    • 2Division of Pediatric Emergency Medicine, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    • 3Section of Pediatric Emergency Medicine, University of Colorado School of Medicine, Aurora
    • 4Children's Hospital Colorado, Aurora
    • 5Department of Pediatrics, Baylor College of Medicine, Houston, Texas
    • 6Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Los Angeles, California
    • 7Keck School of Medicine of the University of Southern California, Los Angeles
    • 8Division of General Pediatrics, Boston Children’s Hospital, Boston, Massachusetts
    JAMA Netw Open. 2021;4(8):e2122248. doi:10.1001/jamanetworkopen.2021.22248
    Key Points

    Question  Which clinical features are associated with delayed diagnosis of pediatric appendicitis, and to what extent are delays preventable?

    Findings  In this case-control study of 748 children, those who experienced delayed diagnosis had less clinically apparent symptoms; among children for whom imaging would have been cost-effective, 22% to 61% received it.

    Meaning  These findings suggest that undertesting for appendicitis may account for a considerable fraction of cases of delayed diagnosis of appendicitis.

    Abstract

    Importance  Delayed diagnosis of appendicitis is associated with worse outcomes than timely diagnosis, but clinical features associated with diagnostic delay are uncertain, and the extent to which delays are preventable is unclear.

    Objective  To determine clinical features associated with delayed diagnosis of pediatric appendicitis, assess the frequency of preventable delay, and compare delay outcomes.

    Design, Setting, and Participants  This case-control study included 748 children treated at 5 pediatric emergency departments in the US between January 1, 2010, and December 31, 2019. Participants were younger than 21 years and had a diagnosis of appendicitis.

    Exposures  Individual features of appendicitis and pretest likelihood of appendicitis were measured by the Pediatric Appendicitis Risk Calculator (pARC).

    Main Outcomes and Measures  Case patients had a delayed diagnosis of appendicitis, defined as 2 emergency department visits leading to diagnosis and a case review showing the patient likely had appendicitis at the first visit. Control patients had a single emergency department visit yielding a diagnosis. Clinical features and pARC scores were compared by case-control status. Preventability of delay was assessed as unlikely, possible, or likely. The proportion of children with indicated imaging based on an evidence-based cost-effectiveness threshold was determined. Outcomes of delayed diagnosis were compared by case-control status, including hospital length of stay, perforation, and multiple surgical procedures.

    Results  A total of 748 children (mean [SD] age, 10.2 [4.3] years; 392 boys [52.4%]; 427 White children [57.1%]) were included in the study; 471 (63.0%) had a delayed diagnosis of appendicitis, and 277 (37.0%) had no delay in diagnosis. Children with a delayed diagnosis were less likely to have pain with walking (adjusted odds ratio [aOR], 0.16; 95% CI, 0.10-0.25), maximal pain in the right lower quadrant (aOR, 0.12; 95% CI, 0.07-0.19), and abdominal guarding (aOR, 0.33; 95% CI, 0.21-0.51), and were more likely to have a complex chronic condition (aOR, 2.34; 95% CI, 1.05-5.23). The pretest likelihood of appendicitis was 39% to 52% lower in children with a delayed vs timely diagnosis. Among children with a delayed diagnosis, 109 cases (23.1%) were likely to be preventable, and 247 (52.4%) were possibly preventable. Indicated imaging was performed in 104 (22.0%) to 289 (61.3%) children with delayed diagnosis, depending on the imputation method for missing data on white blood cell count. Patients with delayed diagnosis had longer hospital length of stay (mean difference between the groups, 2.8 days; 95% CI, 2.3-3.4 days) and higher perforation rates (OR, 7.8; 95% CI, 5.5-11.3) and were more likely to undergo 2 or more surgical procedures (OR, 8.0; 95% CI, 2.0-70.4).

    Conclusions and Relevance  In this case-control study, delayed appendicitis was associated with initially milder symptoms but worse outcomes. These findings suggest that a majority of delayed diagnoses were at least possibly preventable and that many of these patients did not undergo indicated imaging, suggesting an opportunity to prevent delayed diagnosis of appendicitis in some children.

    Introduction

    Appendicitis is the most common surgical emergency in children.1 Significant delays in diagnosis are associated with worse outcomes, particularly perforated appendicitis.2-4 Perforated appendicitis is associated with substantial morbidity, including abdominal abscess, small bowel obstruction, sepsis, bowel resection, longer hospital stays, increased encounters with health care, and higher cost.5-8 Delayed diagnosis of appendicitis is also a top cause of malpractice claims in pediatrics.9

    Delayed diagnosis of appendicitis in the emergency department (ED) is frequently associated with an incorrect diagnosis during an earlier visit to care.10-12 Misdiagnosed conditions include constipation, gastroenteritis, or symptom-based diagnoses, such as vomiting or diarrhea.11,13,14 Imaging may decrease the likelihood of a misdiagnosis, but overuse of computed tomography (CT) increases radiation-related risks, and use of ultrasonography in patients with a low probability of appendicitis has a high rate of both false positives and inconclusive results.15-17 Although duration of symptoms, age, and sex are known to be associated with the likelihood of a delayed diagnosis, the specific symptom and care patterns associated with delays are unclear.11,13,18

    Understanding which patients are at risk of delayed diagnosis would aid efforts to prevent such delays. Therefore, we performed a case-control study comparing children with delayed diagnosis with children with timely diagnosis of appendicitis to assess clinical features that are associated with delay and elements of care that are associated with potentially preventable delays.

    Methods
    Design and Data Sources

    We conducted a case-control study of children and young adults younger than 21 years with appendicitis. Potential cases involved a delayed diagnosis of appendicitis, defined as 2 ED visits within 7 days leading to diagnosis and a case review showing the patient likely had appendicitis at the first visit. Cases were drawn from a cohort of patients treated for appendicitis who visited 1 of 5 US pediatric EDs from January 1, 2010, to December 31, 2019, identified in the Pediatric Health Information System database. The Pediatric Health Information System contains information about patient demographic details, medications, diagnoses, radiology, and procedures for freestanding pediatric hospitals in the US. These records are linkable to local electronic medical records, which allows for detailed case review.

    Control patients were children with a timely diagnosis of appendicitis at their first ED visit and were identified from a prospective study of children with appendicitis that included children from a single pediatric ED between February 26, 2010, and April 20, 2013.17 Controls were identified at the time of presentation, and treating clinicians completed a demographic and symptom survey. The control ED was 1 of the 5 sites used to identify cases. Unlike most case-control studies, we anticipated that we would enroll fewer controls than cases because of the sample size needs of the original case and control studies.

    Appendicitis was defined as either undergoing appendectomy or, if managed nonoperatively, as imaging evidence of appendicitis followed by improvement with antibiotic therapy and/or procedures such as abdominal abscess drainage. Case patients were excluded if there were insufficient records, if the patient left without being seen on the first (ie, index) encounter, if the patient was transferred on the index encounter, or if the case was judged not to have been delayed. Control patients were excluded if there were insufficient records to perform a case review. Institutional review boards at each site approved this study with a waiver of informed consent because of minimal risk to participants. We adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting case-control studies.19

    Outcomes

    The main outcome was delayed diagnosis of appendicitis. Control patients (those without a delayed diagnosis) had a single ED encounter leading to an appendicitis diagnosis. All participants with a potentially delayed diagnosis underwent review of documentation from both the index and the second encounter (when the diagnosis was made, ie, the delay encounter). Delayed diagnosis was defined by reviewer determination that appendicitis was probably or near-definitely present but not diagnosed during the index encounter (definitions in eTable 1 in the Supplement). This definition was developed through a multicenter, multispecialty expert consensus process.20 Case review and data abstraction were performed by board-certified pediatric emergency medicine attending physicians (J.A.G., S.D.R., P.P.C., A.T.C., and K.A.M.). One reviewer from each site was trained through supervised review of 80 case vignettes. Reviewers (J.A.G., S.D.R., P.P.C., A.T.C., and K.A.M.) were coauthors of this study but were unaware of the study hypothesis. We did not perform interrater reliability assessments, as our pilot study showed high interrater reliability in the assessment of delayed diagnosis.21

    A secondary outcome among delayed diagnosis cases was the change in clinical discernibility of appendicitis between the index and delayed encounters. Discernibility was measured using 2 validated clinical scoring systems: the Pediatric Appendicitis Score (PAS) and the Pediatric Appendicitis Risk Calculator (pARC).22,23 The PAS score is measured on a scale of 0 to 10, with higher scores indicating a greater likelihood of appendicitis. The pARC score estimates the likelihood of appendicitis on a scale of 0% to 100%. Both the PAS and pARC scores rely on peripheral white blood cell (WBC) and neutrophil counts. However, the WBC count was not obtained in many cases during the index encounter. Therefore, in evaluations involving the PAS and pARC, we determined the scores using several approaches: (1) analyzed only cases with laboratory values present; (2) imputed missing WBC values from all other clinical variables; and (3) imputed missing WBC values as 5000, 10 000, 15 000, or 20 000 cells per microliter. A complete description appears in the eMethods in the Supplement.

    Cases were also reviewed for the secondary outcome of preventable delayed diagnosis, defined as a missed opportunity to improve diagnosis (MOID).24 To grade the likelihood of a MOID, we used the validated Modified Safer Dx checklist.25 The Safer Dx measures the likelihood of a MOID using 13 items relating to individual elements of an encounter that could lead to MOIDs. The first 12 items include the history, physical examination, data gathering, presence of “red flags,” incomplete caregiver information, completeness of evaluation, diagnostic reasoning, test interpretation, follow-up of results, differential formation, disease evolution, and how closely the index presentation reflected typical features of the disease. The final item, the MOID score, graded the overall likelihood of a MOID. All items are measured on a scale of 1 to 7. We further classified a score of 1 to 2 as an unlikely MOID, 3 to 5 as a possible MOID, and 6 to 7 as a likely MOID.

    Variables

    Potential clinical features associated with delayed diagnosis were chosen a priori based on clinical features related to the ultimate diagnosis of appendicitis from previous literature.22,23 These features included age, sex, duration of abdominal pain, anorexia, nausea or vomiting, migration of pain to the right lower quadrant, history of fever, pain with walking, maximal pain in the right lower quadrant, and abdominal guarding. If the symptom was not mentioned in the record, it was assessed as being absent, as is typical for this type of study.26 We also assessed the presence of a complex chronic condition based on same- and previous-encounter diagnosis codes.27

    We hypothesized that several care features would be associated (positively or negatively) with delayed diagnosis, including obtaining ultrasonographic imaging, obtaining a CT scan, use of a nonopioid analgesic (acetaminophen, ibuprofen, ketorolac, or naproxen), use of an opioid analgesic (fentanyl, hydromorphone, morphine, or oxycodone), or use of ondansetron. Finally, because we planned to evaluate disparities between timely and delayed diagnosis, we collected race, ethnicity, and insurance information as recorded in the source database.

    Statistical Anlysis
    Primary Analysis

    For the primary analysis, we summarized the proportion of delayed cases that were MOIDs. Among cases, we evaluated which elements of care were associated with preventable delay by performing Spearman correlations between each Safer Dx item and the overall MOID score. Correlations were considered strong with a ρ greater than 0.7 and moderate at a ρ of 0.5 to 0.7. Previous research has suggested that imaging is cost-effective if the probability of appendicitis is at least 16%.28 We evaluated the proportion of children who underwent indicated imaging testing (ultrasonography or CT) with an index encounter pARC appendicitis probability of at least 16%.

    Secondary Analysis

    For the secondary analysis, we first compared the characteristics of case and control patients using χ2 tests. To assess the likelihood of selection bias affecting analysis of differences between cases and controls, we compared the characteristics of cases from the single control site to cases from other study sites. An absence of observable differences would suggest a low risk of selection bias.

    Next, we developed 2 logistic regression models to examine features associated with delayed diagnosis. In model 1, we included all clinical features. In model 2, we evaluated how features of care are associated with delayed diagnosis by including all care features after adjusting for all clinical features.

    We hypothesized that cases of delayed diagnosis would be less clinically discernible during the index encounter than controls, so we plotted and compared PAS and pARC scores between cases and controls using Wilcoxon rank sum tests. We then evaluated how clinical features of appendicitis progress in delayed diagnosis cases. Hypothesizing that cases with a delay would become more discernible as time passed, we determined the difference in median PAS and pARC scores between index and delay encounters and applied a Wilcoxon signed rank test. Finally, we quantified the likelihood that a clinical feature would develop between the 2 ED encounters using conditional logistic regression using encounter (delay vs index) as the outcome and clinical features as covariates, and conditioned on the patient. This analysis resulted in estimates of the odds ratio (OR) for a feature being present at the delay encounter compared with the index encounter in a particular patient.

    We planned to evaluate disparities in delayed diagnosis by race, ethnicity, and insurance type; however, we found that the distribution of these characteristics varied too greatly between the source populations of case and control patients. Therefore, we were unable to perform this analysis.

    Length of hospitalization, presence of perforated appendicitis, and whether patients underwent appendectomy were determined for cases by likelihood of MOID and for controls. Appendectomy status was determined by having a procedure code for the procedure (International Classification of Diseases, Ninth Revision [ICD-9] 47.01 or 47.09; International Statistical Classification of Diseases and Related Health Problems, Tenth Revision [ICD-10] 0DTJ4ZZ or 0DTJ0ZZ). We also determined the proportion of patients with multiple abdominal surgical procedures, defined as an abdominal surgical procedure on more than 1 hospital day. These procedures included appendectomy as defined above and abdominal abscess drainage (ICD-9 47.2 or 54.91; ICD-10 stems 0D9J, 0D9W, or 0W9F-0W9J). Outcomes were reported using the mean and 95% CI or proportion and 95% binomial exact CI. Differences between groups were reported using a 95% CI of the difference in means or the conditional maximum likelihood estimate OR. Secondary analyses did not adjust for multiple comparisons and thus were considered exploratory. Statistical significance was defined as P < .05, and all P values were 2-sided. Analyses were performed in R, version 4.0.2 (R Foundation).

    Results

    There were 801 children with delayed diagnosis and 294 children with timely diagnosis of appendicitis. Among those with a delayed diagnosis, 19 were excluded for insufficient records, 32 for leaving without being seen from the index encounter, 81 for transfer from the index encounter, and 198 for not having a delayed diagnosis upon case review. In addition, 17 children with a timely diagnosis were excluded for insufficient records. After exclusions, a total of 748 children (mean [SD] age, 10.2 [4.3] years; 392 boys [52.4%] and 356 girls [47.6%]; 427 White children [57.1%]) were included in the study; 471 (63.0%) had a delayed diagnosis of appendicitis, and 277 (37.0%) had no delay in diagnosis. Of the total cohort, 55 children (7.4%) had a complex chronic condition (Table 1).

    There were few differences between the delayed and timely diagnosis groups in terms of age, sex, complex chronic condition, revisit interval, or index encounter symptoms except for anorexia and migratory pain (eTable 2 in the Supplement). Race, ethnicity, and insurance type each differed significantly. Thus, we regarded selection bias to be limited between cases and controls except with regard to sociodemographic factors. The 5 most common diagnoses given at the index encounter of delayed diagnosis cases were gastroenteritis (132 of 471 [28.0%]), abdominal pain (81 of 471 [17.2%]), constipation (79 of 471 [16.8%]), urinary tract infection (27 of 471 [5.7%]), and viral syndrome (24 of 471 [5.1%]).

    Primary Analysis

    Among cases of delayed diagnosis of appendicitis, 109 (23.1%) were unlikely MOIDs, 247 (52.4%) were possible MOIDs, and 115 (24.4%) were likely MOIDs; in other words, there were possible or probable opportunities to improve diagnosis for 76.8% of patients with a delayed diagnosis. Three of 12 Safer Dx elements were correlated with the overall likelihood of a MOID (eTable 3 in the Supplement): history should have prompted additional diagnostic evaluation (Spearman ρ = 0.77), diagnostic reasoning was not appropriate (ρ = 0.77), and clinical presentation was mostly typical of the ultimate diagnosis (ρ = 0.76). One Safer Dx element was correlated with MOID: alarm symptoms or red flags were not acted upon (ρ = 0.65). The number of patients who received indicated imaging was between 104 of 471 (22.0%; 95% CI, 17.9%-26.5%) and 289 of 471 (61.3%; 95% CI, 49.4%-72.4%) depending on the WBC imputation method.

    Secondary Analysis

    In a model assessing clinical features at the index encounter, patients with delayed diagnosis of appendicitis were more likely to have had abdominal pain for 48 to 96 hours (adjusted OR [aOR], 2.40; 95% CI, 1.16-4.97) compared with less than 24 hours, no pain with walking (aOR, 0.16; 95% CI, 0.10-0.25), abdominal pain maximally outside the right lower quadrant (aOR, 0.95; 95% CI, 0.57-1.58), no abdominal guarding (aOR, 0.33; 95% CI, 0.21-0.51), and a complex chronic condition (aOR, 2.34; 95% CI, 1.05-5.23) (Table 2). Adding care features to the model, children who received nonsteroidal anti-inflammatory drugs (aOR, 3.78; 95% CI, 2.10-6.80) or ondansetron (aOR, 1.99; 95% CI, 1.10-3.61) were more likely to have a delay in diagnosis, whereas those who received opioids (aOR, 0.26; 95% CI, 0.14-0.49), a CT scan (aOR, 0.10; 95% CI, 0.05-0.22), or ultrasonography (aOR, 0.13; 95% CI, 0.08-0.24) were less likely. In this model, the presence of a complex chronic condition was no longer associated with delay.

    The index encounter PAS and pARC scores were lower in children who experienced a delayed diagnosis of appendicitis compared with those with a timely diagnosis (Figure 1). White blood cell values were missing from 53 of 277 index encounters (19.1%) among children with timely diagnosis and 358 of 471 index encounters (76.0%) among children with delayed diagnosis. Regardless of WBC imputation method, PAS and pARC scores were significantly higher from the index encounters of children with timely rather than delayed diagnosis of appendicitis: the median PAS scores differed by 2 to 4 points depending on imputation method, and the median pARC scores differed by 39 to 52 percentage points (P < .001 for each comparison of cases to controls).

    Among children with a delayed diagnosis of appendicitis, the increase in median PAS score between index and delay encounters ranged from 1 to 4 points depending on WBC imputation method (P < .001 for each comparison). The increase in median pARC scores ranged from 12 to 43 percentage points (P < .001 for each comparison). Each clinical feature was more likely to be present at the delay encounter compared with the index encounter except for nausea or vomiting (Figure 2). The symptom most likely to have developed between encounters was migration of pain to the right lower quadrant (aOR, 17.2; 95% CI, 5.9-49.7).

    Outcomes of appendicitis were generally worse among children with delayed vs timely diagnosis (Table 3). Children with delayed diagnosis had longer hospital length of stay (mean difference, 2.8 days; 95% CI, 2.3-3.4 days), higher perforation rates (OR, 7.8; 95% CI, 5.5-11.3), a decreased likelihood of appendectomy at the time of ultimate diagnosis (OR, 0.33; 95% CI, 0.19-0.55), and a higher likelihood of undergoing 2 or more abdominal surgical procedures (OR, 8.0; 95% CI, 2.0-70.4) compared with children with timely diagnosis.

    Discussion

    In this case-control study, children with delayed diagnosis at their index ED encounter were more likely to have a longer duration of abdominal pain, no pain with walking, abdominal pain outside the right lower quadrant, no abdominal guarding, and a complex chronic condition compared with children with a timely diagnosis of appendicitis. Based on PAS or pARC scores, children with delayed diagnoses were less clinically discernible. They were more likely to receive nonsteroidal anti-inflammatory drugs and antiemetic therapy and less likely to receive opioids or undergo imaging. In 76.8% of patients with a delayed diagnosis, there were possibly or probably opportunities to improve diagnosis. Children with delayed diagnosis had worse outcomes. When the diagnosis was made, hospitalizations were longer, appendectomy was less likely, and appendiceal perforation and multiple abdominal surgical procedures were more likely. Taken together, our findings suggest that delayed diagnosis was more likely in patients with less classic presentations, that such patients were less likely to undergo imaging or receive appendicitis-directed treatments, and that a substantial fraction of delayed diagnoses were preventable despite being less clinically apparent.

    Among children with delayed diagnosis of appendicitis, progression of disease occurred between the index and delay ED encounters. Nearly every classic symptom or sign of appendicitis was more likely to be present during the later visit. The PAS and pARC scores, which measure the likelihood of appendicitis, were higher during the delay encounter than during the index encounter. Abdominal guarding and migration of pain to the right lower quadrant were especially likely to develop after the index encounter: each feature was more than 10 times as likely to be present later. The presence of more classic symptoms during a second ED visit makes the diagnosis more discernible to clinicians. Conversely, the absence of classic appendicitis symptoms puts a patient at risk of a delayed diagnosis.

    Nevertheless, in this study, most delayed diagnoses were probably or possibly MOIDs, indicating that there were potential opportunities to prevent delay. The 3 elements of the diagnostic process most associated with a MOID were failures in diagnostic reasoning that indicated that the patient’s history should have prompted a deeper evaluation. Most children with delayed diagnosis had neither a WBC count nor an imaging study performed at the index encounter. Children with a characteristic history or physical examination should undergo these tests in an effort to prevent delayed diagnosis. In this study, only 22.0% to 61.3% of children received indicated imaging based on a pARC score above the threshold at which imaging is cost-effective.28 This finding suggests an opportunity to improve detection.

    Combined with previous work, our results suggest several ways to improve the detection of appendicitis. First, children with an intermediate or high probability of appendicitis should undergo peripheral WBC testing and ultrasonography to evaluate appendicitis.23,28 In this study, the WBC was obtained in only 24% of children with a delayed diagnosis of appendicitis. A visualized normal appendix on ultrasonographic examination in a child with a high clinical probability of appendicitis should typically prompt further diagnostic evaluation (such as cross-sectional imaging), as these ultrasonographic tests have a 19% false-negative rate.17 Second, although children with a low clinical probability of appendicitis should not undergo immediate imaging, longer periods of observation or peripheral WBC measurement could contribute to a correct diagnosis. Evolution of symptoms and additional diagnostic tests that increase case typicality may increase diagnostic accuracy.29 However, defining which clinical features should prompt further testing remains challenging, as patients with appendicitis may have only vague features early in their course.18 Nonetheless, our study suggests that evaluation of clinical decision support systems based on the pARC score may be a promising strategy to improve appendicitis diagnosis because many children with delayed diagnoses had pARC scores that should have prompted additional diagnostic work-up.30

    Limitations

    Our study had several limitations. First, we included only pediatric hospitals, which differ from general hospitals in terms of resources (such as 24-hour ultrasonography and pediatric surgery availability), clinician background, and patient characteristics. Second, cases and controls were not drawn from the same population, which could lead to selection bias. However, this was unlikely to affect our results because the control institution’s case patient characteristics did not differ greatly from the other case institutions’ patients except in sociodemographic domains including race, ethnicity, and insurance. Unfortunately, differences between the source populations of cases and controls meant we could not evaluate the contribution of sociodemographic factors in delayed diagnosis, such as the racial disparities known to exist in appendicitis care.11,31,32 In addition, we could not evaluate whether patients had imaging performed at a transferring hospital before arrival to the index encounter. However, the overall very low rate of imaging in patients with delayed diagnosis compared with those with a timely diagnosis suggests that is unlikely to affect our findings.

    Conclusions

    In this case-control study, children with delayed diagnosis of appendicitis initially had milder symptoms and signs, were likely to have progression of symptoms before diagnosis, and had worse outcomes than those with timely diagnosis. Our findings suggest that cases of delayed diagnosis of appendicitis in children may represent an early missed opportunity to make a diagnosis. Decision support interventions could be useful in preventing delayed diagnosis without leading to overtesting because many delayed diagnoses occur in cases where testing is indicated.

    Back to top
    Article Information

    Accepted for Publication: June 17, 2021.

    Published: August 31, 2021. doi:10.1001/jamanetworkopen.2021.22248

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Michelson KA et al. JAMA Network Open.

    Corresponding Author: Kenneth A. Michelson, MD, MPH, Division of Emergency Medicine, Boston Children’s Hospital, 300 Longwood Ave, BCH 3066, Boston, MA 02115 (kenneth.michelson@childrens.harvard.edu).

    Author Contributions: Dr Michelson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Michelson, Finkelstein, Bachur.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Michelson, Bachur.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Michelson.

    Obtained funding: Michelson.

    Administrative, technical, or material support: Michelson, Dart.

    Supervision: Reeves, Cruz, Chaudhari, Bachur.

    Local institutional review board submission and site principal investigator: Grubenhoff.

    Conflict of Interest Disclosures: Dr Bachur reported receiving a patent for appendicitis biomarkers. No other disclosures were reported.

    Funding/Support: This study was supported by grant K08HS026503 from the Agency for Healthcare Research and Quality (Dr Michelson) and from the Boston Children’s Hospital Office of Faculty Development.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    References
    1.
    Addiss  DG, Shaffer  N, Fowler  BS, Tauxe  RV.  The epidemiology of appendicitis and appendectomy in the United States.   Am J Epidemiol. 1990;132(5):910-925. doi:10.1093/oxfordjournals.aje.a115734 PubMedGoogle ScholarCrossref
    2.
    Serres  SK, Cameron  DB, Glass  CC,  et al.  Time to appendectomy and risk of complicated appendicitis and adverse outcomes in children.   JAMA Pediatr. 2017;171(8):740-746. doi:10.1001/jamapediatrics.2017.0885 PubMedGoogle ScholarCrossref
    3.
    Baxter  KJ, Nguyen  HTMH, Wulkan  ML, Raval  MV.  Association of health care utilization with rates of perforated appendicitis in children 18 years or younger.   JAMA Surg. 2018;153(6):544-550. doi:10.1001/jamasurg.2017.5316 PubMedGoogle ScholarCrossref
    4.
    Papandria  D, Goldstein  SD, Rhee  D,  et al.  Risk of perforation increases with delay in recognition and surgery for acute appendicitis.   J Surg Res. 2013;184(2):723-729. doi:10.1016/j.jss.2012.12.008 PubMedGoogle ScholarCrossref
    5.
    Anandalwar  SP, Cameron  DB, Graham  DA,  et al.  Association of intraoperative findings with outcomes and resource use in children with complicated appendicitis.   JAMA Surg. 2018;153(11):1021-1027. doi:10.1001/jamasurg.2018.2085 PubMedGoogle ScholarCrossref
    6.
    Michelson  KA, Dart  AH, Bachur  RG, Mahajan  P, Finkelstein  JA.  Measuring complications of serious pediatric emergencies using ICD-10.   Health Serv Res. 2021;56(2):225-234. doi:10.1111/1475-6773.13615PubMedGoogle ScholarCrossref
    7.
    Sawin  RS. Appendix and Meckel’s diverticulum. In: Oldham KT, Colombani PM, Foglia RP, Skinner MA, eds. Principles and Practice of Pediatric Surgery. Lippincott Williams & Wilkins; 2005:1271-1282.
    8.
    Young  KA, Neuhaus  NM, Fluck  M,  et al.  Outcomes of complicated appendicitis: is conservative management as smooth as it seems?   Am J Surg. 2018;215(4):586-592. doi:10.1016/j.amjsurg.2017.10.032 PubMedGoogle ScholarCrossref
    9.
    Glerum  KM, Selbst  SM, Parikh  PD, Zonfrillo  MR. Pediatric malpractice claims in the emergency department and urgent care settings from 2001 to 2015. Pediatr Emerg Care. 2021;37(7):e376-e379. doi:10.1097/PEC.0000000000001602PubMed
    10.
    Rusnak  RA, Borer  JM, Fastow  JS.  Misdiagnosis of acute appendicitis: common features discovered in cases after litigation.   Am J Emerg Med. 1994;12(4):397-402. doi:10.1016/0735-6757(94)90047-7 PubMedGoogle ScholarCrossref
    11.
    Mahajan  P, Basu  T, Pai  CW,  et al.  Factors associated with potentially missed diagnosis of appendicitis in the emergency department.   JAMA Netw Open. 2020;3(3):e200612. doi:10.1001/jamanetworkopen.2020.0612 PubMedGoogle Scholar
    12.
    Freedman  SB, Rodean  J, Hall  M,  et al.  Delayed diagnoses in children with constipation: multicenter retrospective cohort study.   J Pediatr. 2017;186:87-94.e16. doi:10.1016/j.jpeds.2017.03.061 PubMedGoogle ScholarCrossref
    13.
    Freedman  SB, Thull-Freedman  J, Manson  D,  et al.  Pediatric abdominal radiograph use, constipation, and significant misdiagnoses.   J Pediatr. 2014;164(1):83-88.e2. doi:10.1016/j.jpeds.2013.08.074 PubMedGoogle ScholarCrossref
    14.
    Naiditch  JA, Lautz  TB, Daley  S, Pierce  MC, Reynolds  M.  The implications of missed opportunities to diagnose appendicitis in children.   Acad Emerg Med. 2013;20(6):592-596. doi:10.1111/acem.12144 PubMedGoogle ScholarCrossref
    15.
    Naoum  JJ, Mileski  WJ, Daller  JA,  et al.  The use of abdominal computed tomography scan decreases the frequency of misdiagnosis in cases of suspected appendicitis.   Am J Surg. 2002;184(6):587-589. doi:10.1016/S0002-9610(02)01086-3 PubMedGoogle ScholarCrossref
    16.
    Anderson  KT, Bartz-Kurycki  M, Austin  MT,  et al.  Approaching zero: implications of a computed tomography reduction program for pediatric appendicitis evaluation.   J Pediatr Surg. 2017;52(12):1909-1915. doi:10.1016/j.jpedsurg.2017.08.050 PubMedGoogle ScholarCrossref
    17.
    Bachur  RG, Callahan  MJ, Monuteaux  MC, Rangel  SJ.  Integration of ultrasound findings and a clinical score in the diagnostic evaluation of pediatric appendicitis.   J Pediatr. 2015;166(5):1134-1139. doi:10.1016/j.jpeds.2015.01.034 PubMedGoogle ScholarCrossref
    18.
    Staab  S, Black  T, Leonard  J, Bruny  J, Bajaj  L, Grubenhoff  JA.  Diagnostic accuracy of suspected appendicitis: a comparative analysis of misdiagnosed appendicitis in children.   Pediatr Emerg Care. 2020. doi:10.1097/PEC.0000000000002323PubMedGoogle Scholar
    19.
    von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.   J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008 PubMedGoogle ScholarCrossref
    20.
    Michelson  KA, Williams  DN, Dart  AH,  et al.  Development of a rubric for assessing delayed diagnosis of appendicitis, diabetic ketoacidosis and sepsis.   Diagnosis (Berl). 2020;8(2):219-225. doi:10.1515/dx-2020-0035 PubMedGoogle ScholarCrossref
    21.
    Michelson  KA, Buchhalter  LC, Bachur  RG, Mahajan  P, Monuteaux  MC, Finkelstein  JA.  Accuracy of automated identification of delayed diagnosis of pediatric appendicitis and sepsis in the ED.   Emerg Med J. 2019;36(12):736-740. doi:10.1136/emermed-2019-208841 PubMedGoogle Scholar
    22.
    Goldman  RD, Carter  S, Stephens  D, Antoon  R, Mounstephen  W, Langer  JC.  Prospective validation of the pediatric appendicitis score.   J Pediatr. 2008;153(2):278-282. doi:10.1016/j.jpeds.2008.01.033 PubMedGoogle ScholarCrossref
    23.
    Kharbanda  AB, Vazquez-Benitez  G, Ballard  DW,  et al.  Development and validation of a novel pediatric appendicitis risk calculator (pARC).   Pediatrics. 2018;141(4):e20172699. doi:10.1542/peds.2017-2699 PubMedGoogle Scholar
    24.
    Singh  H, Sittig  DF.  Advancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework.   BMJ Qual Saf. 2015;24(2):103-110. doi:10.1136/bmjqs-2014-003675 PubMedGoogle ScholarCrossref
    25.
    Singh  H, Khanna  A, Spitzmueller  C, Meyer  AND.  Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety.   Diagnosis (Berl). 2019;6(4):315-323. doi:10.1515/dx-2019-0012 PubMedGoogle ScholarCrossref
    26.
    Dribin  TE, Michelson  KA, Monuteaux  MC,  et al.  Identification of children with anaphylaxis at low risk of receiving acute inpatient therapies.   PLoS One. 2019;14(2):e0211949. doi:10.1371/journal.pone.0211949 PubMedGoogle Scholar
    27.
    Feudtner  C, Feinstein  JA, Zhong  W, Hall  M, Dai  D.  Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation.   BMC Pediatr. 2014;14(1):199. doi:10.1186/1471-2431-14-199 PubMedGoogle ScholarCrossref
    28.
    Jennings  R, Guo  H, Goldin  A, Wright  DR.  Cost-effectiveness of imaging protocols for suspected appendicitis.   Pediatrics. 2020;145(2):e20191352. doi:10.1542/peds.2019-1352 PubMedGoogle Scholar
    29.
    Papa  FJ, Stone  RC, Aldrich  DG.  Further evidence of the relationship between case typicality and diagnostic performance: implications for medical education.   Acad Med. 1996;71(1)(suppl):S10-S12. doi:10.1097/00001888-199601000-00028 PubMedGoogle ScholarCrossref
    30.
    Sittig  DF, Singh  H.  Electronic health records and national patient-safety goals.   N Engl J Med. 2012;367(19):1854-1860. doi:10.1056/NEJMsb1205420 PubMedGoogle ScholarCrossref
    31.
    Goyal  MK, Kuppermann  N, Cleary  SD, Teach  SJ, Chamberlain  JM.  Racial disparities in pain management of children with appendicitis in emergency departments.   JAMA Pediatr. 2015;169(11):996-1002. doi:10.1001/jamapediatrics.2015.1915 PubMedGoogle ScholarCrossref
    32.
    Ponsky  TA, Huang  ZJ, Kittle  K,  et al.  Hospital- and patient-level characteristics and the risk of appendiceal rupture and negative appendectomy in children.   JAMA. 2004;292(16):1977-1982. doi:10.1001/jama.292.16.1977 PubMedGoogle ScholarCrossref
    ×