[Skip to Navigation]
Sign In
Journal Club
May 2, 2011

Reduction in Hospital Mortality Over Time in a Hospital Without a Pediatric Medical Emergency Team: Limitations of Before-and-After Study Designs

Author Affiliations

Author Affiliations: Divisions of Pediatric Intensive Care, Departments of Pediatrics, University of Alberta and Stollery Children's Hospital, Edmonton (Drs Joffe and Anton), and University of Calgary and Alberta Children's Hospital, Calgary (Dr Burkholder), Alberta, Canada.

Arch Pediatr Adolesc Med. 2011;165(5):419-423. doi:10.1001/archpediatrics.2011.47

Objective  To determine whether hospital mortality has decreased over time in a hospital that has not introduced a pediatric medical emergency team (PMET).

Design  Retrospective observational study.

Setting  Quaternary children's hospital.

Participants  All pediatric inpatient separations (defined as any discharge, including death) during 10 fiscal years.

Main Outcome Measures  We searched our hospital administrative database to determine the number of pediatric inpatient separations and deaths, and we searched the hospital switchboard and pediatric intensive care databases to determine ward code and cardiopulmonary arrest rates. Relative risks (RRs) with 95% confidence intervals (CIs) and logistic regression compared results over time.

Results  During the periods of the 2 PMET studies showing a reduction in hospital mortality, we found a decrease in hospital mortality: for 1999-2002 vs 2002-2006, 212 deaths among 14 161 patients (1.50%) vs 219 of 26 767 (0.82%), RR, 0.55 (95% CI, 0.44-0.69); for 2000-2005 vs 2005-2007, 300 deaths among 29 497 patients (1.02%) vs 98 of 14 005 (0.70%), RR, 0.69 (95% CI, 0.55-0.86). During the periods of the 3 PMET studies showing no change in or not examining hospital mortality, we found no significant change in hospital mortality. The annual odds ratio for survival was 1.13 (95% CI, 1.09-1.16). There were no changes in ward code and cardiopulmonary arrest rates over time.

Conclusions  We found a reduction in hospital mortality over time in a children's hospital without a PMET. This demonstrates the limitation of before-and-after study designs, and we hypothesize that multiple cointerventions account for the decrease in mortality. Whether a PMET could have reduced mortality further is unknown.

Medical emergency teams (METs) have been advocated “as a means to decrease in-hospital cardiopulmonary arrest (CPA) and mortality.”1(p2311) Published studies advocate use of a pediatric MET (PMET) to “prevent unexpected in-hospital ward cardiac arrest with subsequent death,”2(p309),3(p1148) “decrease inpatient mortality rates,”4(p2268) “be called before a patient has a CPA in hopes of improving outcomes,”5(p117) “reduce the rate of codes outside the intensive care unit . . . [and] rescue patients and reduce hospital mortality rates,”6(p237) or “intervene before an arrest occurs.”7(p418)Previous studies6 report that 8.5% to 14% of all in-hospital CPAs occur outside the pediatric intensive care unit (PICU); once a CPA occurs, the mortality is 50% to 67%.

We suggest that there are at least 4 “links” in the “chain” of patient rescue from hospital CPA and mortality, as follows: (1) symptoms and signs of deterioration (“calling criteria”) occur and are recognized by ward personnel; (2) the ward personnel are empowered to call for assistance without delay, circumventing established hierarchies if necessary; (3) a readily available urgent response occurs; and (4) this response, along with associated interventions, improves outcomes. To improve the collective chain, there have been several before-and-after studies of PMET implementation with inconsistent effects on hospital mortality and CPA rates (eTable 1).2-7 Nevertheless, the data from these cohort studies using historical controls have been considered sufficient to recommend implementation of a PMET at pediatric hospitals.1

We hypothesized that favorable results in PMET studies may be due to limitations of study design, including the use of historical controls, inadequate risk and temporal trend adjustment, and inadequate accounting for cointerventions that confound the comparisons. To test this hypothesis, we obtained administrative data to determine whether the temporal differences in hospital mortality and ward CPA rates found in some PMET studies also occurred at our quaternary children's hospital, which has not implemented a PMET.


The published PMET studies were reviewed to determine the comparability of our single-center hospital and our study design with (1) the single-center hospitals that have published their experience with PMET; (2) the type of analysis performed in the PMET studies, including years of data collection and adjustment for case mix, severity of illness, and temporal trends; and (3) the primary outcomes that demonstrate the efficacy of PMET in the published studies.

Administrative data were obtained from the following:

  • A hospital patient admission database to determine the number of pediatric inpatient separations (defined as any discharge, including death) and deaths, including and excluding patients in the neonatal intensive care unit (ICU), for each fiscal year (April 1 through March 31) from 1999-2000 through 2008-2009.

  • A hospital switchboard database of code calls to determine the number of pediatric codes called each fiscal year with data available from 2001-2002 through 2008-2009.

  • A PICU database (PICU evaluation) to determine the number of pediatric inpatient ward CPA events (defined as an event requiring chest compression) that resulted in admission to the PICU during each fiscal year with data available from 2002-2003 through 2007-2008. At our center, all ward patients who experience CPA with return of spontaneous circulation are admitted to the PICU, and most patients without return of spontaneous circulation during the first few minutes of chest compressions are also taken to the PICU for ongoing resuscitation and possible extracorporeal cardiopulmonary resuscitation (CPR). We cross-checked data with the minutes from our PICU code committee meetings from April 1, 2004, through March 31, 2009, and did not find any missed ward CPA events. We excluded CPA events occurring before arrival at our hospital, in the emergency department, in the operating room, or in the postanesthetic recovery room.

The primary outcomes were to compare hospital mortality, ward CPA, and code call rates during periods as similar as possible to the before-and-after periods used in the published PMET studies. Secondary outcomes were to compare our hospital mortality, ward CPA, and code call rates during 2 equal periods and using annual trends.

Our hospital is a quaternary referral children's hospital serving a population of 1.7 million, with 134 inpatient beds, including 18 mixed medical-surgical-cardiac PICU, 13 neonatal ICU, and 15 high-dependency unit beds. In 2008-2009, there were 6984 patient separations. We maintain active heart and solid-organ transplant services, neurosurgery and cardiac surgery (>400 cases annually) programs, and an extracorporeal life support program that includes extracorporeal CPR, and we have a full complement of pediatric medical and surgical subspecialties. The PICU has 24-hour coverage by 8 pediatric intensivists. Although intensivists are not required to be in-house overnight, a PICU fellow often is. The code team responds to emergencies on the wards and is composed of a pediatric intensivist (if in-house at the time), a PICU fellow and/or resident, a PICU respiratory therapist, and 2 PICU nurses. In addition, PICU consultations (requested by ward residents or medical staff) are seen by the PICU fellow and/or resident and reviewed with the intensivist. The chair of the health research ethics board of our university approved this project as a quality improvement study for publication.

We compared the data between study periods by presenting relative risks and 95% confidence intervals. We compared annual mortality rates using logistic regression and present odds ratios and 95% confidence intervals.


Review of published pmet studies

The results of the review of published PMET studies are shown in eTable 1.2-7 The results suggest that our hospital is similar to the hospitals in the published PMET studies, that the administrative data on hospital mortality and CPAs are from years similar to those used in the PMET studies, and that lack of adjustment for case mix, severity of illness, and temporal trends in our study design is similar to the published PMET studies.

Hospital mortality

The results for hospital mortality are shown in Table 1. During the periods of the 2 PMET studies2,4 that found a significant reduction in hospital mortality between the before and after periods, we also found a highly significant reduction in mortality at our hospital. During the periods of the 3 PMET studies5-7 that did not find a significant reduction in hospital mortality between the before and after periods, we also did not find a significant change in mortality at our hospital. When we divided our data into equal 4- and 5-year periods, there was a highly significant reduction in hospital mortality in the after periods compared with the before periods at our hospital. By logistic regression, the annual odds ratio for survival was 1.13 (95% confidence interval, 1.09-1.16).

Table 1. 
Hospital Mortality Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies
Hospital Mortality Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies

Ward cardiopulmonary arrests

There was no statistically significant change in CPA rates during the periods similar to those used in the published PMET studies or when we divided our data into equal 3-year periods (Table 2). This finding is similar to those of the published PMET studies,2,3,5,6 with none reporting a statistically significant change in ward CPA rates (eTable 1). We did not review the CPA events to determine mortality from these events or to investigate whether they might have been determined retrospectively to be preventable. The published PMET studies3,5,6 have not shown a significant reduction in hospital mortality from ward CPA events. One center2,3 found a statistically significant reduction in preventable ward CPAs with a statistically significant increase in nonpreventable ward CPAs, while another center6 did not find a statistically significant reduction in preventable ward codes (eTable 1).

Table 2. 
Ward CPA Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies
Ward CPA Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies

Code calls

The rate of calls to our code team each year from 2001-2009 showed no change during the periods similar to those used in the published PMET studies or when we divided our data into 2 equal 4-year periods (Table 3). Two PMET centers2,3,5 reported this outcome, and both found an increase in calls to their single-tier PMET compared with the before period, when they had only a code team (eTable 1). Our code team call rate was 0.43% of hospital admissions during 2005-2009; this rate is similar to the PMET call rate of Brilli et al6 (0.32%) and of Tibballs and Kinney2 and Tibballs et al3 (0.51%-0.69%) but lower than the rate found by the other 3 centers4,5,7 (1.17%-1.97%) (eTable 1).

Table 3. 
Ward Code Team Call Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies
Ward Code Team Call Rates at SCH When Years Are Divided Into Periods Used by Published PMET Studies


The published PMET studies,1-7 despite some inconsistent results (eTable 1), have been used to suggest that hospitals should implement PMETs. In our children's hospital, we demonstrated statistically significant decreases in hospital mortality rate during the periods of the 2 PMET studies2,4 reporting this positive outcome, when our hospital data were divided into equal 4- and 5-year periods, and when our hospital data were analyzed annually by logistic regression. We found no significant change over time in the ward CPA rate, similar to the 4 PMET studies2,3,5,6 reporting this outcome. We also found no change in the rate of code team calls during these periods. These results call into question the strength of the PMET data from the cohort studies with historical controls.

There are several potential reasons implementation of a PMET may not improve hospital outcomes. If, as we suggest, there are at least 4 complex links in the chain of patient rescue, there could be 1 or more links for which a PMET is ineffective. Each PMET study used different calling criteria based on expert opinion with unknown sensitivity and specificity. Whether ward personnel used these criteria or instead used a more subjective assessment (as suggested by 2 of the studies6,7) is unclear. If a more subjective assessment was used, then the problem remains whether ward personnel can recognize the clinically deteriorating patient in time. The highly variable PMET calling rates in the published studies (eTable 1)—all less than the number of calls expected from current pediatric early warning scores8,9—suggest that the PMET is not always activated when triggers are present. This finding suggests that there could be weak links at recognition of clinical deterioration, at empowerment to call a PMET without delay, or both. In addition, the concept of PMETs assumes not only that recognizable clinical deterioration occurs before a CPA but also that intervention by a PMET during this clinical deterioration improves outcomes compared with a PICU consultation and/or a graded response by ward personnel. It is possible that many pediatric CPAs develop acutely and are not easily preventable.5,6 Several PMET studies report no changes in the number of unexpected CPAs occurring in the hospital2,3,5,6 or in hospital mortality from CPAs or codes3,6; one center with a PMET2,3 found an unexplained increase in non-PMET–preventable hospital CPAs. It is also possible that clinical deterioration may be due to a poor response to appropriate therapy rather than to a delay in or lack of available expertise on the wards. Finally, it is possible that there are better ways to improve weak links other than by implementation of a PMET, such as focusing on education, empowerment, respect, and availability of staff.

A systematic review10 of pediatric alert criteria for identifying hospitalized children at risk of critical deterioration found that the tools are unvalidated, are inconsistent, do not address the issue of recognition in time to intervene, and have poor positive predictive value. The authors called for “prospective evaluation of validity, reliability, and utility before widespread adoption into clinical practice.”10(p600) Another systematic review11 of METs found that the mortality benefit in PMET studies is 3 to 100 times higher than the number of CPAs averted. This finding suggests that the benefit reported in PMET studies is likely confounded by interventions unknown to the authors.

Several factors may explain the statistically significant reduction in our hospital mortality rate during the periods studied. Any study with historical controls carries a risk of confounding by known or unknown cointerventions. For example, as a result of largely unclear factors, mortality has improved recently for some conditions, such as acute respiratory distress syndrome12 and hypoplastic left heart syndrome.13 The mortality of critically ill patients depends on many variables, including time of day at admission,14 having an in-house fellow at night,14 source of admission,15 intensivist staffing patterns,16 and surgical volume.17 Many cointerventions occurred at our hospital and at the hospitals that published their PMET experience (eTable 2). These or other unclear cointerventions may account for improved hospital mortality over time.

There are some strengths to our study. First, we used administrative data from existing databases that were collected prospectively and based on objective outcomes of mortality, ward CPAs requiring chest compressions, and code team calls. Second, our children's hospital is similar to most high-acuity referral hospitals with a PICU.18,19 According to data from a 2005 North American survey18 of children's hospitals, our center is in the higher quartiles for the number of acute care, high-dependency unit, and PICU beds and is among those with PICUs that have care always directed by an intensivist (47%), offer extracorporeal life support (56%) or extracorporeal CPR (38%), and have urgent-response personnel (75%). In that survey, the urgent-response personnel were staff from the PICU in 51% of the hospitals, a code team in 8%, and a PMET in 16%. We acknowledge that our ward consultations and code calls may be considered a “functional PMET”; however, the survey results indicate that most hospitals have similar urgent-response personnel systems, supporting the claim that PMETs may not be necessary. According to data from a survey conducted in the United States in 2004,19 our center is in the higher quartiles for the number of PICU beds, pediatric intensivists, and pediatric intensivist in-house coverage hours and is similar to those having pediatric intensivist coverage 24 hours a day (71%). Therefore, our findings are likely to be similar to those of most children's hospitals in North America.

We believe there are 2 main explanations for our results and those of the PMET studies with positive outcomes: improving care over time due to unmeasured cointerventions and having mechanisms that make a PMET redundant. For example, code calls at most pediatric hospitals far outnumber the true ward CPA rate,18 allowing the code team to respond to clinical deteriorations other than CPAs. Our hospital's rate of code calls is similar to the rates in some published PMET studies2,3,6 but much lower than others4,5,7 for unclear reasons (eTable 1). The PICU ward consultations, which we estimate to number at least 1 a week, are similar to the rate of PMET calls in some studies.4,5,7 We also estimate that the interventions performed when our code team is called for many codes or when consultations are done are similar to those described in the PMET studies (eTable 1).

There are some limitations to this study. First, a single-center study may not be generalizable. Second, we cannot identify what factors were responsible for our reduction in hospital mortality; however, we can say with certainty that it was not because of introduction of a PMET. If we had implemented a PMET, we would likely have misattributed the reduction in hospital mortality over time to this intervention. Third, we did not adjust for case-mix, severity of illness, or temporal trends between the periods examined. Nevertheless, this lack of adjustment is similar to the published PMET studies (eTable 1) and may explain the differences between periods. Fourth, we were unable to determine the number of respiratory arrests that occurred at our hospital. Finally, we have not demonstrated that (or even examined whether) a PMET is ineffective in reducing hospital mortality, ward CPA, or respiratory arrest rates. We believe we have shown that before-and-after study designs have limitations that make the published PMET studies unable to prove that PMET implementation improves outcomes. We recognize that some hospital deaths and CPAs are preventable; how to prevent these adverse outcomes requires further study.

We found a reduction in hospital mortality over time even though we have not introduced a PMET. We do not assign our reduction in mortality to absence of a PMET; rather, we claim that this finding demonstrates the limitation of before-and-after study designs (cohort studies with historical controls) in determining the effect of PMET implementation. A large cluster-randomized controlled trial of PMET implementation with validated calling criteria is required to determine whether PMETs are effective.

Correspondence: Ari R. Joffe, MD, Division of Pediatric Intensive Care, Department of Pediatrics, Room 3A3.07, Stollery Children's Hospital, 8440 112th St, Edmonton, AB T6G 2B7, Canada (ari.joffe@albertahealthservices.ca).

Accepted for Publication: September 16, 2010.

Author Contributions: Dr Joffe had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Joffe. Acquisition of data: Joffe. Analysis and interpretation of data: Joffe, Anton, and Burkholder. Drafting of the manuscript: Joffe. Critical revision of the manuscript for important intellectual content: Joffe, Anton, and Burkholder. Statistical analysis: Joffe. Administrative, technical, and material support: Joffe, Anton, and Burkholder.

Financial Disclosure: None reported.

Online-Only Material: This article is featured in the Archives Journal Club. Go here to download teaching PowerPoint slides.

Nowak  JEBrilli  RJ Pediatric rapid response teams: is it time?  JAMA 2007;298 (19) 2311- 2312PubMedGoogle Scholar
Tibballs  JKinney  S Reduction of hospital mortality and of preventable cardiac arrest and death on introduction of a pediatric medical emergency team.  Pediatr Crit Care Med 2009;10 (3) 306- 312PubMedGoogle Scholar
Tibballs  JKinney  SDuke  TOakley  EHennessy  M Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: preliminary results.  Arch Dis Child 2005;90 (11) 1148- 1152PubMedGoogle Scholar
Sharek  PJParast  LMLeong  K  et al.  Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children's hospital.  JAMA 2007;298 (19) 2267- 2274PubMedGoogle Scholar
Hunt  EAZimmer  KPRinke  ML  et al.  Transition from a traditional code team to a medical emergency team and categorization of cardiopulmonary arrests in a children's center.  Arch Pediatr Adolesc Med 2008;162 (2) 117- 122PubMedGoogle Scholar
Brilli  RJGibson  RLuria  JW  et al.  Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit.  Pediatr Crit Care Med 2007;8 (3) 236- 247PubMedGoogle Scholar
Zenker  PSchlesinger  AHauck  M  et al.  Implementation and impact of a rapid response team in a children's hospital.  Jt Comm J Qual Patient Saf 2007;33 (7) 418- 425PubMedGoogle Scholar
Duncan  HHutchison  JParshuram  CS The Pediatric Early Warning System score: a severity of illness score to predict urgent medical need in hospitalized children.  J Crit Care 2006;21 (3) 271- 278PubMedGoogle Scholar
Edwards  EDPowell  CVEMason  BWOliver  A Prospective cohort study to test the predictability of the Cardiff and Vale paediatric early warning system.  Arch Dis Child 2009;94 (8) 602- 606PubMedGoogle Scholar
Chapman  SMGrocott  MPWFranck  LS Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration.  Intensive Care Med 2010;36 (4) 600- 611PubMedGoogle Scholar
Chan  PSJain  RNallmothu  BKBerg  RASasson  C Rapid response teams: a systematic review and meta-analysis.  Arch Intern Med 2010;170 (1) 18- 26PubMedGoogle Scholar
Milberg  JADavis  DRSteinberg  KPHudson  LD Improved survival of patients with acute respiratory distress syndrome (ARDS): 1983-1993.  JAMA 1995;273 (4) 306- 309PubMedGoogle Scholar
Atallah  JDinu  IAJoffe  AR  et al. Western Canadian Complex Pediatric Therapies Follow-up Group, Two-year survival and mental and psychomotor outcomes after the Norwood procedure: an analysis of the modified Blalock-Taussig shunt and right ventricle-to-pulmonary artery shunt surgical eras.  Circulation 2008;118 (14) 1410- 1418PubMedGoogle Scholar
Lee  SKLee  DSCAndrews  WLBaboolal  RPendray  MStewart  SCanadian Neonatal Network, Higher mortality rates among inborn infants admitted to neonatal intensive care units at night.  J Pediatr 2003;143 (5) 592- 597PubMedGoogle Scholar
Odetola  FORosenberg  ALDavis  MMClark  SJDechert  REShanley  TP Do outcomes vary according to the source of admission to the pediatric intensive care unit?  Pediatr Crit Care Med 2008;9 (1) 20- 25PubMedGoogle Scholar
Pronovost  PJAngus  DCDorman  TRobinson  KADremsizov  TTYoung  TL Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review.  JAMA 2002;288 (17) 2151- 2162PubMedGoogle Scholar
Hannan  ELRacz  MKavey  REQuaegebeur  JMWilliams  R Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality.  Pediatrics 1998;101 (6) 963- 969PubMedGoogle Scholar
VandenBerg  SDHutchison  JSParshuram  CSPaediatric Early Warning System Investigators, A cross-sectional survey of levels of care and response mechanisms for evolving critical illness in hospitalized children.  Pediatrics 2007;119 (4) e940- e946PubMedGoogle Scholar
Odetola  FOClark  SJFreed  GLBratton  SLDavis  MM A national survey of pediatric critical care resources in the United States.  Pediatrics 2005;115 (4) e382- e386PubMed10.1542/peds.2004-1920Google Scholar