Prediction of Critical Illness During Out-of-Hospital Emergency Care | Critical Care Medicine | JAMA | JAMA Network
[Skip to Navigation]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
Glance LG, Li Y, Osler TM, Dick A, Mukamel DB. Impact of patient volume on the mortality rate of adult intensive care unit patients.  Crit Care Med. 2006;34(7):1925-193416715030PubMedGoogle ScholarCrossref
Kahn JM, Goss CH, Heagerty PJ, Kramer AA, O’Brien CR, Rubenfeld GD. Hospital volume and the outcomes of mechanical ventilation.  N Engl J Med. 2006;355(1):41-5016822995PubMedGoogle ScholarCrossref
Durairaj L, Torner JC, Chrischilles EA, Vaughan Sarrazin MS, Yankey J, Rosenthal GE. Hospital volume-outcome relationships among medical admissions to ICUs.  Chest. 2005;128(3):1682-168916162775PubMedGoogle ScholarCrossref
Kahn JM, Branas CC, Schwab CW, Asch DA. Regionalization of medical critical care: what can we learn from the trauma experience?  Crit Care Med. 2008;36(11):3085-308818824900PubMedGoogle ScholarCrossref
Institute of Medicine Committee on the Future of Emergency Care in the United States Health System.  Hospital Based Emergency Care: At the Breaking Point. Washington, DC: National Academies Press; 2007
Sporer KA, Johnson NJ, Yeh CC, Youngblood GM. Can emergency medical dispatch codes predict prehospital interventions for common 9-1-1 call types?  Prehosp Emerg Care. 2008;12(4):470-47818924011PubMedGoogle ScholarCrossref
Kidwell CS, Starkman S, Eckstein M, Weems K, Saver JL. Identifying stroke in the field. Prospective validation of the Los Angeles prehospital stroke screen (LAPSS).  Stroke. 2000;31(1):71-7610625718PubMedGoogle ScholarCrossref
Brown LH, Hubble MW, Cone DC,  et al.  Paramedic determinations of medical necessity: a meta-analysis.  Prehosp Emerg Care. 2009;13(4):516-52719731166PubMedGoogle ScholarCrossref
Studnek JR, Fernandez AR, Margolis GS, O’Connor RE. Physician medical oversight in emergency medical services: where are we?  Prehosp Emerg Care. 2009;13(1):53-5819145525PubMedGoogle ScholarCrossref
Yarris LM, Moreno R, Schmidt TA, Adams AL, Brooks HS. Reasons why patients choose an ambulance and willingness to consider alternatives.  Acad Emerg Med. 2006;13(4):401-40516531606PubMedGoogle ScholarCrossref
Brown JP, Mahmud E, Dunford JV, Ben-Yehuda O. Effect of prehospital 12-lead electrocardiogram on activation of the cardiac catheterization laboratory and door-to-balloon time in ST-segment elevation acute myocardial infarction.  Am J Cardiol. 2008;101(2):158-16118178399PubMedGoogle ScholarCrossref
Sivagangabalan G, Ong AT, Narayan A,  et al.  Effect of prehospital triage on revascularization times, left ventricular function, and survival in patients with ST-elevation myocardial infarction.  Am J Cardiol. 2009;103(7):907-91219327414PubMedGoogle ScholarCrossref
Brooks SC, Allan KS, Welsford M, Verbeek PR, Arntz HR, Morrison LJ. Prehospital triage and direct transport of patients with ST-elevation myocardial infarction to primary percutaneous coronary intervention centres: a systematic review and meta-analysis.  CJEM. 2009;11(5):481-49219788793PubMedGoogle Scholar
Feldman MJ, Verbeek PR, Lyons DG, Chad SJ, Craig AM, Schwartz B. Comparison of the medical priority dispatch system to an out-of-hospital patient acuity score.  Acad Emerg Med. 2006;13(9):954-96016894004PubMedGoogle ScholarCrossref
Levine SD, Colwell CB, Pons PT, Gravitz C, Haukoos JS, McVaney KE. How well do paramedics predict admission to the hospital? a prospective study.  J Emerg Med. 2006;31(1):1-516798145PubMedGoogle ScholarCrossref
Price TG, Hooker EA, Neubauer J. Prehospital provider prediction of emergency department disposition: implications for selective diversion.  Prehosp Emerg Care. 2005;9(3):322-32516147483PubMedGoogle ScholarCrossref
Mackersie RC. History of trauma field triage development and the American College of Surgeons criteria.  Prehosp Emerg Care. 2006;10(3):287-29416801263PubMedGoogle ScholarCrossref
Langhelle A, Tyvold SS, Lexow K, Hapnes SA, Sunde K, Steen PA. In-hospital factors associated with improved outcome after out-of-hospital cardiac arrest: a comparison between four regions in Norway.  Resuscitation. 2003;56(3):247-26312628556PubMedGoogle ScholarCrossref
Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care.  Crit Care Med. 2001;29(7):1303-131011445675PubMedGoogle ScholarCrossref
Quan H, Parsons GA, Ghali WA. Validity of procedure codes in International Classification of Diseases, 9th Revision, Clinical Modification administrative data.  Med Care. 2004;42(8):801-80915258482PubMedGoogle ScholarCrossref
Dean JM, Vernon DD, Cook L, Nechodom P, Reading J, Suruda A. Probabilistic linkage of computerized ambulance and inpatient hospital discharge records: a potential tool for evaluation of emergency medical services.  Ann Emerg Med. 2001;37(6):616-62611385330PubMedGoogle ScholarCrossref
Fellegi IP, Sunter AB. A theory for record linkage.  J Am Stat Assoc. 1969;64:1193-1210Google ScholarCrossref
Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model.  BMJ. 2009;338:b60519477892PubMedGoogle ScholarCrossref
Royston P, Moons KG, Altman DG, Vergouwe  Y. Prognosis and prognostic research: Developing a prognostic model.  BMJ. 2009;338:b60419336487PubMedGoogle ScholarCrossref
Dawson DE. National Emergency Medical Services Information System (NEMSIS).  Prehosp Emerg Care. 2006;10(3):314-31616801268PubMedGoogle ScholarCrossref
Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE—Acute Physiology and Chronic Health Evaluation: a physiologically based classification system.  Crit Care Med. 1981;9(8):591-5977261642PubMedGoogle ScholarCrossref
Akaike H. Information theory and an extension of the maximum likelihood principle. Paper presented at: 2nd International Symposium on Information Theory; September 2-8, 1971; Tsahkadsor, Armenia
Moons KG, Harrell FE, Steyerberg EW. Should scoring rules be based on odds ratios or regression coefficients?  J Clin Epidemiol. 2002;55(10):1054-105512464384PubMedGoogle ScholarCrossref
Brier GW. Verification of forecasts expressed in terms of probability.  Mon Weather Rev. 1950;78:1-3Google ScholarCrossref
Mckelvey R, Zavoina W. A statistical model for the analysis of ordinal level dependent variables.  J Math Sociol. 1975;4:103-120Google ScholarCrossref
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve.  Radiology. 1982;143(1):29-367063747PubMedGoogle Scholar
Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models.  Am J Epidemiol. 1982;115(1):92-1067055134PubMedGoogle Scholar
Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited.  Crit Care Med. 2007;35(9):2052-205617568333PubMedGoogle ScholarCrossref
Miller ME, Langefeld CD, Tierney WM, Hui SL, McDonald CJ. Validation of probabilistic predictions.  Med Decis Making. 1993;13(1):49-588433637PubMedGoogle ScholarCrossref
Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study.  BMC Med Res Methodol. 2010;10(1):720085642PubMedGoogle ScholarCrossref
Royston P. Multiple imputation of missing values.  Stata J. 2004;4(3):227-241Google Scholar
Ehlenbach WJ, Hough CL, Crane PK,  et al.  Association between acute care and critical illness hospitalization and cognitive function in older adults.  JAMA. 2010;303(8):763-77020179286PubMedGoogle ScholarCrossref
Barnato AE, Kahn JM, Rubenfeld GD,  et al.  Prioritizing the organization and management of intensive care services in the United States: the PROMIS conference.  Crit Care Med. 2007;35(4):1003-101117334242PubMedGoogle ScholarCrossref
Kahn JM, Linde-Zwirble WT, Wunsch H,  et al.  Potential value of regionalized intensive care for mechanically ventilated medical patients.  Am J Respir Crit Care Med. 2008;177(3):285-29118006884PubMedGoogle ScholarCrossref
Carr BG, Conway PH, Meisel ZF, Steiner CA, Clancy C. Defining the emergency care sensitive condition: a health policy research agenda in emergency medicine.  Ann Emerg Med. 2010;56(1):49-5120045577PubMedGoogle ScholarCrossref
Wang HE, Weaver MD, Shapiro NI, Yealy DM. Opportunities for emergency medical services care of sepsis.  Resuscitation. 2010;81(2):193-19720006419PubMedGoogle ScholarCrossref
Neely KW, Eldurkar J, Drake ME. Can current EMS dispatch protocols identify layperson-reported sentinel conditions?  Prehosp Emerg Care. 2000;4(3):238-24410895919PubMedGoogle ScholarCrossref
Newgard CD. The validity of using multiple imputation for missing out-of-hospital data in a state trauma registry.  Acad Emerg Med. 2006;13(3):314-32416495420PubMedGoogle ScholarCrossref
Weaver WD, Eisenberg MS, Martin JS,  et al.  Myocardial Infarction Triage and Intervention Project— phase I: patient characteristics and feasibility of prehospital initiation of thrombolytic therapy.  J Am Coll Cardiol. 1990;15(5):925-9312312978PubMedGoogle ScholarCrossref
Kothari R, Barsan W, Brott T, Broderick J, Ashbrock S. Frequency and accuracy of prehospital diagnosis of acute stroke.  Stroke. 1995;26(6):937-9417762041PubMedGoogle ScholarCrossref
Sasser SM, Hunt RC, Sullivent EE,  et al; National Expert Panel on Field Triage, Centers for Disease Control and Prevention (CDC).  Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage.  MMWR Recomm Rep. 2009;58(RR-1):1-3519165138PubMedGoogle Scholar
Kahn JM, Asch RJ, Iwashyna TJ,  et al.  Physician attitudes toward regionalization of adult critical care: a national survey.  Crit Care Med. 2009;37(7):2149-215419455025PubMedGoogle ScholarCrossref
Pearse RM. Extending the role of lactate measurement into the prehospital environment.  Crit Care. 2009;13(1):11519232076PubMedGoogle ScholarCrossref
Sasson C, Hegg AJ, Macy M, Park A, Kellermann A, McNally B.CARES Surveillance Group.  Prehospital termination of resuscitation in cases of refractory out-of-hospital cardiac arrest.  JAMA. 2008;300(12):1432-143818812534PubMedGoogle ScholarCrossref
Huemer G, Pernerstorfer T, Mauritz W. Prehospital emergency medicine services in Europe: structure and equipment.  Eur J Emerg Med. 1994;1(2):62-689422140PubMedGoogle ScholarCrossref
Grossman DC, Kim A, Macdonald SC, Klein P, Copass MK, Maier RV. Urban-rural differences in prehospital care of major trauma.  J Trauma. 1997;42(4):723-7299137264PubMedGoogle ScholarCrossref
Barnato AE, Alexander SL, Linde-Zwirble WT, Angus DC. Racial variation in the incidence, care, and outcomes of severe sepsis: analysis of population, patient, and hospital characteristics.  Am J Respir Crit Care Med. 2008;177(3):279-28417975201PubMedGoogle ScholarCrossref
Rehkopf DH, Berkman LF, Coull B, Krieger N. The non-linear risk of mortality by income level in a healthy population: US National Health and Nutrition Examination Survey mortality follow-up cohort, 1988-2001.  BMC Public Health. 2008;8:38319000311PubMedGoogle ScholarCrossref
Caring for the Critically Ill Patient
August 18, 2010

Prediction of Critical Illness During Out-of-Hospital Emergency Care

Author Affiliations

Author Affiliations: Division of Pulmonary and Critical Care Medicine, Harborview Medical Center (Drs Seymour and Watkins), Research Division, Puget Sound Blood Center (Dr Watkins), and Department of Epidemiology (Dr Heckbert) and King County Medic One, Division of General Internal Medicine (Dr Rea), University of Washington, Seattle; Division of Pulmonary, Allergy, and Critical Care, Leonard Davis Institute for Health Economics and Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, University of Pennsylvania Medical Center, Philadelphia (Dr Kahn); and Division of Pulmonary and Critical Care Medicine and Robert Wood Johnson Clinical Scholar Program, University of Michigan, Ann Arbor (Dr Cooke). Dr Kahn is now with the Clinical Research, Investigation, and Systems Modeling of Acute Illness Laboratory, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, and the Department of Health Policy and Management, University of Pittsburgh Graduate School of Public Health, Pittsburgh, Pennsylvania.

JAMA. 2010;304(7):747-754. doi:10.1001/jama.2010.1140

Context Early identification of nontrauma patients in need of critical care services in the emergency setting may improve triage decisions and facilitate regionalization of critical care.

Objectives To determine the out-of-hospital clinical predictors of critical illness and to characterize the performance of a simple score for out-of-hospital prediction of development of critical illness during hospitalization.

Design and Setting Population-based cohort study of an emergency medical services (EMS) system in greater King County, Washington (excluding metropolitan Seattle), that transports to 16 receiving facilities.

Patients Nontrauma, non–cardiac arrest adult patients transported to a hospital by King County EMS from 2002 through 2006. Eligible records with complete data (N = 144 913) were linked to hospital discharge data and randomly split into development (n = 87 266 [60%]) and validation (n = 57 647 [40%]) cohorts.

Main Outcome Measure Development of critical illness, defined as severe sepsis, delivery of mechanical ventilation, or death during hospitalization.

Results Critical illness occurred during hospitalization in 5% of the development (n = 4835) and validation (n = 3121) cohorts. Multivariable predictors of critical illness included older age, lower systolic blood pressure, abnormal respiratory rate, lower Glasgow Coma Scale score, lower pulse oximetry, and nursing home residence during out-of-hospital care (P < .01 for all). When applying a summary critical illness prediction score to the validation cohort (range, 0-8), the area under the receiver operating characteristic curve was 0.77 (95% confidence interval [CI], 0.76-0.78), with satisfactory calibration slope (1.0). Using a score threshold of 4 or higher, sensitivity was 0.22 (95% CI, 0.20-0.23), specificity was 0.98 (95% CI, 0.98-0.98), positive likelihood ratio was 9.8 (95% CI, 8.9-10.6), and negative likelihood ratio was 0.80 (95% CI, 0.79- 0.82). A threshold of 1 or greater for critical illness improved sensitivity (0.98; 95% CI, 0.97-0.98) but reduced specificity (0.17; 95% CI, 0.17-0.17).

Conclusions In a population-based cohort, the score on a prediction rule using out-of-hospital factors was significantly associated with the development of critical illness during hospitalization. This score requires external validation in an independent population.