[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.  Calibration Plot of External Validation
Calibration Plot of External Validation

The validation data were acquired from the Brabant region of the Netherlands. Error bars indicate 95% CIs.

Figure 2.  Screen Shots of the Mobile App
Screen Shots of the Mobile App

EMS indicates emergency medical services; TT app, Trauma Triage App (Synappz).

Table 1.  Baseline Characteristics of the Central Netherlands Region and the Brabant Region
Baseline Characteristics of the Central Netherlands Region and the Brabant Region
Table 2.  Bivariable Logistic Regression Analysis on the Central Netherlands Regiona
Bivariable Logistic Regression Analysis on the Central Netherlands Regiona
Table 3.  Multivariate Analysis of the Predictors for ISS>15 in the Central Netherlands Regiona
Multivariate Analysis of the Predictors for ISS>15 in the Central Netherlands Regiona
1.
MacKenzie  EJ, Rivara  FP, Jurkovich  GJ,  et al.  A national evaluation of the effect of trauma-center care on mortality.  N Engl J Med. 2006;354(4):366-378. doi:10.1056/NEJMsa052049PubMedGoogle ScholarCrossref
2.
Staudenmayer  K, Weiser  TG, Maggio  PM, Spain  DA, Hsia  RY.  Trauma center care is associated with reduced readmissions after injury.  J Trauma Acute Care Surg. 2016;80(3):412-416. doi:10.1097/TA.0000000000000956PubMedGoogle ScholarCrossref
3.
Newgard  CD, Staudenmayer  K, Hsia  RY,  et al.  The cost of overtriage: more than one-third of low-risk injured patients were taken to major trauma centers.  Health Aff (Millwood). 2013;32(9):1591-1599. doi:10.1377/hlthaff.2012.1142PubMedGoogle ScholarCrossref
4.
Champion  HR, Lombardo  LV, Wade  CE, Kalin  EJ, Lawnick  MM, Holcomb  JB.  Time and place of death from automobile crashes: research endpoint implications.  J Trauma Acute Care Surg. 2016;81(3):420-426. doi:10.1097/TA.0000000000001124PubMedGoogle ScholarCrossref
5.
van Rein  EAJ, Houwert  RM, Gunning  AC, Lichtveld  RA, Leenen  LPH, van Heijl  M.  Accuracy of prehospital triage protocols in selecting severely injured patients: a systematic review.  J Trauma Acute Care Surg. 2017;83(2):328-339. doi:10.1097/TA.0000000000001516PubMedGoogle ScholarCrossref
6.
Newgard  CD, Fu  R, Zive  D,  et al.  Prospective validation of the National Field Triage Guidelines for identifying seriously injured persons.  J Am Coll Surg. 2016;222(2):146-158.e2. doi:10.1016/j.jamcollsurg.2015.10.016PubMedGoogle ScholarCrossref
7.
Newgard  CD, Zive  D, Holmes  JF,  et al; WESTRN investigators.  A multisite assessment of the American College of Surgeons Committee on Trauma field triage decision scheme for identifying seriously injured children and adults.  J Am Coll Surg. 2011;213(6):709-721. doi:10.1016/j.jamcollsurg.2011.09.012PubMedGoogle ScholarCrossref
8.
Lewis  TL, Fothergill  RT, Karthikesalingam  A; Aneurysm-FILTR Study Group.  Ambulance smartphone tool for field triage of ruptured aortic aneurysms (FILTR): study protocol for a prospective observational validation of diagnostic accuracy.  BMJ Open. 2016;6(10):e011308. doi:10.1136/bmjopen-2016-011308PubMedGoogle ScholarCrossref
9.
Nogueira  RG, Silva  GS, Lima  FO,  et al.  The FAST-ED app: a smartphone platform for the field triage of patients with stroke.  Stroke. 2017;48(5):1278-1284. doi:10.1161/STROKEAHA.116.016026PubMedGoogle ScholarCrossref
10.
Faul  M, Xu  L, Sasser  SM.  Hospitalized traumatic brain injury: low trauma center utilization and high interfacility transfers among older adults.  Prehosp Emerg Care. 2016;20(5):594-600. doi:10.3109/10903127.2016.1149651PubMedGoogle ScholarCrossref
11.
Sasser  SM, Hunt  RC, Faul  M,  et al; Centers for Disease Control and Prevention (CDC).  Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011.  MMWR Recomm Rep. 2012;61(RR-1):1-20.PubMedGoogle Scholar
12.
The Netherlands Health Care Institute. Urgency must be good—indicators and standards for 6 emergency care indications. https://www.zorginstituutnederland.nl/publicaties/rapport/2015/12/16/spoed-moet-goed---indicatoren-en-normen-voor-zes-spoedzorgindicaties. Published July 12, 2015. Accessed October 31, 2018.
13.
Ambulancezorg Nederland. National Ambulance Care Protocol (LPA). https://www.ambulancezorg.nl/themas/kwaliteit-van-zorg/protocollen-en-richtlijnen/landelijk-protocol-ambulancezorg. Accessed October 31, 2018.
14.
Landelijk Netwerk Acute Zorg. Landelijke Traumaregistratie 2011-2015: Rapportage Nederland. 2016. Landelijk Netwerk Acute Zorg. http://www.lnaz.nl/cms/LTR_landelijk_jaarrapport_2011-2015.pdf. Published November 2, 2016. Accessed October 31, 2018.
15.
Voskens  FJ, van Rein  EAJ, van der Sluijs  R,  et al.  Accuracy of prehospital triage in selecting severely injured trauma patients.  JAMA Surg. 2018;153(4):322-327. doi:10.1001/jamasurg.2017.4472PubMedGoogle ScholarCrossref
16.
Bond  RJ, Kortbeek  JB, Preshaw  RM.  Field trauma triage: combining mechanism of injury with the prehospital index for an improved trauma triage tool.  J Trauma. 1997;43(2):283-287. doi:10.1097/00005373-199708000-00013PubMedGoogle ScholarCrossref
17.
Meisler  R, Thomsen  AB, Abildstrøm  H,  et al.  Triage and mortality in 2875 consecutive trauma patients.  Acta Anaesthesiol Scand. 2010;54(2):218-223. doi:10.1111/j.1399-6576.2009.02075.xPubMedGoogle ScholarCrossref
18.
van Buuren  S, Groothuis-Oudshoorn  K.  mice: multivariate imputation by chained equations in R.  J Stat Softw. 2011;45(3):40-49.Google Scholar
19.
Marshall  A, Altman  DG, Holder  RL, Royston  P.  Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines.  BMC Med Res Methodol. 2009;9:57-64. doi:10.1186/1471-2288-9-57PubMedGoogle ScholarCrossref
20.
Harrell  FE.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001. doi:10.1007/978-1-4757-3462-1
21.
Moons  KGM, Donders  ART, Steyerberg  EW, Harrell  FE.  Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example.  J Clin Epidemiol. 2004;57(12):1262-1270. doi:10.1016/j.jclinepi.2004.01.020PubMedGoogle ScholarCrossref
22.
Vergouwe  Y, Nieboer  D, Oostenbrink  R,  et al.  A closed testing procedure to select an appropriate method for updating prediction models.  Stat Med. 2017;36(28):4529-4539. doi:10.1002/sim.7179PubMedGoogle ScholarCrossref
23.
Janssen  KJM, Moons  KGM, Kalkman  CJ, Grobbee  DE, Vergouwe  Y.  Updating methods improved the performance of a clinical prediction model in new patients.  J Clin Epidemiol. 2008;61(1):76-86. doi:10.1016/j.jclinepi.2007.04.018PubMedGoogle ScholarCrossref
24.
R Development Core Team. R: a language and environment for statistical computing [serial online]. Vienna, Austria: R Foundation for Statistical Computing; 2016. https://www.r-project.org/about.html. Accessed October 31, 2018.
25.
van Laarhoven  JJ, Lansink  KW, van Heijl  M, Lichtveld  RA, Leenen  LP.  Accuracy of the field triage protocol in selecting severely injured patients after high energy trauma.  Injury. 2014;45(5):869-873. doi:10.1016/j.injury.2013.12.010PubMedGoogle ScholarCrossref
26.
Hamada  SR, Gauss  T, Duchateau  FX,  et al.  Evaluation of the performance of French physician-staffed emergency medical service in the triage of major trauma patients.  J Trauma Acute Care Surg. 2014;76(6):1476-1483. doi:10.1097/TA.0000000000000239PubMedGoogle ScholarCrossref
27.
Dinh  MM, Bein  KJ, Oliver  M, Veillard  AS, Ivers  R.  Refining the trauma triage algorithm at an Australian major trauma centre: derivation and internal validation of a triage risk score.  Eur J Trauma Emerg Surg. 2014;40(1):67-74. doi:10.1007/s00068-013-0315-1PubMedGoogle ScholarCrossref
28.
Rubenson Wahlin  R, Ponzer  S, Skrifvars  MB, Lossius  HM, Castrén  M.  Effect of an organizational change in a prehospital trauma care protocol and trauma transport directive in a large urban city: a before and after study.  Scand J Trauma Resusc Emerg Med. 2016;24:26. doi:10.1186/s13049-016-0218-3PubMedGoogle ScholarCrossref
29.
Champion  HR, Sacco  WJ, Copes  WS, Gann  DS, Gennarelli  TA, Flanagan  ME.  A revision of the Trauma Score.  J Trauma. 1989;29(5):623-629. doi:10.1097/00005373-198905000-00017PubMedGoogle ScholarCrossref
30.
Ocak  G, Sturms  LM, Hoogeveen  JM, Le Cessie  S, Jukema  GN.  Prehospital identification of major trauma patients.  Langenbecks Arch Surg. 2009;394(2):285-292. doi:10.1007/s00423-008-0340-4PubMedGoogle ScholarCrossref
31.
Vassar  MJ, Holcroft  JJ, Knudson  MM, Kizer  KW.  Fractures in access to and assessment of trauma systems.  J Am Coll Surg. 2003;197(5):717-725. doi:10.1016/S1072-7515(03)00749-XPubMedGoogle ScholarCrossref
32.
Chang  DC, Bass  RR, Cornwell  EE, Mackenzie  EJ.  Undertriage of elderly trauma patients to state-designated trauma centers.  Arch Surg. 2008;143(8):776-781. doi:10.1001/archsurg.143.8.776PubMedGoogle ScholarCrossref
33.
Ichwan  B, Darbha  S, Shah  MN,  et al.  Geriatric-specific triage criteria are more sensitive than standard adult criteria in identifying need for trauma center care in injured older adults.  Ann Emerg Med. 2015;65(1):92-100.e3. doi:10.1016/j.annemergmed.2014.04.019PubMedGoogle ScholarCrossref
34.
Teasdale  G, Maas  A, Lecky  F, Manley  G, Stocchetti  N, Murray  G.  The Glasgow Coma Scale at 40 years: standing the test of time.  Lancet Neurol. 2014;13(8):844-854. doi:10.1016/S1474-4422(14)70120-6PubMedGoogle ScholarCrossref
35.
Wong  TH, Krishnaswamy  G, Nadkarni  NV,  et al.  Combining the new injury severity score with an anatomical polytrauma injury variable predicts mortality better than the new Injury Severity Score and the Injury Severity Score: a retrospective cohort study.  Scand J Trauma Resusc Emerg Med. 2016;24:25. doi:10.1186/s13049-016-0215-6PubMedGoogle ScholarCrossref
36.
Lefering  R, Huber-Wagner  S, Nienaber  U, Maegele  M, Bouillon  B.  Update of the trauma risk adjustment model of the TraumaRegister DGU™: the Revised Injury Severity Classification, version II.  Crit Care. 2014;18(5):476. doi:10.1186/s13054-014-0476-2PubMedGoogle ScholarCrossref
37.
Scheetz  LJ.  Comparison of type and severity of major injuries among undertriaged and correctly triaged older patients.  J Emerg Med. 2012;43(6):1020-1028. doi:10.1016/j.jemermed.2011.09.036PubMedGoogle ScholarCrossref
38.
Schoell  SL, Doud  AN, Weaver  AA,  et al.  Development of a time sensitivity score for frequently occurring motor vehicle crash injuries.  J Am Coll Surg. 2015;220(3):305-312.e3. doi:10.1016/j.jamcollsurg.2014.11.022PubMedGoogle ScholarCrossref
39.
Cole  TB.  Global road safety crisis remedy sought: 1.2 million killed, 50 million injured annually.  JAMA. 2004;291(21):2531-2532. doi:10.1001/jama.291.21.2531PubMedGoogle ScholarCrossref
40.
Leeflang  MM, Bossuyt  PM, Irwig  L.  Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis.  J Clin Epidemiol. 2009;62(1):5-12. doi:10.1016/j.jclinepi.2008.04.007PubMedGoogle ScholarCrossref
41.
Barnes  J, Hassan  A, Cuerden  R, Cookson  R, Kohlhofer  J.  Comparison of injury severity between AIS 2005 and AIS 1990 in a large injury database.  Ann Adv Automot Med. 2009;53:83-89.PubMedGoogle Scholar
42.
American College of Surgeons Committee on Trauma.  Resources for Optimal Care of the Injured Patient 2006. Chicago, IL: American College of Surgeons; 2006.
43.
Hedges  JR, Feero  S, Moore  B, Haver  DW, Shultz  B.  Comparison of prehospital trauma triage instruments in a semirural population.  J Emerg Med. 1987;5(3):197-208. doi:10.1016/0736-4679(87)90179-XPubMedGoogle ScholarCrossref
44.
Newgard  CD, Nelson  MJ, Kampp  M,  et al.  Out-of-hospital decision making and factors influencing the regional distribution of injured patients in a trauma system.  J Trauma. 2011;70(6):1345-1353. doi:10.1097/TA.0b013e3182191a1bPubMedGoogle ScholarCrossref
45.
Emerman  CL, Shade  B, Kubincanek  J.  A comparison of EMT judgment and prehospital trauma triage instruments.  J Trauma. 1991;31(10):1369-1375. doi:10.1097/00005373-199110000-00009PubMedGoogle ScholarCrossref
46.
Fries  GR, McCalla  G, Levitt  MA, Cordova  R.  A prospective comparison of paramedic judgment and the trauma triage rule in the prehospital setting.  Ann Emerg Med. 1994;24(5):885-889. doi:10.1016/S0196-0644(94)70207-1PubMedGoogle ScholarCrossref
47.
Newgard  CD, Kampp  M, Nelson  M,  et al; WESTRN Investigators.  Deciphering the use and predictive value of “emergency medical services provider judgment” in out-of-hospital trauma triage: a multisite, mixed methods assessment.  J Trauma Acute Care Surg. 2012;72(5):1239-1248. doi:10.1097/TA.0b013e3182468b51PubMedGoogle ScholarCrossref
48.
Harmsen  AM, Giannakopoulos  GF, Moerbeek  PR, Jansma  EP, Bonjer  HJ, Bloemers  FW.  The influence of prehospital time on trauma patients outcome: a systematic review.  Injury. 2015;46(4):602-609. doi:10.1016/j.injury.2015.01.008PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    February 6, 2019

    Development and Validation of a Prediction Model for Prehospital Triage of Trauma Patients

    Author Affiliations
    • 1Department of Traumatology, University Medical Center Utrecht, Utrecht, the Netherlands
    • 2Department of Surgery, Elisabeth-TweeSteden Hospital, Tilburg, the Netherlands
    • 3Utrecht Traumacenter, Utrecht, the Netherlands
    • 4Regional Ambulance Facility Utrecht, Utrecht Regional Ambulance Service, Utrecht, the Netherlands
    • 5Network Emergency Care Brabant, Brabant Trauma Registry, Tilburg, the Netherlands
    • 6Clinical Research Unit, Academic Medical Center, Amsterdam, the Netherlands
    • 7SimQuest Solutions Inc, Annapolis, Maryland
    • 8Section of Surgery, Uniformed Services University of the Health Sciences, Bethesda, Maryland
    • 9Department of Traumatology, Luzerner Kantonsspital, Luzern, Switzerland
    • 10Department of Surgery, Diakonessenhuis Utrecht, Utrecht, the Netherlands
    JAMA Surg. 2019;154(5):421-429. doi:10.1001/jamasurg.2018.4752
    Key Points

    Question  Can a new prehospital prediction model for trauma triage lower the undertriage rate to approximately 10%, with a maximum overtriage rate of 50%?

    Findings  In this multicenter cohort study that included 4950 patients with trauma, 8 highly significant predictors associated with injury severity were selected for the prediction model. The new prehospital trauma triage prediction model was externally validated and may lower the undertriage rate to 11.2% with an overtriage rate of 50.0%.

    Meaning  This prediction model can be integrated in a mobile app for emergency medical services professionals to improve the prehospital trauma triage.

    Abstract

    Importance  Prehospital trauma triage protocols are used worldwide to get the right patient to the right hospital and thereby improve the chance of survival and avert lifelong disabilities. The American College of Surgeons Committee on Trauma set target levels for undertriage rates of less than 5%. None of the existing triage protocols has been able to achieve this target in isolation.

    Objective  To develop and validate a new prehospital trauma triage protocol to improve current triage rates.

    Design, Setting, and Participants  In this multicenter cohort study, all patients with trauma who were 16 years and older and transported to a trauma center in 2 different regions of the Netherlands were included in the analysis. Data were collected from January 1, 2012, through June 30, 2014, in the Central Netherlands region for the design data cohort and from January 1 through December 31, 2015, in the Brabant region for the validation cohort. Data were analyzed from May 3, 2017, through July 19, 2018.

    Main Outcomes and Measures  A new prediction model was developed in the Central Netherlands region based on prehospital predictors associated with severe injury. Severe injury was defined as an Injury Severity Score greater than 15. A full-model strategy with penalized maximum likelihood estimation was used to construct a model with 8 predictors that were chosen based on clinical reasoning. Accuracy of the developed prediction model was assessed in terms of discrimination and calibration. The model was externally validated in the Brabant region.

    Results  Using data from 4950 patients with trauma from the Central Netherlands region for the design data set (58.3% male; mean [SD] age, 47 [21] years) and 6859 patients for the validation Brabant region (52.2% male; mean [SD] age, 51 [22] years), the following 8 significant predictors were selected for the prediction model: age; systolic blood pressure; Glasgow Coma Scale score; mechanism criteria; penetrating injury to the head, thorax, or abdomen; signs and/or symptoms of head or neck injury; expected injury in the Abbreviated Injury Scale thorax region; and expected injury in 2 or more Abbreviated Injury Scale regions. The prediction model showed a C statistic of 0.823 (95% CI, 0.813-0.832) and good calibration. The cutoff point with a minimum specificity of 50.0% (95% CI, 49.3%-50.7%) led to a sensitivity of 88.8% (95% CI, 87.5%-90.0%). External validation showed a C statistic of 0.831 (95% CI, 0.814-0.848) and adequate calibration.

    Conclusions and Relevance  The new prehospital trauma triage prediction model may lower undertriage rates to approximately 10% with an overtriage rate of 50%. The next step should be to implement this prediction model with the use of a mobile app for emergency medical services professionals.

    Introduction

    In first-world countries, systems of trauma care substantially reduce mortality associated with injury.1-3 Multiple studies2,3 focused on optimizing such trauma systems by balancing timely access to expert care, the ability of health care professionals and teams to attain and sustain the necessary expertise, and the cost-effectiveness of the overall trauma system. Fundamental to the trauma system is prehospital trauma triage, the goal of which is to identify at-risk patients and provide early and resuscitative care while transporting the patient to the highest appropriate level of care.4 Identification of severely injured patients is challenging. Only 0.5% of those injured are severely injured.4 Other challenges include assessment of the incident scene and the patient’s physiological state and risk of deterioration, identification of obvious injuries, and consideration of adjuvant factors, such as age.

    Emergency medical services (EMS) professionals must differentiate between patients on the scene, often in adverse situations, without use of advanced diagnostic tools. Therefore, the EMS professionals are often forced to perform the prehospital trauma triage based on incomplete data. The importance of prehospital trauma triage cannot be understated; a structured and reliable process is crucial.

    Worldwide, protocols are used to help identify severely injured patients. However, none of the existing protocols can achieve the recommended triage rates.5-7 All are simplistic and static tools, whereas patients are dynamic, and more advanced methods are available to use on the scene, such as a prediction model in a mobile app.8,9

    Undertriage occurs when severely injured patients are not transported to a higher-level trauma center and results in delayed care and increased mortality and morbidity.1,2 Overtriage occurs when patients without severe injuries are taken to a higher-level trauma center, often incurring preventable cost and resource consumption.3,10 In the Netherlands, level I trauma centers are considered higher-level trauma centers and levels II and III are considered lower-level trauma centers. The American College of Surgeons Committee on Trauma set target levels of undertriage at less than 5% and overtriage as great as 25% to 35%.11 The National Health Institute of the Netherlands recommends an undertriage rate of less than 10%.12 A target for the overtriage rate was not set; the overtriage rate, however, should depend on the regional circumstances and could be as great as 50%.

    In the Netherlands, a protocol based on the Field Triage Decision Scheme is used nationwide (eMethods in the Supplement).13 The mean undertriage rate was 33% in 2015.14 A recent study15 showed an undertriage rate of 22% and an overtriage rate of 31% in a single inclusive trauma system. At present, existing protocols achieve undertriage rates ranging from 22% to 44%, with overtriage rates ranging from 11% to 22% in a general trauma population.6,16,17 The aim of the present study was therefore to develop and validate a new prehospital trauma triage prediction model that attempts to lower the undertriage rate to approximately 10%, with a maximum overtriage rate of 50%.

    Methods
    Study Design and Setting

    In the Netherlands, each ambulance service serves a region. In this prospective, multicenter cohort study, all adult patients with trauma and all trauma centers in 2 different regions were included. Data from the Central Netherlands region were used to develop a diagnostic prediction model that was externally validated using the data of the Brabant region. This study was judged by the Medical Ethical Committee of University Medical Center Utrecht, Utrecht, the Netherlands, as not subject to the Medical Research Involving Human Subjects Act and therefore exempt from the need for informed consent.

    The Central Netherlands region has 1 level I trauma center, the University Medical Center Utrecht, and 7 level II or III trauma centers. The region covers 535 square miles and has 1.2 million residents. Brabant has 1 level I trauma center, Elisabeth-TweeSteden Hospital, Tilburg, and 10 level II or III trauma centers. The region covers 1343 square miles and has 1.7 million residents. In both regions, the Dutch National Ambulance Care Protocol is used (eMethods in the Supplement).13 Patients transported across the borders of these regions were excluded because data were unavailable.

    Patients

    All patients with trauma who were 16 years or older, determined to be highest priority (with flashing lights and siren) by the dispatch center, and transported to a trauma center in 1 of the 2 regions underwent evaluation. In the Central Netherlands, patients were included from January 1, 2012, through June 30, 2014.15 In Brabant, patients were included from January 1 through December 31, 2015.

    Outcomes and Definitions

    Independent predictors associated with severe injury were identified to create a prediction model consisting of a limited number of variables. A severely injured patient was defined as a patient with an Injury Severity Score (ISS) greater than 15 (range, 0-75).

    Undertriage was defined as the proportion of severely injured patients transported to a level II or III trauma center. Overtriage was defined as the proportion of patients without severe injuries transported to a level I trauma center. The protocol allows for these patients to be transported to a level I trauma center if it happens to be the nearest hospital.

    Data Sources

    Prehospital reports from EMS professionals were prospectively collected and included patient demographics, vital signs, description of the trauma mechanism, on-scene physical examination data (including suspected injured body region), and the receiving hospital. The body region suspected of injury by EMS professionals was divided into the head and neck, face, thorax, abdomen, extremities, or skin and others. These regions were chosen based on the categorization of the Abbreviated Injury Scale (AIS) regions that make up the ISS.

    The Dutch National Trauma Database registered injuries for all patients admitted to a hospital. The Central Netherlands patient data were also extracted from the electronic patient documentation for patients discharged directly from the emergency department (ED). The injuries were recorded after discharge or 30 days after admission and coded by trained data managers in both regions (including E.A.J.v.R. and F.J.V.). Before 2015, all injuries were coded using the AIS 1990 Update 98 (AIS98); from 2015 and on, the AIS 2005 Update 2008 (AIS08) was used. Therefore, the injuries were coded according to the AIS98 for the Central Netherlands database and according to the AIS08 for the Brabant database. The ISS was calculated based on AIS scores to determine injury severity.

    Missing Data

    Missing data were analyzed and considered to be missing at random. Multiple imputation by chained equations was used for both regions separately to account for missing prehospital variables rather than deleting patients who had the most data available.18 Missing values were imputed based on all other predictors as well as the ISS. For both regions, respiratory rate, systolic blood pressure, oxygen saturation level, and Glasgow Coma Scale (GCS) score were imputed. Pulse was imputed for the Central Netherlands region only because this variable was missing in the Brabant data. For the Brabant region, ISS was only available for admitted patients. An ISS less than 15 was assumed for patients discharged from the ED because a previous study15 demonstrated that all severely injured patients (ISS >15) were admitted or died in the ED.

    Statistical Analysis

    Data were analyzed from May 3, 2017, through July 19, 2018. Frequencies with percentages were used to describe nominal and ordinal variables, whereas means and SDs were used to describe continuous variables. Bivariable binary logistic regression was used to explore potential predictors associated with severe injury (ISS >15). Analyses were performed on 5 imputed data sets independently and pooled using Rubin’s rules, if applicable.19

    To ensure practical applicability, the maximum number of predictors was limited to 8. A full-model strategy with clinically relevant variables was used to develop the prediction model. To improve the accuracy for future patients and other regions, penalized maximum likelihood estimation was used.20 Penalized maximum likelihood estimation is a rigorous estimation method that potentially results in better generalizability, model reduction, and differential shrinkage of coefficients.21 The functional forms of all continuous predictors were defined before modeling. Restricted cubic splines were used to model nonlinear predictors.

    The performance of the final prediction model was expressed in terms of discrimination and calibration. Discriminative value was quantified by the C statistic. The receiver operating characteristic curve was plotted and a predefined value for specificity—an overtriage rate of 50%—was used to determine a cutoff point. Calibration was graphically assessed using a calibration plot.

    The final model was externally validated with the Brabant data. Owing to heterogeneity between trauma regions (eg, prevalence of severe injury), the model needed to be recalibrated by updating the intercept for the Brabant region.22,23 Calibration and discrimination of the prediction model were assessed in the validation process. Bootstrapped 95% CIs based on percentiles and 1000 resamples were calculated for C statistics and accuracy metrics. Two-sided P values were calculated using univariable logistic regression, and P < .05 indicated significance. All statistical analyses were performed using R software (version 3.2.4; R Foundation for Statistical Computing).24

    Results
    Design Data Set

    A total of 4950 adult patients with trauma were included for the Central Netherlands region constituting the design data set. To account for missing data, multiple imputation was used for respiratory rate in 324 (6.5%), systolic blood pressure in 345 (7.0%), oxygen saturation level in 662 (13.4%), and GCS score in 230 (4.6%). Mean patient age was 47 (21) years; 2887 (58.3%) were male and 2063 (41.7%) were female; and 435 (8.8%) had an ISS greater than 15 (Table 1). In this cohort, the undertriage rate was 21.6%; the overtriage rate, 30.6%.

    Validation Data Set

    In the Brabant region, a total of 6859 adult trauma patients were included in the validation data set. To account for missing data, multiple imputation was used for respiratory rate in 1973 patients (28.8%), systolic blood pressure in 1145 (16.7%), oxygen saturation in 1431 (20.9%), and GCS score in 288 (4.2%). Mean patient age was 51 (22) years; 3583 (52.2%) were male and 3276 (47.8%) were female; and 165 (2.4%) had an ISS greater than 15 (Table 1). The ISS was only available for the admitted patients. In this cohort, the undertriage rate was 27.3%; the overtriage rate, 26.3%.

    Model Development and Specification

    To develop the prediction model, 43 potential prehospital predictors from the Central Netherlands database were explored using bivariable analysis (Table 2). The following 8 predictors were chosen for the final model, based on clinical reasoning: age; systolic blood pressure; GCS score; mechanism criteria; penetrating injury to the head, thorax, or abdomen; signs and/or symptoms of head or neck injury; expected injury in the AIS thorax region; and expected injury in 2 or more AIS regions. The optimal cutoff point with a minimum specificity of 50.0% (95% CI, 49.3%-50.7%) led to a sensitivity of 88.8% (95% CI, 87.5%-90.0%).

    Model Performance

    This prediction model resulted in an undertriage rate of 11.2% (Δ difference, 10.4%) and an overtriage rate of 50.0% (Δ difference, 19.4%) for the Central Netherlands region. Robust estimation using penalized maximum likelihood showed that all variables in the model were significant independent predictors (Table 3). The model had a good discrimination with a C statistic of 0.823 (95% CI, 0.813-0.832). The recalibration method led to an intercept of 0.894 for the Brabant region. External validation using the Brabant region database showed that the model with the new intercept was well calibrated (Figure 1) and had a good discrimination with a C statistic of 0.831 (95% CI, 0.814-0.848).

    Discussion

    In this prospective, multicenter cohort study, we present a ready-to-use prehospital trauma triage prediction model for the presence of severe injury on the scene in patients with trauma. The model performed well in the derivation set and in external validation. To our knowledge, this is the first externally validated protocol showing acceptable triage rates, with a potential undertriage rate of 11.2% and overtriage rate of 50.0% depending on the chosen threshold.

    Worldwide, triage protocols are based on a simple flowchart that includes vital signs, injury type, and mechanism of injury criteria.6,25-28 These triage protocols are simplistic and static: transport to a higher-level trauma center should be considered if just 1 criterion is present. In reality, some factors have a greater association with injury severity than others, and the combination of factors indicates the need for higher-level trauma care. In addition, current protocols often use cutoff points for continuous variables, whereas the prediction model uses coefficients for each predictor to represent each variable’s distinct association with injury severity to increase predictive ability.

    The prediction model was based on the following 3 key elements: (1) inclusion of all adult patients with trauma transported by an ambulance to (2) all trauma centers of an entire geographic region, with (3) prehospital variables measured on the scene by EMS professionals. Previous studies have attempted to develop protocols but have not included these 3 key elements. For example, Dihn and colleagues27 developed a triage protocol based on patients taken to a higher-level trauma center only, thereby excluding the undertriaged patients. Others included admitted patients only,29,30 thus excluding patients discharged from the ED or the potentially overtriaged patients. These models would not be reliable in a general trauma population because they fail to include the patient populations in which improvement is of utmost importance: the undertriaged and overtriaged patients.

    Eight predictors were included in the model based on clinical reasoning to achieve the best accuracy while keeping it user friendly without too many factors. Age was included because previous studies showed a higher undertriage rate in elderly patients.6,31-33 Two continuous predictors of the condition of a patient are systolic blood pressure and GCS score.34-36 Penetrating injury is an obvious predictor associated with potential severe trauma. The brain and thorax are 2 of the most commonly injured body regions, with both associated with a high prevalence of severe injury.37-39 Also, multiregional injury was previously found as a strong predictor associated with severe injury.27 Therefore, these 8 predictors were included in the current model. The prediction model resulted in a undertriage rate of 11.2% and an overtriage rate of 50.0% in the Central Netherlands region.

    After penalized estimation, the updated diagnostic prediction model was externally validated with data from the Brabant region. The prediction model requires an update primarily owing to the difference in prevalence of severe injury, resulting in a difference in baseline risk.40 To account for this difference, the constant value (intercept) in the equation was altered. The constant value can be altered for other regions before applying the prediction model based on prehospital and hospital data of the specific region. External validation—with the altered intercept—showed good discrimination and calibration, indicating that the prediction model would likely be accurate in a region that is heterogeneous with respect to population, prevalence of severe injury, and mechanism of injury. In addition, the injuries were coded differently in both regions: AIS98 was used in the Central Netherlands and AIS08 in the Brabant regions. When using the AIS08, the overall ISS is lower compared with injuries scored with the AIS98.41 External validation using the AIS08 showed that the prediction model functions well using the most recent AIS.

    The prediction model did not achieve the goal of an undertriage rate of less than 5% as targeted by the American College of Surgeons Committee on Trauma.42 However, the model is a significant improvement compared with the existing protocols.5,6,16,43 Whether further improvement of undertriage is achievable by solely improving the protocol is unclear. Previous studies have shown that addition of the judgment of EMS professionals can be useful in the identification of severely injured patients.44-47 In this study, EMS professional judgment could not be quantified because it was not recorded. Including EMS professional judgment may improve the undertriage rate even more as well as increase adherence to the protocol. Other factors could have influenced transport decisions, such as geographical distance to a higher-level trauma center. Although distances are relatively small in the Netherlands compared with other countries, distance could have influenced the destination decision, especially in Brabant, because distances are larger in this region. Unfortunately, the effect of distance on the triage quality could not be evaluated in this study, and the distance at which deviation to the nearest hospital is best remains unclear.48

    In practice, it is not feasible to calculate the risk of severe injury based on an equation that must be memorized and applied on the scene. This problem could be solved by implementing the prediction model in a mobile app; such triage tools are increasingly being developed and used in the prehospital process.8,9 This mobile app includes every variable, calculates the chance of severe injury, and gives advice on where to transport the patient, which is much more practical for EMS professionals compared with an equation. With a mobile device available on every ambulance, the EMS professionals can calculate the risk of severe injury using the prediction model in the app to decide quickly and more accurately where to transport the patient. The EMS professional judgment could be included in the app. A mobile app with the described prediction model is currently being implemented in different regions in the Netherlands. The implementation aims at reducing undertriage specifically.

    Limitations

    This study has several limitations. In the final prediction model, missing data were present for the systolic blood pressure and GCS variables. The data were considered to be missing at random, and multiple imputation was used to minimize selection bias. Second, for Brabant, the ISS was only available for the patients who were admitted or who died in the ED. In the Central Netherlands region—where the ISS was available for all patients—all severely injured patients (ISS >15) were admitted or died in the ED. Accordingly, an ISS less than 15 was assumed for patients discharged from the ED. Last, debate remains on the most accurate definition of a severely injured patient. Legitimate classification is difficult and dependent on multiple factors, such as regional circumstances and trauma center level. An ISS greater than 15 might not represent all patients in need of higher-level trauma center resources. However, an ISS greater than 15 is the most used surrogate marker for a severely injured patient when evaluating prehospital trauma triage; therefore, we chose this variable to define a severely injured patient.5

    Future research should focus on the validation of the prediction model in other regions. Differences in incidence of severely injured patients, and consequently baseline risk, can be large, and therefore different baseline risks should be determined in other populations. A possible solution is to validate the prediction model in other regions using our methods.

    The mobile app has been developed and is currently being implemented in the Netherlands (Figure 2). In this mobile app, the equation is integrated in addition to EMS professional judgment. This combination could be optimal to improve triage rates. In addition, the combination could give insight in the value of EMS professional judgment.

    Conclusions

    To our knowledge, this study is the first to develop and validate a prehospital trauma triage protocol based on all adult patients with trauma to be transported to a trauma center within a region and that can lower the undertriage rate to 11.2%, with an overtriage rate of 50.0%, from 21.6% and 30.6%, respectively. This protocol, based on an equation in which each predictor has its own coefficient, can be implemented with a mobile app for EMS professionals and could be of great help to lower undertriage rates.

    Back to top
    Article Information

    Accepted for Publication: September 14, 2018.

    Corresponding Author: Eveline A. J. van Rein, MD, Department of Traumatology, University Medical Center Utrecht, Ste G04.228, Heidelberglaan 100, 3584 CX, Utrecht, the Netherlands (evelinevanrein@gmail.com).

    Published Online: February 6, 2019. doi:10.1001/jamasurg.2018.4752

    Author Contributions: Dr van Rein had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: van Rein, van der Sluijs, Voskens, Lansink, Houwert, Lichtveld, Leenen, van Heijl.

    Acquisition, analysis, or interpretation of data: van der Sluijs, Voskens, Lansink, de Jongh, Dijkgraaf, Champion, Beeres, van Heijl.

    Drafting of the manuscript: van Rein, van der Sluijs, Lichtveld, Beeres, van Heijl.

    Critical revision of the manuscript for important intellectual content: van der Sluijs, Voskens, Lansink, Houwert, de Jongh, Dijkgraaf, Champion, Beeres, Leenen, van Heijl.

    Statistical analysis: van Rein, van der Sluijs, Dijkgraaf, van Heijl.

    Obtained funding: van Heijl.

    Administrative, technical, or material support: van Rein, Voskens, Lansink, de Jongh, Beeres, van Heijl.

    Supervision: Voskens, Lansink, Houwert, Lichtveld, Beeres, Leenen, van Heijl.

    Conflict of Interest Disclosures: None reported.

    Additional Contributions: The authors thank Jillian Gruber, MD, School of Medicine and Dentistry, University of Rochester Medical Center, Rochester, New York, for copy editing the manuscript. She was compensated for this work.

    References
    1.
    MacKenzie  EJ, Rivara  FP, Jurkovich  GJ,  et al.  A national evaluation of the effect of trauma-center care on mortality.  N Engl J Med. 2006;354(4):366-378. doi:10.1056/NEJMsa052049PubMedGoogle ScholarCrossref
    2.
    Staudenmayer  K, Weiser  TG, Maggio  PM, Spain  DA, Hsia  RY.  Trauma center care is associated with reduced readmissions after injury.  J Trauma Acute Care Surg. 2016;80(3):412-416. doi:10.1097/TA.0000000000000956PubMedGoogle ScholarCrossref
    3.
    Newgard  CD, Staudenmayer  K, Hsia  RY,  et al.  The cost of overtriage: more than one-third of low-risk injured patients were taken to major trauma centers.  Health Aff (Millwood). 2013;32(9):1591-1599. doi:10.1377/hlthaff.2012.1142PubMedGoogle ScholarCrossref
    4.
    Champion  HR, Lombardo  LV, Wade  CE, Kalin  EJ, Lawnick  MM, Holcomb  JB.  Time and place of death from automobile crashes: research endpoint implications.  J Trauma Acute Care Surg. 2016;81(3):420-426. doi:10.1097/TA.0000000000001124PubMedGoogle ScholarCrossref
    5.
    van Rein  EAJ, Houwert  RM, Gunning  AC, Lichtveld  RA, Leenen  LPH, van Heijl  M.  Accuracy of prehospital triage protocols in selecting severely injured patients: a systematic review.  J Trauma Acute Care Surg. 2017;83(2):328-339. doi:10.1097/TA.0000000000001516PubMedGoogle ScholarCrossref
    6.
    Newgard  CD, Fu  R, Zive  D,  et al.  Prospective validation of the National Field Triage Guidelines for identifying seriously injured persons.  J Am Coll Surg. 2016;222(2):146-158.e2. doi:10.1016/j.jamcollsurg.2015.10.016PubMedGoogle ScholarCrossref
    7.
    Newgard  CD, Zive  D, Holmes  JF,  et al; WESTRN investigators.  A multisite assessment of the American College of Surgeons Committee on Trauma field triage decision scheme for identifying seriously injured children and adults.  J Am Coll Surg. 2011;213(6):709-721. doi:10.1016/j.jamcollsurg.2011.09.012PubMedGoogle ScholarCrossref
    8.
    Lewis  TL, Fothergill  RT, Karthikesalingam  A; Aneurysm-FILTR Study Group.  Ambulance smartphone tool for field triage of ruptured aortic aneurysms (FILTR): study protocol for a prospective observational validation of diagnostic accuracy.  BMJ Open. 2016;6(10):e011308. doi:10.1136/bmjopen-2016-011308PubMedGoogle ScholarCrossref
    9.
    Nogueira  RG, Silva  GS, Lima  FO,  et al.  The FAST-ED app: a smartphone platform for the field triage of patients with stroke.  Stroke. 2017;48(5):1278-1284. doi:10.1161/STROKEAHA.116.016026PubMedGoogle ScholarCrossref
    10.
    Faul  M, Xu  L, Sasser  SM.  Hospitalized traumatic brain injury: low trauma center utilization and high interfacility transfers among older adults.  Prehosp Emerg Care. 2016;20(5):594-600. doi:10.3109/10903127.2016.1149651PubMedGoogle ScholarCrossref
    11.
    Sasser  SM, Hunt  RC, Faul  M,  et al; Centers for Disease Control and Prevention (CDC).  Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011.  MMWR Recomm Rep. 2012;61(RR-1):1-20.PubMedGoogle Scholar
    12.
    The Netherlands Health Care Institute. Urgency must be good—indicators and standards for 6 emergency care indications. https://www.zorginstituutnederland.nl/publicaties/rapport/2015/12/16/spoed-moet-goed---indicatoren-en-normen-voor-zes-spoedzorgindicaties. Published July 12, 2015. Accessed October 31, 2018.
    13.
    Ambulancezorg Nederland. National Ambulance Care Protocol (LPA). https://www.ambulancezorg.nl/themas/kwaliteit-van-zorg/protocollen-en-richtlijnen/landelijk-protocol-ambulancezorg. Accessed October 31, 2018.
    14.
    Landelijk Netwerk Acute Zorg. Landelijke Traumaregistratie 2011-2015: Rapportage Nederland. 2016. Landelijk Netwerk Acute Zorg. http://www.lnaz.nl/cms/LTR_landelijk_jaarrapport_2011-2015.pdf. Published November 2, 2016. Accessed October 31, 2018.
    15.
    Voskens  FJ, van Rein  EAJ, van der Sluijs  R,  et al.  Accuracy of prehospital triage in selecting severely injured trauma patients.  JAMA Surg. 2018;153(4):322-327. doi:10.1001/jamasurg.2017.4472PubMedGoogle ScholarCrossref
    16.
    Bond  RJ, Kortbeek  JB, Preshaw  RM.  Field trauma triage: combining mechanism of injury with the prehospital index for an improved trauma triage tool.  J Trauma. 1997;43(2):283-287. doi:10.1097/00005373-199708000-00013PubMedGoogle ScholarCrossref
    17.
    Meisler  R, Thomsen  AB, Abildstrøm  H,  et al.  Triage and mortality in 2875 consecutive trauma patients.  Acta Anaesthesiol Scand. 2010;54(2):218-223. doi:10.1111/j.1399-6576.2009.02075.xPubMedGoogle ScholarCrossref
    18.
    van Buuren  S, Groothuis-Oudshoorn  K.  mice: multivariate imputation by chained equations in R.  J Stat Softw. 2011;45(3):40-49.Google Scholar
    19.
    Marshall  A, Altman  DG, Holder  RL, Royston  P.  Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines.  BMC Med Res Methodol. 2009;9:57-64. doi:10.1186/1471-2288-9-57PubMedGoogle ScholarCrossref
    20.
    Harrell  FE.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001. doi:10.1007/978-1-4757-3462-1
    21.
    Moons  KGM, Donders  ART, Steyerberg  EW, Harrell  FE.  Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example.  J Clin Epidemiol. 2004;57(12):1262-1270. doi:10.1016/j.jclinepi.2004.01.020PubMedGoogle ScholarCrossref
    22.
    Vergouwe  Y, Nieboer  D, Oostenbrink  R,  et al.  A closed testing procedure to select an appropriate method for updating prediction models.  Stat Med. 2017;36(28):4529-4539. doi:10.1002/sim.7179PubMedGoogle ScholarCrossref
    23.
    Janssen  KJM, Moons  KGM, Kalkman  CJ, Grobbee  DE, Vergouwe  Y.  Updating methods improved the performance of a clinical prediction model in new patients.  J Clin Epidemiol. 2008;61(1):76-86. doi:10.1016/j.jclinepi.2007.04.018PubMedGoogle ScholarCrossref
    24.
    R Development Core Team. R: a language and environment for statistical computing [serial online]. Vienna, Austria: R Foundation for Statistical Computing; 2016. https://www.r-project.org/about.html. Accessed October 31, 2018.
    25.
    van Laarhoven  JJ, Lansink  KW, van Heijl  M, Lichtveld  RA, Leenen  LP.  Accuracy of the field triage protocol in selecting severely injured patients after high energy trauma.  Injury. 2014;45(5):869-873. doi:10.1016/j.injury.2013.12.010PubMedGoogle ScholarCrossref
    26.
    Hamada  SR, Gauss  T, Duchateau  FX,  et al.  Evaluation of the performance of French physician-staffed emergency medical service in the triage of major trauma patients.  J Trauma Acute Care Surg. 2014;76(6):1476-1483. doi:10.1097/TA.0000000000000239PubMedGoogle ScholarCrossref
    27.
    Dinh  MM, Bein  KJ, Oliver  M, Veillard  AS, Ivers  R.  Refining the trauma triage algorithm at an Australian major trauma centre: derivation and internal validation of a triage risk score.  Eur J Trauma Emerg Surg. 2014;40(1):67-74. doi:10.1007/s00068-013-0315-1PubMedGoogle ScholarCrossref
    28.
    Rubenson Wahlin  R, Ponzer  S, Skrifvars  MB, Lossius  HM, Castrén  M.  Effect of an organizational change in a prehospital trauma care protocol and trauma transport directive in a large urban city: a before and after study.  Scand J Trauma Resusc Emerg Med. 2016;24:26. doi:10.1186/s13049-016-0218-3PubMedGoogle ScholarCrossref
    29.
    Champion  HR, Sacco  WJ, Copes  WS, Gann  DS, Gennarelli  TA, Flanagan  ME.  A revision of the Trauma Score.  J Trauma. 1989;29(5):623-629. doi:10.1097/00005373-198905000-00017PubMedGoogle ScholarCrossref
    30.
    Ocak  G, Sturms  LM, Hoogeveen  JM, Le Cessie  S, Jukema  GN.  Prehospital identification of major trauma patients.  Langenbecks Arch Surg. 2009;394(2):285-292. doi:10.1007/s00423-008-0340-4PubMedGoogle ScholarCrossref
    31.
    Vassar  MJ, Holcroft  JJ, Knudson  MM, Kizer  KW.  Fractures in access to and assessment of trauma systems.  J Am Coll Surg. 2003;197(5):717-725. doi:10.1016/S1072-7515(03)00749-XPubMedGoogle ScholarCrossref
    32.
    Chang  DC, Bass  RR, Cornwell  EE, Mackenzie  EJ.  Undertriage of elderly trauma patients to state-designated trauma centers.  Arch Surg. 2008;143(8):776-781. doi:10.1001/archsurg.143.8.776PubMedGoogle ScholarCrossref
    33.
    Ichwan  B, Darbha  S, Shah  MN,  et al.  Geriatric-specific triage criteria are more sensitive than standard adult criteria in identifying need for trauma center care in injured older adults.  Ann Emerg Med. 2015;65(1):92-100.e3. doi:10.1016/j.annemergmed.2014.04.019PubMedGoogle ScholarCrossref
    34.
    Teasdale  G, Maas  A, Lecky  F, Manley  G, Stocchetti  N, Murray  G.  The Glasgow Coma Scale at 40 years: standing the test of time.  Lancet Neurol. 2014;13(8):844-854. doi:10.1016/S1474-4422(14)70120-6PubMedGoogle ScholarCrossref
    35.
    Wong  TH, Krishnaswamy  G, Nadkarni  NV,  et al.  Combining the new injury severity score with an anatomical polytrauma injury variable predicts mortality better than the new Injury Severity Score and the Injury Severity Score: a retrospective cohort study.  Scand J Trauma Resusc Emerg Med. 2016;24:25. doi:10.1186/s13049-016-0215-6PubMedGoogle ScholarCrossref
    36.
    Lefering  R, Huber-Wagner  S, Nienaber  U, Maegele  M, Bouillon  B.  Update of the trauma risk adjustment model of the TraumaRegister DGU™: the Revised Injury Severity Classification, version II.  Crit Care. 2014;18(5):476. doi:10.1186/s13054-014-0476-2PubMedGoogle ScholarCrossref
    37.
    Scheetz  LJ.  Comparison of type and severity of major injuries among undertriaged and correctly triaged older patients.  J Emerg Med. 2012;43(6):1020-1028. doi:10.1016/j.jemermed.2011.09.036PubMedGoogle ScholarCrossref
    38.
    Schoell  SL, Doud  AN, Weaver  AA,  et al.  Development of a time sensitivity score for frequently occurring motor vehicle crash injuries.  J Am Coll Surg. 2015;220(3):305-312.e3. doi:10.1016/j.jamcollsurg.2014.11.022PubMedGoogle ScholarCrossref
    39.
    Cole  TB.  Global road safety crisis remedy sought: 1.2 million killed, 50 million injured annually.  JAMA. 2004;291(21):2531-2532. doi:10.1001/jama.291.21.2531PubMedGoogle ScholarCrossref
    40.
    Leeflang  MM, Bossuyt  PM, Irwig  L.  Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis.  J Clin Epidemiol. 2009;62(1):5-12. doi:10.1016/j.jclinepi.2008.04.007PubMedGoogle ScholarCrossref
    41.
    Barnes  J, Hassan  A, Cuerden  R, Cookson  R, Kohlhofer  J.  Comparison of injury severity between AIS 2005 and AIS 1990 in a large injury database.  Ann Adv Automot Med. 2009;53:83-89.PubMedGoogle Scholar
    42.
    American College of Surgeons Committee on Trauma.  Resources for Optimal Care of the Injured Patient 2006. Chicago, IL: American College of Surgeons; 2006.
    43.
    Hedges  JR, Feero  S, Moore  B, Haver  DW, Shultz  B.  Comparison of prehospital trauma triage instruments in a semirural population.  J Emerg Med. 1987;5(3):197-208. doi:10.1016/0736-4679(87)90179-XPubMedGoogle ScholarCrossref
    44.
    Newgard  CD, Nelson  MJ, Kampp  M,  et al.  Out-of-hospital decision making and factors influencing the regional distribution of injured patients in a trauma system.  J Trauma. 2011;70(6):1345-1353. doi:10.1097/TA.0b013e3182191a1bPubMedGoogle ScholarCrossref
    45.
    Emerman  CL, Shade  B, Kubincanek  J.  A comparison of EMT judgment and prehospital trauma triage instruments.  J Trauma. 1991;31(10):1369-1375. doi:10.1097/00005373-199110000-00009PubMedGoogle ScholarCrossref
    46.
    Fries  GR, McCalla  G, Levitt  MA, Cordova  R.  A prospective comparison of paramedic judgment and the trauma triage rule in the prehospital setting.  Ann Emerg Med. 1994;24(5):885-889. doi:10.1016/S0196-0644(94)70207-1PubMedGoogle ScholarCrossref
    47.
    Newgard  CD, Kampp  M, Nelson  M,  et al; WESTRN Investigators.  Deciphering the use and predictive value of “emergency medical services provider judgment” in out-of-hospital trauma triage: a multisite, mixed methods assessment.  J Trauma Acute Care Surg. 2012;72(5):1239-1248. doi:10.1097/TA.0b013e3182468b51PubMedGoogle ScholarCrossref
    48.
    Harmsen  AM, Giannakopoulos  GF, Moerbeek  PR, Jansma  EP, Bonjer  HJ, Bloemers  FW.  The influence of prehospital time on trauma patients outcome: a systematic review.  Injury. 2015;46(4):602-609. doi:10.1016/j.injury.2015.01.008PubMedGoogle ScholarCrossref
    ×