[Skip to Content]
[Skip to Content Landing]
Figure 1.  Patient Flowchart
Patient Flowchart

Abbreviations: ED, emergency department; LOC, loss of consciousness; LWBS, left without being seen (left before physician assessment).

Figure 2.  Calibration Plot of Expected vs Observed Risk in the Validation Cohort
Calibration Plot of Expected vs Observed Risk in the Validation Cohort

Vertical lines represent the 95% CI. The marks on the lower horizontal line represent the frequency distribution of patients across expected risk categories. ROC indicates receiver operating characteristic curve.

Table 1.  The Canadian Syncope Risk Score
The Canadian Syncope Risk Score
Table 2.  Characteristics, Emergency Department Management, and Outcomes
Characteristics, Emergency Department Management, and Outcomes
Table 3.  Thirty-Day Serious Outcomes for Each Canadian Syncope Risk Score Category During the Validation Phase
Thirty-Day Serious Outcomes for Each Canadian Syncope Risk Score Category During the Validation Phase
1.
Probst  MA, Kanzaria  HK, Gbedemah  M, Richardson  LD, Sun  BC.  National trends in resource utilization associated with ED visits for syncope.  Am J Emerg Med. 2015;33(8):998-1001. doi:10.1016/j.ajem.2015.04.030PubMedGoogle ScholarCrossref
2.
Sun  BC, Emond  JA, Camargo  CA  Jr.  Characteristics and admission patterns of patients presenting with syncope to US emergency departments, 1992-2000.  Acad Emerg Med. 2004;11(10):1029-1034. doi:10.1016/j.annemergmed.2004.07.016PubMedGoogle Scholar
3.
Brignole  M, Moya  A, de Lange  FJ,  et al; ESC Scientific Document Group.  2018 ESC guidelines for the diagnosis and management of syncope.  Eur Heart J. 2018;39(21):1883-1948. doi:10.1093/eurheartj/ehy037PubMedGoogle ScholarCrossref
4.
Sun  BC, Thiruganasambandamoorthy  V, Cruz  JD; Consortium to Standardize ED Syncope Risk Stratification Reporting.  Standardized reporting guidelines for emergency department syncope risk-stratification research.  Acad Emerg Med. 2012;19(6):694-702. doi:10.1111/j.1553-2712.2012.01375.xPubMedGoogle ScholarCrossref
5.
Quinn  J, McDermott  D, Stiell  I, Kohn  M, Wells  G.  Prospective validation of the San Francisco Syncope Rule to predict patients with serious outcomes.  Ann Emerg Med. 2006;47(5):448-454. doi:10.1016/j.annemergmed.2005.11.019PubMedGoogle ScholarCrossref
6.
Thiruganasambandamoorthy  V, Kwong  K, Wells  GA,  et al.  Development of the Canadian Syncope Risk Score to predict serious adverse events after emergency department assessment of syncope.  CMAJ. 2016;188(12):E289-E298. doi:10.1503/cmaj.151469PubMedGoogle ScholarCrossref
7.
Sun  BC, Emond  JA, Camargo  CA  Jr.  Direct medical costs of syncope-related hospitalizations in the United States.  Am J Cardiol. 2005;95(5):668-671. doi:10.1016/j.amjcard.2004.11.013PubMedGoogle ScholarCrossref
8.
Shen  WK, Sheldon  RS, Benditt  DG,  et al.  2017 ACC/AHA/HRS guideline for the evaluation and management of patients with syncope: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society.  J Am Coll Cardiol. 2017;70(5):620-663. doi:10.1016/j.jacc.2017.03.002PubMedGoogle ScholarCrossref
9.
Stiell  IG, Wells  GA.  Methodologic standards for the development of clinical decision rules in emergency medicine.  Ann Emerg Med. 1999;33(4):437-447. doi:10.1016/S0196-0644(99)70309-4PubMedGoogle ScholarCrossref
10.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KG.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement  [published correction appears in Ann Intern Med. 2015;162(8):600].  Ann Intern Med. 2015;162(1):55-63. doi:10.7326/M14-0697PubMedGoogle ScholarCrossref
11.
Moya  A, Sutton  R, Ammirati  F,  et al; Task Force for the Diagnosis and Management of Syncope; European Society of Cardiology (ESC); European Heart Rhythm Association (EHRA); Heart Failure Association (HFA); Heart Rhythm Society (HRS).  Guidelines for the diagnosis and management of syncope (version 2009).  Eur Heart J. 2009;30(21):2631-2671. doi:10.1093/eurheartj/ehp298PubMedGoogle ScholarCrossref
12.
Hoefnagels  WA, Padberg  GW, Overweg  J, van der Velde  EA, Roos  RA.  Transient loss of consciousness: the value of the history for distinguishing seizure from syncope.  J Neurol. 1991;238(1):39-43. doi:10.1007/BF00319709PubMedGoogle ScholarCrossref
13.
Sun  BC, Costantino  G, Barbic  F,  et al.  Priorities for emergency department syncope research.  Ann Emerg Med. 2014;64(6):649-55.e2. doi:10.1016/j.annemergmed.2014.04.014PubMedGoogle ScholarCrossref
14.
Steyerberg  EW.  Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer; 2009. doi:10.1007/978-0-387-77244-8
15.
McDonald  JH.  Handbook of Biological Statistics. 2nd ed. Sparky House Publishing; 2009.
16.
Peduzzi  P, Concato  J, Feinstein  AR, Holford  TR.  Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates.  J Clin Epidemiol. 1995;48(12):1503-1510. doi:10.1016/0895-4356(95)00048-8PubMedGoogle ScholarCrossref
17.
Thiruganasambandamoorthy  V, Rowe  BH, Sivilotti  MLA,  et al.  Duration of electrocardiographic monitoring of emergency department patients with syncope.  Circulation. 2019;139(11):1396-1406. doi:10.1161/CIRCULATIONAHA.118.036088PubMedGoogle ScholarCrossref
18.
Fine  MJ, Auble  TE, Yealy  DM,  et al.  A prediction rule to identify low-risk patients with community-acquired pneumonia.  N Engl J Med. 1997;336(4):243-250. doi:10.1056/NEJM199701233360402PubMedGoogle ScholarCrossref
19.
Quinn  JV, Stiell  IG, McDermott  DA, Sellers  KL, Kohn  MA, Wells  GA.  Derivation of the San Francisco Syncope Rule to predict patients with short-term serious outcomes.  Ann Emerg Med. 2004;43(2):224-232. doi:10.1016/S0196-0644(03)00823-0PubMedGoogle ScholarCrossref
20.
Costantino  G, Perego  F, Dipaola  F,  et al; STePS Investigators.  Short- and long-term prognosis of syncope, risk factors, and role of hospital admission: results from the STePS (Short-Term Prognosis of Syncope) study.  J Am Coll Cardiol. 2008;51(3):276-283. doi:10.1016/j.jacc.2007.08.059PubMedGoogle ScholarCrossref
21.
Reed  MJ, Newby  DE, Coull  AJ, Prescott  RJ, Jacques  KG, Gray  AJ.  The ROSE (Risk Stratification of Syncope in the Emergency Department) study.  J Am Coll Cardiol. 2010;55(8):713-721. doi:10.1016/j.jacc.2009.09.049PubMedGoogle ScholarCrossref
22.
Birnbaum  A, Esses  D, Bijur  P, Wollowitz  A, Gallagher  EJ.  Failure to validate the San Francisco Syncope Rule in an independent emergency department population.  Ann Emerg Med. 2008;52(2):151-159. doi:10.1016/j.annemergmed.2007.12.007PubMedGoogle ScholarCrossref
23.
Thiruganasambandamoorthy  V.Retrospective Validation of the San Francisco Syncope Rule for prediction of short-term serious outcomes in adult syncope patients [thesis]. University of Ottawa; 2009.
24.
du Fay de Lavallaz  J, Badertscher  P, Nestelberger  T,  et al.  B-type natriuretic peptides and cardiac troponins for diagnosis and risk-stratification of syncope.  Circulation. 2019;139:2403–2418. doi:10.1161/CIRCULATIONAHA.118.038358Google ScholarCrossref
25.
Colivicchi  F, Ammirati  F, Melina  D, Guido  V, Imperoli  G, Santini  M; OESIL (Osservatorio Epidemiologico sulla Sincope nel Lazio) Study Investigators.  Development and prospective validation of a risk stratification system for patients with syncope in the emergency department: the OESIL risk score.  Eur Heart J. 2003;24(9):811-819. doi:10.1016/S0195-668X(02)00827-8PubMedGoogle ScholarCrossref
26.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-XPubMedGoogle ScholarCrossref
27.
Toarta  C, Mukarram  M, Arcot  K,  et al.  Syncope prognosis based on emergency department diagnosis: a prospective cohort study.  Acad Emerg Med. 2018;25(4):388-396. doi:10.1111/acem.13346PubMedGoogle ScholarCrossref
28.
du Fay de Lavallaz  J, Badertscher  P, Nestelberger  T,  et al; BASEL IX Investigators.  Prospective validation of prognostic and diagnostic syncope scores in the emergency department.  Int J Cardiol. 2018;269:114-121. doi:10.1016/j.ijcard.2018.06.088PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 1,589
    Citations 0
    Original Investigation
    March 23, 2020

    Multicenter Emergency Department Validation of the Canadian Syncope Risk Score

    Author Affiliations
    • 1Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada
    • 2Ottawa Hospital Research Institute, Ottawa Hospital, Ottawa, Ontario, Canada
    • 3University of Ottawa School of Epidemiology and Public Health, Ottawa, Ontario, Canada
    • 4Department of Emergency Medicine, Queen’s University, Kingston, Ontario, Canada
    • 5Department of Biomedical and Molecular Sciences, Queen’s University, Kingston, Ontario, Canada
    • 6Department of Family Medicine and Emergency Medicine, Laval University, Quebec City, Quebec, Canada
    • 7CHU de Québec – Université Laval Research Center, Quebec City, Quebec, Canada
    • 8Division of Emergency Medicine, Western University, London, Ontario, Canada
    • 9Department of Emergency Medicine, University of British Columbia, Vancouver, British Columbia, Canada
    • 10Department of Emergency Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
    • 11Department of Emergency Medicine, University of Calgary, Calgary, Alberta, Canada
    • 12Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
    • 13Department of Emergency Medicine, University of Alberta, Edmonton, Alberta, Canada
    • 14School of Public Health, University of Alberta, Edmonton, Alberta, Canada
    • 15Division of Cardiology, University of British Columbia, Vancouver, British Columbia, Canada
    JAMA Intern Med. Published online March 23, 2020. doi:10.1001/jamainternmed.2020.0288
    Key Points

    Question  Can the Canadian Syncope Risk Score (CSRS) help in decision-making for emergency department (ED) patients with syncope based on short-term serious outcomes?

    Findings  In this cohort study of 3819 ED patients with syncope, the CSRS model performed well. Overall, 1% or fewer of patients with very-low-risk and low-risk CSRS, approximately 20% of patients with high-risk CSRS, and approximately 50% of patients with very-high-risk CSRS experienced 30-day serious outcomes.

    Meaning  Implementation of CSRS has the potential to improve patient safety and health care efficiency.

    Abstract

    Importance  The management of patients with syncope in the emergency department (ED) is challenging because no robust risk tool available has been recommended for clinical use.

    Objective  To validate the Canadian Syncope Risk Score (CSRS) in a new cohort of patients with syncope to determine its ability to predict 30-day serious outcomes not evident during index ED evaluation.

    Design, Setting, and Participants  This prospective multicenter cohort study conducted at 9 EDs across Canada included patients 16 years and older who presented to EDs within 24 hours of syncope. Patients were enrolled from March 2014 to April 2018.

    Main Outcomes and Measures  Baseline characteristics, CSRS predictors, and 30-day adjudicated serious outcomes, including arrhythmic (arrhythmias, interventions for arrhythmia, or unknown cause of death) and nonarrhythmic (myocardial infarction, structural heart disease, pulmonary embolism, or hemorrhage) serious outcomes, were collected. Calibration and discrimination characteristics for CSRS validation were calculated.

    Results  A total of 3819 patients were included (mean [SD] age 53.9 [22.8] years; 2088 [54.7%] female), of whom 139 (3.6%) experienced 30-day serious outcomes, including 13 patients (0.3%) who died. In the validation cohort, there were no differences between the predicted and observed risk, the calibration slope was 1.0, and the area under the receiver operating characteristic curve was 0.91 (95% CI, 0.88-0.93). The empirical probability of a 30-day serious outcome during validation was 3.64% (95% CI, 3.09%-4.28%) compared with the model-predicted probability of 3.17% (95% CI, 2.66%-3.77%; P = .26). The proportion of patients with 30-day serious outcomes increased from 3 of 1631 (0.3%) in the very-low-risk group to 40 of 78 (51.3%) in the very-high-risk group (Cochran-Armitage trend test P < .001). There was a similar significant increase in the serious outcome subtypes with increasing CSRS risk category. None of the very-low-risk and low-risk patients died or experienced ventricular arrhythmia. At a threshold score of −1 (2145 of 3819 patients), the CSRS sensitivity and specificity were 97.8% (95% CI, 93.8%-99.6%) and 44.3% (95% CI, 42.7%-45.9%), respectively.

    Conclusions and Relevance  The CSRS was successfully validated and its use is recommended to guide ED management of patients when serious causes are not identified during index ED evaluation. Very-low-risk and low-risk patients can generally be discharged, while brief hospitalization can be considered for high-risk patients. We believe CSRS implementation has the potential to improve patient safety and health care efficiency.

    Introduction

    Syncope is a sudden transient loss of consciousness followed by spontaneous complete recovery. Syncope accounts for 1% of emergency department (ED) visits, and identifying any serious underlying condition (eg, arrhythmia, pulmonary embolus, internal hemorrhage) that caused the syncope remains the primary focus of ED evaluation.1-4 Most patients have a benign course, yet approximately 1 in 10 patients presenting to the ED has a serious underlying condition identified within 30 days.3 Importantly, among 3% to 5% of patients with syncope, the serious condition will be identified only after ED disposition.5,6 For this reason, and in the absence of accurate risk stratification, more than half of all patients presenting to the ED with syncope are hospitalized, costing in excess of $2.4 billion annually.1-3,7 Given the low yield of hospitalization, national professional medical societies in Europe and North America have called for the development of practical and accurate tools to stratify patients into low-risk, intermediate-risk, and high-risk groups to aid in management decisions.3,8 Previously published tools lack validation or are excessively complex and thus are not supported by guideline recommendations.8 We previously derived a risk stratification tool, the Canadian Syncope Risk Score (CSRS; Table 1), which aims to predict 30-day serious outcomes after the index ED visit with a high degree of discrimination, calibration, and accuracy.6 The tool was developed using rigorous methodological standards, was internally validated using bootstrap validation, and was reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline statement.9,10 The objective of this study was to conduct a multicenter prospective temporal and geographic validation of the CSRS in a new cohort of patients with syncope to determine its ability to predict 30-day serious outcomes not evident during index ED evaluation.

    Methods
    Study Setting and Population

    We conducted a prospective cohort validation study at 9 large Canadian EDs (2 EDs each at the Ottawa Hospital, Ottawa, Ontario; the Kingston Health Sciences Centre, Kingston, Ontario; and the London Health Science Centre, London, Ontario; and 1 each at Hôpital de l’Enfant-Jésus du CHU de Québec-Université Laval, Quebec City, Quebec; St Boniface Hospital, Winnipeg, Manitoba; and Vancouver General Hospital, Vancouver, British Columbia) from March 2014 to April 2018. We sought to enroll consecutive patients 16 years and older who presented to an ED within 24 hours of syncope. Patients with prolonged loss of consciousness (more than 5 minutes), mental status changes from baseline, an obvious witnessed seizure based on previous history or current clinical evaluation, or head trauma leading to loss of consciousness were ineligible because they did not meet the definition of syncope.11,12 We excluded patients requiring hospitalization for traumatic injuries (eg, syncope leading to motorized vehicle collision), because their outcomes may be related to trauma rather than syncope. We also excluded patients from whom obtaining an accurate history was not possible (eg, language barrier, alcohol or drug intoxication). As with the derivation phase,6 we excluded patients adjudicated to have a serious underlying condition identified during the index ED evaluation and included patients both discharged and hospitalized during the index visit. The ethics committee at each study hospital approved this study with the requirement of only verbal informed consent for the collection of existing clinical data and follow-up given the noninterventional study design.

    Data Collection

    Research and clinical personnel screened all consecutive ED patients for enrollment in the study. Patients were prospectively enrolled both prior to and after the publication of the CSRS.6 The CSRS predictors among patients enrolled prior to the CSRS publication were also prospectively collected, although it was not explicitly stated that they were CSRS components. For patients enrolled after the CSRS publication, the ED attending physicians and residents were trained on the study protocol with a 1-hour didactic session to enroll patients with true syncope and to assess and record the CSRS predictors. The treating ED physician or an ED medicine resident under their direct supervision confirmed eligibility, obtained verbal informed consent, and completed the data collection form. The estimated risk of serious outcomes associated with each score level from the derivation study (eTable 1 in the Supplement)6 was not on the data collection form to prevent physicians from making disposition decisions based on the risk score. Research assistants trained in the study protocol reviewed all ED visits during the study period to identify potentially eligible patients who were not enrolled. Individual patient-level data will not be made available to other researchers, but analytical methods can be shared on request.

    Serious Outcomes

    As in the derivation phase,6 we prespecified and classified 30-day serious outcomes (eTable 2 in the Supplement) as either arrhythmic serious conditions (any serious arrhythmias; intervention to treat arrhythmias such as pacemaker/defibrillator insertion, or cardioversion; or any death due to an unknown cause) or nonarrhythmic serious conditions (myocardial infarction, serious structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, significant hemorrhage, subarachnoid hemorrhage, or any other serious condition causing syncope). We assumed that deaths from an unknown cause were due to arrhythmia and collected information on deaths that occurred secondary to the above-listed arrhythmic or nonarrhythmic serious conditions. The serious conditions listed above, including the arrhythmias, were deemed the most clinically relevant short-term outcomes by an international panel of experts.4,13 For patients who experienced serious outcomes, we collected both the time and the phase of care (ie, as an inpatient or after the index visit discharge) during which the patient experienced the serious outcome.

    We used a multistep approach to ascertain 30-day outcomes. First, we undertook a structured review of all available medical records for the index and subsequent ED visits, hospitalizations and/or deaths, and the results of all investigations, including those performed in the outpatient setting. As a second step, we performed a scripted telephone follow-up immediately after 30 days. The third step involved review of administrative health records for return visits, outpatient investigations, or hospitalizations at all local adult hospitals for patients from Ontario, Quebec, and British Columbia , and at all health care facilities in the province of Manitoba. All hospital-based health services are reliably captured in these databases because of the universal health insurance system across Canada. Finally, names of Ontario patients with no 30-day follow-up information were searched in the provincial coroner’s database for matching records, as by Ontario law the coroner is notified of sudden and unexpected deaths. Patients were designated as lost to follow-up if no information was available with the above approaches. A committee of 2 emergency physicians blinded to the CSRS predictors and the total score independently adjudicated each serious outcome, including the time and phase of care during which it occurred. Disagreements were resolved by a third physician.

    Statistical Analysis

    We describe the study patients using means, ranges, and SDs as appropriate for continuous variables and frequencies with proportion for categorical variables. We compared proportions using a χ2 test. Consistent with the derivation phase,6 missing electrocardiography (ECG) or serum troponin values were imputed as normal to calculate the total score. For each patient, we calculated their total score (Table 1), and their risk categorization (eTable 1 in the Supplement), as per the originally derived CSRS. With the total score as the only predictor in a logistic regression model, we calculated the calibration slope and constructed a calibration plot. We also calculated the mean observed vs predicted number of events, and the area under the receiver operating characteristic curve (AUC) with 95% CIs as a measure of discrimination. We compared the observed and expected risk at each score level in the study cohort. Because of the small number of patients with higher scores, consistent with the derivation phase, we integrated scores of 6 or higher. We tested the statistical significance of the trend in observed proportions of events across the risk strata using the Cochran-Armitage trend test. We report 2-tailed tests for statistical significance and considered P less than .05 as statistically significant. We conducted sensitivity analysis by performing multiple imputation for the missing troponin values among study patients. We used SAS, version 9.4 (SAS Institute), for data analysis. The sample size meets recommendations for validation studies of prediction tools, namely a minimum of 100 events and a minimum of 100 nonevents.14-16

    Results

    A total of 4131 patients with syncope were enrolled during the validation phase (Figure 1). After excluding 160 (3.9%) patients with serious conditions identified during the index ED evaluation (eTable 3 in the Supplement) and 152 (3.7%) patients who were lost to follow-up (eTable 4 in the Supplement), 3819 patients were included in the final analysis, representing 80.5% of all potentially eligible patients (Table 2; eTable 5 in the Supplement). Of the 3819 patients analyzed, 139 (3.6%; 95% CI, 3.1%-4.3%) patients experienced 30-day serious outcomes: 107 (2.8%) patients experienced arrhythmic outcomes, including 9 (0.2%) patients who died due to an unknown cause; and 32 (0.8%) patients experienced nonarrhythmic outcomes (eTable 6 in the Supplement). A total of 13 (0.3%) patients died within 30 days, and a cause of death was identified among 4 patients; 1 patient died due to cardiogenic shock, 1 died due to septic shock, and 2 died due to ventricular arrhythmia.

    In this validation study, a total of 114 patients (3.0%) did not have an ECG performed, and 1566 patients (41.0%) did not have troponin measured during the ED evaluation. These patients were generally younger and healthier and they rarely experienced any serious outcomes (eTables 7 and 8 in the Supplement). Hence, consistent with the derivation phase,6 these missing predictors were imputed as normal. Additionally, 2 patients (0.1%) in the study cohort were excluded from the model performance analysis because the total score could not be calculated due to missing systolic blood pressure and neither experienced a 30-day serious outcome.

    The AUC for the model during the validation phase was 0.91 (95% CI, 0.88-0.93). The mean observed probability of a 30-day serious outcome during validation was 3.64% (95% CI, 3.09%-4.28%) compared with the model-predicted probability of 3.17% (95% CI, 2.66%-3.77%; P = .26). The calibration plot for the validation cohort shows a slope of 1.0 for the observed vs the expected risk for 30-day serious outcomes, and the model calibration line is very close to the ideal calibration line (Figure 2). The observed probabilities during the validation phase for each CSRS score level were comparable with the previously published model-based 30-day serious outcome predicted probabilities (eFigure 1 in the Supplement). We conducted a sensitivity analysis by performing multiple imputation for the missing troponin values among study patients and found that the results of validation were similar (estimated outcome probability: 3.24%; 95% CI, 3.07%-3.42%; observed outcome probability: 3.64%; 95% CI, 3.09%-4.28%; P = .34; AUC, 0.91; 95% CI, 0.88-0.93).

    In this validation study, 3 of 1631 (0.3%) patients at very low risk and 9 of 1254 (0.7%) patients at low risk experienced 30-day serious outcomes, and this proportion significantly increased to 40 of 78 (51.3%) total patients in the very-high-risk group (Cochran-Armitage trend test P < .001; Table 3). There were similar steady significant increases in the subtypes of serious outcomes from the very-low-risk to the very-high-risk categories. Among medium-risk patients, 40 patients (5.8%) experienced arrhythmic outcomes and 15 (2.2%) patients nonarrhythmic outcomes within 30 days of ED disposition. Similarly, in the high-risk and very-high-risk groups, 59 (24.1%) patients experienced arrhythmic outcomes and 13 (5.3%) patients nonarrhythmic outcomes. The time of identification of these serious outcomes among medium-risk, high-risk, and very-high-risk patients is shown in eFigure 2 in the Supplement. A total of 99 patients in the medium-risk, high-risk, and very-high-risk groups experienced arrhythmic serious outcomes, of whom 89 (89.9%) patients had them identified within 15 days of the index ED visit. The 13 patients with ventricular arrhythmia had them identified within 9 days of the index ED visit. The sensitivities and specificities for each threshold total score for the validation phase data are detailed in eTable 9 in the Supplement. At a threshold score of −1, including 2145 of 3819 patients, the CSRS performed with a sensitivity of 97.8% (95% CI, 93.8%-99.6%) and a specificity of 44.3% (95% CI, 42.7%-45.9%).

    Among those who were lost to 30-day follow-up, based on the CSRS risk categorization, we designated a proportion of patients as having experienced a 30-day serious outcome (eTable 4 in the Supplement). We found that the model still performed well with an AUC at 0.91 (95% CI, 0.89-0.93).

    Discussion

    In this prospective multicenter study, we validated the CSRS on a new cohort of patients with syncope treated in an ED and confirmed the accurate model performance characteristics of the original decision tool. The results showed that less than 1% of very-low-risk and low-risk patients, approximately 20% of high-risk patients, and 50% of very-high-risk CSRS patients experienced 30-day serious outcomes. Such robust risk classification in conjunction with our 2019 report on ECG monitoring17 offers short-term prognostic information to clinicians that may be translated into meaningful clinical management options.

    The present study found that the short-term serious morbidity and mortality for ED syncope was very low, with 0.3% risk for each of 30-day mortality and ventricular arrhythmia, as previously reported.3,5 Three-quarters of patients in this validation study were designated as being at very low or low risk and fewer than 1% experienced a 30-day serious outcome. Additionally, none of the patients in these categories died or experienced ventricular arrhythmia. Hence, we believe that these patients can be discharged quickly after ED evaluation.

    Overall, a low proportion (2.2%) of medium-risk patients experienced nonarrhythmic serious outcomes within 30 days, approximately 0.5% per day in the first 4 days, after which the incidence was negligible. Given such low probability, discharging these patients with clear instructions to watch for symptoms that indicate evolution of serious conditions might be a reasonable management option. There was a very small risk (0.1%) of mortality among the medium-risk group, and this was lower than the accepted risk for discharge of patients with pneumonia.18 We recognize that clinicians in risk-averse settings owing to medicolegal considerations or patient expectations may choose a brief period of observation using a shared decision-making approach. A short course of hospitalization is a reasonable option for the higher-risk groups.

    Three tools, the San Francisco Syncope Rule (SFSR), Short-Term Prognosis of Syncope, and Risk Stratification of Syncope in the ED (ROSE), have been previously published to risk stratify patients with syncope in the ED for short-term serious outcomes.19-21 The SFSR performed poorly on external validation.5,19,22,23 To our knowledge, the Short-Term Prognosis of Syncope tool has not been validated and the ROSE requires B-type natriuretic peptide (BNP) measurements.20,21 All 3 tools included patients with serious outcomes clearly evident on ED presentation, which may introduce bias toward identification of the obvious, and their application does not lead to clear clinical management options.13

    A 2019 multicenter study compared the prognostic performance of 4 biomarkers, BNP, N-terminal pro-BNP, and high-sensitivity cardiac troponin I and T levels, against the ROSE, SFSR, CSRS, and Osservatorio Epidemiologico sulla Sincope nel Lazio risk tools, to predict major adverse cardiac events.24,25 This study as part of its secondary objective externally validated the CSRS among patients older than 45 years with syncope in 8 countries and reported an AUC of 0.88.

    Given the large number of study patients involving multiple sites, we believe that our results are generalizable. Our validation study adheres to the reporting requirements outlined in the TRIPOD and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.10,26 Given that syncope management is complex, the CSRS is able to risk stratify patients for management decision-making as recommended by the national professional medical societies.3,8

    Limitations

    This study has several limitations. As many as 20% of potentially eligible patients were not enrolled. This is likely an overestimation, as true eligibility for some of these patients could not be ascertained from the medical records and uncertain cases were classified as missed. There were no obvious systematic reasons for failure to enroll these patients. It is possible these patients were at very low risk and were discharged quickly before screening or the emergency physician was too busy with sicker patients to complete the data collection form. However, the demographic characteristics (mean [SD] age, 55.3 [22.8] years; sex, 563 of 1002, 56.2%, female) of these patients were similar to those of the enrolled cohort. In the present study, although the treating physician collecting the data did not have the estimated risk of 30-day serious outcomes on the data collection form, it is possible that the score influenced ordering of investigations and disposition decisions. However, while the proportion hospitalized and those who had loop monitoring increased with the CSRS risk, we did not find similar increases in other outpatient work-up (eTable 5 in the Supplement). In this validation cohort of 3819 patients, 114 (3.0%) patients were missing ECG predictors and 1566 (41.0%) were missing troponin predictors.The proportion of patients with the 2 missing predictors was similar to that in the derivation phase:6 of 4030 patients, 196 (4.9%) were missing ECG predictors and 2101 (52.1%) were missing troponin predictors and, following the same analytical plan as the derivation phase, these missing predictors were imputed as normal. The patients with these missing predictors were younger, with low prevalence of comorbidities, and very few experienced 30-day serious outcomes. Hence, it is likely that physicians elected not to perform these tests. A sensitivity analysis imputing the missing troponin values among study patients did not change the results of the study. Apart from the troponin and ECG predictors, there were only 2 patients (0.1%) for whom the total score could not be calculated because of missing predictors, and neither patient experienced a 30-day serious outcome. Among those enrolled, 3.7% had incomplete 30-day follow-up (eTable 3 in the Supplement). They appear to be at lower risk and, hence, are unlikely to have experienced 30-day serious outcomes. Our outcome assessment process did confirm that those lost to follow-up were unlikely to have experienced 30-day mortality. Given the very small number of patients with missing predictors and incomplete follow-up, the missing data are unlikely to have influenced the study results. A sensitivity analysis including those lost to follow-up and designating 30-day outcomes based on CSRS risk category showed the CSRS still performed with robust discrimination. The tool includes the physician diagnostic impression predictor, which we have shown previously to be reliable and powerful.27 However, as the present study was conducted in academic EDs, the tools’ accuracy in nonacademic settings is unknown. A 2018 study28 with 1490 patients enrolled in 8 countries showed that risk tools that do not incorporate physician diagnostic impression performed poorly and advocated for incorporation of diagnostic impression in tools for improved performance. Additionally, this validation study was limited to Canadian sites, and some of these sites were also part of the derivation phase. A 2019 independent international multicenter study24 that externally validated the CSRS among patients 40 years or older with syncope at 13 EDs in 8 countries reported excellent discrimination abilities for the score. However, the primary objective of this study was to assess the prognostic ability of 4 biomarkers, BNP, N-terminal pro-BNP, and high-sensitivity cardiac troponin I and T levels, and the CSRS validation was an incidental result. Consequently, this study did not report the essential validation results, such as calibration, sensitivity, or specificity, nor the associated risk nor clinical management options for the CSRS risk categories.

    In summary, we believe that patients with serious conditions identified during index ED evaluation need appropriate management, and those strongly suspected to have such conditions because of unstable vital signs or clinical symptoms will need hospitalization for further diagnostic testing and/or monitoring. We envision that the CSRS will be applied at the end of ED evaluation to guide disposition decisions for the remaining patients.

    Conclusions

    Results of this large multicenter prospective study demonstrated an apparent successful validation of the CSRS to risk stratify patients with syncope presenting to the ED. The application of the CSRS may aid in accurate short-term risk stratification after acute syncope ED evaluation. Based on the study results, we recommend that patients with very-low-risk and low-risk CSRS be discharged, patients at medium risk be involved in a shared decision approach regarding disposition, and patients at high risk be hospitalized for a short course. We believe that implementation of the CSRS will improve patient safety and reduce health care resource use.

    Back to top
    Article Information

    Accepted for Publication: January 27, 2020.

    Corresponding Author: Venkatesh Thiruganasambandamoorthy, MBBS, MSc, Ottawa Hospital Research Institute, Ottawa Hospital, 1053 Carling Ave, Civic Campus, 6th Floor, Room F650, Ottawa, ON K1Y 4E9, Canada (vthirug@ohri.ca).

    Published Online: March 23, 2020. doi:10.1001/jamainternmed.2020.0288

    Author Contributions: Dr Thiruganasambandamoorthy has full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Thiruganasambandamoorthy, Sivilotti, Le Sage, Rowe, Wells, Taljaard.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Thiruganasambandamoorthy, Yan, Hegdekar.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Nemnom, Wells, Taljaard.

    Obtained funding: Thiruganasambandamoorthy, Sivilotti, Le Sage, McRae, Taljaard.

    Administrative, technical, or material support: Thiruganasambandamoorthy, Huang, Hegdekar, Mercier, McRae, Rowe.

    Supervision: Thiruganasambandamoorthy, Sivilotti, Yan, Huang, Mukarram.

    Conflict of Interest Disclosures: Dr Le Sage reports receiving grants from the Cardiac Arrhythmia Network of Canada. Dr Sivilotti reports receiving grants from the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada, and the Cardiac Arrhythmia Network of Canada. Dr Thiruganasambandamoorthy reports receiving grants from the Heart and Stroke Foundation of Canada and the Cardiac Arrhythmia Network of Canada, as well as honorarium and travel expenses for attending a focus group on syncope by Medtronic. No other disclosures were reported.

    Funding/Support: This work was funded by grant G-15-0009006 from the Heart and Stroke Foundation of Canada, and grant SRG-15-P10-001 from the Cardiac Arrhythmia Network of Canada as part of the Networks of Centres of Excellence. Dr Thiruganasambandamoorthy holds a salary award through the Heart and Stroke Foundation of Canada. During the study, Dr Rowe’s research was supported by the Canadian Institutes of Health Research through a Tier I Canada Research Chair in Evidence-based Emergency Medicine from the Government of Canada.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We are deeply indebted to all the patients who participated in this study and gratefully acknowledge the emergency physicians at the study sites who recruited the patients and the emergency medicine residents who helped in this process. We acknowledge the following members of our research team. Ottawa Hospital Research Institute, Kingston, Ontario, Canada: Soo-Min Kim, MHA; Sarah Gaudet, MN; Aline Christelle Ishimwe, RN; Faheem Malam, MSc; Zein Ahmed, BScH; My-Linh Tran, MSc; Sheryl Domingo; Angela Marcantonio; Connor Monk, BSc; and Hina Chaudry, MD. All are compensated employees and contributed to acquisition of data. Kingston Health Sciences Center, Kingston, Ontario, Canada: Jane Reid, RN; Laura Walmsley (previously Goodfellow), MD; Nicole O’Callaghan, MSc; and Vlad Latiu, MD. All are compensated employees and contributed to acquisition of data. London Health Sciences Centre: Melanie Columbus, PhD; Kristine Van Aarsen, MSc; and Dimah Azzam, BSc. All are compensated employees and contributed to acquisition of data. CHU de Québec Université Research Centre: Catherine Bédard, RN, MBA; and Suzy Lavoie, RN. All are compensated employees and contributed to acquisition of data. Vancouver site: Rupinder Brar, MPH (University of British Columbia, compensated employee) and Vi Ho, MD (Vancouver Coastal health, compensated employee) contributed to acquisition of data. Corinne Hohl, MD (University of British Columbia, not compensated) contributed to design and acquisition of data. Winnipeg site: Anne Finlayson, MD; and Monica Manhas, MD (St. Boniface Hospital, not compensated). Christine Kennedy, RN (St. Boniface Hospital, compensated). All contributed to acquisition of data.

    References
    1.
    Probst  MA, Kanzaria  HK, Gbedemah  M, Richardson  LD, Sun  BC.  National trends in resource utilization associated with ED visits for syncope.  Am J Emerg Med. 2015;33(8):998-1001. doi:10.1016/j.ajem.2015.04.030PubMedGoogle ScholarCrossref
    2.
    Sun  BC, Emond  JA, Camargo  CA  Jr.  Characteristics and admission patterns of patients presenting with syncope to US emergency departments, 1992-2000.  Acad Emerg Med. 2004;11(10):1029-1034. doi:10.1016/j.annemergmed.2004.07.016PubMedGoogle Scholar
    3.
    Brignole  M, Moya  A, de Lange  FJ,  et al; ESC Scientific Document Group.  2018 ESC guidelines for the diagnosis and management of syncope.  Eur Heart J. 2018;39(21):1883-1948. doi:10.1093/eurheartj/ehy037PubMedGoogle ScholarCrossref
    4.
    Sun  BC, Thiruganasambandamoorthy  V, Cruz  JD; Consortium to Standardize ED Syncope Risk Stratification Reporting.  Standardized reporting guidelines for emergency department syncope risk-stratification research.  Acad Emerg Med. 2012;19(6):694-702. doi:10.1111/j.1553-2712.2012.01375.xPubMedGoogle ScholarCrossref
    5.
    Quinn  J, McDermott  D, Stiell  I, Kohn  M, Wells  G.  Prospective validation of the San Francisco Syncope Rule to predict patients with serious outcomes.  Ann Emerg Med. 2006;47(5):448-454. doi:10.1016/j.annemergmed.2005.11.019PubMedGoogle ScholarCrossref
    6.
    Thiruganasambandamoorthy  V, Kwong  K, Wells  GA,  et al.  Development of the Canadian Syncope Risk Score to predict serious adverse events after emergency department assessment of syncope.  CMAJ. 2016;188(12):E289-E298. doi:10.1503/cmaj.151469PubMedGoogle ScholarCrossref
    7.
    Sun  BC, Emond  JA, Camargo  CA  Jr.  Direct medical costs of syncope-related hospitalizations in the United States.  Am J Cardiol. 2005;95(5):668-671. doi:10.1016/j.amjcard.2004.11.013PubMedGoogle ScholarCrossref
    8.
    Shen  WK, Sheldon  RS, Benditt  DG,  et al.  2017 ACC/AHA/HRS guideline for the evaluation and management of patients with syncope: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society.  J Am Coll Cardiol. 2017;70(5):620-663. doi:10.1016/j.jacc.2017.03.002PubMedGoogle ScholarCrossref
    9.
    Stiell  IG, Wells  GA.  Methodologic standards for the development of clinical decision rules in emergency medicine.  Ann Emerg Med. 1999;33(4):437-447. doi:10.1016/S0196-0644(99)70309-4PubMedGoogle ScholarCrossref
    10.
    Collins  GS, Reitsma  JB, Altman  DG, Moons  KG.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement  [published correction appears in Ann Intern Med. 2015;162(8):600].  Ann Intern Med. 2015;162(1):55-63. doi:10.7326/M14-0697PubMedGoogle ScholarCrossref
    11.
    Moya  A, Sutton  R, Ammirati  F,  et al; Task Force for the Diagnosis and Management of Syncope; European Society of Cardiology (ESC); European Heart Rhythm Association (EHRA); Heart Failure Association (HFA); Heart Rhythm Society (HRS).  Guidelines for the diagnosis and management of syncope (version 2009).  Eur Heart J. 2009;30(21):2631-2671. doi:10.1093/eurheartj/ehp298PubMedGoogle ScholarCrossref
    12.
    Hoefnagels  WA, Padberg  GW, Overweg  J, van der Velde  EA, Roos  RA.  Transient loss of consciousness: the value of the history for distinguishing seizure from syncope.  J Neurol. 1991;238(1):39-43. doi:10.1007/BF00319709PubMedGoogle ScholarCrossref
    13.
    Sun  BC, Costantino  G, Barbic  F,  et al.  Priorities for emergency department syncope research.  Ann Emerg Med. 2014;64(6):649-55.e2. doi:10.1016/j.annemergmed.2014.04.014PubMedGoogle ScholarCrossref
    14.
    Steyerberg  EW.  Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. Springer; 2009. doi:10.1007/978-0-387-77244-8
    15.
    McDonald  JH.  Handbook of Biological Statistics. 2nd ed. Sparky House Publishing; 2009.
    16.
    Peduzzi  P, Concato  J, Feinstein  AR, Holford  TR.  Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates.  J Clin Epidemiol. 1995;48(12):1503-1510. doi:10.1016/0895-4356(95)00048-8PubMedGoogle ScholarCrossref
    17.
    Thiruganasambandamoorthy  V, Rowe  BH, Sivilotti  MLA,  et al.  Duration of electrocardiographic monitoring of emergency department patients with syncope.  Circulation. 2019;139(11):1396-1406. doi:10.1161/CIRCULATIONAHA.118.036088PubMedGoogle ScholarCrossref
    18.
    Fine  MJ, Auble  TE, Yealy  DM,  et al.  A prediction rule to identify low-risk patients with community-acquired pneumonia.  N Engl J Med. 1997;336(4):243-250. doi:10.1056/NEJM199701233360402PubMedGoogle ScholarCrossref
    19.
    Quinn  JV, Stiell  IG, McDermott  DA, Sellers  KL, Kohn  MA, Wells  GA.  Derivation of the San Francisco Syncope Rule to predict patients with short-term serious outcomes.  Ann Emerg Med. 2004;43(2):224-232. doi:10.1016/S0196-0644(03)00823-0PubMedGoogle ScholarCrossref
    20.
    Costantino  G, Perego  F, Dipaola  F,  et al; STePS Investigators.  Short- and long-term prognosis of syncope, risk factors, and role of hospital admission: results from the STePS (Short-Term Prognosis of Syncope) study.  J Am Coll Cardiol. 2008;51(3):276-283. doi:10.1016/j.jacc.2007.08.059PubMedGoogle ScholarCrossref
    21.
    Reed  MJ, Newby  DE, Coull  AJ, Prescott  RJ, Jacques  KG, Gray  AJ.  The ROSE (Risk Stratification of Syncope in the Emergency Department) study.  J Am Coll Cardiol. 2010;55(8):713-721. doi:10.1016/j.jacc.2009.09.049PubMedGoogle ScholarCrossref
    22.
    Birnbaum  A, Esses  D, Bijur  P, Wollowitz  A, Gallagher  EJ.  Failure to validate the San Francisco Syncope Rule in an independent emergency department population.  Ann Emerg Med. 2008;52(2):151-159. doi:10.1016/j.annemergmed.2007.12.007PubMedGoogle ScholarCrossref
    23.
    Thiruganasambandamoorthy  V.Retrospective Validation of the San Francisco Syncope Rule for prediction of short-term serious outcomes in adult syncope patients [thesis]. University of Ottawa; 2009.
    24.
    du Fay de Lavallaz  J, Badertscher  P, Nestelberger  T,  et al.  B-type natriuretic peptides and cardiac troponins for diagnosis and risk-stratification of syncope.  Circulation. 2019;139:2403–2418. doi:10.1161/CIRCULATIONAHA.118.038358Google ScholarCrossref
    25.
    Colivicchi  F, Ammirati  F, Melina  D, Guido  V, Imperoli  G, Santini  M; OESIL (Osservatorio Epidemiologico sulla Sincope nel Lazio) Study Investigators.  Development and prospective validation of a risk stratification system for patients with syncope in the emergency department: the OESIL risk score.  Eur Heart J. 2003;24(9):811-819. doi:10.1016/S0195-668X(02)00827-8PubMedGoogle ScholarCrossref
    26.
    von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-XPubMedGoogle ScholarCrossref
    27.
    Toarta  C, Mukarram  M, Arcot  K,  et al.  Syncope prognosis based on emergency department diagnosis: a prospective cohort study.  Acad Emerg Med. 2018;25(4):388-396. doi:10.1111/acem.13346PubMedGoogle ScholarCrossref
    28.
    du Fay de Lavallaz  J, Badertscher  P, Nestelberger  T,  et al; BASEL IX Investigators.  Prospective validation of prognostic and diagnostic syncope scores in the emergency department.  Int J Cardiol. 2018;269:114-121. doi:10.1016/j.ijcard.2018.06.088PubMedGoogle ScholarCrossref
    ×