Association of Emergency Clinicians' Assessment of Mortality Risk With Actual 1-Month Mortality Among Older Adults Admitted to the Hospital

This cohort study examines the accuracy of clinicians’ assessments of mortality risk among older adults admitted to the hospital from the emergency department compared with the actual 1-month mortality of these patients.


Introduction
Approximately 75% of older adults with serious, life-limiting illnesses visit the emergency department (ED) during the final 6 months of life. 1 Emergency department visits often mark an inflection point in a patient's illness trajectory, signaling a more rapid rate of decline. 2,3Many patients have not formulated their goals for care in the context of their serious illness, 4 and approximately 56% to 99% of patients do not have documentation of such goals available at the time of ED presentation. 5Most patients who are seriously ill have priorities other than simply living as long as possible, 6 yet they are at risk of receiving care that does not align with their goals. 7][10][11][12][13][14][15][16][17] Furthermore, patients with documented serious illness conversations experience a 36% reduction in the cost of end-of-life care, with an mean cost savings of $1041 per patient in the final week of life. 18A study by Smith et al 19 reported that earlier serious illness conversations are 1 of 5 key changes that can reduce the costs for health care.Yet only approximately 37% of older adults who are seriously ill have these conversations with their physicians, 9 and it is often late in the disease course (33 days before death 20 ).
Emergency physicians recognize that an ED visit provides a time and a location for older adults who are seriously ill to discuss serious illness care goals. 21For patients who are seriously ill but clinically stable and who are likely to experience a decline in illness trajectory, engaging them in such a discussion in the ED may be an ideal moment to facilitate serious illness conversations. 22troduction or reinitiation of serious illness conversations before these patients become clinically unstable may be acceptable and feasible in the ED. 23The current clinical practice is constrained by the lack of feasible and reliable approaches to identify patients with limited life expectancy who are most likely to benefit from serious illness conversations after an ED visit.A practical method is needed to help emergency clinicians identify patients who are seriously ill and at the highest risk of mortality and to ensure that such patients receive serious illness conversations after being admitted to the hospital.The so-called surprise question, worded as "Would you be surprised if this patient died in the next 12 months?" is a method to obtain the clinician's overall clinical impression associated with the actual 12-month mortality among patients with a life-limiting illness.The surprise question has demonstrated sensitivity of 21% to 84% and specificity of 51% to 94% in prior studies [24][25][26][27][28][29][30][31] among outpatient patients with advanced cancer or chronic kidney disease undergoing hemodialysis and may be uniquely valuable in the time-pressured ED setting to identify older patients with high risk of mortality. 32The surprise question may help emergency clinicians to identify older adults who are seriously ill and who most urgently require serious illness conversation on admission so that no such patients continue the hospitalization without formulating serious illness care goals.However, to our knowledge, previous studies of the surprise question are limited by their small sample size, variable magnitudes of association, and focus on specific disease populations, limiting the generalizability of this method.
In this large cohort study, we aimed to prospectively test the association of the surprise question with the actual 1-month mortality among a diverse population of undifferentiated older patients in the ED.We propose that emergency clinicians' ability to estimate prognosis is the most accurate in the short term (ie, 1-month vs the traditional 12-month mortality) by the nature of their

Study Design
We conducted a prospective cohort study to examine the association of physicians' response to the surprise question with the actual 1-month mortality of patients who visited the ED at an academic, urban hospital with an annual volume of 70 000 visits, including 24% by patients aged 65 years and older and a hospital admission rate of 47% among these older patients.The study protocol was approved by the Partners Healthcare institutional review board as human subject research.Informed consent was waived because there was no interaction with patients and their risk was strictly the potential breach of confidentiality, which was deemed a minimal risk.

Participants and Procedures
We included all patients 65 years and older who received care in the ED and were admitted to the hospital from January 1, 2014, to December 31, 2015.We chose to include older adults who were admitted to the hospital because they would be more likely to have higher mortality compared with those who were discharged and therefore were the most appropriate patient population to study to identify older adults with the highest 1-month mortality in the ED.The patients who were discharged were more likely to live beyond 1 month, and the treating emergency clinicians were more likely to report that they would be surprised if the patient died in the next 1 month.Therefore, including the discharged patients may have risked overinflating the overall accuracy of the clinical impression.We excluded patients who were admitted to the hospital without going through the ED and those who were transferred from the ED to the cardiac catheterization laboratory or operating room after the ED (ie, expected to have high mortality).
When placing a bed request in the electronic medical record (EMR) for any patient being admitted to the hospital, emergency clinicians were required to answer a mandatory question, "Would you be surprised if your patient died in the next one month?"Placing a bed request in the EMR is a requirement to admit a patient into the hospital from the ED.Enrollment took place consecutively 7 days per week, 24 hours per day.We obtained the death records from the National Death Index (NDI), the central computerized database containing all certified deaths in the United States, on January 1, 2018.We used the complete NDI data from 2014 to 2015 and early release data from January to December 2016, which contained more than 90% of the 2016 data.Mortality information was ascertained by matching the cohort data set from the EMR to NDI death certificate records.We used the Social Security number, first and last names, and date of birth to match the records.Participants were considered a match when all 3 variables were completely matched.
Participants were assigned a vital status code (0 = assumed alive; 1 = assumed deceased) based on their status as of 1 month from the time they visited the ED.Participants with incomplete matches or multiple NDI-matched death records were excluded from the study.

Variables
Our primary outcome was the accuracy of clinicians' response to the surprise question in identifying older patients in the ED with 1-month mortality.Our secondary outcomes included the accuracies of responses by both emergency clinicians and admitting internal medicine clinicians to the surprise question in identifying older patients with high 6-or 12-month mortality.We controlled for the following potential confounders in our analysis: patient demographic information (ie, age, sex, and self-reported race), Charlson Comorbidity Index score, 33

Statistical Analysis
Patients were classified into 2 subgroups based on how clinicians answered the surprise question: ( "No, I would not be surprised if this patient died in the next one month," and (2) "Yes, I would be surprised if this patient died in the next one month."A generalized estimating equation model with binary outcome was used to estimate the association of death and the response to the surprise question along with other potential covariates to take into account the possible correlation between multiple visits from the same patient.We started fitting a bivariate generalized estimating equation model with a pool of potential covariates that may be clinically relevant to the outcome 1 at a time.
Covariates with significant associations were included in the final multivariable model.We determined the sensitivity, specificity, positive and negative predictive values, accuracy, odds ratio (OR), and area under the receiver operating curve at 1 month using SAS statistical software version 9.4 (SAS Institute).P values were 2-tailed, and statistical significance was set at P less than .05.We presented the overall difference in 1-month survival between the 2 subgroups using Kaplan-Maier curves.Based on the study by Lilley et al, 27 we expected the accuracy to be at least 0.60.To achieve the precision of at least 0.02, we hypothesized that we would require at least 10 000 patients for the sample size.We also performed a sensitivity analysis to account for intensive care unit admission status.

Results
We predictive values were 20% and 93%, respectively.The accuracy was 78% (Table 3).The area under the receiver operating curve of the bivariate model was 0.63 (95% CI, 0.61-0.64;P < .001),and the area under the receiver operating curve of the multivariable model was 0.73 (95% CI, 0.72-0.74;P < .001).The model performance improved from 0.70 to 0.73 by adding the surprise question as a covariate.Among 2104 patients for whom clinicians answered that they would not be surprised the patient died in 1 month, only 533 patients actually died within 1 month (Figure 2).Similar test characteristics were demonstrated by the admitting hospital clinicians.Emergency clinicians' ability to accurately identify patients who would die vs patients who would survive was consistent at 1, 6, and 12 months after the ED visit (eTable 1 and eTable 2 in the Supplement).There were no statistically significant differences in our sensitivity analysis to account for intensive care unit admission status (eTable 3 in the Supplement).We did not have any missing data for responses to the surprise question or for 1-month mortality.

Discussion
Emergency clinicians asserted that they would not be surprised if  2).However, the diagnostic test characteristic of the surprise question alone was poor, which makes it a poor screening tool for identifying patients with high risk of 1-month mortality.
[37] Furthermore, our study expanded on the available literature that the overall clinical impressions of emergency clinicians who have no prior knowledge or existing relationships with patients are associated with 1-month mortality among older adults whom the clinicians admit to the hospital.
8][29] This finding may be due to a larger sample size (improving the statistical precision), higher clinical acuity and needs of patients in the ED (ie, emergency clinicians' perception of acuity may be blunted, resulting in downward bias on the magnitude of association), 38,39 and lower study mortality (8.3% vs 6% to 45% 24,26,[29][30][31] ) compared with prior studies in other clinical settings.a At the time of requesting a bed through the electronic medical record system for the patient to be admitted to the hospital, the treating clinician was required to answer the surprise question, "Would you be surprised if your patient died in the next one month?"Comparison is answering, "No, I would not be surprised," vs "Yes, I would be surprised." b Compared with men.
c Compared with white race.b At the time of requesting a bed through the electronic medical record system for the patient to be admitted to the hospital, the treating clinician was required to answer the surprise question, "Would you be surprised if your patient died in the next one month?"Clinicians could respond, "No, I would not be surprised," or "Yes, I would be surprised." c Accurate prediction.
d Inaccurate prediction.

JAMA Network Open | Emergency Medicine
Emergency Clinicians' Assessment of Mortality Risk and Actual 1-Month Mortality Among Older Adults

Downloaded From: https://jamanetwork.com/ on 10/07/2023
34cility), and principal diagnosis using Clinical Classifications Software.34Thesevariables were directly extracted from the EMR as structured data variables.No manual EMR abstraction was conducted.Emergency clinicians commonly consider many clinical variables (eg, vital signs) to derive their overall clinical impressions (ie, the response to the surprise question).Some potentially confounding information may not have been considered immediately (eg, comorbid conditions); thus, we decided to include such additional variables in the multivariable model.We did not include other potential confounders easily accessible to the clinician when determining the overall clinical impression, such as vital signs, laboratory values, and other clinical scoring systems (eg, Acute Physiologic Assessment and Chronic Health Evaluation).Many, if not all, of these variables should be incorporated into clinicians' overall clinical impression.
source of ED arrival (eg, home, another JAMA Network Open | Emergency Medicine Emergency Clinicians' Assessment of Mortality Risk and Actual 1-Month Mortality Among Older Adults JAMA Network Open.2019;2(9):e1911139.doi:10.1001/jamanetworkopen.2019.11139(Reprinted) September 13, 2019 3/11 Downloaded From: https://jamanetwork.com/ on 10/07/2023 hospital, nursing that they would not be surprised if the patient died in 1 month was 2.4-fold that of patients for whom clinicians answered that they would be surprised if the patient died in 1 month (OR, 2.4 [95% CI, 2.2-2.7];P < .001).The surprise question answered by clinicians demonstrated sensitivity of 43% and specificity of 82%.Given the 1-month mortality rate of 8.3% in our cohort, the positive and negative identified 19 284 ED visits by 12 517 patients from January 1, 2014, to December 31, 2015.Sixteen patients were excluded owing to unknown Social Security numbers, inconsistent Social Security number matches to the NDI records, or incomplete matches of the name, date of birth, or Social Security number.We also excluded 1764 patients because they were transferred directly to the cardiac catheterization laboratory or operating room.We identified 16 223 ED visits by 10 737 patients in our final cohort (Figure1).The patients had a mean (SD) age of bivariate analysis (Table2), the odds of death at 1 month in patients for whom clinicians answered that they would not be surprised if the patient died in 1 month was 3.3-fold that of patients for whom clinicians answered that they would be surprised if the patient died in 1 month (OR, 3.3 [95% CI, 3.0-3.7];P < .001).In multivariable analysis controlling for age, sex, race, admission diagnosis, and comorbid conditions (Table 2), the odds of death at 1 month in patients for whom clinicians answered JAMA Network Open | Emergency Medicine Emergency Clinicians' Assessment of Mortality Risk and Actual 1-Month Mortality Among Older Adults JAMA Network Open.2019;2(9):e1911139.doi:10.1001/jamanetworkopen.2019.11139(Reprinted) September 13, 2019 4/11 Downloaded From: https://jamanetwork.com/ on 10/07/2023 Emergency Clinicians' Assessment of Mortality Risk and Actual 1-Month Mortality Among Older Adults 1 in 5 of their older patients in the ED died within the next month.Our study confirms that the emergency clinicians' overall clinical assessment is associated with patients' actual mortality (OR, 2.4 [95% CI, 2.2-2.7];P < .001)JAMA Network Open.2019;2(9):e1911139.doi:10.1001/jamanetworkopen.2019.11139(Reprinted) September 13, 2019 5/11 Downloaded From: https://jamanetwork.com/ on 10/07/2023 (Table

Table 2 .
Bivariate and Multivariable Analysis for 1-Month Mortality Abbreviations: ED, emergency department; NA, not applicable; OR, odds ratio.

Table 3 .
Diagnostic Test Characteristics of the Surprise Question Asked of Emergency Clinicians for the Actual 1-Month Mortality a Analysis was performed at individual patient visit level with general estimate equation model to account for repeated visits by the same patients. a