The Value of Clinical Judgment in the Detection of Clinical Deterioration

Hospitals routinely use rapid response teams (RRTs) to treat and triage unstable patients, but early and reliable identification of high-risk patients remains challenging.¹ Objective, vital sign–based risk prediction scores, such as the Modified Early Warning Score (MEWS), have been developed but have limited accuracy.² The Patient Acuity Rating (PAR), a subjective 7-point Likert scale assessment of the likelihood of cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours, has been proposed as an alternate risk stratification tool.³ However, the PAR has not been externally validated or directly compared with objective metrics such as the MEWS.

This study was approved by the Northwestern University Institutional Review Board (CR3 STU00027683). Hospitalists at an academic medical center prospectively assigned a daily PAR score to consecutive medical in-patients as part of a standardized electronic medical record handoff. Covering physicians were blinded to PAR assignment. A corresponding MEWS was calculated using the closest vital signs. Outcomes were cardiac arrest or ICU transfer, RRT activation, and a composite of any 3, occurring within 24 hours of an observation. These were evaluated using area under the receiver operator characteristics curve (AUC). Logistic regression was used to combine paired PAR and MEWS scores.

Between September 1, 2011, and August 31, 2012, we identified 51 eligible physicians, 34 of whom consented to participate. Participants gave written informed consent. There was no compensation for participation. Among the 34 physicians, 28 provided a total of 7244 PARs in 3249 distinct patients, with a median PAR of 2 (range, 1-7). Among these patients, we observed 2 cardiac arrests, 67 ICU transfers, and 105 RRT activations. In addition, 51 202 MEWS scores were calculated for these patients, with a median of 3 (range, 2-11). There was a median of 84 minutes between corresponding PAR and MEWS scores, which were poorly correlated (Spearman ρ = 0.10), even when the PAR-MEWS time discrepancy was less than or equal to 84 minutes (Spearman ρ = 0.13). The combination of PAR and MEWS (AUC, 0.70) was more accurate for the composite outcome than either MEWS (AUC, 0.65) or PAR alone (AUC, 0.67; P = .01 and .04, respectively). Similar trends were observed for the individual outcomes (Figure 1). Examination of all calculated MEWS scores (N = 51 202) with forward imputation of the preceding PAR, regardless of lag time, resulted in a combined risk score with improved accuracy over MEWS alone for predicting the composite end point in the next 24 hours (Figure 2).

In this prospective, blinded validation study, we found that physician clinical judgment, as quantified by PAR, when added to the MEWS, improves its accuracy for predicting impending clinical deterioration on medical wards.

Clinical judgment-based risk assessments have been shown to be at least as accurate as, and often better than, objective risk scores for predicting mortality in critically ill patients.⁴ In ward patients, aggregated risk scores, such as the MEWS, are more predictive than single-parameter RRT activation criteria.³ However, these scores omit clinician worry, a frequent and early trigger in traditional systems.⁵ The PAR score quantifies this judgment such that it can be incorporated into risk scores.³

The accuracy of PAR for predicting cardiac arrest or ICU transfer in this study was notably lower than in the original study (AUC, 0.82). One potential explanation is that the current cohort had a lower rate of cardiac arrest or ICU transfer (1.0% vs 2.2%), which could reflect a healthier patient population, different ICU transfer or RRT activation thresholds, or the more frequent PAR assignment.

The weak correlation between the PAR and the MEWS is unlikely a factor of the variation in timing between the score assignments, as those close together were still poorly correlated. More likely, physicians were taking into account variables not contained in the MEWS, in assigning a PAR score.

We conclude that the combined use of PAR and MEWS is more accurate than MEWS alone. A generalizable way to incorporate clinical judgment into aggregated weighted scoring systems on the wards may improve detection of clinical deterioration and assist with timely mobilization of resources to address the etiology of clinical deterioration.

Corresponding Author: Dana P. Edelson, MD, MS, Department of Medicine, University of Chicago, 5841 S Maryland Ave, MC 5000, Chicago, IL 60637 (dperes@uchicago.edu).

Published Online: January 5, 2015. doi:10.1001/jamainternmed.2014.7119.

Author Contributions: Mr Zadravecz had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Williams, Edelson.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Patel, Zadravecz, Edelson.

Critical revision of the manuscript for important intellectual content: Patel, Young, Williams, Churpek, Edelson.

Statistical analysis: Zadravecz, Churpek.

Administrative, technical, or material support: Patel, Young, Edelson.

Study supervision: Patel, Churpek, Edelson.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This study was presented as a poster for the Hospital Medicine Annual Meeting; May 17, 2013; National Harbor, Maryland.