Validation of the Caprini Venous Thromboembolism Risk Assessment Model in Critically Ill Surgical Patients

Importance  Appropriate risk stratification for venous thromboembolism (VTE) is essential to providing appropriate thromboprophylaxis and avoiding morbidity and mortality.

Objective  To validate the Caprini VTE risk assessment model in a previously unstudied high-risk cohort: critically ill surgical patients.

Design, Setting, and Participants  We performed a retrospective cohort study of 4844 adults (≥18 years old) admitted to a 20-bed surgical intensive care unit in a large tertiary care academic hospital during a 5-year period (July 1, 2007, through June 30, 2012).

Main Outcomes and Measures  The main study outcome was VTE (defined as patients with deep vein thrombosis or pulmonary embolism) that occurred during the patient’s initial hospital admission.

Results  The study population was distributed among risk levels as follows: low, 5.3%; moderate, 19.9%; high, 31.6%; highest, 25.4%; and superhigh, 14.9%. The overall incidence of inpatient VTE was 7.5% and increased with risk level: 3.5% in low-risk patients, 5.5% in moderate-risk patients, 6.6% in high-risk patients, 8.6% in highest-risk patients, and 11.5% in superhigh-risk patients. Patients with Caprini scores greater than 8 were significantly more likely to develop inpatient VTE events when compared with patients with Caprini scores of 7 to 8 (odds ratio [OR], 1.37; 95% CI, 1.02-1.85; P = .04), 5 to 6 (OR, 1.35; 95% CI, 1.16-1.57; P < .001), 3 to 4 (OR, 1.30; 95% CI, 1.16-1.47; P < .001), or 0 to 2 (OR, 1.37; 95% CI, 1.16-1.64; P < .001). Similarly, patients with Caprini scores of 7 to 8 were significantly more likely to develop inpatient VTE when compared with patients with Caprini scores of 5 to 6 (OR, 1.33; 95% CI, 1.01-1.75; P = .04), 3 to 4 (OR, 1.27; 95% CI, 1.08-1.51; P = .005), or 0 to 2 (OR, 1.38; 95% CI, 1.10-1.74; P = .006).

Conclusions and Relevance  The Caprini VTE risk assessment model is valid. This study supports the use of individual risk assessment in critically ill surgical patients.

Venous thromboembolism (VTE), which encompasses pulmonary embolism (PE) and deep venous thrombosis (DVT), represents a major source of morbidity and mortality among hospitalized patients. An estimated 10% of in-hospital deaths are attributed to PE,¹ whereas 50% of those diagnosed as having DVT develop the long-term sequelae of postthrombotic syndrome.² The incidence of VTE has increased with aging of the population, resulting in treatment costs in the billions of dollars per year.^3,4 The importance of VTE prevention in at-risk patients is highlighted by the Agency for Healthcare Research and Quality’s identification of VTE prevention as the most significant among 79 practices to improve patient safety and the US Surgeon General’s Call to Action for DVT and PE prevention.^5,6

The critically ill are at higher risk of developing VTE than other hospitalized patients.⁷ Multiple patient factors, including trauma, sepsis, immobilization, central venous catheters, and activation of proinflammatory and procoagulant cascades, have been implicated as contributing variables.^7-10 Observational studies^11,12 have described a 24% to 40% incidence of DVT in the intensive care setting with routine screening by duplex ultrasonography. Administration of chemical thromboprophylaxis significantly reduces risk of VTE in patients in intensive care units (ICUs).¹³ However, the benefits of thromboprophylaxis may be countered by the risk of postoperative bleeding and coagulopathy in this patient population.^14,15 Thus, VTE risk stratification represents an important tool for physicians to determine the appropriate thromboprophylaxis regimen. Routine use of VTE risk stratification is recommended by the American College of Chest Physicians.¹⁴

Several different VTE risk assessment models (RAMs) have been developed for use in the postsurgical population.^16-18 In these RAMs, different patient and procedural factors are assigned weighted risk scores, which are summed and used to assign risk. The concept of a weighted risk stratification tool for VTE has been championed by Joseph Caprini since the early 1990s. The 2005 version of the Caprini RAM is the most widely used and well-validated risk prediction for postsurgical patients.^19-22 Despite the increased risk faced by patients in the surgical ICU (SICU), there remains a paucity of data regarding appropriate risk stratification. This study was undertaken to determine whether the Caprini RAM could risk stratify a diverse group of SICU patients for in-hospital VTE events.

This study was performed as a retrospective cohort study of all admissions to a 20-bed SICU, encompassing general surgery, transplant, urology, and orthopedic patients and patients with respiratory failure requiring extracorporeal membrane oxygenation, in a large tertiary care academic hospital for a 5-year period (July 1, 2007, through June 30, 2012). Of the 4844 total patients, 3955 patients underwent a major operative procedure before admission to the SICU. Patients were retrospectively identified with internal billing and quality improvement records. Patients younger than 18 years were excluded. Thromboprophylaxis-prescribing regimens, including date, time, duration, and anticoagulation type and dose, were identified via query of the computer order entry system. The study took place at the University of Michigan Health Care System, was approved by the University of Michigan Institutional Review Board, and was performed in accordance with the Declaration of Helsinki. Informed consent was not required.

A previously validated, computer-generated retrospective risk scoring method based on the 2005 Caprini RAM¹⁹ was used to calculate the risk score for all patients at the time of ICU admission (Table 1). We have previously shown this computer-generated risk scoring method to have acceptable agreement with 2005 Caprini scores calculated by physician assistants (κ = 0.572)¹⁹ and that the score is a more rigorous estimate of VTE risk than physician-reported 2005 Caprini scores.²³ Data for each VTE risk factor before ICU admission were abstracted from multiple electronic resources, including hospital billing data, operating room information system database, and institutional clinical data repository. Quiz Ref IDThe retrospective risk score provided an estimate of VTE risk at the time of SICU admission. The score specifically excluded acquired risk factors, such as sepsis, insertion of central catheters, or additional surgical procedures that occurred after ICU admission. Identified risk factors were weighted according to the 2005 Caprini score, and the aggregate score was used to risk stratify patients (eTables 1 and 2 in the Supplement).

Quiz Ref IDThe primary outcome of interest was VTE (defined as patients with DVT or PE), which occurred during the patient’s initial hospital admission. Investigation for VTE was at the discretion of the ICU and/or surgical attending physicians because no formal screening protocol was in place. Deep vein thrombosis included acute thrombosis of lower-extremity veins (iliac, femoral, popliteal, or calf veins) or upper-extremity veins (axillary, subclavian, brachial, or internal jugular veins). Pulmonary embolism was defined as acute thrombosis within the pulmonary vasculature. Venous thromboembolism was considered present if identified with an objective imaging study, including duplex ultrasonography or PE protocol computed tomography. Patients who experienced sudden death were included if postmortem examination documented definitive evidence of VTE.

Analyses were performed using the STATA version 11 statistical package (StataCorp). The χ² test or Fisher exact test was used as appropriate to examine associations among individual risk factors embedded into the 2005 Caprini score and inpatient VTE events. Patients were stratified by Caprini score at accepted and published cutoffs.^19,20,22,23 Univariate regression examined the odds for VTE among patients at different Caprini risk levels. Rates of inpatient VTE stratified by Caprini score were examined. We used descriptive statistics that examined rates of chemoprophylaxis stratified by Caprini risk level and timing of initial chemoprophylaxis administration. Because chemoprophylaxis was not standardized and was individualized at the patient level, hundreds of potential combinations were present for chemoprophylaxis type, timing, duration, and intensity. Consequently, no further subgroup analyses based on chemoprophylaxis were performed.

This study examined 4844 consecutive admissions to our institution’s SICU, which cares for critically ill postsurgical patients and patients with respiratory failure with a mean Acute Physiology and Chronic Health Evaluation score greater than 50.²³Quiz Ref ID The observed rate of inpatient DVT was 6.4% (n = 308 events), and the inpatient PE rate was 1.6% (n = 79 events). There were a total of 135 patients with upper-extremity DVT, 131 patients with lower-extremity DVT, and 44 patients with both upper- and lower-extremity DVT. The VTE rate was 7.5% (n = 364 events), and 0.5% (n = 23 events) of patients had both DVT and PE. Mean (SD) time to VTE was 5.7 (13.6) days after admission to the ICU. The mean (SD) ICU length of stay was 5.5 (11.3) days. The mean (SD) hospital length of stay was 19.8 (27.5) days, and 308 patients (69.2%) with VTE had their VTE diagnosed before ICU discharge.

Patients were most commonly classified as being at moderate VTE risk at the time of ICU admission; 1533 of the total cohort (31.6%) had Caprini scores of 5 or 6 (Figure 1). A total of 723 patients (14.9%) were classified as being at superhigh risk at admission, with Caprini scores of greater than 8 (Figure 1). Quiz Ref IDMultiple risk factors embedded in the 2005 Caprini score were significantly associated with inpatient VTE events. These factors included younger age, recent sepsis or pneumonia, existing central venous access on admission, personal history of DVT or PE, known thrombophilia, and undergoing an operative procedure (Table 1).

The incidence of VTE increased in a linear fashion with increasing Caprini score (Figure 1). Quiz Ref IDPatients admitted to the ICU who were risk stratified as being at very low risk (Caprini score, 0-2) had a 3.5% rate of inpatient VTE. Those risk stratified as being at superhigh risk (Caprini score, >8) had an inpatient VTE rate of 11.5%. Patients with Caprini scores greater than 8 were significantly more likely to develop inpatient VTE events when compared with patients with Caprini scores of 7 to 8 (odds ratio [OR], 1.37; 95% CI, 1.02-1.85; P = .04), 5 to 6 (OR, 1.35; 95% CI, 1.16-1.57; P < .001), 3 to 4 (OR, 1.30; 95% CI, 1.16-1.47; P < .001), or 0 to 2 (OR, 1.37; 95% CI, 1.16-1.64; P < .001). Similarly, patients with Caprini scores of 7 to 8 were significantly more likely to develop inpatient VTE when compared with patients with Caprini scores of 5 to 6 (OR, 1.33; 95% CI, 1.01-1.75; P = .04), 3 to 4 (OR, 1.27; 95% CI, 1.08-1.51; P = .005), or 0 to 2 (OR, 1.38; 95% CI, 1.10-1.74; P = .006) (Table 2). We examined the predictiveness of the Caprini score for VTE using logistic regression. The Hosmer-Lemeshow goodness-of-fit test revealed acceptable fit, with concordant observed and expected tables. The Hosmer-Lemeshow χ² value was 3.93 (P = .69).

The proportion of patients who received postoperative chemoprophylaxis remained similar as Caprini risk level increased. Patients at higher risk levels (40.8% of patients with Caprini scores >8) were more likely to receive preoperative chemoprophylaxis than patients at lower risk levels (23.6% of patients with Caprini scores of 0-2). Patients at higher risk levels were less likely to receive no chemoprophylaxis than patients at lower risk levels (5.0% in the superhigh-risk cohort vs 25.1% in the lowest risk cohort; Table 3). Preferential intervention among high-risk patients may explain the area under the receiver operating characteristic curve seen with the 2005 Caprini score in this patient population (mean [SD], 0.5846 [0.0155]).

Venous thromboembolism represents one of the most common unsuspected autopsy findings in critically ill patients.²⁴ Indeed, delay in thromboprophylaxis is associated with increased mortality among ICU patients,²⁵ and inadequate risk assessment predicts future VTE events,²³ highlighting the need for rapid and accurate risk stratification. In this study, we found that VTE is common (7.5%) in the ICU, with most patients (69.2%) diagnosed as having VTE during hospitalization in the ICU compared with after transfer to the step-down or general care ward (30.8%). As is true for general, urology, vascular, otolaryngology, and plastic surgery patients,^19-21 we found that the Caprini RAM is a valid tool to predict VTE risk in critically ill surgical patients.

Among postoperative patients, development of VTE is associated with increased 30-day mortality.²⁶ Congruent with data reported from larger national (National Surgical Quality Improvement Program),¹⁹ regional (Michigan Surgical Quality Collaborative),¹⁸ and institutional²⁰ organizations, we found increased VTE risk in the setting of sepsis and central venous catheterization. Although the biological mechanism is poorly understood, others^20,26,27 have found remote infections, such as pneumonia and urinary tract infection, to be among the strongest positive predictors of symptomatic VTE. We also found an increased rate of VTE in the setting of pneumonia but not chronic obstructive pulmonary disease, suggesting that the inflammatory processes associated with acute infection may be causal in the development of venous thrombosis rather than simply pulmonary disease. This finding is supported by the observation that the early initiators of venous thrombosis, namely, elevation of cellular adhesion molecules and activation of neutrophils, is also characteristic of the acute inflammatory response associated with infection.^28-30 Personal history or thrombophilia was associated with increased risk in our patient population and in others,^18,19 underscoring the need for accurate medical records and good history taking because these factors represent the risk factors most commonly missed by physicians.^23,31

The Caprini score can identify a 10- to 15-fold variation in VTE risk among the overall surgical population and can effectively be used to identify those at very low risk (<1% at 30 or 60 days) and superhigh risk (>10% at 30 or 60 days).^20,21 Indeed, this is an accepted scoring system in the 2012 American College of Clinical Pharmacology VTE guidelines. The observed increases in VTE risk with increasing Caprini score were different for ICU patients when compared with the general, vascular, and urology patients, plastic and reconstructive surgery patients, and otolaryngology–head and neck surgery patients. Specifically, the last 3 groups had an exponential-appearing increase in VTE rate as risk score increased,^19-21 whereas ICU patients had a more linear increase in risk (Figure 2). This finding can be explained in several ways. In general, ICU patients at any Caprini risk level were at substantially higher risk than other patient populations. In addition, ICU patients are self-selected to have a higher illness severity than patients admitted to a general care ward, which likely comprised most of the other 3 studies. However, given the widely variable risk at each Caprini risk level, it is clear that there are factors contributing to VTE risk in the critically ill population that are not captured by the 2005 Caprini model. An important consideration is that the length of follow-up varied among studies. Both the study by Bahl et al¹⁹ of general, vascular, and urology surgical patients and the study by Shuman et al²⁰ of otolaryngology–head and neck surgery patients used 30-day VTE as the study end point. The study by Pannucci et al²¹ of plastic and reconstructive surgery patients used 60-day VTE. This study of SICU patients used inpatient VTE only, and the median length of hospitalization among our cohort was 11 days. It is particularly noteworthy that observed VTE rates at escalating Caprini risk levels were substantially higher in the critically ill population when compared with other validation studies. This finding supports our previous statement that risk factors not quantified by the 2005 Caprini score may be present in this population. Identification of these ICU-specific factors represents an important direction for future research, likely as a prospective investigation.

In this retrospective study, receipt of chemoprophylaxis was not standardized; thus, the patient population received widely variable chemoprophylaxis regimens. Because type, timing, duration, and intensity of chemoprophylaxis each varied in the patients, literally hundreds of different chemoprophylaxis patterns were present. As such, the data used for this study cannot rigorously be used to examine whether chemoprophylaxis effectively prevents VTE in the critically ill population, when controlling for baseline Caprini risk. We examined the proportion of patients who received several types of chemoprophylaxis based on Caprini score (Table 3). The proportion of patients who received postoperative chemoprophylaxis only remained similar among Caprini risk levels. As the Caprini score increased, the proportion of patients whose chemoprophylaxis was started before ICU admission increased, and the proportion of patients receiving no chemoprophylaxis decreased. Because surgeons were required to calculate a 2005 Caprini score at admission, they may preferentially have intervened on high-risk patients with chemoprophylaxis, which may explain the linear trend of increasing VTE risk by stratified Caprini score in SICU patients, as opposed to the exponential trend seen in other patient populations.^19-21 This finding may also explain the lower than expected Harrell’s concordance statistic and area under the receiver operator curve (mean [SD], 0.5846 [0.0155]). The 2005 Caprini score as a predictor of 60-day VTE has been associated with higher area under the receiver operating characteristic curve in a large cohort of plastic and reconstructive surgery patients who received no prophylaxis (C statistic = 0.71) (C.J.P. et al, unpublished data, 2014).²¹

Prior validation studies^19-21 of the 2005 Caprini score have identified a 15- to 20-fold variation in risk among the overall surgical population. The 2005 Caprini score was able to identify an approximately 3-fold variation in inpatient VTE risk among SICU patients. We believe that this decreased ability to risk discriminate is due to the high baseline risk of VTE seen among SICU patients (eg, risk of 3.5% in patients with 2005 Caprini scores of 0-2) and to surgeons preferentially providing prophylaxis to patients with higher risk scores on admission.

We acknowledge several limitations in this study. First, the retrospective scoring method used cannot identify all risk factors for patients and may understate the appropriate risk level of approximately 5% of the patient population (V.B., unpublished data, 2010). Second, although this study encompassed a sizable patient population, it was limited to one academic medical center. Replicating this study across a larger patient population would ensure broader applicability of the results. However, to extend the study to other centers, all must be capable of capturing the necessary patient data and applying the retrospective scoring system. Third, VTE outcomes were reported for the length of hospitalization, which may lead to underestimation of VTE risk in this patient population.¹⁶ Others^32,33 have found that the effect of hospitalization on VTE development can exceed 1 month. Ideally, a prospective study with follow-up of patients at 90 days would address this limitation.

Despite a high rate of thromboprophylaxis, critically ill patients are at high risk of breakthrough VTE, suggesting that current prophylaxis regimens could be improved. In addition, the risk of major bleeding in postsurgical critically ill patients highlights the need for an effective thromboprophylaxis agent that lacks the bleeding profile found with traditional therapy. Because the VTE risk in our low-risk cohort (3.5%) approximated that in the American College of Clinical Pharmacology guidelines’ definition of moderate risk (3.0%), it would be prudent to consider chemical thromboprophylaxis in even these low-risk patients, providing their underlying surgical illness does not confer a high bleeding risk. Identifying additional risk factors that elevate SICU patients to risk levels above and beyond the general postsurgical population and further delineation of characteristics of the highest-risk cohort, perhaps with a multicenter prospective study, may lay the foundation for determining which patients should receive further treatment. Higher-risk patients differentially benefit from chemoprophylaxis.³⁴ Some authors have recommended extended-duration prophylaxis (28 days) for those patients at superhigh risk,³⁵ similar to established recommendations for abdominal and pelvic cancer surgery patients.³⁶ Delineating the VTE risk over time and the risk of recurrent VTE may help further define the subset of patients at risk for breakthrough VTE or recurrent events and who may benefit from additional agents, such as aspirin.³⁷ The Caprini RAM is a valid and reliable tool for the assessment of VTE risk in the surgically critically ill.

Accepted for Publication: April 16, 2015.

Corresponding Author: Peter K. Henke, MD, Office of Performance Assessment and Clinical Effectiveness, University of Michigan Health System, 1500 E Medical Center Dr, Cardiovascular Center, Room 5463, Ann Arbor, MI 48109-5867 (henke@umich.edu).

Published Online: August 19, 2015. doi:10.1001/jamasurg.2015.1841.

Author Contributions: Drs Pannucci and Henke had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Obi, Pannucci, Wakefield, Henke.

Acquisition, analysis, or interpretation of data: Obi, Pannucci, Nackashi, Abdullah, Alvarez, Bahl.

Drafting of the manuscript: Obi, Pannucci, Henke.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Pannucci, Nackashi, Abdullah, Bahl.

Administrative, technical, or material support: Obi, Pannucci, Alvarez, Bahl.

Study supervision: Wakefield, Henke.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This study was presented in part at the American College of Surgeons Clinical Congress; October 13, 2013; Washington, DC.