Risk Assessment Models for Venous Thromboembolism in Medical Inpatients

Key Points Question What is the prognostic performance of the simplified Geneva score and other validated risk assessment models (RAMs) to predict venous thromboembolism (VTE) in medical inpatients? Findings In this cohort study providing a head-to-head comparison of validated RAMs among 1352 medical inpatients, sensitivity of RAMs to predict 90-day VTE ranged from 39.3% to 82.1% and specificity of RAMs ranged from 34.3% to 70.4%. Discrimination was poor, with an area under the receiver operating characteristic curve of less than 60% for all RAMs. Meaning This study suggests that the accuracy and prognostic performance of the simplified Geneva score and other validated RAMs to predict VTE is limited and their clinical usefulness is thus questionable.


Introduction
Venous thromboembolism (VTE) represents one of the leading avoidable causes of death among hospitalized patients. 1Although particularly common among patients undergoing surgery, 2 about 75% of hospital-acquired cases of VTE occur in nonsurgical patients. 35][6] However, given the associated small increase in bleeding risk and the low baseline VTE incidence in the overall population of medical inpatients, 4,7 its provision should be targeted to patients at increased risk of VTE. 4,7,8though risk stratification in surgical patients is based mostly on the type of intervention, 2 assessment of VTE risk among medical patients is more challenging and requires integration of various individual risk factors. 9,10To simplify and standardize VTE risk assessment among medical inpatients, risk assessment models (RAMs) such as the original Geneva score, 11 the Padua score, 12 or the IMPROVE (International Medical Prevention Registry on Venous Thromboembolism) score 13 have been developed, and their use is encouraged by clinical guidelines. 7,8,14The practical usefulness of current RAMs is, however, limited by suboptimal sensitivities, 15 nonuniform cutoff values to define risk groups, 13,16 or a large number of variables. 11With the aim of developing a more usable RAM, the simplified Geneva score has been derived. 17A retrospective external validation study showed good discrimination and calibration of the simplified Geneva score 18 ; however, prospective validation is currently lacking.In addition, the comparative performance of validated RAMs has not been examined prospectively.Using data from a prospective multicenter cohort of medical inpatients, we aimed to validate the simplified Geneva score and to perform a head-to-head comparison of its prognostic performance with previously validated RAMs.

Study Design and Setting
RISE (Risk Stratification for Hospital-Acquired Thromboembolism in Medical Patients) is a multicenter prospective cohort study of medical patients admitted to 3 Swiss tertiary care hospitals from June 18, 2020, to January 4, 2022 (ClinicalTrial.govNCT04439383).The methods have been previously described. 19Reporting conforms to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline and checklist for prediction model validation. 20The study was conducted in accordance with all applicable legal and regulatory requirements.Authorization was granted from the responsible ethics committees (Kantonale Ethikkommission Bern, Commission cantonale d'éthique de la recherche sur l'être humain CER-VD, and Commission Cantonale d'Ethique de la Recherche sur l'être humain [CCER]), and written informed consent was obtained from all study participants.

Population
Consecutive adults hospitalized in general internal medicine were screened on weekdays, and eligible patients were enrolled within 72 hours of admission.We included acutely ill patients aged 18 years or older who were admitted for hospitalization for more than 24 hours.Exclusion criteria were indication for therapeutic anticoagulation, estimated life expectancy less than 30 days, transfer from the intensive care unit or other wards, insufficient German or French language proficiency, prior enrollment in the study, and unwillingness to provide informed consent.For patients unable to consent due to mental illness or cognitive impairment, written consent was obtained from an authorized representative.13]17 Treating physicians were not informed of the scores and the use of thromboprophylaxis was not influenced by the study.No RAM was implemented in order sets, but internal guidelines suggested to use the Padua score in 2 centers and the simplified Geneva score in 1 center to assess the indication for thromboprophylaxis.For patients at high risk of VTE, pharmacologic thromboprophylaxis was recommended, or nonpharmacologic prophylaxis for those at high bleeding risk.

Outcome
The primary outcome was symptomatic, objectively confirmed fatal and nonfatal VTE, including pulmonary embolism as well as distal and proximal deep vein thrombosis of the lower and upper extremity within 90 days of admission (eMethods in Supplement 1).To exclude preexisting VTE, we did not consider VTE diagnosed within 48 hours of admission. 21To assess VTE outcomes, study personnel blinded to RAM scores conducted follow-up visits on the day prior to discharge or the day of discharge, and contacted participants, their contact persons, and/or primary care physicians by telephone 90 days after admission. 11,22In case of a VTE outcome, medical and radiologic reports were collected to assess the date, type, and circumstances of the event.For participants who died, the cause was recorded based on medical reports, death certificates, and autopsy reports, if available (eMethods in Supplement 1).All VTE outcomes and deaths were adjudicated by a committee of 3 independent clinical experts blinded to RAM scores.

Statistical Analysis
The sample size was calculated to validate the simplified Geneva score for the prediction of hospitalacquired VTE.Assuming that 67% of patients would be categorized as high risk based on the b Defined as prior deep vein thrombosis or pulmonary embolism.
c Defined as antithrombin deficiency, activated protein C resistance, protein C or protein S deficiency, factor V Leiden, G20210A prothrombin mutation, or antiphospholipid syndrome.
d Defined as metastatic cancer or cancer treated with radiotherapy, chemotherapy, immunotherapy, or cancer surgery within last 6 months.
e Defined as complete bed rest or inability to walk for more than 30 minutes per day for 3 or more days.
f Defined as confinement to chair or bed with or without bathroom privileges for 7 or more days immediately prior to and during hospital admission.
g Defined as anticipated bed rest with or without bathroom privileges for 3 or more days.
h Defined as more than 6 hours within the last 7 days.
j Defined as aspirin plus other antiplatelet therapy.
k None of the patients received thromboprophylaxis with intermediate-dose low-molecular-weight heparin.
simplified Geneva score, and assuming a 90-day VTE incidence of 2.8% among patients at high risk and 0.6% among patients at low risk based on a previous study, 17 we determined that recruitment of 1308 patients would be required to detect an absolute risk difference of 2.2%, with a power of 80% at a 2-sided α of .05.To account for potential dropouts, we aimed to recruit 1350 participants.
Standard descriptive statistical tests were used to compare low and high VTE risk groups based on the simplified Geneva score.Time-to-event analyses with competing risk methods were used to assess the prognostic performance of the simplified Geneva score and the other RAMs and their association with VTE, with non-VTE death representing the competing risk, using a subdistribution hazard model of Fine and Gray. 23Subhazard ratios with 95% CIs were calculated, first unadjusted and then adjusted for pharmacologic thromboprophylaxis use as a time-varying covariate and for study site.Cumulative incidences of VTE among patients at low risk and patients at high risk were assessed using Kaplan-Meier curves, with calculation of P values based on log-rank tests.Sensitivity, specificity, and positive and negative predictive values and likelihood ratios were determined for each RAM.The area under the curve (AUC) was calculated to assess the discriminative power of each continuous score using time-dependent receiver operating characteristic curve analysis, considering censored data and competing events.Calibration was determined using the Hosmer-Lemeshow goodness-of-fit test; use of a calibration plot was not possible because 2 of the RAMs were derived empirically (ie, based on literature or clinical expertise) rather than data driven. 12,17,18tients for whom therapeutic anticoagulation was started for reasons other than VTE during follow-up were censored in the main analysis.Patients who were lost to follow-up were censored at the last visit.
We performed a subgroup analysis of patients who did not receive pharmacologic

Discussion
In this prospective, multicenter cohort of medical inpatients, the simplified Geneva score showed a similarly poor prognostic accuracy and discriminative performance for predicting VTE compared with the original Geneva score, the Padua score, and the IMPROVE score.The cumulative incidence of VTE within 90 days for low-risk and high-risk categories of all 4 RAMs did not significantly differ.Overall, our results suggest that existing RAMs do not perform particularly well in identifying medical inpatients at risk for VTE.
We found no association between risk group and time to a first VTE event for all 4 RAMs.
Although the overall incidence of VTE within 90 days was similar in our study compared with the derivation cohorts of the Geneva score, Padua score, and IMPROVE score, VTE incidence among those in the low-risk categories of our validation cohort was surprisingly high (1.1%-1.8%vs 0.3%-0.6% in the derivation cohorts of the RAMs) [11][12][13] and above the 1% threshold that has been suggested for provision of thromboprophylaxis. 7A potential explanation for the comparatively high VTE incidence in the low-risk groups could possibly be associated with the differing proportions of patients with pharmacologic thromboprophylaxis. 11,17The proportion of patients in the low-risk category receiving thromboprophylaxis was lower in our cohort (37.9% [174 of 459] based on the original Geneva score) 24 than in the derivation cohort (49%) of the original and simplified Geneva score, 17 or other large cohorts. 1,10e sensitivities of the RAMs based on our study were lower than in previous cohorts. 11,25,26For example, sensitivity ranged from 73% to 90% (for the original and simplified Geneva scores, Padua score, and IMPROVE score) in a post hoc analysis from a Swiss prospective cohort, and from 74% to 92% (for the Caprini score, IMPROVE score, and Padua score) in a retrospective analysis from the French PREVENU (Prevention of Venous Thromboembolism Disease in Emergency Departments) study. 17,26Sensitivity is critical in RAMs to select patients for whom a preventive intervention (ie, thromboprophylaxis) can be safely forgone. 27However, specificity should also be considered: use of the simplified and original Geneva scores to target thromboprophylaxis prescription may result in overtreatment due to their low specificity and high sensitivity.
The discriminative performance for 90-day VTE was poor in our study, with an AUC of 53.8% to 58.1%.Although some previous validation studies (based on retrospective data or post hoc analyses of prospectively collected data) showed better discriminative performance (AUC >70%), 16,17 poor results had also been reported in an external validation study of RAMs (including the Padua score and the IMPROVE score) using medical record data from Michigan hospitals, 28 as well as the retrospective analysis of the PREVENU study. 26ere are several potential explanations for the different results of our study and derivation or other validation studies of the Geneva score, Padua score, and IMPROVE score. 16,17First, VTE risk in low-and high-risk groups based on RAMs can be overestimated or underestimated by differing thromboprophylaxis use in the risk groups.Second, differences could be due to the definition of immobility, which differs between RAMs. 29Subjective estimation of mobility is inaccurate, 30 and often surrogates such as the ability to go to the bathroom are used to quantify mobility. 11,16,17As mobility is a highly weighted item in all these RAMs, objective mobility measures (eg, from accelerometry) may improve estimation of VTE risk.Third, data of derivation studies and some validation studies have been collected more than 10 years ago, 11,13,16,18 and inpatient care practices have changed within the last decade (eg, with shorter hospital stays, intensified in-hospital mobilization), with a direct association with VTE risk. 14Fourth, although our cohort is generally comparable with the population of the derivation studies (eTable 9 in Supplement 1), there may be unmeasured variations in characteristics associated with VTE risk.16,17 However, antiplatelet treatment did not have a relevant association with accuracy measures in our study.In addition, subsequent hospitalizations and subsequent use of thromboprophylaxis may be associated with 90-day VTE risk.
Given the overall limited accuracy and prognostic performance of all analyzed RAMs, our results cast doubts on their reliability to identify medical inpatients at risk of VTE for whom thromboprophylaxis is warranted.Even though guidelines, including those from the American College of Chest Physicians or the National Institute for Health and Care Excellence, encourage the use of RAMs to identify medical inpatients at high VTE risk, 7,14 our results emphasize the need for more accurate risk prediction strategies, as already advocated by others. 8For example, it is unclear whether the use of objective mobility measures or artificial intelligence-based models could improve VTE risk prediction. 19,31In addition, the clinical benefit associated with applying RAMs is unclear. 8,25cept for a single randomized trial that showed a reduction in VTE rates with a computer-alert program incorporating the Kucher RAM, 32 no prospective comparative study has, to our knowledge, demonstrated improved clinical or economic outcomes with the application of RAMs in clinical practice. 33The overall necessity of VTE risk stratification to implement targeted thromboprophylaxis may be questioned in light of the uncertain net clinical benefit associated with thromboprophylaxis for medical inpatients. 34,35Randomized clinical trials conducted more than 15 years ago showed up to 63% reductions in VTE with pharmacologic thromboprophylaxis compared with placebo, although the results were mainly due to a reduced risk of asymptomatic VTE of unclear clinical relevance. 6,36,37The recently published SYMPTOMS (Systematic Elderly Medical Patients Thromboprophylaxis: Efficacy on Symptomatic Outcomes) trial did not show significant differences in symptomatic VTE at 30 days in more than 2500 older medical inpatients randomized to enoxaparin or placebo, albeit the trial was underpowered due to premature termination. 38In addition, thromboprophylaxis does not reduce mortality in medical inpatients, 5 but may be associated with a small increase in bleeding risk based on results of a meta-analysis, 4 although we did not find such an association in data from our cohort. 39

Limitations
Our study has some limitations.First, results may have been affected by thromboprophylaxis use, and unadjusted accuracy measures are therefore difficult to interpret.To address this limitation, we conducted a subgroup analysis among patients without thromboprophylaxis, but the size of the subpopulation was small and thus the analysis was underpowered.Thromboprophylaxis was not assigned at random, which may have had a negative association with measures of accuracy and discrimination due to lower actual VTE rates among patients at high risk for VTE.However, the potential for this bias is reduced by the relevant proportion of underuse of thromboprophylaxis for patients at high risk and overuse for patients at low risk, as previously demonstrated in our cohort. 11,24,40In addition, all 4 RAMs were derived in populations of patients with or without thromboprophylaxis, and withholding thromboprophylaxis to perform a derivation or validation study would be unethical.Second, the number of VTE events was low, with large 95% CIs around the estimates.Even though differences in VTE risk between low-risk and high-risk groups were not statistically significant, they may still be clinically relevant.Third, given that patients were recruited from Swiss university hospitals, our results may not be generalizable to health care settings outside of high-income countries or White populations.Fourth, patients at high risk may have been underrepresented in our cohort, given that patients screened but excluded were older than those included, although this may be mostly explained by exclusion of populations for whom RAMs are irrelevant (eg, those receiving therapeutic anticoagulation or with a life expectancy <30 days).Fifth, we did not use specific criteria to define recurrent deep vein thrombosis. 41However, only 1 deep vein thrombosis event occurred in a patient with prior VTE.

Conclusions
To our knowledge, this cohort study provides the first prospective head-to-head comparison of validated RAMs.The easy-to-use simplified Geneva score showed similarly poor performance in predicting the risk for hospital-acquired VTE among medical inpatients compared with other validated RAMs.Overall, accuracy and prognostic performance of all analyzed RAMs were limited, questioning their clinical usefulness.More accurate strategies to predict VTE risk among medical inpatients as well as randomized studies evaluating the effect of risk assessment strategies are needed.
thromboprophylaxis at any time during hospitalization, and a subgroup analysis stratified by antiplatelet treatment during hospitalization.In a sensitivity analysis, we investigated how different outcome scenarios among patients lost to follow-up would be associated with discriminative performance of RAMs.The scenarios included VTE occurring (1) in all patients lost to follow-up, (2) in patients at high risk, and 3) in patients at low risk only.Stata, version 17 (StataCorp LLC), and R, version 4.2.2 (R Project for Statistical Computing), were used for all analyses.A 2-sided P < .05 was considered statistically significant.

Figure 1 .
Figure 1.Kaplan-Meier Plot Showing the Cumulative Incidence of Venous Thromboembolism Among Patients at Low and High Risk

Table 1 .
Baseline Patient Characteristics, Stratified by Low and High Risk of VTE According to the Simplified Geneva Score a

Table 2 .
Risk of Hospital-Acquired Venous Thromboembolism in High-Risk vs Low-Risk Groups Based on Each Risk Assessment Model predictive value of the simplified Geneva score was 2.6% (95% CI, 1.7%-3.9%),while the negative predictive value was 98.8% (95% CI, 97.4%-99.4%); the positive likelihood ratio was 1.25 (95% CI, 1.03-1.52),and the negative likelihood ratio was 0.58 (95% CI, 0.28-1.18).Positive predictive values, negative predictive values, and positive and negative likelihood ratios of the other RAMs a Adjusted for site and use of pharmacologic thromboprophylaxis as a time-varying covariate.positive

Table 3 .
Predictive Accuracy of Each Risk Assessment Model for Hospital-Acquired Venous Thromboembolism Abbreviations: IMPROVE, International Medical Prevention Registry on Venous Thromboembolism; LHR, likelihood ratio; NPV, negative predictive value; PPV, positive predictive value.

eTable 1 .
Venous Thromboembolism Risk Assessment Models eTable 2. Venous Thromboembolism Events in Low-and High-Risk Patients According to the Four Risk Assessment Models eTable 3. Discrimination and Goodness of Fit of Each Risk Assessment Model to Predict Hospital-Acquired Venous Thromboembolism eTable 4. Venous Thromboembolism Events in Patients Without Pharmacological Thromboprophylaxis According to the Four Risk Assessment Models eTable 5. Predictive Accuracy of Risk Assessment Models for Hospital-Acquired Venous Thromboembolism in Patients Without Pharmacological Thromboprophylaxis eTable 6. Venous Thromboembolism Events in Low-and High-Risk Patients According to the Four Risk Assessment Models, Stratified by Antiplatelet Treatment During Hospitalization eTable 7. Predictive Accuracy of Risk Assessment Models for Hospital-Acquired Venous Thromboembolism, Stratified by Antiplatelet Treatment During Hospitalization eTable 8. Sensitivity Analysis Investigating the Discriminative Performance of Risk Assessment Models With Different Outcome Scenarios Among Patients Lost to Follow-up eTable 9. Demographics, Predictors and Outcomes of Participants in the RISE Study and the Derivation Cohorts of the Four Risk Assessment Models eReferences.