Performance of Wells Score for Deep Vein Thrombosis in the Inpatient Setting

Importance  The Wells score to determine the pretest probability of deep vein thrombosis (DVT) was validated in outpatient settings, but it is unclear whether it applies to inpatients.

Objective  To evaluate the utility of the Wells score for risk stratification of inpatients with suspected DVT.

Design, Setting, and Participants  A prospective study was conducted in a 793-bed quaternary care, academic hospital using Wells score clinical predictor findings entered by health care professionals in a computerized physician order entry system at the time lower-extremity venous duplex ultrasound studies were ordered. All adult inpatients suspected of having lower-extremity DVT who underwent lower-extremity venous duplex ultrasound studies between November 1, 2012, and December 31, 2013, were included. Patients with DVT diagnosed within the prior 3 months were excluded. For patients undergoing multiple lower-extremity venous duplex ultrasound studies, only the first was included.

Main Outcomes and Measures  Our primary outcome was the Wells score’s utility for risk stratification among inpatients with suspected DVT as measured by the difference in incidence of proximal DVT among the 3 Wells score categories (low, moderate, and high pretest probability), the discrimination accuracy of the Wells score categories as the area under the receiver operating characteristics curve, the failure rate of Wells score prediction, and the efficiency of the Wells score to exclude DVT.

Results  In a study cohort of 1135 inpatients, 137 (12.1%) had proximal DVT. Proximal DVT incidence in low, moderate, and high pretest probability groups was 5.9% (8 of 135), 9.5% (48 of 506), and 16.4% (81 of 494), respectively (P < .001). The area under the receiver operating characteristics curve for the discriminatory accuracy of the Wells score for risk of proximal DVT identified on lower-extremity venous duplex ultrasound studies was 0.60. The failure rate of the Wells score to classify patients with a low pretest probability was 5.9% (95% CI, 3.0%-11.3%); the efficiency was 11.9% (95% CI, 10.1%-13.9%).

Conclusions and Relevance  The Wells score performed only slightly better than chance for discrimination of risk for DVT in hospitalized patients. It had a higher failure rate and a lower efficiency in the inpatient setting compared with that reported in the outpatient literature. Therefore, the Wells score risk stratification is not sufficient to rule out DVT or influence management decisions in the inpatient setting.

More than 8 million inpatients have a high risk for deep vein thrombosis (DVT) annually in the United States, with potentially fatal complications such as pulmonary embolism.¹ As DVT signs and symptoms are often nonspecific, there is a low threshold to order a lower-extremity ultrasound study (LEUS) to rule out DVT.^2,3 To reduce unnecessary imaging, the Wells score was derived to determine a patient’s pretest probability of DVT.⁴ Subsequently, the Wells score has been validated in outpatient and emergency department settings.^5-7

However, inpatients may have different DVT risk profiles owing to use of thromboprophylaxis⁸ and increased prevalence of risk factors such as heart failure, chronic obstructive pulmonary disease, acute infection, atherosclerosis, cancer, immobilization, and recent surgical procedures.^8,9 While some preliminary studies have suggested that the Wells score might not apply in the inpatient setting,¹⁰ to our knowledge, no large prospective trial to specifically validate the use of the Wells score in hospitalized patients has been conducted. Therefore, the aim of this study was to evaluate the utility of the Wells score for risk stratification of inpatients with suspected DVT.

Brigham and Women’s Hospital Institutional Review Board approval was obtained for this Health Insurance Portability and Accountability Act–compliant prospective study performed at a 793-bed, quaternary care, urban academic hospital. The institutional review board waived the requirement for informed patient consent.

From 5712 patients who underwent extremity ultrasound study in our institution between November 1, 2012, and December 31, 2013, we excluded 3178 outpatients (including those presenting to the emergency department) and 1221 patients who had only upper-extremity studies. To prevent inclusion of follow-up studies, 178 patients with DVT diagnosed within the prior 3 months were also excluded. For patients with multiple LEUS during the study period, only the first examination was considered. We included all remaining (n = 1135) inpatients 16 years or older with suspected DVT who underwent LEUS during the study period.

Our primary outcome was the Wells score’s utility for risk stratification for proximal DVT as measured by the difference in incidence of proximal DVT among the 3 Wells categories (low, moderate, and high pretest probability), the discrimination accuracy of the Wells score categories as the area under the receiver operating characteristics curve (AUC), the failure rate of Wells score prediction, and the efficiency of the Wells score to exclude DVT. Failure rate was defined as mean predicted probability of DVT in patients with a low probability Wells score.⁶ Efficiency was defined as the proportion of patients in the low probability group.⁶

Second, we evaluated the difference in incidence of isolated distal DVT among the 3 Wells categories and compared patients’ baseline characteristics and incidence of proximal DVT by Wells score category among 3 hospital services: surgery, medicine, and hematology/oncology. Among patients’ baseline characteristics, we evaluated the use of pharmacologic thromboprophylaxis, defined as inpatient use of subcutaneous heparin sodium, 5000 U 2 or 3 times daily, or subcutaneous enoxaparin sodium, 30 or 40 mg/d, between the time of hospital admission and LEUS. Heparin and enoxaparin at these dosages are the most commonly used agents for pharmacologic thromboprophylaxis. While other agents, such as fondaparinux sodium and warfarin sodium, may be used in selected patient populations, these are rare occurrences.

Clinical predictors to calculate the Wells score were entered by physicians or physician assistants ordering LEUS using a data capture screen integrated into our computerized physician order entry and clinical decision support systems (Figure 1). A response to each clinical predictor was required to complete the order entry process. To validate the accuracy and completeness of this data entry, we manually reviewed objective clinical data (history of DVT, cancer, immobilization, and surgical procedures) from a random sample of 100 medical records. This number was selected based on a calculation of 80% power, α value of 0.05, and a 0.1 proportion difference. Patients received 1 point for the presence of each clinical predictor (recently bedridden or major surgical procedure; entire leg swollen; localized tenderness along the distribution of deep venous system; active cancer; pitting edema; calf swelling; paralysis, paresis, or recent plaster immobilization of lower extremity; previously documented DVT; or collateral superficial veins), except for the presence of an alternative diagnosis, which subtracted 2 points. The sum of points was used to classify the patient’s pretest probability of DVT as low (≤0), moderate (1 or 2), or high (>2) according to the Wells score.⁵ The ordering and the interpreting physicians or physician assistants were unaware of the patient’s Wells score.

Each LEUS report was classified as positive or negative for lower-extremity DVT, using a natural language processing algorithm developed using General Architecture for Text Engineering (University of Sheffield),¹¹ open-source software that has been successfully used in radiologic report text mining.^12,13 To validate the natural language processing algorithm, we randomly selected 400 deidentified LEUS reports from the full cohort for manual review by a board-certified medicine attending physician (I.K.I.). This sample size was powered for 95% sensitivity (95% CI, 92.5%-97.5%) and 85% specificity (95% CI, 82.5%-87.5%), with an α value of .05. We then manually categorized each positive case as proximal or isolated distal. The thrombus anatomic location was classified as proximal if the popliteal or more proximal veins were involved and distal if isolated to the calf veins. Since proximal DVTs are associated with increased adverse clinical outcomes, cases where proximal and distal DVTs co-occurred were classified as proximal. Therefore, adding the 2 rates will result in the total DVT rate.

Statistical analysis was performed using commercially available software (JMP Pro, version 10, SAS Institute). Area under the receiver operating characteristics curve, in which true-positive rates are plotted against the false-positive rates (ie, sensitivity against 1 – specificity),¹⁴ was used to measure the discriminatory accuracy of the Wells score for risk of proximal DVT identified on the LEUS. An ideal scoring system would have an AUC of 1.0, and a scoring system with no discrimination utility has an AUC of 0.5. A χ² test was used to compare the overall incidence of proximal and isolated distal DVT among the 3 pretest probability categories and by hospital service. Baseline characteristics among hospital services were compared using χ² tests for categorical variables and analysis of variance when comparing means. A low number of patients (5.2%) were not hospitalized under 1 of the 3 primary hospital services and were excluded from the analysis by hospital service. A 2-tailed P < .05 was considered statistically significant. A sample size of 800 patients was calculated to provide 80% power to detect a difference in incidence of proximal DVT of 0.5% to 1%.

Characteristics of 1135 patients with suspected DVT during the study period are summarized in Table 1. Most patients (70.6%; 95% CI, 67.9%-73.2%) were recently bedridden or had a major surgical procedure in the previous 12 weeks and more than one-third (36.8%; 95% CI, 34.1%-39.7%) had active cancer. d-dimer level was elevated in 86 of the 92 patients (93.5%; 95% CI, 86.5%-97.0%) for which it was ordered. Various patients’ baseline characteristics, such as hospital length of stay, comorbidities, thromboprophylaxis use, and Wells score distribution, were significantly different among the 3 major hospital services (Table 2). When compared with the explicit review of medical records, the data capture screen entry had a sensitivity of 91% and specificity of 88%. The natural language processing algorithm to detect positive DVT on LEUS reports had a sensitivity and specificity of 100%.

Lower-extremity venous duplex ultrasound studies showed proximal DVT in 137 of 1135 patients (12.1%; 95% CI, 10.3%-14.1%). Incidence of DVT in the low, moderate, and high pretest probability groups was 8 of 135 (5.9%; 95% CI, 3.0%-11.3%), 48 of 506 (9.5%; 95% CI, 7.2%-12.3%), and 81 of 494 (16.4%; 95% CI, 13.4%-19.9%), respectively (P < .001). The AUC for the discriminatory accuracy of the Wells score for risk of proximal DVT identified on LEUS was 0.60 (Figure 2). The failure rate of the low probability score to rule out proximal DVT was 5.9% (95% CI, 3.0%-11.3%), and the efficiency was 11.9% (95% CI, 10.1%-13.9%).

Isolated distal DVT was identified by LEUS in 104 of 1135 patients (9.2%; 95% CI, 7.6%-11.0%). Incidence of isolated distal DVT in the low, moderate, and high pretest probability groups was 10 of 135 (7.4%; 95% CI, 4.1%-13.1%), 46 of 506 (9.1%; 95% CI, 6.9%-11.9%), and 48 of 494 (9.7%; 95% CI, 7.4%-12.6%), respectively (P = .71).

For the low and moderate pretest probability groups, the incidence of proximal DVT among the 3 hospital services was not significantly different. For the high pretest probability group, the incidence of proximal DVT was 25 of 223 (11.2%; 95% CI, 7.7%-16.0%), 31 of 135 (23.0%; 95% CI, 16.7%-30.7%), and 24 of 116 (20.7%; 95% CI, 14.3%-28.9%) among patients hospitalized in the surgical, medical, and hematology/oncology services, respectively (P = .007). Overall, the incidence of proximal DVT was almost 2 times higher among patients hospitalized in the medical or hematology/oncology services (65 of 423 [15.4%; 95% CI, 12.2%-19.1%] and 33 of 214 [15.4%; 95% CI, 11.2%-20.9%], respectively) compared with those hospitalized in the surgical service (37 of 439 [8.4%; 95% CI, 6.2%-11.4%]).

This is the largest study of which we are aware to evaluate the utility of the Wells score for suspected DVT in the inpatient setting. We found that the incremental difference in incidence of proximal DVT in inpatients across the low, moderate, and high pretest probability groups was narrow in range (5.9%, 9.5%, and 16.4%, respectively) compared with the much broader range for outpatients (3.0%, 16.6%, and 74.6%, respectively) reported by Wells et al.⁵ The Wells score performed only slightly better than chance for discrimination of risk for DVT in hospitalized patients (AUC, 0.60). This finding confirms previous smaller studies’ findings for inpatients (AUC, 0.56)¹⁰ and intensive care unit patients (AUC, 0.59).¹⁴ In addition, we found that the Wells score failure rate was almost 2 times higher in our study compared with that in the outpatient setting (5.9% vs 3.0%).⁵ The efficiency of the Wells score in the inpatient setting was much lower in our study compared with that in the original outpatient validation study (11.9% vs 55.5%).⁵

Our results suggest that the Wells score risk stratification is not sufficient to rule out DVT or influence management decisions (eg, starting anticoagulation while awaiting LEUS result) in the inpatient setting, as has been previously suggested.¹⁰ Potential explanations rely on differences in characteristics of inpatients compared with the outpatient population, such as a 6 times higher prevalence of recent immobilization or surgical procedures, a 3 times higher prevalence of active cancer, and routine use of venous thromboembolism prophylaxis for patients deemed to be at high risk for DVT.⁵ Inpatients are also more likely to have heart failure, chronic obstructive pulmonary disease, acute infection, and atherosclerosis, all of which increase the risk for DVT and none of which are part of the Wells score. In addition, studies have shown that d-dimer testing, part of the outpatient management algorithm, is inefficient in the hospital setting because most inpatients have a positive result owing to other comorbidities,^15,16 concordant with our findings.

A recently published meta-analysis determined that, except for patients with active cancer, the Wells score is a useful DVT pretest probability predictor in the outpatient, primary care, and emergency department settings, but the inpatient subgroup was not evaluated.⁶ Two previous studies described good performance of the Wells score among inpatients.^17,18 However, both the Constans et al¹⁷ and Wells et al¹⁸ studies were performed more than 10 years ago and the sample sizes were 150 and 273 patients, respectively. They found a 9% to 10% incidence of DVT in the low-risk category–clearly too high of a rate to forgo ultrasound study and the discriminatory ability was not assessed. Our contemporary study was performed using state-of-the-art ultrasound scans, included 1135 patients, and assessed multiple other measures (AUC, failure rate, and efficiency) of the Wells score performance.

In the high pretest probability group, patients hospitalized in the medicine or hematology/oncology services had an incidence of proximal DVT 2 times higher than that of surgical patients. While the cause is unclear, it is likely multifactorial, such as patients’ level of physical activity and mobility, length of stay, comorbidities, and use of pharmacologic thromboprophylaxis, which was more common among surgical services than hematology/oncology or medical services (67% vs 40%-59%), confirming prior results.¹⁹ It is also conceivable that patients hospitalized in surgical services undergo ultrasonography more often compared with patients hospitalized in other services. In addition, patients who were hospitalized for elective surgical procedures might be healthier than those in medical services.

Lower-extremity ultrasound study is well established as the definitive test for proximal DVT, with sensitivity and specificity reported to be approximately 94%, while the sensitivity of LEUS for distal DVT is reported to be only 64%.²⁰ The need to test for and manage isolated distal DVTs remains controversial,²¹ and current American College of Chest Physicians recommendations are based on moderate- or low-quality evidence.²² Results from ongoing clinical trials might provide definitive evidence on the topic.^23,24 Generally, the Wells score is considered a predictor only for proximal DVT.^2,5,6 Our findings are consistent with prior evidence that the Wells score is not an effective prediction rule for isolated distal DVT for inpatients.^10,25

Our study also highlights the potential role of health information technology tools, including a computerized physician order entry system, clinical decision support, and a natural language processing algorithm to simplify future large-scale prospective validation of prediction rules. Instead of presenting an alert to guide clinical decision-making in response to an order placed in the computerized physician order entry system, clinical decision support tools can be modified to ensure prospective capture of needed discrete data without the need for laborious manual medical record review that can be compromised by lack of documentation of needed data in the electronic health records.

This study had several limitations. First, we did not evaluate patients with suspected DVT who did not undergo LEUS. However, since LEUS is necessary to definitively diagnose DVT in the inpatient setting, it is likely that patients with a reasonable suspicion of DVT underwent LEUS unless they were empirically treated for DVT without definitive diagnosis, which would be less likely in our practice. Second, LEUS in patients with suspected DVT is limited particularly by false-negative results. Third, the study was conducted in a single academic institution and may not be representative of the general population. We did not control for the use of thromboprophylaxis in this study, which may differ in our institution compared with other hospitals owing to concerted quality improvement efforts.²⁶ Finally, the data entered at the time of LEUS order might be incomplete or discordant with the patient’s actual presentation. However, a previous study has shown that more than 90% of information entered by physicians and physician assistants into an imaging clinical decision support system at our institution is accurate,²⁷ and our review of a random sample of medical records corroborated this finding.

The Wells score performed only slightly better than chance for discrimination of risk for DVT in hospitalized patients. It had a higher failure rate and a lower efficiency in the inpatient setting compared with that reported in the outpatient literature. Therefore, the Wells score risk stratification is not sufficient to rule out DVT or influence management decisions in the inpatient setting.

Accepted for Publication: February 12, 2015.

Corresponding Author: Patricia C. Silveira, MD, Center for Evidence-Based Imaging, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, 20 Kent St, Second Floor, Boston, MA 02445 (pcbsilveira@gmail.com).

Published Online: May 18, 2015. doi:10.1001/jamainternmed.2015.1687.

Author Contributions: Dr Silveira had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Ip, Goldhaber, Piazza, Benson, Khorasani.

Acquisition, analysis, or interpretation of data: Silveira, Ip, Goldhaber, Piazza, Khorasani.

Drafting of the manuscript: Silveira, Ip, Goldhaber, Benson.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Silveira, Ip, Khorasani.

Obtained funding: Khorasani.

Administrative, technical, or material support: Benson, Khorasani.

Study supervision: Ip, Goldhaber, Piazza, Benson, Khorasani.

Conflict of Interest Disclosures: Dr Khorasani is a consultant to Medicalis Corporation. Dr Khorasani is named on US Patent 6,029,138 held by Brigham and Women’s Hospital on clinical decision support–related software licensed to Medicalis Corporation in year 2000. As the result of licensing, Brigham and Women’s Hospital and its parent organization, Partners Healthcare, Inc, have equity and royalty interests in Medicalis. No other disclosures were reported.