A Mantel-Haenszel random-effects model was used to conduct the meta-analysis. The size of each data marker corresponds to the weight of the study assigned by the random-effects model. RR indicates risk ratio.
eTable 1. Search Strategies per Database
eTable 2. MOOSE Checklist
eTable 3. NOS Score for Observational Studies
eFigure 1. Summary of Risk of Bias Assessment
eFigure 2. Forest Plot Comparing Overall Rate of Ordering VTE Prophylaxis Using CCDSSs vs Routine Care
eFigure 3. Forest Plot Comparing Rate of Ordering VTE Prophylaxis Using CDSSs vs Routine Care Subgrouped by CDSSs That Featured Order Autopopulation and Those That Recommended but Could Not Autopopulate the Order
Customize your JAMA Network experience by selecting one or more topics from the list below.
Borab ZM, Lanni MA, Tecce MG, Pannucci CJ, Fischer JP. Use of Computerized Clinical Decision Support Systems to Prevent Venous Thromboembolism in Surgical Patients: A Systematic Review and Meta-analysis. JAMA Surg. 2017;152(7):638–645. doi:10.1001/jamasurg.2017.0131
Do computerized clinical decision support systems decrease the risk of venous thromboembolism (VTE) in surgical patients?
In this systematic review and meta-analysis, increased rates of prophylaxis for VTE and decreased VTE events were associated with the use of computerized clinical decision support systems compared with routine care without decision support.
Computerized clinical decision support systems should be implemented to help clinicians assess the risk of VTE and provide the appropriate prophylaxis to prevent VTE events experienced in surgical patient populations.
Health care professionals do not adequately stratify risk or provide prophylaxis for venous thromboembolism (VTE) among surgical patients. Computerized clinical decision support systems (CCDSSs) have been implemented to assist clinicians and improve prophylaxis for VTE.
To evaluate the effect of implementing CCDSSs on the ordering of VTE prophylaxis and the rates of VTE.
PubMed, MEDLINE via OVID, EMBASE via OVID, Scopus, Cochrane CENTRAL Register of Controlled Trials, and clinicaltrials.gov were searched in June 2016 for articles published in English from October 15, 1991, to February 16, 2016. A manual search of references from relevant articles was also performed.
Clinical trials and observational studies among surgical patients comparing CCDSSs with VTE risk stratification and assistance in ordering prophylaxis vs routine care without decision support were included. Of the 188 articles screened, 11 (5.9%) were eligible for meta-analysis.
Data Extraction and Synthesis
Meta-analysis of Observational Studies in Epidemiology guidelines were followed. Two reviewers extracted data and assessed quality independently.
Main Outcomes and Measures
Rates of prophylaxis for VTE and VTE events. Random- and fixed-effects models were used to summarize odds ratios and risk ratios.
Eleven articles (9 prospective cohort trials and 2 retrospective cohort trials) comprising 156 366 individuals (104 241 in the intervention group and 52 125 in the control group) were included. The use of CCDSSs was associated with a significant increase in the rate of appropriate ordering of prophylaxis for VTE (odds ratio, 2.35; 95% CI, 1.78-3.10; P < .001) and a significant decrease in the risk of VTE events (risk ratio, 0.78; 95% CI, 0.72-0.85; P < .001).
Conclusions and Relevance
Use of CCDSSs increases the proportion of surgical patients who were prescribed adequate prophylaxis for VTE and correlates with a reduction in VTE events.
The leading cause of preventable in-hospital death is deep venous thrombosis (DVT) leading to pulmonary embolism (PE).1,2 Both DVTs and PEs are a significant risk for morbidity and mortality, especially during the postoperative period. Without appropriate prophylaxis, up to 25% to 60% of patients who undergo surgery develop a DVT that is detectable by venography.1 Clinicians are equipped with tools to help decrease the risk of venous thromboembolism (VTE) up to 50% among the patients at highest risk.3-6 At their disposal are risk-reduction methods including modification of anesthetic technique, modification of risk factors, mechanical prophylaxis (eg, elastic compression stockings and intermittent pneumatic compression), and chemical prophylaxis (eg, unfractionated heparin, low-molecular-weight heparin, or newer oral anticoagulants). For example, for patients undergoing orthopedic surgery, it is known that if appropriate prophylaxis is used, symptomatic DVT and PE can be reduced by 55% to 60%.7 Despite the availability of these devices and medications and their profound utility, there remains a barrier to an appropriate assessment of the risk of VTE and the ordering of prophylaxis for VTE.8
The problem of VTE has become so widespread and costly that health care quality improvement initiatives have been launched with prevention of VTE as a major interest. The Joint Commission and the Surgical Care Improvement Project have included measures to assess and mandate risk stratification and prophylaxis for VTE for all surgical patients.9 Six VTE-associated outcome measures were endorsed by the National Quality Forum and aligned with the Centers for Medicare & Medicaid Services.10 Since implementation, the Joint Commission’s technical advisory panel maintains these measures and, through a nationwide program, uses them to evaluate hospitals on core measures.
Currently, guidelines are available to help risk-stratify patients into low, moderate, or high risk for VTE and provide the corresponding prophylaxis measure.7,11,12Quiz Ref ID However, clinicians do not uniformly use existing risk-stratification tools and, when used, clinicians often use the tools incorrectly, producing an underestimation of a patient’s risk for VTE.13,14 This underestimation leaves patients without adequate prophylaxis for VTE and significantly increases their risk for postoperative DVT or PE. As clinician-level risk stratification has been shown to be inaccurate, many hospital systems use computerized clinical decision support systems (CCDSSs) to help physicians assess an individual patient’s risk level and overcome this barrier in providing adequate prophylaxis. A CCDSS is rule- or algorithm-based software that can be integrated into an electronic health record (EHR) and uses data to present evidence-based knowledge at the individual patient level.15
Most surgical patients are at an elevated risk for VTE, and CCDSS risk models embedded within the EHR may enhance the physicians’ ability to more accurately and efficiently assess and mitigate a patient’s risk for VTE. However, the question still remains: Does increased VTE prophylaxis with the use of CCDSSs correlate with downstream risk reduction of VTE events? The aim of this systematic review and meta-analysis is to assess the effect CCDSSs on increasing adherence to guidelines for VTE prophylaxis and on decreasing VTE events postoperatively compared with routine care without CCDSSs.
This study was performed according to the Meta-analysis of Observational Studies in Epidemiology guidelines.16 A systematic literature search was conducted to identify all articles published from October 15, 1991, to February 16, 2016, involving the use of CCDSSs to provide prompts for VTE prophylaxis for individual surgical patients. PubMed, MEDLINE via OVID, EMBASE via OVID, Scopus, Cochrane CENTRAL Register of Controlled Trials, and clinicaltrials.gov were searched. The search strategy used for each database is reported in eTable 1 in the Supplement. A manual search of references from relevant articles was also performed. The final search was performed on June 1, 2016. Articles that included both surgical and medical patients were potentially eligible for inclusion. In this event, we planned to contact the corresponding author directly to request a subgroup analysis specific to surgical patients.
Studies were evaluated for inclusion by 2 reviewers (Z.M.B. and M.A.L.), with a third reviewer (M.G.T.) assigned to resolve any discrepancies. After identifying relevant studies through title and abstract information, evaluation of the full text was performed to include only studies that used a CCDSS to augment the rate of appropriate prophylaxis of VTE. Additional inclusion criteria were patients in surgical services who were of any age- or risk-stratification level, trials that evaluated CCDSSs in a controlled study design, and studies that were published in English. We defined computerized clinical decision support systems as any electronic system designed to aid directly in clinical decision making, in which 2 or more characteristics of individual patients are used to generate patient-specific assessments or recommendations that are then presented to clinicians for consideration.15 The minimum requirement of the CCDSS tool was to provide aid to the user to appropriately prescribe prophylaxis for VTE within the EHR domain. Studies were excluded if they did not meet the above criteria, if outcomes were not evaluated, or if the study did not include a surgical cohort.
Data were extracted independently by 2 members of the study team (Z.M.B. and M.A.L.) using a template data extraction sheet, with a third member (M.G.T.) serving as a tiebreaker when consensus was not achieved. The type of information gathered included study characteristics (study design, year of publication, geographical location, patient population, and cohort size), patient demographics, perioperative details, and outcome measures. The number of patients recorded at follow-up, rather than the enrollment number, was used as the denominator. Primary outcomes included the rate of appropriate prophylaxis for VTE and the rate of VTE events. Studies were rated using the Oxford Centre For Evidence-Based Medicine Level of Evidence Scale.17 Methodological quality was assessed using the Newcastle-Ottawa Quality Assessment Scale and the Cochrane Collaboration Risk of Bias tool (meta-analysis checklist was followed and is available in eTable 2 in the Supplement).18,19
Patients were classified into 2 groups according to intervention: CCDSS-guided care or routine care without the use of a CCDSS. Patient demographics were reported via standard summary statistics. Data points were pooled for meta-analysis when interventions and outcomes were sufficiently similar to compare. A Mantel-Haenszel random or fixed model was used to pool data. Meta-analyses of dichotomous outcomes were reported as odds ratios or risk ratios, 95% CIs, and with P < .05 (2-sided) considered statistically significant. Heterogeneity across studies was assessed using the χ2 test. P = .10 was considered statistically significant for measures of heterogeneity. The I2 statistic was used to quantify heterogeneity.20 Sensitivity analyses were conducted by sequential study omission to explore potential explanations for heterogeneity and any effect on statistical significance. Publication bias was explored by constructing funnel plots where appropriate (ie, when there were ≥10 studies in a pooled analysis).21 All analyses were conducted in Stata IC, version 13.1 (StataCorp), and figures were generated with RevMan, version 5.3 (The Cochrane Collaboration).
The initial literature search yielded 210 articles, and after removal of duplicates, 187 individual articles remained. After screening by title and abstract, 25 articles were identified and underwent review of the full text. A total of 12 articles met the inclusion criteria. A manual reference search of included articles yielded 1 additional article that met inclusion criteria (Figure 1). Two articles identified in the original search could not be included because the data were insufficient for our analysis. We contacted either the first or senior author to directly discuss the data; in both cases, the primary data were no longer available for reanalysis.22,23
Eleven individual articles (9 prospective cohort trials and 2 retrospective cohort trials) were included in this systematic review and meta-analysis.24-34 In total, 156 366 participants were analyzed, of which 104 241 received intervention with a CCDSS to aid in ordering prophylaxis of VTE and 52 125 were not exposed to an intervention. The standardized mean (SD) age of the studied population was 51.6 (11.7) years, and the population was a standardized mean (SD) 47.8% (24.1%) female. Surgical services included general, transplantation, cardiothoracic, vascular, trauma, orthopedic, neurosurgery, otolaryngology, plastic and reconstructive, urology, and gynecology. Details of the included studies are provided in Table 1.
There were no randomized clinical trials that met the inclusion criteria. Of those that met the criteria, most were weakened by the possibility of bias being introduced owing to differences in data collection or recording methods. Owing to the nature of the intervention, the risk of the Hawthorne effect was present in most studies. Four studies were at a high risk of sampling bias owing to an unbalanced patient population or the selective exclusion of high-risk patients24,25,27,30 (eFigure 1 in the Supplement). The Newcastle-Ottawa Quality Assessment Scale was used to assess all studies and assign summary scores (Table 1) (eTable 3 in the Supplement).
All studies tested the effect of the integration of a CCDSS into an EHR. The CCDSSs were either developed in house,24,25,29-31,33,34 purchased,26 or not described in the Methods section of the article.27,28,32 Three of the CCDSSs were linked to the admission note,28,31,34 1 to the preoperative assessment,30 1 to the postoperative note,24 and 6 studies did not specify to which page the CCDSS was linked.25-27,29,32,33 Data points were pulled automatically by the CCDSSs in 4 studies to risk-stratify patients.26,29,33,34 In 7 studies, patient data were entered manually into checklists, and the CCDSS algorithm would risk-stratify based on the data provided.24,25,27,28,30-32 Three of the studies featured CCDSSs that could autopopulate the suggested prophylaxis on the order screen.25,30,31 Suggestions were made by CCDSSs but not linked to the order screen in 6 studies (Table 1).24,26-28,33,34 The control groups in 10 of the studies were patient encounters recorded prior to the implementation of the CCDSS into the EHR.25-34 One study performed a time series experiment in which the CCDSS was enabled for intervention periods and disabled for control periods.24
Ten trials reported the effect of CCDSSs on the appropriate ordering of prophylaxis for VTE.24-32,34Quiz Ref ID Pooled meta-analysis of 98 639 patients who received the CCDSS intervention and 50 175 controls revealed that CCDSSs significantly increased the rate of ordering prophylaxis for VTE when compared with controls (odds ratio, 2.35; 95% CI, 1.78-3.10; P < .001) (Table 2) (eFigure 2 in the Supplement). There was statistically significant heterogeneity (I2 = 98%) in the first outcome measure. Visual inspection of the forest plot did not identify potential outliers. A sensitivity analysis excluding each study individually revealed that no single study contributed to a substantial portion of the measured I2 statistic. The single largest contribution was from the study by Galanter et al,26 which, when excluded, accounted for only 3% of heterogeneity. In addition, there were only minor changes in the pooled-effect estimate when each study was omitted sequentially. With sequential omission, the observed statistical significance was not lost with the exclusion of any study.
Since the CCDSSs had varying capabilities, a subgroup analysis was performed to evaluate whether the feature to autopopulate orders affected the rate of appropriate ordering of prophylaxis for VTE. This feature linked the support system with the order screen so there was not a gap between the support system and the desired action of ordering prophylaxis for VTE. Quiz Ref IDThe CCDSS subgroup with autopopulation significantly increased the rate of appropriate prophylaxis for VTE as compared with CCDSSs that could only recommend but not autopopulate the order (Table 2) (eFigure 3 in the Supplement). Both intervention groups, however, still had better outcomes than the group that received routine care.
Six of the 11 studies, with 97 172 patients who received the CCDSS intervention and 49 978 controls, reported the effect of CCDSSs on the number of VTE events experienced. Quiz Ref IDThe use of a CCDSS was associated with significantly decreased rates of VTE events among surgical patients compared with controls (risk ratio, 0.78; 95% CI, 0.72-0.85; P < .001) (Figure 2 and Table 2).
This systematic review of 11 studies demonstrated that the use of CCDSSs among surgical patients was associated with a significant increase in the proportion of patients with adequately ordered prophylaxis for VTE, especially if the CCDSS could autopopulate patient orders. Furthermore, the use of CCDSSs was associated with a significant decrease in the risk of developing a VTE.
The overall increased rates of adherence and decreased rates of VTE found among surgical patients is comparable to those in a randomized clinical trial by Kucher et al22 evaluating CCDSSs for all types of patients; the authors also found that the application of CCDSSs decreased the incidence of DVT and PE. In a hospital-wide trial, Maynard et al23 found increased adherence to guidelines of 98%, with a subsequent reduction in VTE events. Both of these articles were eligible for inclusion in our study; however, sufficient data for meta-analysis of surgical patients could not be obtained. A recent narrative review evaluated the different strategies that are used to improve prophylaxis of VTE: 16 articles were included, and it was found that the greatest intervention for sustained improvement of prophylaxis for VTE was computerized tools with an educational component.35 A Cochrane review found that multifaceted and electronic alerts were associated with significantly increased rates of thromboprophylaxis.36 This review provided VTE or DVT rates as an outcome measure but did not show a reduced risk with any intervention, likely owing to the inclusion of studies that were underpowered to assess this measure.
Computerized clinical decision support systems have been used successfully in other areas of medicine in the delivery of preventive care.37-40 In surgical fields, CCDSSs have been proven to increase adherence to the guideline of ordering perioperative antibiotics to 99% and reduce the incidence of surgical site infection to less than 1%.41 In addition, CCDSSs have successfully increased adherence to guidelines for blood transfusions, which reduced the number of units transfused by 24% and saved more than $1.6 million annually in a single hospital.42 These positive results display the powerful utility of CCDSSs to implement current evidence-based medicine as the standard of care.
Although the previously cited reviews of this topic also conclude that there are increased rates of prophylaxis for VTE with CCDSSs, there was no consensus on whether that increased rate of prophylaxis correlated with the clinical end point of decreased rates of VTE.35,36 Our study was undertaken to provide evidence to support the clinical end point for CCDSSs in thromboprophylaxis. Through pooling data and meta-analysis, we have demonstrated that the increased rates of prescribing appropriate prophylaxis for VTE with the implementation of CCDSSs do correlate with decreased rates of VTE events (Table 2 and Figure 2).
The key to the successful integration of information technology into medicine is illustrated by the framework of the 5 “rights”: the right information must be presented to the right audience, in the right format, through the right channels, at the right points in patient workflow.43 The majority of the studies in our meta-analysis incorporated most of these critical components, but some of the rights are not yet clearly defined when it comes to prophylaxis for VTE, which accounts for the heterogeneity between systems. Although all the CCDSSs were embedded into EHRs, they differed in several ways, including to which page they were linked, whether they were mandatory, how they communicated with the user, and their capabilities. Nine of the included studies developed CCDSSs from evidence provided by the most recent Antithrombotic Therapy and Prevention of Thrombosis guidelines1,2,7,11,44 available.25-31,33,34 Two studies developed a CCDSS guideline based on local expert consensus.24,32 The CCDSSs were linked to an admission page, preoperative assessment page, postoperative assessment page, or a different page, which could not be bypassed without providing a reason in all but 2 of the studies.29,33 In addition to the CCDSS user interface, 5 of the studies also incorporated electronic alerts to query the EHR to identify patients who did not have adequate prophylaxis for VTE initiated because their clinical course may have changed since admission.25,26,29,32,33 Four of the CCDSSs had the advanced capability to pull data from the EHR to automatically calculate the patient’s risk level.25,31,33,34 The other CCDSSs did not have this capability but provided a risk checklist to the user that calculated risk once the list was completed. As seen in Table 2, optimal rates of appropriate prophylaxis for VTE were achieved when the CCDSS was capable of autopopulating the order. However, autopopulation was not necessary to reach statistical significance when compared with routine care. Overall, the vital function of the CCDSSs may be providing a mandatory “hard stop” for the clinician to consider risk of VTE in every patient.
Quiz Ref IDAlthough this analysis demonstrated that CCDSS-guided prophylaxis for VTE correlated with decreased VTE events, the included studies did not report sufficient data on the adverse events associated with prescribing additional anticoagulation. A meta-analysis of adverse events could not be performed. Increased prophylaxis for VTE with CCDSSs may have a significant effect on adverse events such as hematoma formation, bleeding events, and development of heparin-induced thrombocytopenia. These potential risks of widespread implementation of CCDSSs must be addressed and analyzed. The risk of adverse events from CCDSS-guided prophylaxis may outweigh its benefits for some physicians. This perception, along with inaccurate risk stratification and potentially outdated protocols, may represent the biggest barriers to adequate prophylaxis for VTE.
This review is not without limitations. The patient population varied between studies owing to inclusion or exclusion of specific surgical services and the nature of the services included. Most patient demographics and risk levels could not be extracted because most studies reported these statistics without separating the medical and surgical cohorts. However, overall patient demographics and relevant risk factors for VTE were controlled for between the CCDSS and non-CCDSS cohorts in each study. In addition, the guidelines for prophylaxis for VTE that were developed for each CCDSS differed. The fact that the guidelines differed could affect physician adherence because if a physician was not familiar or not in agreement with the modifications presented by the CCDSS, he or she would be less likely to prescribe the recommended prophylaxis for VTE. A strong educational component could control some of this behavior, but such a component was not mentioned in all studies. The substantial heterogeneity between studies when the data were pooled could also limit interpretations. In eFigure 2 in the Supplement, the I2 was 98%, and even after influence analysis via repeated analyses with sequential exclusion of each study individually, the minimum I2 value observed was 95%, suggesting that the studies may be heterogeneous at baseline. This finding initiated an investigation into the published literature, where it was discovered that Kahn et al36 encountered similar levels of heterogeneity (92%-98%) in their pooled analysis of patients who were ordered prophylaxis for VTE, which could not be attributed to outlier studies.
The measurable effect on VTE events was also limited in several ways. Individually, most studies were not adequately powered to detect a change in VTE events, but this limitation was corrected by pooling data. Of the 6 studies that reported this outcome measure, there were gaps in study design. For example, the detection of patients who experienced VTE postoperatively was subject to either manual record reviews or by searching diagnosis codes, which could alter the incidence of VTE via missed events or failure to discount VTEs present on initial admission. It is known that the risk of perioperative VTE peaks at week 3 and remains elevated for at least 3 to 6 months after surgery in some patients.45 Since the follow-up time ranged from in-hospital stay to 90 days after the surgery, the incidence of VTE could potentially be underestimated.
Some studies only reported the percentage of patients who experienced VTE in each study arm and did not quantify which patients were involved.28,30,31 It would be beneficial for future studies to report whether these patients received adequate prophylaxis, inadequate prophylaxis, or no prophylaxis for VTE. This information would allow for the proper interpretation of potentially preventable VTE (ie, the patient was not receiving adequate prophylaxis before the VTE event).27,46
Of the 11 included studies in this review, 7 CCDSSs were developed in-house, 1 was purchased from Discern Health, and 3 were not mentioned in the text. These CCDSSs were integrated into several different EHRs, including AllScripts, Sunrise, and Cerner. Notably, none of the CCDSSs was integrated into Epic, which is one of the most common EHRs in use today. It is unknown whether the successful CCDSSs reported in this review are scalable to the Epic system. Future studies of updated CCDSSs in the more up-to-date EHRs are warranted.
We should not ignore the strength of computer science in medicine. Although it is known that CCDSSs can improve physician adherence to accepted clinical guidelines, and we have revealed the strength of the autopopulate feature, the optimal CCDSS design is not known. We suggest that future CCDSS programs have the following attributes to achieve accurate patient risk stratification and high rates of adequate prophylaxis for VTE. An appropriate risk stratification for VTE should be suggested based on documented patient variables available in the EHR. The assigned risk level should be accompanied with an evidence-based suggestion for the appropriate prophylaxis, with a hyperlink to supporting literature. More important, the user should be able to modify the assigned risk level (eg, if the patient discloses an undocumented risk factor) and order for prophylaxis. Once an order is selected, it should autopopulate to the physician order screen. Finally, the CCDSS should be able to query the EHR system and provide an alert for any patient who is without adequate prophylaxis for VTE. The successful implementation of a CCDSS and physician acceptance depend on further trials that lend support to the efficacy of CCDSSs, their cost utility, their user acceptability, and, most important, their ability to change patient outcomes.
The implementation of a CCDSS into an EHR, especially with an order autopopulation feature, can significantly improve rates of ordering adequate prophylaxis for VTE and is associated with fewer symptomatic DVT and PE events postoperatively.
Accepted for Publication: January 4, 2017.
Corresponding Author: Zachary M. Borab, BA, Wyss Department of Plastic and Reconstructive Surgery, New York University School of Medicine, 450 E 29th St, Alexandria West Tower, 6th Floor, Room 660D, New York, NY 10016 (firstname.lastname@example.org).
Published Online: March 15, 2017. doi:10.1001/jamasurg.2017.0131
Author Contributions: Mr Borab had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Borab, Lanni, Tecce, Fischer.
Acquisition, analysis, or interpretation of data: Borab, Lanni, Tecce, Pannucci.
Drafting of the manuscript: Borab, Tecce, Pannucci.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Borab, Lanni, Tecce.
Administrative, technical, or material support: Lanni, Pannucci, Fischer.
Study supervision: Tecce, Pannucci, Fischer.
Conflict of Interest Disclosures: None reported.