Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E. A Clinical Decision Support System for Prevention of Venous ThromboembolismEffect on Physician Behavior. JAMA. 2000;283(21):2816-2821. doi:10.1001/jama.283.21.2816
Author Affiliations: Department of Public Health, Faculté de Médecine Broussais Hôtel Dieu (Dr Durieux), Department of Orthopedics, Lariboisière Hospital, Assistance Publique–Hôpitaux de Paris (Dr Nizard), Epidemiology Unit, Bichat Hospital, Assistance Publique–Hôpitaux de Paris (Dr Ravaud), Paris, France; and the Department of Biostatistics and Medical Informatics, Henri Mondor Hospital, Assistance Publique–Hôpitaux de Paris, Créteil, France (Drs Mounier and Lepage).
Context Computer-based clinical decision support systems (CDSSs) have been promoted
for their potential to improve quality of health care. However, given the
limited range of clinical settings in which they have been tested, such systems
must be evaluated rigorously before widespread introduction into clinical
Objective To determine whether presentation of venous thromboembolism prophylaxis
guidelines using a CDSS increases the proportion of appropriate clinical practice
Design Time-series study conducted between December 1997 and July 1999.
Setting Orthopedic surgery department of a teaching hospital in Paris, France.
Participants A total of 1971 patients who underwent orthopedic surgery.
Intervention A CDSS designed to provide immediate information pertaining to venous
thromboembolism prevention among surgical patients was integrated into daily
medical practice during three 10-week intervention periods, alternated with
four 10-week control periods, with a 4-week washout between each period.
Main Outcome Measure Proportion of appropriate prescriptions ordered for anticoagulation,
according to preestablished clinical guidelines, during intervention vs control
Results Physicians complied with guidelines in 82.8% (95% confidence interval
[CI], 77.6%-87.1%) of cases during control periods and in 94.9% (95% CI, 92.5%-96.6%)
of cases during intervention periods. During each intervention period, the
appropriateness of prescription increased significantly (P<.001). Each time the CDSS was removed, physician practice reverted
to that observed before initiation of the intervention. The relative risk
of inappropriate practice decisions during control periods vs intervention
periods was 3.8 (95% CI, 2.7-5.4).
Conclusions In our study, implementation of clinical guidelines for venous thromboembolism
prophylaxis through a CDSS used routinely in an orthopedic surgery department
and integrated into the hospital information system changed physician behavior
and improved compliance with guidelines.
Computer-based clinical decision support systems (CDSSs) are defined
as "any software designed to directly aid in clinical decision making in which
characteristics of individual patients are matched to a computerized knowledge
base for the purpose of generating patient-specific assessments or recommendations
that are then presented to clinicians for consideration."1
Clinical decision support systems have been promoted for their potential to
improve the quality of health care by supporting clinical decision making.
In particular, it has been suggested that physicians have difficulties processing
complex information2 and will improve their
prescription practices in response to electronically delivered recommendations.3 However, given their rapid rate of development and
the limited range of clinical settings in which they have been tested to date,
it has been stressed that CDSSs should be rigorously evaluated before widespread
introduction into clinical practice.1,4
In clinical hospital practice, venous thromboembolism remains a serious
problem and pulmonary embolism is a major cause of death.5
Fatal pulmonary embolism may occur in up to 1% of general surgery patients
and 3% of orthopedic surgical patients who do not receive prophylaxis.6 The most efficient way to prevent both fatal and nonfatal
venous thromboembolism is to use routine prophylaxis for moderate- to high-risk
patients. Despite the publication of several clinical guidelines for venous
thomboembolism prophylaxis in both Europe and North America6- 9
as well as studies suggesting that prophylaxis remains underused, few studies
aimed at improving prophylaxis practices have been performed.10,11
One probable reason is that optimal decisions about the use of anticoagulants
in prevention of venous thromboembolism require access to a large amount of
complex information to evaluate the degree of risk of hospitalized patients.
We have developed a CDSS to implement clinical guidelines on venous thromboembolism
prophylaxis in an orthopedic surgery department. In this study, we evaluated
the effect of this system on physician behavior. We aimed to determine whether
real-time presentation of venous thromboembolism prophylaxis guidelines through
a CDSS increases the proportion of appropriate anticoagulant prescriptions
ordered and whether this behavior change was extinguished after discontinuing
use of the CDSS.
The study was conducted in the orthopedic surgery department of Lariboisière
Hospital, a 1000-acute-bed teaching hospital of the Assistance Publique–Hôpitaux
de Paris group (the Paris, France, metropolitan area public hospital network).
About 2400 patients are hospitalized annually in this department. All surgeons
(7 full-time, 7 part-time) working in the orthopedic surgery department were
involved in the study. All orthopedic patients who underwent surgery in the
department (from December 1997 to July 1999) were included in the study.
The Assistance Publique–Hôpitaux de Paris Institutional
Review Board determined that, according to French policy, the study was exempt
from review requirement and could be conducted without informed consent from
Clinical guidelines for venous thromboembolism prophylaxis were developed
at Assistance Publique–Hôpitaux de Paris.12,13
These local guidelines for general surgery, urologic surgery, gynecologic
surgery, and orthopedic surgery were created by local hospital experts. In
this study, assessment of guideline use with and without the CDSS was restricted
to orthopedic surgery patients.
The risk of thromboembolism in a hospital patient depends on the planned
surgical procedure (or the reason for admission) and on preexisting patient-related
variables. Each of these factors has been classified in existing guidelines
as low, moderate, or high risk.6- 9
There is no published classification of risk that combines patient risk factors
and surgical risk factors to obtain an overall risk of venous thromboembolism.
However, taking into account both types of risk for a given patient is crucial
in clinical practice. Thus, the local hospital expert group proposed a classification
system in which, for each patient, the presence of patient risk factors and
surgical risk factors are combined to classify patients as having a low, moderate,
or high risk for venous thromboembolism. A prophylactic strategy is recommended
for each level of risk. When the risk level is low, no medication is recommended;
when the risk level is moderate, prescription of a low dosage of low-molecular-weight
heparin is recommended; and when the risk level is high, prescription of a
high dosage of low-molecular-weight heparin is recommended (Table 1).
Our CDSS is an online computer application designed as a tool to provide
clinicians with relevant, real-time information pertaining to venous thromboembolism
prevention among surgical patients. This application is linked to the diagnosis
related group–based information system that is implemented in all French
hospitals. Patients' administrative and clinical data are collected by direct
entry in admitting, operating room, and medical care units. These data are
stored in a coded and integrated clinical patient database and are available
for computer-assisted decision making.
The CDSS can be accessed through computer terminals available just outside
each operating room. Following each surgical procedure, after entering an
identification code, the physician enters data related to the clinical situation
of the patient (age, sex, disease, surgical procedure, and preexisting patient
and surgical risk factors of venous thromboembolism). The physician orders
all treatments necessary for patient follow-up (eg, antibiotic therapy, pain
management, immobilization), including venous thromboembolism prophylaxis,
through the computer system. The computer system critiques the orders using
data contained in the patient's database and guideline-based criteria stored
in the system's knowledge base. If the computer detects a discrepancy between
the prescription and the corresponding information in the database, the physician
is immediately notified via a message on the computer screen suggesting the
appropriate prescription and explaining the reasons. The physician can choose
to maintain or change his/her order. At the end of the process, the patient
follow-up and prescription information, including venous thromboembolism prophylaxis,
is printed out and included in the patient's file.
The study had an alternating time-series design, with three 10-week
intervention periods, four 10-week control periods, and a 4-week washout between
During intervention periods, physicians received a message from the
CDSS if their prescriptions were not appropriate according to the guidelines.
During control periods, physicians ordered all treatment related to thromboembolism
prophylaxis through the computer system but received no critiquing messages
from the CDSS.
To evaluate the effects of the CDSS, the proportion of venous prophylaxis
prescriptions that was appropriate according to clinical guidelines was considered
to be the main end point. This proportion was estimated based on the final
prescription order for each patient compared with treatment designated by
algorithms established prior to the study and derived from the guidelines.
Each prescription could be classified as appropriate or not appropriate. A
prescription was classified as not appropriate when no medication was ordered
by the physician when the CDSS recommended prescription of low-molecular-weight
heparin (type 1 error), when the wrong dosage of low-molecular-weight heparin
was prescribed (type 2 error), or when a prescription of low-molecular-weight
heparin was made when the CDSS proposed no medication (type 3 error).
The percentage of inappropriate initial prescriptions that were changed
after advice was given by the CDSS during intervention periods was also calculated
according to each level of risk of venous thromboembolism. We also recorded
the number of pulmonary embolisms and deep vein thromboses diagnosed in the
orthopedic department during the study period.
Analysis included all eligible patients. Comparisons of clinical characteristics
of patients during intervention and control periods were tested using the χ2 test and the t test where appropriate. The
nominal significance level for the end points was .05 (2-sided formulation).
To evaluate the effect of the decision-making application on appropriateness
of prescription, we first chose the patient as the unit of analysis because
the patient experiences the care and generates the original data. Then we
took into account as a unit of analysis the physician (eg, the sequence of
prescriptions of 1 physician). We accounted for the potential nonindependence
of patient observations of a physician resulting from clustering by using
a logistic regression model for binary data with random effect. We assumed
that only the intercept, not the decision-making application effect, varies
among physicians. The decision-making application effect was tested using
the logit of the probability of appropriateness as response, while the period
(control vs intervention) was considered the explicative covariate. The intercept
was regarded as a random effect and the period as a fixed effect. Model parameters
were estimated using the iteratively reweighted restricted likelihood method
and fixed effects were tested with the Fisher exact test (SAS GLIMIX macro).14 Mean probability of appropriateness of prescription
according to study period was then generated by taking the exponential transformation
of the logit.
All statistical analyses were performed using SAS version 6.12 computer
software (SAS Institute Inc, Cary, NC).
A total of 1971 patients were included in the study; 1112 during control
periods and 859 during intervention periods. The computer system was used
in 100% of patients who underwent surgery during the study period. Patient
characteristics were comparable in the intervention and control periods, except
for patient risk factors. There were more patients with no preexisting risk
factors during the control periods than in the intervention periods (36.9%
vs 31.5%; P=.04). However, distribution of venous
thromboembolism risks were comparable in the 2 groups (Table 2). A total of 696 patients (35.3%) were at low risk of venous
thromboembolism, 98 patients (5.0%) were at moderate risk, and 1177 patients
(59.7%) were at high risk. During the study period, the mean number of patients
per surgeon was 141 (range, 4-370). Five surgeons operated on fewer than 40
patients each and 8 surgeons operated on more than 100 patients.
Physicians complied with guidelines in 82.8% (95% confidence interval
[CI], 77.6%-87.1%) of cases during control periods and in 94.9% (95% CI, 92.5%-96.6%)
of cases during intervention periods. Logistic regression analysis, performed
using the physician as the unit of analysis, demonstrated a significant physician
effect (P<.001) and a significant difference between
the 2 study periods on appropriateness of prescription (P<.001). The relative risk of inappropriateness was 3.8 (95% CI,
2.7-5.4) for control periods vs intervention periods, equivalent to a 73%
reduction in risk of inappropriate prescription.
Results according to period are shown in Figure 1. During each intervention period, the proportion of appropriate
prescriptions ordered increased significantly. Each time the CDSS was removed,
physician compliance with guidelines reverted to that observed before initiation
of the intervention.
Results according to venous thromboembolism risk are shown in Table 3. A total of 191 prescriptions (17.2%)
were judged inappropriate by the CDSS during control periods and 113 prescriptions
(13.2%) were judged inappropriate during intervention periods. Among these
113 prescriptions, 69 (61.1%) were modified by the physician according to
the recommendation of the CDSS and 44 (38.9%) remained unchanged. Overall,
the effect of the CDSS was greatest for patients at moderate risk of venous
thromboembolism. In this group, 18(81.8%) of 22 inappropriate prescriptions
were changed after advice given by the CDSS. The CDSS appeared to have less
effect for patients at high risk of venous thromboembolism. In this group,
24 (51.1%) of 47 inappropriate prescriptions were changed.
Table 4 presents the number
of errors by type for the 191 inappropriate prescriptions ordered during the
control periods and the 44 inappropriate prescriptions that were not changed
during the intervention periods. The system did not allow for analysis of
the 69 initial prescriptions that were changed according to the recommendation.
During intervention periods, the error rate decreased by 86% for type 1 errors,
by 59% for type 2 errors, and by 66% for type 3 errors.
The CDSS improved the clinical practice of all physicians except 1 whose
proportion of appropriate prescriptions was close to 100% during control periods.
The greatest improvement was observed among the 5 surgeons who operated on
fewer than 40 patients.
One pulmonary embolism and 2 deep vein thromboses were diagnosed during
control periods. No pulmonary embolisms and 2 deep vein thromboses were diagnosed
during intervention periods.
Our study showed that implementation of clinical guidelines for venous
thromboembolism prophylaxis through a CDSS in an orthopedic surgery department
significantly changed physician behavior and improved compliance with guidelines.
The improvement was greater for patients at moderate risk of venous thromboembolism
than for patients at high risk of venous thromboembolism where practices were
already appropriate for more than 90% of patients before any intervention.
In patients with elective hip surgery and hip fractures, drug regimens including
subcutaneous heparin and low-molecular-weight heparin have been proven effective
in prevention of deep vein thrombosis6- 8
and this strategy is well accepted by French surgeons. However, the moderate
risk constitutes a gray zone of uncertainties and is more difficult to define.
Physicians may also have difficulty remembering the guidelines for this category
involving relatively few patients (5% of the total). This explains the dramatic
effect of the CDSS on physician behavior concerning this subgroup of patients
during intervention periods. For the same group, the percentage of appropriate
initial prescriptions (before advice was given by the CDSS) was much more
important during intervention periods (45%) than during control periods (5.7%).
Due to a Hawthorne effect, the physicians involved in the study were probably
more watchful when the CDSS was in use than when it was not in use. This was
an indirect effect of the CDSS.
The CDSS reduced all types of errors but its input seemed to be particularly
important for type 1 errors (failure to order a medication when prophylaxis
Our study contributes several important considerations to the understanding
of the potential role of CDSSs in clinical guideline implementation. First,
this study confirms that use of a CDSS at the time of prescription constitutes
an effective guideline implementation strategy.15- 18
A significant effect on physician behavior was observed despite a high baseline
compliance to guidelines (84.1%) before intervention. In 2 recent studies
performed in surgical and medical-surgical patients, 86% and 85%, respectively,
received venous thromboembolism prophylaxis before any intervention.10,19
The CDSS was able to maintain a sustained effect of guidelines for a
relatively long period. Failure to do so constitutes a major weakness of most
guideline implementation strategies, including paper reminders.20,21
The guidelines can also be easily updated on the CDSS, which facilitates the
implementation over time of up-to-date guidelines.15,16
The CDSS was integrated into the daily practice of physicians. Thus,
all consecutive patients who underwent surgery during the study period were
included in the study. Since the computer system was used as a data collection
tool, it was easy to evaluate the effect of the system.
Second, when we designed the CDSS, we chose to establish a critiquing
system rather than a reminding system. Such critiquing systems, which advise
clinicians about what should be done after a prescription contrary to guidelines
has been ordered, have been commonly applied.22
A simple reminder system that notifies clinicians before prescription of tasks
that should be done probably can be disregarded more easily by the clinician.
A critiquing system can also be used on a routine basis to calculate physician
deviation rates before intervention, thus facilitating efforts toward continuous
Third, some investigators have considered that reminding or alerting
clinicians about what constitutes appropriate practice is a continuing medical
education strategy.24 The rate of reversion
of compliance to guidelines to baseline values during each control period,
even after 15 months, showed that a CDSS cannot be considered an educational
tool or that education alone is unable to sustain substantial changes in physician
practice as has been suggested previously.25,26
Our study had several limitations. The clinical guidelines, particularly
the combination of patient- and surgery-related risk factors used to generate
venous thromboembolism risk, were developed locally and may not be acceptable
to other groups of physicians. The CDSS was implemented in 1 department of
1 hospital and, therefore, the applicability of our results to other settings
is unknown. Another limitation is that we evaluated the effect of implementing
a CDSS on process, not on patient outcomes. The number of pulmonary embolisms
and deep vein thromboses diagnosed among patients during their hospital stay
is insufficient to evaluate patient outcomes since a thromboembolism event
can occur after discharge. However, the aim of the CDSS was to increase the
appropriateness of prophylaxis, not to demonstrate a relationship between
prophylaxis and thromboembolism. In addition, there is no noninvasive, accurate,
and inexpensive diagnostic test to identify patients with deep vein thrombosis.27 The difficulties in interpreting outcomes are widely
recognized.28 Numerous authors now consider
it better to evaluate process rather than outcomes when assessing quality
of care.29- 31
Outcomes have multiple determinants and it is impossible to know what proportion
of a given health outcome is determined by quality factors (ie, processes
and structure of care) and what proportion is due to patient-related risk
factors.32 Interpretation of health outcomes
is hampered by the problem of case-mix.28 Statistical
analyses require an adequate number of outcomes for the results to be meaningful.29,32 Conversely, the use of process measures
can identify specific shortcomings (eg, proportion of inappropriate prescriptions)
and point toward what needs to be changed.28
Clinical decision support systems have been successfully implemented
for preventive care, drug dosing, and management of diseases.1
Our study shows that implementation of clinical guidelines for venous thromboembolism
prophylaxis through a CDSS used routinely in an orthopedic surgery ward and
integrated into a computerized hospital information system significantly changed
physician behavior and improved compliance with guidelines. This system, integrated
in the daily practice of physicians, appeared to constitute a way to obtain
a sustained effect of clinical guidelines. Given the limited range of clinical
settings and health systems in which CDSSs have been tested, it is important
to evaluate such systems on physician behavior.