Effectiveness of a Hospital-Based Computerized Decision Support System on Clinician Recommendations and Patient Outcomes

This randomized clinical trial assesses the effectiveness of a computerized decision support system that preappraises evidence and provides health professionals with actionable, patient-specific recommendations at the point of care.


Background and rationale 80
Despite the proliferation of clinical guidelines and continued efforts by local and national health care 81 systems to optimize decision-making on patient diagnosis, treatment, and management, the quality of 82 medical care is variable and often suboptimal [1]. There remains an apparent discrepancy between the 83 growing availability of scientific evidence and the application of this evidence into medical care [2,3]. 84 Non-adherence to evidence-based guidelines, medical errors, and omissions in everyday practice may 85 occur because of time pressure, inexperience, reliance on memory, multitasking, and failures in 86 healthcare team coordination. 87 Computerized decision support systems (CDSSs) are information technology-based software that 88 provide health professionals with actionable, patient-specific recommendations or guidelines for 89 clinical care at the point-of-care; these messages are intelligently filtered and presented at appropriate 90 times during the decision-making process in order to enhance patients' health [4,5]. The opportunity to 91 improve patient care by increasing clinicians' accessibility to medical knowledge at the site of practice 92 represents one of the main incentives for investing in the development and evaluation of these 93 sophisticated information systems. 94 In particular, studies focusing on the effectiveness of "new generation" CDSSs demonstrate their 95 potential to assist with problems raised in clinical practice, decrease the rate of medication errors, 96 increase clinicians' adherence to guidelineor protocol-based care, and, ultimately, improve the overall 97 efficiency and quality of health care delivery systems [6][7][8][9][10][11][12][13][14][15][16][17][18][19]. These innovative systems can be 98 integrated into hospital electronic health records (EHRs) and feature authoritative point-of-care 99 information services and evidence-based knowledge [20]. 100 -6 -This has led to some early work: a systematic review assessing the effectiveness of such "new 101 generation" CDSSs demonstrated encouraging results [21]. Although this review did not show CDSSs 102 to affect mortality, they were shown to moderately improve morbidity outcomes. Differences were 103 further observed for costs and health services utilization, but these were often inconsistent in the 104 direction of effect and small in magnitude. The conclusion of a landmark paper, published nearly 15 105 years ago, still reflects the current scenario: "Although the promise of clinical decision support system-106 facilitated evidence-based medicine is strong, substantial work remains to be done to realize the 107 potential benefits" [22]. 108

Trial design 125
The CODES (COmputerized DEcision Support) trial will implement a pragmatic, parallel group, and 126 randomized controlled design with 1:1 allocation ratio. The flow diagram of the study can be found in 127 The study will involve the medical staff of the internal medicine departments of the Vimercate Hospital 131 from Azienda Ospedaliera di Desio e Vimercate (AODV) a multi-site hospital system located in the 132 harms and benefits of about 18,000 drug interactions and adverse events [33]. 158 MediDSS may be used as a stand-alone application, or may integrate structured patient data from EHR 159 to generate patient-specific reminders, therapeutic suggestions, and diagnosis-specific links to full-text 160 guidelines. Reminders are automatically generated and displayed on the monitors of clinicians when 161 they open a patient's EHR, enter a new diagnosis, prescribe a drug, or when new laboratory test results 162 are available. Reminders were formed using international evidence-based guidelines and subsequently 163 approved by an international panel of experts. Our study will use international reminders (n=262) that 164 cover a large number of health conditions across specialties and are derived from the EBMeDS and 165 SFINX database. In addition, 17 local reminders have been carefully selected by a team of doctors at 166 Vimercate hospital, along with members of the trial team. Table 1 reports some examples of the 167 reminders. Figures 2 and 3 show a snapshot of the activation button and of the actual reminders. 168 169 MediDSS reminders will be shown on the EHR of patients only within the intervention group. During 170 the care of control group patients, the generated reminders will not be shown to the physicians, such 171 that the control is usual clinical practice without the use of the MediDSS service. However, physicians 172 -9 -in both groups will have access to the best evidence for usual care at all times during the trial through 173 the active searching of full-text EBM guidelines on Internet. All participating physicians will be 174 informed on the availability and use of the MediDSS system. 175

176
Stepped wedge implementation 177 The intervention is a new technology: its integration in the current hospital system requires the 178 configuration and customization of the software. To allow security controls and successful 179 implementation, the CDSS will be sequentially rolled out to participants over a number of time periods. 180 We anticipate that the number of periods will be limited (i.e. two or three periods). Over an initial 181 period, all participants will receive the intervention. The order in which participants will receive the 182 intervention is not determined at random, but will be determined by selecting physicians prone to 183 provide constructive feedback to the implementation team. The RCT adopts a stepped wedge 184 implementation of the intervention, but not a stepped wedge design [34]. Sequential roll-out of the 185 intervention will not be considered a pilot phase of the trial, but a part of the whole RCT. 186 187

Selection and development of priority reminders 188
In order to encourage the participation of the hospital staff within the study, we invited hospital 189 representatives to assess the priority needs of the hospital wards and develop a set of reminders to 190 address them. One topic of particular interest to the hospitals involved venous thromboembolism 191 (VTE) prevention. The rationale for the prioritization of this condition is provided below: 192 (i) Despite evidence supporting the benefits of VTE prophylaxis based on the risk stratification process 193 [35] as well as the availability of local hospital guidelines, the prophylactic drugs were inconsistently 194 administered among patients. activities. Healthcare service studies on CDSSs, however, consistently suggest that the mere provision 235 of such technology does not guarantee its uptake. In fact, even if a CDSS is readily available within a 236 hospital, clinicians often fail to follow its recommendations, ignoring in some cases up to 96% of its 237 alerts [55]. Given this context, our RCT is informed by qualitative interviews aimed to detect the 238 barriers and facilitators to MediDSS uptake as perceived by diverse health professionals involved in 239 patient care (e.g., physicians and nurses). The interviews are a part of a larger cross-sectional study, 240 which involves three Italian hospitals [56]. The interviews will explore variables that may hinder the 241 use of a CDSS in everyday clinical practice, including technical (e.g., poor usability or knowledge of 242 system), individual (e.g., negative perception of CDSS or EBM, lack of motivation), group or 243 -12 -organizational (e.g., structural or administrative constraints), and cultural factors (e.g., adverse social 244 norms). 245 When feasible, the trial will be tailored to address the specific needs emerging from the qualitative 246 assessment. We will collect feedbacks about usability, possible errors, or inaccuracies of the 247 information and recommendations provided. We will offer the best possible solutions to clinicians and 248 hospital staff to overcome these problems. We will further organize and facilitate group discussions 249 among participants to address negative perceptions or misleading beliefs about CDSSs. The qualitative 250 study seeks to support the use of CDSS by participants, thus increasing the integrity of the intervention 251 and associated compliance. Anonymous patient identification (ID) numbers in the EHR system will be the unit of randomization. 276 An individual external to the study group will generate the anonymous IDs using a formula based on 277 patients' unique fiscal code numbers. 278 We will randomly assign patients to either the control or experimental group with a 1:1 allocation. We 279 will follow a computer generated randomization schedule stratified by gender and age (0-30, 31-60, 280 61-80, >80 years) using permuted blocks of random sizes [59]. Patients will be randomized 281 immediately after the first launch of their EHR (entry of demographic data by physicians at hospital 282 admission), and the allocation will be maintained through successive admissions. 283 Patients and study investigators (i.e., researchers, statisticians, information technology specialists, and 284 hospital representatives) will be blinded to the allocation of participants. We will maintain the blinding 285 up to the dataset disclosure. On the other hand, blinding of physicians is not feasible due to the nature 286 of the intervention: the physician will know that a patient has been allocated to the intervention group if 287 an automatic, patient-specific reminder is displayed on the screen.

Data collection 292
The data collection for this study will follow the standard data collection procedures of the AODV. We 293 will collect demographic (i.e., gender, age) and administrative (i.e., anonymous patient ID, admission 294 and discharge dates, diagnoses) data from the EHR archive on a daily basis. Information on reminders, 295 including all scripts that have been activated in a patient's record, will also be collected daily, but 296 during the night, so as not to disturb or slow down the use of the patient EHR. 297 298

Statistical methods 299
For the primary outcome (i.e., resolution rates), the reminder will serve as the unit of analysis, and the 300 patient the clustering factor. The patient will be the unit of analysis for the secondary outcomes (i.e., 301 length of stay and in-hospital mortality). All analyses will follow the intention-to-treat principle: 302 patients will be analyzed in the group to which they have been randomized. Descriptive statistics will 303 be presented as means ± standard deviations (SD), medians and interquartile ranges (IQR), or 304 percentages when appropriate. We will compare continuous variables using the Student's t-test when 305 normally distributed, and the non-parametric two-sample Wilcoxon rank-sum (Mann-Whitney) test 306 when they are not normally distributed. We will compare categorical variables using the chi-squared 307 test or the Fisher's exact test, as appropriate. To model the resolution rates of the reminders, we will 308 run a random effects logistic regression analysis, accounting for clustering by patient [60]. 309 For hypothesis testing, we will consider a probability level of less than 0.05 as statistically significant. 310 All statistical tests will be two-sided. We will use the Stata software to perform all statistical analyses 311 (Stata Corp., College Station, TX, USA). 312 313

Data monitoring 314
Data monitoring will inform the CODES trial conduct, identifying the potential need for adjustments: 315 -15 -(i) Sample size recalculation: Because the sample size calculation utilizes several assumptions, we will 316 analyze the first batch of data collected and adjust the estimated sample size, if necessary, at the end of 317 the sequential roll-out of the intervention. The 24-month recruitment period may also be adjusted, 318 accordingly. 319 (ii) Interim analysis: We will perform an interim-analysis on the primary endpoint after 50% of the 320 patients have been randomized, after 50% of the expected events have occurred, or after 12 months of 321 the study's initiation (the assumed half-life of the trial), whichever occurs first. An independent 322 statistician that is blind to the patient allocation will perform the analysis. This analysis will inform 323 whether the intervention has been proven for efficacy (beyond reasonable doubt). We will subsequently 324 decide whether (or not) it is necessary to modify the study or prematurely terminate it, if necessary. 325 (iii) End of trial: The end of trial will occur thirty days after the randomization of the last EHR. We 326 will submit an End of Trial notification and final report to the competent Ethical Committee, the 327 AODV, and to the Sponsor. 328 329

Harms 330
We do not anticipate any harms (or other unintended effects) to study participants. Intervention and 331 control groups will differ in the presence (intervention) or absence (control) of automatic reminders 332 displayed on physicians' monitors. Patients assigned to the control group will receive usual care 333 without the reminders. Nevertheless, we will consult an External Advisory Board in the event that the 334 discontinuation of the study becomes an option due to unforeseeable reasons. 335 336

Ethical and Regulatory considerations 337
This study is conducted in accordance with the principles of the Declaration of Helsinki (October 2013) 338 [61]. As the CODES trial has a cluster design (several reminders, the unit of analysis, may derive from 339 -16 -the same EHR, the unit of randomisation), we followed the Ottawa statement to identify research 340 participants and apply ethical and regulatory protections [62,63]. The intervention (electronic CDSS 341 reminders) does not directly target patients, but physicians who can be considered as the participants of 342 the study. The risks associated with the participation of physicians in the CODES trial are negligible. 343 Physicians will be fully informed about the involvement of the AODV in the CODES trial and trained 344 to use the intervention. Requiring the signed consent of each physician is not feasible and will impact 345 on the validity and generalizability of study results. Some have argued that healthcare professionals 346 have an obligation to participate in health system or knowledge translation research [64,65]. We 347 consider that the waiver of signed consents will not adversely affect the rights or welfare of the 348 research participants. 349 350

Protocol amendments 351
Any changes to the research protocol that may impact the study conduct (e.g., changes in study design, 352 eligibility criteria, study outcomes, sample size, study procedures, or significant administrative aspects) 353 will require a formal amendment of the protocol. We will communicate any such amendments to the 354 trial registry (ClinicalTrials.gov), and notify the health authorities in accordance with the Italian 355 regulations. We will further seek the approval of the Ethical Committee for any amendments to the 356 protocol. 357 358

Confidentiality 359
The trial staff will ensure the maintenance of participants' anonymity. The participants will be 360 identified only by their initials and anonymous patient ID number. Depersonalised data will be 361 extracted from the EHR. All documents will be stored securely and accessible only by the trial 362 investigators and authorised personnel. 363 -17 -Clinical data collected during the study will only be accessible to the staff at AODV, thus complying 364 with the current medical practice of the Hospitals. The trial investigators external to the Hospitals 365 (statistician, data manager, information technology personnel, etc.) will not have access to any 366 information at the patients' level. 367 The CODES trial will comply with the Italian Data Protection Act, which requires data to be 368 anonymised as soon as it is practical to do so. 369 370

Dissemination policy 371
The trial results will be posted on ClinicalTrial.gov as well as published in an open access medical 372 journal. 373 We will further disseminate the study results to the health professionals of AODV who are involved in 374 the study. The CODES trial has several strengths. First, the randomized controlled study design is recognized as 380 the "gold standard" for testing intervention-outcome hypotheses, allowing us to maximize the 381 likelihood that the differences observed between groups are due to the intervention rather than potential 382 confounding factors. Second, the pragmatic design of the study under conditions that mimics the actual 383 use of CDSSs in practice increases the generalizability of the results as well as allows a more accurate 384 estimation of the intervention's true effectiveness. Third, the choice of an intention-to-treat analysis 385 helps to ensure the pragmatic design of the study; in other words, although not all physicians may 386 -18 -adhere to the reminders within the study, we anticipate that the lack of compliance with evidence-based 387 recommendations occurs in everyday practice. 388 We must note the methodological limitation that physicians will not be blinded to the treatment 389 allocation. When a patient-specific reminder is automatically displayed on the monitors, the physician 390 will know that the particular patient belongs to the intervention group. We are aware that the unit of 391 allocation (i.e. patient) and the lack of physician blinding can lead to possible group contamination as     , which uses ten common risk factors to identify patients at high risk for VTE. Each risk factor is individually weighted according to a point-based scale.
-Active cancer (defined as presence of methastases or recent chemotherapy), known trombophilic condition, and reduced patient mobility are each assigned a score of 3 points.
-Recent major surgery is assigned a score of 2 points.
-Advanced age (greater than 70 years), obesity (BMI greater than 30), bed rest, and hormone replacement therapy or oral contraceptives are each assigned a score of 1 point.
Patients are identified as high-risk for VTE if they accumulate a sum of 4 or more points. When the risk level is low, no medication is recommended; when the risk level is high, a prophylactic strategy using high dosage low-molecular-weight heparin is recommended.
II. The second part of the algorithm involved the Exclusion Criteria for the use of VTE Prophylaxis.
-Home Anticoagulant Therapy -Contraindications to Pharmacologic Prophylaxis -Active Bleeding 661