[Skip to Navigation]
Sign In
Visual Abstract. Effect of Clinical Decision Support With Audit and Feedback on Prevention of Acute Kidney Injury
Effect of Clinical Decision Support With Audit and Feedback on Prevention of Acute Kidney Injury
Figure 1.  Flow of Patients, Eligible Procedures, and Physicians in the Contrast RISK (Reducing Injury Sustained by Kidneys) Trial
Flow of Patients, Eligible Procedures, and Physicians in the Contrast RISK (Reducing Injury Sustained by Kidneys) Trial

Invasive cardiologists practicing at 3 cardiac catheterization laboratories in Alberta, Canada, were randomly assigned to 1 of 8 start dates (periods) between January 1, 2018, and September 1, 2019, and were included during the intervention period and the control (preintervention) period. Three physicians retired before receiving the intervention.

Figure 2.  Subgroup Analyses for the Effect of the Intervention on the Incidence of Acute Kidney Injury
Subgroup Analyses for the Effect of the Intervention on the Incidence of Acute Kidney Injury

Acute kidney injury was defined according to serum creatinine–based criteria from the Kidney Disease: Improving Global Outcomes acute kidney injury guidelines.5 The last serum creatinine measurement collected before the procedure was used to define the baseline value. Follow-up creatinine testing was performed between 48 hours and 96 hours after the procedures. The predicted risk of acute kidney injury was calculated using the National Cardiovascular Data Registry multivariable acute kidney injury risk prediction model.18 OR indicates odds ratio.

Table 1.  Baseline Characteristics
Baseline Characteristics
Table 2.  Effect of the Intervention on the Primary and Secondary Outcomes
Effect of the Intervention on the Primary and Secondary Outcomes
1.
Mehran  R, Dangas  GD, Weisbord  SD.  Contrast-associated acute kidney injury.   N Engl J Med. 2019;380(22):2146-2155. doi:10.1056/NEJMra1805256 PubMedGoogle ScholarCrossref
2.
Azzalini  L, Candilio  L, McCullough  PA, Colombo  A.  Current risk of contrast-induced acute kidney injury after coronary angiography and intervention: a reappraisal of the literature.   Can J Cardiol. 2017;33(10):1225-1228. doi:10.1016/j.cjca.2017.07.482 PubMedGoogle ScholarCrossref
3.
Amin  AP, McNeely  C, Spertus  JA,  et al.  Incremental cost of acute kidney injury after percutaneous coronary intervention in the United States.   Am J Cardiol. 2020;125(1):29-33. doi:10.1016/j.amjcard.2019.09.042 PubMedGoogle ScholarCrossref
4.
James  MT, Ghali  WA, Knudtson  ML,  et al; Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) Investigators.  Associations between acute kidney injury and cardiovascular and renal outcomes after coronary angiography.   Circulation. 2011;123(4):409-416. doi:10.1161/CIRCULATIONAHA.110.970160 PubMedGoogle ScholarCrossref
5.
Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group.  KDIGO clinical practice guideline for acute kidney injury.   Kidney Int. 2012;2:1-138.Google ScholarCrossref
6.
Valdenor  C, McCullough  PA, Paculdo  D,  et al.  Measuring the variation in the prevention and treatment of CI-AKI among interventional cardiologists.   Curr Probl Cardiol. 2021;46(9):100851. doi:10.1016/j.cpcardiol.2021.100851 PubMedGoogle ScholarCrossref
7.
Myla  M, Blankenship  JC.  The cost of non-kidney conserving catheterization.   Catheter Cardiovasc Interv. 2020;96(6):1198-1199. doi:10.1002/ccd.29354 PubMedGoogle ScholarCrossref
8.
Amin  AP, Bach  RG, Caruso  ML, Kennedy  KF, Spertus  JA.  Association of variation in contrast volume with acute kidney injury in patients undergoing percutaneous coronary intervention.   JAMA Cardiol. 2017;2(9):1007-1012. doi:10.1001/jamacardio.2017.2156 PubMedGoogle ScholarCrossref
9.
Brar  SS, Aharonian  V, Mansukhani  P,  et al.  Haemodynamic-guided fluid administration for the prevention of contrast-induced acute kidney injury: the POSEIDON randomised controlled trial.   Lancet. 2014;383(9931):1814-1823. doi:10.1016/S0140-6736(14)60689-9 PubMedGoogle ScholarCrossref
10.
Huang  C, Li  SX, Mahajan  S,  et al.  Development and validation of a model for predicting the risk of acute kidney injury associated with contrast volume levels during percutaneous coronary intervention.   JAMA Netw Open. 2019;2(11):e1916021. doi:10.1001/jamanetworkopen.2019.16021 PubMedGoogle ScholarCrossref
11.
Malik  AO, Amin  A, Kennedy  K,  et al.  Patient-centered contrast thresholds to reduce acute kidney injury in high-risk patients undergoing percutaneous coronary intervention.   Am Heart J. 2021;234:51-59. doi:10.1016/j.ahj.2020.12.013 PubMedGoogle ScholarCrossref
12.
Qian  G, Fu  Z, Guo  J, Cao  F, Chen  Y.  Prevention of contrast-induced nephropathy by central venous pressure-guided fluid administration in chronic kidney disease and congestive heart failure patients.   JACC Cardiovasc Interv. 2016;9(1):89-96. doi:10.1016/j.jcin.2015.09.026 PubMedGoogle ScholarCrossref
13.
Brown  JR, Solomon  RJ, Sarnak  MJ,  et al; Northern New England Cardiovascular Disease Study Group.  Reducing contrast-induced acute kidney injury using a regional multicenter quality improvement intervention.   Circ Cardiovasc Qual Outcomes. 2014;7(5):693-700. doi:10.1161/CIRCOUTCOMES.114.000903 PubMedGoogle ScholarCrossref
14.
Brown  JR, McCullough  PA, Splaine  ME,  et al; Northern New England Cardiovascular Disease Study Group.  How do centres begin the process to prevent contrast-induced acute kidney injury: a report from a new regional collaborative.   BMJ Qual Saf. 2012;21(1):54-62. doi:10.1136/bmjqs-2011-000041 PubMedGoogle ScholarCrossref
15.
James  MT, Har  BJ, Tyrrell  BD,  et al.  Clinical decision support to reduce contrast-induced kidney injury during cardiac catheterization: design of a randomized stepped-wedge trial.   Can J Cardiol. 2019;35(9):1124-1133. doi:10.1016/j.cjca.2019.06.002 PubMedGoogle ScholarCrossref
16.
Hemming  K, Taljaard  M, McKenzie  JE,  et al.  Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration.   BMJ. 2018;363:k1614. doi:10.1136/bmj.k1614 PubMedGoogle ScholarCrossref
17.
Ghali  WA, Knudtson  ML.  Overview of the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease: on behalf of the APPROACH investigators.   Can J Cardiol. 2000;16(10):1225-1230.PubMedGoogle Scholar
18.
Tsai  TT, Patel  UD, Chang  TI,  et al.  Validated contemporary risk model of acute kidney injury in patients undergoing percutaneous coronary interventions: insights from the National Cardiovascular Data Registry Cath-PCI Registry.   J Am Heart Assoc. 2014;3(6):e001380. doi:10.1161/JAHA.114.001380 PubMedGoogle ScholarCrossref
19.
Ma  B, Faris  P, Har  BJ,  et al.  Development and evaluation of an audit and feedback process for prevention of acute kidney injury during coronary angiography and intervention.   CJC Open. 2021;4(3):271-281. doi:10.1016/j.cjco.2021.10.006 PubMedGoogle ScholarCrossref
20.
Tonelli  M, Wiebe  N, Fortin  M,  et al; Alberta Kidney Disease Network.  Methods for identifying 30 chronic conditions: application to administrative data.   BMC Med Inform Decis Mak. 2015;15(1):31. doi:10.1186/s12911-015-0155-5 PubMedGoogle ScholarCrossref
21.
Vlasschaert  ME, Bejaimal  SA, Hackam  DG,  et al.  Validity of administrative database coding for kidney disease: a systematic review.   Am J Kidney Dis. 2011;57(1):29-43. doi:10.1053/j.ajkd.2010.08.031 PubMedGoogle ScholarCrossref
22.
De Coster  C, Li  B, Quan  H.  Comparison and validity of procedures coded with ICD-9-CM and ICD-10-CA/CCI.   Med Care. 2008;46(6):627-634. doi:10.1097/MLR.0b013e3181649439 PubMedGoogle ScholarCrossref
23.
Waikar  SS, Wald  R, Chertow  GM,  et al.  Validity of International Classification of Diseases, Ninth Revision, Clinical Modification codes for acute renal failure.   J Am Soc Nephrol. 2006;17(6):1688-1694. doi:10.1681/ASN.2006010073 PubMedGoogle ScholarCrossref
24.
Lee  DS, Donovan  L, Austin  PC,  et al.  Comparison of coding of heart failure and comorbidities in administrative and clinical data for use in outcomes research.   Med Care. 2005;43(2):182-188. doi:10.1097/00005650-200502000-00012 PubMedGoogle ScholarCrossref
25.
Austin  PC, Daly  PA, Tu  JV.  A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario.   Am Heart J. 2002;144(2):290-296. doi:10.1067/mhj.2002.123839 PubMedGoogle ScholarCrossref
26.
Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. doi:10.1016/j.cct.2006.05.007 PubMedGoogle ScholarCrossref
27.
Hemming  K, Taljaard  M.  Sample size calculations for stepped wedge and cluster randomised trials: a unified approach.   J Clin Epidemiol. 2016;69:137-146. doi:10.1016/j.jclinepi.2015.08.015 PubMedGoogle ScholarCrossref
28.
Hemming  K, Haines  TP, Chilton  PJ, Girling  AJ, Lilford  RJ.  The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting.   BMJ. 2015;350(1):h391-h391. doi:10.1136/bmj.h391PubMedGoogle ScholarCrossref
29.
Amin  AP, Crimmins-Reda  P, Miller  S,  et al.  Reducing acute kidney injury and costs of percutaneous coronary intervention by patient-centered, evidence-based contrast use.   Circ Cardiovasc Qual Outcomes. 2019;12(3):e004961. doi:10.1161/CIRCOUTCOMES.118.004961 PubMedGoogle ScholarCrossref
30.
Davis  DA, Thomson  MA, Oxman  AD, Haynes  RB.  Changing physician performance: a systematic review of the effect of continuing medical education strategies.   JAMA. 1995;274(9):700-705. doi:10.1001/jama.1995.03530090032018 PubMedGoogle ScholarCrossref
31.
Mansouri  M, Lockyer  J.  A meta-analysis of continuing medical education effectiveness.   J Contin Educ Health Prof. 2007;27(1):6-15. doi:10.1002/chp.88 PubMedGoogle ScholarCrossref
32.
Roshanov  PS, Fernandes  N, Wilczynski  JM,  et al.  Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials.   BMJ. 2013;346(1):f657-f657. doi:10.1136/bmj.f657PubMedGoogle ScholarCrossref
33.
Kawamoto  K, Houlihan  CA, Balas  EA, Lobach  DF.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success.   BMJ. 2005;330(7494):765. doi:10.1136/bmj.38398.500764.8F PubMedGoogle ScholarCrossref
34.
Brehaut  JC, Colquhoun  HL, Eva  KW,  et al.  Practice feedback interventions: 15 suggestions for optimizing effectiveness.   Ann Intern Med. 2016;164(6):435-441. doi:10.7326/M15-2248 PubMedGoogle ScholarCrossref
35.
Ivers  N, Jamtvedt  G, Flottorp  S,  et al.  Audit and feedback: effects on professional practice and healthcare outcomes.   Cochrane Database Syst Rev. 2012;6(6):CD000259. doi:10.1002/14651858.CD000259.pub3PubMedGoogle ScholarCrossref
36.
Sargeant  J, Lockyer  J, Mann  K,  et al.  Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2).   Acad Med. 2015;90(12):1698-1706. doi:10.1097/ACM.0000000000000809 PubMedGoogle ScholarCrossref
37.
Carless  D, Salter  D, Yang  M, Lam  J.  Developing sustainable feedback practices.   Stud High Educ. 2010;36(4):395-407. doi:10.1080/03075071003642449 Google ScholarCrossref
38.
Overeem  K, Driessen  EW, Arah  OA, Lombarts  KM, Wollersheim  HC, Grol  RP.  Peer mentoring in doctor performance assessment: strategies, obstacles and benefits.   Med Educ. 2010;44(2):140-147. doi:10.1111/j.1365-2923.2009.03580.x PubMedGoogle ScholarCrossref
39.
Hysong  SJ.  Meta-analysis: audit and feedback features impact effectiveness on care quality.   Med Care. 2009;47(3):356-363. doi:10.1097/MLR.0b013e3181893f6b PubMedGoogle ScholarCrossref
40.
Nilsen  P.  Making sense of implementation theories, models and frameworks.   Implement Sci. 2015;10(1):53. doi:10.1186/s13012-015-0242-0 PubMedGoogle ScholarCrossref
Original Investigation
September 6, 2022

Effect of Clinical Decision Support With Audit and Feedback on Prevention of Acute Kidney Injury in Patients Undergoing Coronary Angiography: A Randomized Clinical Trial

Author Affiliations
  • 1Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  • 2Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  • 3Libin Cardiovascular Institute, University of Calgary, Calgary, Alberta, Canada
  • 4O’Brien Institute of Public Health, University of Calgary, Calgary, Alberta, Canada
  • 5Department of Cardiac Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  • 6Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada
  • 7CK Hui Heart Centre, University of Alberta, Edmonton, Canada
  • 8Alberta Health Services, Calgary, Canada
  • 9Departments of Biomedical and Health Informatics, University of Missouri, Kansas City
  • 10Saint Luke’s Mid America Heart Institute, Kansas City, Missouri
  • 11Mazankowski Alberta Heart Institute, University of Alberta, Edmonton, Canada
JAMA. 2022;328(9):839-849. doi:10.1001/jama.2022.13382
Visual Abstract. Effect of Clinical Decision Support With Audit and Feedback on Prevention of Acute Kidney Injury
Effect of Clinical Decision Support With Audit and Feedback on Prevention of Acute Kidney Injury
Key Points

Question  What is the effectiveness of clinical decision support, accompanied by audit and feedback to physicians, for prevention of acute kidney injury in patients undergoing coronary angiography, percutaneous coronary intervention, or both?

Findings  In this stepped-wedge, cluster randomized clinical trial conducted in Alberta, Canada, that included 34 cardiologists who performed 7820 procedures, the incidence of acute kidney injury during the intervention period compared with the control period was 7.2% vs 8.6%, a difference that was statistically significant.

Meaning  This multifaceted intervention led to a lower risk of acute kidney injury in patients undergoing coronary angiography, percutaneous coronary intervention, or both; however, whether this intervention would be effective in other settings requires further investigation.

Abstract

Importance  Contrast-associated acute kidney injury (AKI) is a common complication of coronary angiography and percutaneous coronary intervention (PCI) that has been associated with high costs and adverse long-term outcomes.

Objective  To determine whether a multifaceted intervention is effective for the prevention of AKI after coronary angiography or PCI.

Design, Setting, and Participants  A stepped-wedge, cluster randomized clinical trial was conducted in Alberta, Canada, that included all invasive cardiologists at 3 cardiac catheterization laboratories who were randomized to various start dates for the intervention between January 2018 and September 2019. Eligible patients were aged 18 years or older who underwent nonemergency coronary angiography, PCI, or both; who were not undergoing dialysis; and who had a predicted AKI risk of greater than 5%. Thirty-four physicians performed 7820 procedures among 7106 patients who met the inclusion criteria. Participant follow-up ended in November 2020.

Interventions  During the intervention period, cardiologists received educational outreach, computerized clinical decision support on contrast volume and hemodynamic-guided intravenous fluid targets, and audit and feedback. During the control (preintervention) period, cardiologists provided usual care and did not receive the intervention.

Main Outcomes and Measures  The primary outcome was AKI. There were 12 secondary outcomes, including contrast volume, intravenous fluid administration, and major adverse cardiovascular and kidney events. The analyses were conducted using time-adjusted models.

Results  Of the 34 participating cardiologists who were divided into 8 clusters by practice group and center, the intervention group included 31 who performed 4327 procedures among 4032 patients (mean age, 70.3 [SD, 10.7] years; 1384 were women [32.0%]) and the control group included 34 who performed 3493 procedures among 3251 patients (mean age, 70.2 [SD, 10.8] years; 1151 were women [33.0%]). The incidence of AKI was 7.2% (310 events after 4327 procedures) during the intervention period and 8.6% (299 events after 3493 procedures) during the control period (between-group difference, −2.3% [95% CI, −0.6% to −4.1%]; odds ratio [OR], 0.72 [95% CI, 0.56 to 0.93]; P = .01). Of 12 prespecified secondary outcomes, 8 showed no significant difference. The proportion of procedures in which excessive contrast volumes were used was reduced to 38.1% during the intervention period from 51.7% during the control period (between-group difference, −12.0% [95% CI, −14.4% to −9.4%]; OR, 0.77 [95% CI, 0.65 to 0.90]; P = .002). The proportion of procedures in eligible patients in whom insufficient intravenous fluid was given was reduced to 60.8% during the intervention period from 75.1% during the control period (between-group difference, −15.8% [95% CI, −19.7% to −12.0%]; OR, 0.68 [95% CI, 0.53 to 0.87]; P = .002). There were no significant between-group differences in major adverse cardiovascular events or major adverse kidney events.

Conclusions and Relevance  Among cardiologists randomized to an intervention including clinical decision support with audit and feedback, patients undergoing coronary procedures during the intervention period were less likely to develop AKI compared with those treated during the control period, with a time-adjusted absolute risk reduction of 2.3%. Whether this intervention would show efficacy outside this study setting requires further investigation.

Trial Registration  ClinicalTrials.gov Identifier: NCT03453996

Introduction

Between 4% and 10% of patients undergoing coronary angiography or percutaneous coronary intervention (PCI) experience acute kidney injury (AKI),1,2 and the cost of attributable care has been estimated as high as $1.67 billion annually in the US.3 Acute kidney injury has been associated with several adverse kidney and cardiovascular outcomes, including acute dialysis, end-stage kidney disease, heart failure, atherosclerotic events, and death.2,4 Although guidelines recommend minimizing contrast dye exposure and providing periprocedure intravenous hydration for prevention of AKI,5 there is variable uptake of these strategies in care.6-8

Personalized strategies for calculating safe contrast dye volumes and for tailoring intravenous fluid volumes based on hemodynamic criteria have been developed for kidney protection.9-12 However, systematic processes to identify patients at risk of AKI and incorporate prevention strategies have not been implemented at most centers performing invasive coronary procedures.13,14 The large variation in practice patterns among physicians and AKI rates suggest that prevention of AKI in patients undergoing coronary angiography or PCI could be improved,8 and tailored strategies might guide clinicians toward identifying high-risk patients and incorporating tactics to support AKI prevention.

This stepped-wedge, cluster randomized clinical trial was designed to determine whether a multifaceted intervention composed of educational outreach, clinical decision support, and audit and feedback could improve AKI prevention practices and reduce the incidence of AKI after patients undergo coronary angiography, PCI, or both.

Methods
Study Design

The Contrast RISK (Reducing Injury Sustained by Kidneys) trial was a pragmatic, stepped-wedge, cluster randomized clinical trial in which invasive cardiologists in Alberta, Canada, were randomly assigned to 1 of 8 start dates for the intervention between January 1, 2018, and September 1, 2019. Each physician contributed data prior to receiving the intervention for a period ranging from 10 weeks to 80 weeks, with continued data collection ranging from 10 weeks to 80 weeks after introduction to the intervention, such that by the end of the trial all practicing physicians had been exposed to the intervention. Detailed methods have been published15 and appear in the trial protocol in Supplement 1.

This trial was approved by the health research ethics boards at the universities of Alberta and Calgary in Canada, which granted a waiver of patient consent because the intervention was directed at physicians, the cluster design precluded patients from opting out, the intervention promoted evidence-based practices, and the risk to patients was low. All physicians provided informed consent to participate in the study, including for the audit and feedback process.16

Participants

All invasive cardiologists in the province of Alberta, Canada, were eligible to participate, with the exception of the lead physician for the trial at each site. The 3 cardiac catheterization laboratories in the province of Alberta used a common, point-of-care electronic clinical information system (Alberta Provincial Project for Outcomes Assessment in Coronary Heart Disease), which collects demographic and clinical information on all patients undergoing coronary angiography and subsequent procedures.17 For each physician, all Alberta residents undergoing invasive procedures were included in the trial if they were aged 18 years or older at the time of the coronary procedure and had a greater than 5% risk of AKI based on the National Cardiovascular Data Registry18 multivariable AKI risk prediction model. Patients were excluded if they were undergoing dialysis or underwent an emergency primary PCI for ST-segment elevation myocardial infarction.

Randomization

Physicians were randomized to 1 of 8 start dates by an independent statistician using computer-generated random numbers, with randomization stratified by clusters (3-6 physicians per cluster) determined by each physician’s site and practice group (Figure 1). The start-date assignment was concealed from physicians and other members of the research team until the month before their scheduled introduction date to allow sufficient time to plan the educational session at the initiation of the intervention.

The first physician educational session was held on January 22, 2018 (start date), and the trial was registered on February 27, 2018 (without any changes to the trial protocol), due to a delay before the decision support intervention was ready to be delivered to the first cluster of physicians. Given the nature of the intervention, blinding of the physicians, catheterization unit staff, and members of the research team was not possible, and all staff at each site worked with physicians randomized to different start dates.

Intervention Period

The intervention was composed of 3 components.15 First, cardiologists received a 1-hour educational session (provided immediately prior to the period they started to receive the intervention) that included information about AKI, prevention approaches, and the components of the trial intervention (Supplement 1). The training material and an instructional video also were provided.

Second, physicians received clinical decision support that included individualized AKI risk prediction, graphic display of safe contrast volume targets, and automated calculation of hemodynamic-guided intravenous fluid volume targets according to left ventricular end-diastolic pressure measurements obtained during cardiac catheterization (Supplement 1). Catheterization unit staff executed the AKI risk prediction model, which classified patients as low (≤5%), above average (>5%-25%), or at very high risk (>25%) of developing AKI. Safe contrast volume targets were calculated with the ePRISM tool (Health Outcomes Sciences), which used a multivariable model estimating the maximum contrast volume to reduce the relative risk of AKI by 15% when a patient’s absolute risk of AKI exceeded 5%.11 Cardiologists who received the intervention were alerted to the safe contrast volume target by staff before the procedure and again when the safe contrast volume target had been reached. Hemodynamic-guided intravenous fluid volume targets were calculated automatically in the clinical information system based on the patient’s left ventricular end-diastolic pressure and weight9 in those without severe aortic valve disease or active or recent heart failure within the last 2 weeks. The decision to exceed contrast volume targets or vary (provide less or more) the intravenous fluid volume was left to the discretion of the treating physicians.

Third, physicians received audit and feedback every 3 months after their initial introduction to the intervention. Audit and feedback reported on the contrast volume used relative to the safe contrast volume target, the administration of intravenous fluids compared with the hemodynamic-optimized recommendations, and the AKI incidence for patients with an AKI risk greater than 5%.19

Control Period

During the control (preintervention) period, cardiologists provided usual care and did not receive educational outreach, clinical decision support, or audit and feedback.

Outcome Measures

The primary outcome was AKI (defined by serum creatinine–based criteria from the Kidney Disease: Improving Global Outcomes AKI guidelines).5 The last creatinine level collected before the catheterization procedure was used to define the baseline creatinine level and follow-up creatinine testing was performed between 48 hours and 96 hours after procedures for inpatients and outpatients. Alternate, prespecified definitions for AKI also were examined in the secondary analyses (Supplement 1).

Prespecified secondary process of care outcomes were obtained from the clinical information system and hospital medical records and included the contrast volume used, the proportion of procedures with excessive contrast volume (ie, the contrast volume target was exceeded by >15 mL), the intravenous fluid volume administered for AKI prevention up to 6 hours after each procedure, and the proportion of procedures with insufficient fluid (for which the amount of intravenous fluid administered was >50 mL below the hemodynamic-guided target for those without severe aortic valve disease or recent heart failure).

Prespecified secondary clinical outcomes were obtained from provincial hospital and laboratory data sources available for all patients in Alberta, Canada, and included number of days in the hospital within 30 days after a procedure, major adverse cardiovascular events (including death; hospitalization for heart failure, angina, or myocardial infarction; or an unplanned revascularization procedure), and major adverse kidney events (including death, acute dialysis, subsequent hospitalization for AKI, or end-stage kidney disease [defined as undergoing maintenance dialysis, receiving a transplant, or having a sustained estimated glomerular filtration rate {eGFR} <10 mL/min/1.73 m2 for 3 months]) within 1 year after a procedure.

Chronic dialysis and kidney transplant were identified from Alberta Kidney Care programs, whereas validated algorithms using the Canadian version of the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, and the Canadian Classification of Health Interventions were applied to the Alberta Health Services databases to identify cardiovascular events, hospitalizations for AKI, and patients undergoing acute dialysis.20-25 Additional secondary outcomes of the trial included longitudinal changes in eGFR and cost utility of the intervention, which are not reported in this article.

Sample Size Calculation

Because historical data from Alberta, Canada, demonstrated a 10% incidence of AKI in the eligible population, with an intracluster correlation among physicians of 0.054, and we projected 7270 procedures (90%) with follow-up serum creatinine testing performed in eligible patients over the duration of the trial, we estimated 80% power to detect a 30% reduction in the primary outcome of AKI at an α level of .05 with 35 physicians randomly assigned to 8 start dates.26,27

Statistical Analysis

The analyses compared the outcomes of patients treated by physicians who were receiving the intervention at the time of their procedure (intervention group) vs the outcomes of patients treated by physicians not yet receiving the intervention (control group) and were performed according to the intended randomization schedule for the 8 clusters of physicians. For the outcomes of AKI and contrast volume (when data were missing), we used multiple imputation with 20 imputed data sets for the main analyses and used complete case analysis for the sensitivity analyses.

The variables included in the imputation models were sex; intervention group vs control group; procedure date, indication, and type; diabetes; heart failure; baseline serum creatinine level; predicted risk of AKI; physician; and site. Multilevel logistic regression was used to estimate the odds ratios (ORs) and 95% CIs for the categorical outcomes. Mixed-effects linear regression was used to estimate the differences in the continuous outcomes. Random-effects regression was used to account for clustering by physician and site and for repeated procedures in the same patient and fixed-effects regression was used to adjust for time by calendar month.28

Multivariable-adjusted analyses also were performed for each outcome that included further adjustment for patient age, sex, diabetes, heart failure, eGFR, and procedure indication as fixed effects. The effects on the primary outcome also were examined in prespecified subgroup analyses according to age, sex, diabetes, heart failure, chronic kidney disease, procedure type, and baseline AKI risk. Modification of the treatment effect was tested by including a treatment × month interaction term in the time-adjusted model. Because some cardiologists retired from practice prior to receiving the intervention, a post hoc analysis that excluded these physicians also was performed.

The significance threshold was .05 and testing was 2-sided. Because of the potential for type I error due to multiple comparisons, the findings for the analyses of the secondary outcomes should be interpreted as exploratory. The statistical analyses were performed using SAS version 9.4 (SAS Institute Inc).

Results
Study Population

Thirty-four invasive cardiologists in Alberta, Canada, contributed data to the trial, of whom 3 contributed data only to the control group (preintervention period) because they retired from practice before receiving the intervention (Figure 1). The intervention was delivered as intended to the remaining 31 physicians (mean age, 51.2 years [SD, 7.2 years]; 4 were women [12.9%]) who had been practicing for a mean of 19.6 years (SD, 8.1 years). A total of 29 418 coronary angiography or PCI procedures were performed in 26 110 patients during the study period (eFigure 1 in Supplement 2).

After excluding 19 004 patients undergoing dialysis, urgent primary PCI for ST-segment elevation myocardial infarction, and those with less than a 5% risk of AKI, there were 7820 procedures in 7106 patients eligible for inclusion (eFigure 1 in Supplement 2). Of the 7820 procedures, 794 (10.2%) were missing data for the primary outcome (serum creatinine was not measured during follow-up to ascertain AKI) and 698 (8.9%) were missing data on contrast volume. The differences in the baseline characteristics between the procedures with complete data and those with incomplete data for the outcome of AKI appear in eTable 1 in Supplement 2.

The number of procedures performed by each physician among patients with a greater than 5% risk of AKI ranged from 4 to 334 across the 31 physicians during the intervention period (median, 130 procedures per physician) and from 22 to 325 across the 34 physicians during the control period (median, 87 procedures per physician). There were small differences in procedure indication between the intervention group and the control group, but patient demographic characteristics, comorbidities, preprocedure eGFR, predicted risk of AKI, and fluoroscopy time (a surrogate of procedure complexity) were similar between the groups (Table 1).

Primary Outcome

Over the 2-year duration of the trial, there were 310 AKI events after 4327 procedures (7.2%) performed by physicians in the intervention group compared with 299 AKI events after 3493 procedures (8.6%) performed by physicians in the control group (Table 2). In the primary analysis accounting for clustering and adjusted for time, the intervention resulted in a significant odds reduction in AKI (time-adjusted OR, 0.72 [95% CI, 0.56 to 0.93], P = .01; time-adjusted risk difference, −2.3% [95% CI, −0.6% to −4.1%]) with a consistent effect over the duration of the trial (P = .27 for treatment × time interaction).

The results were similar in the multivariable analyses further adjusted for age, sex, comorbidities, eGFR, and procedure indication (adjusted OR, 0.67 [95% CI, 0.52-0.86]) and in the complete case analysis (eTable 2 in Supplement 2). This effect was consistently observed in subgroups defined by age, sex, with or without heart failure or chronic kidney disease, at moderate or high risk of AKI, and in those who underwent coronary angiography alone or procedures including PCI (Figure 2) as well as when alternate serum creatinine-based definitions for AKI were examined (eTable 3 in Supplement 2). In a post hoc analysis, the results remained consistent after excluding data from the 3 physicians who retired before receiving the intervention (eTable 4 in Supplement 2).

Secondary Process of Care Outcomes

The contrast volume administered declined to a mean of 93.1 mL (SD, 61.2 mL) during the intervention period from a mean of 112.7 mL (SD, 67.7 mL) during the control (preintervention) period (time-adjusted mean difference, −17.7 mL [95% CI, −12.8 to −22.5 mL) (eTable 5 and eFigure 2 in Supplement 2). The proportion of cases exceeding the patient-specific contrast volume target was reduced to 38.1% during the intervention period from 51.7% during the control period (between-group difference, −12.0% [95% CI, −14.4% to −9.4%]; time-adjusted OR, 0.77 [95% CI, 0.65 to 0.91]; Table 2). The results were similar in the complete case analysis (eTable 2 in Supplement 2).

The intravenous fluid volume administered increased to a mean of 851.4 mL (SD, 596.4 mL) during the intervention period from 650.0 mL (SD, 467.4 mL) during the control (preintervention) period (time-adjusted mean difference, 112.0 mL [95% CI, 55.4 to 168.7 mL]). The proportion of eligible patients who received less than the hemodynamic-guided intravenous fluid volume target decreased to 60.8% during the intervention period from 75.1% during the control period (between-group difference, −15.8% [95% CI, −19.7% to −12.0%]; time-adjusted OR, 0.68 [95% CI, 0.53 to 0.87]; eTable 6 and eFigure 3 in Supplement 2).

Secondary Clinical Outcomes

The number of days patients were in the hospital within 30 days after a procedure was not significantly different between the groups (median, 2.5 days [IQR, 1.1 to 8.1 days] in the intervention group vs 2.9 days [IQR, 1.1 to 8.0 days] in the control group; mean log difference, −0.3 days [95% CI, −0.9 to 0.4 days]; Table 2). There were no significant between-group differences in major adverse cardiovascular events (28.6% in the intervention group vs 31.5% in the control group; time-adjusted OR, 0.96 [95% CI, 0.83 to 1.12]) or major adverse kidney events (9.4% and 11.2%, respectively; time-adjusted OR, 0.89 [95% CI, 0.72 to 1.10]). There were also no significant differences in the components of these secondary outcomes (Table 2).

Discussion

Among cardiologists in Alberta, Canada, randomized to an intervention that included clinical decision support with audit and feedback, patients undergoing coronary procedures during the intervention period were less likely to develop contrast-associated AKI compared with those treated during the control (preintervention) period.

Prior observational studies and quality improvement initiatives have reported that administration of lower contrast volumes is associated with reduced risk of AKI. An initiative that implemented a personalized approach to determine contrast volume limits at 1 US center reported a mean reduction in contrast volume of 55 mL among 3377 patients and an increased odds of AKI when the contrast volume limit was exceeded (OR, 1.95 [95% CI, 1.41-2.69]).29 A multicenter quality improvement initiative that included contrast volume reductions among 21 067 patients at 6 US hospitals was associated with a 20% relative reduction in risk of AKI.13 In a US cohort of almost 1 million patients with greater than average risk of AKI, exceeding a personalized contrast volume target was associated with an 8% higher absolute incidence of AKI.11 Small randomized clinical trials (N = 396 patients9 and N = 264 patients12) that used individualized intravenous fluid regimens tailored to left ventricular end-diastolic pressure and central venous pressure measurements have also reported reductions in AKI incidence after patients undergo coronary procedures.

The current stepped-wedge, cluster randomized clinical trial strengthens the evidence base for these personalized strategies for AKI prevention when implemented across a large health jurisdiction. In this trial, providing decision support with personalized targets for contrast volume and intravenous fluid volume led to significant overall changes in practice by invasive cardiologists treating patients identified as being at risk of AKI; however, these targets were still not met for many patients. Higher contrast volume targets may be required for technical reasons and may be perceived as justified by physicians. Further research is needed to identify barriers that prevent physicians from achieving these targets more frequently and the effect that additional facilitators could have on reducing AKI.

This trial demonstrated that a combination of educational outreach, point-of-care clinical decision support tailored to readily measurable patient characteristics, and audit and feedback for AKI prevention was effective in changing the practices of invasive cardiologists and reducing AKI; however, this study cannot disentangle which aspects of this intervention mediated this effect. Educational interventions alone often have limited effects, particularly in passive, large group settings.30,31 Therefore, the current trial provided a 1-on-1 educational intervention to physicians on available strategies (reducing contrast volume and hemodynamic-guided intravenous fluid administration) and evidence for identifying patients at risk of AKI, which may have been more effective in contributing to improvement. A large body of evidence has demonstrated that clinical decision support can be effective.32,33 The current trial used decision support incorporating features that have been demonstrated to underlie its effectiveness, including automatic provision of support within routine clinical workflow, information provided in a manner that is actionable, support provided at the time and location of decision-making, and use of computer-based tools with documentation of actions. The audit and feedback aspect of the intervention was included to augment the trial intervention by promoting accountability and reinforcing and sustaining behavioral changes targeted by the decision support. Audit and feedback alone generally leads to small improvements in practice, so the study followed established best practices for its implementation, including providing it via a colleague, more than once, delivering it in both verbal and written formats, and including explicit targets and prompting individuals to develop action plans.34-39

This trial was designed to detect a significant reduction in the incidence of AKI but was not powered to detect beneficial differences in the secondary outcomes of major adverse cardiovascular and kidney events, many of which would be expected to accrue during longer-term follow-up. In the context of this limitation, the intervention did not lead to measurable increases in the shorter-term risks of recurrent acute coronary syndromes, urgent revascularization procedures, or heart failure, which could be unintended consequences of reducing the contrast volume and increasing the administration of intravenous fluids. Studies with longer follow-up that are robustly powered to examine downstream kidney outcomes associated with AKI are needed to determine effects on long-term kidney outcomes. Although this intervention may have similar effects on process of care outcomes and AKI in many other jurisdictions,6,8 the effect may vary based on the state of existing performance across clinical centers. Implementation of this intervention is feasible when electronic medical records are available at the point of care, when clinical informatics resources are available to audit and report on care practices, and when a culture of quality improvement exists, which are all variables that are increasingly common in modern health care systems. However, successful implementation of interventions of this nature requires tailoring to the local context.40

The strengths of this trial include the pragmatic, evidence-informed design of the intervention, evaluation within cardiac catheterization unit practices, inclusion of all eligible invasive cardiologists in the provincial health system, and examination of the effect of implementation on process of care outcomes as well as clinical outcomes.

Limitations

This study has several limitations. First, although a stepped-wedge, cluster randomized clinical trial design may have lower risk of bias than nonrandomized designs, the stepped-wedge aspect can be vulnerable to secular trends if outcomes are already improving without the intervention.16,28 The incidence of AKI was stable over the trial period and there were no other relevant interventions delivered at the participating cardiac catheterization laboratories during the implementation period of the trial, which indicates that secular trends are unlikely to explain the findings.

Second, the stepped-wedge design may also be vulnerable to contamination if physicians who had not yet received the intervention were aware of the trial and changed their behaviors before receiving the intervention. This would be expected to attenuate the effect of the intervention, suggesting that the treatment effect observed in the trial may be a conservative estimate of the effect of the intervention.

Third, data for the primary outcome of AKI were missing for 10.2% of the cohort. Multiple imputation was used under the assumption that the incomplete data were missing at random such that the missingness of the AKI ascertainment was associated only with the observed data. Similar effects were found in other analyses using complete case analysis and multiple imputation.

Fourth, the trial included a relatively small number of physician clusters and the magnitude of the effect size was modest and less than the clinically important difference specified for the design of the trial. This reduced the precision of the estimate of the treatment effect and resulted in greater fragility to the findings.

Fifth, this multifactorial intervention was designed for cardiologists practicing in Canada. Findings may not be generalizable to other countries, and appropriate tailoring and further evaluation is required in other settings.

Conclusions

Among cardiologists randomized to an intervention including clinical decision support with audit and feedback, patients undergoing coronary procedures during the intervention period were less likely to develop AKI compared with those treated during the control period, with a time-adjusted absolute risk reduction of 2.3%. Whether this intervention would show efficacy outside this study setting requires further investigation.

Back to top
Article Information

Corresponding Author: Matthew T. James, MD, PhD, Departments of Medicine and Community Health Sciences, Cumming School of Medicine, University of Calgary, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada (mjames@ucalgary.ca).

Correction: This article was corrected November 15, 2022, to fix 2 sentences in the first paragraph of the Primary Outcome section in the Results section because of incorrect data placement.

Accepted for Publication: July 15, 2022.

Author Contributions: Dr James had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: James, Har, Tyrrell, Spertus, Wilton, Ghali, Knudtson, Sajobi, Pannu, Klarenbach, Graham.

Acquisition, analysis, or interpretation of data: James, Har, Faris, Tan, Wilton, Knudtson, Sajobi.

Drafting of the manuscript: James, Tyrrell, Faris, Tan.

Critical revision of the manuscript for important intellectual content: Har, Tyrrell, Faris, Spertus, Wilton, Ghali, Knudtson, Sajobi, Pannu, Klarenbach, Graham.

Statistical analysis: James, Faris, Tan, Sajobi.

Obtained funding: James, Ghali, Sajobi, Graham.

Administrative, technical, or material support: James, Har, Spertus, Wilton, Pannu, Graham.

Supervision: James, Har, Tyrrell, Ghali, Graham.

Conflict of Interest Disclosures: Dr James reported receiving grants from Amgen Canada. Dr Spertus reported being a medical adviser to Health Outcomes Sciences, Terumo, which is the maker of ePRISM software; receiving grants from Abbott Vascular; receiving personal fees from Janssen, United Healthcare, Merck, Novartis, and Bayer; holding a patent and receiving royalties for the Seattle Angina Questionnaire, the Kansas City Cardiomyopathy Questionnaire, and the Peripheral Artery Questionnaire; and being on the board of directors for Blue Cross Blue Shield of Kansas City. Dr Wilton reported receiving grants from Medtronic Canada, Abbott, and Boston Scientific and receiving personal fees from Arca Biopharma. Dr Klarenbach reported being the co-scientific director of the kidney health section of the Medicine Strategic Clinical Network within Alberta Health Services and receiving grants from the Real World Evidence Consortium (universities of Alberta and Calgary, Institute of Health Economics). No other disclosures were reported.

Funding/Support: Supported by partnership for research and innovation in the health system grant 201600425 from Alberta Innovates and Alberta Health Services. Dr James was supported by a Canadian Institutes of Health Research new investigator award. The Alberta Provincial Project for Outcomes Assessment in Coronary Heart Disease was supported by the Alberta Health Services (Cardiac Sciences), the Libin Cardiovascular Institute, and the Mazankowski Alberta Heart Institute.

Role of the Funder/Sponsor: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The interpretation and conclusions contained herein are those of the authors and do not necessarily represent the views of Alberta Health Services.

Data Sharing Statement: See Supplement 3.

Additional Information: This study is based in part on data provided by Alberta Health Services.

References
1.
Mehran  R, Dangas  GD, Weisbord  SD.  Contrast-associated acute kidney injury.   N Engl J Med. 2019;380(22):2146-2155. doi:10.1056/NEJMra1805256 PubMedGoogle ScholarCrossref
2.
Azzalini  L, Candilio  L, McCullough  PA, Colombo  A.  Current risk of contrast-induced acute kidney injury after coronary angiography and intervention: a reappraisal of the literature.   Can J Cardiol. 2017;33(10):1225-1228. doi:10.1016/j.cjca.2017.07.482 PubMedGoogle ScholarCrossref
3.
Amin  AP, McNeely  C, Spertus  JA,  et al.  Incremental cost of acute kidney injury after percutaneous coronary intervention in the United States.   Am J Cardiol. 2020;125(1):29-33. doi:10.1016/j.amjcard.2019.09.042 PubMedGoogle ScholarCrossref
4.
James  MT, Ghali  WA, Knudtson  ML,  et al; Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) Investigators.  Associations between acute kidney injury and cardiovascular and renal outcomes after coronary angiography.   Circulation. 2011;123(4):409-416. doi:10.1161/CIRCULATIONAHA.110.970160 PubMedGoogle ScholarCrossref
5.
Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group.  KDIGO clinical practice guideline for acute kidney injury.   Kidney Int. 2012;2:1-138.Google ScholarCrossref
6.
Valdenor  C, McCullough  PA, Paculdo  D,  et al.  Measuring the variation in the prevention and treatment of CI-AKI among interventional cardiologists.   Curr Probl Cardiol. 2021;46(9):100851. doi:10.1016/j.cpcardiol.2021.100851 PubMedGoogle ScholarCrossref
7.
Myla  M, Blankenship  JC.  The cost of non-kidney conserving catheterization.   Catheter Cardiovasc Interv. 2020;96(6):1198-1199. doi:10.1002/ccd.29354 PubMedGoogle ScholarCrossref
8.
Amin  AP, Bach  RG, Caruso  ML, Kennedy  KF, Spertus  JA.  Association of variation in contrast volume with acute kidney injury in patients undergoing percutaneous coronary intervention.   JAMA Cardiol. 2017;2(9):1007-1012. doi:10.1001/jamacardio.2017.2156 PubMedGoogle ScholarCrossref
9.
Brar  SS, Aharonian  V, Mansukhani  P,  et al.  Haemodynamic-guided fluid administration for the prevention of contrast-induced acute kidney injury: the POSEIDON randomised controlled trial.   Lancet. 2014;383(9931):1814-1823. doi:10.1016/S0140-6736(14)60689-9 PubMedGoogle ScholarCrossref
10.
Huang  C, Li  SX, Mahajan  S,  et al.  Development and validation of a model for predicting the risk of acute kidney injury associated with contrast volume levels during percutaneous coronary intervention.   JAMA Netw Open. 2019;2(11):e1916021. doi:10.1001/jamanetworkopen.2019.16021 PubMedGoogle ScholarCrossref
11.
Malik  AO, Amin  A, Kennedy  K,  et al.  Patient-centered contrast thresholds to reduce acute kidney injury in high-risk patients undergoing percutaneous coronary intervention.   Am Heart J. 2021;234:51-59. doi:10.1016/j.ahj.2020.12.013 PubMedGoogle ScholarCrossref
12.
Qian  G, Fu  Z, Guo  J, Cao  F, Chen  Y.  Prevention of contrast-induced nephropathy by central venous pressure-guided fluid administration in chronic kidney disease and congestive heart failure patients.   JACC Cardiovasc Interv. 2016;9(1):89-96. doi:10.1016/j.jcin.2015.09.026 PubMedGoogle ScholarCrossref
13.
Brown  JR, Solomon  RJ, Sarnak  MJ,  et al; Northern New England Cardiovascular Disease Study Group.  Reducing contrast-induced acute kidney injury using a regional multicenter quality improvement intervention.   Circ Cardiovasc Qual Outcomes. 2014;7(5):693-700. doi:10.1161/CIRCOUTCOMES.114.000903 PubMedGoogle ScholarCrossref
14.
Brown  JR, McCullough  PA, Splaine  ME,  et al; Northern New England Cardiovascular Disease Study Group.  How do centres begin the process to prevent contrast-induced acute kidney injury: a report from a new regional collaborative.   BMJ Qual Saf. 2012;21(1):54-62. doi:10.1136/bmjqs-2011-000041 PubMedGoogle ScholarCrossref
15.
James  MT, Har  BJ, Tyrrell  BD,  et al.  Clinical decision support to reduce contrast-induced kidney injury during cardiac catheterization: design of a randomized stepped-wedge trial.   Can J Cardiol. 2019;35(9):1124-1133. doi:10.1016/j.cjca.2019.06.002 PubMedGoogle ScholarCrossref
16.
Hemming  K, Taljaard  M, McKenzie  JE,  et al.  Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration.   BMJ. 2018;363:k1614. doi:10.1136/bmj.k1614 PubMedGoogle ScholarCrossref
17.
Ghali  WA, Knudtson  ML.  Overview of the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease: on behalf of the APPROACH investigators.   Can J Cardiol. 2000;16(10):1225-1230.PubMedGoogle Scholar
18.
Tsai  TT, Patel  UD, Chang  TI,  et al.  Validated contemporary risk model of acute kidney injury in patients undergoing percutaneous coronary interventions: insights from the National Cardiovascular Data Registry Cath-PCI Registry.   J Am Heart Assoc. 2014;3(6):e001380. doi:10.1161/JAHA.114.001380 PubMedGoogle ScholarCrossref
19.
Ma  B, Faris  P, Har  BJ,  et al.  Development and evaluation of an audit and feedback process for prevention of acute kidney injury during coronary angiography and intervention.   CJC Open. 2021;4(3):271-281. doi:10.1016/j.cjco.2021.10.006 PubMedGoogle ScholarCrossref
20.
Tonelli  M, Wiebe  N, Fortin  M,  et al; Alberta Kidney Disease Network.  Methods for identifying 30 chronic conditions: application to administrative data.   BMC Med Inform Decis Mak. 2015;15(1):31. doi:10.1186/s12911-015-0155-5 PubMedGoogle ScholarCrossref
21.
Vlasschaert  ME, Bejaimal  SA, Hackam  DG,  et al.  Validity of administrative database coding for kidney disease: a systematic review.   Am J Kidney Dis. 2011;57(1):29-43. doi:10.1053/j.ajkd.2010.08.031 PubMedGoogle ScholarCrossref
22.
De Coster  C, Li  B, Quan  H.  Comparison and validity of procedures coded with ICD-9-CM and ICD-10-CA/CCI.   Med Care. 2008;46(6):627-634. doi:10.1097/MLR.0b013e3181649439 PubMedGoogle ScholarCrossref
23.
Waikar  SS, Wald  R, Chertow  GM,  et al.  Validity of International Classification of Diseases, Ninth Revision, Clinical Modification codes for acute renal failure.   J Am Soc Nephrol. 2006;17(6):1688-1694. doi:10.1681/ASN.2006010073 PubMedGoogle ScholarCrossref
24.
Lee  DS, Donovan  L, Austin  PC,  et al.  Comparison of coding of heart failure and comorbidities in administrative and clinical data for use in outcomes research.   Med Care. 2005;43(2):182-188. doi:10.1097/00005650-200502000-00012 PubMedGoogle ScholarCrossref
25.
Austin  PC, Daly  PA, Tu  JV.  A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario.   Am Heart J. 2002;144(2):290-296. doi:10.1067/mhj.2002.123839 PubMedGoogle ScholarCrossref
26.
Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. doi:10.1016/j.cct.2006.05.007 PubMedGoogle ScholarCrossref
27.
Hemming  K, Taljaard  M.  Sample size calculations for stepped wedge and cluster randomised trials: a unified approach.   J Clin Epidemiol. 2016;69:137-146. doi:10.1016/j.jclinepi.2015.08.015 PubMedGoogle ScholarCrossref
28.
Hemming  K, Haines  TP, Chilton  PJ, Girling  AJ, Lilford  RJ.  The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting.   BMJ. 2015;350(1):h391-h391. doi:10.1136/bmj.h391PubMedGoogle ScholarCrossref
29.
Amin  AP, Crimmins-Reda  P, Miller  S,  et al.  Reducing acute kidney injury and costs of percutaneous coronary intervention by patient-centered, evidence-based contrast use.   Circ Cardiovasc Qual Outcomes. 2019;12(3):e004961. doi:10.1161/CIRCOUTCOMES.118.004961 PubMedGoogle ScholarCrossref
30.
Davis  DA, Thomson  MA, Oxman  AD, Haynes  RB.  Changing physician performance: a systematic review of the effect of continuing medical education strategies.   JAMA. 1995;274(9):700-705. doi:10.1001/jama.1995.03530090032018 PubMedGoogle ScholarCrossref
31.
Mansouri  M, Lockyer  J.  A meta-analysis of continuing medical education effectiveness.   J Contin Educ Health Prof. 2007;27(1):6-15. doi:10.1002/chp.88 PubMedGoogle ScholarCrossref
32.
Roshanov  PS, Fernandes  N, Wilczynski  JM,  et al.  Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials.   BMJ. 2013;346(1):f657-f657. doi:10.1136/bmj.f657PubMedGoogle ScholarCrossref
33.
Kawamoto  K, Houlihan  CA, Balas  EA, Lobach  DF.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success.   BMJ. 2005;330(7494):765. doi:10.1136/bmj.38398.500764.8F PubMedGoogle ScholarCrossref
34.
Brehaut  JC, Colquhoun  HL, Eva  KW,  et al.  Practice feedback interventions: 15 suggestions for optimizing effectiveness.   Ann Intern Med. 2016;164(6):435-441. doi:10.7326/M15-2248 PubMedGoogle ScholarCrossref
35.
Ivers  N, Jamtvedt  G, Flottorp  S,  et al.  Audit and feedback: effects on professional practice and healthcare outcomes.   Cochrane Database Syst Rev. 2012;6(6):CD000259. doi:10.1002/14651858.CD000259.pub3PubMedGoogle ScholarCrossref
36.
Sargeant  J, Lockyer  J, Mann  K,  et al.  Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2).   Acad Med. 2015;90(12):1698-1706. doi:10.1097/ACM.0000000000000809 PubMedGoogle ScholarCrossref
37.
Carless  D, Salter  D, Yang  M, Lam  J.  Developing sustainable feedback practices.   Stud High Educ. 2010;36(4):395-407. doi:10.1080/03075071003642449 Google ScholarCrossref
38.
Overeem  K, Driessen  EW, Arah  OA, Lombarts  KM, Wollersheim  HC, Grol  RP.  Peer mentoring in doctor performance assessment: strategies, obstacles and benefits.   Med Educ. 2010;44(2):140-147. doi:10.1111/j.1365-2923.2009.03580.x PubMedGoogle ScholarCrossref
39.
Hysong  SJ.  Meta-analysis: audit and feedback features impact effectiveness on care quality.   Med Care. 2009;47(3):356-363. doi:10.1097/MLR.0b013e3181893f6b PubMedGoogle ScholarCrossref
40.
Nilsen  P.  Making sense of implementation theories, models and frameworks.   Implement Sci. 2015;10(1):53. doi:10.1186/s13012-015-0242-0 PubMedGoogle ScholarCrossref
×