High-Intensity vs Low-Intensity Knowledge Translation Interventions for Surgeons and Their Association With Process and Outcome Measures Among Patients Undergoing Rectal Cancer Surgery

IMPORTANCE Surgeon-directed knowledge translation (KT) interventions for rectal cancer surgery are designed to improve patient measures, such as rates of permanent colostomy and in-hospital mortality, and to improve survival. OBJECTIVE To evaluate the association of sustained, iterative, integrated KT rectal cancer surgery interventions directed at all surgeons with process and outcome measures among patients undergoing rectal cancer surgery in a geographic region. DESIGN, SETTING, AND PARTICIPANTS This quality improvement study used administrative data frompatientswhounderwentrectalcancersurgeryfromApril1,2004,toMarch31,2015,in14health regions in Ontario, Canada. Follow-up was completed on March 31, 2020. EXPOSURES Surgeons in 2 regions were offered intensive KT interventions, including annual workshops, audit and feedback sessions, and, in 1 of the 2 regions, operative demonstrations, from 2006 to 2012 (high-intensity KT group). Surgeons in the remaining 12 regions did not receive these interventions (low-intensity KT group). MAIN OUTCOMES AND MEASURES Among patients undergoing rectal cancer surgery, proportions of preoperative pelvic magnetic resonance imaging (MRI), preoperative radiotherapy, and type of surgery were evaluated, as were in-hospital mortality and overall survival. Logistic regression models with an interaction term between group and year were used to assess whether process measures and in-hospital mortality differed between groups over time. RESULTS total of 15683 patients male; whom Abstract(continued) high-intensity group and 20.0% to 18.3% in the low-intensity group) and in-hospital mortality (0.8% to 0.8% in the high-intensity group and 2.2% to 1.8% in the low-intensity group). Time trends were similar between groups for measures that did or did not change over time. Patient overall survival was similar between groups (hazard ratio, 1.00; 95% CI, 0.90-1.11; P = .99). CONCLUSIONS AND RELEVANCE In this quality improvement study, between-group differences were found in only 2 measures (permanent stoma and in-hospital mortality), but these differences were stable over time. High-intensity KT group interventions were not associated with improved patient measures and outcomes. Proper evaluation of KT or quality improvement interventions may help avoid opportunity costs associated with ineffective strategies.


Introduction
Knowledge translation (KT) interventions, such as audit and feedback, are used with the expectation of improving related health care quality. 1 However, most evaluations of KT interventions have demonstrated minimal association with targeted patient process or outcome measures. [2][3][4][5][6] In response, stakeholders suggest that KT interventions would be more effective with the use of integrated KT approaches; theory to plan, implement, and evaluate any KT strategy; and sustained and iterative efforts. [7][8][9][10][11] An integrated KT approach involves KT users and implementers working together to design, implement, and evaluate a study or an initiative. [7][8][9] Key theories (eg, social learning theory) that may affect clinician behavior were used to construct the Knowledge-to-Action (KTA) Cycle. 11 The KTA Cycle reinforces the importance of an iterative sustained effort to close quality gaps. Thus, for example, an ongoing annual audit and feedback intervention should be more effective than a single audit and feedback episode. Despite the intuitive appeal of these concepts, there has been little research at a population level that has tested associated assumptions.
Rectal cancer surgery is a useful model to test surgical quality improvement strategies because rectal cancer is a common cancer with serious clinical consequences for patients, and diagnostic, neoadjuvant treatment and surgical standards (ie, total mesorectal techniques) have changed markedly in the past 20 years. [12][13][14] Improving the quality of rectal cancer surgery should result in higher rates of magnetic resonance imaging (MRI) scans, optimal use of preoperative radiotherapy, optimal selection of surgical procedures, and improved outcomes such as survival. Our research group has implemented population-level KT strategies to optimize rectal cancer surgery. Our most recent effort incorporated integrated KT, theory, and a sustained iterative multiyear effort.
In preparation for an evaluation of the aforementioned strategy, KT experts reviewed data on rectal cancer surgery KT interventions implemented from 2006 to 2014 in each of 14 Ontario health regions. 15 Each region was given a score based on methods of intervention implementation (eg, integrated KT) and which interventions were implemented (eg, audit and feedback). Data were gathered using a structured interview guide that integrated an exhaustive list of KT or quality improvement interventions. Two regions had more progressive methods, a greater number of KT interventions, and high scores (mean, 78 of 100 [range, 73-83]), whereas 12 regions had low scores (mean, 30.5 of 100 [range, [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38]). For the present study, we labeled the former and latter regions as the high-intensity and low-intensity KT groups, respectively.
The substantive changes over time in rectal cancer surgery standards and the use of highintensity interventions in only 2 of Ontario's 14 health regions provided an opportunity to evaluate the association of population-level KT interventions with process and outcome measures among patients undergoing rectal cancer surgery. We hypothesized that relevant measures and trends for measures would be better among patients treated by surgeons in the high-intensity group than among patients treated by surgeons in the low-intensity KT group.

Study Groups and Patient Population
In a previous study, 15 KT 19,20 Surgeons and patients in these 2 regions constituted the exposed group and were labeled the high-intensity KT group. Front-line surgeons helped organize selection of measures for audit and feedback and planning of meetings (ie, integrated KT methods).
Interventions included audit and feedback of relevant measures, with feedback occurring at annual workshops (ie, KTA-Cycle-like approaches). In 1 high-intensity region, surgeons were also encouraged to invite surgeon researchers to assist with surgery at the home hospital of the participating surgeon (ie, an operative demonstration). In both regions, an ongoing explicit effort was made to improve patient outcomes at the region level. Engagement with any intervention by front- Patients were attached to the high-intensity or low-intensity region where their rectal cancer surgery occurred. The start date of April 1, 2004, ensured capture of cases before relevant activities in the high-intensity KT group. We excluded individuals who underwent a resection more than 2 months before or more than 8 months after the date of their diagnosis. Captured cases outside these 2 intervals likely reflected surgery with an unknown cancer diagnosis or surgery for palliative reasons, respectively; both scenarios could influence process and outcome measures. Other exclusion criteria included a hospital stay of less than 2 days, which would be too short for patient surgery and recovery, and procedures performed in facilities in which 3 or fewer resections for rectal cancer were performed annually for patient confidentiality reasons.

Patient and Hospital Variables
We determined patient age (grouped as Յ50, 51-65, 66-80, and Ն81 years), sex, area-based income quintile, and urban or rural dwelling. Area-based income was assessed at the level of dissemination areas based on individual postal code. We determined whether patients' surgery was elective or urgent and whether it resulted in an anastomosis (eg, low anterior resection), potentially reversible stoma (eg, low anterior resection and diverting ileostomy or Hartman resection), or permanent stoma (eg, abdominoperineal resection). We also captured TNM cancer stage at the time of surgery (data comprehensively available after 2007 only) and medical comorbidities present on the index admission using the Charlson Comorbidity Index. 21 In addition, we identified hospital teaching status (yes or no) and hospital procedure volume, grouped into low-, low-medium-, medium-high-, or high-volume institutions. All Ontario hospitals were first ranked and then placed into volume categories, with each group containing approximately equal numbers of patients.

Process and Outcome Measures
Process measures of interest were pelvic MRI 5 months or less before surgery, preoperative radiotherapy administered 5 months or less before surgery, postoperative radiotherapy use 6 months or less after surgery, and type of surgery (resection with anastomosis, resection with reversible stoma, and resection with permanent stoma). Time intervals ensured processes were associated with incident cases of rectal cancer surgery. Preoperative pelvic MRI is accepted as the best test to determine whether a patient has a stage II or III tumor and is thus a candidate for preoperative radiotherapy. 13 The use of radiotherapy lowers the risk of local tumor recurrence but also confers a risk of serious adverse effects, including bowel and sexual dysfunction. 22,23 Because the benefits and risks of radiotherapy must be balanced, in the absence of a criterion standard rate, a priori we did not know whether superior use of radiotherapy at a group level would be associated with a higher or lower proportion of patients receiving radiotherapy. Of note, preoperative radiotherapy is associated with lower rates of local recurrence than postoperative radiotherapy. 24 Most patients would prefer to avoid a permanent stoma and instead receive an anastomosis (ie, reconnection of the bowel). 13 A guideline 13 recommends that patients with a low anastomosis receive a potentially reversible stoma (eg, ileostomy) to decrease the long-term risk of a permanent stoma despite closure of the ileostomy requiring an additional surgery. Outcomes of interest were rates of in-hospital mortality and overall survival among patients discharged from the hospital.

Statistical Analysis
We used 2-tailed t and χ 2 tests for between-group comparisons of patient and hospital characteristics, process measures, and in-hospital mortality. We used logistic regression models with an interaction term between group and year to determine whether time trends process measures and in-hospital mortality differed between groups. We used multivariable hierarchical logistic regression models to examine the association of study group with measures after adjustment for patient age, sex, income, rural dwelling, admission category, the presence of comorbidities, cancer type (rectosigmoid or rectal), hospital volume group, hospital teaching status, and year of surgery.
We used generalized estimating equations with robust SEs to account for clustering of patients within hospitals in all models. To examine the association of group with survival, we used a multilevel Cox proportional hazards regression model adjusting for the aforementioned covariates. This model used a frailty approach to account for patient clustering within hospitals. Patients were followed up until death or March 31, 2020. We used adjusted hierarchical logistic regression models to determine potential differences between groups over time in slopes. As a sensitivity analysis, we examined the effects of adding cancer stage as a covariate to all logistic and Cox proportional hazards regression patients owing to 5451 not having a corresponding hospital record, 5200 not having a relevant resection code, and 1400 meeting the remaining exclusion criteria (Figure 1) or direction of the association for any process measure, in-hospital mortality, or overall survival.

JAMA Network Open | Surgery
Over time, marked increases occurred in the use of MRI, preoperative radiotherapy, and temporary stoma (Figure 2). For example, the proportion of preoperative MRI increased from approximately 6.3% to 67.1%, the proportion of preoperative radiotherapy increased from 16.5% to  44.7%, and the proportion of temporary stoma increased from 26.9% to 42.0%. In contrast, the proportion of permanent stoma (25.4% to 25.3% in the high-intensity group and 20.0% to 18.3% in the low-intensity group) and in-hospital mortality (0.8% to 0.8% in the high-intensity group and 2.2% to 1.8% in the low-intensity group) remained similar ( Figure 2). There were no significant time trend differences between groups for measures that did or did not change over time. In multivariable models controlling for group, changes in slope were significant for MRI (0.33; P < .001), preoperative radiotherapy (0.11; P < .001), and temporary stoma (0.08; P < .001) and were not significant for permanent stoma (0.0002; P = .98) and in-hospital mortality (−0.01; P = .64).

Discussion
In this quality improvement study, we found that the resource-intense methods and interventions used in the high-intensity KT regions were not associated with improved patient measures. In multivariable models, there were no significant differences between groups in most measures, including survival. There were marked changes in proportions over time for use of MRI, preoperative radiotherapy, and temporary stoma. However, changes occurred in parallel for patients in the highintensity and low-intensity KT groups. There were modest differences in proportions between groups for permanent stoma and in-hospital mortality. However, absolute values and between-group differences for these 2 measures did not change significantly over time.     differences between groups, P = .54 for preoperative magnetic resonance imaging (MRI); P = .17 for preoperative radiotherapy; P = .99 for postoperative radiotherapy; P = .46 for permanent stoma; and P = .68 for potentially reversible stoma. C, P = .64 for slope; P = .43 for differences between groups over time.

Limitations
This study has limitations. First, this was a quality improvement study using administrative data, which may not comprehensively capture relevant variables and measures or important case mix differences between groups. However, the databases used in this study are accurate for key data points, such as patient characteristics, procedures provided, and mortality status, limiting potential confounding. 17,34,35 Our main results are also based on multivariable models. In addition, previous research 36 showed that patients with rectal cancer were frequently treated in their health region of origin. Patient and tumor characteristics are unlikely to vary markedly among regions, and this was supported by our results. 36

Conclusions
In this quality improvement study, high-intensity KT methods and interventions offered to all surgeons in 2 of 14 health regions in Ontario from 2006 to 2012 were not associated with improvements in patient process or outcome measures. Proper evaluation of KT or quality improvement interventions may help avoid opportunity costs associated with ineffective strategies.