Can an individualized audit and feedback intervention reduce musculoskeletal diagnostic imaging usage among high-requesting general practitioners in Australia?
In this cluster-randomized clinical trial that included 3819 Australian general practitioners from 2271 practices, an intervention that involved written individualized feedback and ordering rate comparisons with peers significantly decreased the overall rate of requests for any of 11 targeted imaging tests over 12 months compared with a control group that did not receive feedback (adjusted mean rate, 27.7 vs 30.4 requests per 1000 consultations; adjusted mean difference, −2.66 per 1000 consultations).
Among Australian high-requesting general practitioners, an individualized audit and feedback intervention significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
Audit and feedback can improve professional practice, but few trials have evaluated its effectiveness in reducing potential overuse of musculoskeletal diagnostic imaging in general practice.
To evaluate the effectiveness of audit and feedback for reducing musculoskeletal imaging by high-requesting Australian general practitioners (GPs).
Design, Setting, and Participants
This factorial cluster-randomized clinical trial included 2271 general practices with at least 1 GP who was in the top 20% of referrers for 11 imaging tests (of the lumbosacral or cervical spine, shoulder, hip, knee, and ankle/hind foot) and for at least 4 individual tests between January and December 2018. Only high-requesting GPs within participating practices were included. The trial was conducted between November 2019 and May 2021, with final follow-up on May 8, 2021.
Eligible practices were randomized in a 1:1:1:1:1 ratio to 1 of 4 different individualized written audit and feedback interventions (n = 3055 GPs) that varied factorially by (1) frequency of feedback (once vs twice) and (2) visual display (standard vs enhanced display highlighting highly requested tests) or to a control condition of no intervention (n = 764 GPs). Participants were not masked.
Main Outcomes and Measures
The primary outcome was the overall rate of requests for the 11 targeted imaging tests per 1000 patient consultations over 12 months, assessed using routinely collected administrative data. Primary analyses included all randomized GPs who had at least 1 patient consultation during the study period and were performed by statisticians masked to group allocation.
A total of 3819 high-requesting GPs from 2271 practices were randomized, and 3660 GPs (95.8%; n = 727 control, n = 2933 intervention) were included in the primary analysis. Audit and feedback led to a statistically significant reduction in the overall rate of imaging requests per 1000 consultations compared with control over 12 months (adjusted mean, 27.7 [95% CI, 27.5-28.0] vs 30.4 [95% CI, 29.8-30.9], respectively; adjusted mean difference, −2.66 [95% CI, −3.24 to −2.07]; P < .001).
Conclusions and Relevance
Among Australian general practitioners known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
ANZCTR Identifier: ACTRN12619001503112
Regional musculoskeletal conditions such as low back, neck, shoulder, hip, knee, and foot pain were common and significant contributors to global disability between 1990 and 2019.1,2 Evidence-based primary care guidelines for many of these conditions consistently recommend against early diagnostic imaging in the absence of features suggestive of serious and/or specific underlying cause or unexplained symptom progression.3 Unwarranted imaging does not benefit patients, may cause harm and/or lead to further unnecessary care, and wastes health resources that could be better used elsewhere.4 Yet overuse of diagnostic imaging by general practitioners (GPs) for people with musculoskeletal conditions has persisted.5-9
Audit and feedback is a widely used strategy for improving professional practice.10 The ability of audit and feedback used at scale to reduce musculoskeletal imaging by GPs is unclear, as available trials have included small numbers of practices and GPs.11-15 The primary objective of this trial was to evaluate the effectiveness of audit and feedback used at a national level for reducing requests for 11 commonly overused musculoskeletal diagnostic imaging tests by high-requesting Australian GPs compared with a control condition of no intervention. A secondary objective was to evaluate which forms of audit and feedback were most effective.
This was a 5-group factorial cluster-randomized clinical trial conducted within the context of Australian general clinical practice. A cluster design was used to minimize crossover of intervention effects within practices and a no-intervention control condition tested the effectiveness of any form of audit and feedback. A 2 × 2 factorial design was used to simultaneously assess the effect of 2 factors: (1) frequency of feedback delivery (once vs twice) and (2) enhanced visual display (no vs yes). The trial used routinely collected Australian Medicare Benefits Schedule (MBS) data to identify the study population, apply eligibility criteria, generate individualized feedback for the interventions, and source trial outcomes. Medicare is Australia’s universal health insurance scheme that guarantees all Australian residents access to a range of health care services by physicians and other health care professionals at low or no cost. The Bond University Human Research Ethics Committee (LA03323) approved the study and waived participant consent, as the research carried no more than low risk to participants, there was adequate protection of participants’ privacy, and obtaining consent would create a burden greater than the intervention itself.16 Participants were not aware they were in a trial. The trial protocol and statistical analysis plan are available in Supplement 1.
General practices were included if (1) at least 1 GP within the practice was in the top 20% of Australian GP referrers for 11 targeted musculoskeletal imaging tests overall, and for at least 4 individual tests, within the 12-month period from January 1 to December 31, 2018, according to Australian MBS administrative data and (2) the practice was located in an Australian state or territory. The MBS database is comprehensive and records all diagnostic imaging that has been requested by GPs, rendered by a radiologist, and claimed from Medicare. The 11 targeted imaging tests were lumbosacral spine x-ray and computed tomography (CT); cervical spine x-ray, CT, and magnetic resonance imaging (MRI); shoulder x-ray and ultrasound; hip x-ray and ultrasound; knee x-ray; and ankle/hind foot ultrasound. These imaging tests were selected for inclusion by the research team in consultation with the Australian Government Department of Health and Aged Care medical advisors and reflect tests that are generally considered overused in primary care. Only high-requesting GPs (ie, those in the top 20% of GP referrers for the targeted tests overall and for at least 4 individual tests within the specified 12-month period) practicing within one of the participating practices were included.
General practitioners were excluded if they (1) did not make any requests for the 11 targeted tests between January 1 and December 31, 2018; (2) had less than 1000 patient consultations over the specified 12-month period; (3) participated in user testing that informed intervention development; or (4) participated in an Australian government compliance activity between January 1, 2018, and October 25, 2019.
Randomization and Masking
General practices with at least 1 eligible high-requesting GP, clustered based on exact-matched primary practice addresses, were randomized on a single occasion to 1 of 4 interventions or a no-intervention control group in a 1:1:1:1:1 ratio by a statistician (B.M.) using the “cluster_ra” procedure in R statistical software, stratified by geographic region (metropolitan and regional/rural/remote).
Due to the nature of the audit and feedback intervention, participating GPs were not masked to group allocation; however, the risk of bias was considered to be minimal as GPs were not aware of the variations of audit and feedback being tested or outcome measures and analytical approach. The statistical analysis plan was developed by the senior trial statistician (L.B.), who was masked to group allocation (see Supplement 1).17 Analyses were independently conducted by 2 statisticians (B.M. and A.G.) using randomly shuffled group allocations. Real allocations were only revealed once analyses were completed and agreement between statisticians was reached.
General practitioners allocated to the control group did not receive any audit and feedback on their musculoskeletal imaging requests.
General practitioners allocated to 1 of 4 intervention groups received individualized written audit and feedback on their imaging requesting rates in a letter from the chief medical officer of Australia (eAppendix 1 in Supplement 2). The audit and feedback provided to all intervention groups was consistent in content and length other than frequency and visual display, the potential effect modifiers evaluated in the trial. The letters were delivered by mail and graphically displayed recipients’ overall request rate for 11 targeted imaging tests, and tabular data for individual test request rates, per 1000 consultations, each compared with the median rate of their peers in the same geographic stratum. The letter described the benefits of tackling imaging overuse, provided links to educational resources, and encouraged recipients to reflect on the feedback and limit imaging requests to situations in which it had a reasonable likelihood of changing a patient’s management plan.
Factor 1: Frequency of Feedback
Two groups received feedback on 1 occasion (November 8, 2019), the current standard approach of the Australian government, based on GPs’ January to December 2018 audit data. Another 2 groups received feedback on 2 occasions (November 8, 2019, and November 9, 2020) to investigate the potential for increased effects; delivery of the second feedback letter provided updated requesting data for the period November 8, 2019, to March 7, 2020, a shorter period given the effect of COVID-19 on health care practice.
Factor 2: Inclusion of Enhanced Visual Display
Two groups received standard feedback display (request rates for all 11 tests without highlighting imaging types with rates >80% of peers), while 2 groups received enhanced display (use of yellow shading to direct recipients’ attention to imaging for which ordering rates were >80% of peers, suggesting behavior change may be required) (eAppendix 1 in Supplement 2).
The interventions were user tested and refined with GPs prior to evaluation.
The primary outcome was the overall rate of requests for any of the 11 targeted musculoskeletal diagnostic imaging tests for each GP over 12 months, with rates over 6 and 18 months considered secondary outcomes. The request rates for the 11 individual imaging tests plus ankle x-ray for each GP over 6, 12, and 18 months were secondary outcomes. Ankle x-ray was not targeted by the audit and feedback but was considered a possible substitute for ankle/hind foot ultrasound. All outcomes are expressed as requests per 1000 patient consultations.
The following baseline data were also collected: GPs’ age, sex, years of practice, geographical location of primary practice (metropolitan vs regional/rural/remote), and rates of musculoskeletal diagnostic imaging requests for primary and secondary outcomes at baseline (November 1, 2018, to October 31, 2019).
A total of 2271 unique general practices across Australia were identified as eligible for inclusion in the study. Using all eligible practices and a 4-to-1 randomization ratio, the sample size provided greater than 90% power to detect a mean difference as small as 1.1 requests per 1000 consultations in the rate of imaging requests. This calculation assumed a standard deviation of 7 as observed in a previous data extract and a 2-sided type I error rate of 5%. It conservatively assumed that each cluster would be of size 1 (ie, a single GP per practice), thus negating the need to adjust for intracluster correlation. Given that several practices included 2 or more GPs with cluster sizes ranging from 1 to 12 (median, 1 [IQR, 1-2], equal between groups), this approach guaranteed at least 90% power regardless of the intracluster correlation.
This sample size also provided greater than 90% power to detect a difference of 1.1 requests per 1000 consultations in the mean rate of imaging requests between those receiving feedback on 1 vs 2 occasions and between those receiving feedback with vs without enhanced feedback display, assuming a 2-sided α = .025 for each of these secondary comparisons to control for multiplicity.
Based on the trial protocol and statistical analysis plan, the main analysis was to include all randomized GPs and analyze them according to the group to which they were randomized, regardless of the number of patient consultations conducted. However, during data analysis, it was discovered that a limited number of GPs did not have any patient consultations during the follow-up period and could not be included in the main analysis. Therefore, the primary analysis set consisted of all GPs with at least 1 patient consultation over the relevant analysis period. Given that MBS data systematically capture all consultations, no missing outcome data were assumed.
The main analysis compared the mean rate of diagnostic imaging requests per 1000 patient consultations between the 4 audit and feedback intervention groups, considered as 1 group, with the control group. Data were aggregated at the GP level and analyzed using multilevel mixed-effects linear regression adjusted for the baseline imaging rate of each GP and regional/rural/remote location. All assumptions for multivariable regression analysis, including lack of multicollinearity, independence of observations, and multivariable normality, were met. To remove skewness and potential heteroskedasticity, a natural log transformation was applied to the 6-, 12-, and 18-month rates as well as to the baseline rate included as a covariate. Resulting estimates and confidence intervals were back-transformed to the original scale. Clustering of GPs by practice was accounted for by including a random intercept by practice. The effect of the intervention was estimated as the adjusted mean difference in the overall rate of musculoskeletal imaging requests between the intervention and the control groups with its 95% CI. The same mixed-effects linear regression model was used to estimate the effect of the number of audit and feedback occasions (1 vs 2) and the effect of the enhanced feedback display (no vs yes) by adding terms for each intervention as well as their interaction. The effect of each intervention was estimated using adjusted mean differences and 97.5% CIs to account for multiple comparisons. A similar approach was used to analyze the secondary outcomes. All analyses were repeated using an adjusted model, controlling for GP demographic characteristics (age, sex, and years of practice) and baseline volume of imaging requests (eAppendix 2 in Supplement 2).
Shifts in percentage ranks over time for the main comparison (intervention vs control) and according to the 4 intervention groups for the primary outcome were investigated using Sankey diagrams using 4 time points (baseline, month 6, month 12, and month 18), categorizing ranks into 3 categories (lowest: <80th percentile; medium: 80th to <90th percentile; high: ≥90th percentile).
A limited number of prespecified subgroup analyses were conducted to identify potential differences in intervention effects for factors documented in the analysis plan, such as geographical location (metropolitan vs regional/rural/remote), years practicing as a GP (0 to <5 years, 5 to <10 years, 10 to <15 years, or 15 or more years), sex, baseline overall percentile ranking (80th to <90th or 90th to 99th), and baseline volume of imaging tests with requesting percentile at least 80th (4-6 vs 7-11). Subgroup analyses were conducted by adding the subgroup variable of interest as well as its interaction with the randomized group to the main analysis model.
Additionally, prespecified sensitivity analyses were conducted to test whether the effect of the intervention remained stable under various conditions: (1) including only those who completed the trial per protocol, ie, excluding those whose mail returned to sender unopened or who requested not to receive a second audit and feedback letter and (2) including only those who had at least 1500 patient consultations within the 18-month trial duration. An additional post hoc sensitivity analysis was undertaken excluding practices randomized into 2 study groups due to an administrative error in practice locality classification, to assess the effect of practices misclassified in both strata (ie, metropolitan, nonmetropolitan [regional/rural/remote]) leading to GPs within the same practice having a chance of being randomized to different study groups. A post hoc estimate of the reduction in diagnostic imaging requests as a result of the audit and feedback intervention was calculated with its 95% CI based on the difference between the observed number of imaging requests made by GPs allocated to audit and feedback and expected number of imaging requests made by these GPs in absence of the intervention, based on regression model estimates. For the main analysis, all practices were included with GPs analyzed according to their allocated group.
All analyses were performed using Stata version 16 (StataCorp LLC) and R version 4.0.0 (R Foundation). Level of significance was set at P < .05 using a 2-sided test for the primary comparison and a Bonferroni-corrected P < .025 to compare 1 vs 2 feedback occasions and feedback with vs without enhanced display. We did not formally adjust for multiplicity across secondary outcomes; therefore, secondary outcome results should be interpreted as exploratory and as assisting the interpretation of the primary outcome results.
On November 8, 2019, 3819 high-requesting GPs from 2271 unique practices were simultaneously randomized to control or active intervention groups. Due to an administrative error in general practice locality classification prior to randomization, 395 practices were stratified into both metropolitan and nonmetropolitan areas. This resulted in some GPs from the same practice having a chance of being randomized to different study groups. Of the 395 affected practices, 307 were randomized to 2 groups (111 practices to both control and intervention, with potential for crossover of 336 GPs in these practices; and 196 practices to 2 different intervention groups, with potential for crossover of 647 GPs); 88 practices were randomized into the same group.
Overall, 533 practices (n = 764 GPs) were randomized to control and 2133 practices (n = 3055 GPs) were randomized to active interventions (Figure 1). For the main analysis, all practices were included with GPs analyzed according to their allocated group. Requesting rates for 159 GPs (4.2%) who did not have any patient consultations during the trial period could not be calculated. The main analysis therefore included 3660 GPs from 2578 practices. The geographic location, sex, and baseline imaging request rates between GPs included in and excluded from the main analysis were comparable, yet excluded GPs were older and had more years in practice than included GPs (eTable 1 in Supplement 2). A per-protocol analysis included 3370 GPs from 2423 practices and excluded the 290 GPs who did not receive the allocated intervention (ranging from 37 [4.9%] to 112 [14.7%] per group) because their first audit and feedback letter was returned to sender unopened (n = 206) or there was no follow-up (n = 84), in addition to the 159 GPs with no patient consultations.
The characteristics of high-requesting GPs included in the trial were similar across all study groups at baseline (Table 1). The median overall rate of musculoskeletal imaging at baseline was 32.3 (IQR, 27.3-38.1) per 1000 patient consultations and was similar across groups. Request rates for individual imaging tests were also similar across groups at baseline (eTable 2 in Supplement 2).
The overall rate of musculoskeletal imaging requests by GPs following the audit and feedback intervention was statistically significantly lower than in the control group over 12 months despite substantial reductions in imaging requests across all groups during the COVID-19 pandemic (Figure 2 and Table 2; eFigure 1 in Supplement 2). The mean overall rate of imaging requests per 1000 consultations was 27.7 (95% CI, 27.5-28.0) with the intervention vs 30.4 (95% CI, 29.8-30.9) in the control group, for an adjusted mean difference of −2.66 requests (95% CI, −3.24 to −2.07; P < .001).
Audit and feedback also statistically significantly reduced the overall request rate compared with control over 6 months (mean, 24.8 [95% CI, 24.5-25.0] vs 27.6 [95% CI, 27.1-28.1] requests per 1000 consultations, respectively; adjusted mean difference, −2.81 [95% CI, −3.36 to −2.26]; P < .001) and 18 months (mean, 26.2 [95% CI, 26.0-26.4] vs 28.3 [95% CI, 27.8-28.8] requests per 1000 consultations, respectively; adjusted mean difference, −2.11 [95% CI, −2.63 to −1.58]; P < .001). An estimated 47 318 (95% CI, 35 562 to 59 074) fewer diagnostic imaging tests were requested due to any audit and feedback intervention over the study period.
Audit and feedback on 2 occasions (at baseline and 12 months later) statistically significantly reduced the overall request rate per 1000 consultations at 18 months vs audit and feedback on 1 occasion (mean, 25.8 [95% CI, 25.4-26.2] vs 26.6 [95% CI, 26.2-27.0], respectively; adjusted mean difference, −0.79 [97.5% CI, −1.30 to −0.26]; P < .001).
Enhanced display significantly reduced the overall request rate per 1000 consultations vs standard display over 6 months (mean, 24.4 [95% CI, 24.0-25.1] vs 25.2 [95% CI, 24.8-25.5], respectively; adjusted mean difference, −0.79 [97.5% CI, −1.35 to −0.24]; P = .001) and 12 months (mean, 27.4 [95% CI, 26.9-27.8] vs 28.1 [95% CI, 27.7-28.5], respectively; adjusted mean difference, −0.70 [97.5% CI, −1.28 to −0.11]; P = .008) but not over 18 months (mean, 26.0 [95% CI, 25.6-26.4] vs 26.4 [95% CI, 25.9-26.7], respectively; adjusted mean difference, −0.36 [97.5% CI, −0.90 to 0.17]; P = .13). There was no statistically significant interaction between number of occasions and type of display (P = .51). Relative rate reductions in overall diagnostic imaging rates at each time point are presented in eFigure 2 in Supplement 2 (see also eTable 3 in Supplement 2).
Audit and feedback statistically significantly reduced the request rate per 1000 consultations of all individual imaging tests compared with control over the first 6 months, with the greatest reductions observed for hip x-ray (adjusted mean difference, −0.65 [95% CI, −0.86 to −0.45]; P < .001), ankle x-ray (adjusted mean difference, −0.62 [95% CI, −0.78 to −0.46]; P < .001) and lumbosacral spine CT (adjusted mean difference, −0.47 [95% CI, −0.58 to −0.37]; P < .001) (eTable 4 in Supplement 2). Request rates remained statistically significantly lower compared with control for all but 2 individual imaging tests (lumbosacral and cervical spine x-ray) over 12 and 18 months (Table 3; eTable 4 in Supplement 2). Audit and feedback on 2 occasions statistically significantly reduced request rates of all but 3 individual imaging tests (lumbosacral spine, knee, and ankle x-ray) vs once-only delivery over 18 months (eTable 5 in Supplement 2). Enhanced display statistically significantly reduced request rates of all but 4 individual tests (lumbosacral spine x-ray and CT, cervical spine x-ray and MRI) vs standard display over 6 months. However, there was no statistically significant difference in request rates between groups over 12 and 18 months except for hip x-ray and shoulder, hip, and ankle ultrasound (eTable 6 in Supplement 2). The intracluster correlation for the primary outcome was 0.10 (95% CI, 0.06-0.16) and ranged between 0 and 0.15 for other outcomes (eTable 7 in Supplement 2).
Within the first 6 months of the intervention, 46.9% of GPs in the intervention group (767/1634) and 33.0% of those in the control group (133/403) shifted from the highest percentile ranking (90th to 99th) to medium (80th to <90th) or lowest (<80th) (P < .001) (Figure 2), while 57.8% of GPs in the intervention group (623/1078) and 41.5% in the control group (120/289) shifted from medium to lowest percentile ranking. At 12 months, the proportion of highest percentile ranking GPs in the intervention group was statistically significantly lower compared with control (39.9% [1169/2933] vs 46.4% [337/727]; P = .001). The difference between groups increased following the second mailing, reaching 36.6% (1073/2933) for intervention and 49.7% (361/727) for control (P < .001). Percentile rankings for the 4 intervention groups at each time point are presented in eFigure 3 in Supplement 2.
Subgroup and Sensitivity Analyses
Subgroup analyses showed no statistically significant heterogeneity in intervention effects based on geographical location, years practicing as a GP, sex, and baseline overall percentile ranking and baseline volume of imaging tests with requesting percentile at least 80th (4-6 vs 7-11) (eFigure 4 in Supplement 2).
Prespecified sensitivity analyses showed that there was little change in effect estimates for the main and adjusted models after restricting analyses to per-protocol or to active GPs (ie, excluding those with <1500 consultations during follow-up). Post hoc sensitivity analyses excluding practices inadvertently randomized into 2 study groups due to an administrative practice locality classification error also led to consistent results (eTable 8 in Supplement 2).
Among Australian GPs known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months in the context of substantial reductions in requests across all groups during the COVID-19 pandemic.
Audit and feedback also led to statistically significant reductions in overall imaging requests over 6 and 18 months, as well as for most individual imaging tests over 6, 12, and 18 months, compared with a control condition of no audit and feedback. A second round of feedback to GPs at 12 months led to a further modest statistically significant reduction in overall imaging requests over 18 months compared with once-only delivery. Enhanced display calling GPs’ attention to ordering rates greater than 80% of peers’ rates also resulted in further modest statistically significant reductions in overall imaging over 6 and 12 months but not 18 months.
Early in the COVID-19 pandemic, governments in Australia implemented a range of restrictions on health care delivery, greater access to telehealth services, and, in some jurisdictions, stay-at-home orders to limit virus transmission and protect hospital capacity. A substantial decline in imaging services was observed in the first few months of the pandemic and across all groups within the first 6 months of the trial but rebounded thereafter.
The favorable effects observed in the current study are at odds with results from 3 of 5 previous trials of audit and feedback for musculoskeletal diagnostic imaging.11-13 Robling et al13 found that providing audit and feedback accompanied by guidelines on 1 occasion to GPs in the UK did not lead to statistically significant improvements in guideline-concordant knee and lumbar spine MRI ordering compared with guideline dissemination alone. Kerry et al12 showed that providing audit and feedback to GPs in the UK on 1 occasion accompanied by guidelines on hip, knee, spine, chest, skull, and sinus imaging did not lead to a statistically significant reduction in the overall request rate compared with a no-intervention control (mean difference, 10; 95% CI, −1 to 21). Eccles et al11 showed that providing audit and feedback on 2 occasions over 12 months did not lead to statistically significant reductions in the volume or appropriateness of lumbar spine and knee x-ray requests by GPs in England and Scotland compared with guideline dissemination alone (absolute change in radiograph requests per 1000 patients, −0.07 [95% CI, −1.3 to 0.9] for lumbar spine and 0.04; [95% CI, −0.95 to 1.03] for knee x-ray requests; concordance odds ratios for lumbar spine, 0.75 [95% CI, 0.52-1.07] and knee, 0.82 [95% CI, 0.50-1.33]). However, the audit and feedback intervention evaluated in these trials was provided at the level of the GP practice, rather than individualized to the GP recipient as implemented in the current study, so this could explain differences in the observed effects. Furthermore, the interventions in the current study were user tested with GPs prior to evaluation and sent to GPs known to frequently request musculoskeletal diagnostic imaging by the most senior medical advisor in the Australian Government Department of Health and Aged Care, so this may have also contributed to the beneficial effects observed.
Two previous trials have reported benefit with more intensive audit and feedback interventions compared with control. Winkens et al15 showed that providing audit and feedback on requesting of cervical spine, lumbar spine, pelvis, knee, and ankle imaging to 79 family physicians in the Netherlands on 5 occasions over a 2-year period led to a statistically significant reduction in the proportion of inappropriate requests compared with control (effect estimate not reported). Schectman et al14 showed that providing audit and feedback as part of a multicomponent quality improvement strategy (including small-group education, guideline dissemination, patient educational materials, and individual follow-up visits) on 2 occasions led to a small, statistically significant improvement in guideline-concordant lumbar spine imaging compared with a no-intervention control (5.4% increase with intervention vs 2.7% reduction with control; P = .05).
The finding that providing audit and feedback on 2 occasions led to a statistically significant reduction in overall imaging requests compared with once-only delivery is in keeping with broader empirical and theoretical knowledge about audit and feedback10,18 and with theories suggesting audit and feedback is more likely to be effective when delivered on multiple occasions.19-21 To our knowledge, no prior randomized trial has compared different ways of displaying feedback to direct recipients’ attention to where action is needed.21 This study provides evidence in support of a simple design enhancement (ie, use of yellow shading to direct recipients’ attention to their high imaging requesting) that led to modest, statistically significant reductions in overall and individual imaging request rates compared with standard display, especially within the first 6 months of feedback.
It is unclear to what extent the observed effect of audit and feedback will be maintained over time and whether 1 or more additional interventions will be needed to sustain the effect. It is also not known whether audit and feedback interventions like that used in this study influence other related imaging behaviors or low-value care practices. Future research could explore these issues as well as investigate how audit and feedback can be further optimized to reduce overuse of musculoskeletal imaging.
This study has several limitations. First, the study assessed volume and not appropriateness of imaging, so it is uncertain whether high request rates represent inappropriate use in all instances. Second, while audit and feedback letters were mailed to GPs’ nominated mailing address and it was noted when letters were returned to sender unopened, delivery of the letters was not tracked and no attempts were made to locate alternative mailing addresses for returned letters. It is also not known whether the letters that were not returned were opened and read. Therefore, estimates from the main analysis are likely to be conservative. Third, during data cleaning, an error was identified in the locality classification of some practices, resulting in some GPs located within the same practice being randomized into different groups. While it is possible that GPs within these practices discussed their feedback, sensitivity analysis suggests that exclusion of affected practices resulted in minimal overall changes to inference.
Among Australian general practitioners known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
Corresponding Author: Denise A. O’Connor, PhD, Department of Epidemiology and Preventive Medicine, School of Public Health and Preventive Medicine, Monash University, and Monash-Cabrini Department of Musculoskeletal Health and Clinical Epidemiology, Cabrini Health, 553 St Kilda Rd, Melbourne, VIC 3004, Australia (email@example.com).
Accepted for Publication: August 4, 2022.
Author Contributions: Mss Maguire and Gorelik had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: O’Connor, Glasziou, Maher, McCaffery, Schram, Maguire, Billot, Traeger, Albarqouni, Checketts, Vyas, Buchbinder.
Acquisition, analysis, or interpretation of data: O’Connor, Glasziou, Maher, Schram, Maguire, Ma, Billot, Gorelik, Traeger, Checketts, Vyas, Clark, Buchbinder.
Drafting of the manuscript: O’Connor, Gorelik.
Critical revision of the manuscript for important intellectual content: O’Connor, Glasziou, Maher, McCaffery, Schram, Maguire, Ma, Billot, Gorelik, Traeger, Albarqouni, Checketts, Vyas, Clark, Buchbinder.
Statistical analysis: O’Connor, Maguire, Billot, Gorelik.
Obtained funding: Schram, Glasziou, Maher, McCaffery, Buchbinder.
Administrative, technical, or material support: Glasziou, Schram, Ma, Albarqouni, Clark.
Supervision: O’Connor, Glasziou, Billot.
Conflict of Interest Disclosures: Dr Maher reported receipt of nonfinancial support from FlexEze. No other disclosures were reported.
Funding/Support: This study was funded by the Australian Government Department of Health and Aged Care and supported by Australian National Health and Medical Research Council (NHMRC) Program Grant APP1113532. Dr O’Connor is supported by an Australian NHMRC Translating Research Into Practice (TRIP) Fellowship (grant APP1168749). Dr Glasziou is supported by an Australian NHMRC Investigator Fellowship (grant APP1155009). Dr Maher is supported by an Australian NHMRC Investigator Fellowship (grant APP1194283). Dr Traeger is supported by an Australian NHMRC Early Career Fellowship (grant APP1144026). Dr Buchbinder is supported by an Australian NHMRC Investigator Fellowship (grant APP1194483).
Role of the Funder/Sponsors: Six members of the study team are employed by the Australian Government Department of Health and Aged Care and contributed to design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication. The sponsor did not have the right to veto publication or control the decision regarding to which journal the manuscript was submitted.
Data Sharing Statement: See Supplement 3.
Additional Contributions: We acknowledge the GPs who participated in user testing of the interventions and thank them for their time and feedback.
Additional Information: This trial was a collaboration between the Australian Government Department of Health and Aged Care and Wiser Healthcare, a collaboration of researchers investigating the causes of and solutions to overtesting, overdiagnosis, and overtreatment in health care.
GBD 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet
. 2020;396(10258):1204-1222. doi:10.1016/S0140-6736(20)30925-9PubMedGoogle ScholarCrossref
et al. Prevalence, deaths, and disability-adjusted life years due to musculoskeletal disorders for 195 countries and territories 1990-2017. Arthritis Rheumatol
. 2021;73(4):702-714. doi:10.1002/art.41571PubMedGoogle ScholarCrossref
et al. What does best practice care for musculoskeletal pain look like? eleven consistent recommendations from high-quality clinical practice guidelines: systematic review. Br J Sports Med
. 2020;54(2):79-86. doi:10.1136/bjsports-2018-099878PubMedGoogle ScholarCrossref
et al. How common is imaging for low back pain in primary and emergency care? systematic review and meta-analysis of over 4 million imaging requests across 21 years. Br J Sports Med
. 2020;54(11):642-651. doi:10.1136/bjsports-2018-100087PubMedGoogle ScholarCrossref
et al. EVOLVE: the Australian Rheumatology Association’s “top five” list of investigations and interventions doctors and patients should question. Intern Med J
. 2018;48(2):135-143. doi:10.1111/imj.13654PubMedGoogle ScholarCrossref
J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on x-ray referrals from general practice: a randomized controlled trial. Fam Pract
. 2000;17(1):46-52. doi:10.1093/fampra/17.1.46PubMedGoogle ScholarCrossref
et al. General practitioners’ use of magnetic resonance imaging: an open randomized trial comparing telephone and written requests and an open randomized controlled trial of different methods of local guideline dissemination. Clin Radiol
. 2002;57(5):402-407. doi:10.1053/crad.2001.0864PubMedGoogle ScholarCrossref
National Health and Medical Research Council, Australian Research Council, Universities Australia. National Statement on Ethical Conduct in Human Research. National Health and Medical Research Council; 2007. Updated 2018.
et al. Evaluating an audit and feedback intervention for reducing musculoskeletal diagnostic imaging requests by Australian general practitioners: statistical analysis plan for a partial cluster randomised trial. OSF Preprints
. Preprint posted August 6, 2021. doi:10.31219/osf.io/prnd2
et al. Growing literature, stagnant science? systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med
. 2014;29(11):1534-1541. doi:10.1007/s11606-014-2913-yPubMedGoogle ScholarCrossref
et al. Clinical performance feedback intervention theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci
. 2019;14(1):40. doi:10.1186/s13012-019-0883-5PubMedGoogle ScholarCrossref