[Skip to Content]
[Skip to Content Landing]
Figure 1.
Project Timeline at 4 Hospitals in the Netherlands, August 1, 2016, Through April 30, 2018
Project Timeline at 4 Hospitals in the Netherlands, August 1, 2016, Through April 30, 2018
Figure 2.
Slopes and Differences for Laboratory Test Volume Before and After the Start of the Interventiona
Slopes and Differences for Laboratory Test Volume Before and After the Start of the Interventiona

aThe intervention included creating awareness through education and feedback, intensified supervision of residents, and changes in ordering entry systems.

Table 1.  
Characteristics of Participating Departments at Initiation of the Project
Characteristics of Participating Departments at Initiation of the Project
Table 2.  
Changes in Volumes and Costs for Diagnostics Following the Interventiona
Changes in Volumes and Costs for Diagnostics Following the Interventiona
Table 3.  
Clinical Outcomes
Clinical Outcomes
1.
National Institute for Health and Care Excellence. NICE Do not Do prompts. https://www.nice.org.uk/sharedlearning/nice-do-not-do-prompts. Published December 2013. Accessed June 16, 2019.
2.
American Board of Internal Medicine Foundation. Choosing Wisely. http://www.choosingwisely.org/. Accessed October 24, 2018.
3.
Levinson  W, Kallewaard  M, Bhatia  RS, Wolfson  D, Shortt  S, Kerr  EA; Choosing Wisely International Working Group.  ‘Choosing Wisely’: a growing international campaign.  BMJ Qual Saf. 2015;24(2):167-174. doi:10.1136/bmjqs-2014-003821PubMedGoogle ScholarCrossref
4.
Zhi  M, Ding  EL, Theisen-Toupal  J, Whelan  J, Arnaout  R.  The landscape of inappropriate laboratory testing: a 15-year meta-analysis.  PLoS One. 2013;8(11):e78962. doi:10.1371/journal.pone.0078962PubMedGoogle ScholarCrossref
5.
Geleris  JD, Shih  G, Logio  L.  Analysis of diagnostic test ordering habits among internal medicine residents.  JAMA Intern Med. 2018;178(12):1719-1721. doi:10.1001/jamainternmed.2018.3519PubMedGoogle ScholarCrossref
6.
Kobewka  DM, Ronksley  PE, McKay  JA, Forster  AJ, van Walraven  C.  Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review.  Clin Chem Lab Med. 2015;53(2):157-183. doi:10.1515/cclm-2014-0778PubMedGoogle ScholarCrossref
7.
Solomon  DH, Hashimoto  H, Daltroy  L, Liang  MH.  Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework.  JAMA. 1998;280(23):2020-2027. doi:10.1001/jama.280.23.2020PubMedGoogle ScholarCrossref
8.
Bindraban  RS, ten Berg  MJ, Naaktgeboren  CA, Kramer  MHH, van Solinge  WW, Nanayakkara  PWB.  Reducing test utilization in hospital settings: a narrative review.  Ann Lab Med. 2018;38(5):402-412. doi:10.3343/alm.2018.38.5.402PubMedGoogle ScholarCrossref
9.
van Walraven  C, Naylor  CD.  Do we know what inappropriate laboratory utilization is? a systematic review of laboratory clinical audits.  JAMA. 1998;280(6):550-558. doi:10.1001/jama.280.6.550PubMedGoogle ScholarCrossref
10.
Vegting  IL, van Beneden  M, Kramer  MH, Thijs  A, Kostense  PJ, Nanayakkara  PW.  How to save costs by reducing unnecessary testing: lean thinking in clinical practice.  Eur J Intern Med. 2012;23(1):70-75. doi:10.1016/j.ejim.2011.07.003PubMedGoogle ScholarCrossref
11.
Bindraban  RS, van Beneden  M, ten Berg  MJ, Nanayakkara  PWB.  Long-term sustainability of a multi-step intervention to reduce unnecessary diagnostic testing.  Eur J Intern Med. 2017;44:e38-e39. doi:10.1016/j.ejim.2017.07.025PubMedGoogle ScholarCrossref
12.
Nederlandse Federatie van UMC. To do or not to do? [in Dutch]. https://www.doenoflaten.nl/. Accessed October 24, 2018.
13.
Bindraban  RS, van Beneden  ML, Kramer  MH,  et al.  A multicenter before-after study on reducing unnecessary diagnostics by changing the attitude of caregivers: protocol for the RODEO Project.  JMIR Res Protoc. 2018;7(8):e10473. doi:10.2196/10473PubMedGoogle ScholarCrossref
14.
Basisset Medisch Specialistische Zorg Kwaliteitsindicatoren.  Inspectie voor de Gezondheidszorg. Utrecht, the Netherlands: Ministerie van Volksgezondheid, Welzijn en Sport; 2017.
15.
Performation. https://performation.com/. Accessed October 24, 2018.
16.
Grol  R, Wensing  M.  What drives change? barriers to and incentives for achieving evidence-based practice.  Med J Aust. 2004;180(6)(suppl):S57-S60.PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,312
    Original Investigation
    Pathology and Laboratory Medicine
    July 24, 2019

    Association of a Multifaceted Intervention With Ordering of Unnecessary Laboratory Tests Among Caregivers in Internal Medicine Departments

    Author Affiliations
    • 1Section of Acute Medicine, Department of Internal Medicine, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
    • 2Department of Clinical Chemistry and Hematology, University Medical Center Utrecht, Utrecht, the Netherlands
    • 3Department of Epidemiology and Biostatistics, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
    • 4Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
    • 5Department of Internal Medicine, Zaans Medical Center, Zaandam, the Netherlands
    • 6Department of Clinical Pharmacy, Zaans Medical Center, Zaandam, the Netherlands
    • 7Department of Internal Medicine, North-West Hospital Group, Alkmaar, the Netherlands
    • 8Department of Internal Medicine, Spaarne Gasthuis, Haarlem, the Netherlands
    • 9Department of Internal Medicine, Spaarne Gasthuis, Hoofddorp, the Netherlands
    • 10Atalmedial Medical Diagnostics Centers, Hoofddorp, the Netherlands
    • 11Department of Internal Medicine, Meander Medical Center, Amersfoort, the Netherlands
    • 12Department of Clinical Chemistry, Meander Medical Center, Amersfoort, the Netherlands
    JAMA Netw Open. 2019;2(7):e197577. doi:10.1001/jamanetworkopen.2019.7577
    Key Points español 中文 (chinese)

    Question  What is the association of a multifaceted intervention aimed at changing the mindset of caregivers with the amount of unnecessary laboratory testing?

    Findings  In this before-after quality improvement study conducted in the internal medicine departments of 4 large teaching hospitals in the Netherlands, the volume of laboratory tests ordered per patient contact was reduced in all 4 departments by 11.4% overall. In contrast, volume increased by 2.4% in 19 comparable hospitals in the Netherlands.

    Meaning  The approach used in this study can be used as a framework for future projects aiming to reduce unnecessary laboratory diagnostic tests in routine clinical practice.

    Abstract

    Importance  Inappropriate use of laboratory testing is a challenging problem. Estimated overuse rates of approximately 20% have been reported. Effective, sustainable solutions to stimulate optimal use are needed.

    Objective  To determine the association of a multifaceted intervention with laboratory test volume.

    Design, Setting, and Participants  A before-after quality improvement study was performed between August 1, 2016, and April 30, 2018, in the internal medicine departments of 4 teaching hospitals in the Netherlands. Data on laboratory order volumes from 19 comparable hospitals were used as controls. The participants were clinicians ordering laboratory tests.

    Interventions  The intervention included creating awareness through education and feedback, intensified supervision of residents, and changes in order entry systems. Interventions were performed by local project teams and guided by a central project team during a 6-month period. Sustainability was investigated during an 8-month follow-up period.

    Main Outcomes and Measures  The primary outcome was the change in slope for laboratory test volume. Secondary outcomes were change in slope for laboratory expenditure, order volumes and expenditure for other diagnostic procedures, and clinical outcomes. Data were collected on duration of hospital stay, rate of repeated outpatient visits, 30-day readmission rate, and rate of unexpected prolonged duration of hospital stay for patients admitted for pneumonia.

    Results  The numbers of internists and residents ordering tests in hospitals 1 to 4 were 16 and 30, 18 and 20, 13 and 17, and 21 and 60, respectively. Statistically significant changes in slope for laboratory test volume per patient contact were found at hospital 1 (change in slope, −1.55; 95% CI, −1.98 to −1.11; P < .001), hospital 3 (change in slope, −0.74; 95% CI, −1.42 to −0.07; P = .03), and hospital 4 (change in slope, −2.18; 95% CI, −3.27 to −1.08; P < .001). At hospital 2, the change in slope was not statistically significant (−0.34; 95% CI, −2.27 to 1.58; P = .73). Laboratory test volume per patient contact decreased by 11.4%, whereas the volume increased by 2.4% in 19 comparable hospitals. Statistically significant changes in slopes for laboratory costs and volumes and costs for other diagnostic procedures were also observed. Clinical outcomes were not associated with negative changes. Important facilitators were education, continuous attention for overuse, feedback, and residents’ involvement. Important barriers were difficulties in data retrieval, difficulty in incorporation of principles in daily practice, and high resident turnover.

    Conclusions and relevance  A set of interventions aimed at changing caregivers’ mindset was associated with a reduction in the laboratory test volume in all departments, whereas the volume increased in comparable hospitals in the Netherlands. This study provides a framework for nationwide implementation of interventions to reduce unnecessary laboratory testing.

    Introduction

    In recent years, the concept of low-value care has gained attention, and international campaigns have been launched to discourage the unnecessary use of tests and procedures. In the United Kingdom, Do Not Do recommendations1 were formulated, and in the United States, the Choosing Wisely campaign was introduced.2 Thereafter, many other countries, including the Netherlands, followed suit with programs aimed at deimplementing unnecessary care.3

    Inappropriate use of laboratory tests is a well-recognized phenomenon, and estimated overuse rates of approximately 20% have been reported.4 Overuse is also reflected in high interphysician variability of test orders. In a recent study5 among internal medicine residents, some residents ordered 7 to 8 times more tests than their peers. Many interventions have proven effective in reducing unnecessary laboratory testing.6-8 In addition to financial consequences, overuse is less patient friendly and may increase the number of false-positive results, which leads to more, potentially harmful tests.9

    In 2012, our study group published the findings of a multifaceted intervention10 begun in 2008 aimed at reducing unnecessary diagnostic testing through increasing awareness at the Internal Medicine Department of the Vrije Universiteit Medical Center in Amsterdam, the Netherlands. Although we mainly focused on laboratory testing, the use of other diagnostic procedures also decreased. A 13% gross reduction in diagnostic expenditure was observed compared with the previous year and was sustained over subsequent years.11

    In the Reduction of Unnecessary Diagnostics Through Attitude Change of the Caregivers (RODEO) project, we implemented this same multifaceted intervention in the internal medicine departments of 4 large teaching hospitals in the Netherlands. The goal of this project was to reduce total laboratory testing by 5%. Our primary focus was on laboratory testing, although associations with other diagnostic procedures were also assessed. Furthermore, we assessed the facilitators and barriers to deimplementation of unnecessary testing.

    Methods

    This project is a part of the To Do or Not to Do? Reducing Low-Value Care program,12 a national program initiated by the Dutch Federation of University Medical Centers. The study protocol providing a detailed description of the methods has been published previously.13 Therefore, we provide only a brief overview of the methods here.

    This report follows the Standards for Quality Improvement Reporting Excellence (SQUIRE) reporting guideline for quality improvement studies. The medical ethics review committee of Vrije Universiteit Medical Center assessed the project protocol, determined that official approval by the committee was not required, and waived the need for informed consent because data were collected anonymously. Local ethics committees and boards of directors of all participating hospitals approved the study.13

    Study Design and Setting

    We conducted a before-after quality improvement study at the departments of internal medicine of 4 teaching hospitals in the Netherlands. The study was performed by a coordinating project team together with a local project team at each hospital (eAppendix 1 in the Supplement). The inclusion criteria were previously described.13

    Characteristics of Participating Departments

    Table 1 shows the characteristics of participating departments at initiation of the project. The departments differ mainly in annual patient load and in number of physicians working at the departments. In the remainder of this article, the participating hospitals are referred to as hospitals 1 to 4. Numbers were selected randomly.

    Outcomes, Data Sources, and Measurements

    Our primary outcome was change in slope for laboratory test volume per patient contact. Secondary outcomes were change in slope for laboratory test costs; order volumes and costs for radiology, microbiology, and nuclear medicine tests per patient contact; and clinical outcomes. Orders placed for the internal medicine specialty by inpatient and outpatient departments and the emergency department were included.

    To investigate whether our intervention influenced patient care, we assessed the mean duration of hospital stay, rate of repeated outpatient visits, 30-day readmission rate, and rate of unexpected prolonged duration of hospital stay for patients admitted for pneumonia. The last 2 outcomes are quality indicators assessed yearly by the Healthcare Inspectorate in the Netherlands.14

    In addition, we collected data on laboratory order volumes from 19 comparable hospitals in the Netherlands. Most of these were large peripheral hospitals, reflecting the national mix of hospital type. The criterion for inclusion was availability of data for the duration of the project. The benchmark data used in this study are based on standardized and validated production data of these of 19 hospitals. These data were acquired through Performation, a Dutch data-driven consultancy firm.15

    Assessment of Facilitators and Barriers

    Facilitators and barriers were identified through questionnaires with the project teams13 at initiation, during joint conferences, and through a questionnaire filled out by all physicians at participating departments at the end of the project (eAppendix 2 in the Supplement).

    Deimplementation Strategy

    A timeline of the project is shown in Figure 1. After consent by the board of directors of each hospital, cooperation agreements were signed. Thereafter, project teams consisting of 1 or more internists, residents, and clinical chemists and a business intelligence or control specialist were formed.

    Preintervention Period

    Ordering patterns from the preceding years were analyzed with the aim of recognizing patterns in test use. A joint conference was organized with all members of the project teams to exchange ideas and experiences and to discuss target laboratory items for reduction. In addition, anticipated facilitators and barriers were discussed. Each department was encouraged to develop interventions specifically for their department, in addition to the generic intervention that is described in this article.

    Intervention Period

    The local project teams performed the interventions and had monthly meetings with the coordinating project team, during which the progress of the study and ordering patterns were discussed. A second joint conference was organized with all project teams to discuss interim results, facilitators and barriers, and ideas and experiences.

    Postintervention Period

    A third joint conference was organized, during which the main goal was to discuss sustainability. The local project teams continued the actions introduced earlier, performed new actions, and had 2 to 3 monthly progress meetings with the coordinating project team.

    Description of Interventions

    The intervention consisted of several items. Education and feedback were used to create awareness and were provided through, for example, presentations and newsletters. In addition, supervision of residents by experienced physicians regarding test ordering was intensified. This entailed explicitly focusing on indications for tests and asking critical questions during morning reports and other clinical meetings. In addition, modifications were made to order entry systems, for example by enacting time limits on repeated orders.

    The actions performed in each clinic are displayed in eFigure 1 in the Supplement. Details on specific actions are published in the study protocol.13 An overview of the modifications made to the order entry systems is provided in eTable 1 in the Supplement.

    The project teams placed specific focus on a set of tests that are known to be frequently overused: blood urea nitrogen, creatinine, amylase, aspartate aminotransferase, alanine aminotransferase, C-reactive protein, and erythrocyte sedimentation rate. These were used to create a change in physician mindset, which we believed would lead to greater awareness when requesting diagnostic procedures in general. A more-detailed description of the deimplementation strategy is published in the protocol.13

    Statistical Analysis
    Volumes and Costs of Diagnostic Procedures

    For our primary outcome, we collected the weekly number of tests performed during the intervention and postintervention period and the 2 preceding years. Secondary outcomes were reported by the hospitals per month. We adjusted volumes and costs for patient load using the number of patient contacts, defined as the sum of the number of visits, day admissions, and patient days for the internal medicine department. We decided against using standardized patient units to adjust for patient load, which we previously described in the study protocol,13 because we observed that a large proportion of diagnostic procedures were ordered during repeated outpatient visits that were not included in the calculation of standardized patient units.

    Clinical Outcomes

    The mean duration of hospital stay and rate of repeated outpatient visits were assessed monthly. The 30-day readmission rate and the rate of unexpected prolonged duration of hospital stay for patients admitted for pneumonia were assessed yearly for 2015, 2016, and 2017.

    Analysis

    All statistical analyses were performed using the forecast package in R statistical software version 3.4.3 (R Project for Statistical Computing). Interrupted time series analyses were performed using the weekly or monthly data to assess the associations of the intervention with volumes and costs. We used an autoregressive integrated moving average model to analyze whether a (more profound) change in the number or costs of tests per patient contact was observed after starting the intervention. We adjusted for seasonal variation using autocorrelations with a yearly period. The trend in mean was modeled using a regression model with separate parameters for slope before and after implementation of the intervention and allowing for a direct change in the period after the start of the intervention. The autocorrelation and moving average parameters were selected using the automated auto.arima function in R, minimizing the corrected Akaike information criterion statistic. Outcomes are expressed as the difference in slope between the 2 years before the intervention started and the 14 months after the intervention started (6-month intervention period plus 8-month postintervention period). We used a z test to determine whether regression coefficients in the autoregressive integrated moving average model differed from 0. All P values refer to 2-sided tests. P < .05 was considered statistically significant. The percentage change in laboratory test volume per patient contact was calculated using the mean number of laboratory tests per patient contact during the final 6 months of the project and the mean number during the same 6 months of the previous year.

    Results

    The numbers of internists and residents ordering tests in hospitals 1 to 4 were 16 and 30, 18 and 20, 13 and 17, and 21 and 60, respectively.

    Laboratory Test Volume

    Our main goal was to assess the association of our multifaceted intervention with laboratory test volume. Slopes and changes in slopes for laboratory test volume per patient contact per year are presented in Figure 2 and Table 2. Changes in slope were statistically significant at hospital 1 (−1.55; 95% CI, −1.98 to −1.11; P < .001), hospital 3 (−0.74; 95% CI, −1.42 to −0.07; P = .03), and hospital 4 (−2.18; 95% CI, −3.27 to −1.08; P < .001). At hospital 2, the change in slope was not statistically significant (−0.34; 95% CI, −2.27 to 1.58; P = .73). For hospitals 1 to 4, reductions in numbers of tests per patient contact in the last 6 months of the intervention were 11.5%, 5.9%, 8.3%, and 14.7%, respectively, compared with the same period in the year before, yielding an overall decrease of 11.4%. In eFigures 2, 3, 4, and 5 in the Supplement, we present the changes in laboratory test volumes relative to the same months in the preceding year during the 14 months, along with the interventions performed.

    Control Group

    In the departments of internal medicine of 19 comparable hospitals, we saw a significant increase of 0.32 laboratory tests per patient contact per year (95% CI, 0.21-0.42; P < .001) between January 1, 2015, until our intervention started in the first 2 hospitals on November 1, 2016, and a comparable increase (0.33; 95% CI, 0.20-0.47; P < .001) between November 1, 2016, and April 30, 2018, which was when the project was concluded. The number of laboratory tests per patient contact increased by 2.4% in the last 6 months of the project compared with the same period in the year before.

    Specific Tests

    The departments targeted all tests described in the Methods section, except for hospital 2, which did not focus on blood urea nitrogen and creatinine. Significant changes in slope for number of requests were observed for blood urea nitrogen and creatinine (hospitals 1 and 4), amylase (hospitals 1, 2, and 3), aspartate aminotransferase (hospitals 3 and 4), and alanine aminotransferase (hospital 4). Full data are shown in eTable 2 in the Supplement. The associations of the multifaceted intervention with costs of laboratory tests, and with volumes and costs of radiology, microbiology, and nuclear medicine tests are presented in Table 2.

    Clinical Outcomes

    Clinical outcomes are shown in Table 3. For mean duration of hospital stay, no significant changes in slope were observed. For rate of repeated outpatient visits, a significant decreasing change in slope was found at hospital 1. The 30-day readmission rates remained unchanged. The rate of unexpected prolonged duration of hospital stay for patients admitted with pneumonia increased in all departments, which was consistent with the national trend and presumably was attributable to the severe influenza epidemic in 2017.

    Facilitators and Barriers to Deimplementation

    The factors identified through the initial questionnaire and first and second conference, categorized into different levels as proposed by Grol and Wensing,16 were used as input to adjust our strategy (eTable 3 in the Supplement). During the final conference and through the final questionnaire, we evaluated which factors were of greatest influence (eTables 4, 5, and 6 in the Supplement).

    Important facilitators were education, continuous attention for diagnostic testing, and feedback. Involvement of clinical chemists and establishing clear working agreements were also considered important. During the project, the teams were expanded with physicians representing internal medicine subspecialties. This facilitated our efforts to obtain widespread support, which was a crucial element in the RODEO project. Having enthusiastic internists and residents function as role models was considered a strong facilitating factor. Members of the coordinating project team viewed the involvement of residents as the main factor contributing to the success of the project.

    Although the teams aimed to establish clear working agreements, incorporation in daily practice was impeded by the high rate of turnover of residents, which required regular repetition of RODEO principles. The most important barrier was obtaining reliable data on order volumes and costs, which made it difficult to monitor progress in some clinics. In addition, it took several months before reduction efforts translated into consistent changes in ordering patterns (eFigures 2, 3, 4, and 5 in the Supplement). It was challenging to maintain the efforts needed for the project, especially when only 1 resident was included in the project team and that resident changed rotation. Moreover, we noticed that lessening attention was associated with an almost immediate change in ordering patterns (eFigures 2, 3, 4, and 5 in the Supplement). Although modifying order systems was an important facilitator, their rigidness was seen as a barrier in 1 hospital, because clinicians were not immediately informed when a test was not performed and the time limits for repeated requests were considered too strict.

    Discussion

    In this project, we aimed to reduce inappropriate laboratory testing by implementing interventions aimed at changing the mindset of health care professionals. In 3 of 4 departments, a significant decrease in slope was found for laboratory test volume, whereas an increase in slope was observed in 19 comparable hospitals in the Netherlands. The laboratory test volume per patient contact decreased by 11.4% overall. Although every department performed interventions from all categories, nuances were different. In hospital 1, the role of the clinical chemist in the project team was substantial, whereas hospital 3 mainly focused on order system changes by instituting time limits for repeat orders, and hospital 4 placed emphasis on educating physicians.

    For other diagnostic procedures, significant changes in slopes were also found. At hospital 1, statistically significant changes in slope were found for volume and/or costs of laboratory, radiology, and nuclear medicine tests. At hospital 2, although a 5.9% reduction in volume of laboratory tests per patient contact was achieved, the change in slope was not statistically significant. For nuclear medicine, we did find statistically significant decreasing changes in slope for volume and costs, which we think are a result of the departments’ increased focus on indications for positron emission tomography scans during the intervention period. At hospital 3, the decreases in slope for order volumes of microbiology and nuclear medicine tests were statistically significant, together with laboratory test volume. For hospital 4, data on diagnostic procedures other than laboratory tests were not available. The observed changes in ordering patterns for diagnostic procedures on which little focus was placed suggest that the intervention led to a change in caregivers’ mindset for ordering diagnostic procedures in general. Clinical outcomes were not associated with negative changes following the reduction in diagnostic testing.

    Overall, important facilitators to deimplementation were education, feedback, continuous attention for diagnostic testing, and involvement of residents. In contrast to hospitals 3 and 4, only 1 resident was included in the project team at hospital 2. In addition, this resident changed rotation during the project. Also, the outpatient department at hospital 2 was not as emphatically targeted, although we observed that in all participating hospitals, a large number of orders were placed at the outpatient department. We believe that this finding, in part, reflects the importance of obtaining widespread support for the success of the project. This knowledge was used as input for performing the project at hospitals 3 and 4, where several residents were included in the local project team from initiation, and much attention was paid to orders placed at the outpatient department. The most important barriers were difficulties in data collection, difficulties in incorporation of working agreements in daily practice, and a high rate of resident turnover.

    The total cost for performing the project was approximately €250 000. The largest part of this amount was spent on the personnel who coordinated the project (€200 000). At the end of the project, the intervention was integrated into daily routine without hiring extra staff. As for potential cost reduction through reducing laboratory testing, an estimated reduction of approximately €1.2 million could be made. This yields an 11.4% reduction of the total €10.9 million spent on laboratory testing in 2017 by all 4 departments. Because of the fixed costs for personnel and laboratory equipment, the actual cost reduction will be lower. Cost reduction through the reduction of other diagnostic procedures as well as downstream costs were not included in this calculation.

    Comparison With Previous Literature

    In line with previous literature,6-8 a multifaceted intervention was associated with a reduction in laboratory testing. Contrary to most other studies, we used a multicenter approach, introducing our intervention in 4 large hospitals. Data were collected during the 6-month intervention period and, unlike most other studies, we also assessed sustainability during an additional 8-month period. Furthermore, we collected clinical data to investigate patient-related outcomes. This study is one of the first to investigate all these aspects, to our knowledge, and we further demonstrate that with the help of local teams, these interventions can be implemented successfully in daily practice. Previous literature6-8 does not show that any particular intervention is the most effective, and combined interventions are advocated. However, in the questionnaires conducted at the end of the project, the physicians in our project reported perceiving the educational sessions to be the most effective intervention, followed by modifications in order systems.

    Strengths and Limitations

    The intervention used in this study was originated and first applied in an academic medical center in Amsterdam.10,11 In the current study, we have shown that it is feasible to implement this intervention in 4 large peripheral teaching hospitals. The positive and sustained changes observed in different settings suggest that our approach may also be effective in other hospitals and for other services. By assessing facilitators and barriers, we point out specific issues to take into account in future studies. Another strength entails assessment of sustainability. Although we investigated short-term sustainability, long-term sustainability can be expected through our approach, as we have previously shown.11 Measures to ensure sustainability include, for example, repeated education, modifications in ordering systems, posters stating important principles regarding test ordering displayed in work spaces, mouse pads with reminders regarding test ordering, and inclusion of these principles in the introductory program for new employees. Another strength of this project is the study of clinical outcomes. Another strength involves the intervention itself. By providing departments the liberty to place focus on different elements of the standard intervention, we ensured that the actions suited the departmental structure.

    This project has several limitations. First, because multiple interventions were performed at once, it was not possible to assess the effectiveness of individual interventions. Second, data collection was difficult and incomplete for 1 hospital. Also, it was not possible to assess the numbers of blood samples obtained at all clinics, which would be a desirable end point in future studies in the context of delivering patient-friendly care. The lack of patient involvement in this project is another limitation.

    Conclusions

    A set of interventions aimed at changing caregivers’ mindset was associated with a reduction in the laboratory test volume in all departments, whereas the volume increased in comparable hospitals in the Netherlands. In 3 of 4 departments, the change in slope was significant. Laboratory costs and other diagnostic procedures were also reduced following the intervention, and clinical outcomes were unchanged. Furthermore, we identified facilitators and barriers to deimplementation. The approach used in this study can be extended to other types of services and clinics. This study provides a framework for nationwide implementation of these interventions and might be complemented with involving patients and emphasizing patient-friendly care in future efforts.

    Back to top
    Article Information

    Accepted for Publication: May 21, 2019.

    Published: July 24, 2019. doi:10.1001/jamanetworkopen.2019.7577

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Bindraban RS et al. JAMA Network Open.

    Corresponding Author: Prabath W. B. Nanayakkara, MD, PhD, Section of Acute Medicine, Department of Internal Medicine, Amsterdam University Medical Center, Location VU University Medical Center, De Boelelaan 1118, 1081 HZ Amsterdam, the Netherlands (p.nanayakkara@vumc.nl).

    Author Contributions: Dr Nanayakkara had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Nanayakkara was the principal investigator.

    Concept and design: Bindraban, van Beneden, Kramer, van Solinge, Al-Dulaimy, Bandt, Castelijn, ten Berg, Nanayakkara.

    Acquisition, analysis, or interpretation of data: Bindraban, van Beneden, van Solinge, van de Ven, Naaktgeboren, van der Wekken, Bandt, Stam, Neppelenbroek, Griffioen-Keijzer, Castelijn, Wevers, Boerman, van Wijnen, ten Berg, Nanayakkara.

    Drafting of the manuscript: Bindraban, van Beneden, van de Ven, Naaktgeboren, Al-Dulaimy, ten Berg, Nanayakkara.

    Critical revision of the manuscript for important intellectual content: Bindraban, van Beneden, Kramer, van Solinge, van de Ven, Naaktgeboren, van der Wekken, Bandt, Stam, Neppelenbroek, Griffioen-Keijzer, Castelijn, Wevers, Boerman, van Wijnen, ten Berg, Nanayakkara.

    Statistical analysis: Bindraban, van Beneden, van de Ven, Naaktgeboren, Bandt, ten Berg, Nanayakkara.

    Obtained funding: Bindraban, van Beneden, van Solinge, ten Berg, Nanayakkara.

    Administrative, technical, or material support: Bindraban, van der Wekken, Bandt, Neppelenbroek, Castelijn, Wevers, van Wijnen, ten Berg, Nanayakkara.

    Supervision: Bindraban, Kramer, van Solinge, Naaktgeboren, Al-Dulaimy, Stam, Griffioen-Keijzer, ten Berg, Nanayakkara.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: This work was supported by grant 80-83920-98-400 from the Netherlands Organization for Health Research and Development.

    Role of the Funder/Sponsor: The Netherlands Organization for Health Research and Development had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Additional Contributions: We thank the individual members of participating project teams for their uncompensated contribution to the study: J.W. Plaisier, Department of Internal Medicine, N. Osmanovic, Laboratory of Clinical Chemistry, and J. Dinkelaar, Laboratory of Clinical Chemistry, Zaans Medical Center, Zaandam, the Netherlands; B. van Dam, Department of Internal Medicine, D. ten Oever, Department of Internal Medicine, V.A. de Weger, Department of Internal Medicine, E. ten Boekel, Department of Clinical Chemistry, Hematology and Immunology, and J. de Gans-de Wit, Department of Business Control, North-West Hospital Group, Alkmaar, the Netherlands; R. Soetekouw, Department of Internal Medicine, K. Farhat, Department of Internal Medicine, J.S. ten Kulve, Department of Internal Medicine, N.N. Radhakishun, Department of Internal Medicine, C.I.M. de Roij van Zuijdewijn, Department of Internal Medicine, N. Slager, Department of Planning and Control, and M.M. Buijs, Atalmedial Medical Diagnostics Centers, Spaarne Gasthuis, Hoofddorp and Haarlem, the Netherlands; and R. Fijnheer, Department of Internal Medicine, J.A.J. Traa, Department of Finance and Control, and R. Goedegebuure, Department of Internal Medicine, Meander Medical Center, Amersfoort, the Netherlands.

    References
    1.
    National Institute for Health and Care Excellence. NICE Do not Do prompts. https://www.nice.org.uk/sharedlearning/nice-do-not-do-prompts. Published December 2013. Accessed June 16, 2019.
    2.
    American Board of Internal Medicine Foundation. Choosing Wisely. http://www.choosingwisely.org/. Accessed October 24, 2018.
    3.
    Levinson  W, Kallewaard  M, Bhatia  RS, Wolfson  D, Shortt  S, Kerr  EA; Choosing Wisely International Working Group.  ‘Choosing Wisely’: a growing international campaign.  BMJ Qual Saf. 2015;24(2):167-174. doi:10.1136/bmjqs-2014-003821PubMedGoogle ScholarCrossref
    4.
    Zhi  M, Ding  EL, Theisen-Toupal  J, Whelan  J, Arnaout  R.  The landscape of inappropriate laboratory testing: a 15-year meta-analysis.  PLoS One. 2013;8(11):e78962. doi:10.1371/journal.pone.0078962PubMedGoogle ScholarCrossref
    5.
    Geleris  JD, Shih  G, Logio  L.  Analysis of diagnostic test ordering habits among internal medicine residents.  JAMA Intern Med. 2018;178(12):1719-1721. doi:10.1001/jamainternmed.2018.3519PubMedGoogle ScholarCrossref
    6.
    Kobewka  DM, Ronksley  PE, McKay  JA, Forster  AJ, van Walraven  C.  Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review.  Clin Chem Lab Med. 2015;53(2):157-183. doi:10.1515/cclm-2014-0778PubMedGoogle ScholarCrossref
    7.
    Solomon  DH, Hashimoto  H, Daltroy  L, Liang  MH.  Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework.  JAMA. 1998;280(23):2020-2027. doi:10.1001/jama.280.23.2020PubMedGoogle ScholarCrossref
    8.
    Bindraban  RS, ten Berg  MJ, Naaktgeboren  CA, Kramer  MHH, van Solinge  WW, Nanayakkara  PWB.  Reducing test utilization in hospital settings: a narrative review.  Ann Lab Med. 2018;38(5):402-412. doi:10.3343/alm.2018.38.5.402PubMedGoogle ScholarCrossref
    9.
    van Walraven  C, Naylor  CD.  Do we know what inappropriate laboratory utilization is? a systematic review of laboratory clinical audits.  JAMA. 1998;280(6):550-558. doi:10.1001/jama.280.6.550PubMedGoogle ScholarCrossref
    10.
    Vegting  IL, van Beneden  M, Kramer  MH, Thijs  A, Kostense  PJ, Nanayakkara  PW.  How to save costs by reducing unnecessary testing: lean thinking in clinical practice.  Eur J Intern Med. 2012;23(1):70-75. doi:10.1016/j.ejim.2011.07.003PubMedGoogle ScholarCrossref
    11.
    Bindraban  RS, van Beneden  M, ten Berg  MJ, Nanayakkara  PWB.  Long-term sustainability of a multi-step intervention to reduce unnecessary diagnostic testing.  Eur J Intern Med. 2017;44:e38-e39. doi:10.1016/j.ejim.2017.07.025PubMedGoogle ScholarCrossref
    12.
    Nederlandse Federatie van UMC. To do or not to do? [in Dutch]. https://www.doenoflaten.nl/. Accessed October 24, 2018.
    13.
    Bindraban  RS, van Beneden  ML, Kramer  MH,  et al.  A multicenter before-after study on reducing unnecessary diagnostics by changing the attitude of caregivers: protocol for the RODEO Project.  JMIR Res Protoc. 2018;7(8):e10473. doi:10.2196/10473PubMedGoogle ScholarCrossref
    14.
    Basisset Medisch Specialistische Zorg Kwaliteitsindicatoren.  Inspectie voor de Gezondheidszorg. Utrecht, the Netherlands: Ministerie van Volksgezondheid, Welzijn en Sport; 2017.
    15.
    Performation. https://performation.com/. Accessed October 24, 2018.
    16.
    Grol  R, Wensing  M.  What drives change? barriers to and incentives for achieving evidence-based practice.  Med J Aust. 2004;180(6)(suppl):S57-S60.PubMedGoogle Scholar
    ×