Cost-effectiveness and Economic Benefit of Continuous Professional Development for Drug Prescribing: A Systematic Review

Key Points

Question  What are the comparative costs and benefits of physician continuous professional development (CPD) for drug prescribing?

Findings  In this systematic review of 38 studies, CPD was associated with reduced health care costs (median drug cost savings of $79 373) compared with no training. More intensive CPD (compared with no training or less intensive interventions) was associated with improved effectiveness (prescribing) outcomes but incurred greater education costs (incremental cost of $3 to $4105 per physician per standardized effectiveness change).

Meaning  These results suggest that physician CPD for drug prescribing is associated with reduced health care costs and that both effectiveness outcomes and costs should be considered when making education decisions.

Importance  The economic impact of continuous professional development (CPD) education is incompletely understood.

Objective  To systematically identify and synthesize published research examining the costs associated with physician CPD for drug prescribing.

Evidence Review  MEDLINE, Embase, PsycInfo, and the Cochrane Database were searched from inception to April 23, 2020, for comparative studies that evaluated the cost of CPD focused on drug prescribing. Two reviewers independently screened all articles for inclusion and reviewed all included articles to extract data on participants, educational interventions, study designs, and outcomes (costs and effectiveness). Results were synthesized for educational costs, health care costs, and cost-effectiveness.

Findings  Of 3338 articles screened, 38 were included in this analysis. These studies included at least 15 659 health care professionals and 1 963 197 patients. Twelve studies reported on educational costs, ranging from $281 to $183 554 (median, $15 664). When economic outcomes were evaluated, 31 of 33 studies (94%) comparing CPD with no intervention found that CPD was associated with reduced health care costs (drug costs), ranging from $4731 to $6 912 000 (median, $79 373). Four studies found reduced drug costs for 1-on-1 outreach compared with other CPD approaches. Regarding cost-effectiveness, among 5 studies that compared CPD with no intervention, the incremental cost-effectiveness ratio for a 10% improvement in prescribing ranged from $15 390 to $437 027 to train all program participants. Four comparisons of alternative CPD approaches found that 1-on-1 educational outreach was more effective but more expensive than group education or mailed materials (incremental cost-effectiveness ratio, $18-$4105 per physician trained).

Conclusions and Relevance  In this systematic review, CPD for drug prescribing was associated with reduced health care (drug) costs. The educational costs and cost-effectiveness of CPD varied widely. Several CPD instructional approaches (including educational outreach) were more effective but more costly than comparators.

Inappropriate prescribing (including prescription errors, overuse and underuse of medications, and unnecessarily expensive medications) harms patients and wastes resources.¹ Up to one-third of prescriptions are inappropriate,² and these inappropriate prescriptions are associated with suboptimal clinical outcomes³ and large financial burden.^4,5 With expanding drug formularies, increasing patient comorbidities, and progressively individualized treatment recommendations, ensuring appropriate prescribing will become even more challenging. Systematic reviews have found that restrictive interventions to optimize prescribing (eg, prioritized formularies, prior authorization, and computerized decision support) have benefits^6-8 but may be associated with treatment delays and adverse effects on clinicians’ professional identity and culture.⁶ By contrast, educational interventions (eg, audit and feedback as well as educational outreach) have overall favorable effects on practitioner performance and patient outcomes.^6,9-14

Continuous professional development (CPD) is essential to clinicians’ efforts to maintain competency after completion of training and includes formal educational interventions and unstructured learning activities. Although CPD programs to improve prescribing practices are increasingly common, their cost and cost-effectiveness remain incompletely characterized. Reviews^6,9-14 of interventions to improve drug prescribing have touched only briefly on the economic outcomes (educational costs and health care cost savings) of CPD interventions. Only 1 review¹⁵ (published in 2002) focused on cost of CPD, and that review included only 1 study of CPD for drug prescribing.

A comprehensive synthesis of evidence regarding the comparative costs and benefits of physician CPD for drug prescribing would provide clinicians, educators, and administrators information to reduce wasted effort (cost and physician time), identify resource-efficient instructional approaches, and promote more effective health care. We conducted a systematic review to determine the comparative costs and benefits (including cost-effectiveness and health care cost savings) of physician CPD for drug prescribing and the CPD features that are associated with improved cost-benefit outcomes.

This systematic review (part of a larger systematic review of economic outcomes of CPD¹⁶) was planned, conducted, and reported in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline¹⁷ and reporting guidelines for economic evaluations.^18-20

We worked with an experienced reference librarian to create a search strategy to identify studies that examined the cost of CPD. This strategy incorporated key concepts and terms related to the population (eg, physicians), intervention (eg, education continuing), and outcomes²¹ (eg, economics medical; costs and cost analysis). We used this strategy (detailed in the eBox in the Supplement) to search MEDLINE, Embase, PsycInfo, and the Cochrane Database from each database’s inception through April 23, 2020. We subsequently identified 3 additional relevant studies cited in other included articles.

We included all original comparative economic evaluation studies, published in any language, that investigated CPD for practicing physicians. We defined CPD as “activities intended to promote or measure the clinical knowledge/skills of physicians in independent medical practice through courses or assessments delivered in any modality or venue, whether or not continuing medical education (CME) credit is awarded; or self-directed learning or self-assessment activities for which CME credit is awarded.”¹⁶ (See the eBox in the Supplement for additional operational definitions.) From these studies we then identified all CPD activities that addressed the clinical topic of drug prescribing.

Reviewers (all authors) worked in assigned pairs to screen studies for inclusion, first reviewing the title and abstract and then reviewing the full text if needed (interreviewer agreement, κ = 0.73). Reviewers resolved all disagreements by consensus.

We implemented a data abstraction form in software designed for systematic reviews (DistillerSR, Evidence Partners Inc). Two reviewers (D.A.C. and C.R.S.) independently extracted information on study design, participants, CPD interventions, methodological quality, effectiveness outcomes (including prescribing rates, errors, or costs), educational costs, and health care costs. For educational costs, we used Levin’s “ingredients” approach,²² organizing costs according to defined categories¹⁶ related to education planning and implementation. We resolved conflicts by consensus.

We appraised general methodological quality using the Medical Education Research Study Quality Instrument,²³ which evaluates the study design, sampling, outcomes (type [counting drug prescriptions and drug costs as behavior outcomes], objectivity, and validity), and statistical analyses. We also appraised methods specific to economic analyses, selected from the 1996 BMJ guidelines.²⁴

We converted all monetary estimates to 2021 US dollars, first adjusting for inflation in the original currency (starting with the final year of data collection and ending in 2021) and then converting the original currency into US dollars using exchange rates on May 6, 2021. Given well-documented concerns about meta-analysis of cost-effectiveness studies (including between-context heterogeneity in resource use and pricing^25-28), we did not attempt such analyses. Instead, to evaluate cost-effectiveness, we adopted the dominance ranking matrix approach²⁹ advocated by the Joanna Briggs Institute.²⁷ We also calculated monetary net benefit (health care costs averted minus educational costs) for all studies that reported educational costs and health care costs.

We identified 3338 potentially eligible studies of which 38 met criteria for inclusion (eFigure 1 in the Supplement).^30-67 These studies enrolled more than 15 659 health care professionals (median, 180.5; IQR, 68-353; 28 studies reporting this information) and collected data on more than 1 963 197 patients (median, 4763; IQR, 1622-169 447; 16 studies reporting). Three studies^40,42,62 were translated from a non-English language (2 German and 1 Spanish). Five studies included more than 2 arms,^{31,33,48,54,59} resulting in more than 1 relevant comparison per study. After describing overall study features and quality, we report educational costs, health care cost impact, and educational costs combined with effectiveness outcomes (cost-effectiveness and net benefit).

Table 1 reports a summary of key features, and eTable 1 in the Supplement contains detailed information for each study. We distinguished studies that evaluated the cost of (expenses incurred in implementing) the CPD activity (12 studies^{31,32,38,46-48,54,56,59,60,63,66}) and studies that evaluated health care costs after a CPD activity (35 studies^{30-58,61-65,67}).

All studies involved physicians in independent practice, most of whom were family, internal, or general medicine physicians (26 studies [68%]) or all physicians in the facility (4 studies [11%]). Thirteen studies^{33,35,41,47,49,51,52,54-57,61,66} (34%) involved other health care professionals (besides physicians), including postgraduate physician trainees (n = 5), nurse practitioners and physician assistants (n = 5), pharmacists (n = 5), nurses (n = 3), and medical students (n = 1).

Most studies focused on optimal use of specific drugs or drug classes, including antibiotics (n = 15 studies), analgesics (n = 3 studies), antihyperlipidemics (n = 3 studies), and neuropsychiatric drugs (n = 3 studies). Three studies focused on general drug-drug interactions or prescribing errors. One study⁶⁶ compared 2 approaches to increase attendance at a CPD course on drug prescribing.

Instructional modalities included paper materials (26 studies^{30-39,41,45,47-55,60,61,63,64,67}), face-to-face small groups or 1-on-1 outreach (22 studies^{30,31,33,35,36,40,42-44,46,48-50,53,54,56,58-60,62,63,65}), performance audit and feedback (19 studies^{35,37-39,42-46,49,51-53,57-59,63-65}), and face-to-face large groups (14 studies^{30,32,34,35,37,43,47,52,55-57,59,60,65}). Four studies^41,51,57,67 also implemented changes in clinical practice (eg, clinical checklists).

Considering the context of education, most studies were conducted in private or independent practice^{31-38,40-50,53-56,58-61,63-67} (32 [84%]); 6 (16%) were in academic settings^{30,34,39,40,52,57} and 2 (5%) in government facilities.^51,62 Thirty studies^{31,33-35,38,40-51,53-56,58-61,63-67} (79%) involved patients in an ambulatory setting, 7 (18%) involved patients in a hospital ward,^{30,32,36,37,52,57,62} and 3 (8%) involved a medical procedure.^32,37,39 Geographic origins included the US (15 [39%]), the UK (5 [13%]), other European countries (11 [29%]), Asia (3 [8%]), Canada (3 [8%]), and Oceania (1 [3%]). Funding sources included industry^{30,42,49,58,61} (5 [13%]), government^{31,32,48,50,51,59,63-66} (10 [26%]), private^34,53 (2 [5%]), and local⁵⁷ (1 [3%]) support. Twenty studies (53%) did not report funding^{33,35-41,43-47,52,54-56,60,62,67}; among these, for 5 studies^{33,35,41,43,44} the authors were employed by a capitated health plan (ie, a potential conflict of interest).

Twelve studies (32%) used randomized group assignment,^{31,33,45,48,50,53-55,59,63-65} 12 (32%) used 2 or more nonrandomized groups,^{32,35,38,41,44,46,47,58,60,61,66,67} and 14 (36%) involved a single-group pre/post or time-series design.^{30,34,36,37,39,40,42,43,49,51,52,56,57,62} Nearly all studies (35 [92%]) compared CPD against no intervention^{30-53,56-65,67} (baseline performance or a no intervention control group), whereas 10 (26%) compared one CPD intervention against another^{31,33,48,50,54,55,59,60,65,66} (eg, comparing different instructional approaches or implementation strategies).

Outcome measures were blinded or obtained from an unbiased data source (eg, pharmacy claims database) in 22 studies^{31,38,39,41,43-48,50,51,53-56,58,61,63-65,67} (58%). Evidence to support the validity of the outcome measure was infrequently reported: 10 studies (26%) reported content evidence^{32,45,46,48,51,57,59,63,64,67} (eg, information about the database) and 5 (13%) reported internal structure evidence^{33,41,48,51,64} (such as interrater reliability); none reported correlation with another variable. Participant retention was high in 27 studies^{31-33,38-48,50,51,53-56,58,59,61,63-65,67} (71%) (explicitly reported as ≥75% or data obtained from a database presumed to reflect all eligible clinicians). Thirty-two studies^{30-35,37-43,45-48,50,53-59,61-67} (84%) specified the time horizon, 3 (8%) stated a discount rate²⁴ or justified its absence,^31,33,63 and 3 (8%) conducted a sensitivity analysis.^31,48,64 See eTable 2 in the Supplement for additional details of economic study quality. Methodological quality (Medical Education Research Study Quality Instrument) scores ranged from 7 to 16 (of 18 possible), with a mean (SD) of 11.4 (2.3) (median, 11.5; IQR, 9.5-13.0).

Twelve studies^{31,32,38,46-48,54,56,59,60,63,66} (32%) reported costs associated with implementing CPD interventions. For each study, we extracted 12 possible cost ingredients^16,22 (Table 2). The ingredients reported varied widely across studies. Three studies (5 interventions) reported only the total estimated educational cost without any mention of specific ingredients, and another 3 studies (4 interventions) reported some or all costs as a lump sum while delineating the ingredients that counted in that sum. The most commonly reported ingredient was the cost of course faculty (7 studies), whereas 6 counted postage and 5 included the cost of educational materials. Four studies reported specific costs for 5 ingredients or more, and 6 studies reported costs for 3 ingredients or more. No studies itemized costs for equipment. Data sources for educational costs were reported infrequently, with 3 studies^59,60,63 (25%) reporting details of ingredient quantification and 5 studies^{38,56,59,60,63} (42%) reporting details of pricing.

The mean ingredient cost per intervention varied widely, ranging from $58 897 for learner attendance (ie, compensated hourly or daily wage or estimated lost revenue; mean of 3 interventions) (all costs reported in 2021 US dollars) to $1022 for catering (mean of 2 interventions). The total reported educational cost likewise varied widely, ranging from $281 (a study⁴⁵ that analyzed only postage and paper and reached 36 physicians) to $183 554 (a study⁶³ that analyzed 6 ingredients, training 127 physicians), with a mean (SD) of $32 676 ($51 215) (median, $15 664; IQR, $5080-$29 290).

Thirty-five studies^{30-58,61-65,67} (92%) evaluated health care costs after a CPD activity (see Table 3 for key features). In 11 studies^{32,34,49,52,53,55,57,61-63,65} the CPD intervention focused on wise stewardship (using the most appropriate drug [eg, antibiotic] for a given condition), 10 studies^{36,37,39,41,43-45,50,51,67} encouraged choice of less expensive drugs, 7 studies^{30,38,40,42,46,47,58} targeted fewer prescriptions overall, and 7 studies^{31,33,35,48,54,56,64} encouraged increased use of an effective treatment.

All 35 studies^{30-58,61-65,67} reported an outcome of drug cost, and this outcome was often the only health care cost outcome; thus, we used drug cost as the (common) metric of health care costs. For 1 study⁵⁶ the investigators expected higher short-term drug costs if prescribing recommendations were followed (ramipril in patients with diabetes); for the other 34 studies,^{30-55,57,58,61-65,67} the economic goal appeared to be immediate cost savings. The appropriateness of drug prescription was appraised in 5 studies^{32,33,37,57,67} (14%). Twenty-five studies^{33,38,46,47,54,55,58,63-65} took steps to ensure an appropriate (less biased) comparison by using a concurrent control group^{33,38,46,47,54,55,58,63-65} (10 [29%]) or making historical comparison matched for time of year^{30-32,34,40-42,45,46,55,56,58,63-65} (15 [43%]). Twenty-four studies^{31-34,37-39,41,43,46,48,50-56,58,61,63-65,67} (69%) reported details of resource quantification, and 26 studies^{31-35,38,39,41,43-48,50-52,54,56-58,61,63-65,67} (74%) reported details of pricing. Thirty studies^{30-35,37-43,45-48,50,53-58,61-65,67} (86%) reported the period of data collection, which ranged from 1 to 182 weeks.

Thirty-three studies (37 comparisons) evaluated drug costs for CPD compared with no intervention (a separate control group^{31-33,35,38,41,43-48,50,53,58,61,63-65,67} [n = 23 comparisons] or baseline metrics^{30,34,36,37,39,40,42,43,49,51,52,56,57,62} [n = 14 comparisons]). In all but 2 instances,^56,65 these studies found that CPD was associated with lower drug costs (Table 4). Considering these exceptions, in 1 study⁵⁶ CPD led to increased prescribing of ramipril as intended for patients with diabetes and a concomitant (expected) increase in drug costs. In the other study,⁶⁵ the rate and cost of antibiotic prescriptions were reduced as intended, but recommendations for nonantibiotic symptomatic treatments increased, resulting in higher total drug costs.

Among the remaining 31 studies,^{30-53,57,58,61-64,67} the total reported savings ranged from $4731 to $6 912 000 (median, $79 373; n = 22 studies reporting information sufficient to calculate), and the percentage of savings compared with the control group or baseline costs ranged from 2.8% to 62.0% (n = 29 studies). Substantial variation in study time period and sample size precluded meaningful cross-study comparisons of total costs, so when possible we standardized these variables to reflect a fixed period and number of patients or physicians (100 for 1 year). Although admittedly imperfect (it implies a linear response across time and sample sizes, and cross-study cost comparisons are still inherently challenging), this approach offers useful insights. Savings for 100 patients ranged from $1.00 to $186 862 per year (n = 24 studies), and savings for 100 physicians ranged $11 058 to $28 800 000 per year (n = 5 studies).

Seven studies^{31,33,48,50,54,55,65} (9 comparisons) compared drug costs for 2 or more alternative CPD approaches (Table 4). Four studies (5 comparisons) found that 1-on-1 outreach education was associated with reduced drug costs in comparison with mailed materials,^31,48,54 group education,⁵⁴ or conventional CPD.⁵⁰ Studies also found reduced drug costs for seminars with added role play and self-reflection activities (vs less interactive seminars)⁶⁵ and year-round (vs seasonal) activities.⁵⁵ Contrary to their authors’ hypotheses, studies found higher drug costs for group outreach (vs mailed materials)⁵⁴ and for 1-on-1 outreach augmented with vivid case studies (vs augmented with statistical information from a review article).³³

Nine studies^{31,32,38,46,48,56,59,60,63} (24%) reported costs and clinical (prescribing) outcomes for CPD compared with no education (a no intervention control group or baseline metrics). Eight of these studies^{32,38,46,48,56,59,60,63} reported an effectiveness outcome (rate of drug prescribing [n = 5], prescribing errors [potential drug-drug interactions or errors in writing; n = 2], prescription counts [n = 1]), thus allowing estimation of the incremental cost-effectiveness ratio (ICER). All these studies^{32,38,46,48,56,59,60,63} found improved prescribing outcomes with CPD, but costs were of course higher than for no education (Table 5). The dominance ranking matrix (eFigure 2 in the Supplement) classifies such studies as unclear dominance, and decisions require judgment of “whether intervention [is] preferable considering incremental cost effectiveness measures and priorities/willingness to pay.”²⁷

To permit cross-study comparisons, we standardized the ICERs using change thresholds that reflect a subjectively meaningful difference, namely a 10% change in prescribing rates, a 1% change in prescribing errors, or an absolute change of 100 prescriptions. The standardized ICER for a 10% improvement in prescribing rates ranged from $15 390 to $437 027 to train all program participants (5 studies). When further standardized to 1 physician trained, the ICERs ranged from $152 to $3441 per physician. Standardized ICERs for other outcomes ranged from $179 to $3258 per physician.

Four studies^31,32,38,63 (11%) reported information that allowed estimation of the net benefit (ie, health care cost savings minus educational costs) for CPD compared with no education. Three studies^31,32,38 found a favorable net benefit (Table 5). The fourth study⁶³ found that educational costs exceeded health care savings (net benefit of −$879 per physician trained), but extrapolation of benefits suggested a break-even point (ie, when savings [costs averted] equal educational costs) approximately 3.5 years after the intervention.

Studies that compare the cost-effectiveness of alternative educational approaches are vital to efforts to optimize CPD. Five studies^{31,48,54,59,60} (13%) (6 comparisons) compared the costs and clinical (prescribing) outcomes of 2 or more alternative CPD approaches (Table 5). All these studies^{31,48,54,59,60} found improved prescribing after enhancements intended to improve educational effectiveness but at the expense of increased education costs (ie, unclear dominance) (eFigure 2 in the Supplement). Specifically, 3 studies^48,54,59 (4 comparisons) found that 1-on-1 educational outreach was more effective but more expensive than group education or mailed materials (ICERs ranging from $18 to $4105 per physician trained). Two other studies^31,60 compared face-to-face instruction with mailed materials; one study⁶⁰ found that face-to-face instruction was more effective but more expensive (ICER of $3 per physician trained), and the other study³¹ found a net benefit of $221 per physician favoring face-to-face instruction. One additional study⁶⁶ compared 2 approaches to recruit participants to a CPD activity on drug prescribing and found that informal publicity through local organizations was more effective (higher attendance) and less expensive than a formal advertising campaign (ie, the informal approach dominated the formal).

This systematic review examined the costs and economic benefits of CPD for practicing physicians. Among studies^{30-58,61-65,67} that evaluated health care costs (drug costs), we found that CPD is associated with substantial cost savings. Among studies^{31,32,38,46-48,54,56,59,60,63,66} that evaluated educational costs, cost-effectiveness estimates varied widely, even after attempts to standardize the outcome measures. Along with expected differences in local contexts, educational approaches, and effectiveness outcomes, this heterogeneity arises from large differences in the accounting of educational costs. For example, only 4 studies accounted for 5 cost ingredients or more in their cost estimates, and 3 studies reported no discrete ingredients. Across all studies, we found that different CPD approaches can be associated with substantial differences in health care costs, educational costs, and incremental cost-effectiveness.

Studies reported information on participants (physicians and patients) inconsistently. Studies likewise frequently omitted details on ingredient (resource) quantification and pricing for educational costs (missing in >50%) and drug costs (missing in >25% of studies).

Previous reviews^6,9-14 of drug prescribing have suggested that outreach, audit and feedback, and multimodal educational interventions are consistently associated with improved drug prescribing (ie, effectiveness). Our review extends these findings by examining cost along with effectiveness and confirms that 1-on-1 educational outreach is effective but incurs higher cost. Audit and feedback was frequently used in the studies we identified, but study designs did not allow direct cost comparisons.

The findings of this review support 4 important messages. First, CPD was associated with substantially reduced drug costs (in 31 of 33 studies^{30-53,57,58,61-64,67}; median immediate savings of $79 373). Two studies^55,65 found increased costs, but in 1 study⁵⁶ this was intentional (increased prescribing of a drug with long-term health benefits), and in the other⁶⁵ there was a decrease in the prescription rate and cost of the target drug (antibiotics) despite an increase in overall drug costs. These exceptions underscore the importance of explicitly stating and justifying the intended impact (eg, fewer prescriptions, more prescriptions, wise stewardship, or cheaper drug choices) and illustrate that focused, immediate outcomes may differ from broad, delayed economic measures.

Second, our findings highlight differences in effectiveness and cost among different CPD approaches. Most notably, 1-on-1 educational outreach was more effective but more costly than comparators. Indeed, all cost-effectiveness studies^{,32,38,46,48,54,56,59,60,63} had unclear dominance (and the Dominance Ranking Matrix^27,29 [eFigure 2 in the Supplement] was unfortunately not very helpful in this synthesis). In the absence of dominance, local factors, resources, priorities, and values will influence local decisions. These findings underscore the importance of moving beyond questions of effectiveness alone (“What works?”) and adding economic data to inform judgments of value (“Is it worth the investment?”).^68-70

Third, our findings indicate that some ingredients are more important (ie, contribute more to total cost) than others. We observed a 50-fold difference in highest vs lowest mean cost per ingredient. The most expensive ingredients reflected time (opportunity cost of lost wages or productivity) invested by learners, administrators, and faculty, and facility costs. Indeed, itemized time expenses collectively represented 55% of all educational costs (Table 2), and the total would likely have been greater if all studies had itemized time. Acknowledging that these data are far from definitive and that specifics will vary for other interventions and topics, our findings offer guidance in prioritizing cost ingredients. We urge that time—both quantity and price—for learners, teachers, and administrators be measured and reported accurately and consistently in economic studies of education.

Fourth and most importantly, our findings amplify prior requests for more and better educational cost evaluations.^22,71,72 Our method to standardize educational costs, cost-effectiveness, and net benefits to a common time period, sample size, and level of effectiveness offers an example that others may choose to adopt when synthesizing economic data. However, this synthesis would have been facilitated by more complete accounting of educational costs,²² more consistent measurement of clinical outcomes (effectiveness and health care costs using standardized metrics and defined periods of follow-up), and better reporting^16,18,24,71 in the original studies. We encourage education researchers to measure and report costs more frequently, robustly, and completely.

This study has several strengths, including the robust literature search, duplicate review at all stages, and synthesis of key study findings using recommended approaches²⁷ and a variety of tabular summaries.

This review also has some limitations. This study was limited by poor reporting in many original studies, which impaired our ability to identify key elements of methods and results. In particular, data on the number of physicians trained or patients treated were reported inconsistently, such that it was not always possible to rescale data to a common unit that allowed straightforward synthesis or comparison across studies. In addition, both effectiveness and costs are sensitive to numerous design features, including sample size, outcome selection and measurement, and duration of follow-up (all of which varied widely). Robust standardization would require a direct association among cost, effectiveness, and scale (eg, sites, participants, and duration), which is unlikely.⁶⁸ Moreover, differences in local systems, funding, culture, and values will constrain local applications of cost-related outcomes regardless of rigor in the original data collection or meta-synthesis.^25,27,28,68 However, we propose that syntheses such as our can help establish boundaries of plausible outcomes (eg, approximate ranges for educational costs, health care costs averted, or net benefits) that decision makers can apply locally. Finally, our findings regarding what works to enhance CPD effectiveness were limited by the paucity of studies and the wide variation in research questions.

In this systematic review, CPD for drug prescribing was associated with reduced health care (drug) costs. The educational costs and cost-effectiveness of CPD varied widely. Several CPD instructional approaches (including educational outreach) were more effective but more costly than comparators.

Accepted for Publication: December 1, 2021.

Published: January 26, 2022. doi:10.1001/jamanetworkopen.2021.44973

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Cook DA et al. JAMA Network Open.

Corresponding Author: David A. Cook, MD, MHPE, Division of General Internal Medicine, Mayo Clinic College of Medicine, 200 First St SW, Mayo 17-W, Rochester, MN 55905 (cook.david33@mayo.edu).

Author Contributions: Dr Cook had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Cook, Stephenson, Maloney, Foo.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Cook, Stephenson, Wilkinson.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Cook.

Administrative, technical, or material support: Cook.

Supervision: Cook.

Conflict of Interest Disclosures: None reported.

Additional Contributions: Barbara L. Baasch-Thomas, BSN, MA, Mayo Clinic School of Continuous Professional Development, Mayo Clinic College of Medicine and Science, Rochester, Minnesota, screened studies for inclusion, and Larry J. Prokop, MLIS, Mayo Clinic Libraries, Mayo Clinic College of Medicine and Science, Rochester, Minnesota, helped create the search strategy. They were not compensated for this work beyond their usual salaries.