Evaluation of a Peer-to-Peer Data Transparency Intervention for Mohs Micrographic Surgery Overuse | Dermatology | JAMA Dermatology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Mean Stages per Case by Group
Mean Stages per Case by Group
Figure 2.  Individual Surgeon Change in Mean Stages per Case Among Outlier Surgeons
Individual Surgeon Change in Mean Stages per Case Among Outlier Surgeons
Table 1.  Preintervention Characteristics of Intervention and Control Surgeons Performing Mohs Micrographic Surgery by Outlier Status
Preintervention Characteristics of Intervention and Control Surgeons Performing Mohs Micrographic Surgery by Outlier Status
Table 2.  Change in Stages per Case by Outlier Status and Notification Group
Change in Stages per Case by Outlier Status and Notification Group
Table 3.  Multivariable Linear Regression Model for Factors Associated With Surgeon-Level Change in Mean Stages per Case
Multivariable Linear Regression Model for Factors Associated With Surgeon-Level Change in Mean Stages per Case
1.
Guy  GPG  Jr, Machlin  SR, Ekwueme  DU, Yabroff  KR.  Prevalence and costs of skin cancer treatment in the US, 2002-2006 and 2007-2011.  Am J Prev Med. 2015;48(2):183-187. doi:10.1016/j.amepre.2014.08.036PubMedGoogle ScholarCrossref
2.
Rogers  HW, Weinstock  MA, Harris  AR,  et al.  Incidence estimate of nonmelanoma skin cancer in the United States, 2006.  Arch Dermatol. 2010;146(3):283-287. doi:10.1001/archdermatol.2010.19PubMedGoogle ScholarCrossref
3.
Leibovitch  I, Huilgol  SC, Selva  D, Hill  D, Richards  S, Paver  R.  Cutaneous squamous cell carcinoma treated with Mohs micrographic surgery in Australia I: experience over 10 years.  J Am Acad Dermatol. 2005;53(2):253-260. doi:10.1016/j.jaad.2005.02.059PubMedGoogle ScholarCrossref
4.
Smeets  NWJ, Krekels  GAM, Ostertag  JU,  et al.  Surgical excision vs Mohs’ micrographic surgery for basal-cell carcinoma of the face: randomised controlled trial.  Lancet. 2004;364(9447):1766-1772. doi:10.1016/S0140-6736(04)17399-6PubMedGoogle ScholarCrossref
5.
Reeder  VJ, Gustafson  CJ, Mireku  K, Davis  SA, Feldman  SR, Pearce  DJ.  Trends in Mohs surgery from 1995 to 2010: an analysis of nationally representative data.  Dermatol Surg. 2015;41(3):397-403. doi:10.1097/DSS.0000000000000285PubMedGoogle ScholarCrossref
6.
Viola  KV, Rezzadeh  KS, Gonsalves  L,  et al.  National utilization patterns of Mohs micrographic surgery for invasive melanoma and melanoma in situ.  J Am Acad Dermatol. 2015;72(6):1060-1065. doi:10.1016/j.jaad.2015.02.1122PubMedGoogle ScholarCrossref
7.
Mosterd  K, Krekels  GAM, Nieman  FH,  et al.  Surgical excision versus Mohs’ micrographic surgery for primary and recurrent basal-cell carcinoma of the face: a prospective randomised controlled trial with 5-years’ follow-up.  Lancet Oncol. 2008;9(12):1149-1156. doi:10.1016/S1470-2045(08)70260-2PubMedGoogle ScholarCrossref
8.
Pugliano-Mauro  M, Goldman  G.  Mohs surgery is effective for high-risk cutaneous squamous cell carcinoma.  Dermatol Surg. 2010;36(10):1544-1553. doi:10.1111/j.1524-4725.2010.01576.xPubMedGoogle ScholarCrossref
9.
Viola  KV, Jhaveri  MB, Soulos  PR,  et al.  Mohs micrographic surgery and surgical excision for nonmelanoma skin cancer treatment in the Medicare population.  Arch Dermatol. 2012;148(4):473-477. doi:10.1001/archdermatol.2011.2456PubMedGoogle ScholarCrossref
10.
Wang  DM, Morgan  FC, Besaw  RJ, Schmults  CD.  An ecological study of skin biopsies and skin cancer treatment procedures in the United States Medicare population, 2000 to 2015.  J Am Acad Dermatol. 2018;78(1):47-53. doi:10.1016/j.jaad.2017.09.031PubMedGoogle ScholarCrossref
11.
Krishnan  A, Xu  T, Hutfless  S,  et al; American College of Mohs Surgery Improving Wisely Study Group.  Outlier practice patterns in Mohs micrographic surgery: defining the problem and a proposed solution.  JAMA Dermatol. 2017;153(6):565-570. doi:10.1001/jamadermatol.2017.1450PubMedGoogle ScholarCrossref
12.
Sebaratnam  DF, Choy  B, Lee  M, Paver  R, Fernández Peñas  P.  Direct cost-analysis of Mohs micrographic surgery and traditional excision for basal cell carcinoma at initial margin clearance.  Dermatol Surg. 2016;42(5):633-638. doi:10.1097/DSS.0000000000000756PubMedGoogle ScholarCrossref
13.
Makary  MA, Mehta  A, Xu  T.  Improving wisely using physician metrics.  Am J Med Qual. 2018;33(1):103-105. doi:10.1177/1062860617704504PubMedGoogle ScholarCrossref
14.
Research Data Assistance Center. CMS Virtual Research Data Center (VRDC). https://www.resdac.org/cms-virtual-research-data-center-vrdc. Accessed April 4, 2019.
15.
Research Data Assistance Center. Medicare Data on Provider Practice and Specialty (MD-PPAS): 2008-2016. https://www.resdac.org/cms-data/files/md-ppas. Accessed April 4, 2019.
16.
US Centers for Medicare & Medicaid Services. Physician Compare datasets. https://data.medicare.gov/data/physician-compare. Accessed April 4, 2019.
17.
Loehrer  AP, Chang  DC, Scott  JW,  et al.  Association of the Affordable Care Act Medicaid expansion with access to and quality of care for surgical conditions.  JAMA Surg. 2018;153(3):e175568. doi:10.1001/jamasurg.2017.5568PubMedGoogle ScholarCrossref
18.
Lyu  H, Xu  T, Brotman  D,  et al.  Overtreatment in the United States.  PLoS One. 2017;12(9):e0181970. doi:10.1371/journal.pone.0181970PubMedGoogle ScholarCrossref
19.
Porter  ME.  What is value in health care?  N Engl J Med. 2010;363(26):2477-2481. doi:10.1056/NEJMp1011024PubMedGoogle ScholarCrossref
20.
Emanuel  EJ, Ubel  PA, Kessler  JB,  et al.  Using behavioral economics to design physician incentives that deliver high-value care.  Ann Intern Med. 2016;164(2):114-119. doi:10.7326/M15-1330PubMedGoogle ScholarCrossref
21.
Rosenberg  A, Agiro  A, Gottlieb  M,  et al.  Early trends among seven recommendations from the Choosing Wisely campaign.  JAMA Intern Med. 2015;175(12):1913-1920. doi:10.1001/jamainternmed.2015.5441PubMedGoogle ScholarCrossref
22.
Mostofian  F, Ruban  C, Simunovic  N, Bhandari  M.  Changing physician behavior: what works?  Am J Manag Care. 2015;21(1):75-84.PubMedGoogle Scholar
23.
Ivers  N, Jamtvedt  G, Flottorp  S,  et al.  Audit and feedback: effects on professional practice and healthcare outcomes.  Cochrane Database Syst Rev. 2012;(6):CD000259.PubMedGoogle Scholar
24.
Forsetlund  L, Bjørndal  A, Rashidian  A,  et al.  Continuing education meetings and workshops: effects on professional practice and health care outcomes.  Cochrane Database Syst Rev. 2009;(2):CD003030.PubMedGoogle Scholar
25.
Flodgren  G, Eccles  MP, Shepperd  S, Scott  A, Parmelli  E, Beyer  FR.  An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes.  Cochrane Database Syst Rev. 2011;(7):CD009255.PubMedGoogle Scholar
26.
Giguère  A, Légaré  F, Grimshaw  J,  et al.  Printed educational materials: effects on professional practice and healthcare outcomes.  Cochrane Database Syst Rev. 2012;10:CD004398.PubMedGoogle Scholar
27.
Flodgren  G, Parmelli  E, Doumit  G,  et al.  Effectiveness of the use of local opinion leaders to promote evidence-based practice and improving patient outcomes.  Cochrane Database Syst Rev. 2011;(8):CD000125. doi:10.1002/14651858.CD000125.pub4Google Scholar
28.
Sirovich  BE, Woloshin  S, Schwartz  LM.  Too little? too much? primary care physicians’ views on US health care: a brief report.  Arch Intern Med. 2011;171(17):1582-1585. doi:10.1001/archinternmed.2011.437PubMedGoogle ScholarCrossref
29.
Graham  ID, Logan  J, Harrison  MB,  et al.  Lost in knowledge translation: time for a map?  J Contin Educ Health Prof. 2006;26(1):13-24. doi:10.1002/chp.47PubMedGoogle ScholarCrossref
30.
Althabe  F, Buekens  P, Bergel  E,  et al; Guidelines Trial Group.  A behavioral intervention to improve obstetrical care.  N Engl J Med. 2008;358(18):1929-1940. doi:10.1056/NEJMsa071456PubMedGoogle ScholarCrossref
31.
Gagliardi  AR, Kothari  A, Graham  ID.  Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know.  J Epidemiol Community Health. 2017;71(2):105-106. doi:10.1136/jech-2016-207743PubMedGoogle ScholarCrossref
32.
Colquhoun  HL, Squires  JE, Kolehmainen  N, Fraser  C, Grimshaw  JM.  Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review.  Implement Sci. 2017;12(1):30. doi:10.1186/s13012-017-0560-5PubMedGoogle ScholarCrossref
33.
Brehaut  JC, Colquhoun  HL, Eva  KW,  et al.  Practice feedback interventions: 15 suggestions for optimizing effectiveness.  Ann Intern Med. 2016;164(6):435-441. doi:10.7326/M15-2248PubMedGoogle ScholarCrossref
34.
Sikkens  JJ, van Agtmael  MA, Peters  EJG,  et al.  Behavioral approach to appropriate antimicrobial prescribing in hospitals: the Dutch Unique Method for Antimicrobial Stewardship (DUMAS) participatory intervention study.  JAMA Intern Med. 2017;177(8):1130-1138. doi:10.1001/jamainternmed.2017.0946PubMedGoogle ScholarCrossref
35.
Deci  EL, Ryan  RM.  The empirical exploration of intrinsic motivational processes.  Adv Exp Soc Psychol. 1980;13:39-80. doi:10.1016/S0065-2601(08)60130-6Google Scholar
36.
McDermott  L, Yardley  L, Little  P, Ashworth  M, Gulliford  M; eCRT Research Team.  Developing a computer delivered, theory based intervention for guideline implementation in general practice.  BMC Fam Pract. 2010;11:90. doi:10.1186/1471-2296-11-90PubMedGoogle ScholarCrossref
37.
Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement.  JAMA. 1999;282(15):1458-1465. doi:10.1001/jama.282.15.1458PubMedGoogle ScholarCrossref
38.
Council  ML, Alam  M, Gloster  HM  Jr,  et al.  Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry.  J Am Acad Dermatol. 2016;74(4):739-745.PubMedGoogle ScholarCrossref
39.
Leitenberger  JJ, Rogers  H, Chapman  JC,  et al.  Defining recurrence of nonmelanoma skin cancer after Mohs micrographic surgery: report of the American College of Mohs Surgery Registry and Outcomes Committee.  J Am Acad Dermatol. 2016;75(5):1022-1031.PubMedGoogle ScholarCrossref
Original Investigation
May 5, 2019

Evaluation of a Peer-to-Peer Data Transparency Intervention for Mohs Micrographic Surgery Overuse

Author Affiliations
  • 1The Skin Surgery Center, Winston Salem, North Carolina
  • 2Department of Plastic and Reconstructive Surgery, Wake Forest University School of Medicine, Winston Salem, North Carolina
  • 3Department of Surgery, Johns Hopkins University, Baltimore, Maryland
  • 4Department of Epidemiology, Johns Hopkins University, Baltimore, Maryland
  • 5Department of Medicine, Johns Hopkins University, Baltimore, Maryland
  • 6Department of Dermatology, University of Oklahoma, Oklahoma City
  • 7Cleveland Clinic, Cleveland, Ohio
  • 8Penn State Health, Hershey, Pennsylvania
  • 9University of Cincinnati Hospital, Cincinnati, Ohio
  • 10Bennett Surgery Center, Santa Monica, California
  • 11Geisinger Medical Center, Danville, Pennsylvania
JAMA Dermatol. 2019;155(8):906-913. doi:10.1001/jamadermatol.2019.1259
Key Points

Question  Can the use of benchmarked peer-to-peer reports reduce overuse in Mohs micrographic surgery (MMS)?

Findings  In this nonrandomized controlled intervention study including 2329 US surgeons, sharing individual reports of MMS practice patterns with physicians resulted in an immediate and sustained reduction in mean stages per case among outlier physicians and an associated cost savings compared with physicians who did not receive these reports.

Meaning  A nonpunitive, confidential audit-and-feedback intervention may represent an opportunity to reduce overuse of MMS among physicians identified as outliers by their clinical specialty society.

Abstract

Importance  Mohs micrographic surgery (MMS) is a skin cancer treatment that uses staged excisions based on margin status. Wide surgeon-level variation exists in the mean number of staged resections used to treat a tumor, resulting in a cost disparity and question of appropriateness.

Objective  To evaluate the effectiveness of a behavioral intervention aimed at reducing extreme overuse in MMS, as defined by the specialty society, by confidentially sharing stages-per-case performance data with individual surgeons benchmarked to their peers nationally.

Design, Setting, and Participants  This nonrandomized controlled intervention study included 2329 US surgeons who performed MMS procedures from January 1, 2016, to March 31, 2018. Physicians were identified using a 100% capture of Medicare Part B claims. The intervention group included physicians affiliated with the American College of Mohs Surgery, and the control group included physicians not affiliated with the American College of Mohs Surgery.

Interventions  Individualized performance reports were delivered to all outlier surgeons, defined by the specialty society as those with mean stages per case 2 SDs above the mean, and inlier surgeons in the intervention group.

Main Outcomes and Measures  The primary outcome was surgeon-level change in mean stages per case between the prenotification (January 2016 to January 2017) and postnotification (March 2017 to March 2018) periods. A multivariable linear regression model was used to evaluate the association of notification with this surgeon-level outcome. The surgeon-level metric of mean stages per case was not risk adjusted. The mean Medicare cost savings associated with changes in practice patterns were calculated.

Results  Of the 2329 included surgeons, 1643 (70.5%) were male and 2120 (91.0%) practiced in metropolitan areas. In the intervention group (n = 1045), 53 surgeons (5.1%) were outliers; in the control group (n = 1284), 87 surgeons (6.8%) were outliers. Among the outliers in the intervention group, 44 (83%) demonstrated a reduction in mean stages per case compared with 60 outliers in the control group (69%; difference, 14%; 95% CI of difference, −1 to 27; P = .07). There was a mean stages-per-case reduction of 12.6% among outliers in the intervention group compared with 9.0% among outliers in the control group, and outliers in the intervention group had an adjusted postintervention differential decrease of 0.14 stages per case (95% CI, −0.19 to −0.09; P = .002). The total administrative cost of the intervention program was $150 000, and the estimated reduction in Medicare spending was $11.1 million.

Conclusions and Relevance  Sharing personalized practice pattern data with physicians benchmarked to their peers can reduce overuse of MMS among outlier physicians.

Introduction

Skin cancer is the most common malignancy in the United States, accounting for more than 5.4 million cases annually at an estimated cost of more than $8.1 billion to the US health care system.1,2 Mohs micrographic surgery (MMS) is a surgical approach that treats skin cancer using a potential series of staged excisions determined by intraoperative histologic examination. Mohs micrographic surgery is an effective and often curative treatment for cutaneous basal and squamous cell carcinoma.3,4 In addition, the procedure is highly efficient, since the surgeon serves the simultaneous roles of oncologic surgeon, pathologist, and generally also reconstructive surgeon. Mohs micrographic surgery use has increased over the last decade owing to the procedure’s superior outcomes, its expanding application to melanoma, an emphasis of MMS in training programs, and increased rates of skin cancer.5-12

We have previously described wide variation among surgeons performing MMS in the mean number of stages used to resect a skin cancer.11 Potential overuse may stem from insufficient technical expertise or the current fee-for-service payment model, which compensates surgeons based on the number of stages performed per tumor, creating a perverse incentive for a surgeon to use an excessive number of staged resections to remove a lesion. Overuse of stages per case burdens patients with unnecessary surgical resections and taxes the health care system with avoidable costs.11 For these reasons, the American College of Mohs Surgery (ACMS), the largest specialty society for MMS, considers a surgeon’s annual mean stages per case as an appropriateness metric for Mohs surgery of the head and neck. To address this disparity in appropriateness and cost, we designed an audit-and-feedback intervention aimed at reducing extreme overuse in MMS by confidentially sharing stages-per-case performance data with individual surgeons benchmarked to their peers nationally for similar head and neck lesions. We examined the association of the intervention with a surgeon’s practice pattern and the overall cost to Medicare.

Methods
Study Design, Setting, and Participants

We conducted a nonrandomized controlled intervention study of surgeons in the United States who billed MMS procedures to Medicare. The intervention group consisted of all ACMS members, and they received an individualized notification that reported their annual mean stages per case benchmarked to national peer performance. The control group consisted of non-ACMS members who did not receive a notification. The prenotification performance period was January 1, 2016, through January 31, 2017. The intervention group received an individualized notification in February 2017. The postnotification period was March 1, 2017, through March 31, 2018. We excluded surgeons who performed 10 or fewer MMS procedures in either the prenotification or postnotification period, as required by our user agreement with Medicare designed to protect patient identity. This study received waived authorization of institutional review board approval by the Johns Hopkins Medical Institutions Institutional Review Board and informed consent was not needed because this study was based on administrative claims data only and did not involve active recruitment of study participants.

Expert Consensus on Best Practices

The Improving Wisely collaborative is a Robert Wood Johnson Foundation–funded initiative that uses national data to identify overuse patterns for various diagnostic, medical, and surgical procedures. In the Improving Wisely model, an overuse metric is developed by expert consensus, and Medicare data are queried to identify physician outliers and inliers.13 Confidential, benchmarked, personalized audit-and-feedback data reports are shared with physicians in an effort to change physician behavior and reduce overuse and cost waste.

As part of the Improving Wisely collaborative, ACMS created a Physician Engagement Council of highly respected national experts and clinical leaders, including the association’s executive leadership as elected by the ACMS membership, and representation from large, small, urban, and rural practice settings. Through this workgroup, the ACMS developed and endorsed a consensus threshold of what constitutes an outlier physician practice pattern of overuse. This consensus definition of an outlier was defined as a physician’s annual mean stages per case for MMS for skin cancers of the head, neck, genitalia, hands, and feet being greater than 2 SDs above the national mean.11 This metric was developed specifically for this study, taking into consideration our inability to control for surgeon case mix.

Intervention

In the month prior to the notification intervention, national leaders in the specialty introduced the Improving Wisely collaborative and objectives in a printed and electronic newsletter. In addition, surgeons in the intervention cohort also received an email notification detailing the upcoming confidential, personalized, audit-and-feedback report 2 weeks in advance of the mailing. All physicians in the intervention cohort were mailed their individualized performance report using the most recent addresses listed with their professional association. Accompanying the individual performance data, a cover letter signed by national peer leaders in the specialty described the benchmarking approach and the goals of the program.

Each report listed the individual surgeon’s mean number of stages per case, the national mean of stages per case, and the consensus metric definition and included a cover letter explaining the Improving Wisely program and the intervention (eAppendix in the Supplement). A graphical representation of an individual surgeon’s performance was depicted using an arrow pointing to the surgeon’s position on the national distribution. Surgeons were classified into 3 groups that were differentiated by color; green represented surgeons considered inliers (ie, performance fell within the acceptable range of clinical variation), yellow represented surgeons whose mean stages per case fell within 1.0 to 2.0 SDs above the national mean, and red represented surgeons considered outliers (ie, performance was more than 2.0 SDs above the national mean). Surgeons who fell in the lattermost category also had the word “outlier” appear on their report next to their mean stages per case along with their percentile rank.

Communications from the ACMS president to the membership at the time of notification described the Improving Wisely reports and invited surgeons to contact the leadership with any questions or concerns. Confidentiality, collegiality, respect, and a spirit of quality improvement were emphasized in all communications and announcements about the study. A follow-up letter was sent to all outlier members offering confidential mentoring and educational resources, including a slide deck on surgical and laboratory techniques and select readings to improve one’s proficiency.

Data Sources

Mean stages per case for head and neck procedures were calculated from the Physician Supplier Part B claims file accessed via the US Centers for Medicare & Medicaid Services Data Center.14 This file includes all fee-for-service claims submitted on a CMS-1500 claims form and includes the claims from physicians and free-standing facilities. We identified the first stage by Current Procedural Terminology (CPT) code 17311 and additional stages by CPT code 17312. The mean stages per case were calculated by dividing the sum of the first and additional stages by the number of first stages (ie, [total CPT codes 17311 + 17312]/CPT code 17311).

We identified surgeon characteristics, including sex, practice location, and region, from the Medicare Data on Provider Practice and Specialty.15 We obtained surgeon year of medical school graduation from the Physician Compare National Downloadable File.16 Surgeon information was linked to claims data using National Provider Identifier numbers.

Outcomes

The primary outcome was surgeon-level change in mean stages per case in the post-notification period compared with the prenotification period. Cost was defined as the Medicare-allowed amount in US dollars paid to surgeons directly for MMS (professional fee only). The cost savings for each outlier was calculated by multiplying their number of cases performed during the postnotification period by their change in mean stages per case and by their mean reimbursed amount for each additional stage (CPT code 17312) during the postnotification period. We calculated the mean cost savings per case for each group by dividing the total cost savings by the total case volume in the postnotification period.

Statistical Analyses

We compared physician characteristics between intervention and control groups using 2-sample t tests for continuous variables and χ2 tests for categorical variables. We used paired t tests to compare surgeons’ prenotification and postnotification mean stages per case within each arm of the study. We used independent 2-sample t tests to compare surgeons’ change in mean stages per case between the intervention and control groups stratified by outlier status. A power calculation was not performed because our study included the entire population of eligible MMS surgeons who billed to Medicare in the specified time period.

We used a multivariable linear regression model to evaluate factors associated with surgeon-level change in mean stages per case.16,17 We modeled surgeons’ postnotification minus prenotification change in mean stages per case as a continuous outcome, with the independent variables being surgeon sex, years since medical school graduation, practice region and location, case volume in the prenotification period, outlier status, notification status, and an interaction term for outlier × notification status. All P values were from 2-sided tests, and results were deemed statistically significant at a P value less than .05. All statistical analyses were performed using SAS Enterprise Guide version 7.1 (SAS Institute).

Results

The number of eligible MMS surgeons in the United States was 2329 (Table 1). There were 140 outliers (53 in the intervention group and 87 in the control group) and 2189 inliers (992 in the intervention group and 1197 in the control group). The outliers in the intervention group paralleled the outliers in the control group in regards to mean stages per case in the prenotification period, male predominance, and metropolitan location of practice (Table 1). Compared with outliers in the control group, intervention group outliers had fewer years in practice (median time, 16 vs 22 years), a higher procedural volume (median prenotification volume, 423 vs 102 cases), and were more likely to practice in the northeast region of the United States (38% [20 of 53] vs 15% [13 of 87]).

Following the intervention, outliers in the intervention group demonstrated a decrease in mean stages per case in the first quarter of 2017, which was sustained through the first quarter of 2018 (Figure 1). Outliers in the control group demonstrated a smaller decrease in mean stages per case beginning in the second quarter of 2017, which continued for the remainder of the study period. Following the intervention, the overall mean stages per case decreased from 2.55 to 2.31 among outliers in the intervention group and decreased from 2.56 to 2.46 among outliers in the control group. Outlier surgeons in the intervention group exhibited a mean reduction of 0.26 stages per case, whereas outliers in the control group exhibited a mean reduction of 0.11 stages per case, corresponding to a differential reduction of 0.14 stages per case (95% CI, −0.23 to −0.05; P = .002) (Table 2). Of the 53 intervention group outliers notified, 44 (83%) demonstrated a reduction in mean stages per case in the postnotification period (median [range] reduction, 0.21 [0.05-1.22] stages per case). In contrast, 60 of 87 control group outliers (69%) demonstrated a decrease in mean stages per case in the postnotification period (median [range] reduction, 0.20 [0.002-0.94] stages per case), a difference of 14% in rate of response (95% CI of difference, −1 to 27; P = .07) (Figure 2). Among the 53 intervention group outliers, 9 surgeons (17%) demonstrated a median (range) increase of 0.06 (0.02-0.16) stages per case compared with 27 control group outliers (31%) with a median (range) increase of 0.08 (0.001-0.45) stages per case. Inlier surgeons in the intervention group exhibited a mean reduction of 0.03 stages per case, whereas inliers in the control group exhibited a mean reduction of 0.01 stages per case, corresponding to a differential reduction of 0.01 stages per case (95% CI, −0.03 to −0.003; P = .01) (Table 2).

In the multivariable linear regression model that adjusted for baseline surgeon characteristics (Table 3), the intervention was associated with an additional 0.14 reduction in mean stages per case (95% CI, −0.19 to −0.09; P < .001) among outliers and an additional 0.02 reduction in mean stages per case (95% CI, −0.03 to −0.001; P = .03) among inliers. The intervention was associated with greater change among outliers compared with inliers (coefficient for interaction, −0.13; 95% CI, −0.18 to −0.07). Holding other factors equal, surgeons practicing for less than 10 years since graduation decreased a mean of 0.03 stages per case (95% CI, −0.05 to −0.01) more than those practicing for 30 years or more since graduation. Surgeon sex, practice region and location, and case volume were not significantly associated with a change in mean stages per case.

Excluding research salaries, approximately $150 000 ($144 per surgeon) was used to cover the costs of the intervention. Using the mean stages per case in the prenotification period as a reference, surgeons’ stage reduction in the postnotification period was associated with a total cost savings of $11 143 882 to Medicare. This included a savings of $2 458 672 by the outliers in the intervention group and $788 262 by the outliers in the control group as well as a savings of $5 114 177 by the intervention group inliers and $2 782 771 by the control group inliers. The mean cost savings per case was $95 for intervention group outliers vs $46 for control group outliers and $11 for intervention group inliers vs $10 for control group inliers.

Discussion

Engaging physicians to promote behavior change around best practices has been a significant barrier to reducing waste in health care.13,18-22 The emerging field of implementation science seeks to address this problem and has categorized quality improvement interventions by their approach. These categories include audit and feedback, use of local opinion leaders, reminders, and educational sessions.23-27 Of these approaches, interventions that use opinion leaders and those that use audit and feedback have been observed to have the greatest association with physician behavior change. In a 2012 meta-analysis of 70 studies using the audit-and-feedback approach,23 the Cochrane Effective Practice and Organization of Care group described a 1% to 4% change in desired physician behavioral outcomes. Similarly, a 2011 Effective Practice and Organization of Care review of 18 cluster randomized clinical trials involving 296 hospitals27 found that using opinion leaders in an intervention results in 9% to 14% change in desired behavior.

We combined 2 established approaches of using audit and feedback and opinion leaders, which resulted in an immediate and sustained reduction in physician behavior around low-value care in MMS. Among outliers in the intervention group, 83% demonstrated a reduction in mean stages per case compared with 69% of outliers in the control group. There was a mean reduction in stages per case of 12.6% among outliers in the intervention group compared with 9.0% among outliers in the control group. Adjusted for physician baseline characteristics, the intervention was associated with an additional reduction of 0.14 stages per case (95% CI, −0.19 to −0.09; P < .001) among outliers. We posit that the improvement we observed in this study may be attributed to 4 elements characteristic of the intervention: (1) endorsement of an overuse metric by a clinical specialty society, (2) awareness and education by the specialty society around overuse, (3) the use of an achievable target, and (4) a confidential, nonpunitive audit-and-feedback comparison process.

The potential latent crossover effect observed among control group outliers may be attributed to 2 factors. First, the educational efforts and increased awareness efforts by the specialty society at the time of the intervention may have influenced surgeons in the control group. We believe that having peers from the specialty association prepare members in the intervention group to receive their notification reports was critical to surgeons being receptive to them. Second, clinicians in the intervention group may have had an impact on control group participants through personal conversations or a perception of an increased culture of accountability fostered by the initiative. Further, we observed a small yet statistically significant decrease in stages per case among inliers in the intervention group, suggesting that while outliers were more likely to demonstrate a greater reduction in mean stages per case, inliers may have also modified their behavior. Notably, even a slight decrease in stages per case in the inlier group could result in a significant cost savings, given that outlier volume only comprises a small percentage of all MMS procedures.

Physician demand for benchmarking data has been described to be strong. A 2011 study of primary physicians28 revealed that 95% of physicians surveyed believe there is excessive variation in care, and 76% are interested to learn how their individual practice patterns compare with their peers. We posit these views are shared by other specialist groups and hypothesize that additional iterations of Improving Wisely report dissemination could elicit even greater reductions in overuse.29

This study and its approach through the Improving Wisely model may have implications for broader quality improvement and waste reduction efforts in health care.13 We have identified critical factors that are likely needed for subsequent replication of this design. First, as a prerequisite, the intervention requires a well-respected peer physician leadership group to endorse a metric and the broader initiative a priori.30,31 In the present study, the physician engagement council and ACMS endorsed the overuse measure and broader program. Second, a clear achievable benchmark that is supported by peer leaders in the field is required, with a consensus threshold of what constitutes outlier practice patterns. In this study, experts created a consensus threshold of 2 SDs above the national mean. Third, the addition of educational and training resources as intervention components may be valuable to help outliers seeking tools to improve. Finally, the manner in which reports are messaged and delivered to the physician is important.32 Our methods were consistent with audit-and-feedback delivery best practices.33 Specifically, we ensured the reports were easily accessible, short in length and easy to interpret (ie, reduced cognitive load), and provided a combination of text and visual feedback.33 The reports were nonpunitive and confidential, provided clear targets for the desired behavior, were endorsed by respected leaders and governing bodies, provided repeated messaging and communication to targeted surgeons, and emphasized a behavior that was in the control of individual surgeons. This concept of autonomy is critical, with data suggesting that behavior change effects are stronger if an individual believes they are acting autonomously rather than simply responding to external influences.34-37

Limitations

This study has several limitations. First, while claims data are powerful for benchmarking and not subject to reporting bias, they do not provide detailed clinical data, such as patient or tumor characteristics. While our proposed metric adopted an appropriateness range based on expert consensus regarding a surgeons’ typical case mix, it is possible that some outliers’ high mean stages per case might be attributed to a high percentage of large and complex cases and/or high-risk patients in their practice. Despite this, the intervention group still showed a greater reduction in mean stages per case. Additional data sources would expand the analysis beyond the Medicare population. Detailed registries, such as the ACMS Clinical Data Registry (MohsAIQ) or the American Academy of Dermatology DataDerm Registry, may facilitate risk-adjusted comparisons and may enable individual physicians to examine their mean number of stages across all payers and compare themselves to benchmarks more easily in the future. Additional research will be needed to determine whether any unintended negative consequences resulted from the reduction in mean stages per case, such as tumor recurrence. Second, our study intervened on a single metric, which addressed only 1 aspect of appropriateness in MMS. However, additional metrics (eg, appropriate use of MMS for trunk and extremities, complications, and recurrences) can and should be explored.38,39 Further, we emphasize that the metric used to identify outliers was developed for the purpose of this study, in consideration of the claims data limitations. Future studies should revisit these benchmarks and provide adjustments for those clinicians with extensive case mix. Third, to facilitate intervention delivery, a nonrandomized design was used. While we adjusted for some physician characteristics, differences between surgeons in the intervention and control groups, including history of fellowship training and clinical practice setting, could lead to residual confounding. Future quality improvement initiatives should aim to target a representative sample of surgeons performing MMS. Fourth, it is unknown which educational resources were used by intervention group outliers and which resources may have contributed to the improvement observed. The sustainability and durable impact of this intervention should be explored in the long term.

Conclusions

We observed an immediate and sustained reduction in MMS procedures using an intervention that integrated the use of experts, a specialty association partnership, and educational resources along with individualized, confidential data report cards that provided a benchmark for peer comparison. The relatively low cost of this intervention relative to its resultant cost savings suggests that application to other areas of medicine could yield larger savings to the health care system. In an era where reducing low-value care is increasingly paramount given the escalating costs of health care, this intervention has the promise to provide a high return on investment.

Back to top
Article Information

Accepted for Publication: April 14, 2019.

Corresponding Author: John G. Albertini, MD, The Skin Surgery Center, 1450 Professional Park Dr, Ste 150, Winston Salem, NC 27103 (john.albertini@skinsurgerycenter.net).

Published Online: May 5, 2019. doi:10.1001/jamadermatol.2019.1259

Author Contributions: Dr Makary had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Albertini, Hutfless, Stasko, Vidimos, Coldiron, Bennett, Marks, Overton, Bruhn, Xu, Krishnan, Makary.

Acquisition, analysis, or interpretation of data: Albertini, Wang, Fahim, Hutfless, Stasko, Vidimos, Leshin, Billingsley, Park, Bruhn, Xu, Krishnan.

Drafting of the manuscript: Albertini, Wang, Fahim, Hutfless, Park, Makary.

Critical revision of the manuscript for important intellectual content: Albertini, Wang, Fahim, Hutfless, Stasko, Vidimos, Leshin, Billingsley, Coldiron, Bennett, Marks, Overton, Bruhn, Xu, Krishnan.

Statistical analysis: Wang, Hutfless, Park, Xu, Krishnan.

Obtained funding: Albertini, Makary.

Administrative, technical, or material support: Fahim, Vidimos, Bruhn, Makary.

Study supervision: Albertini, Fahim, Bennett, Xu, Makary.

Conflict of Interest Disclosures: Dr Albertini has received grants and nonfinancial support from the American College of Mohs Surgery as well as personal fees from Novascan for serving on a scientific advisory board and serves on the board of directors and is a shareholder of QualDerm Partners. Dr Leshin has received personal fees from Novascan for serving on a scientific advisory board and is a shareholder of QualDerm Partners. Dr Overton is supported by grant 5T32CA126607-10 from the National Institutes of Health. Dr Makary has received grants from the Robert Wood Johnson Foundation. No other disclosures were reported.

Funding/Support: This study was funded by a grant from the Robert Wood Johnson Foundation.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Meeting Presentation: This paper was presented at the 2019 Annual Meeting of the American College of Mohs Surgery; May 5, 2019; Baltimore, MD.

References
1.
Guy  GPG  Jr, Machlin  SR, Ekwueme  DU, Yabroff  KR.  Prevalence and costs of skin cancer treatment in the US, 2002-2006 and 2007-2011.  Am J Prev Med. 2015;48(2):183-187. doi:10.1016/j.amepre.2014.08.036PubMedGoogle ScholarCrossref
2.
Rogers  HW, Weinstock  MA, Harris  AR,  et al.  Incidence estimate of nonmelanoma skin cancer in the United States, 2006.  Arch Dermatol. 2010;146(3):283-287. doi:10.1001/archdermatol.2010.19PubMedGoogle ScholarCrossref
3.
Leibovitch  I, Huilgol  SC, Selva  D, Hill  D, Richards  S, Paver  R.  Cutaneous squamous cell carcinoma treated with Mohs micrographic surgery in Australia I: experience over 10 years.  J Am Acad Dermatol. 2005;53(2):253-260. doi:10.1016/j.jaad.2005.02.059PubMedGoogle ScholarCrossref
4.
Smeets  NWJ, Krekels  GAM, Ostertag  JU,  et al.  Surgical excision vs Mohs’ micrographic surgery for basal-cell carcinoma of the face: randomised controlled trial.  Lancet. 2004;364(9447):1766-1772. doi:10.1016/S0140-6736(04)17399-6PubMedGoogle ScholarCrossref
5.
Reeder  VJ, Gustafson  CJ, Mireku  K, Davis  SA, Feldman  SR, Pearce  DJ.  Trends in Mohs surgery from 1995 to 2010: an analysis of nationally representative data.  Dermatol Surg. 2015;41(3):397-403. doi:10.1097/DSS.0000000000000285PubMedGoogle ScholarCrossref
6.
Viola  KV, Rezzadeh  KS, Gonsalves  L,  et al.  National utilization patterns of Mohs micrographic surgery for invasive melanoma and melanoma in situ.  J Am Acad Dermatol. 2015;72(6):1060-1065. doi:10.1016/j.jaad.2015.02.1122PubMedGoogle ScholarCrossref
7.
Mosterd  K, Krekels  GAM, Nieman  FH,  et al.  Surgical excision versus Mohs’ micrographic surgery for primary and recurrent basal-cell carcinoma of the face: a prospective randomised controlled trial with 5-years’ follow-up.  Lancet Oncol. 2008;9(12):1149-1156. doi:10.1016/S1470-2045(08)70260-2PubMedGoogle ScholarCrossref
8.
Pugliano-Mauro  M, Goldman  G.  Mohs surgery is effective for high-risk cutaneous squamous cell carcinoma.  Dermatol Surg. 2010;36(10):1544-1553. doi:10.1111/j.1524-4725.2010.01576.xPubMedGoogle ScholarCrossref
9.
Viola  KV, Jhaveri  MB, Soulos  PR,  et al.  Mohs micrographic surgery and surgical excision for nonmelanoma skin cancer treatment in the Medicare population.  Arch Dermatol. 2012;148(4):473-477. doi:10.1001/archdermatol.2011.2456PubMedGoogle ScholarCrossref
10.
Wang  DM, Morgan  FC, Besaw  RJ, Schmults  CD.  An ecological study of skin biopsies and skin cancer treatment procedures in the United States Medicare population, 2000 to 2015.  J Am Acad Dermatol. 2018;78(1):47-53. doi:10.1016/j.jaad.2017.09.031PubMedGoogle ScholarCrossref
11.
Krishnan  A, Xu  T, Hutfless  S,  et al; American College of Mohs Surgery Improving Wisely Study Group.  Outlier practice patterns in Mohs micrographic surgery: defining the problem and a proposed solution.  JAMA Dermatol. 2017;153(6):565-570. doi:10.1001/jamadermatol.2017.1450PubMedGoogle ScholarCrossref
12.
Sebaratnam  DF, Choy  B, Lee  M, Paver  R, Fernández Peñas  P.  Direct cost-analysis of Mohs micrographic surgery and traditional excision for basal cell carcinoma at initial margin clearance.  Dermatol Surg. 2016;42(5):633-638. doi:10.1097/DSS.0000000000000756PubMedGoogle ScholarCrossref
13.
Makary  MA, Mehta  A, Xu  T.  Improving wisely using physician metrics.  Am J Med Qual. 2018;33(1):103-105. doi:10.1177/1062860617704504PubMedGoogle ScholarCrossref
14.
Research Data Assistance Center. CMS Virtual Research Data Center (VRDC). https://www.resdac.org/cms-virtual-research-data-center-vrdc. Accessed April 4, 2019.
15.
Research Data Assistance Center. Medicare Data on Provider Practice and Specialty (MD-PPAS): 2008-2016. https://www.resdac.org/cms-data/files/md-ppas. Accessed April 4, 2019.
16.
US Centers for Medicare & Medicaid Services. Physician Compare datasets. https://data.medicare.gov/data/physician-compare. Accessed April 4, 2019.
17.
Loehrer  AP, Chang  DC, Scott  JW,  et al.  Association of the Affordable Care Act Medicaid expansion with access to and quality of care for surgical conditions.  JAMA Surg. 2018;153(3):e175568. doi:10.1001/jamasurg.2017.5568PubMedGoogle ScholarCrossref
18.
Lyu  H, Xu  T, Brotman  D,  et al.  Overtreatment in the United States.  PLoS One. 2017;12(9):e0181970. doi:10.1371/journal.pone.0181970PubMedGoogle ScholarCrossref
19.
Porter  ME.  What is value in health care?  N Engl J Med. 2010;363(26):2477-2481. doi:10.1056/NEJMp1011024PubMedGoogle ScholarCrossref
20.
Emanuel  EJ, Ubel  PA, Kessler  JB,  et al.  Using behavioral economics to design physician incentives that deliver high-value care.  Ann Intern Med. 2016;164(2):114-119. doi:10.7326/M15-1330PubMedGoogle ScholarCrossref
21.
Rosenberg  A, Agiro  A, Gottlieb  M,  et al.  Early trends among seven recommendations from the Choosing Wisely campaign.  JAMA Intern Med. 2015;175(12):1913-1920. doi:10.1001/jamainternmed.2015.5441PubMedGoogle ScholarCrossref
22.
Mostofian  F, Ruban  C, Simunovic  N, Bhandari  M.  Changing physician behavior: what works?  Am J Manag Care. 2015;21(1):75-84.PubMedGoogle Scholar
23.
Ivers  N, Jamtvedt  G, Flottorp  S,  et al.  Audit and feedback: effects on professional practice and healthcare outcomes.  Cochrane Database Syst Rev. 2012;(6):CD000259.PubMedGoogle Scholar
24.
Forsetlund  L, Bjørndal  A, Rashidian  A,  et al.  Continuing education meetings and workshops: effects on professional practice and health care outcomes.  Cochrane Database Syst Rev. 2009;(2):CD003030.PubMedGoogle Scholar
25.
Flodgren  G, Eccles  MP, Shepperd  S, Scott  A, Parmelli  E, Beyer  FR.  An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes.  Cochrane Database Syst Rev. 2011;(7):CD009255.PubMedGoogle Scholar
26.
Giguère  A, Légaré  F, Grimshaw  J,  et al.  Printed educational materials: effects on professional practice and healthcare outcomes.  Cochrane Database Syst Rev. 2012;10:CD004398.PubMedGoogle Scholar
27.
Flodgren  G, Parmelli  E, Doumit  G,  et al.  Effectiveness of the use of local opinion leaders to promote evidence-based practice and improving patient outcomes.  Cochrane Database Syst Rev. 2011;(8):CD000125. doi:10.1002/14651858.CD000125.pub4Google Scholar
28.
Sirovich  BE, Woloshin  S, Schwartz  LM.  Too little? too much? primary care physicians’ views on US health care: a brief report.  Arch Intern Med. 2011;171(17):1582-1585. doi:10.1001/archinternmed.2011.437PubMedGoogle ScholarCrossref
29.
Graham  ID, Logan  J, Harrison  MB,  et al.  Lost in knowledge translation: time for a map?  J Contin Educ Health Prof. 2006;26(1):13-24. doi:10.1002/chp.47PubMedGoogle ScholarCrossref
30.
Althabe  F, Buekens  P, Bergel  E,  et al; Guidelines Trial Group.  A behavioral intervention to improve obstetrical care.  N Engl J Med. 2008;358(18):1929-1940. doi:10.1056/NEJMsa071456PubMedGoogle ScholarCrossref
31.
Gagliardi  AR, Kothari  A, Graham  ID.  Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know.  J Epidemiol Community Health. 2017;71(2):105-106. doi:10.1136/jech-2016-207743PubMedGoogle ScholarCrossref
32.
Colquhoun  HL, Squires  JE, Kolehmainen  N, Fraser  C, Grimshaw  JM.  Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review.  Implement Sci. 2017;12(1):30. doi:10.1186/s13012-017-0560-5PubMedGoogle ScholarCrossref
33.
Brehaut  JC, Colquhoun  HL, Eva  KW,  et al.  Practice feedback interventions: 15 suggestions for optimizing effectiveness.  Ann Intern Med. 2016;164(6):435-441. doi:10.7326/M15-2248PubMedGoogle ScholarCrossref
34.
Sikkens  JJ, van Agtmael  MA, Peters  EJG,  et al.  Behavioral approach to appropriate antimicrobial prescribing in hospitals: the Dutch Unique Method for Antimicrobial Stewardship (DUMAS) participatory intervention study.  JAMA Intern Med. 2017;177(8):1130-1138. doi:10.1001/jamainternmed.2017.0946PubMedGoogle ScholarCrossref
35.
Deci  EL, Ryan  RM.  The empirical exploration of intrinsic motivational processes.  Adv Exp Soc Psychol. 1980;13:39-80. doi:10.1016/S0065-2601(08)60130-6Google Scholar
36.
McDermott  L, Yardley  L, Little  P, Ashworth  M, Gulliford  M; eCRT Research Team.  Developing a computer delivered, theory based intervention for guideline implementation in general practice.  BMC Fam Pract. 2010;11:90. doi:10.1186/1471-2296-11-90PubMedGoogle ScholarCrossref
37.
Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement.  JAMA. 1999;282(15):1458-1465. doi:10.1001/jama.282.15.1458PubMedGoogle ScholarCrossref
38.
Council  ML, Alam  M, Gloster  HM  Jr,  et al.  Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry.  J Am Acad Dermatol. 2016;74(4):739-745.PubMedGoogle ScholarCrossref
39.
Leitenberger  JJ, Rogers  H, Chapman  JC,  et al.  Defining recurrence of nonmelanoma skin cancer after Mohs micrographic surgery: report of the American College of Mohs Surgery Registry and Outcomes Committee.  J Am Acad Dermatol. 2016;75(5):1022-1031.PubMedGoogle ScholarCrossref
×