[Skip to Content]
[Skip to Content Landing]
Figure.
Adjusted 5-Year, Surgeon-Specific Endoscopic Sinus Surgery Revision Rates for Chronic Rhinosinusitis
Adjusted 5-Year, Surgeon-Specific Endoscopic Sinus Surgery Revision Rates for Chronic Rhinosinusitis

Overall crude revision rates were 10.6% for both the observed and risk-adjusted revisions (vertical dashed lines). Error bars indicate 95% CIs.

Table 1.  
Association of Adjusted Variables With Surgeon-Specific, 5-Year Endoscopic Sinus Surgery Revision Rate for Chronic Rhinosinusitis
Association of Adjusted Variables With Surgeon-Specific, 5-Year Endoscopic Sinus Surgery Revision Rate for Chronic Rhinosinusitis
Table 2.  
Comparison of the Rates of Important Characteristics Between the Top- and Low-Performing Surgeons
Comparison of the Rates of Important Characteristics Between the Top- and Low-Performing Surgeons
1.
Duclos  A, Carty  MJ.  Value of health care delivery.  JAMA. 2011;306(3):267.PubMedGoogle ScholarCrossref
2.
Kaplan  RS, Porter  ME.  How to solve the cost crisis in health care.  Harv Bus Rev. 2011;89(9):46-52, 54, 56-61 passim.PubMedGoogle Scholar
3.
Vila  PM, Schneider  JS, Piccirillo  JF, Lieu  JE.  Understanding quality measures in otolaryngology–head and neck surgery.  JAMA Otolaryngol Head Neck Surg. 2016;142(1):86-90.PubMedGoogle ScholarCrossref
4.
Blumenthal  D.  Better health care: a way forward.  JAMA. 2016;315(13):1333-1334.PubMedGoogle ScholarCrossref
5.
Dimick  JB, Weeks  WB, Karia  RJ, Das  S, Campbell  DA  Jr.  Who pays for poor surgical quality? building a business case for quality improvement.  J Am Coll Surg. 2006;202(6):933-937.PubMedGoogle ScholarCrossref
6.
Smith  TL, Mace  JC, Rudmik  L,  et al.  Comparing surgeon outcomes in endoscopic sinus surgery for chronic rhinosinusitis.  Laryngoscope. 2017;127(1):14-21.PubMedGoogle ScholarCrossref
7.
Kiernan  F, Rahman  F.  Measuring surgical performance: a risky game?  Surgeon. 2015;13(4):213-217.PubMedGoogle ScholarCrossref
8.
Maruthappu  M, El-Harasis  MA, Nagendran  M,  et al.  Systematic review of methodological quality of individual performance measurement in surgery.  Br J Surg. 2014;101(12):1491-1498.PubMedGoogle ScholarCrossref
9.
Cassel  CK, Conway  PH, Delbanco  SF, Jha  AK, Saunders  RS, Lee  TH.  Getting more performance from performance measurement.  N Engl J Med. 2014;371(23):2145-2147.PubMedGoogle ScholarCrossref
10.
Bhatt  DL, Drozda  JP  Jr, Shahian  DM,  et al; ACC/AHA Task Force on Performance Measures.  ACC/AHA/STS statement on the future of registries and the performance measurement enterprise: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures and the Society of Thoracic Surgeons.  Ann Thorac Surg. 2015;100(5):1926-1941.PubMedGoogle ScholarCrossref
11.
Govaert  JA, van Dijk  WA, Fiocco  M,  et al. Nationwide outcomes measurement in colorectal cancer surgery: improving quality and reducing costs. J Am Coll Surg.2016;222(1):19-29, e12.
12.
Graham  B, Green  A, James  M, Katz  J, Swiontkowski  M.  Measuring patient satisfaction in orthopaedic surgery.  J Bone Joint Surg Am. 2015;97(1):80-84.PubMedGoogle ScholarCrossref
13.
Shahian  DM, He  X, Jacobs  JP,  et al.  The Society of Thoracic Surgeons composite measure of individual surgeon performance for adult cardiac surgery: a report of the Society of Thoracic Surgeons Quality Measurement Task Force.  Ann Thorac Surg. 2015;100(4):1315-1324.PubMedGoogle ScholarCrossref
14.
Rudmik  L, Mace  J, Soler  ZM, Smith  TL.  Long-term utility outcomes in patients undergoing endoscopic sinus surgery.  Laryngoscope. 2014;124(1):19-23.PubMedGoogle ScholarCrossref
15.
Soler  ZM, Wittenberg  E, Schlosser  RJ, Mace  JC, Smith  TL.  Health state utility values in patients undergoing endoscopic sinus surgery.  Laryngoscope. 2011;121(12):2672-2678.PubMedGoogle ScholarCrossref
16.
Patel  ZM, Thamboo  A, Rudmik  L, Nayak  JV, Smith  TL, Hwang  PH.  Surgical therapy vs continued medical therapy for medically refractory chronic rhinosinusitis: a systematic review and meta-analysis.  Int Forum Allergy Rhinol. 2017;7(2):119-127.PubMedGoogle ScholarCrossref
17.
Bhattacharyya  N.  Ambulatory sinus and nasal surgery in the United States: demographics and perioperative outcomes.  Laryngoscope. 2010;120(3):635-638.PubMedGoogle ScholarCrossref
18.
Wier  LM, Steiner  CA, Owens  PL. Surgeries in Hospital-Owned Outpatient Facilities: Agency for Healthcare Research and Quality. http://www.hcup-us.ahrq.gov/reports/statbriefs/sb188-Surgeries-Hospital-Outpatient-Facilities-2012.pdf. Published February 2015. Accessed May 8, 2017.
19.
Rudmik  L, Bird  C, Dean  S, Dort  JC, Schorn  R, Kukec  E.  Geographic variation of endoscopic sinus surgery in Canada: an Alberta-based small area variation analysis.  Otolaryngol Head Neck Surg. 2015;153(5):865-874.PubMedGoogle ScholarCrossref
20.
Rudmik  L, Holy  CE, Smith  TL.  Geographic variation of endoscopic sinus surgery in the United States.  Laryngoscope. 2015;125(8):1772-1778.PubMedGoogle ScholarCrossref
21.
Venkatraman  G, Likosky  DS, Morrison  D, Zhou  W, Finlayson  SR, Goodman  DC.  Small area variation in endoscopic sinus surgery rates among the Medicare population.  Arch Otolaryngol Head Neck Surg. 2011;137(3):253-257.PubMedGoogle ScholarCrossref
23.
Dautremont  JF, Rudmik  L.  When are we operating for chronic rhinosinusitis? a systematic review of maximal medical therapy protocols prior to endoscopic sinus surgery.  Int Forum Allergy Rhinol. 2015;5(12):1095-1103.PubMedGoogle ScholarCrossref
24.
American Academy of Otolaryngology. Quality Measures. 2017. http://www.entnet.org/content/quality-measures. Accessed May 11, 2017.
25.
Bhattacharyya  N.  Unplanned revisits and readmissions after ambulatory sinonasal surgery.  Laryngoscope. 2014;124(9):1983-1987.PubMedGoogle ScholarCrossref
26.
Ramakrishnan  VR, Kingdom  TT, Nayak  JV, Hwang  PH, Orlandi  RR.  Nationwide incidence of major complications in endoscopic sinus surgery.  Int Forum Allergy Rhinol. 2012;2(1):34-39.PubMedGoogle ScholarCrossref
27.
Rudmik  L, Xu  Y, Kukec  E, Liu  M, Dean  S, Quan  H.  A validated case definition for chronic rhinosinusitis in administrative data: a Canadian perspective.  Int Forum Allergy Rhinol. 2016;6(11):1167-1172.PubMedGoogle ScholarCrossref
28.
Philpott  C, Hopkins  C, Erskine  S,  et al.  The burden of revision sinonasal surgery in the UK—data from the Chronic Rhinosinusitis Epidemiology Study (CRES): a cross-sectional study.  BMJ Open. 2015;5(4):e006680.PubMedGoogle ScholarCrossref
29.
Hosmer  DW, Lemeshow  S.  Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.Crossref
30.
Hopkins  C, Slack  R, Lund  V, Brown  P, Copley  L, Browne  J.  Long-term outcomes from the English national comparative audit of surgery for nasal polyposis and chronic rhinosinusitis.  Laryngoscope. 2009;119(12):2459-2465.PubMedGoogle ScholarCrossref
31.
Mendelsohn  D, Jeremic  G, Wright  ED, Rotenberg  BW.  Revision rates after endoscopic sinus surgery: a recurrence analysis.  Ann Otol Rhinol Laryngol. 2011;120(3):162-166.PubMedGoogle ScholarCrossref
32.
Rudmik  L, Xu  Y, Liu  M, Bird  C, Kukec  E, Quan  H.  Utilization patterns of topical intranasal steroid therapy for chronic rhinosinusitis: a Canadian population-based analysis.  JAMA Otolaryngol Head Neck Surg. 2016;142(11):1056-1062.PubMedGoogle ScholarCrossref
33.
Xu  Y, Quan  H, Faris  P,  et al.  Prevalence and incidence of diagnosed chronic rhinosinusitis in Alberta, Canada.  JAMA Otolaryngol Head Neck Surg. 2016;142(11):1063-1069.PubMedGoogle ScholarCrossref
34.
Hopkins  C, Andrews  P, Holy  CE.  Does time to endoscopic sinus surgery impact outcomes in chronic rhinosinusitis? retrospective analysis using the UK clinical practice research data.  Rhinology. 2015;53(1):18-24.PubMedGoogle Scholar
35.
Smith  KA, Smith  TL, Mace  JC, Rudmik  L.  Endoscopic sinus surgery compared to continued medical therapy for patients with refractory chronic rhinosinusitis.  Int Forum Allergy Rhinol. 2014;4(10):823-827.PubMedGoogle ScholarCrossref
36.
Council  ML, Alam  M, Gloster  HM  Jr,  et al.  Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry.  J Am Acad Dermatol. 2016;74(4):739-745.PubMedGoogle ScholarCrossref
37.
Erekson  EA, Iglesia  CB.  Improving patient outcomes in gynecology: the role of large data registries and big data analytics.  J Minim Invasive Gynecol. 2015;22(7):1124-1129.PubMedGoogle ScholarCrossref
38.
Norrving  B, Bray  BD, Asplund  K,  et al; European Implementation Score Collaboration.  Cross-national key performance measures of the quality of acute stroke care in Western Europe.  Stroke. 2015;46(10):2891-2895.PubMedGoogle ScholarCrossref
39.
Southwood  JE, Loehrl  TA, Poetker  DM.  Advances in surgery: extended procedures for sinonasal polyp disease.  Adv Otorhinolaryngol. 2016;79:148-157.PubMedGoogle Scholar
40.
Rudmik  L, Soler  ZM, Orlandi  RR,  et al.  Early postoperative care following endoscopic sinus surgery: an evidence-based review with recommendations.  Int Forum Allergy Rhinol. 2011;1(6):417-430.PubMedGoogle ScholarCrossref
41.
Maruthappu  M, Gilbert  BJ, El-Harasis  MA,  et al.  The influence of volume and experience on individual surgical performance: a systematic review.  Ann Surg. 2015;261(4):642-647.PubMedGoogle ScholarCrossref
42.
Sahni  NR, Dalton  M, Cutler  DM, Birkmeyer  JD, Chandra  A.  Surgeon specialization and operative mortality in United States: retrospective analysis.  BMJ. 2016;354:i3571.PubMedGoogle ScholarCrossref
43.
Soler  ZM, Rudmik  L, Hwang  PH, Mace  JC, Schlosser  RJ, Smith  TL.  Patient-centered decision making in the treatment of chronic rhinosinusitis.  Laryngoscope. 2013;123(10):2341-2346.PubMedGoogle Scholar
44.
Hopkins  C, Rudmik  L, Lund  VJ.  The predictive value of the preoperative Sinonasal Outcome Test-22 score in patients undergoing endoscopic sinus surgery for chronic rhinosinusitis.  Laryngoscope. 2015;125(8):1779-1784.PubMedGoogle ScholarCrossref
45.
Rudmik  L, Soler  ZM, Hopkins  C.  Using postoperative SNOT-22 to help predict the probability of revision sinus surgery.  Rhinology. 2016;54(2):111-116.PubMedGoogle Scholar
46.
Rudmik  L, Soler  ZM, Mace  JC, DeConde  AS, Schlosser  RJ, Smith  TL.  Using preoperative SNOT-22 score to inform patient decision for endoscopic sinus surgery.  Laryngoscope. 2015;125(7):1517-1522.PubMedGoogle ScholarCrossref
Original Investigation
September 2017

Evaluating Surgeon-Specific Performance for Endoscopic Sinus Surgery

Author Affiliations
  • 1Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Calgary, Calgary, Alberta, Canada
  • 2Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
  • 3Division of Otolaryngology–Head and Neck Surgery, Rhinology–Sinus and Skull Base Surgery Program, Department of Surgery, University of Utah, Salt Lake City
  • 4Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of California, San Diego
  • 5Division of Rhinology and Sinus/Skull Base Surgery, Oregon Sinus Center, Department of Otolaryngology–Head and Neck Surgery, Oregon Health and Science University, Portland
  • 6Division of Rhinology and Sinus Surgery, Department of Otolaryngology–Head and Neck Surgery, Medical University of South Carolina, Charleston
JAMA Otolaryngol Head Neck Surg. 2017;143(9):891-898. doi:10.1001/jamaoto.2017.0752
Key Points

Question  When using a risk-adjusted, 5-year surgery revision rate as a surrogate marker for performance, are there differences in surgeon-specific performance for endoscopic sinus surgery?

Findings  After evaluating 43 surgeons within the province of Alberta, Canada, there were differences in surgeon-specific performance for endoscopic sinus surgery. Three variables demonstrated significant associations with 5-year endoscopic sinus surgery revision rate: presence of nasal polyps, more annual systemic corticosteroid courses, and concurrent septoplasty.

Meaning  Evaluating surgeon-specific performance for endoscopic sinus surgery may provide information to assist in quality improvement. Given the findings from this study, the surgeon-specific, risk-adjusted, 5-year endoscopic sinus surgery revision rate may represent a quality metric to assess surgical performance during management of chronic rhinosinusitis.

Abstract

Importance  Several identified factors have raised questions concerning the quality of care for endoscopic sinus surgery (ESS), including the presence of large geographic variation in the rates and extent of surgery, poorly defined indications, and lack of ESS-specific quality metrics. Combined with the risk of major complications, ESS represents a high-value target for quality improvement.

Objective  To evaluate differences in surgeon-specific performance for ESS using a risk-adjusted, 5-year ESS revision rate as a quality metric.

Design, Setting, and Participants  This retrospective study used a population-based administrative database to study adults (≥18 years of age) with chronic rhinosinusitis (CRS) who underwent primary ESS in Alberta, Canada, between March 1, 2007, and March 1, 2010. The study period ended in 2015 to provide 5 years of follow-up.

Interventions  Endoscopic sinus surgery for CRS.

Main Outcomes and Measures  Primary outcomes were the 5-year observed and risk-adjusted ESS revision rate. Logistic regression was used to develop a risk adjustment model for the primary outcome.

Results  A total of 43 individual surgeons performed primary ESS in 2168 patients with CRS. Within 5 years after the primary ESS procedure, 239 patients underwent revision ESS, and the mean crude 5-year ESS revision rate was 10.6% (range, 2.4%-28.6%). After applying the risk adjustment model and 95% CI to each surgeon, 7 surgeons (16%) had lower-than-expected performance and 2 surgeons (5%) had higher-than-expected performance. Three variables had significant associations with surgeon-specific, 5-year ESS revision rates: presence of nasal polyps (odds ratio [OR], 2.07; 95% CI, 1.59-2.70), more annual systemic corticosteroid courses (OR, 1.33; 95% CI, 1.19-1.48), and concurrent septoplasty (OR, 0.70; 95% CI, 0.55-0.89).

Conclusions and Relevance  Evaluating surgeon-specific performance for ESS may provide information to assist in quality improvement. Although most surgeons had comparable risk-adjusted, 5-year ESS revision rates, 16% of surgeons had lower-than-expected performance, indicating a potential to improve quality of care. Future studies are needed to evaluate more surgeon-specific variables and validate a risk adjustment model to provide appropriate feedback for quality improvement.

Introduction

Health care systems are focused on improving the quality and value of health care delivery.1-3 Because of increasing technology costs, constrained surgical resources, and large practice variation, there is significant interest in measuring the performance of individual surgeons to drive quality improvement.4-6 Information obtained from measuring and appropriately risk adjusting surgeon performance would be useful to various stakeholders. This information might give surgeons real-time individual feedback, assist patients to make informed decisions about their care, and allow third-party payers to link remuneration with quality outcomes. However, the measurement of surgeon performance for endoscopic sinus surgery (ESS) lags behind other medical specialties.7,8 A recent study by Smith et al6 evaluated surgeon-specific outcomes for ESS among 3 academic institutions, and the results provided the first insights into how outcomes can vary among surgeons. When conducted appropriately, evaluating surgeon-specific performance may represent an important strategy to improve the overall quality of the health care system.9-13

It is well accepted that ESS can provide significant short- and long-term benefit to patients with refractory chronic rhinosinusitis (CRS).14-16 With more than 250 000 operations performed each year in the United States,17 this volume of ESS rivals inguinal hernia repair and breast lumpectomy and exceeds the rates of pacemaker placement and colorectal resections.18 Recently, several identified factors have raised questions concerning the quality of care for ESS, including the presence of large geographic variation in the rates and extent of surgery,19-22 poorly defined indications,23 and lack of ESS-specific quality metrics.24 These factors, combined with the risk of major complications,25,26 make ESS a high-value target for quality improvement, and measuring individual surgeon performance may assist in further quality improvement.

The objective of this study was to evaluate surgeon-specific performance for ESS by using a risk-adjusted, 5-year ESS revision rate as a quality metric. Evaluation of differences in adjusted surgeon-specific ESS revision rates may provide novel insight into how quality of care may vary during management of refractory CRS.

Methods

The Data Integration, Measurement, and Reporting (DIMR) administrative database of Alberta Health Service was used to obtain data for this study. The DIMR database collects physician claims on all government-funded health care encounters in Alberta, Canada. There is no private provision of health care for CRS in Alberta; therefore, all medical care for CRS is captured in the DIMR database. Inclusion criteria for this study were adults (≥18 years of age) with CRS (defined by validated case definition27) in Alberta who underwent primary ESS between March 1, 2007, and March 1, 2010 (3-year time horizon). Because more than 95% of ESS revision cases occur within 5 years after the primary ESS procedure,28 this study evaluated a 5-year follow-up time. An ESS revision was defined as an additional ESS procedure within 5 years after the primary ESS date. Patients were excluded from the final analysis if they had an ESS claim before 2007. An ESS case was defined as a minimum physician claim of a maxillary antrostomy (Alberta procedure code 34.1A) and ethmoidectomy (Alberta procedure code 34.54A). The use of physician claims to identify cases of ESS was validated in a prior study.19 Conjoint Health Research Ethics Board approval from the University of Calgary was obtained, no informed consent was required, and all data were deidentified.

All patients included in this analysis were cross-linked to the providing surgeon’s anonymous practice identification number along with the Pharmaceutical Information Network and the Discharge Abstract Database to collect drug- and comorbidity-related variables. Because physician claims were used to identify ESS, all cases of ESS revision were included regardless of whether the additional operation was performed by the primary ESS surgeon.

The outcome variable of interest was the 5-year observed and risk-adjusted ESS revision rates. Logistic regression was used to develop a risk adjustment model for the 5-year ESS revision rate. Multiple potential confounding variables were included in the risk adjustment model (Box). Race/ethnicity is only reported in aggregate per geographic region in the DIMR database; therefore, the regression analysis could not include this variable. On the basis of the 2011 Canadian Census, most of the Alberta population is white (75%), with the next visible minorities being Aboriginal (6%), South Asian (4%), Chinese (3%), and black (2.7%); therefore, the association of potential geographic variation with race/ethnicity was determined to be minimal. To assess the association of the extent of surgery with outcomes, we assumed that the addition of a frontal sinusotomy claim (Alberta procedure code 34.32B or 34.32C) indicated that the surgeon performed a complete ESS as opposed to a limited ESS when only maxillary antrostomy (Alberta procedure code 34.1A) and ethmoidectomy (Alberta procedure code 34.54A) claims were submitted. To maintain surgeon anonymity in this study, the only surgeon factor included in the model was annual ESS case volume. Other identifiable variables, such as age, year of training, fellowship training status, and location, were not included to ensure surgeon anonymity.

Box Section Ref ID
Box.

Variables Used to Adjust Surgeon-Specific ESS Revision Rates

Patient Sociodemographic Characteristic
  • Age

  • Sex

  • Geographic region (urban or rural)

Patient Comorbidity
  • Nasal polyps

  • Asthma

  • Allergic rhinitis

  • Depression

  • Chronic obstructive pulmonary disease

  • Congestive heart failure

  • Hypertension

  • Coagulopathy

  • Cardiac arrhythmia

  • Diabetes

  • Renal failure

  • Obesity

  • Liver disease

  • Psychosis

Chronic Rhinosinusitis Markers
  • Topical intranasal steroid treatment (No. of bottles per year)

  • Oral antibiotic treatment (No. of courses per year)

  • Oral corticosteroid treatment (No. of courses per year)

Other
  • Underwent a concurrent septoplasty with ESS

  • No. of ENT visits before ESS

  • Annual surgeon primary ESS case volume

  • Frontal sinusotomy (marker for complete ESS)

Abbreviations: ENT, ear, nose, and throat; ESS, endoscopic sinus surgery.

The model determined the probability of an ESS revision for each patient based on the combination of clinical risk variables present in the patient. The expected ESS revision rate for each surgeon was then calculated using the mean probabilities from the model. The calculated observed and expected ESS revision rate for each surgeon was used to generate surgeon-specific observed-expected revision ratios. To obtain risk-adjusted revision rates, we then multiplied the surgeon-specific observed-expected revision ratios by the overall revision rate for the 43 surgeons. The 95% CIs of the observed and adjusted revision rates were derived by applying the binomial method. The surgeon-specific adjusted ESS revision rate was considered to be higher or lower than the expected rate when the 95% CI did not cross the crude observed revision rate.

The association between the adjusted variables and surgeon-specific, 5-year ESS revision rate is measured by odds ratio (OR) and 95% CI. The performance of the logistic regression model was assessed using the C statistic, and a value of greater than 0.7 was considered to be a good model.29 Because of the multiple variables included in the logistic regression model (n = 24 variables), the Bonferroni correction was applied to reduce the risk of a type I error (ie, incorrectly rejecting the null hypothesis). For this study, a 2-sided P ≤ .002 was considered statistically significant.

Results

A total of 43 surgeons performed primary ESS in 2168 patients for CRS. The mean number of ESS cases per surgeon within the 3-year study period was 50 (range, 16-167). Within 5 years after the primary ESS, 239 patients underwent revision ESS, and the mean crude 5-year ESS revision rate was 10.6% (range, 2.4%-28.6%) (Figure).

The associations between the adjusted variables and surgeon-specific, 5-year ESS revision rate as measured by OR and 95% CI are reported in Table 1. With use of P ≤ .002 to account for multiple comparisons, the following 3 variables demonstrated significant associations with surgeon-specific, 5-year ESS revision rates: presence of nasal polyps (OR, 2.07; 95% CI, 1.59-2.70), more annual systemic corticosteroid courses (OR, 1.33; 95% CI, 1.19-1.48), and concurrent septoplasty (OR, 0.70; 95% CI, 0.55-0.89).

Risk-adjusted, 5-year ESS revision rates per surgeon are shown in the Figure. After applying the 95% CI to each surgeon’s adjusted revision rate, 2 surgeons (5%) had lower-than-expected revision rates (ie, performed better than expected) and 7 surgeons (16%) had higher-than-expected revision rates (ie, performed lower than expected). The mean annual volume of primary ESS cases for the top 2 performing surgeons was 26 cases compared with a mean annual volume of 10 primary ESS cases (range, 5-19 cases) for the 7 surgeons with lower-than-expected performance. Although low subgroup sample size precludes the ability to perform meaningful statistical analysis, the rates for each of the 3 patient characteristics that were significantly associated with 5-year ESS revision rate for the top- and low-performing surgeons are reported in Table 2.

Discussion

This study evaluated the differences in surgeon-specific, 5-year ESS revision rates for CRS. The results demonstrate that the crude 5-year ESS revision rate for CRS is 10.6%, which is similar to previously reported ESS revision rates.30,31 When evaluating the performance of individual surgeons, the results indicate that after risk adjusting, most surgeons had comparable outcomes, with 7 surgeons (16%) having higher-than-expected ESS revision rates (ie, lower than expected performance). The finding that nasal polyps and more systemic corticosteroid courses negatively affect ESS revision is expected because the presence of nasal polyps has been associated with higher rates of ESS revisions in a prior study28 and surgeons tend to prescribe systemic corticosteroids for nasal polyps. Furthermore, the finding that concurrent septoplasty positively affected the rate of revision ESS (ie, reduced rate) was similar to the results from a recent US-based study.6 Using larger surgeon sample sizes, future studies need to determine whether there are other variables that need to be accounted for in the risk-adjustment model before surgeon performance can be evaluated using revision rate as a quality metric.

In theory, with expanding research into the pathophysiology and clinical outcomes for CRS combined with improved surgical technology, the quality of care for this chronic disease should have increased during the past 3 decades. However, despite substantial advances in the management of CRS, persistent gaps still exist in the quality of care for CRS, such as large practice variation,19,20,32,33 delayed access to ESS,34,35 and postoperative complications that require hospital care.25 As we continue to search for ways to improve how we deliver care for CRS, it is important to begin evaluating the role that surgeons play in this quality-improvement puzzle. Although quality improvement is a highly complex process that involves multiple stakeholders, evaluating the performance of surgeons and providing feedback in real time represent a promising strategy.4 Although other medical and surgical specialties have started registries focused on performance measurement,10,36-38 research focused on evaluating the performance of surgeons during management of CRS has been limited.

A variety of metrics can reasonably be examined when looking at surgeon-specific outcomes for management of CRS. This study examined the ESS revision rate, but other potential metrics include degree of quality-of-life improvement, work productivity, need for subsequent medications and physician visits, cost of surgical treatment, or prevention of CRS progression. In 2016, a US-based study by Smith et al6 evaluated surgeon-specific outcomes for CRS across 3 academic institutions. The results indicated that there were unadjusted differences in patient outcomes across the 3 institutions, and the differences appeared to be driven by variations in the degree of baseline CRS-specific quality-of-life impairment, oral corticosteroid dependence, and concurrent septoplasty. Objective measures of CRS (ie, computed tomography and endoscopy) failed to explain the differences in patient outcomes among institutions and were not included in the risk-adjustment model, a finding that suggests that objective disease severity was not driving the differences in surgeon performance. The effects of other patient characteristics, such as depression, prior ESS, and acetylsalicylic acid sensitivity, were not assessed because the prevalence was similar among all 3 institutions. The primary limitation of the study by Smith et al6 was that it only included surgeons from 3 tertiary-level practices and did not include community-based practices. This model reduces the generalizability of the findings to a broader population-level perspective of care for CRS.

Although the study by Smith et al6 represents the first insights into how outcomes can vary across academic institutions, the current study evaluated 43 different surgeons across a Canadian province and includes tertiary-level and community-based rhinology practices. Overall, the results support the notion that it is feasible to compare surgeon-specific performance for ESS after appropriate risk adjustment has been applied. Although most of the surgeons had similar outcomes, 7 surgeons (16%) had lower-than-expected performance when using an adjusted 5-year ESS revision rate with the 95% CI as a quality metric. This finding indicates that there is variation in surgical outcomes, and delivery of ESS to patients with CRS can be improved.

Three variables were identified to affect surgeon performance outcomes and thus may be important to include in risk-adjustment models for ESS. First, the presence of nasal polyps was associated with higher rates of ESS revision, and this finding has been supported by several previous studies.28,30,39 Second, receipt of more systemic corticosteroid courses before ESS was associated with a higher risk of ESS revision. This finding may be attributable to patients with nasal polyps often being treated with courses of systemic corticosteroids before ESS and therefore having a higher risk of ESS revision. Another potential reason may indicate that patients treated with multiple courses of systemic corticosteroids had a delay in undergoing ESS and thus may have worse long-term outcomes.34 Third, a concurrent septoplasty was associated with a reduced risk of undergoing an ESS revision. Potential explanations for this finding may include patients breathing better after ESS, having improved access to topical medical therapies after ESS, or perhaps undergoing septoplasty that augmented the technical success of ESS by allowing better access to the sinuses. On the other hand, concurrent septoplasty may be a confounding variable for patients receiving technically more complete primary ESS and appropriate postoperative care because surgeons who are more comfortable performing ESS (ie, sinus surgeons performing a large volume of operations and/or fellowship-trained rhinologists) may have a lower threshold to perform a concurrent septoplasty. Although it is unknown why concurrent septoplasty positively affects surgeon performance by using the ESS revision rate, the study by Smith et al6 evaluated only the performance of fellowship-trained rhinologists, and the results also indicated that performing a concurrent septoplasty positively affected outcomes after ESS. Although it is important to prevent unnecessary procedures, failure to perform a concurrent septoplasty when a deviated septum is present may negatively affect outcomes after ESS.

Although most surgeons had similar adjusted ESS revision rates, this study found that 7 surgeons (16%) had higher-than-expected revision rates, suggesting there may be an opportunity to improve the care provided by these surgeons. However, this study does not provide an answer as to why these surgeons have higher rates of revision. Because of the need to maintain surgeon anonymity, this study was unable to evaluate more detailed surgeon variables, such as age, fellowship training status, practice location (ie, rural or urban), surgical technique, or quality of postoperative care.40 Despite the limitations of trying to elucidate reasons for the lower-than-expected performance from 7 surgeons, variables such as annual ESS case volume and fellowship training status may warrant future investigation using larger surgeon sample sizes. First, we were able to evaluate the annual case volume of ESS per surgeon without compromising surgeon anonymity. Although there was no observed or adjusted association between overall surgeon volume and 5-year ESS revision rate in the logistic regression analysis, an examination of the extremes revealed that the top-performing surgeons had higher annual case volumes (mean annual ESS case volume, 17) compared with surgeons with lower performance (mean annual ESS case volume, 8). Because of the known effect of surgeon volume on patient outcomes from other procedures,41,42 future studies should continue to evaluate the effect of ESS volumes using larger sample sizes. Second, the low-performing surgeons had a lower rate of concurrent septoplasty during primary ESS (Table 2). Although the significance of concurrent septoplasty needs further investigation, current evidence suggests that concurrent septoplasty is associated with reduced risk of ESS revision. Although there are several potential reasons to explain the differences in surgeon performance, this study provides insight into understanding that differences in surgeon-specific outcomes exist for ESS and there is a need to identify factors that lead to lower performance. Once factors that affect surgeon performance are identified, they can be used to improve the quality of care.

Limitations

Despite the possibility to compare surgeons based on adjusted 5-year ESS revision rates, it is important to emphasize that the model used in this study may not represent the most accurate risk adjustment for surgeon comparison. Specifically, because of the limitations of using administrative data, there may be unaccounted variables that are outside the surgeon’s control that were not included in the risk-adjustment model. For example, the degree of baseline quality-of-life impairment has a significant effect on treatment choice43 and surgical outcomes44-46 and therefore may need to be used to adjust patient outcomes after ESS. The risk-adjustment model in this study failed to include baseline or follow-up quality-of-life data because the database did not collect these variables. Furthermore, additional factors that are often not collected in administrative data must be considered when evaluating surgeon performance for ESS, for example, patient adherence to postoperative medical therapy, access to appropriate surgical equipment (eg, image guidance), and measurement of whether surgeons assessed patient preferences during the decision-making process or ensured that patients had appropriate expectations for performing revision ESS. Because risk adjustment is critical to enable appropriate and transparent comparison of surgeon performance for CRS, future studies will need to evaluate the effect of other potential confounding variables in risk-adjustment models.

Another limitation of this study is the inability to include cases of ESS revision that occurred outside Alberta. However, because there is no private health care for CRS in other provinces of Canada and Alberta has 2 tertiary-level otolaryngology institutions at which complex revision ESS procedures are performed, it would be rare for patients from Alberta to seek care outside their province.

Last, the intended and unintended consequences that could arise from reporting an ESS quality metric such as 5-year ESS revision rate must be considered. Positive benefits would arise if surgeons with high ESS revision rates critically evaluate their practices and make quality improvements. Patients are also more likely to benefit when ESS, including revision, is performed for appropriate indications by well-trained surgeons. From a negative perspective, surgeons may start to cream-skim by inappropriately adjusting their practice to deny primary ESS to patients at higher risk of additional operations, such as those with nasal polyps or other severe CRS phenotypes. Surgeons could also game the quality metric by not offering revision surgery to their own patients who might otherwise benefit from surgery or by being more liberal in offering revision surgery to patients who underwent primary surgery performed by another surgeon. These and other issues will need to be addressed before implementation of comparing surgeon performance for ESS.

Conclusions

This study evaluated differences in surgeon-specific performance for ESS. Although most surgeons had comparable risk-adjusted, 5-year ESS revision rates, 16% of surgeons had lower-than-expected performance, indicating a potential to improve quality of care. The presence of nasal polyps and more prior systemic corticosteroid courses were associated with increased risk of undergoing ESS revision, whereas concurrent septoplasty was associated with a reduced risk of ESS revision. Future studies need to evaluate more surgeon-specific variables and validate a risk-adjustment model to provide appropriate feedback for quality improvement.

Back to top
Article Information

Corresponding Author: Luke Rudmik, MD, MSc, Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Calgary, 1820 Richmond Rd SW, Calgary, AB T2T 5C7, Canada (lukerudmik@gmail.com).

Accepted for Publication: April 17, 2017.

Published Online: June 22, 2017. doi:10.1001/jamaoto.2017.0752

Author Contributions: Drs Rudmik and Xu had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Rudmik, Xu, Alt, DeConde, Quan.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Rudmik, Alt, DeConde, Smith, Schlosser.

Critical revision of the manuscript for important intellectual content: Rudmik, Xu, Alt, DeConde, Smith, Quan, Soler.

Statistical analysis: Rudmik, Xu, Quan.

Obtained funding: Rudmik.

Administrative, technical, or material support: Alt, DeConde, Smith, Soler.

Study supervision: Rudmik, Alt, DeConde, Smith, Schlosser, Quan.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Rudmik reported serving on the scientific advisory board for BioInspire and 480 Biomedical. Dr Alt reported serving as a consultant for Medtronic, Spirox, and GlycoMira Therapeutics. Dr Deconde reported serving as a consultant for IntersectENT and Stryker Endoscopy. Dr Schlosser reported serving as a consultant for Olympus, Medtronic, and Arrinex and receiving grant support from Intersect and Entellus. Dr Soler reported serving as a consultant for Olympus Inc and 480 Biomedical Inc. No other disclosures were reported.

Funding/Support: This study was supported by an MSI Foundation Grant and a Petro-Canada Young Innovator in Community Health Sciences of Canada Award (Dr Rudmik).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

Additional Contributions: We thank the MSI Foundation and Petro-Canada Young Innovator in Community Health Sciences of Canada Award for the grants to support this project.

References
1.
Duclos  A, Carty  MJ.  Value of health care delivery.  JAMA. 2011;306(3):267.PubMedGoogle ScholarCrossref
2.
Kaplan  RS, Porter  ME.  How to solve the cost crisis in health care.  Harv Bus Rev. 2011;89(9):46-52, 54, 56-61 passim.PubMedGoogle Scholar
3.
Vila  PM, Schneider  JS, Piccirillo  JF, Lieu  JE.  Understanding quality measures in otolaryngology–head and neck surgery.  JAMA Otolaryngol Head Neck Surg. 2016;142(1):86-90.PubMedGoogle ScholarCrossref
4.
Blumenthal  D.  Better health care: a way forward.  JAMA. 2016;315(13):1333-1334.PubMedGoogle ScholarCrossref
5.
Dimick  JB, Weeks  WB, Karia  RJ, Das  S, Campbell  DA  Jr.  Who pays for poor surgical quality? building a business case for quality improvement.  J Am Coll Surg. 2006;202(6):933-937.PubMedGoogle ScholarCrossref
6.
Smith  TL, Mace  JC, Rudmik  L,  et al.  Comparing surgeon outcomes in endoscopic sinus surgery for chronic rhinosinusitis.  Laryngoscope. 2017;127(1):14-21.PubMedGoogle ScholarCrossref
7.
Kiernan  F, Rahman  F.  Measuring surgical performance: a risky game?  Surgeon. 2015;13(4):213-217.PubMedGoogle ScholarCrossref
8.
Maruthappu  M, El-Harasis  MA, Nagendran  M,  et al.  Systematic review of methodological quality of individual performance measurement in surgery.  Br J Surg. 2014;101(12):1491-1498.PubMedGoogle ScholarCrossref
9.
Cassel  CK, Conway  PH, Delbanco  SF, Jha  AK, Saunders  RS, Lee  TH.  Getting more performance from performance measurement.  N Engl J Med. 2014;371(23):2145-2147.PubMedGoogle ScholarCrossref
10.
Bhatt  DL, Drozda  JP  Jr, Shahian  DM,  et al; ACC/AHA Task Force on Performance Measures.  ACC/AHA/STS statement on the future of registries and the performance measurement enterprise: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures and the Society of Thoracic Surgeons.  Ann Thorac Surg. 2015;100(5):1926-1941.PubMedGoogle ScholarCrossref
11.
Govaert  JA, van Dijk  WA, Fiocco  M,  et al. Nationwide outcomes measurement in colorectal cancer surgery: improving quality and reducing costs. J Am Coll Surg.2016;222(1):19-29, e12.
12.
Graham  B, Green  A, James  M, Katz  J, Swiontkowski  M.  Measuring patient satisfaction in orthopaedic surgery.  J Bone Joint Surg Am. 2015;97(1):80-84.PubMedGoogle ScholarCrossref
13.
Shahian  DM, He  X, Jacobs  JP,  et al.  The Society of Thoracic Surgeons composite measure of individual surgeon performance for adult cardiac surgery: a report of the Society of Thoracic Surgeons Quality Measurement Task Force.  Ann Thorac Surg. 2015;100(4):1315-1324.PubMedGoogle ScholarCrossref
14.
Rudmik  L, Mace  J, Soler  ZM, Smith  TL.  Long-term utility outcomes in patients undergoing endoscopic sinus surgery.  Laryngoscope. 2014;124(1):19-23.PubMedGoogle ScholarCrossref
15.
Soler  ZM, Wittenberg  E, Schlosser  RJ, Mace  JC, Smith  TL.  Health state utility values in patients undergoing endoscopic sinus surgery.  Laryngoscope. 2011;121(12):2672-2678.PubMedGoogle ScholarCrossref
16.
Patel  ZM, Thamboo  A, Rudmik  L, Nayak  JV, Smith  TL, Hwang  PH.  Surgical therapy vs continued medical therapy for medically refractory chronic rhinosinusitis: a systematic review and meta-analysis.  Int Forum Allergy Rhinol. 2017;7(2):119-127.PubMedGoogle ScholarCrossref
17.
Bhattacharyya  N.  Ambulatory sinus and nasal surgery in the United States: demographics and perioperative outcomes.  Laryngoscope. 2010;120(3):635-638.PubMedGoogle ScholarCrossref
18.
Wier  LM, Steiner  CA, Owens  PL. Surgeries in Hospital-Owned Outpatient Facilities: Agency for Healthcare Research and Quality. http://www.hcup-us.ahrq.gov/reports/statbriefs/sb188-Surgeries-Hospital-Outpatient-Facilities-2012.pdf. Published February 2015. Accessed May 8, 2017.
19.
Rudmik  L, Bird  C, Dean  S, Dort  JC, Schorn  R, Kukec  E.  Geographic variation of endoscopic sinus surgery in Canada: an Alberta-based small area variation analysis.  Otolaryngol Head Neck Surg. 2015;153(5):865-874.PubMedGoogle ScholarCrossref
20.
Rudmik  L, Holy  CE, Smith  TL.  Geographic variation of endoscopic sinus surgery in the United States.  Laryngoscope. 2015;125(8):1772-1778.PubMedGoogle ScholarCrossref
21.
Venkatraman  G, Likosky  DS, Morrison  D, Zhou  W, Finlayson  SR, Goodman  DC.  Small area variation in endoscopic sinus surgery rates among the Medicare population.  Arch Otolaryngol Head Neck Surg. 2011;137(3):253-257.PubMedGoogle ScholarCrossref
23.
Dautremont  JF, Rudmik  L.  When are we operating for chronic rhinosinusitis? a systematic review of maximal medical therapy protocols prior to endoscopic sinus surgery.  Int Forum Allergy Rhinol. 2015;5(12):1095-1103.PubMedGoogle ScholarCrossref
24.
American Academy of Otolaryngology. Quality Measures. 2017. http://www.entnet.org/content/quality-measures. Accessed May 11, 2017.
25.
Bhattacharyya  N.  Unplanned revisits and readmissions after ambulatory sinonasal surgery.  Laryngoscope. 2014;124(9):1983-1987.PubMedGoogle ScholarCrossref
26.
Ramakrishnan  VR, Kingdom  TT, Nayak  JV, Hwang  PH, Orlandi  RR.  Nationwide incidence of major complications in endoscopic sinus surgery.  Int Forum Allergy Rhinol. 2012;2(1):34-39.PubMedGoogle ScholarCrossref
27.
Rudmik  L, Xu  Y, Kukec  E, Liu  M, Dean  S, Quan  H.  A validated case definition for chronic rhinosinusitis in administrative data: a Canadian perspective.  Int Forum Allergy Rhinol. 2016;6(11):1167-1172.PubMedGoogle ScholarCrossref
28.
Philpott  C, Hopkins  C, Erskine  S,  et al.  The burden of revision sinonasal surgery in the UK—data from the Chronic Rhinosinusitis Epidemiology Study (CRES): a cross-sectional study.  BMJ Open. 2015;5(4):e006680.PubMedGoogle ScholarCrossref
29.
Hosmer  DW, Lemeshow  S.  Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.Crossref
30.
Hopkins  C, Slack  R, Lund  V, Brown  P, Copley  L, Browne  J.  Long-term outcomes from the English national comparative audit of surgery for nasal polyposis and chronic rhinosinusitis.  Laryngoscope. 2009;119(12):2459-2465.PubMedGoogle ScholarCrossref
31.
Mendelsohn  D, Jeremic  G, Wright  ED, Rotenberg  BW.  Revision rates after endoscopic sinus surgery: a recurrence analysis.  Ann Otol Rhinol Laryngol. 2011;120(3):162-166.PubMedGoogle ScholarCrossref
32.
Rudmik  L, Xu  Y, Liu  M, Bird  C, Kukec  E, Quan  H.  Utilization patterns of topical intranasal steroid therapy for chronic rhinosinusitis: a Canadian population-based analysis.  JAMA Otolaryngol Head Neck Surg. 2016;142(11):1056-1062.PubMedGoogle ScholarCrossref
33.
Xu  Y, Quan  H, Faris  P,  et al.  Prevalence and incidence of diagnosed chronic rhinosinusitis in Alberta, Canada.  JAMA Otolaryngol Head Neck Surg. 2016;142(11):1063-1069.PubMedGoogle ScholarCrossref
34.
Hopkins  C, Andrews  P, Holy  CE.  Does time to endoscopic sinus surgery impact outcomes in chronic rhinosinusitis? retrospective analysis using the UK clinical practice research data.  Rhinology. 2015;53(1):18-24.PubMedGoogle Scholar
35.
Smith  KA, Smith  TL, Mace  JC, Rudmik  L.  Endoscopic sinus surgery compared to continued medical therapy for patients with refractory chronic rhinosinusitis.  Int Forum Allergy Rhinol. 2014;4(10):823-827.PubMedGoogle ScholarCrossref
36.
Council  ML, Alam  M, Gloster  HM  Jr,  et al.  Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry.  J Am Acad Dermatol. 2016;74(4):739-745.PubMedGoogle ScholarCrossref
37.
Erekson  EA, Iglesia  CB.  Improving patient outcomes in gynecology: the role of large data registries and big data analytics.  J Minim Invasive Gynecol. 2015;22(7):1124-1129.PubMedGoogle ScholarCrossref
38.
Norrving  B, Bray  BD, Asplund  K,  et al; European Implementation Score Collaboration.  Cross-national key performance measures of the quality of acute stroke care in Western Europe.  Stroke. 2015;46(10):2891-2895.PubMedGoogle ScholarCrossref
39.
Southwood  JE, Loehrl  TA, Poetker  DM.  Advances in surgery: extended procedures for sinonasal polyp disease.  Adv Otorhinolaryngol. 2016;79:148-157.PubMedGoogle Scholar
40.
Rudmik  L, Soler  ZM, Orlandi  RR,  et al.  Early postoperative care following endoscopic sinus surgery: an evidence-based review with recommendations.  Int Forum Allergy Rhinol. 2011;1(6):417-430.PubMedGoogle ScholarCrossref
41.
Maruthappu  M, Gilbert  BJ, El-Harasis  MA,  et al.  The influence of volume and experience on individual surgical performance: a systematic review.  Ann Surg. 2015;261(4):642-647.PubMedGoogle ScholarCrossref
42.
Sahni  NR, Dalton  M, Cutler  DM, Birkmeyer  JD, Chandra  A.  Surgeon specialization and operative mortality in United States: retrospective analysis.  BMJ. 2016;354:i3571.PubMedGoogle ScholarCrossref
43.
Soler  ZM, Rudmik  L, Hwang  PH, Mace  JC, Schlosser  RJ, Smith  TL.  Patient-centered decision making in the treatment of chronic rhinosinusitis.  Laryngoscope. 2013;123(10):2341-2346.PubMedGoogle Scholar
44.
Hopkins  C, Rudmik  L, Lund  VJ.  The predictive value of the preoperative Sinonasal Outcome Test-22 score in patients undergoing endoscopic sinus surgery for chronic rhinosinusitis.  Laryngoscope. 2015;125(8):1779-1784.PubMedGoogle ScholarCrossref
45.
Rudmik  L, Soler  ZM, Hopkins  C.  Using postoperative SNOT-22 to help predict the probability of revision sinus surgery.  Rhinology. 2016;54(2):111-116.PubMedGoogle Scholar
46.
Rudmik  L, Soler  ZM, Mace  JC, DeConde  AS, Schlosser  RJ, Smith  TL.  Using preoperative SNOT-22 score to inform patient decision for endoscopic sinus surgery.  Laryngoscope. 2015;125(7):1517-1522.PubMedGoogle ScholarCrossref
×