Evaluation of Patient- and Surgeon-Specific Variations in Patient-Reported Urinary Outcomes 3 Months After Radical Prostatectomy From a Statewide Improvement Collaborative

Key Points

Question  Which patient- and surgeon-specific factors are associated with urinary outcomes 3 months after radical prostatectomy?

Findings  In this cohort study including 4637 men following radical prostatectomy, there was broad variation in patient-reported urinary outcomes 3 months after surgery. The percentage of patients reporting good urinary function at 3 months varied from 0% to 55% across surgeons, and this difference persisted after accounting for patient factors associated with differences in urinary outcomes.

Meaning  Patient-reported urinary outcomes varied substantially across surgeons performing radical prostatectomy, and identifying technical factors associated with superior outcomes may limit morbidity for patients with prostate cancer.

Importance  Understanding variation in patient-reported outcomes following radical prostatectomy may inform efforts to reduce morbidity after this procedure.

Objective  To describe patient-reported urinary outcomes following radical prostatectomy in the diverse practice settings of a statewide quality improvement program and to explore whether surgeon-specific variations in observed outcomes persist after accounting for patient-level factors.

Design, Setting, and Participants  This prospective population-based cohort study included 4582 men in the Michigan Urological Surgery Improvement Collaborative who underwent radical prostatectomy as primary management of localized prostate cancer between April 2014 and July 2018 and who agreed to complete validated questionnaires prior to surgery and at 3, 6, and 12 months after surgery. Data were analyzed from 2019 to June 2019.

Exposures  Radical prostatectomy.

Main Outcomes and Measures  Patient- and surgeon-level analyses of patient-reported urinary function 3 months after radical prostatectomy. Outcomes were measured using validated questionnaires with results standardized using previously published methods. Urinary function survey scores are reported on a scale from 0 to 100 with good function established as a score of 74 or higher.

Results  For the 4582 men undergoing radical prostatectomy within the Michigan Urological Surgery Improvement Collaborative who agreed to complete surveys, mean (SD) age was 63.3 (7.1) years. Survey response rates varied: 3791 of 4582 (83%) responded at baseline, 3282 of 4137 (79%) at 3 months, 2975 of 3770 (79%) at 6 months, and 2213 of 2882 (77%) at 12 months. Mean (SD) urinary function scores were 88.5 (14.3) at baseline, 53.6 (27.5) at 3 months, 68.0 (25.1) at 6 months, and 73.7 (23.0) at 12 months. Regression analysis demonstrated that older age, lower baseline urinary function score, body mass index (calculated as weight in kilograms divided by height in meters squared) of 30 or higher, clinical stage T2 or higher, and lack of bilateral nerve-sparing surgery were associated with a lower probability of reporting good urinary function 3 months after surgery. When evaluating patients with good baseline function, the rate at which individual surgeons’ patients reported good urinary function 3 months after surgery varied broadly (0% to 54.5%; P < .001). Patients receiving surgery from top-performing surgeons were more likely to report good 3-month function. This finding persisted after accounting for patient risk factors.

Conclusions and Relevance  In this study, patient- and surgeon-level urinary outcomes following prostatectomy varied substantially. Documenting surgeon-specific variations after accounting for patient factors may facilitate identification of surgical factors associated with superior outcomes.

Radical prostatectomy has long been considered a criterion-standard treatment for prostate cancer. Although this treatment is often curative,^1,2 it also carries substantial potential morbidities.^3-5 Urinary dysfunction is a common adverse effect of this procedure.^5-10 Up to two-thirds of men undergoing prostatectomy report having to use incontinence pads 2 months after surgery.⁵ Although some recover continence over time, the interval to regaining urinary control for most men is variable, and nearly 20% continue to report some urinary incontinence 15 years after prostatectomy.¹⁰

Urinary morbidity estimates, made possible through the collection and analysis of large volumes of patient-reported outcome (PRO) data, have improved patient and physician understanding of adverse effects associated with prostatectomy. However, to our knowledge, such data have not been analyzed in a way that drives improvement in postsurgical outcomes. Collection of PRO data across a population with granular risk adjustment to account for patient-level differences and subsequent comparison across surgeons may present a meaningful use of PRO data to facilitate improvement. If it is possible to identify surgeons who consistently achieve superior outcomes despite other factors, there may be opportunities to learn from those surgeons to limit morbidity for populations of men seeking surgical management of prostate cancer.

In this context, we evaluated patient-reported urinary function during the first year after prostatectomy using data from patients undergoing surgery within the statewide Michigan Urological Surgery Improvement Collaborative (MUSIC). Patient-reported urinary outcomes were collected at baseline and at 3, 6, and 12 months after surgery. We also explored patient- and surgeon-specific variation in 3-month urinary outcomes. We evaluated whether there are surgeons who consistently achieve superior outcomes even after accounting for patient age, body mass index (calculated as weight in kilograms divided by height in meters squared), race, diabetes status, baseline function, prostate-specific antigen (PSA) level, Gleason score, clinical stage, and receipt of nerve sparing surgery.

MUSIC was established in 2011 in partnership with Blue Cross Blue Shield of Michigan. The quality improvement collaborative currently comprises 44 diverse community and academic urology practices and more than 250 urologists—more than 90% of the urologists in the state. The collaborative maintains a registry of all patients newly diagnosed with prostate cancer in participating sites. The registry includes detailed demographic, laboratory, and clinicopathologic data related to diagnostic testing, subsequent treatment(s), and outcomes of treatment. Previous reports have described MUSIC’s data quality-control activities, including annual data audits at each practice and validation analyses based on insurance claims.^11,12 Each MUSIC practice obtained an exemption or approval for collaborative participation from a local institutional review board and separate institutional review board approval for participation in the MUSIC PRO program. Given the quality-improvement focus of MUSIC, both the collaborative’s registry and the PRO initiative were deemed exempt from obtaining informed consent. Every patient was provided with an acknowledgement statement prior to their first survey explaining the purpose of the program and given the choice to opt out of the program.

MUSIC PRO started data collection in April 2014, and the program has been described elsewhere.¹³ Participating surgeons enroll patients undergoing radical prostatectomy to take part in MUSIC PRO for prospective collection of validated PRO surveys.

From the establishment of the program through September 2016, MUSIC PRO evaluated outcomes using the Prostate Quality of Life Survey developed and validated at Memorial Sloan Kettering Cancer Center, New York, New York.¹⁴ From September 2016 to 2020, MUSIC PRO has used the 26-item Expanded Prostate Cancer Index Composite Short Form questionnaire (EPIC-26) (eAppendix in the Supplement).¹⁵ The Memorial Sloan Kettering Cancer Center tool evaluates urinary function in a 5-question survey where scores range from 0 to 21 and good function is defined as a score of 17 or higher.⁹ The EPIC-26 survey assesses urinary function on a scale from 0 to 100 points. In previous work, a crosswalk algorithm has been developed to standardize results from each survey on a scale from 0 to 100 points with good function defined as a score of 74 or higher. All results in this study are reported on this 100-point scale.¹⁶

Each participant was asked to complete a questionnaire to assess the above outcomes at baseline and at 3, 6, and 12 months after surgery. Surveys were initially distributed via email and completed online through a web-based platform. For patients without email access, a paper version was sent with a self-addressed and stamped return envelope. The MUSIC coordinating center provided practices with tablets for online data collection in the clinic when feasible. For each survey period, an acceptable window for completion was defined, and surveys completed outside of the specified window were excluded from analysis (eg, 3-month surveys were excluded if not completed between 60 and 120 days after date of surgery) (eFigure in the Supplement). Surveys that were not completed or completed outside of the acceptable window were excluded from analysis.

Between April 2014 and July 2018, 4582 men agreed to participate in MUSIC PRO and all were considered eligible for this analysis. Across Michigan, 95 surgeons from 29 urology practices treated these patients with a median (range; interquartile range) of 21 (1-536; 7-55) patients per surgeon.

Clinical and demographic characteristics were reported for all MUSIC PRO participants and compared across groups of survey responders and nonresponders at 3 months using the Wilcoxon rank sum test for continuous measures and χ² testing for categorical measures. Urinary function scores were reported for responders at baseline and 3, 6, and 12 months on a scale from 0 to 100 points. We used 3-month urinary function data to fit a multivariable logistic regression model to evaluate factors associated with patients reporting good 3-month urinary function. In this model, we included variables that have been previously associated with functional outcomes after prostatectomy, including age, body mass index, baseline urinary function score, and log-transformed preoperative PSA level as continuous covariates and race/ethnicity (defined in the registry based on medical record documentation), diabetes status, biopsy Gleason score, clinical T stage, surgical approach (open vs robotic-assisted), and receipt of nerve-sparing surgery as categorical covariates. Men with missing data for any of the evaluated covariates were not included in the model. Regression results were used to stratify patients into risk quintiles based on the predicted probability that an individual would report a 3-month urinary function score of 74 or higher where individuals in the lowest risk quintile were most likely to report good urinary function.

For evaluation of surgeon-level performance, we included surgeons with at least 10 patients reporting a urinary function score of 74 or higher at baseline who also responded to the 3-month PRO survey. We determined the percentage of each surgeon’s patients who reported good urinary function both at baseline and at 3 months to define surgeon-specific variation in outcomes. From this, we identified a group of top-performing surgeons who were above the 75th percentile in the overall sample for the percentage of patients reporting good function 3 months after surgery.

Next, we performed a stratified analysis to evaluate whether top-performing surgeons demonstrated superior outcomes regardless of patient characteristics or because of favorable patient risk profiles. To do this, we evaluated whether the percentage of patients reporting good urinary function at 3 months was different for patients receiving surgery from top-performing surgeons vs those under the care of the remaining surgeons from the collaborative within each of the patient risk quintiles defined during patient-level regression analyses.

Finally, in an effort to evaluate if there was an association between functional outcomes achieved by top-performing surgeons and cancer control, we compared top-performing surgeons with the remaining surgeons across 2 short-term measures of cancer control¹: the percentage of patients achieving a PSA level of less than 0.1 ng/mL (to convert to micrograms per liter, multiply by 1) within 6 months of surgery and the percentage of patients with organ-confined disease (ie, stage pT2) and negative surgical margins. Multivariable regression models were fit in this step to account for patient-level covariates either previously demonstrated to affect these outcomes^17,18 or likely to be associated with outcomes, including patient age, race/ethnicity, body mass index, preoperative PSA level, biopsy Gleason score, and clinical T stage. All statistical testing was 2-tailed, performed using Statistical Analysis System software, version 9.4 (SAS Institute), and P < .05 was considered statistically significant.

Of 4582 men enrolled in MUSIC PRO prior to radical prostatectomy, 4137 (median [IQR] age, 63.8 [58.2-68.5] years) were considered eligible to answer the 3-month survey (ie, were at least 60 days from the date of their prostatectomy). Of these, 3791 (82.7%) completed a baseline survey and 3282 (79.3%) completed a 3-month survey. Of the men who responded to the 3-month survey, 2818 (85.9%) completed surveys within acceptable proximity to surgery and were included for analysis of 3-month outcomes (eFigure in the Supplement). Table 1 contains demographic information of all individuals in the analysis, including survey responders and nonresponders. Compared with men excluded from the 3-month outcome analysis, those included in the 3-month functional analyses were older (mean [SD] age of 63.5 [7.0] years vs 62.7 [7.3] years), more likely to be White (87.8% [2777 of 2818] vs 76.1% [918 of 1319]; P < .001), less likely to have diabetes (11.1% [312 of 2818] vs 14.4% [190 of 1319]), and had slightly higher baseline urinary function scores (mean [SD] score of 88.9 [13.3] points vs 86.5 [16.8] points). They did not differ across the other characteristics reported in Table 1.

Mean (SD) urinary function scores were 88.5 (14.3) at baseline, 53.6 (27.5) at 3 months, 68.8 (25.1) at 6 months, and 73.7 (23.0) at 12 months (Figure 1). The observed variation in patient scores was greatest at the 3-month time point (median, interquartile range [IQR]: 52, 31.3-77.4) (Figure 1). Table 2 presents findings from the patient-level multivariable analysis used to identify factors associated with 3-month urinary outcomes. Older age, body mass index of 30 or higher, and clinical stage T2a or higher were associated with worse 3-month urinary outcomes, whereas patients with higher baseline urinary function and those who received bilateral nerve-sparing surgery were more likely to report good function 3 months after surgery.

A total of 48 surgeons were included in the surgeon-level analyses. Across these surgeons, the percentage of patients who reported good urinary function at both baseline and 3 months varied from 0% to 54.5% (P < .001) (Figure 2). Twelve surgeons were above the 75th percentile for percentage of patients reporting good 3-month function and these were classified as the top-performing surgeons (Figure 2). Annualized prostatectomy volume did not differ for top-performing surgeons (median [IQR] cases per year, 28 [18-50]) relative to those in the 3 lower quartiles for outcomes (median [IQR] cases per year: quartile 1, 17 [14-29]; P = .07; quartile 2, 30 [17-46]; P = .92; quartile 3, 41 [26-52]; P = .58), although there was a nonsignificant difference in volume (median [IQR] number of prostatectomies) between the lowest-performing quartile (17 [14-29]) and highest-performing quartile (28 [18-50]) (P = .07).

Within patient risk quintiles, 3-month outcomes were compared based on whether surgery was performed by a top-performing surgeon or another MUSIC surgeon. Patients who underwent surgery with top-performing surgeons were more likely to report good urinary function 3 months after surgery in all risk quintiles (Figure 3).

After adjusting for patient age, body mass index, race/ethnicity, preoperative PSA, and biopsy Gleason score, the adjusted rate of negative margins in men with pT2 disease was 80.3% (95% CI, 73.1-85.9) among patients who underwent surgery with top-performing surgeons vs an adjusted rate of negative margins of 70.0% (95% CI, 61.3-77.4) among patients of remaining surgeons (odds ratio, 1.75; 95% CI, 1.31-2.32; P < .001). After similar adjustments, there were no differences in the percentage of patients with a PSA level of less than 0.1 ng/mL following surgery between patients of top-performing surgeons (89.4%) vs those of remaining surgeons (87.7%) (odds ratio, 1.18; 95% CI, 0.90-1.54; P = .23).

We report urinary function outcomes following radical prostatectomy from a diverse group of community and academic urology practices in Michigan. Postoperative urinary function outcomes were most substantial 3 months after surgery with improvement in urinary function scores at 6 and 12 months, indicating continued recovery over the first postoperative year. Regression analyses demonstrated that patient age, body mass index, preoperative function, clinical T stage, and receipt of bilateral nerve-sparing surgery were predictors of whether a patient would report good urinary function 3 months after surgery. Surgeon-specific analyses revealed significant variability across surgeons in how often patients with good baseline function also reported good function at 3 months. Top-performing surgeons achieved superior urinary outcomes regardless of patient risk group. An association between surgeon annual case volume and performance quartile was not observed, although there was a nonsignificant difference in volume between the lowest-performing quartile and highest-performing quartile. Top-performing surgeons were also more likely to achieve negative surgical margins among patients with pT2 disease and had similar rates of patients achieving a PSA of less than 0.1 ng/mL following surgery, indicating that superior functional outcomes were achieved without an apparent decrease in surgical rates of cancer control.

These findings build on previous reports of PROs following radical prostatectomy.^4-8 In 2008, one of the first multi-institutional prospectively measured investigations of PROs after prostate cancer treatment demonstrated a substantial decrease in mean urinary function following prostatectomy.⁵ These results have been consistently reproduced over the last decade in various cohort studies.^6-8 Our current report from MUSIC PRO confirmed this while adding perspective given data were captured from more than 90% of urology practices in the state with both community and academic settings being well represented. Our patient-level regression also demonstrated that age, body mass index, baseline function, clinical T stage, and nerve-sparing surgery were important factors for predicting 3-month outcomes. Knowledge of the factors associated with outcomes may augment patient risk stratification and counseling.

To our knowledge, this is the first report of surgeon-specific variation in PRO outcomes following prostatectomy. We conducted this study in an attempt to use a large and diverse data set to move beyond the documentation of outcomes toward the identification of improvement opportunities. The finding that top-performing surgeons achieved consistently superior outcomes after accounting for differences in patient risk is important. This serves as evidence against the hypothesis that top-performing surgeons achieve better outcomes solely because they have favorable patient populations. In their landmark article on surgical technique and bariatric surgery outcomes, the Michigan Bariatric Surgery Collaborative¹⁹ demonstrated that surgeon skill is closely linked to improved postoperative outcomes. It is possible that there is a yet-to-be delineated relationship between surgical skill and outcomes following prostatectomy. In light of this, the ability to reliably identify surgeons who achieve superior urinary outcomes is significant. If specific techniques used by top-performing surgeons can be identified, there may be opportunities to share those strategies with other surgeons to improve outcomes for patients undergoing prostatectomy. Ongoing efforts at MUSIC may help evaluate technical performance²⁰ and identify techniques that should be avoided and those that help top-performing surgeons achieve more favorable outcomes. With growing evidence that coaching interventions can improve surgeons’ skill,^21-23 there may be an opportunity to disseminate these techniques via coaching workshops.

This study has important implications. Prostate cancer remains one of the leading causes of cancer death for men in the US,²⁴ and radical prostatectomy is considered a criterion-standard, often curative treatment.¹ However, as detailed here and elsewhere, treatment-related morbidity can be significant. The ability to limit morbidities associated with prostatectomy through quality-improvement efforts, such as those ongoing in MUSIC, may substantially diminish the risks of prostatectomy for men in need of curative intervention for prostate cancer. Objective identification of surgeons who achieve better outcomes, as we have done in this analysis, represents an important first step.

As we work within MUSIC to improve prostatectomy outcomes for men in Michigan, we are actively using these analyses to guide our efforts. We are striving for 75% of men to achieve social continence within 3 months of surgery and 90% to reach that mark within 6 months. To achieve this, we have launched a multidimensional improvement initiative that leverages MUSIC’s infrastructure and the expertise of objectively identified top-performing surgeons. Our efforts include teaching sessions for MUSIC surgeons led by top-performing surgeons to discuss the technical aspects of prostatectomy, peer-to-peer in-person review of surgical videos, anonymous peer-review of surgical videos, and individualized reports of PROs benchmarked against performance of other surgeons in the collaborative. As we collect videos and develop a formal objective process for reviewing videos to evaluate surgical quality,²⁰ we are also working to link technical evaluations to PROs. This may help us to identify specific techniques associated with better outcomes. The collaborative nature of MUSIC would in-turn allow for rapid dissemination of such findings to surgeons in Michigan and elsewhere. We are hopeful this will represent a path toward a meaningful reduction in morbidities associated with prostatectomy in Michigan and serve as a model to more broadly improve prostatectomy outcomes.

Our analysis had several limitations. First, urinary morbidity was not the only potential adverse effect of radical prostatectomy. For instance, there are well-documented decreases in erectile function and other rare but significant sources of morbidity after prostatectomy. Although we fully recognize the importance of limiting all morbidity after radical prostatectomy and not urinary morbidity alone, we chose to focus our initial efforts on urinary recovery given its substantial effect on quality of life.²⁵ Second, within the surgeon-specific analyses, the inclusion of surgeons with as few as 10 patients reporting good baseline function included some surgeons in the analysis with a relatively small sample size by which to evaluate their performance. Although this may have introduced instability to estimates of individual performance, we performed a sensitivity analysis to increase certainty where we only evaluated surgeons with at least 20 patients in the cohort (27 surgeons were included in this sensitivity analysis), and the results did not substantively differ from the primary findings of the overall analysis. Third, as indicated above and as with any survey-based analysis, possible biases may have been introduced because of missing data or survey nonresponse. Although it is worth recognizing this possible bias, we believe the survey response rate of approximately 80% in all survey periods is sufficient to reduce the effect of limited survey response. The exclusion of 12% of 3-month survey respondents because of missing data is similarly of note, but relatively unlikely to have had a major influence on our findings. Additionally, although we made an effort to account for patient factors that can clearly influence outcomes, there is a possibility that unmeasured elements, such as the utilization of postoperative pelvic floor rehabilitation, may have influenced outcomes in a way that we were unable to account for. Although plausible, we believe it is unlikely that unmeasured factors would substantially change our primary findings.

We have detailed urinary function outcomes for men who underwent prostatectomy procedures in Michigan. We reported substantial patient- and surgeon-level variation in 3-month urinary outcomes. In identifying surgeons whose outcomes consistently exceed those of their peers regardless of patient factors, we have begun to identify a path to improve urinary outcomes following prostatectomy.

Accepted for Publication: November 7, 2020.

Published Online: January 20, 2021. doi:10.1001/jamasurg.2020.6359

Corresponding Author: Gregory B. Auffenberg, MD, MS, Department of Urology, Northwestern University Feinberg School of Medicine, 676 N St Clair St, Arkes 2300, Chicago, IL 60611 (gregory.auffenberg@northwestern.edu).

Author Contributions: Dr Auffenberg and Mr Qi had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Auffenberg, Dunn, Kim, Miller, Sarle, Ghani, Peabody.

Acquisition, analysis, or interpretation of data: Auffenberg, Qi, Dunn, Linsell, Kim, Miller, Tosoian, Sarle, Johnston, Kleer, Montie, Peabody.

Drafting of the manuscript: Auffenberg, Johnston, Peabody.

Critical revision of the manuscript for important intellectual content: Qi, Dunn, Linsell, Kim, Miller, Tosoian, Sarle, Johnston, Kleer, Ghani, Montie, Peabody.

Statistical analysis: Auffenberg, Qi, Dunn, Kleer.

Obtained funding: Miller, Ghani.

Administrative, technical, or material support: Linsell, Kim, Ghani, Montie.

Supervision: Kim, Miller, Sarle, Johnston, Montie.

Conflict of Interest Disclosures: Dr Miller reports support from Blue Cross Blue Shield of Michigan during the conduct of the study. Dr Tosoian reports personal fees and nonfinancial support from LynxDx outside the submitted work. Dr Kleer reports compensation for administrative work from Blue Cross Blue Shield of Michigan during the conduct of the study. Dr Ghani reports grants from Blue Cross Blue Shield of Michigan during the conduct of the study. Dr Montie reports salary support from Blue Cross Blue Shield of Michigan during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was supported in part by the Blue Cross Blue Shield of Michigan Value Partnerships program.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge the significant contributions of the clinical champions, urologists, administrators, and data abstractors in each participating Michigan Urological Surgery Improvement Collaborative practice. Details on specific participating urologists and practices can be found at https://musicurology.com. We also acknowledge members of the Michigan Urological Surgery Improvement Collaborative Coordinating Center at the University of Michigan. In addition, we acknowledge the support provided by the Blue Cross Blue Shield of Michigan Value Partnerships program, specifically David Share MD, MPH; Tom Leyden, MBA; and Rozanne Darland, BSBA. None of these individuals were compensated for their contributions.