[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.205.176.107. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1.
CUSUM Graph of Risk-Adjusted Morbidity of 193 Sequential Cases
CUSUM Graph of Risk-Adjusted Morbidity of 193 Sequential Cases
Figure 2.
CUSUM Graph of Risk-Adjusted Mortality of 241 Sequential Cases
CUSUM Graph of Risk-Adjusted Mortality of 241 Sequential Cases

FIR indicates fast initial response.

1.
Maguire  T, Mayne  CJ, Terry  T, Tincello  DG.  Analysis of the surgical learning curve using the cumulative sum (CUSUM) method. Neurourol Urodyn. 2013;32(7):964-967.
PubMedArticle
2.
Provost  LP, Murray  SK. The Health Care Data Guide: Learning From Data Improvement. San Francisco, CA: John Wiley & Sons, Inc; 2011.
3.
Koshti  VV.  Cumulative sum control chart. Int J Physics Math Sci. 2011;1(1):28-32. http://www.cibtech.org/J-PHYSICS-MATHEMATICAL-SCIENCES/PUBLICATIONS/2011/Vol%201%20No.%201/01-Koshti.pdf. Accessed November 20, 2014.
Views 578
Citations 0
Research Letter
Association of VA Surgeons
March 2015

Statistical Methods of Risk-Adjusted Statistical Process Control Charts to Assess Surgical Performance in Consecutive Colorectal Operations at a Single Institution

Author Affiliations
  • 1Pittsburgh Research Office, Research StatCore, Pittsburgh, Pennsylvania
  • 2Department of Surgery, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania
JAMA Surg. 2015;150(3):271-272. doi:10.1001/jamasurg.2014.1773

Risk-adjusted cumulative summation is an iterative statistical process control chart centered on 0 that allows for local and individualized risk adjustment.13 Risk-adjusted cumulative summation scores signal special-cause variation if data points cross predefined control limits. We used cumulative summation charts and individual patient-level data at the Veterans Affairs (VA) Pittsburgh Healthcare System to visualize common- and special-cause variations in surgical outcomes in a cohort of patients who underwent colorectal operations. The cumulative summation scores were adjusted for individual risk based on differences between surgical outcome observed (O) (0 = no complication; 1 = any complication) and expected (E) risk derived from the predicted probability of complication (from the VA Surgical Quality Improvement Program [VASQIP]). Risk-adjusted (O − E) cumulative summation charts based on VASQIP probability are a novel and intuitive method of objectively assessing performance in real time with a minimum of 20 patients.

Methods

We created 2 risk-adjusted cumulative summation statistic charts: 30-day postoperative morbidity (n = 193) and mortality (n = 241) among patients undergoing sequential colorectal operations between October 1, 2006, and September 30, 2012 (n = 241, abstracted for the VASQIP). If a patient was assessed to be at high risk for complications (with a score of 0.78) and if the patient did indeed have complications (score of 1), then the patient’s adjusted score would be 0.22 (1 − 0.78). However, if another patient had the same risk for complications (a score of 0.78) but did not have complications (a score of 0), then that patient’s risk-adjusted score would be −0.78 (0 − 0.78). Graphically, the presence of complications is presented as upward spikes. To test for excess risk, two 1-sided C curves, CUSUM statistics C+ and C, were calculated using each patient’s risk-adjusted cumulative summation score so as to be sensitive to the variability of excess successes (C) and failures (C+). Surrounding C+ and C were upper and lower control limits that, when crossed, indicate that variation in the curves surpassed what was expected (special-cause variation).

Results

The cumulative summation chart for 30-day morbidity (75 complications) indicated that excess morbidity occurred between February 21, 2007, and April 3, 2007 (cases 25 and 26). Immediately afterward, there was a sharp decline in the number of complications, and the number of complications continued to decrease until case 36 (June 16, 2007), when the lower limit was crossed to indicate significantly lower-than-expected morbidity until July 17, 2007. Beyond July 17, 2007, morbidity occurred at expected rates, with actual outcome getting progressively closer to expected outcome (ie, both curves hovered closer to 0 over time) (Figure 1).

The curve patterns for 30-day mortality (with a total of 10 patients) showed that mortality occurred as expected up until case 96, although that did not signal special-cause variation. Excess mortality was indicated when 2 patients with a less than 10% expected risk of death died within a short time span (cases 124 and 127 in August 2009). The other 8 deaths were spread out randomly over time and did not signal excess death (Figure 2).

Discussion

The VA uses the O to E ratio method, which estimates performance by using group-level data that produces an O to E ratio that almost always has a value greater than 0. When comparing patient-level outcomes, the ability to adjust for risk using the O to E ratio is limited for patients for whom no complications occurred, and the curve is more dependent on the number of events in a short time span, rather than being dependent on each patient’s outcome related to presurgical risk.

The visual presentation of the risk-adjusted cumulative summation chart (O − E) allows for an examination of the process in control events, in better-than-expected performance, and in out-of-control events. The use of real-time cumulative summation (O − E) complements the existing O to E ratio method to provide an informative, real-time visual representation of ongoing surgical performance. It may provide a more intuitive presentation of the process of care to health care providers and administrators.

Back to top
Article Information

Corresponding Author: Malak Bokhari, MD, MPH, Department of Surgery, VA Pittsburgh Healthcare System, University Drive Campus (112 U), Pittsburgh, PA 15240 (malak.bokhari@va.gov).

Published Online: January 21, 2015. doi:10.1001/jamasurg.2014.1773.

Author Contributions: Dr Bokhari had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Boudreaux-Kelly, Bokhari.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Boudreaux-Kelly.

Administrative, technical, or material support: Wilson, Bokhari.

Study supervision: Bokhari.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 6, 2014; New Haven, Connecticut.

References
1.
Maguire  T, Mayne  CJ, Terry  T, Tincello  DG.  Analysis of the surgical learning curve using the cumulative sum (CUSUM) method. Neurourol Urodyn. 2013;32(7):964-967.
PubMedArticle
2.
Provost  LP, Murray  SK. The Health Care Data Guide: Learning From Data Improvement. San Francisco, CA: John Wiley & Sons, Inc; 2011.
3.
Koshti  VV.  Cumulative sum control chart. Int J Physics Math Sci. 2011;1(1):28-32. http://www.cibtech.org/J-PHYSICS-MATHEMATICAL-SCIENCES/PUBLICATIONS/2011/Vol%201%20No.%201/01-Koshti.pdf. Accessed November 20, 2014.
×