FIR indicates fast initial response.
Boudreaux-Kelly M, Wilson M, Bokhari M. Statistical Methods of Risk-Adjusted Statistical Process Control Charts to Assess Surgical Performance in Consecutive Colorectal Operations at a Single Institution. JAMA Surg. 2015;150(3):271-272. doi:10.1001/jamasurg.2014.1773
Risk-adjusted cumulative summation is an iterative statistical process control chart centered on 0 that allows for local and individualized risk adjustment.1- 3 Risk-adjusted cumulative summation scores signal special-cause variation if data points cross predefined control limits. We used cumulative summation charts and individual patient-level data at the Veterans Affairs (VA) Pittsburgh Healthcare System to visualize common- and special-cause variations in surgical outcomes in a cohort of patients who underwent colorectal operations. The cumulative summation scores were adjusted for individual risk based on differences between surgical outcome observed (O) (0 = no complication; 1 = any complication) and expected (E) risk derived from the predicted probability of complication (from the VA Surgical Quality Improvement Program [VASQIP]). Risk-adjusted (O − E) cumulative summation charts based on VASQIP probability are a novel and intuitive method of objectively assessing performance in real time with a minimum of 20 patients.
We created 2 risk-adjusted cumulative summation statistic charts: 30-day postoperative morbidity (n = 193) and mortality (n = 241) among patients undergoing sequential colorectal operations between October 1, 2006, and September 30, 2012 (n = 241, abstracted for the VASQIP). If a patient was assessed to be at high risk for complications (with a score of 0.78) and if the patient did indeed have complications (score of 1), then the patient’s adjusted score would be 0.22 (1 − 0.78). However, if another patient had the same risk for complications (a score of 0.78) but did not have complications (a score of 0), then that patient’s risk-adjusted score would be −0.78 (0 − 0.78). Graphically, the presence of complications is presented as upward spikes. To test for excess risk, two 1-sided C curves, CUSUM statistics C+ and C−, were calculated using each patient’s risk-adjusted cumulative summation score so as to be sensitive to the variability of excess successes (C−) and failures (C+). Surrounding C+ and C− were upper and lower control limits that, when crossed, indicate that variation in the curves surpassed what was expected (special-cause variation).
The cumulative summation chart for 30-day morbidity (75 complications) indicated that excess morbidity occurred between February 21, 2007, and April 3, 2007 (cases 25 and 26). Immediately afterward, there was a sharp decline in the number of complications, and the number of complications continued to decrease until case 36 (June 16, 2007), when the lower limit was crossed to indicate significantly lower-than-expected morbidity until July 17, 2007. Beyond July 17, 2007, morbidity occurred at expected rates, with actual outcome getting progressively closer to expected outcome (ie, both curves hovered closer to 0 over time) (Figure 1).
The curve patterns for 30-day mortality (with a total of 10 patients) showed that mortality occurred as expected up until case 96, although that did not signal special-cause variation. Excess mortality was indicated when 2 patients with a less than 10% expected risk of death died within a short time span (cases 124 and 127 in August 2009). The other 8 deaths were spread out randomly over time and did not signal excess death (Figure 2).
The VA uses the O to E ratio method, which estimates performance by using group-level data that produces an O to E ratio that almost always has a value greater than 0. When comparing patient-level outcomes, the ability to adjust for risk using the O to E ratio is limited for patients for whom no complications occurred, and the curve is more dependent on the number of events in a short time span, rather than being dependent on each patient’s outcome related to presurgical risk.
The visual presentation of the risk-adjusted cumulative summation chart (O − E) allows for an examination of the process in control events, in better-than-expected performance, and in out-of-control events. The use of real-time cumulative summation (O − E) complements the existing O to E ratio method to provide an informative, real-time visual representation of ongoing surgical performance. It may provide a more intuitive presentation of the process of care to health care providers and administrators.
Corresponding Author: Malak Bokhari, MD, MPH, Department of Surgery, VA Pittsburgh Healthcare System, University Drive Campus (112 U), Pittsburgh, PA 15240 (firstname.lastname@example.org).
Published Online: January 21, 2015. doi:10.1001/jamasurg.2014.1773.
Author Contributions: Dr Bokhari had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Boudreaux-Kelly, Bokhari.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Boudreaux-Kelly.
Administrative, technical, or material support: Wilson, Bokhari.
Study supervision: Bokhari.
Conflict of Interest Disclosures: None reported.
Previous Presentation: This paper was presented at the 38th Annual Surgical Symposium of the Association of VA Surgeons; April 6, 2014; New Haven, Connecticut.