Flowchart showing steps to develop predictive models required to price case-specific warranties to cover the cost of adverse outcomes (AOs) associated with elective colon resections at effective and efficient hospitals. LOS indicates length of stay.
Adverse outcome (AO) rates and total costs of care for hospitals with significantly higher-than-standard risk-adjusted rates of AOs and for hospitals with substantially higher-than-standard risk-adjusted individualized costs compared with corresponding average rates and costs for reference hospitals.
Effects of risk factors on predicted mortality rates and on predicted rates of adverse outcomes (AOs) (ie, mortality plus prolonged risk-adjusted postoperative length of stay). For all study cases, the average mortality rate is 1.0% and the average AO rate is 7.8%. Predicted mortality and AOs for the least severe cases are labeled “intercept” because no additional risk factors are present. Patient A is a 68-year-old woman with chronic lung disease. Patient B is a 72-year-old man with chronic liver disease. Patient C is a 76-year-old woman with chronic renal failure.
Effects of risk adjustment on the costs of routine cases, on the price of surgical warranties, and on the total costs of care. For cases at effective and efficient hospitals, average routine costs are $9843, average warranty costs are $1294, and average total costs (including average stop-loss allocations) are $11 413. Corresponding costs for the least severe cases are labeled “intercept” because no additional risk factors are present. Patient A is a 68-year-old woman with chronic lung disease. Patient B is a 72-year-old man with chronic liver disease. Patient C is a 76-year-old woman with chronic renal failure. Warranty costs assume a larger proportion of total costs as patients' overall risks increase.
Fry DE, Pine M, Jones BL, Meimban RJ. Surgical Warranties to Improve Quality and Efficiency in Elective Colon Surgery. Arch Surg. 2010;145(7):647-652. doi:10.1001/archsurg.2010.106
Copyright 2010 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2010
Uncomplicated surgical care has highly variable costs. High costs of complications have led payers to deny additional payments even for predictable complications.
A payment warranty indexed to effective and efficient hospitals can promote quality and economic stewardship in surgical care.
Analysis of hospital costs for elective colon surgery in the Healthcare Cost and Utilization Project's National Inpatient Sample from 2002 through 2005.
A 20% sample of acute care hospitals in the United States.
Patients and Methods
Data for elective colon resections were used to create predictive models for adverse outcomes (AOs) and costs. Total hospital costs were determined using cost-to-charge ratios. Costs of AOs were computed as total costs minus predicted costs of uncomplicated care. Surgical warranties were computed as the probability of AOs times per-case predicted costs of AOs. Final predictive models were calibrated using data only from effective and efficient hospitals.
We studied 51 602 cases from 632 hospitals. There were 4048 (7.8%) AOs with 505 deaths (1.0%); 19 hospitals had excessive AOs and 95 hospitals had excessive costs. For 518 effective and efficient hospitals, total per-case costs for routine care were $9843 with an average warranty of $1294 and a $276 stop-loss allocation. This cost model would reduce national expenditures for colon surgery by 6%.
Complications and costs of care can be indexed to quality performing hospitals. Warranties for surgical care can reward effective and efficient care and preclude the need for additional payments for complications.
Costs of treating complications of care are frequently cited as important contributors to the high cost of health care. However, the existing payment system often financially rewards poor quality. Fee-for-service, per diem, and even diagnosis-related, group–based payments may increase hospital and physician revenue when complications require additional services. Current payment systems may actually penalize hospitals and physicians (ie, providers) who invest in lowering complication rates and reducing the severity of complications. In these situations, payments are reduced but providers do not share in the savings.
Payers have introduced pay-for-performance programs to encourage higher-quality care.1 These programs offer bonuses to providers who score well on predominantly process-based performance measures. Unfortunately, little evidence shows that these programs substantially reduce rates or severity of complications.2
The Centers for Medicare and Medicaid Services have begun denying payment to hospitals to cover the cost of selected hospital-acquired complications (ie, “never” events).3 Private insurers have introduced similar programs. While this policy may be appropriate for true “never” events like wrong-site surgery, the occurrence of most complications (eg, surgical site infection or Clostridium difficile enterocolitis) is at least partly dependent on risk factors beyond the control of providers.4 Patient characteristics, comorbidities, and limitations of current technology affect outcomes. As a result, nonpayment strategies unjustly penalize providers and create strong financial disincentives to treat higher-risk patients.
The Geisinger Clinic offers a fixed-price warranty that limits payers' financial responsibility for added costs of treating complications.5 Because Geisinger retains warranty payments in excess of the total cost of caring for patients with potentially avoidable complications, it can benefit financially by reducing rates and severity of complications.
Health care warranties can create an equitable financial arrangement that encourages improved quality and reduced costs. However, warranties such as those offered by Geisinger have several critical limitations. First, because the price of a warranty for a procedure is the same for all patients, there is a financial disincentive to treat patients with an increased risk of complications. Second, the price of the warranty is set arbitrarily at 50% of the total costs of complications. Third, there is no protection against costs associated with single catastrophic complications that may be unrelated to quality of care.
Payment reform initiatives could benefit by incorporating a more refined approach to pricing and administering health care warranties. This is particularly important for surgeons and hospitals as bundled payment systems become increasingly prevalent. The Medicare Payment Advisory Commission has recommended that the Centers for Medicare and Medicaid Services explore systems to bundle payments,6 and the latter has initiated the Medicare Acute Care Episode Demonstration Project, which bundles payments for all hospital, physician, and postdischarge care associated with coronary artery bypass graft and joint replacement procedures.7 This study examines the effect of a warranty calculation for the index hospitalization that (1) adjusts payments based on patient risk, (2) accurately reflects the costs of complications, and (3) protects providers from extraordinary costs associated with catastrophic complications.
We analyzed hospital administrative claims data from the National Inpatient Sample of the Healthcare Cost and Utilization Project database for 2002 through 2005. Elective colon resections (International Classification of Diseases, Ninth Revision, Clinical Modification procedure codes 45.71 through 45.79, 45.8, 48.62, and 48.63)8 were studied for patients older than 17 years who had operations within 2 days of elective admissions to hospitals that performed more than 20 procedures during the study period. Abdominoperineal resections were excluded. Analyses were performed using SAS statistical software, version 18.104.22.168 The flowchart in Figure 1 outlines the steps used to derive and price the surgical warranty. Cases at hospitals that failed quality screens for administrative coding were eliminated.10 Costs were derived from total billed charges and hospital-specific cost-to-charge ratios.11 Predicted costs were adjusted to local wage indexes and geographic locations.
Candidate risk factors were crafted from administrative data elements.12 Hospital dummy variables were included in initial predictive models to reduce the effect of possible correlations between case mix and quality of care on variable selection and weighting.13 These hospital variables were removed from final models so that only valid patient risk factors were used to predict outcomes and costs. For outcomes other than mortality, we included only live patients who were discharged.
A list of general surgical complication codes was prepared. Specific complications of colectomy (eg, leaking anastomosis or abdominal reoperation) were added. Total postoperative hospital days beyond a 3-day minimum were computed for each discharged live patient to avoid the disproportionate influence of patients with very short lengths of stay.14
An adverse outcome (AO) was defined as an inpatient death or a risk-adjusted postoperative length of stay (RApoLOS) outlier. Forward stepwise regressions were used to derive predictive models for inpatient deaths and for postoperative lengths of stay. After identifying RApoLOS outliers, a predictive model for prolonged RApoLOS was derived.10
The probability of an AO was computed as P(death) + [1 – P(death) × P(RApoLOS outlier)], where P(death) is the probability of death and P(RApoLOS outlier) is the probability of a discharged live patient being a RApoLOS outlier. Predicted AO rates for each hospital were the sum of predictions divided by the total number of cases.
Cases were divided into 2 groups for cost modeling: those without AOs (routine cases) and those with AOs.
A lower limit for hospital costs was derived by computing the 10th-percentile cost of cases at each hospital and setting the lower limit equal to the 10th percentile of these costs at all hospitals. This minimum cost was used to ensure that estimates would not be biased by cases in which essential services had not been provided. Costs exceeding this lower limit in cases without AOs were considered to represent the costs of individualized routine services. A risk-adjustment model for the hospital cost of colon resection without an AO was derived using forward stepwise linear regression.
Costs attributable to AOs were computed by subtracting the predicted routine hospital costs from the total observed hospital costs of each case with an AO. These excess costs of AOs were limited to 10 times the routine predicted individualized costs to avoid penalizing providers for catastrophic complications, with these additional costs assigned to a stop-loss pool. A model to compute predicted excess costs of AOs was derived by applying forward stepwise linear regression to cases with AOs, using candidate variables and inpatient mortality as predictors of excess costs.
For each case, a predicted excess cost was computed as the sum of (1) the probability of dying times the predicted excess cost associated with inpatient mortality (p[death] × p[excess cost of death]) and (2) the probability of a prolonged RApoLOS times the excess costs of prolonged RApoLOS (p[RApoLOS outlier] × p[excess cost of RApoLOS outlier]).
To remove the added cost of ineffective or inefficient care, ineffective hospitals with significantly higher-than-average risk-adjusted AO rates and, subsequently, inefficient hospitals with excessive risk-adjusted routine costs were removed from analyses to create a reference database. In both cases, outliers were identified using predetermined quantitative and statistical criteria and were removed only if the standard deviation of z statistics for all hospitals was significantly greater than 1 (P < .05) and there were more than 2 outlier hospitals. The process of identifying and removing outlier hospitals was repeated until no further outliers were eligible for removal.
To permit accurate identification of outlier hospitals, the predicted AO rate for all cases was set equal to the observed AO rate. A hospital was defined as ineffective if the observed AO rate exceeded its predicted rate by more than 2.56 SDs (P < .005) and the hospital had more than 4.5 predicted AOs or more than 7 observed AOs. Volume requirements kept small-volume hospitals from being discarded for small numbers of outlier events. After removing ineffective hospitals, the total predicted individualized routine cost for all remaining cases was set equal to the total observed routine individualized cost using a standard mathematical transformation. A hospital was considered inefficient if (1) its average observed individualized routine cost exceeded its corresponding predicted cost by more than 3.29 SDs (P < .001) and (2) its observed individualized costs exceeded twice the predicted costs for that hospital or exceeded by 1 SD the difference between observed and predicted individualized costs for all hospitals.
To ensure that warranty prices would be adequate to cover the cost of complications observed at reference hospitals, predictive models for clinical outcomes and costs were recalibrated to set total observed and predicted values equal for all cases remaining in the reference data set. Recalibrated models were used to compute the risk of an AO and its associated predicted excess cost for each case at each hospital. The price of a warranty for each case was computed as the product of the risk of an AO and its predicted excess cost. Because predictions reflected the average performance of effective and efficient hospitals in the reference database, warranties were revenue neutral for effective and efficient hospitals with average incidence and cost of AOs.
There were 632 hospitals with 51 602 cases in the analytic database. The linear predictive model for length of stay had 18 significant variables, an intercept (ie, minimum) of 3.08 days (ie, minimum predicted total hospital stay of 6.08 days), an r2 statistic of 0.236 with hospital variables included, but an r2 statistic of 0.085 after hospital variables were removed. The logistic predictive model for mortality consisted of 15 variables with a C statistic of 0.836 after removal of hospital variables. The logistic prediction model for RApoLOS outliers consisted of 18 variables with a C statistic of 0.669 after removal of hospital variables. There were 505 inpatient deaths (1.0%) and 3543 discharged live patients (6.9%) with prolonged RApoLOS for a total of 4048 AOs (7.8%).
The cost of required hospital care was $4723. Observed individualized hospital costs were calculated by subtracting the required cost from the total cost of 47 554 routine cases. The linear predictive model for individualized costs had 27 variables and an r2 statistic of 0.515 with hospital variables included and 0.074 after removing hospital variables. The intercept for the individualized cost model for all hospitals was $4499. Risk adjustment increased this individualized cost by an average of $1219 per case.
The linear predictive models of the excess cost of AOs in the 4048 cases with AOs had 10 variables with an r2 statistic of 0.277 with hospital variables included and 0.076 after hospital variables were removed. The intercept for predicted excess costs of an AO was $14 122. Risk adjustment increased excess costs by an average of $4235. Death added an additional $2339 to the predicted excess cost of any case with an AO. The minimum per-case price of a revenue-neutral warranty for AOs at all hospitals was $357. The average price of a warranty to cover these costs was $1446. An additional $282 per case was allocated to the stop-loss pool to cover extraordinary costs of catastrophic AOs.
No hospitals were outliers based on death rates, but 19 had higher-than-predicted AO rates. Ninety-five hospitals had excessive risk-adjusted individualized costs. The remaining 518 (82.0%) effective and efficient hospitals were used as a reference standard for quality and cost comparisons (Table). Standardized (ie, risk-adjusted) AO rates were 7.5% at 518 effective and efficient (ie, reference) hospitals, 15.1% at 19 hospitals with suboptimal effectiveness, and 7.4% at 95 high-cost hospitals.
On the basis of reference hospital data, minimum per-case individualized costs were $4029 and minimum excess costs for each AO were $12 442. Average per-case observed and predicted individualized costs both were $5120 at reference hospitals; $4728 and $5310, respectively, at hospitals with suboptimal quality; and $11 041 and $5072, respectively, at inefficient hospitals. Average observed and predicted excess costs of AOs both were $1294 at reference hospitals; $2327 and $1278, respectively, at hospitals with suboptimal quality; and $2061 and $1406, respectively, at inefficient hospitals. Per-case stop-loss allocations were $276 at reference hospitals, $650 at ineffective hospitals, and $149 at inefficient hospitals. The relationship between AO rates (ie, effectiveness) and total cost (ie, efficiency) is illustrated in Figure 2.
Indexing predicted costs to effective and efficient hospitals reduced average per-case predicted individualized costs by $603 and reduced average predicted excess costs per AO by $1079 (ie, an average savings of $152 per case). Using this reference standard, the revenue-neutral average price of a warranty would be $1294. Figure 3 and Figure 4 illustrate differences in projected AO rates and warranty prices in prototype patients.
Bundled payments for episodes of care are being promoted as an alternative to fee-for-service reimbursement. As hospitals and surgeons become increasingly interdependent and payers continue shifting risk to providers, it is important for surgeons to become familiar with payment strategies that will protect their incomes and reward high-quality care. This study presents an equitable solution to payer concerns about current financial arrangements that often reward poor-quality providers. Methods described in this study can support warranty payments that eliminate inequities associated with complication severity, hold providers to realistic and achievable standards, reward quality, and encourage innovation. With expanded data sets, total physician and postdischarge costs can be integrated into the comprehensive warranty.
Serious AOs that substantially altered the cost of care were identified using an objective composite measure. For example, diagnosis-based measures might classify both a small draining stitch sinus and an extensive deep incisional infection with fascial necrosis as surgical site infections and treat both as surgical complications. In contrast, because the proposed measure recognizes the substantive effect of AOs on subsequent care, it would identify only the deep infection as an AO.
Robust risk adjustment enables warranty prices to reflect each patient's probability of experiencing an AO so that providers are not penalized for treating sicker patients. It also permits analysts to distinguish excess costs associated with AOs from risk-adjusted costs of routine care in the absence of serious complications. Additional risk adjustment permits warranty prices to reflect patient-specific differences in predicted costs of treating AOs when they occur.
Accurate predictions of the cost of routine care are required to ensure that differences between observed costs and predicted routine costs are fair estimates of the excess costs of AOs. In this study, minimum routine costs were established to protect providers against underestimations of the true cost of high-quality routine care. Stop-loss limits were applied to these costs to limit the influence of extreme cases on predictive models and to prevent relatively rare catastrophic AOs from unfairly penalizing high-quality providers. These stop-loss limits were set high enough to create substantial financial penalties for consistently poor-quality care.
Warranty payments are based on the actual performance of a reference set of effective and efficient providers: providers who demonstrate high-quality outcomes and good economic stewardship. This approach ensures that fee schedules are fair and reflect the true cost of high-quality care by ordinary providers. In this study, 82.0% of hospitals met requirements of effectiveness and efficiency and were included in the reference standard. The process of setting predicted costs used to price warranties equal to observed (ie, actual) costs for reference hospitals ensures that the total amount paid on the warranty will equal the total cost of effective and efficient care. This protects providers from insufficient payments to cover the costs of high-quality care. The warranty design is equivalent to an outcomes-based pay-for-performance program. Providers that have lower rates of AOs or less severe AOs will benefit financially from the warranty.
Although the methods used may appear complex, they are based on concepts that relate directly to clinical practice and can be applied to other surgical and acute medical conditions. Hybrid databases that combine coded procedures and diagnoses with present-on-admission modifiers and admission laboratory data can provide sufficient information to monitor and risk-adjust outcomes without expensive abstraction of medical records.15- 17 When readmission and outpatient data are available, warranties can be expanded to cover postdischarge events.
Correspondence: Donald E. Fry, MD, Michael Pine and Associates, 5020 S Lake Shore Dr, Ste #304N, Chicago, IL 60615 (firstname.lastname@example.org).
Accepted for Publication: February 8, 2010.
Author Contributions:Study concept and design: Fry and Pine. Acquisition of data: Fry, Pine, Jones, and Meimban. Analysis and interpretation of data: Fry and Pine. Drafting of the manuscript: Fry and Pine. Critical revision of the manuscript for important intellectual content: Fry, Pine, Jones, and Meimban. Statistical analysis: Fry, Pine, and Jones. Administrative, technical, and material support: Meimban. Study supervision: Fry and Pine.
Financial Disclosure: None reported.
Previous Presentation: This paper was presented at the 117th Scientific Session of the Western Surgical Association; November 10, 2009; San Antonio, Texas; and is published after peer review and revision. The discussions that follow this are based on the originally submitted manuscript and not the revised manuscript.
John Weigelt, MD, Milwaukee, Wisconsin: Dr Fry and colleagues bring a payment topic to our attention, one that tries to marry quality and cost. Payment reform is definitely part of health care reform today. Unfortunately, what the reform will be is unknown, but some marriage of quality and cost will likely prevail. Thus, this manuscript is timely for a surgical meeting, although I will admit that the devil is truly in the details.
Now, we all experience warranties in our daily life. A warranty is offered by the supplier of the service or product when the supplier believes risks can be controlled. This paper tries to define risks for colon resections and demonstrates a method of payment that warrants the care. The concept is not new. Dr Fry mentioned proven care by Geisinger. There is also a Prometheus project for warrants of an acute and chronic care condition, so there are a number of methodologies, and the question is, are Dr Fry and his colleagues predicting a reimbursement future? Before we go there, I have 3 questions.
First, is your methodology equal to or better than existing warranty calculations? Second, did you do appropriate sensitivity analyses on your assumptions to examine which factors might alter your results or impact payment the greatest? Third, you also used colon resection in your model. Will a model need to be developed for each condition we have a warranty for? If so, based on the length of your manuscript, how difficult will this be?
Dr Fry: In answer to Dr Weigelt's question, how does this methodology differ from that proposed by the Prometheus project? How does it differ from ProvenCare by the Geisinger Clinic? The Prometheus and the ProvenCare warranties have one size fits all. I would argue that that is the best way I know of to implement cherry picking in health care. Everybody will take the favorable-risk patients and send the bad-risk patients elsewhere. That is what will happen. There has to be risk adjustment. Our warranty is different because of risk adjustment. We would even argue that a prejudice or bias in the warranty ought to be implemented to enhance payment for high-risk patients so that medical centers will feel inclined to take them.
Regarding the sensitivity of the analysis, we have done more sensitivity analyses than you could ever believe. I actually believe that our threshold for identifying AOs is very sensitive. I think we would agree that our AO methodology is very effective in the identification of severe outliers, but it may leave some of the lesser degree of outliers within the reference group of patients. We are engaged in extensive efforts to identify whether 3 σ is the correct threshold or whether it should be geared down to 2.5 σ. This is really a quality control effort that is taking the industrial manufacturing model and seeing if we can make it work in health care. This clearly remains controversial.
We have now done 20 surgical models. We have now developed a diabetes model in which we have taken the episode of care as being a full year in the treatment of a diabetic patient and have studied several hundred thousand cases. I think that the surgical models will eventually come down to having common risk factors, but the complications are going to be different and the thresholds for the AOs are going to be different. To that end, there is probably going to be some commonality in terms of those risk factors among the operations that predict an AO. However, complication rates and AO rates will be specific to the specific operation. These rates of AOs will even vary more between operations when postdischarge readmissions are brought into the analysis. We hope that we can report on the impact of readmissions next year.