Percentage reduction of error before and after the cost card was introduced. Percentage error is the amount by which the estimates varied from the actual cost—in some cases by greater than 100%.
Performance improvement vs postgraduate year of training. Percentage error is the amount by which the estimates varied from the actual cost—in some cases by greater than 100%.
Chandawarkar RY, Taylor S, Abrams P, Duffy A, Voytovich A, Longo WE, Kozol RA. Cost-Aware CareCritical Core Competency. Arch Surg. 2007;142(3):222-226. doi:10.1001/archsurg.142.3.222
Copyright 2007 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2007
Resident core competence can be improved by learning to accurately estimate the costs of postoperative complications.
Prospective, institutional review board–approved study. In step 1, residents were provided 3 clinical vignettes detailing specific treatment measures for postsurgical complications and asked to assign total cost estimates for the treatment for each vignette; in step 2 they were given a pocket-sized cost card listing hospital costs, and in step 3, after 2 weeks, they were retested using the same clinical vignettes as in step 1.
University of Connecticut, Farmington, and the Yale University School of Medicine, New Haven.
Fifty-three general surgery residents.
Main Outcome Measures
Cost estimates for steps 1 and 3 were compared using the paired t test and analysis of variance to examine whether there is a difference between the baseline cost estimates and the actual cost; whether introduction of the cost card improves performance; and whether responses correlate to postgraduate year level or to the clinical vignette.
There was a statistically significant difference between the baseline cost estimates (before introduction of the cost card) and the actual cost of the treatment (P = .03). Introduction of the cost card resulted in a statistically significant improvement between the cost estimates before and after the intervention (P = .002), with a drop in average percentage error by 35% (range, 32%-38%). Level of postgraduate training or type of test vignette (at analysis of variance) did not seem to be a significant factor.
There is a lack of awareness among surgical residents of the cost of treatment of postoperative complications. Introduction of a simple educational tool such as a cost card measurably improves their overall understanding of the cost of care and can be easily incorporated into the residency curriculum.
It is not difficult to recognize that the health care climate is rapidly changing from one that used to be blind to the cost of treatment to one that is under national scrutiny as a result of rising health care costs. Recent developments in health care management and delivery foreshadow these changes. They are centered heavily on 2 fundamental issues: cost and quality of care.1- 3 Newer initiatives including the Pay-for-Performance program initiated by the Centers for Medicare and Medicaid Services and the Federation of Hospitals seek to reward health care systems that deliver the best care at the lowest cost and to “disincentivize” systems that do not. In this competitive and charged environment, only physicians who are willing to learn about the strengths and weaknesses of health care systems and can use them effectively can elicit improvements in patient care.4,5
Physicians have a prominent role in highly interdependent medical systems, be it in large university hospitals or smaller community centers. This centrality places a huge responsibility on physicians to deliver the best and most cost-effective care. Digressing from the understanding of issues of cost and quality would not only be limiting but, in the current health care climate, perilous. Physicians who choose to focus only on patient outcomes and not tally costs may soon find their ability to practice medicine fettered by financial constraints.
We explored how education in health care costs can be imparted at the level of residency training, in the hope that greater cost awareness, instilled early in one's formative years, will result in physicians better equipped to meet the challenges of the near future. Systems-Based Practice is the sixth core competency6 defined by the Accreditation Council for Graduate Medical Education Outcome Project. Specifically, Systems-Based Practice requires that “residents demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.”7 This competency can be divided into the following subcompetencies, all of which are integral to training physicians in general surgery residency programs: resources, providers, and systems; cost-appropriate care; delivery systems; and patient advocacy.
Effective instruction in Systems-Based Practice must incorporate training in the practice of cost-effective health care and resource allocation that does not compromise quality of care. As medical educators, we have an ethical responsibility to the US public, who fund medical education, to train physicians to develop an acute awareness of the costs of medical care in this country and to use the resources wisely.
This prospective study was approved by the institutional review boards at both participating institutions. It included a cohort of 53 participating general surgery residents from departments of surgery in 2 residency programs, at the University of Connecticut, Farmington, and the Yale University School of Medicine, New Haven, Conn. The 3-step educational exercise and its analysis are given in Table 1. In brief, in step 1, residents were provided 3 clinical vignettes (Table 2) detailing common postsurgical complications and their specific treatment measures, and asked to assign cost estimates for the total treatment for each vignette; in step 2, each resident was given a pocket-sized card (Table 3) listing hospital costs, and asked to periodically review it during the course of 2 weeks; and in step 3, they were retested, without access to the cost card, and their estimates for the same clinical vignettes in step 1 were recorded. Percentage error is the amount by which the estimates varied from the actual cost—in some cases by greater than 100%. Cost estimates for steps 1 and 3 were compared using the paired t test (for the entire group) and analysis of variance (for differences categorized by postgraduate year [PGY] or type of vignette). Specific questions addressed were as follows: Is there is a difference between the baseline cost estimates and the actual cost? Does introduction of the cost card improve their performance? Do the responses correlate to PGY level or to the clinical vignette?
Salient features of the conduct of this study included the following. First, its experimental design, which was an entirely fictional academic exercise for resident educational purposes, did not involve any actual patients, nor did it try to change patient care. Second, concerns that, as faculty, we would be coercing the residents to participate were addressed by a simple announcement that residents were free to not participate and that there would be no repercussions for nonparticipation; that not only would their individual identity not be revealed but the study would also record only the PGY level and not the name or identity of the individual resident; that both tests would be conducted by a person not connected to the residency program in any way, thereby completely protecting the personal identity of the residents who participated and those who chose not to; and that the principal investigators (R.Y.C. and R.A.K.) would not be present in the test room at any time. Third, concerns that the study would somehow induce residents to deny required care based on cost were allayed by clear and unambiguous verbalization of the purpose of the study. It was made amply clear that it did not seek to change patient care but only to improve the residents' understanding of treatment costs. Fourth, it was also made clear to all participants that this exercise was an approved research project and that the data collected would be published.
Steps 1 and 3 were compared using the paired t test and analysis of variance. Statistical significance was set at P<.05.
Fifty-three residents in the general surgery program at 2 institutions, University of Connecticut and the Yale University School of Medicine, participated in the study. None of the residents refused participation. Of the participants, 18 (35%) were PGY1; 9 (16%), PGY2; 11 (21%), PGY3; 10 (19%), PGY4; and 5 (9%), PGY5. There was a statistically significant difference between the baseline cost estimates (before introduction of the cost card) and the actual cost of the treatment (P = .002). As noted in Table 1, introduction of the cost card resulted in a statistically significant improvement between the cost estimates for each individual clinical scenario (marked S1, S2, and S3) before (preintervention) and after (postinvervention) the intervention. Cumulative results, shown in Figure 1, indicate a statistically significant decrease in average percentage error of 35% (range, 32%-38%; P = .002). The level of PGY training or type of test vignette (at analysis of variance) did not seem to be a significant factor in the changes in the cost estimates (Figure 2). Although there was clear improvement in each segment, none of the individual scenarios showed any differences (data not shown). Further, there was no difference in the level of improvement that correlated with the type of scenario or the level of training.
In the health care field of the near future, quality of care and costs of postoperative complications of treatment will not only guide reimbursements but also direct referral patterns and enrollment.1- 3,8- 12 This trend will need a concurrent evolution of 2 methods: accurately assessing indicators of quality including outcomes, costs, risks, and benefits; and incorporation of cost education and quality-improvement measures in the training of future physicians. Both methods are critical in meeting the challenges the new health care system poses. Surgical training has long recognized the need for quality improvement in patient care and the need for important measures such as peer review reporting in morbidity and mortality conferences and evidence-based teaching initiatives. Resident education in terms of actual costs will add a new dimension to training initiatives. Our study provides useful guidelines to begin to address the issue of cost education. Study materials such as the cost card are a simple interventional tool that provides education and a reference point in terms of understanding costs of postoperative complications. We recommend a phased approach to enable a more streamlined improvement in the overall awareness of cost of care that will ultimately result in more efficient patient care. This process will require the development, testing, and further refinement of several tools that will add to the current study methods.
Critics might argue that studies such as this lack specificity of results and, hence, the conclusions drawn are of limited value. Admittedly, our study has some inherent limitations. First, it lacks specificity. It is important to emphasize that residents given the cost card were encouraged, but not mandated, to refer to it between the 2 tests. In that sense, this approach raises a valid criticism, that is, this study cannot accurately assess the maximum benefit the cost card potentially could have delivered because there would always be some participants who chose to ignore using the card altogether. Second, the number of residents in each PGY level was small, discounting some of the observations made in the subgroup level. Third, since the study group was not held accountable for their performance (in the traditional sense of a training exercise), their replies could lack overall seriousness and involvement. Studies incorporating concerns with numbers are easily addressable. However, those that pertain to seriousness of the replies and participation can only be addressed if these exercises are performed as part of the residency training curriculum. At the least, we believe the data from this study make a strong argument for the inclusion of educational tools that provide cost-awareness in the curriculum. The fourth and last, but extremely important, concern that studies of this type raise is that residents will use cost as a factor and inappropriately withhold a required treatment or investigation. The only viable defense to this criticism is that adequate education and understanding of why and when certain interventions and investigations are performed will enable trainees to use these resources more efficiently. Limiting care based on cost is not the desired goal; efficient deployment of health care systems is. By raising awareness of the cost of treatment complications, we hope to shed light on the importance of their prevention.
Our data clearly indicate the gap in residents' assessment of the cost of complications in actual dollar values. Introduction of an educational tool such as a pocket-sized cost card measurably improves their overall understanding of cost of care. The interventional measures described in this study are simple and inexpensive and can be easily incorporated into the residency curriculum. We recommend that these educational measures be included in the residency training curriculum, with modifications that may be specific to each program. Measures such as these are proactive ways by which to meet the growing demands of a cost- and quality-conscious health care system, recognizing that design, discipline, and data are essential forebearers of change.
Correspondence: Rajiv Y. Chandawarkar, MD, Division of Plastic Surgery, University of Connecticut School of Medicine, MC 1601, 263 Farmington Ave, Farmington, CT 06030-1601 (Chandawarkar@uchc.edu).
Accepted for Publication: May 30, 2006.
Author Contributions:Study concept and design: Chandawarkar, Taylor, Abrams, Duffy, Longo, and Kozol. Acquisition of data: Chandawarkar, Taylor, Abrams, Duffy, Longo, and Kozol. Analysis and interpretation of data: Taylor, Voytovich, Longo, and Kozol. Drafting of the manuscript: Chandawarkar, Taylor, and Duffy. Critical revision of the manuscript for important intellectual content: Voytovich, Longo, and Kozol. Statistical analysis: Voytovich. Administrative, technical, and material support: Kozol. Study supervision: Chandawarkar, Abrams, Duffy, Longo, and Kozol.
Financial Disclosure: None reported.
Previous Presentation: This study was presented at the annual meeting of the Association of Program Directors in Surgery; April 25, 2006; Tucson, Ariz.