[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.166.89.187. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Table 1. 
Continuous Quality Improvement Project Template
Continuous Quality Improvement Project Template
Table 2. 
Resident Continuous Quality Improvement Curriculum Pretest and Posttest Values
Resident Continuous Quality Improvement Curriculum Pretest and Posttest Values
Table 3. 
Examples of Project Themes From Surgical Residency Continuous Quality Improvement Curriculum
Examples of Project Themes From Surgical Residency Continuous Quality Improvement Curriculum
1.
 Accreditation Council on Graduate Medical Education Web site. General competencies. http://www.acgme.org/outcome/comp/compFull.asp. Accessed October 28, 2006
2.
Rosenfeld  JC Using the morbidity and mortality conference to teach and assess the ACGME general competencies. Curr Surg 2005;62664- 669
PubMedArticle
3.
Murayama  KMDerossis  AMDaRosa  DASherman  HBFryer  JP A critical evaluation of the morbidity and mortality conference. Am J Surg 2002;183246- 250
PubMedArticle
4.
Kravet  SJHowell  EWright  SM Morbidity and mortality conference, grand rounds, and the ACGME's core competencies. J Gen Intern Med 2006;211192- 1194
PubMedArticle
5.
Coleman  MTNasraty  SOstapchuk  MWheeler  SLooney  SRhodes  S Introducing practice-based learning and improvement ACGME core competencies into a family medicine residency curriculum. Jt Comm J Qual Saf 2003;29238- 247
PubMed
6.
Paukert  JLChumley-Jones  HSLittlefield  JH Do peer chart audits improve residents' performance in providing preventive care? Acad Med 2003;78 ((suppl)) S39- S41
PubMedArticle
7.
Sachdeva  AK The new paradigm of continuing education in surgery. Arch Surg 2005;140264- 269
PubMedArticle
8.
Brindis  RGSpertus  J The role of academic medicine in improving health care quality. Acad Med 2006;81802- 806
PubMedArticle
9.
Ogrinc  GHeadrick  LAMorrison  LJFoster  T Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med 2004;19496- 500
PubMedArticle
10.
Holmboe  ESPrince  LGreen  M Teaching and improving quality of care in a primary care internal medicine residency clinic. Acad Med 2005;80571- 577
PubMedArticle
11.
Djuricich  AMCiccarelli  MSwigonski  ML A continuous quality improvement curriculum for residents addressing core competency, improving systems. Acad Med 2004;79 ((suppl)) S65- S67
PubMedArticle
12.
Djuricich  AM A continuous quality improvement (CQI) curriculum for residents. MedEdPORTAL ID 468. 2006;http://www.aamc.org/mededportal. Accessed December 13, 2006
13.
Headrick  LARichardson  APriebe  GPBergman  D Continuous improvement learning for residents. Pediatrics 1998;101768- 774
PubMed
Paper
May 01, 2007

Practice-Based Learning and ImprovementA Curriculum in Continuous Quality Improvement for Surgery Residents

Author Affiliations

Author Affiliations: Departments of Surgery (Drs Canal and Torbeck), Medicine (Dr Djuricich), and Pediatrics (Dr Djuricich), Indiana University School of Medicine, Indianapolis.

Arch Surg. 2007;142(5):479-483. doi:10.1001/archsurg.142.5.479
Abstract

Hypothesis  Surgery residents can learn continuous quality improvement (CQI) principles within a structured curriculum and propose quality improvement projects.

Design  Curriculum within a surgical residency program.

Setting  A university surgical residency program with multiple hospital training sites.

Participants  Fifteen surgical residents during the dedicated research year.

Intervention  A curriculum in CQI that focuses on devising a quality improvement project.

Main Outcome Measures  Resident self-reported attitudes about quality improvement and implementation of resident-initiated quality improvement projects.

Results  Resident survey data demonstrated an improvement in knowledge, self-efficacy, and experiences within CQI. Fifteen individual residents, within smaller teams, created 4 quality improvement projects worthy of implementation.

Conclusions  A structured CQI curriculum can be successfully integrated into a general surgery residency program. Residents can learn the skill of constructing CQI project ideas within the framework of the plan-do-study-act cycle. Residents are eager to make improvements in their local system of residency. By giving them the tools to critically investigate systems improvement and a much needed ear to hear their concerns and suggestions for improvement, we found ways to potentially enhance patient care and developed ideas to improve the education of future surgeons. In doing so, we provided the residents with “buy-in” into their residency program, while addressing the competency of practice-based learning and improvement required by the Accreditation Council for Graduate Medical Education for resident education.

Residency training has undergone many changes within the last several years. Besides the work-hour restrictions, which began in July 2003, the other major change has been the integration of the 6 core Accreditation Council for Graduate Medical Education competencies into residency training. Every residency program must document individual resident competence in patient care, medical knowledge, professionalism, interpersonal and communication skills, practice-based learning and improvement (PBLI), and systems-based practice.

One of the competencies, PBLI, has several important performance-based criteria. One of these criteria is that residents “must perform improvement activities using a systematic methodology.”1 How individual residency programs define improvement activities is variable. Many programs use a morbidity and mortality conference to help the learners think about mistakes and about how to improve one's practice.24 Another method for assessing PBLI is by the use of medical record audits.5,6 However, these methods do not teach residents about the science of improvement. Addressing the topic of continuing education in surgery, Sachdeva wrote:

Continuous professional development activities of practicing surgeons should be integrated with the core competency of practice-based learning and improvement (PBLI), which involves a cycle of 4 steps—identifying areas for improvement, engaging in learning, applying new knowledge and skills to practice, and checking for improvement.7(p264)

Academic medical centers with residency programs should be leaders in teaching quality improvement and should develop educational models for skills training in quality improvement.8

Other specialty training programs have demonstrated that residents can learn quality improvement principles and integrate them into their daily activities, specifically within the purview of the PBLI competency.9,10 To our knowledge, there is no literature about teaching quality improvement to surgical residents.

One of us (A.M.D.) wrote a standard curriculum for internal medicine and pediatrics residents on continuous quality improvement (CQI).11,12 In this curriculum focusing on primary care residents, a curriculum on CQI was provided to residents during a rotation in an outpatient ambulatory setting. After learning about the plan-do-study-act (PDSA) cycle, each resident was asked to apply this model to a real-world project related to improving the residency program or patient care within his or her training venue. Each resident completed a quality improvement project by the end of the month and answered questions related to quality improvement learning before and after the curriculum. We hypothesized that surgical residents could learn about quality improvement and perform improvement activities in the form of a quality improvement project within the department.

METHODS

The Department of Surgery Residency Program at the Indiana University School of Medicine, Indianapolis, graduates 9 chief residents per year. The training venues include a Veterans Affairs hospital, children's hospital, tertiary care adult hospital, large community hospital, and county hospital caring primarily for the underserved community. Residents also rotate through clinics associated with each of these hospitals. All residents are offered the opportunity to pursue 1 or 2 years of research after their second clinical year.

The surgical residents were introduced to a new 6-week quality improvement curriculum in which didactics and discussion were provided by 1 or 2 faculty members for 1 hour each week. A curricular structure was created that included objectives, didactics, a project template, and an evaluation method. Specific content about quality improvement was designed and integrated into a didactic lecture of 60 minutes. The content of this lecture focused on key principles of quality improvement, primarily on understanding the PDSA cycle as it applies to actual health care. One of us (A.M.D.) had written a similar curriculum for primary care residents; therefore, the preparation time for the didactic program was minimal.

The curricular objectives were (1) surgical residents will learn principles and methods of CQI, specifically the PDSA cycle and (2) surgical residents will design and attempt to implement quality improvement projects that focus on improving a particular aspect of the residency program or patient care within their current system.

To assess resident attitudes and knowledge about CQI, a pretest survey was given to the residents as previously described.11 Survey questions covered perceived knowledge, experiences, self-efficacy, and interest. The template used by the residents to organize their projects is given in Table 1.

The curriculum was divided into 3 portions. During the first 2 weeks, residents were taught the PDSA cycle within the model for improvement; residents were taught the importance of identifying and planning an intervention, focusing on the importance of identifying a goal, finding a way to measure it, and carrying out an intervention via implementation.13 Explicit expectations of what they needed to accomplish were provided. These included proposing an idea for a quality improvement project, planning it, and identifying how to measure improvement. Opportunity for resident feedback to the instructors was provided with adequate time for questions.

In the second portion, residents were asked to design a project of their own and to write a preliminary template of how they would implement the project. Options for the project themes were “improve the residency program” or “improve patient care within one of the hospitals or clinics.” These themes were intentionally open-ended for the residents to consider what things they might want to improve, and the final decision for the project theme was left to the residents. They were required to search the literature on the topic of choice, and feedback via e-mail was provided to each of the residents on his or her written project. The mean time for feedback to each resident was about 20 minutes of faculty time per project. Each resident presented his or her project to the other residents, and the residents voted on which projects they would choose to work on together and implement. Teams were created, and team members began to work on their projects as a group. Teams were encouraged to seek faculty help for the projects, depending on the theme. The amount of time each team spent working on the project was variable but was manageable given the timing of this curriculum within the research year. Therefore, clinical duties did not supersede time needed to complete the projects.

In the third portion, residents were required to report their findings about their project to the entire surgical residency in a formal presentation at our surgical resident education hour. Each team had to identify what their specific project goal was, what they were going to do to achieve the goal, and how they were going to measure it. Time was given for questions and answers. The program director and the surgical educator, knowledgeable in quality improvement methods, were present for these presentations. They, along with the entire house staff, offered each presenter feedback on the projects in the form of suggestions and formal written evaluations.

The residents were given a posttest survey about their experiences; these surveys were compared with the pretest surveys with identical questions. Paired t tests were used to test for a statistically significant difference between pretest and posttest scores.

RESULTS

Fifteen research residents were available to participate in this CQI curriculum during a 2-year period. They were all postgraduate year 3 residents assigned to the research year of their residency. Descriptive statistics and results of paired t tests are given in Table 2. Residents showed a statistically significant higher score in the posttest domains of knowledge ( = .001) and experience (P<.001) compared with their pretest scores. Changes in pretest to posttest scores in efficacy and interest were not statistically significant but improved slightly.

The project themes that the residents generated are given in Table 3. Of these, none of the projects from the first year were implemented, and all 4 projects from the second year have been implemented.

COMMENT

Although one might surmise that the outcome of this curriculum would be true implementation of each project with demonstration of improvement, observing the final outcome of each of the projects was not the end point of this curriculum. Rather, having the resident learn the process of working to create effective improvement was the aim. The 4 projects in the second year were implemented with varying degrees of completion. The project on beginning a mentoring system between residents and interns has been put into place. Preimplementation satisfaction survey data were obtained, and the mentors meet regularly with the interns. The satisfaction survey will be repeated at the end of the year to try to identify change, and the overall sense from the interns is that the mentoring system is well received. The project on improving attendance at the chairman's conference demonstrated a 2-fold improvement in attendance after a 6-month period. This was achieved by structuring the topics with reading assignments and by instituting text paging reminders. The project on improving the wait time for non–English-speaking patients requiring cholecystectomy for biliary colic has been started. Data indicated 3 months' wait time from initial presentation to the emergency department or to primary clinics until surgical intervention. Multilingual instructions outlining clinic telephone numbers, locations of facilities, and patient instructions have been designed, printed, and distributed among the emergency department physicians and nurses and in the primary care clinics. A project on improving communication between the emergency department personnel and the trauma surgery residents was delayed because of the inability to obtain funding needed to purchase a wireless telephone, but hospital leadership has responded to the residents' need, with the possibility of purchase in the future. Barriers and obstacles to implementation were experienced by the 15 residents as they worked on their projects. Some of the financial constraints identified and the challenges encountered when working with the hospital administration were eye-opening but were necessary for residents to learn how systems operate.

We discovered barriers to successful project implementation during the first year and made changes to the second year's curriculum. Although it was easy to deliver the didactics both years, getting the residents to follow through on the projects was challenging during the first year. We began the didactic session in February, which left a shorter time than needed to implement the projects. While it was important for residents to learn CQI methods, we discerned that working on a project was needed to solidify the process. This mirrors literature suggesting that experiential learning of quality improvement is ideal.8 Therefore, in the second year we focused on attempting to achieve outcomes, that is, actually following through on the project proposals. We started much earlier in the year, and after each individual had completed the PDSA cycle curriculum, we asked the residents to narrow down their number of projects and to work in groups. We were able to provide protected time for the projects. We believe that providing this curriculum during the research year, when clinical demands are not competing, is more likely to produce projects that could come to fruition.

The culture of our residency program has already changed, possibly based on this curriculum. For example, the team that organized the mentoring system has conducted several meetings of research residents with the interns. The interns seem excited to have an ombudsman within the department and indicate that this resident and intern mentoring process is working well. The resident and intern mentoring system complements a faculty and intern mentoring program that was already in place.

The residents evaluated the curriculum; they stated that the process is a good one and that the curriculum gives them a formal way to learn the science of improvement. They commented that this curriculum gives them a way to have a say in their training, specifically how to improve the training. We believe that this curriculum has greater possibilities and that residents can be agents of change and future leaders within departments of surgery and within hospital systems. By learning the process of CQI and the PDSA cycle, residents can tackle system issues and learn to integrate their innovative solutions into their daily activities to achieve change that represents improvement. It also allows hospital leadership and residency educators to understand the frustrations and barriers of the training environment that residents face and the trainees' ideas about remedying the barriers. Residents experienced that what initially may have seemed to be a simple concern or problem can prove to be much more complicated than expected to fix. This is a key learning point for residents, that barriers to implementation must be considered when planning improvement.

Another success from this curriculum is that residency programs can assess and document competency in PBLI. Each team's written project is placed into its members' competency portfolio files. This is important for the residency review committee evaluations and formally documents individual resident involvement in quality improvement activities.

CONCLUSIONS

A structured CQI curriculum can be successfully integrated into a general surgery residency program. Residents can obtain knowledge of the CQI principles, as well as skills in constructing CQI project ideas within the framework of the PDSA cycle.

Residents are eager to make improvements in their local system of residency. By providing them with the tools to critically investigate systems improvement and a much needed ear to hear their concerns and suggestions for improvement, we are finding ways to potentially enhance patient care and developing ideas to improve the education of future surgeons. In doing so, we provide the residents with buy-in into their residency program, while addressing the competency of PBLI required by the Accreditation Council for Graduate Medical Education for resident education.

Integrating PBLI skills early in a surgeon's career will form a foundation for lifelong quality monitoring. The resulting competence will be useful throughout his or her practice in providing improved patient care.

Back to top
Article Information

Correspondence: David F. Canal, MD, Department of Surgery Residency Program, Indiana University School of Medicine, Emerson Hall 202, 545 Barnhill Dr, Indianapolis, IN 46202 (dcanal@iupui.edu).

Accepted for Publication: January 17, 2007.

Author Contributions:Study concept and design: Canal, Torbeck, and Djuricich. Acquisition of data: Canal and Torbeck. Analysis and interpretation of data: Canal, Torbeck, and Djuricich. Drafting of the manuscript: Canal, Torbeck, and Djuricich. Critical revision of the manuscript for important intellectual content: Canal and Djuricich. Administrative, technical, and material support: Canal, Torbeck, and Djuricich. Study supervision: Canal and Torbeck.

Financial Disclosure: None reported.

Previous Presentation: This study was presented at the 114th Annual Meeting of the Western Surgical Association; November 15, 2006; Los Cabos, Mexico. The discussions that follow this article are based on the originally submitted manuscript and not the revised manuscript.

References
1.
 Accreditation Council on Graduate Medical Education Web site. General competencies. http://www.acgme.org/outcome/comp/compFull.asp. Accessed October 28, 2006
2.
Rosenfeld  JC Using the morbidity and mortality conference to teach and assess the ACGME general competencies. Curr Surg 2005;62664- 669
PubMedArticle
3.
Murayama  KMDerossis  AMDaRosa  DASherman  HBFryer  JP A critical evaluation of the morbidity and mortality conference. Am J Surg 2002;183246- 250
PubMedArticle
4.
Kravet  SJHowell  EWright  SM Morbidity and mortality conference, grand rounds, and the ACGME's core competencies. J Gen Intern Med 2006;211192- 1194
PubMedArticle
5.
Coleman  MTNasraty  SOstapchuk  MWheeler  SLooney  SRhodes  S Introducing practice-based learning and improvement ACGME core competencies into a family medicine residency curriculum. Jt Comm J Qual Saf 2003;29238- 247
PubMed
6.
Paukert  JLChumley-Jones  HSLittlefield  JH Do peer chart audits improve residents' performance in providing preventive care? Acad Med 2003;78 ((suppl)) S39- S41
PubMedArticle
7.
Sachdeva  AK The new paradigm of continuing education in surgery. Arch Surg 2005;140264- 269
PubMedArticle
8.
Brindis  RGSpertus  J The role of academic medicine in improving health care quality. Acad Med 2006;81802- 806
PubMedArticle
9.
Ogrinc  GHeadrick  LAMorrison  LJFoster  T Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med 2004;19496- 500
PubMedArticle
10.
Holmboe  ESPrince  LGreen  M Teaching and improving quality of care in a primary care internal medicine residency clinic. Acad Med 2005;80571- 577
PubMedArticle
11.
Djuricich  AMCiccarelli  MSwigonski  ML A continuous quality improvement curriculum for residents addressing core competency, improving systems. Acad Med 2004;79 ((suppl)) S65- S67
PubMedArticle
12.
Djuricich  AM A continuous quality improvement (CQI) curriculum for residents. MedEdPORTAL ID 468. 2006;http://www.aamc.org/mededportal. Accessed December 13, 2006
13.
Headrick  LARichardson  APriebe  GPBergman  D Continuous improvement learning for residents. Pediatrics 1998;101768- 774
PubMed

Leigh Anne Neumayer, MD, Salt Lake City, Utah: The authors presented a very important experiment, if you will, of incorporating quality improvement education into their residency program. This article is timely not only for residency programs that have been struggling to comply with the last 2 of the 6 competencies required by the ACGME [Accreditation Council for Graduate Medical Education] but also for practicing surgeons who will need to comply with maintenance of the certification component “Evaluation of Performance in Practice.” Although last year I told you that participation in a program such as NSQIP [National Surgical Quality Improvement Program] was sufficient for this component, the American Board of Medical Specialties [ABMS] has stated that they expect more. Specifically, for this component the ABMS expects that each physician at least once every 5 years has identified an area of their practice in need of improvement, designed and implemented a plan for improvement, and measured the results. What many fail to realize is that many of us already do at least part of this. In fact, at least 11 (39%) of the 28 papers presented at this meeting could be considered examples of practice improvement.

Dr Canal and his colleagues have successfully designed a curriculum to teach residents methodology enabling them to meet these competencies. I have 4 questions: (1) How many hours were spent with this curriculum in lectures or small group sessions over the 6-week period mentioned? Were there weekly or more frequent meetings? Was this required? (2) Are the syllabus and lecture slides available to those of us who would like to copy your work in our own programs? (3) Do you have plans for tracking the results of the projects that appear to get off the ground (nothing beats positive feedback to improve behaviors)? (4) Do you have plans to expand this to your entire residency?

Dr Canal: The curriculum was 1 hour that always stretched to about an hour and a half or more on Tuesday mornings at 7 AM with the research residents. We met weekly for 6 weeks initially. That was the groundwork. Then, the residents proceeded with their project work. We met probably every 2 or 3 months to follow up on how things were going. This was required of all the research residents 1 time. If they were in the laboratory for 2 or 3 years, they were not expected to repeat the didactics, but we did ask them to come for the project review.

We can certainly make the syllabus and lecture slides available as necessary.

I think it is a great idea of tracking the results of the projects. I think it is a nice idea to keep a list in our house staff room that shows what the residents have initiated and what the residents have completed. The results of the projects are also kept in the resident portfolios.

Do we plan to expand this to the entire residency? I want to expand it so that everybody in our residency has this experience. Specifically, we need to capture the residents who do not spend time in research.

Alden H. Harken, MD, Oakland, Calif: I think it is a nice example of how asking the question will change the answer in a very constructive fashion.

As I looked through the various competencies, you glided through the first 4, saying it is pretty easy to deal with those. In my experience, the interpersonal skills and professionalism competencies are the stuff that get us and our residents (any folks, even out in practice) in trouble. As identified earlier this week, our surgical personalities occasionally glide into arrogance and sometimes even evolve into being a jerk. That is what gets us in trouble. You indicated that there was a difference in the way the lower-level residents (PGY [postgraduate year]-1’s, -2’s, and -3’s vs the PGY-4’s and -5’s) looked at interpersonal skills and professionalism. Is this something that your own elementary school teachers and moms taught you that doesn't change?

Dr Canal: I think we can read communications skills into several of the projects. The residents were frustrated at the inability to easily talk to the ER [emergency room] nurses on busy trauma nights. Getting a wireless phone that they can call directly on was appealing. In our county hospital, we have such a phone available to the trauma team. At the private hospital that also cares for a lot of trauma, they do not have one. The residents had their eyes opened when they saw how expensive it was to get a wireless phone. They were surprised that the county hospital had 1 and the private one did not. I think that project helped improve communication skills.

Professionalism. I think you are right that these projects did not address it directly. The project trying to improve communication with non–English-speaking patients demonstrated insight into professionalism.

I think the difference in the PGY years had to do more with who perceived it was a problem. The chief residents thought the wireless phone would not be useful, but they would not be the ones using it. The midlevel residents, who would be using it all the time, all thought it would be useful. So that is what I meant by the different years had very different perspectives.

Terry M. Gilliland, MD, Denver, Colo: Do you plan to expand your experience with residents reflected in this paper and use it for your own staff? Our mentors are heroes in many ways, and yet at times we have all witnessed or emulated behavior that would not be considered the best way to enact change. I believe this great program you have enacted for residents could be transformational for staff surgeons as well. Can you comment?

Dr Canal: It is something I have not thought about in detail. I think I will take that back to think about, to work on options to get our faculty to go through the thought process of change. I think faculty are quick to suggest changes, be very aggressive in their attitudes, without thinking through the true measurements and the outcome changes that they are going to get at the end. That is a great idea.

Financial Disclosure: None reported.

×