Design of the study. Baseline assessment (BA) consisted of a survey and the Pediatric Advanced Life Support test. The outcome assessment (OA) consisted of a survey, the Pediatric Advanced Life Support test, a short-answer test, skills stations, and a videotaped resuscitation.
Nadel FM, Lavelle JM, Fein JA, Giardino AP, Decker JM, Durbin DR. Teaching Resuscitation to Pediatric ResidentsThe Effects of an Intervention. Arch Pediatr Adolesc Med. 2000;154(10):1049-1054. doi:10.1001/archpedi.154.10.1049
To evaluate the effectiveness of an educational intervention on pediatric residents' resuscitation fund of knowledge, technical skills, confidence, and overall performance.
Prospective, nonconcurrent, controlled interventional trial.
Urban pediatric tertiary care hospital.
An intervention group (IG) of 28 pediatric residents graduating in 1997, and a control group (CG) of 30 pediatric residents graduating in 1996.
Resuscitation course with didactic lectures and skills practice stations, as well as a minimum of 3 practice mock resuscitations with immediate feedback throughout postgraduate year 3.
Main Outcome Measures
Fund of knowledge, using the Pediatric Advanced Life Support test and short answer test; technical skills, using the Airway and Vascular Access Skills Assessment; experience and confidence, using an anonymous survey; and overall performance, evaluated using a videotaped mock resuscitation test.
The IG scored better on the short answer test (P<.001). A larger number of IG residents were successful in the completion of ancillary airway maneuvers and femoral vascular access (P = .02), as well as endotracheal intubation (P = .004) and intraosseous access (-/P = .002). The IG was more confident in their leadership role (P = .0001) and technical skills (P = .05). Trends toward improved overall performance were noted for the IG mock resuscitations. Residents in the IG were more likely to assess the airway in fewer than 2 minutes (P = .02), recognize the threat to life in fewer than 5 minutes (P = .02), and complete the primary survey in a timely fashion (P = .05). They required fewer prompts (P = .04) and made fewer mistakes (P = .07).
A structured, formal curriculum can improve the necessary fund of knowledge, skills, confidence, and leadership required for resuscitation.
A SUCCESSFUL pediatric resuscitation requires a knowledge of the common etiologies of serious illness, an understanding of how they manifest on physical examination, an array of technical skills using specially sized equipment, and a basic treatment strategy. Although it is important for pediatricians to acquire these skills during their residency training, several studies demonstrate deficiencies in these areas.1- 9 Popular methods of educating residents in the treatment of the critically ill or injured patient rely on organized brief courses, such as Pediatric Advanced Life Support (PALS), individual residents' experiences, random exposure to scheduled didactic lectures, and personal reading.10- 12 Several studies document poor retention of knowledge and inability to perform skills correctly shortly after successful completion of advanced life support courses.5,6 Even in a tertiary hospital, a resident's individual exposure to pediatric resuscitations is limited by the rarity of these events. Furthermore, these events frequently demand more senior physicians to assume leadership, responsibility, and perform the necessary procedures. These occurrences may offer little in the way of resident education, even if effort is made by the senior staff to review the events of the resuscitation with the participants. In a recently published assessment of resident resuscitation skills, White et al8 called for greater attention to resuscitation skills during training. The objective of this study was to evaluate an educational intervention designed to improve pediatric residents' knowledge, skills, confidence, and performance in resuscitation.
The educational intervention was evaluated via a prospective, nonconcurrent, controlled trial. The control group (CG) consisted of 29 pediatric residents who graduated in June 1996. This group received the routine resuscitation experience and education that was in place at the time of their residency training as described below. The intervention group (IG) was composed of 28 residents who graduated in 1997. This group received the study intervention as outlined in Figure 1. The institutional review board at our hospital exempted this project from review as it was an educational intervention study. Verbal consent was obtained from each participating resident.
Our pediatrics residency program is based at a large, urban university-affiliated pediatric hospital. It is a regional and international referral center for pediatric subspecialty care and a level I trauma center. At the time of the study, there were approximately 300 inpatient beds, 15,000 annual patient admissions, and 140,000 outpatient visits, 54,000 of which were emergency department visits. The pediatric residency training program at our hospital fulfills all Residency Review Committee requirements as specified by the Accreditation Council of Graduate Medical Education in 1996.13
During residency training of both the IG and the CG, clinical experience in resuscitation occurred primarily during rotations in pediatric intensive care (2 months), neonatal intensive care (3-4 months), and pediatric emergency medicine (4-6 months). Additionally, all residents completed PALS in both postgraduate year (PGY) 1 and PGY-3. Weekly mock resuscitations were included in the morning conference schedule during the emergency department rotation.
The study intervention consisted of a resuscitation course and participation in a minimum of 3 mock resuscitations during PGY-3. In May 1996, just before starting PGY-3, the IG residents participated in an 8-hour resuscitation course given by pediatric emergency medicine faculty. This course included lectures on important resuscitation topics, practice sessions in advanced life support skills with feedback, and participation in the critique of previously videotaped resuscitations. Throughout the course, important learning objectives were stressed, including the sequential assessment and prioritization of the critically ill patient's airway, breathing, circulation, disability, and exposure (ABCDEs). Important algorithms were also reviewed. From July to April of PGY-3, each of the IG residents led a minimum of 3 mock resuscitations facilitated by one of the investigators. These mock resuscitations were designed to reinforce the recognition and treatment of the life-threatening condition, completion of the ABCDEs, accurate execution of any skills required, and reassessment. Immediately after the mock resuscitations, the group joined in a feedback session. During this session, the facilitator completed a standardized form that emphasized leadership skills, decision making, and the above learning objectives. Each resident's record served as a history of previous experience and performance to provide individualized focus for the facilitator during subsequent mock resuscitations.
Both the IG and CG underwent a baseline assessment of fund of knowledge, experience, and confidence at the beginning of PGY-3. Fund of knowledge was assessed using the standardized PALS examination, which includes 48 multiple-choice questions. Experience and confidence were evaluated with an anonymous survey. Each resident estimated the number of times he or she had been the first responder, leader, and participant in an actual or simulated resuscitation. Each also estimated the number of times that he or she performed or supervised a list of 22 important technical skills, including bag-valve mask ventilation, endotracheal intubation, venipuncture, and intraosseous needle placement. A 4-point ordinal scale (1, very confident; 2, confident; 3, somewhat confident; and 4, not confident) was used to rate each resident's level of confidence in leading resuscitations and in performing technical skills. Residents also reported the number of times they received performance feedback.
At the end of PGY-3, both the IG and CG underwent a standardized assessment of fund of knowledge, technical skills, experience, and confidence. Fund of knowledge was evaluated with the PALS examination in conjunction with 12 short-answer questions. These questions were developed by us and stressed problem solving using case scenarios. For example, residents were given patient situations that required use of PALS resuscitation protocols, calculation of equipment size, selection of the appropriate medication and dose, and description of landmarks for lifesaving procedures. The questions were reviewed for clarity and content by pediatric emergency medicine faculty and former pediatric chief residents.
For technical skills assessment, one of us (J.A.F.) observed each resident as he or she performed 4 advanced life support procedures, including ancillary airway maneuvers, endotracheal intubation, intraosseous placement, and femoral vein access using the Seldinger technique. Ancillary airway maneuvers included use of nasopharyngeal airway and bag-valve mask ventilation. All airway skills were performed on the Laerdal infant mannequin (Laerdal Medical Corp, Wappinger Falls, NY), Seldinger line placement was performed on a plastic arm model, and intraosseous needle placement was performed on chicken legs. If the resident reached a point where he or she could no longer continue with the procedure, the investigator would record and then provide a prompt to allow him or her to continue. At the end of the procedure, the resident received feedback on his/her performance. To standardize performance evaluations, a stepwise, weighted checklist was designed by the investigators. This checklist included discrete, observable steps, such as identification of landmarks, selection of appropriate equipment size, and adequacy of the end point of the procedure.9
Finally, residents in both groups were videotaped leading a mock resuscitation facilitated by 2 of us (F.M.N. and J.M.L.). The 4 scripted scenarios (asystole, status epilepticus, ventricular tachycardia, and supraventricular tachycardia) included our learning objectives, standard sequence of patient events, and prompts to redirect the resident after common errors.
An attempt was made to equalize the difficulty of each scenario. For instance, the asystole scenario does not require the resident to memorize a complicated protocol for drug therapy or perform cardioversion or defibillation. Therefore, when we presented this scenario, the resident was required to recognize the additional complications of an esophageal intubation by emergency medical service personnel and a right main-stem intubation. Residents were videotaped in the resuscitation room in the emergency department, which provided a familiar environment well stocked with resuscitation equipment. They performed procedures on the Laerdal infant mannequin or the Resusci Junior (Laerdal Medical Corp). The HeartSim 2000 Cardiac Rhythm Simulator (Laerdal Medical Corp) was used to generate appropriate rhythms. Immediate feedback was provided after each of the scenarios.
The investigators designed a 2-component grading sheet. The first component (the critical element component) evaluated resident performance of the critical elements common to all resuscitations, including ABCDEs, recognition of the life threat, order and appropriateness of interventions, and reassessment of the patient's condition. The second component was scenario-specific, designed to evaluate resident knowledge of the relevant PALS protocol, appropriate use of medications, drug dose and route, performance of necessary procedures, and reassessment of the patient's condition. Time to important interventions was recorded, as was the need for prompts to redirect the resident. Major mistakes were defined and recorded for each scenario. Finally, an overall grade of resident performance was assigned. Videotapes were graded by one of us (J.M.D.), who had no prior familiarity with the residents in our training program.
Descriptive statistics, including frequencies for categorical variables, and mean, median, range, and SD for continuous variables, were calculated. Forced choice and ordinal scaling of confidence level was simplified into a binary scale as confident vs not confident. For the skill stations, the percentage of residents who successfully completed each procedure was determined. Success was defined a priori as completion of 85% or more of the necessary steps. Finally, data from the videotaped mock resuscitations were analyzed for completion of critical steps, time to completion, need for prompting, number of mistakes, adherence to the PALS protocol, and an overall grade. Differences in each outcome under study were then calculated for intervention vs control groups using χ2 or exact tests for categorical variables, and the t test for continuous variables.
More than 80% of residents in both groups participated in all phases of the evaluation. Only 1 resident in the IG declined to participate in the videotaped mock code. One resident's scenario in the IG group and 2 in the CG were not recorded because of videotape malfunction. A mutually convenient time for the videotape testing was unable to be scheduled for 2 of the IG and 1 of the CG residents.
The baseline assessment revealed no significant differences between the 2 groups. Mean PALS test scores were identical (84% ± 7%). There was no difference in resident self-report regarding experience in leading real resuscitations (IG: median, 0, range, 0-8; CG: median, 0, range, 0-5). The same was true for the number of times residents reported receiving feedback (IG: median, 0, range, 0-2; CG: median, 0, range, 0-2). The majority of residents were confident in performing resuscitation procedures: bag-valve mask ventilation (IG, 100%; CG, 100%); intravenous placement (IG, 70%; CG, 64%); and endotracheal intubation (IG, 96%; CG, 81%). Conversely, few in either group were confident in their leadership skills (IG, 11%; CG, 21%).
As noted in Table 1, both groups performed well on the PALS test. The IG showed significant improvement in both the short answer test and in the execution of all technical skills. There was no difference in the reported number of times residents had performed important technical skills in actual resuscitations. However, residents in the IG had performed more of these procedures during mock resuscitations. The IG residents also led more mock resuscitations and were more confident in the leadership or participant role.
Table 2 demonstrates the proportion of the IG and the CG who successfully completed several tasks during the videotape mock resuscitations. A larger percentage of residents in the IG completed the critical elements of the primary survey. There was also improvement in the IG group's time to airway assessment, time to recognition of the life threat, and in completion of all of the components of the primary survey (ABCDE). Residents in the IG were also less likely to make major mistakes. Significantly, more IG residents required no prompts (IG, 67%; CG, 44%; P = .04) during their resuscitation.
When focusing analysis on the asystole scenario (IG, 6; CG, 11), a larger percentage of IG residents performed the following steps: (1) recognition of the life-threatening condition within 5 minutes (50% vs 9%); (2) reassessment of airway and breathing after the first dose of epinephrine (67% vs 18%); (3) oxygen delivery within 1 minute of patient arrival (100% vs 46%); and (4) termination of the resuscitation (100% vs 45%). No resident in the IG made more than 1 mistake in their scenario as compared with 36% in the CG.
The IG resident's average time to performance of key interventions was also better than the CG in the asystole scenario. For instance, the IG group more quickly connected the patient to oxygen (IG: 0.3 ± 0.3 vs CG: 1.3 ± 0.9 minutes); recognized the rhythm (IG: 5.0 ± 3.3 vs CG: 7.1 ± 3.0 minutes); and reassessed the airway and breathing after epinephrine (IG: 6.4 ± 1.5 vs CG: 8.1 ± 0.1 minutes). A relatively small sample size precluded meaningful statistical analysis.
Our study demonstrates that an educational intervention has the potential to improve overall performance of residents beyond that seen during the routine training experience. Specifically, there was significant improvement in the IG's knowledge, technical skills, confidence, and performance of critical elements in resuscitation. In addition, the IG residents performed many of these critical elements more quickly.
The IG residents tended to perform better on almost all critical elements of the mock resuscitation. Although some of these differences did not achieve statistical significance because of small sample size, we suggest that these would be clinically significant if replicated in actual practice.
Other investigators have attempted to evaluate the effectiveness of a resuscitation educational intervention.14- 20 Most have focused on a single aspect of resuscitation skills, such as technical skills or specific fund of knowledge.14- 16 Often the evaluation of effectiveness of an intervention has been more descriptive in nature, focused on system errors and not individual performance, and lacked a control group.17,18 Only one study mentioned the use of an evaluator who was blind to the learner's group designation.19 Frequently, the intervention was a single teaching episode and the time between intervention to evaluation was minimal.14- 16,19 Our study is unique in that our evaluation was a more comprehensive assessment of the multiple skills required for a resuscitation. Additionally, the use of a control group allowed us to measure the effects of our intervention independent of the usual resident maturation. Finally, the videotapes were evaluated by an investigator blinded to group designation and had never worked with the IG or CG residents and thus had no preconceived ideas about any specific resident's abilities.
Our intervention included some basic educational principles as well as those specific to adult learners.21,22 First, adult learners want education specific to their perceived needs; multiple studies have demonstrated that pediatric resuscitation is a relevant part of training.3,11,13,23,24 Second, adults also learn best in situations in which they can participate, and in which they receive specific feedback.25,26 Feedback is an essential part of learning; the learners need specific recommendations for their improvement to correct inadequate performance.25 Our baseline assessment revealed that feedback is rarely given after these acute events, thus emphasizing another area for improvement of teacher performance. Third, our intervention also incorporated a number of characteristic components of "ideal clinical precepting," in which a knowledgeable and skillful attending physician is paired with an interested resident physician to work through a clinical problem.22,27 The mock resuscitation simulated a form of bedside teaching that is highly valued by many residents. In such a setting, the resident and attending phsyician spend time specifically dealing with a clinical scenario that builds on the abilities of the resident and enhances their knowledge and skill based on what they need to learn.22,27- 29 Fourth, our intervention also incorporated a structured curriculum. Didactic sessions, when provided in conjunction with regular patient care, have been shown to aid in the acquisition of a fund of knowledge that is broader than that which can be acquired from clinical experience alone.30 Finally, the use of simulation technology aids in creating "teachable moments" and allows for practice without putting the patient at risk.31 The rarity of actual pediatric resuscitation makes mock resuscitations a necessity. All of these interventions may have particular importance in our current complex medical environment
Our study has several limitations beyond its relatively small sample size. In an effort to provide variability, 4 mock scenarios were used for video testing. Although the specific medication protocols may differ, the general principles of resuscitation remain the same between scenarios. We therefore considered it reasonable to combine the data from the critical element component of the grading sheet.
To make the mock code a positive learning experience for residents in both groups, the facilitators often prompted a resident to correct decisions. Given that the CG residents required more prompts, this may have enabled the CG to seem more similar in its performance of several tasks.
We designed our evaluation tools, including the short answer test, the technical skills checklist, and the mock resuscitation grading sheet. Although these were based on PALS and Advanced Trauma Life Support curricula, as well as the experience of practicing pediatric emergency medicine specialists, these tools were not rigorously validated. Some of the complexity of decision making required in being a successful leader of a resuscitation may not have been measured by our approach. Additionally, simulated resuscitation performance may not reflect performance in actual resuscitations. However, leading a videotaped resuscitation in front of 2 attending pediatric emergency medicine subspecialists may have approximated some of the stress of a real resuscitation. Finally, further research may be needed to evaluate the retention of skills over time.
During this intervention, the investigators spent approximately more than 200 hours that year providing teaching to small groups of residents, which was less than 1 hour weekly per investigator. Although this may seem like a daunting task, it is important that we continue to explore innovative teaching methods and to measure their effectiveness.
Proficiency in pediatric resuscitation is an important component of pediatric residency training. A structured curriculum can improve a resident's resuscitation fund of knowledge, skills performance, confidence, and overall performance beyond what one would expect during the routine training experience. Continued study and refinement will be essential to improving pediatric resuscitation education. The value of using videotaped simulations may enhance resident education and performance, and, it is hoped, improve patient care.
Accepted for publication June 7, 2000.
This study was supported by a grant from the Emergency Medicine Foundation, Pittsburgh, Pa.
Presented in part as a poster at the Society for Academic Emergency Medicine Annual Meeting, Chicago, Ill, May 18, 1998, and at the Ambulatory Pediatric Association Annual Meeting, New Orleans, La, May 3, 1998.
We thank Stephen Ludwig, MD, and Andrew Costarino, MD, for their invaluable support and advice, the attending physicians and fellows in the Division of Emergency Medicine, Mike Sicilia, RN, RRT, EMT, for providing ongoing feedback and assistance in teaching, Patricia Parkinson, for her manuscript preparation, research assistants Katelyn Dyer and Titi Alabi, and the residents who participated in this study and provided essential feedback.
Purpose: This section is intended to share information concerning educational efforts in the broad field of pediatrics. We welcome studies on the following topics: undergraduate and graduate education in medicine and allied health occupations; continuing education of health professionals; education of patients and families; and health education for the general public, the community, and organizations that contribute to the promotion and improvement of the health of children and adolescents.
Corresponding author: Frances M. Nadel, MD, The Children's Hospital of Philadelphia, Division of Emergency Medicine, 34th Street and Civic Center Boulevard, Philadelphia, PA 19104 (e-mail: firstname.lastname@example.org).