Copyright 2007 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2007
To evaluate the impact of simulation-based education on patient safety during pediatric procedural sedation.
A prospective, observational, single-blind, controlled study of pediatric procedural sedation outside the operating room.
Two university teaching hospitals in Israel.
Nonanesthesiologists, with or without training in simulation-based education on patient safety, who routinely perform procedural sedation outside the operating room. These comprise full-time pediatricians practicing emergency medicine and a cohort of pediatric gastroenterologists.
The study investigators used the internally developed, 9-criteria Sedation Safety Tool to observe and evaluate nonanesthesiologists who were trained in sedation safety and compared their performance with that of colleagues who did not receive similar training.
For each of the 9 criteria on the evaluation form, odds ratios and 95% confidence intervals were calculated to compare the actions of the individuals in the 2 study groups.
Thirty-two clinicians were evaluated. Half of the physicians were graduates of the simulation-based sedation safety course. Significant differences in performance pertaining to patient safety were found between those physicians who did and those who did not complete simulation-based training.
Pediatric procedural sedations conducted by simulator-trained nonanesthesiologists were safer. The simulation-based sedation safety course enhanced physician performance during pediatric procedural sedation.
The management of acute pediatric pain and anxiety outside of the operating room (OR) has developed significantly during the past few years. Procedural sedation should be practiced effectively to control pain and anxiety and to provide an appropriate degree of decreased awareness of one's environment.1,2 Procedural sedation is practiced by multiple subspecialists in a variety of settings. As a result, patient safety has become a concern because of the different skill sets of these practitioners and the use of various protocols incorporating multiple different sedative agents.3- 6
In 2003, the Israeli Ministry of Health7 published formal guidelines for pediatric procedural sedation by nonanesthesiologists. Per these guidelines, nonanesthesiologists performing procedural sedation outside of the OR were required to undergo specific training in pediatric sedation, including a component pertaining to patient safety. More than 400 nonanesthesiologists have been trained and qualified in a simulation-based pediatric sedation safety course at the Israel Center for Medical Simulation.8
To our knowledge, the contribution of physician's training to patient safety in this setting has never been examined. This study examined the impact of training on patient safety during pediatric sedation performed outside the OR. Our hypothesis was that trained nonanesthesiologists would perform more safely than those who did not complete the simulator-based course.
The study was conducted in Israel at 2 university-affiliated teaching hospitals from March 3 through April 29, 2005. Pediatric sedations were performed in the emergency department and in the gastroenterology clinic. Institutional review boards at both hospitals waived the need for informed consent.
Pediatricians practicing emergency medicine (PEM) or gastroenterology (PGE) full-time were included in the study. Observed procedures were exclusively minor, such as laceration repair, lumbar puncture, and elective endoscopies. Unstable patients and those undergoing major procedures, such as chest tube insertion, were excluded.
Any PEM or PGE meeting the inclusion criteria and planning to perform sedation was asked to participate. Potential study subjects received explanations about the study but were blinded to the final objective concerning patient safety. They were informed that enrollment was voluntary and that individual data would be kept confidential because only aggregate information would be made public.
To our knowledge, accepted tools or standards for evaluating patient safety during pediatric sedation do not exist and, consequently, an evaluation instrument was developed for this study. The Sedation Safety Tool (SST) includes 9 criteria deemed relevant for patient safety. The SST clusters measurements over 3 distinct periods: before sedation (history of operations or procedural sedation, time of the last meal, history of known adverse effect to medication, and measurement of vital signs before drug administration), during sedation (maintaining eye contact with the patient throughout the procedure, titration of medication, and monitoring), and recovery time (monitoring and national guideline–recommended discharge criteria). The tool evaluates physician behaviors that are conducive to safe patient outcomes. To overcome potential investigator bias, the SST was designed dichotomously. For each criterion, the study investigator marked “done” or “not done.” Three experts in pediatric sedation (including I.S. and I.K.) independently reviewed the SST and unanimously endorsed the face validity of the instrument. Before starting the study, the tool's interrater reliability was tested by 2 of us (Y.H. and L.M.). In the setting of the OR, the 2 testers evaluated 11 sedations in a blinded fashion, and overall agreement was achieved for 87 of 99 safety variables (Cohen κ, 0.73).
In this prospective observational study, the evaluation process lasted from the moment the patient entered the room until discharge. Study subjects were not allowed to see their evaluation. Once the evaluation was completed, it could not be modified. Completed evaluation forms were immediately faxed to the principal investigator (I.S.). Information about the study subject's previous training in pediatric sedation was ascertained retrospectively.
Two study investigators, one from each site (Y.H. and L.M.), evaluated all completed SST forms on participants from the other site. The investigators were blinded to information on the participant's previous training in pediatric sedation. Participants who provided this information to the investigators were excluded from the study.
For each of the 9 criteria in the evaluation form, adjusted odds ratios and 95% confidence intervals were calculated using the logistic regression model. The analyses were performed using SAS statistical software (SAS Institute Inc, Cary, NC).
Forty-six nonanesthesiologists who potentially met the inclusion criteria for the study were identified and solicited. Thirty-two completed evaluations are included in the study: 20 from PGEs and 12 from PEMs. The exclusion of 14 others was based on the following: 4 declined to participate, 8 volunteered information regarding previous training, and 2 were replaced by a colleague during the actual sedation and, consequently, were excluded.
All sedations were successfully completed. Procedures included 14 upper gastrointestinal endoscopies, 6 colonoscopies, 8 laceration repairs, 2 lumbar punctures, and 2 fracture reductions. Sixteen participating physicians had previously completed training in pediatric sedation safety, and 16 had not. Experience with sedation was similar in both groups (2-4 years). A regimen of intravenous midazolam and ketamine was used in 27 sedations (by 12 PEMs and 15 PGEs); a midazolam and fentanyl regimen was used in 5 sedations (by 5 PGEs).
For each of the 9 safety criteria measured via the SST, those physicians who received previous training in pediatric procedural sedation performed better than those who did not receive such training (Table).
Procedural sedation outside the OR is a growing practice. Guidelines for pediatric procedural sedation and analgesia have been published by various organizations and specialty societies. The most widely disseminated were published by the American Academy of Pediatrics,3,4 the American Society of Anesthesiologists,5 and the American College of Emergency Physicians.6
Recent studies9,10 suggest that pediatric procedural sedations performed outside of the OR are unlikely to yield serious adverse outcomes, but emphasize that further research on the safety of this practice should continue to be explored. An important resource on pediatric sedations is a large Web-based database created by the Pediatric Sedation Research Consortium. Findings from the Pediatric Sedation Research Consortium suggest that lethal critical adverse events are rare, with only 1 cardiac arrest reported in more than 30 000 cases. However, potentially critical events occur relatively commonly, because 1 in 44 procedures was associated with stridor, laryngospasm, wheezing, or apnea.10
An evidence-based review of the literature failed to identify factors associated with enhanced patient safety during pediatric procedural sedation. Anesthesiologists are well trained in the practice of procedural sedation. The reality is that because of increased demand for sedation services, procedural sedation is performed by nonanesthesiologists without uniform training. Considering the Pediatric Sedation Research Consortium data that show that critical events do commonly occur, the nonuniformity of the training of nonanesthesiologists performing these sedations may be a concern.11 Our study suggests that the adequacy of the sedating physician's training is an important component in ensuring a safe outcome.
We demonstrated that nonanesthesiologists with specific training in pediatric procedural sedation performed tasks associated with safer outcomes in a superior fashion compared with their untrained counterparts. The trained study physicians participated in a course that included a presimulation, Web-based, cognitive distant learning component, followed by a workshop with lectures culminating in comprehensive hands-on sessions using mannequin and human simulator models. By use of the simulator, course participants practice necessary techniques to assist in avoiding sedation-related complications and reviewing patient monitoring skills and rescue measures necessary to manage adverse reactions. They are then provided with immediate feedback using a real-time audio-video debriefing system.12,13 On completion of the course, participants are required to pass a written examination and a safety skills session that includes an evaluation of the participant's ability to assess and manage airway complications.
We believe that this patient safety training is a significant factor responsible for the superior performance of the trained physicians. The necessity of this type of training is consistent with the recommendations of other colleagues.3,14
Our study has limitations. The SST as an evaluation instrument was consistent among physicians and has face validity; however, “construct” validity has not been proved (ideally, the SST should have been validated in a simulator environment before the study; however, the behaviors listed in the SST cannot be measured in this environment). The convenience sample size was small, the study was not randomized, and we had a high rate of enrollment failure. The 2 groups of physicians were heterogeneous in their scope of practice and were not controlled for other variables, such as the type of equipment used during sedation or sedation protocols. In addition, patient outcomes were positive in all cases, despite the differences in training background. Pediatric emergency medicine is not yet a recognized specific subspecialty in Israel and, consequently, training programs do not exist. The cohort of PEM who participated in the study included pediatricians working in the emergency department who have 2 to 4 years of experience in practicing procedural sedations. These individuals did not have specific training in procedural sedation or in managing airway complications as part of their general training. The participating PGE completed fellowship programs in pediatric gastroenterology and had similar years of experience in practicing procedural sedation and in managing airway complications. Consequently, we do not believe that the differences in training background between the PEMs and the PGEs are a confounding variable in this study.
Despite its limitations, our study shows that specific physician-related factors may influence patient safety in pediatric procedural sedation. To our knowledge, our study is the first to examine the influence of simulation-based training on sedation safety.
The results of this study indicate that nonanesthesiologists with pediatric procedural sedation simulator training performed in a safer manner than colleagues who did not have similar training. Consequently, simulation-based training in sedation safety is recommended for nonanesthesiologists practicing pediatric procedural sedation.
Correspondence: Itai Shavit, MD, PO Box 369, Nahariyya 22103, Israel (firstname.lastname@example.org).
Accepted for Publication: March 1, 2007.
Author Contributions: Dr Shavit has full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Shavit, Keidan, and Ziv. Acquisition of data: Shavit, Keidan, Hoffmann, Mishuk, and Ziv. Analysis and interpretation of data: Shavit, Keidan, Hoffmann, Mishuk, Rubin, and Steiner. Drafting of the manuscript: Shavit, Mishuk, and Steiner. Critical revision of the manuscript for important intellectual content: Mishuk, Rubin, Ziv, and Steiner. Statistical analysis: Rubin. Study supervision: Ziv.
Financial Disclosure: None reported.
Previous Presentation: This study was presented at the Annual Meeting of the Israeli Association for Clinical Pediatrics; April 12, 2006; Tel Aviv, Israel.
Additional Contributions: We thank Edna Keidar, director of administration at the Israel Center for Medical Simulation, for her ongoing help on this project.
Shavit I, Keidan I, Hoffmann Y, Mishuk L, Rubin O, Ziv A, Steiner IP. Enhancing Patient Safety During Pediatric SedationThe Impact of Simulation-Based Training of Nonanesthesiologists. Arch Pediatr Adolesc Med. 2007;161(8):740-743. doi:10.1001/archpedi.161.8.740