[Skip to Content]
[Skip to Content Landing]
Figure 1.  Stepped-Wedge Cluster Nonrandomized Clinical Trial Design
Stepped-Wedge Cluster Nonrandomized Clinical Trial Design

Implementation of the individualized preoperative and postoperative Surgical Patient Safety System (SURPASS) checklists in 3 surgical clusters in a tertiary hospital in Western Norway, November 2012 to March 2015 (29 months). A indicates gynecology; B, orthopedics; and C, neurosurgery.

aIndicates pilot SURPASS intervention.

bIndicates 3-week pilot during June and July 2012.

Figure 2.  Surgical Checklist Flow
Surgical Checklist Flow

1 indicates surgical pathway; 2, checkpoints for clinicians; and 3, Surgical Patient Safety System (SURPASS) or World Health Organization surgical safety checklist (WHO SSC) applications. PACU indicates postanesthesia care unit.

Table 1.  Characteristics of 9009 Surgical Procedures in a Stepped-Wedge Cluster Nonrandomized Clinical Trial
Characteristics of 9009 Surgical Procedures in a Stepped-Wedge Cluster Nonrandomized Clinical Trial
Table 2.  Characteristics of Outcomes Before and After Intervention With SURPASS Checklists Added to WHO SSC
Characteristics of Outcomes Before and After Intervention With SURPASS Checklists Added to WHO SSC
Table 3.  Results From Logistic Regression of the Effects of Preoperative and Postoperative SURPASS Checklists Plus WHO SSC on 1 or More Complications in 9002 Surgical Proceduresa
Results From Logistic Regression of the Effects of Preoperative and Postoperative SURPASS Checklists Plus WHO SSC on 1 or More Complications in 9002 Surgical Proceduresa
1.
Abbott  TEF, Ahmad  T, Phull  MK,  et al; International Surgical Outcomes Study (ISOS) group.  The surgical safety checklist and patient outcomes after surgery: a prospective observational cohort study, systematic review and meta-analysis.   Br J Anaesth. 2018;120(1):146-155. doi:10.1016/j.bja.2017.08.002 PubMedGoogle ScholarCrossref
2.
Bergs  J, Hellings  J, Cleemput  I,  et al.  Systematic review and meta-analysis of the effect of the World Health Organization surgical safety checklist on postoperative complications.   Br J Surg. 2014;101(3):150-158. doi:10.1002/bjs.9381 PubMedGoogle ScholarCrossref
3.
Borchard  A, Schwappach  DL, Barbir  A, Bezzola  P.  A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery.   Ann Surg. 2012;256(6):925-933. doi:10.1097/SLA.0b013e3182682f27 PubMedGoogle ScholarCrossref
4.
Boyd  J, Wu  G, Stelfox  H.  The impact of checklists on inpatient safety outcomes: a systematic review of randomized controlled trials.   J Hosp Med. 2017;12(8):675-682. doi:10.12788/jhm.2788 PubMedGoogle ScholarCrossref
5.
Lau  CSM, Chamberlain  RS.  The World Health Organization surgical safety checklist improves post-operative outcomes: a meta-analysis and systematic review.   Surg Sci. 2016;7(4):206-217. doi:10.4236/ss.2016.74029 Google ScholarCrossref
6.
Thomassen  Ø, Storesund  A, Søfteland  E, Brattebø  G.  The effects of safety checklists in medicine: a systematic review.   Acta Anaesthesiol Scand. 2014;58(1):5-18. doi:10.1111/aas.12207 PubMedGoogle ScholarCrossref
7.
Urbach  DR, Govindarajan  A, Saskin  R, Wilton  AS, Baxter  NN.  Introduction of surgical safety checklists in Ontario, Canada.   N Engl J Med. 2014;370(11):1029-1038. doi:10.1056/NEJMsa1308261 PubMedGoogle ScholarCrossref
8.
Reames  BN, Krell  RW, Campbell  DA  Jr, Dimick  JB.  A checklist-based intervention to improve surgical outcomes in Michigan: evaluation of the Keystone Surgery program.   JAMA Surg. 2015;150(3):208-215. doi:10.1001/jamasurg.2014.2873 PubMedGoogle ScholarCrossref
9.
Urbach  DR.  The challenge of quality improvement in surgical care.   JAMA Surg. 2015;150(3):215. doi:10.1001/jamasurg.2014.2908 PubMedGoogle ScholarCrossref
10.
Leape  LL.  The checklist conundrum.   N Engl J Med. 2014;370(11):1063-1064. doi:10.1056/NEJMe1315851 PubMedGoogle ScholarCrossref
11.
Russ  S, Rout  S, Sevdalis  N, Moorthy  K, Darzi  A, Vincent  C.  Do safety checklists improve teamwork and communication in the operating room? a systematic review.   Ann Surg. 2013;258(6):856-871. doi:10.1097/SLA.0000000000000206 PubMedGoogle ScholarCrossref
12.
Greenberg  CC, Regenbogen  SE, Studdert  DM,  et al.  Patterns of communication breakdowns resulting in injury to surgical patients.   J Am Coll Surg. 2007;204(4):533-540. doi:10.1016/j.jamcollsurg.2007.01.010 PubMedGoogle ScholarCrossref
13.
Griffen  FD, Stephens  LS, Alexander  JB,  et al.  The American College of Surgeons’ closed claims study: new insights for improving care.   J Am Coll Surg. 2007;204(4):561-569. doi:10.1016/j.jamcollsurg.2007.01.013 PubMedGoogle ScholarCrossref
14.
de Vries  EN, Hollmann  MW, Smorenburg  SM, Gouma  DJ, Boermeester  MA.  Development and validation of the Surgical Patient Safety System (SURPASS) checklist.   Qual Saf Health Care. 2009;18(2):121-126. doi:10.1136/qshc.2008.027524 PubMedGoogle ScholarCrossref
15.
de Vries  EN, Prins  HA, Crolla  RM,  et al; SURPASS Collaborative Group.  Effect of a comprehensive surgical safety system on patient outcomes.   N Engl J Med. 2010;363(20):1928-1937. doi:10.1056/NEJMsa0911535 PubMedGoogle ScholarCrossref
16.
Mehta  N, Amaranathan  A, Jayapal  L, Kundra  P, Nelamangala Ramakrishnaiah  VP.  Effect of comprehensive surgical safety system on patients’ outcome: a prospective clinical study.   Cureus. 2018;10(5):e2601. doi:10.7759/cureus.2601 PubMedGoogle Scholar
17.
Brown  C, Hofer  T, Johal  A,  et al.  An epistemology of patient safety research: a framework for study design and interpretation—part 2, study design.   Qual Saf Health Care. 2008;17(3):163-169. doi:10.1136/qshc.2007.023648 PubMedGoogle ScholarCrossref
18.
Hemming  K, Taljaard  M, McKenzie  JE,  et al.  Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration.   BMJ. 2018;363:k1614. doi:10.1136/bmj.k1614 PubMedGoogle ScholarCrossref
19.
Mdege  ND, Man  MS, Taylor Nee Brown  CA, Torgerson  DJ.  Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation.   J Clin Epidemiol. 2011;64(9):936-948. doi:10.1016/j.jclinepi.2010.12.003 PubMedGoogle ScholarCrossref
20.
Haynes  AB, Weiser  TG, Berry  WR,  et al; Safe Surgery Saves Lives Study Group.  A surgical safety checklist to reduce morbidity and mortality in a global population.   N Engl J Med. 2009;360(5):491-499. doi:10.1056/NEJMsa0810119 PubMedGoogle ScholarCrossref
21.
World Alliance for Patient Safely; World Health Organization. Implementation Manual: Surgical Safety Checklist. Published May 2008. Accessed February 16, 2020. https://www.who.int/patientsafety/safesurgery/tools_resources/SSSL_Manual_finalJun08.pdf
22.
Storesund  A, Haugen  AS, Wæhle  HV,  et al.  Validation of a Norwegian version of Surgical Patient Safety System (SURPASS) in combination with the World Health Organizations’ surgical safety checklist (WHO SSC).   BMJ Open Qual. 2019;8(1):e000488. doi:10.1136/bmjoq-2018-000488 PubMedGoogle Scholar
23.
Haugen  AS, Søfteland  E, Almeland  SK,  et al.  Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial.   Ann Surg. 2015;261(5):821-828. doi:10.1097/SLA.0000000000000716 PubMedGoogle ScholarCrossref
24.
Powell  BJ, Waltz  TJ, Chinman  MJ,  et al.  A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project.   Implement Sci. 2015;10(21):21. doi:10.1186/s13012-015-0209-1 PubMedGoogle ScholarCrossref
25.
Conley  DM, Singer  SJ, Edmondson  L, Berry  WR, Gawande  AA.  Effective surgical safety checklist implementation.   J Am Coll Surg. 2011;212(5):873-879. doi:10.1016/j.jamcollsurg.2011.01.052 PubMedGoogle ScholarCrossref
26.
Haynes  AB, Berry  WR, Gawande  AA.  What do we know about the safe surgery checklist now?   Ann Surg. 2015;261(5):829-830. doi:10.1097/SLA.0000000000001144 PubMedGoogle ScholarCrossref
27.
Storesund  A, Haugen  AS, Hjortås  M,  et al.  Accuracy of surgical complication rate estimation using ICD-10 codes.   Br J Surg. 2019;106(3):236-244. doi:10.1002/bjs.10985 PubMedGoogle ScholarCrossref
28.
Hemming  K, Haines  TP, Chilton  PJ, Girling  AJ, Lilford  RJ.  The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting.   BMJ. 2015;350:h391. doi:10.1136/bmj.h391 PubMedGoogle ScholarCrossref
29.
de Vries  EN, Eikens-Jansen  MP, Hamersma  AM, Smorenburg  SM, Gouma  DJ, Boermeester  MA.  Prevention of surgical malpractice claims by use of a surgical safety checklist.   Ann Surg. 2011;253(3):624-628. doi:10.1097/SLA.0b013e3182068880 PubMedGoogle ScholarCrossref
30.
van Boxtel  AGM, van Veghel  D, Soliman Hamad  MA, Schulz  DN, Stepaniak  PS, van Straten  AHM.  Use of an intraoperative checklist to decrease the incidence of re-exploration for postoperative bleeding after cardiac surgery.   Interact Cardiovasc Thorac Surg. 2017;25(4):555-558. doi:10.1093/icvts/ivx130 PubMedGoogle ScholarCrossref
31.
Lepänluoma  M, Rahi  M, Takala  R, Löyttyniemi  E, Ikonen  TS.  Analysis of neurosurgical reoperations: use of a surgical checklist and reduction of infection-related and preventable complication-related reoperations.   J Neurosurg. 2015;123(1):145-152. doi:10.3171/2014.12.JNS141077 PubMedGoogle ScholarCrossref
32.
Kim  SW, Maturo  S, Dwyer  D,  et al.  Interdisciplinary development and implementation of communication checklist for postoperative management of pediatric airway patients.   Otolaryngol Head Neck Surg. 2012;146(1):129-134. doi:10.1177/0194599811421745 PubMedGoogle ScholarCrossref
33.
Graan  SM, Botti  M, Wood  B, Redley  B.  Nursing handover from ICU to cardiac ward: standardised tools to reduce safety risks.   Aust Crit Care. 2016;29(3):165-171. doi:10.1016/j.aucc.2015.09.002 PubMedGoogle ScholarCrossref
34.
Johnston  MJ, Arora  S, King  D,  et al.  A systematic review to identify the factors that affect failure to rescue and escalation of care in surgery.   Surgery. 2015;157(4):752-763. doi:10.1016/j.surg.2014.10.017 PubMedGoogle ScholarCrossref
35.
Nagpal  K, Vats  A, Lamb  B,  et al.  Information transfer and communication in surgery: a systematic review.   Ann Surg. 2010;252(2):225-239. doi:10.1097/SLA.0b013e3181e495c2 PubMedGoogle ScholarCrossref
36.
Drake  K, McBride  M, Bergin  J, Vandeweerd  H, Higgins  A.  Ensuring safe discharge with a standardized checklist and discharge pause.   Nursing. 2017;47(8):65-68. doi:10.1097/01.NURSE.0000521042.81195.86 PubMedGoogle ScholarCrossref
37.
Rodella  S, Mall  S, Marino  M,  et al.  Effects on clinical outcomes of a 5-year surgical safety checklist implementation experience: a large-scale population-based difference-in-differences study.   Health Serv Insights. 2018;11:1178632918785127. doi:10.1177/1178632918785127 PubMedGoogle Scholar
38.
The Office of the Auditor General of Norway. The Office of the Auditor General’s investigation of medical coding practice within the health enterprises. Document 3:5 (2016-2017). Accessed February 16, 2020. https://www.riksrevisjonen.no/globalassets/reports/en-2016-2017/codingpracticehealthenterprises.pdf
39.
de Jager  E, Gunnarsson  R, Ho  YH.  Implementation of the World Health Organization surgical safety checklist correlates with reduced surgical mortality and length of hospital admission in a high-income country.   World J Surg. 2019;43(1):117-124. doi:10.1007/s00268-018-4703-x PubMedGoogle ScholarCrossref
40.
Ministry of Health and Care Services. Report No. 47 (2008–2009) to the Storting. The Coordination Reform—proper treatment at the right place and right time. Full version in Norwegian. Summary in English: Oslo, Norway. Published 2008. Accessed September 30, 2019. https://www.regjeringen.no/en/dokumenter/report.no.-47-to-the-storting-2008-2009/id567201/?ch=1&q=#?ch=1&q=&_t_dtq=true&_suid=146037692843204369084028942121
41.
Russ  SJ, Sevdalis  N, Moorthy  K,  et al.  A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: lessons from the “Surgical Checklist Implementation Project”.   Ann Surg. 2015;261(1):81-91. doi:10.1097/SLA.0000000000000793 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,038
    Citations 0
    Original Investigation
    May 13, 2020

    Clinical Efficacy of Combined Surgical Patient Safety System and the World Health Organization’s Checklists in Surgery: A Nonrandomized Clinical Trial

    Author Affiliations
    • 1Department of Anesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway
    • 2Department of Clinical Medicine, Faculty of Medicine, University of Bergen, Bergen, Norway
    • 3Centre for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway
    • 4Centre for Clinical Research, Haukeland University Hospital, Bergen, Norway
    • 5Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway
    • 6Department of Surgery, Academic Medical Center, Amsterdam, the Netherlands
    • 7Center for Implementation Science, Health Service and Population Research Department, King’s College, London, United Kingdom
    • 8Department of Neurosurgery, Haukeland University Hospital, Bergen, Norway
    • 9Department of Orthopedic Surgery, Haukeland University Hospital, Bergen, Norway
    • 10Department of Gynecology and Obstetrics, Haukeland University Hospital, Bergen, Norway
    • 11Department of Anesthesiology, Haugesund Hospital, Health Trust Fonna, Haugesund, Norway
    • 12Section of Cardiothoracic Surgery, Department of Heart Disease, Haukeland University Hospital, Bergen, Norway
    • 13Department of Clinical Science, Faculty of Medicine, University of Bergen, Bergen, Norway
    • 14Department of Surgery, Førde Central Hospital, Førde, Norway
    JAMA Surg. Published online May 13, 2020. doi:10.1001/jamasurg.2020.0989
    Key Points

    Question  Does patient safety improve when adding the preoperative and postoperative Surgical Patient Safety System checklists to the World Health Organization’s established surgical safety checklist?

    Findings  In this stepped-wedge cluster nonrandomized clinical trial with parallel controls that included 9009 surgical procedures, reductions in complications and emergency reoperations occurred when the preoperative Surgical Patient Safety System was added to the surgical safety checklist. The postoperative Surgical Patient Safety System reduced readmissions, whereas overall increased complications were found in the 9678 parallel controls.

    Meaning  These findings suggest that joint use of the preoperative and postoperative Surgical Patient Safety System with the intraoperative surgical safety checklist is beneficial for patients.

    Abstract

    Importance  Checklists have been shown to improve patient outcomes in surgery. The intraoperatively used World Health Organization surgical safety checklist (WHO SSC) is now mandatory in many countries. The only evidenced checklist to address preoperative and postoperative care is the Surgical Patient Safety System (SURPASS), which has been found to be effective in improving patient outcomes. To date, the WHO SSC and SURPASS have not been studied jointly within the perioperative pathway.

    Objective  To investigate the association of combined use of the preoperative and postoperative SURPASS and the WHO SSC in perioperative care with morbidity, mortality, and length of hospital stay.

    Design, Setting, and Participants  In a stepped-wedge cluster nonrandomized clinical trial, the preoperative and postoperative SURPASS checklists were implemented in 3 surgical departments (neurosurgery, orthopedics, and gynecology) in a Norwegian tertiary hospital, serving as their own controls. Three surgical units offered additional parallel controls. Data were collected from November 1, 2012, to March 31, 2015, including surgical procedures without any restrictions to patient age. Data were analyzed from September 25, 2018, to March 29, 2019.

    Interventions  Individualized preoperative and postoperative SURPASS checklists were added to the intraoperative WHO SSC.

    Main Outcomes and Measures  Primary outcomes were in-hospital complications, emergency reoperations, unplanned 30-day readmissions, and 30-day mortality. The secondary outcome was length of hospital stay (LOS).

    Results  In total, 9009 procedures (5601 women [62.2%]; mean [SD] patient age, 51.7 [22.2] years) were included, with 5117 intervention procedures (mean [SD] patient age, 51.8 [22.4] years; 2913 women [56.9%]) compared with 3892 controls (mean [SD] patient age, 51.5 [21.8] years; 2688 women [69.1%]). Parallel control units included 9678 procedures (mean [SD] patient age, 57.4 [22.2] years; 4124 women [42.6%]). In addition to the WHO SSC, adjusted analyses showed that adherence to the preoperative SURPASS checklists was associated with reduced complications (odds ratio [OR], 0.70; 95% CI, 0.50-0.98; P = .04) and reoperations (OR, 0.42; 95% CI, 0.23-0.76; P = .004). Adherence to the postoperative SURPASS checklists was associated with decreased readmissions (OR, 0.32; 95% CI, 0.16-0.64; P = .001). No changes were observed in mortality or LOS. In parallel control units, complications increased (OR, 1.09; 95% CI, 1.01-1.17; P = .04), whereas reoperations, readmissions, and mortality remained unchanged.

    Conclusions and Relevance  In this nonrandomized clinical trial, adding preoperative and postoperative SURPASS to the WHO SSC was associated with a reduction in the rate of complications, reoperations, and readmissions.

    Trial Registration  ClinicalTrials.gov Identifier: NCT01872195

    Introduction

    The World Health Organization surgical safety checklist (WHO SSC), now used globally, has been found to reduce complications and mortality,1-6 although negative findings have also been published.7,8 Questions have been raised regarding whether negative results are due to a lack of emphasis on the implementation and local tailoring.8-10 The WHO SSC is used within the operating room, aiming to improve teamwork, with shared awareness of the safety aspects of surgery.11

    However, surgical complications often originate before and after operating room activities.12-14 The comprehensive Surgical Patient Safety System (SURPASS) developed in the Netherlands is the only surgical safety checklist to date to include specific preoperative and postoperative checklists for individual clinicians in addition to team checks in the operating room. Like the WHO SSC, SURPASS has also been found to reduce complications and mortality.15 The effect of implementing SURPASS has been replicated only in a smaller study from India,16 which found that use of SURPASS alone reduced the rate of postoperative complications in both elective and emergency operations.

    To date, whether surgical safety can improve further when combining the intraoperative WHO SSC with the preoperative and postoperative SURPASS remains unknown. This study aimed to evaluate the associations of adding the preoperative and postoperative SURPASS to the intraoperative WHO SSC with surgical complications, all-cause 30-day mortality, and subsequent length of hospital stay (LOS).

    Methods
    Study Design, Setting, and Oversight

    The trial protocol is given in Supplement 1. A prospective, stepped-wedge cluster nonrandomized clinical trial design17 was used. The study implemented the preoperative and postoperative SURPASS checklists in 3 surgical departments in a tertiary hospital in western Norway (Figure 1) in addition to the WHO SSC. The study was approved by the regional ethical research committee, the data privacy units at the Health Trust Førde, and Health Trust Fonna of Norway. After approval, the patients in the intervention departments received written information on the study and their option to refrain from data sharing. The study was exempt from written informed consent. This study followed the extension of the 2010 Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.18

    At the time of the study (and to date), the WHO SSC is mandatory in Norwegian operating rooms. The study design allowed introduction of the SURPASS to each department sequentially19 as opposed to a classic before-after design, wherein the switch from before/control to after/intervention is introduced for all the trial departments at the same time. The original SURPASS15 and WHO SSC20 intervention trials are classic before-after studies. Our current design allows adjustment for time trends and is advantageous in health care settings with limited resources, involving continuous advancements and change.19

    Following advice from the WHO checklist implementation guideline, the trial departments were invited to participate based on their management commitment, frontline positive engagement, and high adherence to the WHO SSC.21 The SURPASS intervention followed a stepwise introduction in 1 department at a time. The departments each contributed patient data before and after the study intervention and served as their own controls, thus minimizing selection bias. Contamination between study departments and the parallel control departments—caused by information bias due to personnel working in several disciplines, sections, or departments—was avoided: The operating rooms and surgical teams were geographically separate with their own organizational units and specialized personnel (neurosurgery, orthopedic surgery, and gynecology and the parallel control departments of thoracic surgery, general surgery, vascular surgery, gastroenterology, urology, orthopedics, and ear, nose, and throat surgery). Because single surgical procedures were subjects of investigation, it was unlikely that any participant could have been in both control and intervention groups, hence within-department contamination was avoided. Three separate surgical units in different hospitals (a tertiary hospital serving a population of 1.1 million, a community hospital serving a population of 110 000, and a community hospital serving a population of 180 000) constituted additional parallel controls, with the WHO SSC as standard care but without SURPASS.

    Intervention

    The original SURPASS system was developed to include known risk factors described in the literature, validated against actual registered adverse events.14 The preoperative and postoperative SURPASS checklists are individualized to be performed by key clinicians in the surgical pathway. Each checklist is to be used as a last point of check before transfer to the next segment of the pathway, ensuring good planning and compliance with existing perioperative care protocols at all transfer points. Figure 2 displays the combined SURPASS and WHO SSC checklists across the surgical pathway as implemented in this study.

    Before the trial, validation of the SURPASS checklist content into a Norwegian context was performed in a neurosurgical setting.22 Before checklist implementation in a new department, tailoring for specific conditions in the different Norwegian departments was performed in accordance with advice in the WHO implementation manual.21 Implementation of SURPASS was informed by our team’s extensive experience with implementing the WHO SSC in Norway23 and also by recently compiled implementation strategies for health care compendium developed by implementation scientists.24 In brief, the implementation strategy included educational sessions with frontline clinicians emphasizing why the checklists should be used, their evidence, and the practicalities of how to apply them.25,26 Individual coaching was offered by the research team. Moreover, an information campaign in the trial departments was performed through distribution of printed posters and emails to staff. Service managers and key clinicians in the different departments were designated champions of the SURPASS intervention. Last, audit and feedback on SURPASS implementation fidelity (ie, quality of application) was provided through regular compliance reports sent directly to all service managers by the research team.

    Outcome Measures

    Primary outcomes were in-hospital morbidity (complications, emergency reoperations, and 30-day readmissions) and all-cause 30-day postoperative mortality. The secondary outcome measure was LOS.

    Inclusion and Exclusion Criteria

    The study included in-hospital patients of all ages undergoing an elective or an emergency surgical procedure. Excluded were radiological interventions, donor surgery, extracorporeal membrane oxygenation procedures, outpatients, and patients who declined to consent to the study.

    Data Collection and Handling

    Complications were investigated according to the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) as routinely recorded by physicians. The method for validating the reported in-hospital complications has been described previously.27 For the present study, all 155 ICD-10 complication codes included in the analyses were verified against each patient’s medical records by the research team.

    Data on mortality, LOS, patient characteristics, and surgical procedures were collected from the hospitals’ electronic administrative systems and verified against each patient’s medical record. Checklist data were combined with outcome data after this verification procedure. Patient characteristics included age, sex, American Society of Anesthesiologists (ASA) physical health classification, urgency of surgery, type of surgery, type of anesthesia, and time (month and year) of operation. Checklist adherence (ie, fidelity of application) was recorded per SURPASS checklist item and as the proportion of individual checklists with all items checked. Thus, for the preoperative SURPASS, the proportions were 0, 0.20 for 1 checklist, 0.40 for 2, 0.60 for 3, 0.80 for 4, and 1.00 for 5 (because these have 5 parts in all). For the postoperative SURPASS and WHO SSC, the proportions were 0, 0.33 for 1 checklist, 0.66 for 2, and 1.00 for 3 (because these have 3 parts in all). All data were collected as part of daily routine patient documentation, with staff blinded to outcome measures. All data handlers were blinded to checklists used in the care of individual patients.

    Statistical Analysis

    Data were analyzed from September 25, 2018, to March 29, 2019. Characteristics of the preoperative and postintervention procedures and patient data were compared using the Pearson exact test with Bonferroni corrections for categorical variables and Gosset t tests for continuous variables. Intention-to-treat analyses were performed to evaluate changes in complication rates with comparison before and after the intervention regardless of SURPASS compliance. Multiple logistic regression analysis was used to evaluate associations of SURPASS with patient outcomes and including actual adherence to the checklists. Multiple Cox proportional hazards regression was used to analyze LOS. The individual checklists included in the analyses had all items checked. Preintervention and postintervention stages were analyzed while adjusting for time associations and other possible confounders in the logistic regression model,18,28 including age, sex, urgency of operation, ASA classification, anesthesia given, surgical specialty, point of time for inclusion in the study, and WHO SSC and SURPASS checklist adherence. Intention-to-treat analyses were adjusted for the same variables, except SURPASS adherence (as a proportion, as described above). With an expected mortality rate decrease from 0.015 to 0.008, a sample size of 3641 patients undergoing surgery in both preintervention and postintervention groups was required to achieve a power of 80% with an α value set to .05 (2 tailed). Intracluster correlation was not considered to affect the study power owing to heterogeneity within and between departments. The results for complications and mortality are reported as odds ratios (ORs) with 95% CIs and for LOS as hazard ratios (HRs). Two-sided P ≤ .05 was set as statistically significant. Power calculations were performed with SPPS Sample Power 2. Statistical analyses were performed using SPSS, version 24 (IBM Corporation).

    Results

    The study included 3892 procedures at baseline and 5117 procedures in the intervention periods during the 29 months, from November 1, 2012, to March 31, 2015 (Figure 1). A total of 7772 unique patients underwent 9009 procedures (mean [SD] patient age, 51.7 [22.2] years) in 8515 admissions within the study. Characteristics of patients and surgical procedures are reported in Table 1. The inclusion of gynecology as one of the study departments contributed to an overall higher proportion of women (5601 women [62.2%] and 3408 men [37.8%]; P < .001). A total of 5117 intervention procedures (mean [SD] patient age, 51.8 [22.4] years; 2913 women [56.9%] and 2204 men [43.1%]) and 3892 controls (mean [SD] patient age, 51.5 [21.8] years; 2688 women [69.1%] and 1204 men [30.9%]) were included.

    In total, 1418 of 9009 procedures (15.7%) were associated with 1 or more complications (Table 2). In adjusted intention-to-treat analyses, the number of complications decreased (OR, 0.73; 95% CI, 0.54-0.98; P = .04).

    To analyze associations of complications per procedure with preoperative and postoperative SURPASS added to the WHO SSC, multiple logistic regression analyses were performed accounting for the level of adherence. When adherence to the preoperative SURPASS checklists was achieved, adjusted analysis demonstrated a decrease in in-hospital complications (OR, 0.70; 95% CI, 0.50-0.98; P = .04) and emergency reoperations (OR, 0.42; 95% CI, 0.23-0.76; P = .004) (Table 3). Furthermore, adherence to the 3 postoperative SURPASS checklists was associated with a reduction of unplanned 30-day readmissions (OR, 0.32; 95% CI, 0.16-0.64; P = .001) in adjusted analyses.

    Analyzing time trends in adjusted Cox proportional hazards regression showed an overall shorter LOS from early to later in the study period (ie, an increasing chance of earlier discharge; HR, 1.07 per year; 95% CI, 1.02-1.13; P = .003). Added use of the SURPASS checklists showed no significant associations with LOS.

    The 30-day in-hospital mortality associated with using the preoperative SURPASS was nonsignificant (OR, 0.28; 95% CI, 0.04-1.78; P = .17). For postoperative SURPASS, the association was likewise nonsignificant (OR, 0.86; 95% CI, 0.68-1.08; P = .18). Similarly, there was no change in 30-day mortality after discharge associated with use of either the preoperative SURPASS (OR, 1.67; 95% CI, 0.38-7.44; P = .50) or the postoperative SURPASS (OR, 0.64; 95% CI, 0.17-2.45; P = .51) in all adjusted analyses.

    The 3 parallel control surgical units included 9678 procedures during the study period (mean [SD] patient age, 57.4 [22.2] years; 4124 women [42.6%] and 5554 men [57.4%]). A CONSORT flow diagram describing eligible procedures is represented in the eFigure in Supplement 2. Characteristics of the procedures and outcome measures are reported in eTables 1 and 2 in Supplement 2, respectively. There was an overall decrease in LOS in the control units during the study period, with an increased chance of being more rapidly discharged (HR, 1.07 per year; 95% CI, 1.04-1.11; P < .001). We also found an increase in complications over time (OR, 1.09; 95% CI, 1.01-1.17; P = .04) (eTable 3 in Supplement 2). In adjusted analyses, no changes were observed in emergency reoperations (OR, 0.96; 95% CI, 0.82-1.12; P = .57), 30-day readmissions (OR, 1.17; 95% CI, 0.96-1.42; P = .11), 30-day in-hospital mortality (OR, 0.95; 95% CI, 0.70-1.29; P = .73), or 30-day mortality after discharge (OR, 1.14; 95% CI, 0.81-1.59; P = .46) in these departments.

    Discussion

    Findings from this study demonstrate that adding the preoperative and postoperative SURPASS checklists to the intraoperative WHO SSC may be clinically advantageous. We found that the joint application of the 2 surgical checklist systems was associated with reduced in-hospital complications, emergency reoperations, and hospital readmissions.

    A decade ago, the WHO SSC was initially implemented in 2 Norwegian hospitals (one being the present trial hospital), resulting in a 42% relative risk reduction of complications from 19.9% to 12.4%.23 Although the WHO SSC has become national clinical policy for surgery, evidence shows that surgical complications often originate outside the operating room.12-14 Logically, this outcome suggests that a checklist to improve flow of information and completeness of requisite clinical care protocols before a patient reaches the operating room can reduce unwanted variation in preparation and planning and improve care.29 Our findings suggest that effective application of the preoperative SURPASS may achieve this. The reduction in emergency reoperations when preoperative SURPASS had been used replicates studies showing a decrease in reoperations after implementing intraoperative surgical checklists.20,30,31

    Furthermore, better use of the 3 postoperative SURPASS checklists was associated with decreased readmissions to hospital within 30 days. Improved communications optimize care delivery in transfer of patients to other units.12,32-35 The clinical associations we observed could be owing to the SURPASS discharge checklists supporting better preparation of patients when leaving the hospital (ie, plans for their medications and expectations regarding their ongoing recovery). Other studies have found that patient discharge is strengthened by use of checklists,36 and decreased readmissions have been linked to use of the WHO SSC.37

    The parallel control units had increased complication rates over time, whereas rates of emergency reoperations, 30-day readmissions, and mortality remained unchanged. Over time, we observed an overall increased complication rate in both trial and control units. We do not have data directly addressing this finding. We hypothesize, however, that the national context of the study may account for this pattern. National economic incentive systems reimbursing ICD-10 codes for complicated admissions have increased hospitals’ focus on coding practices.38 A possible explanation may be increased hospital focus on more accurate coding practice for reimbursement purposes by individual physicians throughout the study period. The increase in complications is unlikely to reflect a lack of effect from the checklist intervention, because when adjusted regression analyses were performed, the intervention was associated with lower risk of complications. Also, stricter adherence to the SURPASS checklists had a lower risk of complications than looser adherence, indicating a dose-response effect. Furthermore, use of the stepped-wedge design allowed us to adjust for time trends in complication rates.17 Both the trial departments and the control units had an overall decrease in LOS over time, and LOS was not associated with use of the SURPASS in the intervention departments. This finding contrasts with those of previous studies, which showed reduction in LOS with checklist use.23,39 We consider it possible that maximum reduction of LOS had been reached in our study owing to the national Norwegian context. Specifically, a national coordination reform took effect in January 2012.40 One of the main goals of the reform was to reduce LOS in hospitals by a build-up and enhancement of publicly funded nursing homes. This national policy program likely affected discharge decision-making throughout the study period and thus affected our findings. Our findings cannot directly support this explanation, which can be evaluated further through longitudinal outcome studies.

    Strengths and Limitations

    This study has several strengths, including the prestudy SURPASS validation process, study design, long-term collection of data, and strong engagement from hospital leaders, managers, and influential clinicians when implementing the SURPASS intervention, thus achieving good fidelity. In addition, the validation procedures with exact and extensive verification of in-hospital ICD-10 codes for complications, emergency reoperations, readmissions, mortality, and LOS from patient records linked to actual checklist adherence allowed reliable outcome measurement.27 Furthermore, the study allows distinguishing which checklists are associated with improvements on which outcomes. For example, combined use of the preoperative and postanesthesia care unit nurses’ SURPASS with the WHO SSC may improve in-hospital complications, in-hospital mortality, and LOS. Use of preoperative SURPASS checklists and the WHO SSC may influence emergency reoperations. A combined use of preoperative and postoperative SURPASS and WHO SSC may influence unplanned hospital readmissions and mortality after discharge.

    The study also has limitations. The nonsignificant change in mortality could be owing to an underpowered sample size. The calculation was performed in 2012 based on the published literature.15,20 However, the number of patients dying in our sample was lower than anticipated. Furthermore, an important consideration is whether there could be any residual confounders explaining the observed higher rate of complications after the intervention. For example, were more complex procedures performed in sicker patients after the intervention? In Table 1, we showed that there is no difference in comorbidity (ASA classification) between control and intervention departments. In the regression analyses, we have adjusted for case mixes, including age, sex, emergency procedures, ASA classification, anesthesia given, surgical specialty, point of time for inclusion in the study, and checklist use. Additional comorbidity measures such as the Charlson comorbidity index were not part of the original study protocol. However, with these rigorous adjusted analyses, we believe that very little residual confounding has remained unexplained.

    The parallel control units contributed different surgical procedures and specialties to the trial compared with the intervention departments. Comparing outcome data on similar procedures and specialties would have been ideal. However, morbidity and mortality trends in the parallel controls were similar to those of the intervention departments. The actual complexity of the SURPASS intervention, involving different professional groups across different departments, added an inherent limitation, because randomizing the start-up of the intervention with the time and resources available became unfeasible.

    In addition, overall high-fidelity application of all checklists across all professional groups for all surgical procedures was not obtained. Known implementation barriers affect checklist use globally (eg, information technology systems, checklist and personnel flow, checklist resistance, and/or checklist fatigue) and could have resulted in underestimations of the sizes of associations of intervention and clinical outcomes in our analyses. Other investigators41 have also raised these issues. Further studies of how to improve fidelity in delivering clinically effective checklists in surgical pathways are warranted.

    Conclusions

    Our findings suggest that combinations of the WHO and SURPASS checklists throughout the perioperative pathway may be clinically advantageous in improving processes of care and patient safety further with reductions in complications, reoperations, and readmissions beyond what sole use of the WHO checklist in the operating room achieves. The WHO checklist has been adopted globally for use in operating rooms. The next step to increase surgical patient safety is to use safety checklists throughout the perioperative pathway, as when combining the WHO checklist with SURPASS checklists. Rigorous large-scale multicenter randomized clinical trials are recommended to investigate this further.

    Back to top
    Article Information

    Accepted for Publication: March 5, 2020.

    Published Online: May 13, 2020. doi:10.1001/jamasurg.2020.0989

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Storesund A et al. JAMA Surgery.

    Corresponding Author: Anette Storesund, MCCN(Hons), PhD, Department of Anesthesia and Intensive Care, Haukeland University Hospital, Jonas Lies vei 65, N-5021 Bergen, Norway (anette.storesund@helse-bergen.no).

    Author Contributions: Dr Storesund had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Storesund, Haugen, Flaatten, Nortvedt, Boermeester, Mahesparan, Fevang, Thornhill, Søfteland.

    Acquisition, analysis, or interpretation of data: Storesund, Haugen, Nortvedt, Eide, Sevdalis, Tveiten, Hjallen, Størksen, Sjøen, Kolseth, Haaverstad, Sandli, Søfteland.

    Drafting of the manuscript: Storesund, Haugen, Tveiten, Fevang, Thornhill, Søfteland.

    Critical revision of the manuscript for important intellectual content: Storesund, Haugen, Flaatten, Nortvedt, Eide, Boermeester, Sevdalis, Tveiten, Mahesparan, Hjallen, Størksen, Sjøen, Kolseth, Haaverstad, Sandli, Søfteland.

    Statistical analysis: Storesund, Haugen, Eide, Tveiten, Hjallen.

    Obtained funding: Storesund, Flaatten, Sevdalis, Søfteland.

    Administrative, technical, or material support: Storesund, Hjallen, Fevang, Størksen, Søfteland.

    Supervision: Haugen, Flaatten, Nortvedt, Eide, Boermeester, Sevdalis, Mahesparan, Fevang, Haaverstad, Søfteland.

    Conflict of Interest Disclosures: Dr Storesund reported receiving grants from the Western Norway Regional Health Authority Trust and Norwegian Nurses Organization during the conduct of the study. Dr Haugen reported receiving grants from Western Norway Regional Health Authority Trust during the conduct of the study. Dr Boermeester reported receiving grants from KCI, Johnson & Johnson, and New Compliance and serving as a speaker and/or instructor from Bard Pharmaceuticals, Gore, and Smith & Nephew plc outside the submitted work and having a patent to the Surgical Patient Safety System (SURPASS) checklist licensed. Dr Sevdalis reported receiving grants from the National Institute for Health Research (NIHR) and Economic and Social Research Council (ESRC) during the conduct of the study and personal fees from London Safety & Training Solutions Ltd outside the submitted work. No other disclosures were reported.

    Funding/Support: This study was supported by grants HV1173 (Dr Storesund) and HV1172 (Dr Haugen) from the Western Norway Regional Health Authority Trust; grant NSF15-0023 from the Norwegian Nurses Organization (Dr Storesund); the NIHR Applied Research Collaboration (ARC) South London at King’s College Hospital NHS (National Health Service) Foundation Trust (Dr Sevdalis); King’s Improvement Science, which offers co-funding to the NIHR ARC South London and consists of a specialist team of improvement scientists and senior researchers based at King’s College London, from King’s Health Partners (Guy’s and St Thomas’ NHS Foundation Trust, King’s College Hospital NHS Foundation Trust, King’s College London and South London and Maudsley NHS Foundation Trust), Guy’s and St Thomas’ Charity, and the Maudsley Charity (Dr Sevdalis); and the ASPIRES research program (Antibiotic Use Across Surgical Pathways–Investigating, Redesigning and Evaluating Systems), funded by the ESRC (Dr Sevdalis).

    Role of the Funder/Sponsor: The sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The views expressed in this publication are those of the author(s) and not necessarily those of the Norwegian Nurses Organization, the Western Norway Regional Health Authority Trust, the NIHR, the ESRC, or the Department of Health and Social Care.

    Additional Contributions: We thank the dedicated managers and clinicians who made the implementation feasible. Contributions to data collection and data handling were provided by the information technology consultants Harry Waldeland, BSc, from the Western Norway Regional Health Authority Trust, Nils Eivind Johansen Widnes, RN, and Øystein Eeg, RN, from the Department of Research and Development, Bergen, Norway, and Thor-Ludvig Thorsen, BSc, Webport, Grimstad, Norway. Continuous support was provided from Stig Harthug, MD, PhD, from the Department of Research and Development, Haukeland University Hospital, Bergen, Norway and the Department of Clinical Science, University of Bergen, Bergen, Norway, Dagny Strand Klausen, MD, from the Department of Anesthesiology, Haugesund Hospital, Health Trust Fonna, Haugesund, Norway, and Nils Sletteskog, MD, from the Department of Surgery, Førde Central Hospital, Førde, Norway and the Department of Clinical Medicine, Faculty of Medicine, University of Bergen, Bergen, Norway. None of them were compensated for their support, except Mr Thorsen, who received payment (US $4150) through grant NSF15-0023 from the Norwegian Nurses Organization.

    References
    1.
    Abbott  TEF, Ahmad  T, Phull  MK,  et al; International Surgical Outcomes Study (ISOS) group.  The surgical safety checklist and patient outcomes after surgery: a prospective observational cohort study, systematic review and meta-analysis.   Br J Anaesth. 2018;120(1):146-155. doi:10.1016/j.bja.2017.08.002 PubMedGoogle ScholarCrossref
    2.
    Bergs  J, Hellings  J, Cleemput  I,  et al.  Systematic review and meta-analysis of the effect of the World Health Organization surgical safety checklist on postoperative complications.   Br J Surg. 2014;101(3):150-158. doi:10.1002/bjs.9381 PubMedGoogle ScholarCrossref
    3.
    Borchard  A, Schwappach  DL, Barbir  A, Bezzola  P.  A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery.   Ann Surg. 2012;256(6):925-933. doi:10.1097/SLA.0b013e3182682f27 PubMedGoogle ScholarCrossref
    4.
    Boyd  J, Wu  G, Stelfox  H.  The impact of checklists on inpatient safety outcomes: a systematic review of randomized controlled trials.   J Hosp Med. 2017;12(8):675-682. doi:10.12788/jhm.2788 PubMedGoogle ScholarCrossref
    5.
    Lau  CSM, Chamberlain  RS.  The World Health Organization surgical safety checklist improves post-operative outcomes: a meta-analysis and systematic review.   Surg Sci. 2016;7(4):206-217. doi:10.4236/ss.2016.74029 Google ScholarCrossref
    6.
    Thomassen  Ø, Storesund  A, Søfteland  E, Brattebø  G.  The effects of safety checklists in medicine: a systematic review.   Acta Anaesthesiol Scand. 2014;58(1):5-18. doi:10.1111/aas.12207 PubMedGoogle ScholarCrossref
    7.
    Urbach  DR, Govindarajan  A, Saskin  R, Wilton  AS, Baxter  NN.  Introduction of surgical safety checklists in Ontario, Canada.   N Engl J Med. 2014;370(11):1029-1038. doi:10.1056/NEJMsa1308261 PubMedGoogle ScholarCrossref
    8.
    Reames  BN, Krell  RW, Campbell  DA  Jr, Dimick  JB.  A checklist-based intervention to improve surgical outcomes in Michigan: evaluation of the Keystone Surgery program.   JAMA Surg. 2015;150(3):208-215. doi:10.1001/jamasurg.2014.2873 PubMedGoogle ScholarCrossref
    9.
    Urbach  DR.  The challenge of quality improvement in surgical care.   JAMA Surg. 2015;150(3):215. doi:10.1001/jamasurg.2014.2908 PubMedGoogle ScholarCrossref
    10.
    Leape  LL.  The checklist conundrum.   N Engl J Med. 2014;370(11):1063-1064. doi:10.1056/NEJMe1315851 PubMedGoogle ScholarCrossref
    11.
    Russ  S, Rout  S, Sevdalis  N, Moorthy  K, Darzi  A, Vincent  C.  Do safety checklists improve teamwork and communication in the operating room? a systematic review.   Ann Surg. 2013;258(6):856-871. doi:10.1097/SLA.0000000000000206 PubMedGoogle ScholarCrossref
    12.
    Greenberg  CC, Regenbogen  SE, Studdert  DM,  et al.  Patterns of communication breakdowns resulting in injury to surgical patients.   J Am Coll Surg. 2007;204(4):533-540. doi:10.1016/j.jamcollsurg.2007.01.010 PubMedGoogle ScholarCrossref
    13.
    Griffen  FD, Stephens  LS, Alexander  JB,  et al.  The American College of Surgeons’ closed claims study: new insights for improving care.   J Am Coll Surg. 2007;204(4):561-569. doi:10.1016/j.jamcollsurg.2007.01.013 PubMedGoogle ScholarCrossref
    14.
    de Vries  EN, Hollmann  MW, Smorenburg  SM, Gouma  DJ, Boermeester  MA.  Development and validation of the Surgical Patient Safety System (SURPASS) checklist.   Qual Saf Health Care. 2009;18(2):121-126. doi:10.1136/qshc.2008.027524 PubMedGoogle ScholarCrossref
    15.
    de Vries  EN, Prins  HA, Crolla  RM,  et al; SURPASS Collaborative Group.  Effect of a comprehensive surgical safety system on patient outcomes.   N Engl J Med. 2010;363(20):1928-1937. doi:10.1056/NEJMsa0911535 PubMedGoogle ScholarCrossref
    16.
    Mehta  N, Amaranathan  A, Jayapal  L, Kundra  P, Nelamangala Ramakrishnaiah  VP.  Effect of comprehensive surgical safety system on patients’ outcome: a prospective clinical study.   Cureus. 2018;10(5):e2601. doi:10.7759/cureus.2601 PubMedGoogle Scholar
    17.
    Brown  C, Hofer  T, Johal  A,  et al.  An epistemology of patient safety research: a framework for study design and interpretation—part 2, study design.   Qual Saf Health Care. 2008;17(3):163-169. doi:10.1136/qshc.2007.023648 PubMedGoogle ScholarCrossref
    18.
    Hemming  K, Taljaard  M, McKenzie  JE,  et al.  Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration.   BMJ. 2018;363:k1614. doi:10.1136/bmj.k1614 PubMedGoogle ScholarCrossref
    19.
    Mdege  ND, Man  MS, Taylor Nee Brown  CA, Torgerson  DJ.  Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation.   J Clin Epidemiol. 2011;64(9):936-948. doi:10.1016/j.jclinepi.2010.12.003 PubMedGoogle ScholarCrossref
    20.
    Haynes  AB, Weiser  TG, Berry  WR,  et al; Safe Surgery Saves Lives Study Group.  A surgical safety checklist to reduce morbidity and mortality in a global population.   N Engl J Med. 2009;360(5):491-499. doi:10.1056/NEJMsa0810119 PubMedGoogle ScholarCrossref
    21.
    World Alliance for Patient Safely; World Health Organization. Implementation Manual: Surgical Safety Checklist. Published May 2008. Accessed February 16, 2020. https://www.who.int/patientsafety/safesurgery/tools_resources/SSSL_Manual_finalJun08.pdf
    22.
    Storesund  A, Haugen  AS, Wæhle  HV,  et al.  Validation of a Norwegian version of Surgical Patient Safety System (SURPASS) in combination with the World Health Organizations’ surgical safety checklist (WHO SSC).   BMJ Open Qual. 2019;8(1):e000488. doi:10.1136/bmjoq-2018-000488 PubMedGoogle Scholar
    23.
    Haugen  AS, Søfteland  E, Almeland  SK,  et al.  Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial.   Ann Surg. 2015;261(5):821-828. doi:10.1097/SLA.0000000000000716 PubMedGoogle ScholarCrossref
    24.
    Powell  BJ, Waltz  TJ, Chinman  MJ,  et al.  A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project.   Implement Sci. 2015;10(21):21. doi:10.1186/s13012-015-0209-1 PubMedGoogle ScholarCrossref
    25.
    Conley  DM, Singer  SJ, Edmondson  L, Berry  WR, Gawande  AA.  Effective surgical safety checklist implementation.   J Am Coll Surg. 2011;212(5):873-879. doi:10.1016/j.jamcollsurg.2011.01.052 PubMedGoogle ScholarCrossref
    26.
    Haynes  AB, Berry  WR, Gawande  AA.  What do we know about the safe surgery checklist now?   Ann Surg. 2015;261(5):829-830. doi:10.1097/SLA.0000000000001144 PubMedGoogle ScholarCrossref
    27.
    Storesund  A, Haugen  AS, Hjortås  M,  et al.  Accuracy of surgical complication rate estimation using ICD-10 codes.   Br J Surg. 2019;106(3):236-244. doi:10.1002/bjs.10985 PubMedGoogle ScholarCrossref
    28.
    Hemming  K, Haines  TP, Chilton  PJ, Girling  AJ, Lilford  RJ.  The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting.   BMJ. 2015;350:h391. doi:10.1136/bmj.h391 PubMedGoogle ScholarCrossref
    29.
    de Vries  EN, Eikens-Jansen  MP, Hamersma  AM, Smorenburg  SM, Gouma  DJ, Boermeester  MA.  Prevention of surgical malpractice claims by use of a surgical safety checklist.   Ann Surg. 2011;253(3):624-628. doi:10.1097/SLA.0b013e3182068880 PubMedGoogle ScholarCrossref
    30.
    van Boxtel  AGM, van Veghel  D, Soliman Hamad  MA, Schulz  DN, Stepaniak  PS, van Straten  AHM.  Use of an intraoperative checklist to decrease the incidence of re-exploration for postoperative bleeding after cardiac surgery.   Interact Cardiovasc Thorac Surg. 2017;25(4):555-558. doi:10.1093/icvts/ivx130 PubMedGoogle ScholarCrossref
    31.
    Lepänluoma  M, Rahi  M, Takala  R, Löyttyniemi  E, Ikonen  TS.  Analysis of neurosurgical reoperations: use of a surgical checklist and reduction of infection-related and preventable complication-related reoperations.   J Neurosurg. 2015;123(1):145-152. doi:10.3171/2014.12.JNS141077 PubMedGoogle ScholarCrossref
    32.
    Kim  SW, Maturo  S, Dwyer  D,  et al.  Interdisciplinary development and implementation of communication checklist for postoperative management of pediatric airway patients.   Otolaryngol Head Neck Surg. 2012;146(1):129-134. doi:10.1177/0194599811421745 PubMedGoogle ScholarCrossref
    33.
    Graan  SM, Botti  M, Wood  B, Redley  B.  Nursing handover from ICU to cardiac ward: standardised tools to reduce safety risks.   Aust Crit Care. 2016;29(3):165-171. doi:10.1016/j.aucc.2015.09.002 PubMedGoogle ScholarCrossref
    34.
    Johnston  MJ, Arora  S, King  D,  et al.  A systematic review to identify the factors that affect failure to rescue and escalation of care in surgery.   Surgery. 2015;157(4):752-763. doi:10.1016/j.surg.2014.10.017 PubMedGoogle ScholarCrossref
    35.
    Nagpal  K, Vats  A, Lamb  B,  et al.  Information transfer and communication in surgery: a systematic review.   Ann Surg. 2010;252(2):225-239. doi:10.1097/SLA.0b013e3181e495c2 PubMedGoogle ScholarCrossref
    36.
    Drake  K, McBride  M, Bergin  J, Vandeweerd  H, Higgins  A.  Ensuring safe discharge with a standardized checklist and discharge pause.   Nursing. 2017;47(8):65-68. doi:10.1097/01.NURSE.0000521042.81195.86 PubMedGoogle ScholarCrossref
    37.
    Rodella  S, Mall  S, Marino  M,  et al.  Effects on clinical outcomes of a 5-year surgical safety checklist implementation experience: a large-scale population-based difference-in-differences study.   Health Serv Insights. 2018;11:1178632918785127. doi:10.1177/1178632918785127 PubMedGoogle Scholar
    38.
    The Office of the Auditor General of Norway. The Office of the Auditor General’s investigation of medical coding practice within the health enterprises. Document 3:5 (2016-2017). Accessed February 16, 2020. https://www.riksrevisjonen.no/globalassets/reports/en-2016-2017/codingpracticehealthenterprises.pdf
    39.
    de Jager  E, Gunnarsson  R, Ho  YH.  Implementation of the World Health Organization surgical safety checklist correlates with reduced surgical mortality and length of hospital admission in a high-income country.   World J Surg. 2019;43(1):117-124. doi:10.1007/s00268-018-4703-x PubMedGoogle ScholarCrossref
    40.
    Ministry of Health and Care Services. Report No. 47 (2008–2009) to the Storting. The Coordination Reform—proper treatment at the right place and right time. Full version in Norwegian. Summary in English: Oslo, Norway. Published 2008. Accessed September 30, 2019. https://www.regjeringen.no/en/dokumenter/report.no.-47-to-the-storting-2008-2009/id567201/?ch=1&q=#?ch=1&q=&_t_dtq=true&_suid=146037692843204369084028942121
    41.
    Russ  SJ, Sevdalis  N, Moorthy  K,  et al.  A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: lessons from the “Surgical Checklist Implementation Project”.   Ann Surg. 2015;261(1):81-91. doi:10.1097/SLA.0000000000000793 PubMedGoogle ScholarCrossref
    ×