In total, 21 hospitals with poor baseline culture experienced significant improvement in 1 or more culture domains following the implementation of ISQIC. Although several hospitals experienced improvement in a single domain, most hospitals improved in 2 or more domains.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Yuce TK, Yang AD, Johnson JK, et al. Association Between Implementing Comprehensive Learning Collaborative Strategies in a Statewide Collaborative and Changes in Hospital Safety Culture. JAMA Surg. 2020;155(10):934–940. doi:10.1001/jamasurg.2020.2842
Does an association exist between participation in a comprehensive, multicomponent, statewide surgical quality improvement collaborative and hospital safety culture?
In this survey study of 1024 respondents from 36 hospitals, a diverse group of hospitals enrolled in ISQIC (Illinois Surgical Quality Improvement Collaborative) experienced a significant improvement in hospital safety culture. The largest improvements were reported by hospitals with the lowest baseline hospital safety culture.
Participation in a comprehensive, multicomponent, statewide surgical quality improvement collaborative was associated with positive changes in hospital safety culture, particularly for hospitals with the poorest baseline culture.
Hospital safety culture remains a critical consideration when seeking to reduce medical errors and improve quality of care. Little is known regarding whether participation in a comprehensive, multicomponent, statewide quality collaborative is associated with changes in hospital safety culture.
To examine whether implementation of a comprehensive, multicomponent, statewide surgical quality improvement collaborative is associated with changes in hospital safety culture.
Design, Setting, and Participants
In this survey study, the Safety Attitudes Questionnaire, a 56-item validated survey covering 6 culture domains (teamwork, safety, operating room safety, working conditions, perceptions of management, and employee engagement), was administered to a random sample of physicians, nurses, operating room staff, administrators, and leaders across Illinois hospitals to assess hospital safety culture prior to launching a new statewide quality collaborative in 2015 and then again in 2017. The final analysis included 1024 respondents from 36 diverse hospitals, including major academic, community, and rural centers, enrolled in ISQIC (Illinois Surgical Quality Improvement Collaborative).
Participation in a comprehensive, multicomponent statewide surgical quality improvement collaborative. Key components included enrollment in a common standardized data registry, formal quality and process improvement training, participation in collaborative-wide quality improvement projects, funding support for local projects, and guidance provided by surgeon mentors and process improvement coaches.
Main Outcomes and Measures
Perception of hospital safety culture.
The overall survey response rate was 43.0% (580 of 1350 surveys) in 2015 and 39.0% (444 of 1138 surveys) in 2017 from 36 hospitals. Improvement occurred in all the overall domains, with significant improvement in teamwork climate (change, 3.9%; P = .03) and safety climate (change, 3.2%; P = .02). The largest improvements occurred in individual measures within domains, including physician-nurse collaboration (change, 7.2%; P = .004), reporting of concerns (change, 4.7%; P = .009), and reduction in communication breakdowns (change, 8.4%; P = .005). Hospitals with the lowest baseline safety culture experienced the largest improvements following collaborative implementation (change range, 11.1%-14.9% per domain; P < .05 for all). Although several hospitals experienced improvement in safety culture in 1 domain, most hospitals experienced improvement across several domains.
Conclusions and Relevance
This survey study found that hospital enrollment in a statewide quality improvement collaborative was associated with overall improvement in safety culture after implementing multiple learning collaborative strategies. Hospitals with the poorest baseline culture reported the greatest improvement following implementation of the collaborative.
Hospital safety culture has been defined as the individual and organizational perceptions and behaviors that determine commitment to patient safety.1,2 The link between hospital safety culture, quality of care, and patient outcomes has been well documented.3-7 Given the current focus on quality of care in the United States health care system, an increasing emphasis has been placed on improving hospital safety culture.8 Previous interventions aimed at improving the delivery of care in operating rooms and intensive care units, such as teamwork training and implementation of checklists, have been linked to both improved perception of safety culture and decreased morbidity and mortality rates.8-13 However, many of these interventions have been implemented in specific clinical areas, rather than across an entire department or hospital. Less is known regarding the effects of interventions designed to influence department- or hospital-wide safety culture. Therefore, statewide quality improvement (QI) collaboratives provide an ideal mechanism for implementing and evaluating strategies aimed at improving safety culture on a larger scale.8,11,14
The Illinois Surgical Quality Improvement Collaborative (ISQIC) represents a diverse partnership of hospitals that performs over 80% of complex surgical procedures15 in the state of Illinois. The collaborative has grown since its foundation in 2014 to include 55 hospitals, including major academic centers, community hospitals, and rural hospitals. All hospitals (1) are enrolled in a common standardized data registry (American College of Surgeons National Surgical Quality Improvement Program [ACS NSQIP] plus an Illinois-specific registry for best-practice measure adherence monitoring), (2) receive formal quality and process improvement training, (3) participate in collaborative-wide QI projects while also receiving support for local hospital projects, (4) receive funding for QI initiatives, (5) create leadership engagement plans, (6) present quality reports annually to hospital leadership, and (7) receive guidance from surgeon mentors and process improvement coaches.16 Each participating hospital also receives benchmarked hospital-level reports, detailing adherence to process measures in the collaborative as well as their postoperative outcomes.
Whereas some specific interventions have been shown to improve hospital safety culture, less is known regarding the effect of participation in a comprehensive, multicomponent statewide quality collaborative on safety culture. As a component of the continued evaluation of ISQIC, the safety culture of participating hospitals was iteratively assessed using the Safety Attitudes Questionnaire (SAQ)17 prior to launching the collaborative (baseline, 2015) and then 3 years later (year 2, 2017). Therefore, the present study was conducted to examine changes in physician (surgeon and anesthesiologist), nurse, operating room staff, administrator, and hospital leadership perception of hospital safety culture associated with the implementation of a comprehensive, multicomponent statewide surgical QI collaborative.
The SAQ was administered to personnel at all ISQIC hospitals between January and May 2015, and again between May and June 2017. Data from the 2015 administration were collected prior to the full implementation of ISQIC and were considered a baseline assessment of safety culture. Each ISQIC hospital provided separate personnel lists, including all hospital administrators and leadership members, surgeons, anesthesiologists, nursing staff (floor, intensive care unit, and operating room), and hospital quality improvement staff, to ensure capture of a variety of perspectives from each hospital. Survey participants were then randomly identified to ensure adequate sampling of each of the populations or roles. A unique REDCap link was emailed to each participant to ensure the survey was completed securely. Nonrespondents were sent 2 subsequent reminder emails. The 19 hospitals that joined ISQIC after the 2015 SAQ administration were excluded from the present study because they lacked baseline data. This study was deemed nonhuman subjects research by the Northwestern University institutional review board office. No one received compensation or was offered any incentive for participating in this study.
The SAQ has been validated17-20 and widely used to measure hospital safety culture in previous studies.21-26 The questionnaire contains 6 domains (teamwork climate, safety climate, or specific safety climate, perceptions of management, working conditions, and employee engagement) each consisting of between 2 and 24 questions (Table 1). These domains are designed to assess the culture of teamwork, safety, communication, and patient care. A 5-point Likert scale format was used to collect responses to each question (strongly disagree, disagree, neutral, agree, or strongly agree). The participants were asked to consider each response in terms of the specific patient care environment in their clinical unit.
Responses to the SAQ were evaluated for each individual question as well as by domain. Hospital-level domain scores were calculated as a composite of all respondents at each hospital. The percentage of positive responses was measured to evaluate safety culture, similar to previous work.6 Positive responses included each item answered with agree or strongly agree, unless the question was negatively worded, in which case disagree or strongly disagree was used. Descriptive statistics were calculated using t tests to evaluate the change in the percentage of positive responses between 2015 and 2017. Hospitals were further stratified into quartiles by baseline safety culture (2015 scores). Quartiles were established based on the 2015 score of each hospital in the 6 SAQ domains. The first quartile represented hospitals with the lowest (poorest) baseline culture in the respective domain, and the fourth quartile represented hospitals with the most positive culture in the respective domain. This stratification was then used to evaluate differences in the magnitude of change between years based on baseline culture. An analysis of variance with pairwise comparisons was used to assess significant differences between hospital groups. Only respondents who completed the survey were included. Hospital-specific covariates were obtained using the 2016 American Hospital Association Annual Survey. All tests were 2-sided, and the level of statistical significance was set to .05. Statistical analyses were performed using Stata, version 15.1 (StataCorp).
Responses to the SAQ for 2015 and 2017 were available from 36 hospitals with a total of 1024 respondents. The overall SAQ response rate was 580 of 1350 surveys (43.0%) for 2015 and 444 of 1138 surveys (39.0%) for 2017. Individual hospital response rates ranged from 15.2% to 79.6%, whereas the hospital-specific number of respondents ranged from 5 to 30 (median, 19; interquartile range, 12-24). A diverse group of respondents and hospital types were captured in the survey (Table 2). The proportion of respondents remained similar between both years of the SAQ administration. Most of the 36 hospitals had more than 200 inpatient beds (22 [61.1%]), were affiliated with a medical school (26 [72.2%]) or residency program (22 [61.1%]), first enrolled in ACS NSQIP on joining ISQIC (28 [77.8%]), and were accredited by the Joint Commission (31 [86.1%]). Most of the ISQIC hospitals in this cohort were operating as nonprofit organizations (22 [61.1%]), whereas hospitals with religious affiliation (13 [36.1%]) and government or for-profit organizations (1 [2.8%]) represented the minority.
Operating room safety was the SAQ domain with the most positive perceived safety culture in 2015 (of 580 respondents, 543 [93.6%] were positive responses) and remained as such in 2017 (of 444 respondents, 420 [94.7%] were positive responses) (Table 3). The least positive perception of safety culture was observed in the perceptions of management domain in both 2015 (434 positive responses [74.9%]) and 2017 (345 positive responses [77.6%]). A significant increase in the percentage of positive responses for the teamwork climate domain (change, 3.9%; P = .03) and safety climate domain (change, 3.2%; P = .02) occurred between 2015 and 2017. Although not statistically significant, increases in the percentages of positive responses were also observed for the remaining safety culture domains. Robust improvement occurred in hospital-level individual measures in domains, including physician-nurse collaboration (change, 7.2%; P = .004), improved reporting of concerns (change, 4.7%; P = .009), surgeon participation in quality improvement design (change, 8.7%; P = .04), and reduction in communication breakdowns (change, 8.4%; P = .005). These hospital-level results are mirrored in the respondent-level analysis. Furthermore, evaluation of differences in the change in hospital safety culture by hospital characteristics (eg, hospital management, Joint Commission accreditation status, and teaching status) yielded no significant findings.
Hospitals in the lowest quartile of baseline 2015 culture (first quartile) experienced the largest increase in percentage of positive responses for all safety culture domains between 2015 and 2017 (range, 11.1%-14.9% per domain; P < .05 for all) (Table 4). The largest increase in perceived hospital safety culture for those hospitals in the lowest quartile of the 2015 survey occurred in the domains of teamwork climate (69.3% vs 84.2%; P < .001) and perceptions of management (62.6% vs 76.8%; P < .001). Hospitals in the highest quartile at baseline 2015 (fourth quartile) experienced minimal change in the perception of safety culture among their employees, with the exception of less positive perceptions of both institutional management (84.5% vs 75.4%; P < .001) and working conditions (90.9% vs 83.1%; P = .04). In total, 21 (58.3%) of 36 hospitals that had 1 or more baseline culture domains in the lowest quartile at the beginning of the collaborative saw a significant improvement after the implementation of ISQIC (Figure). Two (5.6%) hospitals scored in the lowest quartile for all 6 culture domains and experienced significant increases across each domain. Hospitals that experienced a significant improvement in 1 or more culture domains more frequently had fewer than 200 beds (57.5% vs 33.1%; P = .02), an ACGME-accredited residency program (76.2% vs 66.7%; P = .03), or were classified as a rural location (19.0% vs 7.4%; P = .006).
Hospital safety culture has been recognized as an important component for the delivery of high-quality care. In the present study, changes in the perceptions of hospital safety culture were evaluated following the implementation of a comprehensive, multicomponent, statewide quality improvement learning collaborative. Although perceptions of hospital safety culture improved across all 6 SAQ domains, statistically significant improvements were observed in teamwork climate and overall hospital safety climate. Furthermore, hospitals with the lowest baseline perceptions of hospital safety culture experienced the largest improvement following the implementation of ISQIC.
Although variation in hospital safety culture was found, improvement was observed in hospitals participating for 3 years in a comprehensive, multicomponent, statewide QI collaborative. These changes likely reflected, in part, the multitude of tools, assistance, and other resources made available to hospitals participating in ISQIC. In particular, hospitals participating in ISQIC reported detailed data to ACS NSQIP, which were used to create risk-adjusted benchmark comparison reports on a variety of postoperative outcomes. These tailored reports go beyond the traditional comparisons provided by ACS NSQIP to member hospitals. Specifically, they enabled hospitals to compare their performance across clinical domains with their peers. As a result, they enabled ISQIC hospitals to identify areas of poor performance and target them for QI. The ISQIC hospitals also received comparative feedback regarding adherence to critical process measures (ie, best practice adherence), further augmenting the ability of the hospital to recognize areas for improvement. In combination, these detailed reports, combining patient outcomes and process measure adherence, were designed to equip hospitals with the motivation and data necessary to successfully implement QI projects and improve care.
Beyond the use of hospital-specific data, a collaborative-wide emphasis was placed on providing teamwork-specific resources to each participating hospital. These resources in ISQIC included formal QI and process adherence training, networking via semiannual collaborative meetings with achievement recognition, and QI coaching and mentoring. These tools were designed to ensure each hospital was appropriately equipped to implement site-specific changes aimed at improving quality. Furthermore, the combination of training, networking, and mentorship through the collaborative facilitated the creation of a social network between hospitals through which knowledge, skills, and lessons learned were shared.
In addition, ISQIC participants took part in several mandatory annual statewide initiatives, with examples including comprehensive venous thromboembolism prophylaxis and opioid reduction. These projects were designed to promote QI across the collaborative by targeting both hospital-level and surgeon-level components while fostering a sense of collaboration. Hospitals were also encouraged to design and implement hospital-specific QI initiatives with financial support for projects provided by the collaborative. These supported initiatives enabled hospitals to focus on specific areas that had been identified for improvement. Taken together, these collaborative-wide and hospital-specific interventions likely played a key role in the improvement of hospital safety culture.
Operating room safety was the domain with the most positive perception of safety culture in both years. These findings were likely a reflection of the long-standing importance placed on patient safety in an operating room.13,27 In particular, the use of safety protocols, including preprocedure patient verification, surgical site marking, and the “time out” prior to the procedure, emphasize patient safety in the operating room and have been linked with improved patient outcomes.28 Given the strong safety culture in the operating room, this environment may serve as an example of how to successfully implement safety interventions for departments and hospitals as a whole. Furthermore, with little room for improvement in operating room safety, the improvements in the domains of teamwork and safety across ISQIC hospitals likely reflected changes that occurred on the wards, intensive care units, and postanesthesia care units.
Hospitals with the poorest safety culture at the launch of ISQIC experienced the largest increase in perception of culture after 3 years in the collaborative. These findings suggested that improvements can be achieved in hospitals with low safety culture through a collaborative approach that places an emphasis on teamwork, enhances employee and management engagement, and fosters an environment of safety. Previous work by members of our group has supported the association between positive safety culture and decreases in serious morbidity and mortality rates.6 Numerous studies have further described the association between promoting safety culture and patient outcomes at the hospital level as well as in operating rooms or intensive care units.8,9,11 Those findings support the idea that hospital safety culture serves as a potential target for enhancing quality of care. Thus, the increase in positive perceptions of safety culture observed across numerous hospitals in ISQIC may have important implications regarding patient outcomes and the benefits associated with participating in statewide QI collaboratives.
Key components that influence the overall safety culture of a hospital include communication, teamwork, and the ability to identify and report medical errors in the workplace. These factors reflect interactions between clinicians and administrators in hospitals rather than the organizational makeup of a hospital.29 As a result, interventions designed to improve hospital safety culture would likely not require marked organizational changes. The employee-specific benefits of enacting these interventions and promoting positive hospital safety culture have been well documented. High levels of health care professional satisfaction and patient satisfaction,30 along with low rates of staff turnover,24 have been found in hospitals with positive safety culture. These staffing-related benefits, coupled with the clinical benefits of improved patient outcomes, have highlighted the importance of measuring and working to improve hospital safety culture.
The results of this study should be interpreted in light of several limitations. First, the response rate for the survey decreased between 2015 and 2017, from 43.0% to 39.0%. Although these response rates are considered reasonable for health care survey research, nonresponse bias may influence the results given the lack of information on nonresponders. Second, respondents also represent a random sampling of hospital employees; therefore, perceptions of safety culture in each facility may not reflect the views of all staff. However, a diverse group of both clinical and administrative staff completed the survey, increasing the likelihood that the reported perceptions reflected those of surgical departments and service lines as a whole. Third, owing to the lack of SAQ results from a control group of hospitals or of hospitals not participating in ISQIC between 2015 and 2017, we were unable to compare improvements in safety culture noted in ISQIC hospitals to potential contemporaneous changes occurring nationally. However, the improvements observed in hospital safety culture in this study are similar to the improvements reported in studies evaluating single interventions aimed at boosting safety culture.8,11,24,29 Therefore, it is more likely that the present results indicate improvements associated with the implementation of ISQIC rather than changes occurring nationally.
Our findings suggested that participating in a comprehensive, multicomponent, statewide quality collaborative was associated with improvements in the perception of hospital safety culture. Those hospitals with low baseline safety culture experienced the largest increase while participating in the collaborative. Given the evidence supporting an association between safety culture and patient outcomes, the implementation of statewide quality collaboratives to improve safety culture may prove beneficial in subsequently improving outcomes for patients. Further work is needed to determine the clinical effects of these findings as well as those components of comprehensive, multicomponent, statewide QI collaboratives with the greatest influence on safety culture.
Accepted for Publication: April 18, 2020.
Corresponding Author: Karl Y. Bilimoria, MD, MS, Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery, Feinberg School of Medicine, Northwestern Medicine, 633 N St Clair St, 20th Floor, Chicago, IL 60611 (firstname.lastname@example.org).
Published Online: August 12, 2020. doi:10.1001/jamasurg.2020.2842
Author Contributions: Drs Yuce and Bilimoria had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Yuce, Yang, Odell, Kreutzer, O’Leary, Bilimoria.
Acquisition, analysis, or interpretation of data: Yuce, Yang, Johnson, Odell, Love, Schlick, Zambrano, Shan, Halverson.
Drafting of the manuscript: Yuce, Odell, Zambrano, Bilimoria.
Critical revision of the manuscript for important intellectual content: Yuce, Yang, Johnson, Odell, Love, Kreutzer, Schlick, Shan, O’Leary, Halverson, Bilimoria.
Statistical analysis: Yuce, Odell, Schlick, Zambrano, Shan, Bilimoria.
Obtained funding: Bilimoria.
Administrative, technical, or material support: Yang, Johnson, Odell, Love, Kreutzer, O’Leary, Halverson.
Supervision: Yang, Odell, Bilimoria.
Conflict of Interest Disclosures: Dr Yuce was supported by postdoctoral research fellowship 5T32HS000078 from the Agency for Healthcare Research and Quality (AHRQ). Dr Yang’s research is supported by award K08HL145139 from the National Heart, Lung, and Blood Institute (NHLBI). Dr Odell reported receiving grants from the AHRQ, American College of Surgeons, and National Cancer Institute (NCI) during the conduct of the study. Dr O’Leary reported receiving grants from AHRQ during the conduct of the study. Dr Bilimoria reported receiving grants from AHRQ, Blue Cross and Blue Shield of Illinois (Health Care Services Corporation), Mallinckrodt, NCI, NHLBI, and Pacira during the conduct of the study. No other disclosures were reported.
Funding/Support: This work was funded by grant R01 5R01HS024516-04 from AHRQ and by support from Blue Cross and Blue Shield of Illinois, both to Dr Bilimoria.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: Views expressed in this work represent those of the authors only.
Meeting Presentation: This paper was presented at the American College of Surgeons Clinical Congress; October 29, 2019; San Francisco, California.
Additional Information: The leaders of the Illinois Surgical Quality Improvement Collaborative are all authors on this paper. The 36 hospitals in Illinois associated with the collaborative are Presence Covenant Medical Center, Urbana; Blessing Hospital, Quincy; Central DuPage Hospital, Winfield; Northshore University/Evanston, Evanston; OSF St. Joseph Medical Center, Bloomington; Memorial Hospital of Carbondale, Carbondale; Northwestern Memorial Hospital, Chicago Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago; Loyola University Medical Center, Maywood; Mercy Hospital and Medical Center, Chicago; Sinai Health System, Chicago; Presence Resurrection Medical Center, Chicago; Presence Saint Joseph Hospital, Chicago; Swedish Covenant Hospital, Chicago; The University of Chicago Medicine/ Comer Childrens, Chicago; Advocate Good Samaritan Hospital, Downers Grove; Elmhurst Memorial Hospital, Elmhurst; Presence Saint Francis Hospital, Evanston; OSF Saint Mary Medical Center, Galesburg; Delnor Hospital, Geneva; Herrin Hospital, Herrin; Jersey Community Hospital, Jerseyville; Silver Cross Hospital, New Lenox; Riverside Medical Center, Kankakee; Northwestern Lake Forest Hospital, Lake Forest; Anderson Hospital, Maryville; Edward Hospital, Naperville; Rush Oak Park, Oak Park; OSF St. Elizabeth Medical Center, Ottawa; Northwest Community Hospital, Arlington Heights; OSF Saint Francis Medical Center, Peoria; OSF St. James, Pontiac; OSF St. Anthony Medical Center, Rockford; Swedish American Hospital, Rockford; Carle Foundation Hospital, Urbana; and Iroquois Memorial Hospital, Watseka.
Create a personal account or sign in to: