Hospital Quality Improvement Interventions, Statewide Policy Initiatives, and Rates of Cesarean Delivery for Nulliparous, Term, Singleton, Vertex Births in California | Pregnancy | JAMA | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Flowchart of California Hospitals by Participation in the Supporting Vaginal Birth Collaborative
Flowchart of California Hospitals by Participation in the Supporting Vaginal Birth Collaborative

aAdditional information appears in eFigure 1 in the Supplement.

Figure 2.  Rates of Cesarean Delivery for NTSV Births Over Time in the US vs in California
Rates of Cesarean Delivery for NTSV Births Over Time in the US vs in California

Starting in 2016, a multifaceted initiative (a mentor-led collaborative to support hospital-level quality improvement initiatives and statewide interventions that affected all hospitals) was implemented in California to decrease the rates of cesarean delivery for nulliparous, term, singleton, vertex (NTSV) births.

Table 1.  Hospital and Patient Characteristics by Hospital Participation Status in California Maternal Quality Care Collaborative Supporting Vaginal Birth Collaborative
Hospital and Patient Characteristics by Hospital Participation Status in California Maternal Quality Care Collaborative Supporting Vaginal Birth Collaborative
Table 2.  Interventions Used by Hospitals Participating in the California Maternal Quality Care Collaborative Supporting Vaginal Birth Collaborative
Interventions Used by Hospitals Participating in the California Maternal Quality Care Collaborative Supporting Vaginal Birth Collaborative
1.
Main  EK, Morton  CH, Melsop  K,  et al.  Creating a public agenda for maternity safety and quality in cesarean delivery.   Obstet Gynecol. 2012;120(5):1194-1198. PubMedGoogle ScholarCrossref
2.
Betran  AP, Torloni  MR, Zhang  JJ,  et al.  WHO statement on caesarean section rates.   BJOG. 2016;123(5):667-670. PubMedGoogle ScholarCrossref
3.
Office of Disease Prevention and Health Promotion. Healthy People 2020: topics and objectives, maternal, infant, and child health. Accessed August 26, 2020. https://www.healthypeople.gov/2020/topics-objectives/topic/maternal-infant-and-child-health/objectives
4.
Spong  CY, Berghella  V, Wenstrom  KD,  et al.  Preventing the first cesarean delivery.   Obstet Gynecol. 2012;120(5):1181-1193. PubMedGoogle ScholarCrossref
5.
National Center for Health Statistics. Chapter 26: Maternal, infant, and child health. In:  Healthy People 2020 Midcourse Review. National Center for Health Statistics; 2016.
6.
Martin  JA, Hamilton  BE, Osterman  MJK, Driscoll  AK.  Births: final data for 2018.   Natl Vital Stat Rep. 2019;68(13):1-47. PubMedGoogle Scholar
7.
Cáceres  IA, Arcaya  M, Declercq  E,  et al.  Hospital differences in cesarean deliveries in Massachusetts (US) 2004-2006.   PLoS One. 2013;8(3):e57817. PubMedGoogle Scholar
8.
Kozhimannil  KB, Law  MR, Virnig  BA.  Cesarean delivery rates vary tenfold among US hospitals.   Health Aff (Millwood). 2013;32(3):527-535. PubMedGoogle ScholarCrossref
9.
White VanGompel  E, Perez  S, Datta  A,  et al.  Cesarean overuse and the culture of care.   Health Serv Res. 2019;54(2):417-424. PubMedGoogle ScholarCrossref
10.
Valdes  EG.  Examining cesarean delivery rates by race.   J Racial Ethn Health Disparities. Published online August 17, 2020. doi:10.1007/s40615-020-00842-3 PubMedGoogle Scholar
11.
Getahun  D, Strickland  D, Lawrence  JM,  et al.  Racial and ethnic disparities in the trends in primary cesarean delivery based on indications.   Am J Obstet Gynecol. 2009;201(4):422.e1-422.e7. PubMedGoogle ScholarCrossref
12.
Main  EK, Cape  V, Abreo  A,  et al.  Reduction of severe maternal morbidity from hemorrhage using a state perinatal quality collaborative.   Am J Obstet Gynecol. 2017;216(3):298.e1-298.e11. PubMedGoogle ScholarCrossref
13.
Main  EK, Dhurjati  R, Cape  V,  et al.  Improving maternal safety at scale with the mentor model of collaborative improvement.   Jt Comm J Qual Patient Saf. 2018;44(5):250-259. PubMedGoogle Scholar
14.
Main  EK, Markow  C, Gould  J.  Addressing maternal mortality and morbidity in California through public-private partnerships.   Health Aff (Millwood). 2018;37(9):1484-1493. doi:10.1377/hlthaff.2018.0463PubMedGoogle ScholarCrossref
15.
Damschroder  LJ, Aron  DC, Keith  RE,  et al.  Fostering implementation of health services research findings into practice.   Implement Sci. 2009;4(1):50. PubMedGoogle ScholarCrossref
16.
Davidoff  F, Batalden  P.  Toward stronger evidence on quality improvement: draft publication guidelines.   Qual Saf Health Care. 2005;14(5):319-325. PubMedGoogle ScholarCrossref
17.
Agency for Healthcare Research and Quality. Module 10: academic detailing as a quality improvement tool. Accessed August 26, 2020. https://www.ahrq.gov/ncepcr/tools/pf-handbook/mod10.html
18.
Smith  H, Peterson  N, Lagrew  D, Main  E.  Toolkit to Support Vaginal Birth and Reduce Primary Cesareans: A Quality Improvement Toolkit. California Maternal Quality Care Collaborative; 2016.
19.
Alliance for Innovation on Maternal Health. Patient Safety Bundle: Safe Reduction of Primary Cesarean Births. Accessed August 26, 2020. https://safehealthcareforeverywoman.org/wp-content/uploads/Safe-Reduction-of-Primary-Cesarean-Bundle.pdf
20.
Chaillet  N, Dumont  A, Abrahamowicz  M,  et al.  A cluster-randomized trial to reduce cesarean delivery rates in Quebec.   N Engl J Med. 2015;372(18):1710-1721. PubMedGoogle ScholarCrossref
21.
Gude  WT, Brown  B, van der Veer  SN,  et al.  Clinical performance comparators in audit and feedback.   Implement Sci. 2019;14(1):39. PubMedGoogle ScholarCrossref
22.
Chaillet  N, Dumont  A.  Evidence-based strategies for reducing cesarean section rates.   Birth. 2007;34(1):53-64. PubMedGoogle ScholarCrossref
23.
Covered California. Attachment 7 to Covered California 2017 individual market QHP issuer contract: quality, network management, delivery system standards and improvement strategy. Accessed August 26, 2020. https://hbex.coveredca.com/insurance-companies/PDFs/Attachment-7_2020_Clean_Final-Model.pdf
24.
US Centers for Disease Control and Prevention; National Center for Health Statistics. Vital statistics online. Accessed January 16, 2021. https://www.cdc.gov/nchs/data_access/VitalStatsOnline.htm
25.
Danielsen  B. Probabilistic record linkages for generating a comprehensive epidemiological research file on maternal and infant health. Accessed August 26, 2020. http://www.health-info-solutions.com/Probabilistic%20Record%20Linkages%20may%202002.pdf
26.
American Academy of Pediatrics Committee on Fetus and Newborn.  Levels of neonatal care.   Pediatrics. 2012;130(3):587-597. doi:10.1542/peds.2012-1999PubMedGoogle ScholarCrossref
27.
Health Resources and Services Administration. List of rural counties and designated eligible census tracts in metropolitan counties. Accessed August 26, 2020. https://www.hrsa.gov/sites/default/files/ruralhealth/resources/forhpeligibleareas.pdf
28.
Joint Commission. Specifications manual for Joint Commission national quality measures (v2019A): PC-06 unexpected complications in term newborns. Accessed March 29, 2021. https://manual.jointcommission.org/releases/TJC2019A/MIF0393.html#:~:text=Severe%20Complications%3A&text=11.37%20Severe%20Hypoxia%2FAsphyxia,11.40%20Neonatal%20Severe%20Infection%5C
29.
Warton  EM, Parker  MM. Paper 25-2018: Oops, I D-I-D it again! advanced difference-in-differences models in SAS. Accessed March 23, 2021. https://www.lexjansen.com/cgi-bin/xsl_transform.php?x=wuss2018#wuss2018.025.st
30.
Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. PubMedGoogle ScholarCrossref
31.
Ahern  J, Hubbard  A, Galea  S.  Estimating the effects of potential public health interventions on population disease burden.   Am J Epidemiol. 2009;169(9):1140-1147. PubMedGoogle ScholarCrossref
32.
White VanGompel  E, Main  EK, Tancredi  D, Melnikow  J.  Do provider birth attitudes influence cesarean delivery rate.   BMC Pregnancy Childbirth. 2018;18(1):184. PubMedGoogle ScholarCrossref
33.
Sebastião  YV, Womack  L, Vamos  CA,  et al.  Hospital variation in cesarean delivery rates.   Am J Obstet Gynecol. 2016;214(1):123.e1-123.e18. PubMedGoogle ScholarCrossref
34.
Main  EK.  Reducing cesarean birth rates with data-driven quality improvement activities.   Pediatrics. 1999;103(1)(suppl E):374-383.PubMedGoogle Scholar
35.
Gilbert  WM, Bliss  MC, Johnson  A,  et al.  Improving recording accuracy, transparency, and performance for obstetric quality measures in a community hospital–based obstetrics department.   Jt Comm J Qual Patient Saf. 2013;39(6):258-266. PubMedGoogle Scholar
36.
Main  EK, Chang  S-C, Cape  V,  et al.  Safety assessment of a large-scale improvement collaborative to reduce nulliparous cesarean delivery rates.   Obstet Gynecol. 2019;133(4):613-623. PubMedGoogle ScholarCrossref
37.
Flamm  BL, Berwick  DM, Kabcenell  A.  Reducing cesarean section rates safely.   Birth. 1998;25(2):117-124. PubMedGoogle ScholarCrossref
38.
Nilsen  P, Birken  SA, eds.  Handbook on Implementation Science. Edward Elgar Publishing; 2020.
39.
Lazenbatt  AE, ed.  The Evaluation Handbook for Health Professionals. Routledge; 2002. doi:10.4324/9780203994498
40.
Landry  S, Collie-Akers  V, Foster  K,  et al.  Assessing the development of collective impact initiatives addressing maternal and child health.   Matern Child Health J. 2020;24(4):405-411. doi:10.1007/s10995-020-02894-7PubMedGoogle ScholarCrossref
Original Investigation
April 27, 2021

Hospital Quality Improvement Interventions, Statewide Policy Initiatives, and Rates of Cesarean Delivery for Nulliparous, Term, Singleton, Vertex Births in California

Author Affiliations
  • 1California Maternal Quality Care Collaborative, Stanford University, Stanford
  • 2Division of Maternal-Fetal Medicine, Department of Obstetrics, Gynecology, and Reproductive Sciences, University of California, San Francisco
  • 3Division of Neonatal and Developmental Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California
  • 4California Health Care Foundation, Oakland
  • 5Covered California, Sacramento
  • 6California Department of Health Care Services, Sacramento
  • 7California Public Employees’ Retirement System, Sacramento
  • 8Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Stanford University School of Medicine, Stanford, California
JAMA. 2021;325(16):1631-1639. doi:10.1001/jama.2021.3816
Key Points

Question  What were the rates of cesarean delivery for nulliparous, term, singleton, vertex (NTSV) births in California from 2014 to 2019 in the context of a mentor-led hospital collaborative intervention and statewide policy initiatives to reduce the rates of cesarean delivery?

Findings  In this observational study of 7 574 889 NTSV births that compared the rates of cesarean delivery between 2014 and 2019, the rates in California had a statistically significant decrease from 26.0% to 22.8% (relative risk, 0.88). The cesarean delivery rate for NTSV births in the US (excluding California) was 26.0% in both 2014 and 2019.

Meaning  Between 2014 and 2019, the rates of cesarean delivery for NTSV births in California decreased over time in the setting of a coordinated hospital-level collaborative and statewide initiatives designed to support vaginal birth.

Abstract

Importance  Safe reduction of the cesarean delivery rate is a national priority.

Objective  To evaluate the rates of cesarean delivery for nulliparous, term, singleton, vertex (NTSV) births in California in the context of a statewide multifaceted intervention designed to reduce the rates of cesarean delivery.

Design, Setting, and Participants  Observational study of cesarean delivery rates from 2014 to 2019 among 7 574 889 NTSV births in the US and at 238 nonmilitary hospitals providing maternity services in California. From 2016 to 2019, California Maternal Quality Care Collaborative partnered with Smart Care California to implement multiple approaches to decrease the rates of cesarean delivery. Hospitals with rates of cesarean delivery greater than 23.9% for NTSV births were invited to join 1 of 3 cohorts for an 18-month quality improvement collaborative between July 2016 and June 2019.

Exposures  Within the collaborative, multidisciplinary teams implemented multiple strategies supported by mentorship, shared learning, and rapid-cycle data feedback. Partnerships among nonprofit organizations, state governmental agencies, purchasers, and health plans addressed the external environment through transparency, award programs, and incentives.

Main Outcomes and Measures  The primary outcome was the change in cesarean delivery rates for NTSV births in California and a difference-in-differences analysis was performed to compare cesarean delivery rates for NTSV births in California vs the rates in the rest of the US. A mixed multivariable logistic regression model that adjusted for patient-level and hospital-level confounders also was used to assess the collaborative and the external statewide actions. The cesarean delivery rates for NTSV births at hospitals participating in the collaborative were compared with the rates from the nonparticipating hospitals and the rates in the participating hospitals prior to participation in the collaborative.

Results  A total of 7 574 889 NTSV births occurred in the US from 2014 to 2019, of which 914 283 were at 238 hospitals in California. All California hospitals were exposed to the statewide actions to reduce the rates of cesarean delivery, including the 149 hospitals that had baseline rates of cesarean delivery greater than 23.9% for NTSV births, of which 91 (61%) participated in the quality improvement collaborative. The rate of cesarean delivery for NTSV births in California decreased from 26.0% (95% CI, 25.8%-26.2%) in 2014 to 22.8% (95% CI, 22.6%-23.1%) in 2019 (relative risk, 0.88; 95% CI, 0.87-0.89). The rate of cesarean delivery for NTSV births in the US (excluding California births) was 26.0% in both 2014 and 2019 (relative risk, 1.00; 95% CI, 0.996-1.005). The difference-in-differences analysis revealed that the reduction in the rate of cesarean delivery for NTSV births in California was 3.2% (95% CI, 1.7%-3.5%) higher than in the US (excluding California). Compared with the hospitals and the periods not exposed to the collaborative activities, and after adjusting for patient characteristics and time using a modified stepped-wedge analysis, exposure to collaborative activities was associated with a lower odds of cesarean delivery for NTSV births (24.4% vs 24.6%; adjusted odds ratio, 0.87 [95% CI, 0.85-0.89]).

Conclusions and Relevance  In this observational study of NTSV births in California from 2014 to 2019, the rates of cesarean delivery decreased over time in the setting of the implementation of a coordinated hospital-level collaborative and statewide initiatives designed to support vaginal birth.

Introduction

High rates of cesarean delivery are a concern worldwide.1-4 The US Centers for Disease Control and Prevention set a target rate for cesarean delivery of 23.9% for low-risk first birth (also known as nulliparous, term, singleton, vertex [NTSV] birth) as part of their Healthy People 2020 goals; however, the cesarean delivery rate in the US was 25.9% in 2018, which was still above the target rate.5,6Quiz Ref ID There is marked variation in the overall rates of cesarean delivery (between 7%-70%) at US hospitals and in persistent racial disparities that suggest hospital policies, hospital unit culture, and individual clinician attitudes rather than patient factors alone drive the rates of cesarean delivery.7-11

California Maternal Quality Care Collaborative (CMQCC) is a statewide perinatal quality collaborative dedicated to improving maternity care; previous successful interventions have focused on reducing morbidity associated with hemorrhage and hypertension.12-14 In 2016, partnering with the California Health Care Foundation, CMQCC launched a multifaceted, multilevel initiative to reduce the cesarean delivery rate for NTSV births in California. Guided by the Consolidated Framework for Implementation Research,15 efforts were aimed at both the internal (hospital level) and the external (statewide) environment. Hospitals with rates of cesarean delivery for NTSV births greater than the national target (23.9%) were invited to join a quality improvement collaborative and mentorship program to offer support for hospital-level initiatives. Strategic partnerships with Smart Care California, the California Health and Human Services Agency, the California Health Care Foundation, and individual health plans and purchasers enabled statewide efforts to encourage vaginal birth.

The objective of this study was to evaluate the rates of cesarean delivery for NTSV births in California in the context of a statewide multifaceted initiative designed to reduce the rates of cesarean deliveries.

Methods
Overview of Study Design

Institutional review board approval was obtained from Stanford University as the study host and from the California Committee for the Protection of Human Subjects for the use of the state data sets, which waived the requirement for individual informed consent. The Standards for Quality Improvement Reporting Excellence (version 2.0) guidelines were followed during the development of this study.16

This was an observational study of a multifaceted quality improvement initiative designed to decrease the cesarean delivery rates for NTSV births in California. In 2016, a mentor-led collaborative to support hospital quality improvement efforts was started along with the simultaneous development of statewide interventions. The overall initiative was evaluated by comparing the cesarean delivery rates in California with the cesarean delivery rates in the US between 2014 and 2019. The specific components of the intervention were evaluated by examining patient-level data for all deliveries that occurred between January 2015 and June 2019 in California.

Overview of Collaborative Activities

All California hospitals with 2015 cesarean delivery rates greater than 23.9% for NTSV births were invited to join the CMQCC Supporting Vaginal Birth Collaborative at no cost. Although all eligible hospitals were invited at the outset, the collaborative was designed to have multiple cohorts to accommodate finite coordinating resources and varying levels of hospital interest and availability. Hospitals that declined to participate during the first cohort round were reinvited during subsequent study rounds. Round 1 was launched in June 2016, round 2 was launched in January 2017, and round 3 was launched in January 2018 (Figure 1). The collaborative followed a previously described modified Institute for Healthcare Improvement Breakthrough Series model with added mentorship.13 This approach brought together a multidisciplinary team (physicians, nurses, and quality improvement professionals) from 6 to 8 hospitals supported by a physician and nurse mentor pair.

The mentorship groups met monthly for virtual collaborative learning to share their efforts, challenges, and progress over an 18-month period in addition to regional in-person all-day kickoff and closing meetings. Mentors conducted site visits featuring a grand rounds presentation and individualized education and support based on the Agency for Healthcare Research and Quality recommended strategy of academic detailing (ie, peer-to-peer educational outreach).17 The 2 key support materials for improvement activities were the CMQCC Toolkit to Support Vaginal Birth and Reduce Primary Cesareans18 (a 159-page open-source compendium of evidence-based tools, algorithms, and guidelines) and the Alliance for Innovation on Maternal Health National Patient Safety Bundle: Safe Reduction of Primary Cesarean Births.19

Quiz Ref IDAn important component of the collaborative was the use of the CMQCC Maternal Data Center (a web-based system that generates near real-time performance metrics from which trends for cesarean delivery rates can be benchmarked and analyzed). The CMQCC Maternal Data Center system allows hospitals to compare their cesarean delivery rates with other similar hospitals across the state as a type of “audit and feedback” intervention, which is an implementation science strategy using clinical performance comparators previously identified to be important for cesarean delivery improvement projects.20,21 Another key strategy of the collaborative was a formal analysis of barriers and facilitators to allow for application of tailored interventions via a readiness survey and analyses using the CMQCC Maternal Data Center system to help determine specific drivers for each hospital’s cesarean delivery rate.22 Hospitals were encouraged to select approaches from the CMQCC Toolkit to Support Vaginal Birth and Reduce Primary Cesareans that were most relevant to their particular setting and to record and track structure and process measures from the Alliance for Innovation on Maternal Health National Patient Safety Bundle: Safe Reduction of Primary Cesarean Birth.19

Overview of Statewide Activities

During this period, Smart Care California, a public-private purchaser partnership, together with CMQCC and the California Health Care Foundation, coordinated statewide policy and public awareness efforts. Starting in October 2016, the California Health and Human Services Agency released annual honor rolls highlighting hospitals that had cesarean delivery rates of 23.9% or less for NTSV births.

Beginning in June 2017, the cesarean delivery rate for NTSV births at every California hospital was made publicly available at http://www.CalHospitalCompare.org and on each hospital’s Yelp.com landing page. Through 2016 and 2017, multiple health plans encouraged or incentivized participation in the CMQCC collaborative. A patient education website (MyBirthMatters.org) was launched in early 2018.

From 2016 through 2019, the California Department of Health Care Services used the US Centers for Medicare & Medicaid Services §1115 waiver process to incentivize public hospitals to address overuse of cesarean delivery. Beginning in 2016, Covered California (the state health insurance exchange) encouraged health plans to manage performance variation, including the potential to exclude hospitals that demonstrated persistently elevated rates of cesarean delivery.23

Analysis of Collaborative Activities

Data on hospital collaborative activities were compiled from 3 sources: transcripts of monthly conference calls among each mentor group, structure measures entered into the CMQCC Maternal Data Center system, and summary documents highlighting the best practices of each hospital. If an activity was mentioned in any of these sources, it was considered as having occurred. Hospital interventions were categorized into groups that were defined by the CMQCC Toolkit to Support Vaginal Birth and Reduce Primary Cesareans18 and frequencies were described.

Creation of Data Sets

The US rates of cesarean delivery for NTSV births from 2014 through 2019 were obtained from the National Center for Health Statistics.24 The CMQCC Maternal Data Center system received monthly birth certificate data from the California Department of Public Health Center for Health Statistics and Informatics and semiannual maternal and neonatal hospital discharge diagnosis files (containing diagnosis, procedure, and disposition codes) from the California Office of Statewide Health Planning and Development. These files were linked using a previously validated probabilistic algorithm with linkage rates exceeding 98%.25 The California Office of Statewide Health Planning and Development patient-level data for 2019 were not available for analysis. The 2019 discharge diagnosis files provided directly from the hospitals to the CMQCC Maternal Data Center system were used instead. These data represented 96% of California births during that period.

Teaching hospital status, hospital geographic region, and hospital ownership were obtained from the California Office of Statewide Health Planning and Development. The level of neonatal care was determined and reported by the nursery directors to the CMQCC Maternal Data Center system based on the 2012 definition from the American Academy of Pediatrics.26 Hospitals in rural areas were identified using definitions provided by the Health Resources and Services Administration.27

The specifications for PC-02 (NTSV) from the Joint Commission were used to classify NTSV births. Gestational age (between 37 and 44 weeks) and parity were obtained from the birth certificate; plurality and presentation status were obtained using maternal discharge diagnosis codes. To control for potential confounders over time, the following demographic and clinical data were collected: prepregnancy body mass index (calculated as weight in kilograms divided by height in meters squared), education level, use of prenatal care, and self-reported race/ethnicity (selected from fixed categories and obtained from the birth certificate).

Insurance status was obtained from the birth certificate only if the discharge record was incomplete. The maternal comorbidities of preeclampsia, chronic hypertension, diabetes, and gestational diabetes were identified using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) or International Statistical Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis and procedure codes from the discharge data.

In addition, severe unexpected newborn complications (code 0716 [National Quality Forum] and code PC-06 [The Joint Commission]) were examined over the same period as a balancing measure. The term severe unexpected newborn complications includes hypoxic ischemic encephalopathy, seizures, requirement of ventilation, neonatal sepsis, birth injury, and hospital transfer to a higher level of care among term neonates without preexisting conditions.28 These data also were obtained using ICD-9-CM or ICD-10-CM codes from the neonatal discharge data. During the baseline years (2014-2015), severe unexpected newborn complications were collected using ICD-9-CM codes and during subsequent years using ICD-10-CM codes, which provide a slightly lower rate, so formal statistical testing was not performed on these data due to these limitations.

Outcomes

The primary outcome was the change in cesarean delivery rates for NTSV births in California compared with the cesarean delivery rates in the rest of the US during the same period. Secondary outcomes included the association of cesarean delivery for NTSV birth with the collaborative activities involving participating hospitals as well as the statewide interventions that affected all hospitals.

Statistical Analyses

To assess statewide trends, annual cesarean delivery rates from all NTSV births in California from 2014 through 2019 were calculated. The cesarean delivery rates for NTSV births in California were compared with the US cesarean delivery rates for NTSV births (excluding California births) and were calculated annually by the National Center for Health Statistics.6 Risk difference, relative risk (RR), and their 95% CIs comparing cesarean delivery rates for 2019 with 2014 were estimated. In addition, to compare cesarean delivery rates for California vs a control group unaffected by statewide activities, we used a difference-in-differences approach to evaluate whether the risk difference in California and in the US (excluding California births) between 2014 and 2019 was statistically different.29

The risk difference model used included a time variable (2014 vs 2019), an indicator for the population (California vs the US [excluding California]), and their interaction term. The coefficient for the interaction term then represents the magnitude of changes in the risk of cesarean delivery for 2014 to 2019 between the 2 groups. Patient-level and hospital-level data for US births were not available so these analyses were unadjusted.

A patient-level analysis using NTSV births from all maternity hospitals in California between January 2015 and June 2019 was then performed to examine associations more closely with the collaborative activities and the statewide actions. To accommodate secular trends and the fact that rounds 1, 2, and 3 started at different times, the analysis was designed to be similar to a stepped-wedge design.30

The cesarean delivery rates were analyzed in 6-month time blocks and each period was identified as either an intervention period (time when hospitals were participating in the collaborative), a sustainment period (time following participation in the collaborative), or a control period (included baseline time for participating hospitals and all times for hospitals that did not participate in the collaborative). The cesarean delivery rates for NTSV births at hospitals participating in the collaborative were compared with rates from nonparticipating hospitals as well as vs the rates from participating hospitals prior to participation in the collaborative using a modified stepped-wedge analysis (eFigure 1 in the Supplement).

A multivariable patient-level mixed logistic regression model was created to calculate the adjusted odds of cesarean delivery during the combined intervention and sustainment phases compared with the baseline period while controlling for period (to control for secular trends, namely the statewide activities) and other confounders (including delivery hospital as a random intercept to account for clustering by hospital). Potential confounders included patient-level characteristics, hospital-level characteristics, and whether the hospital started with a cesarean delivery rate greater than 23.9% for NTSV births. A subgroup analysis was performed that excluded hospitals that were ineligible for participation in the collaborative because of baseline cesarean delivery rates of 23.9% or less for NTSV births.

All hypothesis testing was 2-sided with a prespecified significance threshold of P < .05. SAS version 9.4 (SAS Institute Inc) was used for the statistical analysis. Missing data were rare and cases with missing data were excluded from the patient-level analysis but included in the statewide cesarean delivery rate comparisons with the US.

Results

From 2014 to 2019, there were 7 574 889 NTSV births in the US and 914 283 NTSV births in California and these data were used for the US vs California comparison. For the patient-level analysis, there were 679 086 NTSV births in California that occurred between January 2015 and June 2019.

Participation in the Collaborative

There were 238 nonmilitary hospitals in California providing maternity services between January 2015 and June 2019. Of the 149 hospitals with baseline cesarean delivery rates greater than 23.9% for NTSV births, 91 participated in the collaborative (24 hospitals during round 1, 42 during round 2, and 25 during round 3). Of the remaining 147 hospitals, 89 had cesarean delivery rates of 23.9% or less for NTSV births and were not eligible to participate in the collaborative and 58 were eligible but declined to participate. All 238 hospitals were exposed to statewide actions regardless of their baseline cesarean delivery rate for NTSV births and regardless of their participation in the quality collaborative (Figure 1).

Although largely representative of the entire state, hospitals participating in the collaborative were less likely to be an academic center, have fewer than 1000 births per year, and be located in a rural area and more likely to be located in southern California (a region with higher cesarean delivery rates; Table 1). Women who delivered at a hospital participating in the collaborative in 2015 were more likely to be older than aged 35 years, have a college degree, and have private insurance compared with women who delivered at a hospital not participating (eligible and ineligible) in the collaborative (Table 1 and eTable 1 in the Supplement).

Analysis of Cesarean Delivery Rate

In California, the rate of cesarean delivery for NTSV births decreased from 26.0% (95% CI, 25.8% to 26.2%) in 2014 to 22.8% (95% CI, 22.6% to 23.1%) in 2019 with a statistically significant RR of 0.88 (95% CI, 0.87 to 0.89) and an absolute risk difference of 3.2% (95% CI, 2.9% to 3.5%). During this period, the rate of cesarean delivery for NTSV births in the US (excluding California births) did not change (26.0% in 2014 and in 2019; RR, 1.00 [95% CI, 0.996 to 1.005]; risk difference, 0.01% [95% CI −0.01% to 0.10%). A difference-in-differences analysis revealed that the reduction in the rate of cesarean delivery for NTSV births in California was 3.2% (95% CI, 1.7% to 3.5%; P < .001) higher than in the US (excluding California; Figure 2).

Analysis of the Collaborative and Statewide Activities

Among women delivering at hospitals participating in the collaborative, the rate of cesarean delivery for NTSV births decreased from 28.6% (95% CI, 28.2% to 29.1%; n = 9858/34 437) for January-June 2015 to 24.2% (95% CI, 23.7% to 24.7%; n = 7439/30 728) for January-June 2019 and the absolute difference was 4.4% (95% CI, 3.7% to 5.1%; P < .001) (RR, 0.85 [95% CI, 0.82 to 0.87]; P < .001). Examining only the eligible hospitals (had baseline cesarean delivery rates >23.9% for NTSV births) that did not participate in the collaborative, the rate of cesarean delivery for NTSV births changed from 27.0% (95% CI, 26.3% to 27.7%; n = 4291/15 910) for January-June 2015 to 24.3% (95% CI, 23.6% to 25.1%; n = 2973/12 217) for January-June 2019 and the absolute difference was 2.7% (95% CI, 1.6% to 3.7%; P < .001) (RR, 0.90 [95% CI, 0.87 to 0.94]; P < .001).

Of the 89 hospitals that had a cesarean delivery rate for NTSV births of 23.9% or less (which was the target rate and these hospitals were ineligible to participate in the collaborative), the cesarean delivery rate was 20.5% (95% CI, 20.0% to 21.0%; n = 5284/25 767) for January-June 2015 and 20.8% (95% CI, 20.3% to 21.3%; n = 5046/24 281) for January-June 2019 and the absolute difference was 0.3% (95% CI −0.4% to 1.0%; P = .45) (RR, 1.01 [95% CI, 0.98 to 1.05]; P = .45). Considering all 147 hospitals not participating in the collaborative (89 ineligible and 58 eligible), the rate of cesarean delivery for NTSV births changed from 23.0% (95% CI, 22.6% to 23.4%; n = 9575/41 677) for January-June 2015 to 22.0% (95% CI, 21.6% to 22.4%; n = 8019/36 498) for January-June 2019 and the absolute difference was 1.0% (95% CI, 0.4% to 1.6%; P < .001) (RR, 0.96 [95% CI, 0.93 to 0.98]; P < .001) (eFigures 1-2 in the Supplement).

In the multivariable analysis (after adjusting for 6-month time blocks and other potential confounders in a modified stepped-wedge analysis), the collaborative activities were associated with decreased odds of cesarean delivery for NTSV births (adjusted odds ratio, 0.87 [95% CI, 0.85-0.89]). In the same analysis, the statewide initiatives (as represented by the time variable) also were associated with lower odds of cesarean delivery regardless of whether a hospital participated in the collaborative (eTable 2 in the Supplement). In the multivariable analysis excluding the hospitals that were ineligible (due to baseline cesarean delivery rates ≤23.9%), the collaborative activities were associated with decreased odds of cesarean delivery for NTSV births (adjusted odds ratio, 0.91 [95% CI, 0.89-0.94]). Combining all the rates used in the analysis (eFigure 1 in the Supplement), the mean rate of cesarean delivery for NTSV births was 24.6% (95% CI, 24.4%-24.8%) for the baseline group (included hospitals that started above or below the target delivery rate of 23.9% and decreased over time) and was 24.4% (95% CI, 24.2%-24.5%) for the intervention or sustainment group (included rates that decreased over time).

Collaborative Activities

During the course of the collaborative, participating hospitals implemented a median of 7 distinct interventions (interquartile range, 5-9) within a median of 5 intervention categories (interquartile range, 4-6). The most frequently used intervention was clinician education (99%), which included both physician and nurse education (Table 2). The second and third most common interventions were enhanced support for women in labor and standardization of labor management (90% [82 of 91 hospitals] used all 3 of these intervention categories). Another widely used intervention strategy (occurring in 85% of hospitals participating in the collaborative) was the sharing of unblinded physician-level cesarean delivery rates for NTSV births, allowing physicians to see their own rates and compare themselves with their peers.

Severe Unexpected Newborn Complications

The statewide rate of severe unexpected newborn complications decreased from 2.1% to 1.5% between January 2015 and June 2019. Among hospitals participating in the collaborative, the rate of severe unexpected newborn complications decreased from 2.1% in January 2015 to 1.4% in January-June 2019. Among hospitals not participating in the collaborative, the rate of severe unexpected newborn complications decreased from 1.9% in January 2015 to 1.4% in January-June 2019. Due to the fact that the coding changed from using the ICD-9-CM to the ICD-10-CM during this period, the rates cannot be compared exactly, and statistical testing was not performed.

Discussion

Quiz Ref IDIn the current study, a collaborative approach to encourage a variety of quality improvement approaches within hospitals that had high baseline rates of cesarean delivery for NTSV births was combined with an approach aimed to influence the external environment through a series of statewide policy and transparency efforts. Together, this multifaceted approach was associated with a reduction in the rate of cesarean delivery for NTSV births in California from 26.0% in 2014 to 22.8% in 2019, whereas the rest of the US had no change in the rate of cesarean delivery. These findings suggest that hospital and clinician policies, attitudes, and practices (rather than individual patient characteristics) are related to hospital rates of cesarean delivery.7,31,32

Quiz Ref IDWomen who delivered at hospitals during or after participation by the hospitals in the collaborative had lower odds of cesarean delivery compared with women who delivered at eligible hospitals that declined to participate in the collaborative. An important feature of the collaborative was its flexibility along with access to rapid-cycle data in a user-friendly format and a mentor-led shared learning model with multidisciplinary teams to identify barriers and select evidence-based interventions based on local need. This peer support system enhanced the strategies provided in the widely available CMQCC Toolkit to Support Vaginal Birth and Reduce Primary Cesareans and was likely one of the reasons why joining the collaborative was associated with further reductions in cesarean delivery compared with the periods before joining.

The 3 intervention categories most commonly used were clinician and nursing education on labor management, attention to improving support for women in labor, and implementing guidelines or other strategies to improve labor management. Most hospitals implemented at least 1 strategy within these 3 intervention categories. Three-quarters of the hospitals participating in the collaborative also shared unblinded physician-specific cesarean delivery rates, suggesting this strategy of transparency was a powerful adjunct to the other intervention categories, as has been shown in other studies.33,34

More than one-third of California hospitals began the study period with cesarean delivery rates below the target (≤23.9%) and maintained a low rate, indicating that lower rates of cesarean delivery are feasible and sustainable. Previously, the safety of lowering the cesarean delivery rate for NTSV births for both the mother and the neonate was demonstrated using data from the first 2 cohorts in the collaborative.35 The current study included the entire state of California and was unable to demonstrate any increase in the rates of severe neonatal complications statewide, nor in the subgroups of hospitals that did vs did not participate in the collaborative.

The strengths of this study include the expansion of an intervention to reduce cesarean delivery rates for NTSV births to a larger scale than has previously been reported to our knowledge, demonstrating results in both small and large community hospitals. The US cesarean delivery rate has risen steadily during the last 20 years despite efforts from major professional societies and public health authorities.3,4,36 Notwithstanding the success found in some smaller studies, reducing the cesarean delivery rate at scale has been elusive.20,37

Quiz Ref IDThe change in rates of cesarean delivery for NTSV births observed in association with this initiative may have been related to the added focus on the external environment, which supported the collaborative activities and identified reducing unnecessary cesarean deliveries as a clear quality and safety priority for the entire state. This likely eased the difficult task of changing physician behavior by mobilizing health plans, hospital executives, and patients in a shared goal to support vaginal birth. Implementation science principles emphasize the importance of including the external context into the design of any intervention, particularly one that attempts large-scale change.38

Limitations

The study had several limitations. First, the study was observational rather than randomized, and selection bias and unmeasured confounding could have affected the results. Hospitals that chose to participate in the collaborative may have been more motivated to make changes to lower their cesarean delivery rates, and this motivation, in addition to the support offered by the collaborative, likely did play a part in the ability of these hospitals to make changes.

Second, it was difficult to identify which facets of the mentor-led collaborative structure were the most useful to individuals at the participating hospitals. Third, the analyses comparing the changes in the cesarean delivery rates for the US and California were unadjusted due to lack of availability of national patient-level and hospital-level covariates.

Fourth, even though the odds of cesarean delivery associated with the collaborative activities and the statewide initiatives were quantified, it is highly likely that leakage (the informal uptake of interventions by the control group) was present as is often seen in community-based interventions.39 The CMQCC Toolkit to Support Vaginal Birth and Reduce Primary Cesareans was widely available to hospitals not participating in the collaborative (there have been >7000 downloads of the Toolkit), and some individual hospitals and hospital systems used this document to develop their own improvement initiatives independent from the collaborative. In addition, the statewide external actions reinforced participation in the collaborative and may have contributed to their success. However, even though leakage may be a limitation of the analysis, the combination of the collaborative interventions and the external statewide actions was a strength in the reach and sustainability overall. Further research is needed to determine whether these interventions were associated with changes in cesarean delivery rates equally among different racial/ethnic groups.

Fifth, this was conducted in a single state with a well-established perinatal quality collaborative, which raises generalizability concerns. However, even though not all states have access to the CMQCC Maternal Data Center system, the emphasis on building partnerships with multiple state agencies, private organizations, and individual mentors and clinical leaders from the community would seem to be achievable in other geographical areas. A model that focuses on collaboration, coalition development, and strategic planning, together leading to a “collective impact” should be considered in future large-scale efforts to reduce cesarean delivery rates.40

Conclusions

In this observational study of NTSV births in California from 2014 to 2019, the rates of cesarean delivery decreased over time in the setting of the implementation of a coordinated hospital-level collaborative and statewide initiatives designed to support vaginal birth.

Back to top
Article Information

Corresponding Author: Melissa G. Rosenstein, MD, MAS, California Maternal Quality Care Collaborative, Stanford Medical School Office Building, 1265 Welch Road, MS 5415, Stanford, CA 94305 (melissa.rosenstein@ucsf.edu).

Accepted for Publication: February 27, 2021.

Author Contributions: Drs Rosenstein and Main had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Rosenstein, Markow, Teleki, Lang, Logan, Main.

Acquisition, analysis, or interpretation of data: Rosenstein, Chang, Sakowski, Cape, Main.

Drafting of the manuscript: Rosenstein, Markow, Cape, Main.

Critical revision of the manuscript for important intellectual content: Rosenstein, Chang, Sakowski, Teleki, Lang, Logan, Main.

Statistical analysis: Chang, Main.

Obtained funding: Markow, Teleki, Main.

Administrative, technical, or material support: Sakowski, Markow, Teleki, Lang, Logan, Cape, Main.

Supervision: Main.

Conflict of Interest Disclosures: Dr Teleki reported working for the California Health Care Foundation, which funded a substantial portion of the cesarean delivery work that was undertaken in California and described in this article. No other disclosures were reported.

Funding/Support: Funding for the development of the California Maternal Quality Care Collaborative (CMQCC) Toolkit to Support Vaginal Birth and Reduce Primary Cesareans was provided by the California Health Care Foundation. Labor support education materials were supported by the Yellow Chair Foundation. The core administrative support for CMQCC was provided by the Maternal Child and Adolescent Division of the California Department of Public Health.

Role of the Funder/Sponsor: Dr Teleki is the director of Learning and Impact for the California Health Care Foundation and was involved in the preparation, review, and approval of the manuscript. The funders were involved in the design of the interventions but were not involved in the design and conduct of the study, nor the collection, management, analysis, and interpretation of the data, nor in the decision to submit the manuscript for publication or where it should be sent.

Additional Contributions: California Maternal Quality Care Collaborative (CMQCC) thanks the following mentors and clinical leaders for providing paid leadership and support (compensated by CMQCC) to their hospital teams and making this study possible: Paola Aghajanian, MD (Cedars Sinai Medical Center); Robert Altman, MD, MBA (Sutter Health); Katy Berkowitz, MD (Cleveland Clinic); Jennifer Butler, MD (University of California, Irvine); Kenneth Chan, MD (Miller Children’s and Women’s Hospital); Susan Crowe, MD (Stanford University School of Medicine); Terri Deeds, RN, MSN (Saddleback Memorial Medical Center); Sharon Dey-Layne, CNS, MSN (Riverside University Health System); Ty Gopal, MD (Sutter Health); Kimberly Gregory, MD (Cedars Sinai Medical Center); Valerie Huwe, RNC, MS, CNS (University of California, San Francisco); David Lagrew, MD (St Joseph Health); Timothy Leach, MD (John Muir Health); Katie Lydon, MSN, RN (NorthBay Healthcare); Alina Miller, MHI, BSN, RN (Hoag Orthopedic Institute); Marlin Mills, MD (Hoag Medical Center); Sunnie Mitchell, RN (Providence Health); Jennifer McNulty, MD (Miller Children’s and Women’s Hospital); Karen Perdion, CNM, MSN (University of California, San Diego); Jenna Quarn-Ogborn, RN (Miller Women’s and Children’s Hospital); Nancy Peterson, MSN, PNNP (Dominican Hospital); Hellen Rodriguez, MD (Pomona Valley Medical Center); Cynthia Sawyer, MSN, CNS (PIH Health Whittier); Emily Seet, MD (Long Beath Memorial Medical Center); Holly Smith, MPH, MSN, CNM (CMQCC); Pat Spier, RNC, BSN (Perinatal Advisory Council for Leadership, Advocacy, and Consultation); Beth Stephens-Hennessy, MSN, CNS (Sutter Medical Center); Maryam Tarsa, MD (University of California, San Diego); Janet Trial, EdD, CNM, MSN (Keck School of Medicine, University of Southern California); Julie Vasher, DNP, CNS (Salinas Valley Medical Center); and Jamie Vincent, MSN, APRN-CNS (John Muir Health).

References
1.
Main  EK, Morton  CH, Melsop  K,  et al.  Creating a public agenda for maternity safety and quality in cesarean delivery.   Obstet Gynecol. 2012;120(5):1194-1198. PubMedGoogle ScholarCrossref
2.
Betran  AP, Torloni  MR, Zhang  JJ,  et al.  WHO statement on caesarean section rates.   BJOG. 2016;123(5):667-670. PubMedGoogle ScholarCrossref
3.
Office of Disease Prevention and Health Promotion. Healthy People 2020: topics and objectives, maternal, infant, and child health. Accessed August 26, 2020. https://www.healthypeople.gov/2020/topics-objectives/topic/maternal-infant-and-child-health/objectives
4.
Spong  CY, Berghella  V, Wenstrom  KD,  et al.  Preventing the first cesarean delivery.   Obstet Gynecol. 2012;120(5):1181-1193. PubMedGoogle ScholarCrossref
5.
National Center for Health Statistics. Chapter 26: Maternal, infant, and child health. In:  Healthy People 2020 Midcourse Review. National Center for Health Statistics; 2016.
6.
Martin  JA, Hamilton  BE, Osterman  MJK, Driscoll  AK.  Births: final data for 2018.   Natl Vital Stat Rep. 2019;68(13):1-47. PubMedGoogle Scholar
7.
Cáceres  IA, Arcaya  M, Declercq  E,  et al.  Hospital differences in cesarean deliveries in Massachusetts (US) 2004-2006.   PLoS One. 2013;8(3):e57817. PubMedGoogle Scholar
8.
Kozhimannil  KB, Law  MR, Virnig  BA.  Cesarean delivery rates vary tenfold among US hospitals.   Health Aff (Millwood). 2013;32(3):527-535. PubMedGoogle ScholarCrossref
9.
White VanGompel  E, Perez  S, Datta  A,  et al.  Cesarean overuse and the culture of care.   Health Serv Res. 2019;54(2):417-424. PubMedGoogle ScholarCrossref
10.
Valdes  EG.  Examining cesarean delivery rates by race.   J Racial Ethn Health Disparities. Published online August 17, 2020. doi:10.1007/s40615-020-00842-3 PubMedGoogle Scholar
11.
Getahun  D, Strickland  D, Lawrence  JM,  et al.  Racial and ethnic disparities in the trends in primary cesarean delivery based on indications.   Am J Obstet Gynecol. 2009;201(4):422.e1-422.e7. PubMedGoogle ScholarCrossref
12.
Main  EK, Cape  V, Abreo  A,  et al.  Reduction of severe maternal morbidity from hemorrhage using a state perinatal quality collaborative.   Am J Obstet Gynecol. 2017;216(3):298.e1-298.e11. PubMedGoogle ScholarCrossref
13.
Main  EK, Dhurjati  R, Cape  V,  et al.  Improving maternal safety at scale with the mentor model of collaborative improvement.   Jt Comm J Qual Patient Saf. 2018;44(5):250-259. PubMedGoogle Scholar
14.
Main  EK, Markow  C, Gould  J.  Addressing maternal mortality and morbidity in California through public-private partnerships.   Health Aff (Millwood). 2018;37(9):1484-1493. doi:10.1377/hlthaff.2018.0463PubMedGoogle ScholarCrossref
15.
Damschroder  LJ, Aron  DC, Keith  RE,  et al.  Fostering implementation of health services research findings into practice.   Implement Sci. 2009;4(1):50. PubMedGoogle ScholarCrossref
16.
Davidoff  F, Batalden  P.  Toward stronger evidence on quality improvement: draft publication guidelines.   Qual Saf Health Care. 2005;14(5):319-325. PubMedGoogle ScholarCrossref
17.
Agency for Healthcare Research and Quality. Module 10: academic detailing as a quality improvement tool. Accessed August 26, 2020. https://www.ahrq.gov/ncepcr/tools/pf-handbook/mod10.html
18.
Smith  H, Peterson  N, Lagrew  D, Main  E.  Toolkit to Support Vaginal Birth and Reduce Primary Cesareans: A Quality Improvement Toolkit. California Maternal Quality Care Collaborative; 2016.
19.
Alliance for Innovation on Maternal Health. Patient Safety Bundle: Safe Reduction of Primary Cesarean Births. Accessed August 26, 2020. https://safehealthcareforeverywoman.org/wp-content/uploads/Safe-Reduction-of-Primary-Cesarean-Bundle.pdf
20.
Chaillet  N, Dumont  A, Abrahamowicz  M,  et al.  A cluster-randomized trial to reduce cesarean delivery rates in Quebec.   N Engl J Med. 2015;372(18):1710-1721. PubMedGoogle ScholarCrossref
21.
Gude  WT, Brown  B, van der Veer  SN,  et al.  Clinical performance comparators in audit and feedback.   Implement Sci. 2019;14(1):39. PubMedGoogle ScholarCrossref
22.
Chaillet  N, Dumont  A.  Evidence-based strategies for reducing cesarean section rates.   Birth. 2007;34(1):53-64. PubMedGoogle ScholarCrossref
23.
Covered California. Attachment 7 to Covered California 2017 individual market QHP issuer contract: quality, network management, delivery system standards and improvement strategy. Accessed August 26, 2020. https://hbex.coveredca.com/insurance-companies/PDFs/Attachment-7_2020_Clean_Final-Model.pdf
24.
US Centers for Disease Control and Prevention; National Center for Health Statistics. Vital statistics online. Accessed January 16, 2021. https://www.cdc.gov/nchs/data_access/VitalStatsOnline.htm
25.
Danielsen  B. Probabilistic record linkages for generating a comprehensive epidemiological research file on maternal and infant health. Accessed August 26, 2020. http://www.health-info-solutions.com/Probabilistic%20Record%20Linkages%20may%202002.pdf
26.
American Academy of Pediatrics Committee on Fetus and Newborn.  Levels of neonatal care.   Pediatrics. 2012;130(3):587-597. doi:10.1542/peds.2012-1999PubMedGoogle ScholarCrossref
27.
Health Resources and Services Administration. List of rural counties and designated eligible census tracts in metropolitan counties. Accessed August 26, 2020. https://www.hrsa.gov/sites/default/files/ruralhealth/resources/forhpeligibleareas.pdf
28.
Joint Commission. Specifications manual for Joint Commission national quality measures (v2019A): PC-06 unexpected complications in term newborns. Accessed March 29, 2021. https://manual.jointcommission.org/releases/TJC2019A/MIF0393.html#:~:text=Severe%20Complications%3A&text=11.37%20Severe%20Hypoxia%2FAsphyxia,11.40%20Neonatal%20Severe%20Infection%5C
29.
Warton  EM, Parker  MM. Paper 25-2018: Oops, I D-I-D it again! advanced difference-in-differences models in SAS. Accessed March 23, 2021. https://www.lexjansen.com/cgi-bin/xsl_transform.php?x=wuss2018#wuss2018.025.st
30.
Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. PubMedGoogle ScholarCrossref
31.
Ahern  J, Hubbard  A, Galea  S.  Estimating the effects of potential public health interventions on population disease burden.   Am J Epidemiol. 2009;169(9):1140-1147. PubMedGoogle ScholarCrossref
32.
White VanGompel  E, Main  EK, Tancredi  D, Melnikow  J.  Do provider birth attitudes influence cesarean delivery rate.   BMC Pregnancy Childbirth. 2018;18(1):184. PubMedGoogle ScholarCrossref
33.
Sebastião  YV, Womack  L, Vamos  CA,  et al.  Hospital variation in cesarean delivery rates.   Am J Obstet Gynecol. 2016;214(1):123.e1-123.e18. PubMedGoogle ScholarCrossref
34.
Main  EK.  Reducing cesarean birth rates with data-driven quality improvement activities.   Pediatrics. 1999;103(1)(suppl E):374-383.PubMedGoogle Scholar
35.
Gilbert  WM, Bliss  MC, Johnson  A,  et al.  Improving recording accuracy, transparency, and performance for obstetric quality measures in a community hospital–based obstetrics department.   Jt Comm J Qual Patient Saf. 2013;39(6):258-266. PubMedGoogle Scholar
36.
Main  EK, Chang  S-C, Cape  V,  et al.  Safety assessment of a large-scale improvement collaborative to reduce nulliparous cesarean delivery rates.   Obstet Gynecol. 2019;133(4):613-623. PubMedGoogle ScholarCrossref
37.
Flamm  BL, Berwick  DM, Kabcenell  A.  Reducing cesarean section rates safely.   Birth. 1998;25(2):117-124. PubMedGoogle ScholarCrossref
38.
Nilsen  P, Birken  SA, eds.  Handbook on Implementation Science. Edward Elgar Publishing; 2020.
39.
Lazenbatt  AE, ed.  The Evaluation Handbook for Health Professionals. Routledge; 2002. doi:10.4324/9780203994498
40.
Landry  S, Collie-Akers  V, Foster  K,  et al.  Assessing the development of collective impact initiatives addressing maternal and child health.   Matern Child Health J. 2020;24(4):405-411. doi:10.1007/s10995-020-02894-7PubMedGoogle ScholarCrossref
×