[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure.
Risk-Standardized Severe Maternal and Neonatal Morbidity by Quality Indicators
Risk-Standardized Severe Maternal and Neonatal Morbidity by Quality Indicators

aN = 41 hospitals. Top 10 on the quality measures indicates lowest rate for elective and cesarean deliveries; bottom 10, highest rates.

Table 1.  
Sociodemographic and Clinical Characteristics for Deliveries by Presence of Severe Maternal Morbidity
Sociodemographic and Clinical Characteristics for Deliveries by Presence of Severe Maternal Morbidity
Table 2.  
Sociodemographic and Clinical Characteristics for Deliveries by Presence of Moderate and Severe Neonatal Term Morbidity Among Term Newborns Without Anomaliesa
Sociodemographic and Clinical Characteristics for Deliveries by Presence of Moderate and Severe Neonatal Term Morbidity Among Term Newborns Without Anomaliesa
Table 3.  
New York City Delivery Hospital Characteristics and Performance Indicators (n = 41)
New York City Delivery Hospital Characteristics and Performance Indicators (n = 41)
Table 4.  
Association Between Hospital-Level Quality Indicators and Characteristics With Severe Maternal and Neonatal Morbidity
Association Between Hospital-Level Quality Indicators and Characteristics With Severe Maternal and Neonatal Morbidity
1.
Callaghan  WM, Mackay  AP, Berg  CJ.  Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991-2003 [published online February 15, 2008]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2007.12.020.
2.
Callaghan  WM, Creanga  AA, Kuklina  EV.  Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstet Gynecol. 2012;120(5):1029-1036.
PubMed
3.
Korst  LM, Fridman  M, Michael  CL,  et al.  Monitoring childbirth morbidity using hospital discharge data: further development and application of a composite measure [published online March 11, 2014]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2014.03.011.
4.
Creanga  AA, Berg  CJ, Ko  JY,  et al.  Maternal mortality and morbidity in the United States: where are we now? J Womens Health (Larchmt). 2014;23(1):3-9.
PubMedArticle
5.
Howell  EA, Hebert  P, Chatterjee  S, Kleinman  LC, Chassin  MR.  Black/white differences in very low birth weight neonatal mortality rates among New York City hospitals. Pediatrics. 2008;121(3):e407-e415.
PubMedArticle
6.
Panting-Kemp  A, Geller  SE, Nguyen  T, Simonson  L, Nuwayhid  B, Castro  L.  Maternal deaths in an urban perinatal network, 1992-1998. Am J Obstet Gynecol. 2000;183(5):1207-1212.
PubMedArticle
7.
Holt  J, Fagerli  I, Holdø  B, Vold  IN.  Audit of neonatal deaths of nonmalformed infants of 34 or more weeks’ gestation: unavoidable catastrophic events or suboptimal care? Acta Obstet Gynecol Scand. 2002;81(10):899-904.
PubMedArticle
8.
Nannini  A, Weiss  J, Goldstein  R, Fogerty  S.  Pregnancy-associated mortality at the end of the twentieth century: Massachusetts, 1990-1999. J Am Med Womens Assoc. 2002;57(3):140-143.
PubMed
9.
De Lange  TE, Budde  MP, Heard  AR, Tucker  G, Kennare  R, Dekker  GA.  Avoidable risk factors in perinatal deaths: a perinatal audit in South Australia. Aust N Z J Obstet Gynaecol. 2008;48(1):50-57.
PubMedArticle
10.
The Joint Commission. Specifications Manual for Joint Commission National Quality Measures (v2012B). The Joint Commission website. http://manual.jointcommission.org/releases/TJC2012B/. 2012. Accessed March 29, 2014.
11.
Center for Medicare & Medicaid Services (CMS). Hospital Compare. CMS website. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalCompare.html. Accessed March 29, 2014.
12.
Glantz  JC.  Elective induction vs. spontaneous labor associations and outcomes. J Reprod Med. 2005;50(4):235-240.
PubMed
13.
Bailit  JL, Love  TE, Dawson  NV.  Quality of obstetric care and risk-adjusted primary cesarean delivery rates. Am J Obstet Gynecol. 2006;194(2):402-407.
PubMedArticle
14.
Kuklina  EV, Whiteman  MK, Hillis  SD,  et al.  An enhanced method for identifying obstetric deliveries: implications for estimating maternal morbidity. Matern Child Health J. 2008;12(4):469-477.
PubMedArticle
15.
Gregory  KD, Fridman  M, Shah  S, Korst  LM.  Global measures of quality- and patient safety-related childbirth outcomes: should we monitor adverse or ideal rates? Am J Obstet Gynecol. 2009;200(6):e1-e7.
PubMedArticle
16.
National Quality Forum. Healthy Term Newborn Measure. National Quality Forum website. http://www.qualityforum.org/QPS/. Accessed July 23, 2014.
17.
Gray  KE, Wallace  ER, Nelson  KR, Reed  SD, Schiff  MA.  Population-based study of risk factors for severe maternal morbidity. Paediatr Perinat Epidemiol. 2012;26(6):506-514.
PubMedArticle
18.
Srinivas  SK, Fager  C, Lorch  SA.  Evaluating risk-adjusted cesarean delivery rate as a measure of obstetric quality. Obstet Gynecol. 2010;115(5):1007-1013.
PubMedArticle
19.
Grobman  WA, Feinglass  J, Murthy  S.  Are the Agency for Healthcare Research and Quality obstetric trauma indicators valid measures of hospital safety? Am J Obstet Gynecol. 2006;195(3):868-874.
PubMedArticle
20.
Grobman  WA, Bailit  JL, Rice  MM,  et al.  Can differences in obstetric outcomes be explained by differences in the care provided? the MFMU Network APEX Study [published online March 12, 2014]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2014.03.017.
21.
Olsen  IE, Groveman  SA, Lawson  ML, Clark  RH, Zemel  BS.  New intrauterine growth curves based on United States data. Pediatrics. 2010;125(2):e214-e224.
PubMedArticle
22.
Ash  AS, Normand  ST, Stukel  TA, Utts  J; Committee of Presidents of Statistical Societies (COPSS). Statistical Issues in Assessing Hospital Performance. Centers for Medicare & Medicaid Services website. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Downloads/Statistical-Issues-in-Assessing-Hospital-Performance.pdf. 2012. Accessed September 16, 2014.
23.
Kleinman  LC, Norton  EC.  What’s the risk? a simple approach for estimating adjusted risk measures from nonlinear models including logistic regression. Health Serv Res. 2009;44(1):288-302.
PubMedArticle
24.
Green  W. Econometric Analysis. Upper Saddle River, NJ: Prentice Hall; 2000.
25.
Howell  EA, Zeitlin  J, Hebert  P, Balbierz  A, Egorova  N.  Paradoxical trends and racial differences in obstetric quality and neonatal and maternal mortality. Obstet Gynecol. 2013;121(6):1201-1208.
PubMedArticle
26.
American College of Obstetricians and Gynecologists (ACOG) Task Force on Hyperension in Pregnancy. Hypertension in Pregnancy. Washington, DC: ACOG; 2013.
27.
California Maternal Quality Care Collaborative (CMQCC). OB Hemorrhage Toolkit. CMQCC website. https://www.cmqcc.org/ob_hemorrhage. 2010. Accessed March 29, 2014.
28.
The Joint Commission. Sentinel Event Alert: Preventing Infant Death and Injury During Delivery. The Joint Commission website. http://www.jointcommission.org/assets/1/18/SEA_30.PDF. July 21, 2004. Accessed September 16, 2014.
29.
The Joint Commission. Sentinel Event Alert: Preventing Maternal Death. The Joint Commission website. http://www.jointcommission.org/assets/1/18/SEA_44.PDF. January 26, 2010. Accessed September 16, 2014.
30.
Bailit  JL, Grobman  WA, Rice  MM,  et al.  Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals [published online July 24, 2013]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2013.07.019.
31.
Werner  RM, Bradlow  ET.  Relationship between Medicare’s hospital compare performance measures and mortality rates. JAMA. 2006;296(22):2694-2702.
PubMedArticle
32.
Morse  RB, Hall  M, Fieldston  ES,  et al.  Hospital-level compliance with asthma care quality measures at children’s hospitals and subsequent asthma-related outcomes. JAMA. 2011;306(13):1454-1460.
PubMedArticle
33.
Bilimoria  KY, Chung  J, Ju  MH,  et al.  Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure. JAMA. 2013;310(14):1482-1489.
PubMedArticle
34.
Conway  PH, Mostashari  F, Clancy  C.  The future of quality measurement for improvement and accountability. JAMA. 2013;309(21):2215-2216.
PubMedArticle
35.
Panzer  RJ, Gitomer  RS, Greene  WH, Webster  PR, Landry  KR, Riccobono  CA.  Increasing demands for quality measurement. JAMA. 2013;310(18):1971-1980.
PubMedArticle
36.
Phibbs  CS, Baker  LC, Caughey  AB, Danielsen  B, Schmitt  SK, Phibbs  RH.  Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med. 2007;356(21):2165-2175.
PubMedArticle
37.
Snowden  JM, Cheng  YW, Kontgis  CP, Caughey  AB.  The association between hospital obstetric volume and perinatal outcomes in California [published online October 21, 2012] . Am J Obstet Gynecol. doi:10.1016/j.ajog.2012.09.029.
38.
Klein  MC, Spence  A, Kaczorowski  J, Kelly  A, Grzybowski  S.  Does delivery volume of family physicians predict maternal and newborn outcome? CMAJ. 2002;166(10):1257-1263.
PubMed
39.
Piper  JM, Mitchel  EF  Jr, Snowden  M, Hall  C, Adams  M, Taylor  P.  Validation of 1989 Tennessee birth certificates using maternal and newborn hospital records. Am J Epidemiol. 1993;137(7):758-768.
PubMed
40.
DiGiuseppe  DL, Aron  DC, Ranbom  L, Harper  DL, Rosenthal  GE.  Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern Child Health J. 2002;6(3):169-179.
PubMedArticle
41.
Yasmeen  S, Romano  PS, Schembri  ME, Keyzer  JM, Gilbert  WM.  Accuracy of obstetric diagnoses and procedures in hospital discharge data. Am J Obstet Gynecol. 2006;194(4):992-1001.
PubMedArticle
42.
Romano  PS, Yasmeen  S, Schembri  ME, Keyzer  JM, Gilbert  WM.  Coding of perineal lacerations and other complications of obstetric care in hospital discharge data. Obstet Gynecol. 2005;106(4):717-725.
PubMedArticle
43.
New York City Maternal Mortality Review Project Team. Pregnancy-Associated Mortality, New York City 2001-2005. NYC.gov website. http://www.nyc.gov/html/doh/downloads/pdf/ms/ms-report-online.pdf. 2010. Accessed September 16, 2014.
Original Investigation
October 15, 2014

Association Between Hospital-Level Obstetric Quality Indicators and Maternal and Neonatal Morbidity

Author Affiliations
  • 1Department of Population Health Science & Policy, Icahn School of Medicine at Mount Sinai, New York, New York
  • 2Department of Obstetrics, Gynecology, and Reproductive Science, Icahn School of Medicine at Mount Sinai, New York, New York
  • 3Inserm UMR 1153, Obstetrical, Perinatal and Pediatric Epidemiology Research Team (Epopé), Center for Epidemiology and Biostatistics Sorbonne Paris Cité, DHU Risks in Pregnancy, Paris Descartes University, Paris, France
  • 4University of Washington School of Public Health, Seattle
JAMA. 2014;312(15):1531-1541. doi:10.1001/jama.2014.13381
Abstract

Importance  In an effort to improve the quality of care, several obstetric-specific quality measures are now monitored and publicly reported. The extent to which these measures are associated with maternal and neonatal morbidity is not known.

Objective  To examine whether 2 Joint Commission obstetric quality indicators are associated with maternal and neonatal morbidity.

Design, Setting, and Participants  Population-based observational study using linked New York City discharge and birth certificate data sets from 2010. All delivery hospitalizations were identified and 2 perinatal quality measures were calculated (elective, nonmedically indicated deliveries at 37 or more weeks of gestation and before 39 weeks of gestation; cesarean delivery performed in low-risk mothers). Published algorithms were used to identify severe maternal morbidity (delivery associated with a life-threatening complication or performance of a lifesaving procedure) and morbidity in term newborns without anomalies (births associated with complications such as birth trauma, hypoxia, and prolonged length of stay). Mixed-effects logistic regression models were used to examine the association between maternal morbidity, neonatal morbidity, and hospital-level quality measures while risk-adjusting for patient sociodemographic and clinical characteristics.

Main Outcomes and Measures  Individual- and hospital-level maternal and neonatal morbidity.

Results  Severe maternal morbidity occurred among 2372 of 115 742 deliveries (2.4%), and neonatal morbidity occurred among 8057 of 103 416 term newborns without anomalies (7.8%). Rates for elective deliveries performed before 39 weeks of gestation ranged from 15.5 to 41.9 per 100 deliveries among 41 hospitals. There were 11.7 to 39.3 cesarean deliveries per 100 deliveries performed in low-risk mothers. Maternal morbidity ranged from 0.9 to 5.7 mothers with complications per 100 deliveries and neonatal morbidity from 3.1 to 21.3 neonates with complications per 100 births. The maternal quality indicators elective delivery before 39 weeks of gestation and cesarean delivery performed in low-risk mothers were not associated with severe maternal complications (risk ratio [RR], 1.00 [95% CI, 0.98-1.02] and RR, 0.99 [95% CI, 0.96-1.01], respectively) or neonatal morbidity (RR, 0.99 [95% CI, 0.97-1.01] and RR, 1.01 [95% CI, 0.99-1.03], respectively).

Conclusions and Relevance  Rates for the quality indicators elective delivery before 39 weeks of gestation and cesarean delivery performed in low-risk mothers varied widely in New York City hospitals, as did rates of maternal and neonatal complications. However, there were no correlations between the quality indicator rates and maternal and neonatal morbidity. Current quality indicators may not be sufficiently comprehensive for guiding quality improvement in obstetric care.

Introduction

Although great progress has been made in reducing obstetric complications, they persist. Severe maternal complications include renal failure and eclampsia or the need for lifesaving interventions such as prolonged mechanical ventilation or transfusions.1,2 Neonatal complications may occur in low-risk term infants and include hypoxia and shock.3 Severe maternal morbidity occurs in about 60 000 women (1.6 per 100 deliveries) annually in the United States, and 1 in 10 term infants experience neonatal complications.3,4 Variation in complication rates between hospitals exists and suggests that the quality of obstetric care can be improved.5 More than one-third of maternal deaths and severe morbidities, and a significant proportion of neonatal mortality and morbidity, may be preventable by changes in patient, clinician, and system factors.4,69

As part of its core measure set, The Joint Commission now recommends 2 perinatal quality measures that address important aspects of obstetric care during childbirth: elective deliveries performed prior to 39 weeks of gestation and cesarean deliveries performed in low-risk nulliparous women.10 The elective delivery measure, which includes nonmedically indicated deliveries associated with medical induction or cesarean delivery at more than 37 weeks and prior to 39 weeks of gestation, is also mandated by the Centers for Medicare & Medicaid Services.11 The elective delivery before 39 weeks of gestation indicator is intended to reduce neonatal complications among term infants. Assessing rates of cesarean delivery performed in low-risk patients is intended to reduce unnecessary variation in rates of cesarean delivery. Both of these measures may be associated with maternal outcomes.12,13 However, how well hospital performance on these quality indicators correlates with maternal or neonatal morbidity is not known.

We investigated whether elective deliveries performed prior to 39 weeks of gestation and cesarean deliveries performed in low-risk nulliparous women were associated with severe maternal or neonatal morbidity in New York City hospitals.

Methods
Study Sample

We used birth certificate data linked with New York State discharge abstract data reported in the Statewide Planning and Research Cooperative System (SPARCS) for all delivery and newborn hospitalizations in New York City in 2010. Data linkage was conducted by the New York State Department of Health. Ninety-seven percent of maternal and 98% of infant discharge abstracts were linked with infant birth certificates. Institutional review board approvals were obtained from the New York City Department of Health and Mental Hygiene, the New York State Department of Health, and the Icahn School of Medicine at Mount Sinai. A waiver of consent was approved by the Icahn School of Medicine. Delivery hospitalizations were identified based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes and diagnosis related group delivery codes.14 Singleton nonanomalous term newborns were identified using the newborn flag. Gestational age was ascertained from birth certificate data and congenital anomalies from SPARCS data.

Identifying Severe Maternal Morbidity

We identified severe maternal morbidity using a Centers for Disease Control and Prevention (CDC)–endorsed surveillance algorithm (available at http://www.cdc.gov/reproductivehealth/MaternalInfantHealth/SevereMaternalMorbidity.html), using diagnoses for life-threatening conditions and procedure codes for life-saving procedures.2 Codes were selected by experts and reviewed for their association with in-hospital maternal mortality. The algorithm excludes hospitalizations with a length of stay less than the 90th percentile as calculated separately for vaginal and primary and repeat cesarean deliveries.2 As recommended, length of stay reclassification was not applied to in-hospital deaths, transfers, or delivery hospitalizations with severe complications identified by procedure codes.2

Identifying Neonatal Morbidity at Term

We identified neonatal morbidity at term based on diagnoses and procedure codes as defined by Korst et al3 to monitor childbirth morbidity using hospital discharge data.3,15,16 The list includes ICD-9-CM code–based categories that capture indicators of death or neonatal complications (eg, birth trauma, disseminated intravascular coagulation, neonatal intensive care unit [NICU] procedures, renal failure, respiratory conditions, necrotizing enterocolitis, shock, neonatal length of stay >5 days, neonatal death) and was ascertained from SPARCS data.3

Quality Measures

We used birth certificate data and SPARCS data to calculate 2 perinatal quality measures using algorithms designated by the Joint Commission.10 The first measure, elective deliveries at 37 or more weeks and prior to 39 weeks of gestation, was defined as all deliveries associated with medical inductions of labor or cesarean delivery at 37 or more weeks and prior to 39 weeks of gestation as a proportion of all deliveries at 37 or more weeks and prior to 39 weeks. All conditions possibly justifying delivery prior to 39 weeks of gestation were excluded, as specified.10 We also excluded cesarean deliveries that were associated with a trial of labor but were not inductions. The second measure, cesarean deliveries in low-risk women, was defined as the proportion of cesarean deliveries among nulliparous patients with singleton vertex deliveries of newborns at 37 or more weeks of gestation. We excluded patients with ICD-9-CM codes that signify contraindications to vaginal deliveries.10 Gestational age, parity, multiple birth, vertex presentation, and trial of labor were ascertained from birth certificate data. ICD-9-CM codes were obtained from SPARCS.

For each hospital we calculated the rate of elective deliveries and low-risk cesarean deliveries. No risk adjustment is required for the elective delivery measure, but direct standardization by maternal age is suggested for the cesarean delivery measure. We standardized this indicator for each hospital by 5-year age groups using the observed maternal age distribution in the entire sample.10

Covariates

To risk-adjust hospital-level rates of maternal morbidity we used variables from birth certificate records, including mothers’ sociodemographics (maternal age, self-identified race/ethnicity, parity, education), prenatal care visits, and clinical and obstetric factors (multiple births, history of previous cesarean delivery). We ascertained patient insurance status from SPARCS. We also included diagnoses codes for patient risk factors that could lead to maternal morbidity but were likely present on admission to the hospital (eg, diabetes, hypertension, obesity, premature rupture of membranes, disorders of placentation). These conditions have been used to risk-adjust for severe maternal morbidity,17 cesarean deliveries, and other maternal outcomes (eTable in the Supplement).18,19

We risk-adjusted neonatal morbidity by maternal sociodemographics (maternal age, self-identified race and ethnicity, parity, education), behaviors (prenatal care visit, tobacco use, illicit drug use), maternal clinical factors (hypertension or diabetes during or before pregnancy), obstetric factors (previous cesarean delivery, premature rupture of membranes), and infant factors (gestational age, size for gestational age).20 Small for gestational age was defined as a birth weight in less than the 10th percentile; large for gestational age was defined as a birth weight in more than the 90th percentile for gestational age and sex.21

We obtained teaching status from the American Hospital Association, ownership and nursery level from the New York State Department of Health, and volume of deliveries in each hospital from SPARCS to assess how other hospital characteristics are correlated with measures of quality and morbidity.

Statistical Analyses

We compared the sociodemographic characteristics and clinical conditions of women with and without severe maternal morbidity and of infants with and without neonatal morbidity using χ2 tests.

We used mixed-effects logistic regression with a random hospital-specific intercept to generate risk-standardized severe maternal morbidity rates and risk-standardized neonatal morbidity rates at term for each hospital. The models included covariates described above. Hospital risk-standardized rates were computed from these models using methods recommended by Centers for Medicare & Medicaid Services Hospital Compare22 (eAppendix in the Supplement).

We used Spearman rank correlations to assess correlations among hospital-level rates of severe maternal morbidity, neonatal morbidity, and the 2 measures of hospital quality. We divided hospitals into quartiles based on each quality measure and examined hospital rankings on both risk-standardized morbidity measures using the nonparametric Kruskal-Wallis Test. Quality measures were coded as continuous for our primary multivariable analyses. Our final models explored the association of the 2 quality measures with severe maternal morbidity and then with neonatal morbidity using mixed-effects logistic regression in 3 steps, with (1) no adjustment, (2) adjustment for patient characteristics, and (3) adjustment for patient and hospital-level characteristics, including ownership, volume, level of nursery, and teaching status. Risk ratios were calculated using recycled predictions, which yields the marginal effect of a variable while holding all other patient characteristics constant23; confidence intervals were estimated using the delta method.24 Intraclass correlations were calculated to measure the between-hospital variance as a proportion of the total variance.

Sensitivity analyses included coding quality measures in quartiles; examining the association of the quality measures with severe maternal morbidity without blood transfusion; and using 2 simpler measures of neonatal outcomes (prolonged length of stay [>5 days] and NICU admission).

Statistical analyses were performed using SAS version 9.3 (SAS Institute Inc) and Stata version 12 (StataCorp). P < .05 (2-sided) was considered statistically significant.

Results
Patient Sample Characteristics, Severe Maternal Morbidity, and Neonatal Morbidity at Term

Of the 43 hospitals with deliveries, we excluded 2 hospitals with annual delivery volumes fewer than 5 births and 1360 deliveries with missing hospital identifiers. The final sample included 115 742 deliveries, of which 2732 (2.4%) were associated with severe maternal morbidity. Of the 119 793 newborns identified, we excluded 4672 multiple births, 4447 with congenital anomalies, and 7258 with gestational age less than 37 weeks. The final sample included 103 416 newborns; of these, 8057 (7.8%) were associated with neonatal morbidity. The associations of sociodemographic and clinical characteristics with severe maternal morbidity and neonatal morbidity are described in Table 1 and Table 2.

Hospital Characteristics and Performance on Perinatal Quality and Morbidity Measures

Table 3 shows that the majority of hospitals were private, had level 3/4 nurseries, and were teaching hospitals. Hospital performance per 100 deliveries ranged from 15.5 to 41.9 for elective deliveries before 39 weeks; 11.7 to 39.3 for age-standardized rates of cesarean deliveries in low-risk nulliparous women; 0.9 to 5.7 for risk-standardized severe maternal morbidity rates; and 3.1 to 21.3 for risk-standardized neonatal morbidity rates among these hospitals. Both quality measures were correlated with each other (Spearman ρ = 0.45; P = .003). Severe maternal morbidity and neonatal morbidity at term were also correlated (Spearman ρ = 0.39; P = .01).

Hospital Rankings on Quality Measures and Risk-Standardized Maternal and Neonatal Morbidity

Hospital rankings on both quality measures were not associated with hospital rankings for maternal or neonatal morbidity as demonstrated in the Figure. In fact, among the 10 hospitals with the best performance (lowest rates) on elective deliveries, only 3 were in the lowest quartile of risk-standardized severe maternal morbidity. Among the 10 hospitals with the best performance (lowest rates) on the low-risk cesarean delivery measure, only 3 were in the lowest quartile of risk-standardized severe maternal morbidity. The rankings were similarly discordant for neonatal morbidity.

Nonparametric correlations between obstetric quality indicators and both maternal and neonatal morbidity were not significant (Spearman ρ = −0.01, P = .94 for cesarean delivery and maternal morbidity; Spearman ρ = −0.10, P = .52 for cesarean delivery and neonatal morbidity; Spearman ρ = 0.14, P = .39 for elective delivery and maternal morbidity; Spearman ρ = −0.14, P = .39 for elective delivery and neonatal morbidity).

Mixed-Effects Logistic Regression Models

In mixed-effects models including both quality measures as independent variables, neither quality indicator was associated with severe maternal morbidity (model 1, Table 4); the risk ratio (RR) for elective delivery was 1.00 (95% CI, 0.97-1.03) and for low-risk cesarean deliveries was 0.98 (95% CI, 0.95-1.01). In models that also included patient-level variables (model 2) and patient-level and other hospital-level variables (model 3), the RRs for both quality measures remained essentially unchanged. Sensitivity analyses with the quality measures coded in quartiles and analyses excluding blood transfusions from the severe maternal morbidity outcome corroborated our findings. There was a substantial between-hospital variation in severe maternal morbidity: the intraclass correlation coefficient (ICC) was 28% in the unadjusted model; after adjusting for patient characteristics, ICC was reduced to 22%.

Our findings were similar in models examining the 2 obstetric quality measures and neonatal morbidity (Table 4). Volume of deliveries and nursery level were associated with neonatal morbidity. The association between the 2 obstetric quality measures with neonatal length of stay longer than 5 days and NICU admission showed similar results. In a multilevel model without adjustment, the ICC was 20% for neonatal morbidity. The variance between hospitals remained (ICC = 16%) after accounting for patient characteristics.

Discussion

Severe maternal morbidity and neonatal morbidity at term remain important health issues. We found that in New York City hospitals 2.4% of all mothers and 7.8% of term nonanomalous newborns have major complications and that these rates vary substantially between hospitals. Severe maternal morbidity rates varied 4- to 5-fold between hospitals, and there was a 7-fold variation in neonatal morbidity at term between hospitals. Although there was substantial variation in morbidity rates, they were not correlated with the performance measures designed to assess hospital-level obstetric quality of care.

Measuring quality of care in obstetrics is complex: it involves assessing care for 2 separate individuals (mother and infant). Improving the quality of care requires reducing the use of obstetric interventions that can harm infants (eg, early delivery) and mothers (eg, unnecessary cesarean delivery) and avoiding suboptimal care such as underutilization of antenatal steroids or antibiotics, which can lead to neonatal or maternal complications or death. Measures of overutilization may be important obstetric quality indicators and enable the tracking of utilization of obstetric interventions; however, the measures we assessed do not reflect the quality of care in terms of severe maternal or neonatal morbidity.

The Joint Commission set of perinatal quality measures includes 3 other indicators in addition to the 2 we studied. Those measures assess other dimensions of perinatal care—breastfeeding, health care–associated infections among very low-birth-weight infants, and antenatal steroid use among premature deliveries. The measures may be indicators of obstetric quality, but the outcomes primarily concern newborns and not their mothers and 2 of the measures only assess care provided in the neonatal period. Our findings suggest that other quality measures should be developed that focus on suboptimal care. Examples include whether hemorrhage and preeclampsia protocols are used in the delivery suite.2529 Given that we did not find a relationship between the quality indicators we examined and maternal or neonatal morbidity, these measures do not adequately capture obstetric hospital quality.

Our results are consistent with findings from the Maternal Fetal Medicine Network and studies in NICUs demonstrating that performance assessment based on isolated measures do not accurately characterize the overall quality of care in a hospital.20,30 Numerous other studies have found a lack of correlation between quality indicators and the clinical outcomes they are supposed to reflect. Hospital Compare measures only predict small differences in hospital risk-adjusted mortality rates for heart failure, pneumonia, and acute myocardial infarction.31 Adherence with infection control practices was not associated with lower perioperative infection rates (http://www.ncbi.nlm.nih.gov/pubmed/20571014). There was no association between adherence to home management care quality measures and emergency department visits and readmissions for pediatric asthma.32 Adherence to perioperative prophylaxis for venous thromboembolism was not associated with venous thromboembolism rates, and surveillance bias resulted in unfair penalization of hospitals that more rigorously looked for the disorder.33 The growing list of quality measures often does not reflect the care these measures are supposed to measure and capture only a narrow slice of hospital quality.34,35 There is a need to reassess how these measures are designed and implemented and to think more broadly about constructing meaningful quality measures tightly linked with patient outcomes.

In our analyses, the number of deliveries at a hospital and the availability of high-level nursery facilities were associated with neonatal morbidity at term. These hospital characteristics have been consistently related to outcomes for very low-birth-weight infants.36 The association is less clear for outcomes of lower-risk infants.37,38 Few studies have investigated how hospital characteristics affect severe maternal morbidity.

Our analysis has limitations. We used administrative (ICD-9-CM procedure and diagnosis codes) and birth certificate data rather than medical chart review for our analysis. Both birth certificate and SPARCS data may be limited by the reliability of certain types of information contained in these databases. Previous assessments demonstrated good reliability for maternal age, race, and gestational age.39 Previous studies of birth certificate data used for hospital quality assessment studies showed that they perform as well as medical records information.40

Algorithms for morbidity based on diagnosis and procedure codes are imperfect. However, discharge data in obstetrics are reliable for delivery and procedure codes and moderately reliable for comorbidities.41,42 It is possible that the codes used in our analyses inadequately captured the outcomes of interest. However, many of the diagnosis codes included in the severe maternal morbidity algorithm (eg, eclampsia, shock, acute renal failure) are designated as major complications and comorbidities codes in diagnosis related group coding and when present may affect diagnosis related group assignment for obstetric deliveries, making these diagnoses less likely to be underreported. In a sensitivity analyses we found that coding for prolonged neonatal length of stay and NICU admission (outcomes known to be less subject to coding problems) were not associated with the 2 quality measures. If administrative codes inadequately captured the clinical information we assessed, we would have expected null findings in the patient-level analysis. This was not the case. We were able to demonstrate significant associations between age, race/ethnicity, parity, education, prenatal care, pregnancy type, and multiple comorbidities with severe maternal morbidity.

Although we have no reason to believe that systematic coding biases (eg, the probability that a hospital undercodes maternal morbidity is a positive function of its cesarean or elective delivery rates) affected our results, the existence of random coding errors could bias our findings toward the null. Our estimate for severe maternal morbidity is higher than the recent national estimate of 1.6% that was derived using similar methods4 but is consistent with data from New York City showing significantly higher than average maternal mortality rates,43 and the rates of neonatal morbidity at term are consistent with previous findings.3 Further, without medical chart review we were unable to address the recent Joint Commission recommendation to exclude women with specific types of previous uterine surgery (eg, classic cesarean delivery and myomectomy) from the measure on elective delivery before 39 weeks. We constructed risk-adjustment models that included multiple comorbidities and clinical conditions consistent with those in previous studies. Our sensitivity analyses examining the association of the quality measures with subcomponents of both outcomes measures (severe maternal morbidity without blood transfusion, prolonged neonatal length of stay, and NICU admission) reinforced our findings. By linking hospital discharge and birth certificate data we were able to control for maternal confounders, such as self-identified race/ethnicity, education, and prenatal visits associated with maternal and neonatal morbidity.

Conclusions

Performance on elective delivery before 39 weeks of gestation, cesarean deliveries performed on low-risk mothers, and maternal and neonatal morbidity varied widely between New York City hospitals. The obstetric quality indicators we examined were not associated with lower morbidity. Our findings highlight the need for an expanded array of obstetric quality measures.

Back to top
Article Information

Corresponding Author: Elizabeth A. Howell, MD, MPP, Departments of Population Health Science and Policy and Obstetrics, Gynecology, and Reproductive Science, PO Box 1077, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Pl, New York, NY 10029 (elizabeth.howell@mountsinai.org).

Author Contributions: Dr Egorova had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Howell, Zeitlin, Hebert, Egorova.

Acquisition, analysis, or interpretation of data: Howell, Zeitlin, Balbierz, Egorova.

Drafting of the manuscript: Howell, Zeitlin.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Zeitlin, Hebert, Egorova.

Obtained funding: Howell, Balbierz.

Administrative, technical, or material support: Balbierz.

Study supervision: Howell.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: This study was funded by grant R21HD068765 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development. Dr Zeitlin received funding from the European Commission, Research Directorate, Marie Curie, IOF Fellowship grant 254171.

Role of the Funders/Sponsors: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References
1.
Callaghan  WM, Mackay  AP, Berg  CJ.  Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991-2003 [published online February 15, 2008]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2007.12.020.
2.
Callaghan  WM, Creanga  AA, Kuklina  EV.  Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstet Gynecol. 2012;120(5):1029-1036.
PubMed
3.
Korst  LM, Fridman  M, Michael  CL,  et al.  Monitoring childbirth morbidity using hospital discharge data: further development and application of a composite measure [published online March 11, 2014]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2014.03.011.
4.
Creanga  AA, Berg  CJ, Ko  JY,  et al.  Maternal mortality and morbidity in the United States: where are we now? J Womens Health (Larchmt). 2014;23(1):3-9.
PubMedArticle
5.
Howell  EA, Hebert  P, Chatterjee  S, Kleinman  LC, Chassin  MR.  Black/white differences in very low birth weight neonatal mortality rates among New York City hospitals. Pediatrics. 2008;121(3):e407-e415.
PubMedArticle
6.
Panting-Kemp  A, Geller  SE, Nguyen  T, Simonson  L, Nuwayhid  B, Castro  L.  Maternal deaths in an urban perinatal network, 1992-1998. Am J Obstet Gynecol. 2000;183(5):1207-1212.
PubMedArticle
7.
Holt  J, Fagerli  I, Holdø  B, Vold  IN.  Audit of neonatal deaths of nonmalformed infants of 34 or more weeks’ gestation: unavoidable catastrophic events or suboptimal care? Acta Obstet Gynecol Scand. 2002;81(10):899-904.
PubMedArticle
8.
Nannini  A, Weiss  J, Goldstein  R, Fogerty  S.  Pregnancy-associated mortality at the end of the twentieth century: Massachusetts, 1990-1999. J Am Med Womens Assoc. 2002;57(3):140-143.
PubMed
9.
De Lange  TE, Budde  MP, Heard  AR, Tucker  G, Kennare  R, Dekker  GA.  Avoidable risk factors in perinatal deaths: a perinatal audit in South Australia. Aust N Z J Obstet Gynaecol. 2008;48(1):50-57.
PubMedArticle
10.
The Joint Commission. Specifications Manual for Joint Commission National Quality Measures (v2012B). The Joint Commission website. http://manual.jointcommission.org/releases/TJC2012B/. 2012. Accessed March 29, 2014.
11.
Center for Medicare & Medicaid Services (CMS). Hospital Compare. CMS website. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalCompare.html. Accessed March 29, 2014.
12.
Glantz  JC.  Elective induction vs. spontaneous labor associations and outcomes. J Reprod Med. 2005;50(4):235-240.
PubMed
13.
Bailit  JL, Love  TE, Dawson  NV.  Quality of obstetric care and risk-adjusted primary cesarean delivery rates. Am J Obstet Gynecol. 2006;194(2):402-407.
PubMedArticle
14.
Kuklina  EV, Whiteman  MK, Hillis  SD,  et al.  An enhanced method for identifying obstetric deliveries: implications for estimating maternal morbidity. Matern Child Health J. 2008;12(4):469-477.
PubMedArticle
15.
Gregory  KD, Fridman  M, Shah  S, Korst  LM.  Global measures of quality- and patient safety-related childbirth outcomes: should we monitor adverse or ideal rates? Am J Obstet Gynecol. 2009;200(6):e1-e7.
PubMedArticle
16.
National Quality Forum. Healthy Term Newborn Measure. National Quality Forum website. http://www.qualityforum.org/QPS/. Accessed July 23, 2014.
17.
Gray  KE, Wallace  ER, Nelson  KR, Reed  SD, Schiff  MA.  Population-based study of risk factors for severe maternal morbidity. Paediatr Perinat Epidemiol. 2012;26(6):506-514.
PubMedArticle
18.
Srinivas  SK, Fager  C, Lorch  SA.  Evaluating risk-adjusted cesarean delivery rate as a measure of obstetric quality. Obstet Gynecol. 2010;115(5):1007-1013.
PubMedArticle
19.
Grobman  WA, Feinglass  J, Murthy  S.  Are the Agency for Healthcare Research and Quality obstetric trauma indicators valid measures of hospital safety? Am J Obstet Gynecol. 2006;195(3):868-874.
PubMedArticle
20.
Grobman  WA, Bailit  JL, Rice  MM,  et al.  Can differences in obstetric outcomes be explained by differences in the care provided? the MFMU Network APEX Study [published online March 12, 2014]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2014.03.017.
21.
Olsen  IE, Groveman  SA, Lawson  ML, Clark  RH, Zemel  BS.  New intrauterine growth curves based on United States data. Pediatrics. 2010;125(2):e214-e224.
PubMedArticle
22.
Ash  AS, Normand  ST, Stukel  TA, Utts  J; Committee of Presidents of Statistical Societies (COPSS). Statistical Issues in Assessing Hospital Performance. Centers for Medicare & Medicaid Services website. http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Downloads/Statistical-Issues-in-Assessing-Hospital-Performance.pdf. 2012. Accessed September 16, 2014.
23.
Kleinman  LC, Norton  EC.  What’s the risk? a simple approach for estimating adjusted risk measures from nonlinear models including logistic regression. Health Serv Res. 2009;44(1):288-302.
PubMedArticle
24.
Green  W. Econometric Analysis. Upper Saddle River, NJ: Prentice Hall; 2000.
25.
Howell  EA, Zeitlin  J, Hebert  P, Balbierz  A, Egorova  N.  Paradoxical trends and racial differences in obstetric quality and neonatal and maternal mortality. Obstet Gynecol. 2013;121(6):1201-1208.
PubMedArticle
26.
American College of Obstetricians and Gynecologists (ACOG) Task Force on Hyperension in Pregnancy. Hypertension in Pregnancy. Washington, DC: ACOG; 2013.
27.
California Maternal Quality Care Collaborative (CMQCC). OB Hemorrhage Toolkit. CMQCC website. https://www.cmqcc.org/ob_hemorrhage. 2010. Accessed March 29, 2014.
28.
The Joint Commission. Sentinel Event Alert: Preventing Infant Death and Injury During Delivery. The Joint Commission website. http://www.jointcommission.org/assets/1/18/SEA_30.PDF. July 21, 2004. Accessed September 16, 2014.
29.
The Joint Commission. Sentinel Event Alert: Preventing Maternal Death. The Joint Commission website. http://www.jointcommission.org/assets/1/18/SEA_44.PDF. January 26, 2010. Accessed September 16, 2014.
30.
Bailit  JL, Grobman  WA, Rice  MM,  et al.  Risk-adjusted models for adverse obstetric outcomes and variation in risk-adjusted outcomes across hospitals [published online July 24, 2013]. Am J Obstet Gynecol. doi:10.1016/j.ajog.2013.07.019.
31.
Werner  RM, Bradlow  ET.  Relationship between Medicare’s hospital compare performance measures and mortality rates. JAMA. 2006;296(22):2694-2702.
PubMedArticle
32.
Morse  RB, Hall  M, Fieldston  ES,  et al.  Hospital-level compliance with asthma care quality measures at children’s hospitals and subsequent asthma-related outcomes. JAMA. 2011;306(13):1454-1460.
PubMedArticle
33.
Bilimoria  KY, Chung  J, Ju  MH,  et al.  Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure. JAMA. 2013;310(14):1482-1489.
PubMedArticle
34.
Conway  PH, Mostashari  F, Clancy  C.  The future of quality measurement for improvement and accountability. JAMA. 2013;309(21):2215-2216.
PubMedArticle
35.
Panzer  RJ, Gitomer  RS, Greene  WH, Webster  PR, Landry  KR, Riccobono  CA.  Increasing demands for quality measurement. JAMA. 2013;310(18):1971-1980.
PubMedArticle
36.
Phibbs  CS, Baker  LC, Caughey  AB, Danielsen  B, Schmitt  SK, Phibbs  RH.  Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med. 2007;356(21):2165-2175.
PubMedArticle
37.
Snowden  JM, Cheng  YW, Kontgis  CP, Caughey  AB.  The association between hospital obstetric volume and perinatal outcomes in California [published online October 21, 2012] . Am J Obstet Gynecol. doi:10.1016/j.ajog.2012.09.029.
38.
Klein  MC, Spence  A, Kaczorowski  J, Kelly  A, Grzybowski  S.  Does delivery volume of family physicians predict maternal and newborn outcome? CMAJ. 2002;166(10):1257-1263.
PubMed
39.
Piper  JM, Mitchel  EF  Jr, Snowden  M, Hall  C, Adams  M, Taylor  P.  Validation of 1989 Tennessee birth certificates using maternal and newborn hospital records. Am J Epidemiol. 1993;137(7):758-768.
PubMed
40.
DiGiuseppe  DL, Aron  DC, Ranbom  L, Harper  DL, Rosenthal  GE.  Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern Child Health J. 2002;6(3):169-179.
PubMedArticle
41.
Yasmeen  S, Romano  PS, Schembri  ME, Keyzer  JM, Gilbert  WM.  Accuracy of obstetric diagnoses and procedures in hospital discharge data. Am J Obstet Gynecol. 2006;194(4):992-1001.
PubMedArticle
42.
Romano  PS, Yasmeen  S, Schembri  ME, Keyzer  JM, Gilbert  WM.  Coding of perineal lacerations and other complications of obstetric care in hospital discharge data. Obstet Gynecol. 2005;106(4):717-725.
PubMedArticle
43.
New York City Maternal Mortality Review Project Team. Pregnancy-Associated Mortality, New York City 2001-2005. NYC.gov website. http://www.nyc.gov/html/doh/downloads/pdf/ms/ms-report-online.pdf. 2010. Accessed September 16, 2014.
×