Prenatal Health Care Outcomes Before and During the COVID-19 Pandemic Among Pregnant Individuals and Their Newborns in an Integrated US Health System

This cohort study compares a multimodal model of in-office and telemedicine prenatal health care model implemented during the COVID-19 pandemic with prepandemic in-office only health care and its association with maternal and newborn health outcomes.


Introduction
Across the US, the COVID-19 pandemic has been an unprecedented challenge for the health care of pregnant individuals, a population in need of frequent contacts with the health care system. The COVID-19 pandemic has also led to a rapid adoption of telemedicine, 1 offering a unique opportunity to reevaluate and reorganize prenatal health care delivery. Telemedicine may become an important modality integrated into prenatal health care beyond the pandemic since it can reach patients in rural areas and address other barriers to health care, such as lack of transportation and limited time due to work or childcare. 2 However, data on the integration of telemedicine in prenatal health care and prenatal health outcomes are sparse.
To reduce unnecessary exposure of pregnant individuals to COVID-19, Kaiser Permanente Northern California (KPNC) implemented telemedicine in prenatal health care. Taking advantage of the pandemic-related change constituted by the implementation of a multimodal prenatal health care model of in-office and telemedicine, we investigated the association of this multimodal model with maternal and newborn health outcomes.
We hypothesized that the delivery of prenatal health care by a multimodal model of telemedicine and in-office visits during the pandemic was equally effective regarding health outcomes as standard in-office health care before the pandemic. We also explored whether the outcomes of the multimodal health care model were similar among individuals of different races and ethnicities, those with a disadvantaged socioeconomic background, with a preferred language other than English, or those living in a rural area.

Methods
This cohort study was approved by the KPNC institutional review board, which waived the requirement for participant informed consent because the study used only electronic data and the large sample size did not make it feasible to obtain authorization from each individual. This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. 3 This data-only project was conducted at KPNC, an integrated health care delivery system, using longitudinal data from electronic health records (EHRs). KPNC serves 4.5 million individuals (including approximately 60 000 pregnant individuals per year), accounting for approximately 40% of the population residing in the served geographic area (including both urban and rural areas), and is representative of the underlying population regarding race and ethnicity, education level, and obesity rates, differing only slightly from the underlying population in regard to income. [4][5][6] Using EHR data, we identified all pregnant individuals who delivered a live birth or stillbirth between July 1, 2018, and October 21, 2021. We searched EHRs for all encounter types (eg, in-office or telemedicine) and identified diagnoses and procedures according to the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes (eTable 1 in Supplement 1), laboratory tests and results, and medications from 2 years before the last menstrual period through pregnancy.
Prior to March 13, 2020, KPNC followed a prenatal health care delivery schedule as described by the American College of Obstetricians and Gynecologists, 7 with all standard visits and screenings taking place in person. As of March 13, 2020, changes in prenatal health care were implemented. For patients at low risk (ie, patients younger than 40 years) and patients with specific moderate risk conditions including gestational diabetes (GD) receiving diet therapy only, chronic hypertension without medication management, prior cesarean delivery, and other conditions per clinician discretion, the following 4 standard in-office prenatal health care visits were converted to telemedicine visits (either by video or by telephone according to the patient's preference): 10-to 12-week visit (after completion of the first trimester office physical examination and ultrasonography examination), 22-week visit, 28-week visit, and 32-week visit.
To reflect the levels that a pregnancy was exposed to the multimodal prenatal health care model related to the COVID-19 pandemic, dates of birth delivery were categorized in 3 time intervals: T1, unexposed (July 1, 2018, to February 29, 2020); T2, partially exposed (March 1, 2020, to December 5, 2020); and T3, fully exposed (December 6, 2020, to October 31, 2021). Individuals with birth delivery in T2 did not experience the multimodal prenatal health care model for the entire duration of the pregnancy; the fraction of a pregnancy exposed to the multimodal health care model increased from 0% to 100% with each additional day during this time interval. In contrast, individuals with birth delivery in T3 experienced the multimodal health care model for the entire duration of the pregnancy.
Primary outcomes included rates of preeclampsia and eclampsia, severe maternal morbidity, cesarean delivery, preterm birth, and neonatal intensive care unit (NICU) admission. We defined preeclampsia and eclampsia by ICD-10 codes or meeting both of the following criteria after 20 weeks of gestation: (1) at least 2 blood pressure readings above 140/90 mm Hg on 2 occasions at least 4 hours apart or at least 2 readings above 160/110 mm Hg on 2 occasions within 1 hour and (2) new onset of proteinuria, thrombocytopenia, or pulmonary edema. 8 We used the Centers for Disease Control and Prevention Callaghan criteria 9-11 to define the presence of severe maternal morbidity, which comprised 21 conditions occurring at any time during pregnancy or delivery. 10 Preterm birth was defined using a previously validated, 12 obstetric, estimate-based measure of gestational age at delivery (<37 weeks). We further classified preterm birth as spontaneous (based on hospital discharge codes for early spontaneous onset of delivery and/or premature rupture of membranes) or medically indicated (if individuals had no indication of spontaneous birth and a hospital discharge diagnosis of induced and/or cesarean delivery). We used ICD-10 codes to define cesarean deliveries and NICU admissions.
searched EHRs for 5-minute Apgar scores. We defined being small for gestational age (SGA) and large for gestational age (LGA) as gestational age-specific and sex-specific birth weight less than the 10th percentile and greater than the 90th percentile, respectively. 16 Age, self-reported race and ethnicity, residential addresses, and preferred language were captured in the outpatient setting or at hospital admission. Parity, smoking, prepregnancy height and weight, and health outcomes were obtained from EHRs, where data are collected by health care clinicians. Race was defined as American Indian or Alaska Native, Asian or Pacific Islander, Black, multiracial, or White. All patients who self-identified as Hispanic ethnicity, regardless of race, were categorized as Hispanic. Race and ethnicity were included because of known associations with perinatal outcomes. 17 Residential addresses were used to calculate the neighborhood deprivation index (NDI), a composite index where higher values indicate more disadvantaged neighborhood characteristics, 18 and for identifying whether an individual lived in an urban or rural area 19 (eAppendix in Supplement 1). According to the World Health Organization recommendations on specific body mass index (BMI, calculated as weight in kilograms divided by height in meters squared) cutoffs for individuals of different races and ethnicities, 20 we categorized individuals as underweight, normal weight, overweight, and obese.

Statistical Analysis
The distributions of demographic and clinical characteristics, care processes, and health outcomes for birth deliveries within each of the 3 intervals of interest were assessed with standardized mean differences (SMDs) calculated for between-interval contrasts. Analyses examined changes in rates of perinatal outcomes in relation to the multimodal prenatal health care model implemented during the COVID-19 pandemic using an interrupted time series (ITS) design. 21 Log binomial regression was used to examine each binary outcome in relation to date of birth delivery (week), providing point and interval estimates of relative percentage change in outcome probability associated with change in time, scaled to a 4-week change for the purpose of interpretability of small effect sizes. Poisson regression was used for point and interval estimation of relative percentage change in means for outcomes characterized as counts in relation to a 4-week change in birth delivery date. Analyses were performed with and without adjustment for age, race and ethnicity, prepregnancy BMI, and NDI.
In accordance with the ITS method, 21,22 prenatal outcomes before and during the pandemic were modeled using piecewise linear segmented regression, allowing a change in slope associated with time at 2 inflection points: March 1, 2020 (initiation of the multimodal health care model), and December 6, 2020 (date when individuals were fully exposed to the multimodal health care model). Given the nature of exposure to the multimodal health care model, with the proportion of a pregnancy with a postpandemic birth delivery date that was exposed to the change gradually increasing from 0% to 100% during T2 (March 1, 2020, to December 6, 2020), we hypothesized and allowed for a change in slope associated with calendar time with no immediate or abrupt outcome level change at each of the 2 inflection points. Formal tests of statistical significance for differences in slopes between the 3 time intervals were calculated (2-sided Wald test). P < .05 was considered statistically significant. ITS analyses were also performed after stratifying individuals by various sociodemographic factors including race and ethnicity, NDI quartiles, living in an urban or rural area, whether English was the preferred language, and whether they had a low-risk pregnancy. We also presented plots of weekly proportions of birth deliveries with each binary outcome of interest and means among deliveries in each week for variables characterized as counts, with an overlaid plot of estimated values from a segmented linear regression with no covariate adjustment as a visual aid in interpretation of time trends (see eAppendix in Supplement 1 for more details on the statistical analyses). We used multiple imputation 23,24 for variables with missing data such as age (0.01%), race and ethnicity (2.30%), NDI (0.15%), prepregnancy BMI (3.36%), smoking (0.89%), GD (4.30%), cesarean delivery (5.15%), Apgar score (4.74%), NICU admission (4.74%), and birth weight for gestational age categories (4.76%) (eAppendix and eTable 1 in Supplement 1). Analyses were conducted using SAS/STAT statistical software version 9.4 (SAS Institute Inc) from January 2022 to May 2023. health care model (T1), 34 799 (23.0%) were partially exposed (T2), and 40 829 (26.9%) were fully exposed (T3). There were negligible differences in the sociodemographic and clinical characteristics between individuals who were partially exposed or fully exposed compared with those who were unexposed ( Table 1). SMD T3 vs T1, 0.649) ( Table 2). The majority of telemedicine visits were performed by telephone and both the mean number of telephone visits and video conferencing visits increased across the 3 intervals (Table 2). ITS analysis showed that there was no change in the mean number of telemedicine visits per individual during T1 (change per 4-week interval, −0.05%; 95% CI, −0.14% to 0.04%) followed by an increase during T2 (change per 4-week interval, 9.75%; 95% CI, 9.49% to 10.00%) and a decrease during T3 (change per 4-week interval, −13.10%; 95% CI, −13.40% to −12.90%) (Figure 1 and Table 3).

Prenatal Visits and Care Process
There were no clinically notable differences in the mean number of blood pressure measurements, rates of GD, or depression screenings across the 3 time intervals (Table 2). ITS analysis found that there were decreases in the mean number of blood pressure measurements per individual during T1 (change per 4-week interval, −0.10%; 95% CI, −0.13% to −0.08%) and T2 (change per 4-week interval, −0.68%; 95% CI, −0.74% to −0.61%) that were followed by an increase during T3 (change per 4-week interval, 1.32%; 95% CI, 1.23% to 1.41%) ( Figure 1 and Table 3). The screening rate for depression was stable during T1, followed by a decrease during T2 (change per 4-week interval, −0.27%; 95% CI, −0.37% to −0.16%) and an increase during T3 (change per 4-week interval, 0.78%; 95% CI, 0.64% to 0.93%) (eFigure 1 in Supplement 1). There were no changes across the 3 intervals for the GD screenings (Table 3 and Figure 1). Similar outcomes in prenatal visits (totals and by modality) and care processes were observed across individuals of different races and ethnicities, among individuals in the highest NDI quartile, among individuals who preferred a language other than English, among those living in a rural area, and those with a low-risk pregnancy (eTables 2-13 in Supplement 1).

Primary Health Outcomes
As shown in Table 2, rates of primary health outcomes were similar across the 3 intervals. Small changes were observed for rates of NICU admission, which decreased from T1 (7014 admissions   (Figure 2).

Secondary Health Outcomes
Rates of gestational hypertension, GD, and depression slightly increased from T1 to T2 and from T2 to T3 (Table 2). ITS analysis (eFigure 2 in Supplement 1 and Overall, similar rates of prenatal visits and care processes and primary and secondary health outcomes were found among individuals in the highest NDI quartile, among those who preferred a language other than English, lived in a rural area, had a low-risk pregnancy, and among individuals of different races and ethnicities, with a few exceptions (eTables 2-13 in Supplement 1). Among Black

Discussion
In this cohort study in an integrated health care delivery system with prepaid insurance, after implementing a multimodal prenatal health care model with use of both in-office and telemedicine visits during the COVID-19 pandemic, the mean number of prenatal visits via telemedicine increased to 21.3% of the total visits per individual compared with 11.1% of total visits during the prepandemic era. There were no clinically significant differences in prenatal health care process measures such as blood pressure measurements, GD, or depression screenings across pandemic periods (T2 and T3).
Most importantly, there were no clinically significant changes in most of our primary health outcomes, including preeclampsia and eclampsia, severe maternal morbidity, cesarean delivery, and preterm birth, with the exception of NICU admission rates. Although NICU admission rates were stable from the prepandemic period to the pandemic period, ITS analysis showed a decrease in risk during the first pandemic period (T2) and an increase during the second pandemic period (T3) when individuals were fully exposed to the multimodal prenatal health care model. Possible reasons for this increase in the risk of NICU admission rates require further investigation. Regarding secondary health outcomes, although the rates of gestational hypertension and depression increased from the prepandemic era through the pandemic era, ITS analysis showed that there were already increases in the risk of these conditions during the prepandemic period. Our results were consistent with a study 25   Unexposed deliveries Partially exposed deliveries Fully exposed deliveries COVID-19-related changes in perinatal health care Unexposed deliveries Partially exposed deliveries Fully exposed deliveries COVID-19-related changes in perinatal health care Unexposed deliveries Partially exposed deliveries Fully exposed deliveries COVID-19-related changes in perinatal health care The figure shows mean prenatal visits per individual per week of birth delivery (A), mean blood pressure measurements per individual per week of birth delivery (B), the percentage of individuals screened for gestational diabetes per week of birth delivery (C), and rates of neonatal intensive care unit admission (D) among 151 464 individuals. Vertical lines indicate periods of exposure to the multimodal health care model. In the first period, individuals with birth delivery between July 1, 2018, and February 29, 2020, were unexposed to the model; in the second period, individuals with birth delivery between March 1, 2020, and December 5, 2020, were partially exposed to the model; and in the third period, individuals with birth delivery between December 6, 2020, and October 31, 2021, were fully exposed to the model.    Abbreviations: T1, birth delivery between July 1, 2018, and February 29, 2020; T2, birth delivery between March 1, 2020, and December 5, 2020; T3, birth delivery between December 6, 2020, and October 31, 2021. a Interrupted time series method for prenatal outcomes before and during the implementation of COVID-19related changes in prenatal health care used piecewise linear segmented regression, allowing a change in slope associated with time at 2 inflection points: March 1, 2020 (initiation of changes in prenatal health care), and December 6, 2020 (date when pregnancies were fully exposed to the prenatal health care changes). Log binomial regression was used to examine each binary outcome in relation to date of birth delivery (week), providing point and interval estimates of relative percentage change in outcome probability associated with change in time, scaled to a 4-week change. Poisson regression was used for point and interval estimation of relative percentage change in means for outcomes characterized as counts in relation to a 4-week change in birth delivery date.
b Individuals with delivery in T2 did not experience the multimodal prenatal health care model implemented during COVID-19 pandemic for the entire duration of the pregnancy. Individuals with delivery in T3 experienced the multimodal prenatal health care model for the entire duration of the pregnancy.  The figure shows rates of preeclampsia and eclampsia (A), severe maternal morbidity (B), cesarean delivery (C), and preterm birth (D) among 151 464 individuals.
Vertical lines indicate periods of exposure to the multimodal health care model. In the first period, individuals with birth delivery between July 1, 2018, and February 29, 2020, were unexposed to the model; in the second period, individuals with birth delivery between March 1, 2020, and December 5, 2020, were partially exposed to the model; and in the third period, individuals with birth delivery between December 6, 2020, and October 31, 2021, were fully exposed to the model. found no differences in the weekly rates of GD, preeclampsia, preterm birth, or NICU admission rates between the 2 study periods. Our results were also consistent with 2 studies 26,27 conducted in the US with smaller numbers (ie, 12 607 and 1894 pregnant individuals) that used a before-and-after design to compare individuals who delivered during the COVID-19 pandemic and received a hybrid health care model of in-office and telemedicine with individuals who delivered before the pandemic and received in-office prenatal visits only. Rates of preeclampsia and cesarean delivery did not differ between groups. 26,27 However, none of these studies assessed whether performance of these hybrid health care models was similar across sociodemographic factors.
It is worth noting we observed similar results among individuals living in areas with a low NDI index, who preferred a language other than English, or were living in a rural area. Similar results were also obtained among individuals of different races and ethnicities, with a few exceptions. The most relevant differences compared with the whole cohort were those that showed an increase in the risk among those fully exposed to the multimodal health care model (ie, during T3) such as increases in the risk of severe maternal morbidity among Black individuals and increases in the risk of a 5-minute Apgar score of less than 7 among Asian and Pacific Islander individuals. These findings need further investigation to understand whether they were directly associated with the multimodal prenatal health care model or exposure to inequitable conditions within or outside of the health care system, or whether they might be chance findings in the context of the large number of regression analyses we performed.
Prenatal health care delivery is currently under evaluation and redesign as a result of the COVID-19 pandemic. 1 Integrating the use of telemedicine in prenatal health care provides an alternative to exclusive in-office care for populations living in underserved areas or facing access barriers and has received both promising attention and some concern. 2 Although populations living in underserved areas are also at risk for health disparities due to less access to technology and possible language barriers, as shown in the general population, 28-30 we did not observe differences in the uptake of telemedicine according to sociodemographic characteristics, and qualitative assessments of telemedicine programs show that pregnant individuals and their health care clinicians in rural or urban regions of the US perceive telemedicine as a positive experience. 2,31-34

Strengths and Limitations
Strengths of this study are the cohort design, inclusion of a large number of pregnant individuals and newborns, the ability to ascertain prenatal outcomes using robust longitudinal clinical data, the representativeness of the study sample and its racial and ethnic diversity, and use of ITS analysis, a rigorous study design for evaluating the outcomes of natural experiments, which accounted for partial and full exposure to the multimodal prenatal health care model implemented during the COVID-19 pandemic. Another strength of the study was its ability to assess the uptake and outcomes of the multimodal prenatal health care model by several sociodemographic factors.
Limitations of this study include different levels of exposure to the multimodal prenatal health care model among individuals who delivered during the first pandemic period (T2); therefore, analyses of some outcomes were subject to possible exposure misclassification. However, analyses of outcomes during the second pandemic period (T3), when pregnancies were homogeneously and fully exposed to the multimodal prenatal health care model, were subject to no exposure misclassification. During the first pandemic period (T2), lockdown policies and reduced individual comfort levels with being in person may also have affected complete assessment of the health outcomes. We were not able to accurately define pregnancies at low-risk or with specific moderaterisk conditions, since these definitions also include clinician discretion, for which we do not have complete documentation. Furthermore, we did not have data on patients' and health care clinicians' satisfaction with telemedicine.
This study has several characteristics that ensure the generalizability of its results in similar health care settings given the representativeness of the KPNC pregnant population, 4-6 which was also racially and geographically diverse. Although prenatal health care is covered without cost sharing