Trends in Medicaid Enrollment and Disenrollment During the Early Phase of the COVID-19 Pandemic in Wisconsin

This cohort study examines whether maintenance of eligibility requirements or employment shocks are associated with increased Medicaid enrollment during the first 7 months of the COVID-19 pandemic in Wisconsin.


eAppendix. Additional Details for Models
Additional Details for Model 1 As described in brief in the text, we estimate the model using data from 2015-2017 and predict enrollment from 2018-2020, using 2018 and 2019 as test periods.
To adjust for any differences in the composition of enrollees between our testing and prediction cohorts (those enrolled as of March 2018, 2019, and 2020) and our estimation cohort (those enrolled as of March 2017), we implement nearest neighbor propensity score matching to select samples from the 2017 cohort that is similar to the 2018, 2019, and 2020 cohorts. On the individual data, we estimate logit models with controls for gender, a quadratic in age, eligibility category (parent, child, pregnant woman, childless adult, or other), a quadratic in months of continuous enrollment, race (indicators for white, Black, Asian, Pacific Islander, American Indian, and missing information), Hispanic ethnicity, income, high school degree, citizenship, tribe member, and metropolitan county of residence that reflect the probability of being enrolled in March 2018, 2019, or 2020 relative to March 2017. The results of the logit model are presented in eTable 1. Each column 1 to 3 presents the probability of belonging to the sample in March 2018, 2019, and 2020, respectively, relative to March 2017. We then use these model estimates to construct the propensity score associated with being a member of the March 2018, 2019, and 2020 cohorts. We then select a nearest neighbor sample from the March 2017 cohort using these propensity scores to construct our estimation samples. Each year thus has a separate matched sample of March 2017 enrollees reflecting its demographics.
The first step in the model itself is estimating continued enrollment among those enrolled when the policy was in place (March 2020). We estimate the nonparametric (Kaplan-Meier) survival curve for individuals enrolled in March 2017 in each matched sample to generate nearest-neighbor propensity score matched versions of the survival curve for 2018-2020. This provides an estimate of the share of individuals initially enrolled in March that remained enrolled in subsequent months. These survival curves are listed in eTable 2 rounded to the nearest full percent. For example, the interpretation for someone enrolled in March 2018 would be: any individual enrolled in March 2018 has a probability of being enrolled in April 2018 of 97% and so on, based on estimates from the matching sample. Applying these probabilities to the number of enrollees in March of the corresponding year provides the number of enrollees that remain continuously enrolled.
Turnover, or churn, means that some individuals who exit in a month will re-enroll in subsequent months. Again using the matched sample of enrollees from the March 2017 cohort, we estimate the probability someone is from the March cohort is re-enrolled n months after an exit, with n=1, 2, …,12 (using a simple logit model and no additional controls). Using these probabilities we calculate the aggregate re-enrollment rates of the individuals that exit each month in each year and report these in eTable 3. For example, if a 2018 cohort individual exited at a given month, there is a 29% probability they will be enrolled again in Medicaid 2 months after the month they were not enrolled. We apply these probabilities to exiting enrollees each month.
Finally, to estimate new enrollment, we regress the number of new enrollees in each month (from April through February of the following year) on calendar month in 2015-2017 data. We then predict the average number of new enrollees by calendar month (e.g., average of Aprils in 2015, 2016, and 2017) as strong seasonal enrollment patterns make it important to control for time of year in some way. Since some of the new enrollees will also exit Medicaid, we apply survival functions to these new enrollees as well, estimated similarly as described above and presented in eTable 2. Survival functions for new enrollees tender to differ from the average cohort enrollee after the first six months but are otherwise very similar.
We then aggregate by month the number of predicted continued enrollees, new enrollees, and re-enrollees beginning with each March cohort in the test years (2018, 2019) and year of interest (2020) to come up with our total enrollment estimates.

Additional Details for Models 2-3
Models 2 and 3 augment Model 1 by incorporating information on the recent employment experience of new enrollees and continued enrollees. Survival curves and re-entry are estimated separately for those who have had a recent employment shock, defined as any member of the Medicaid case having a drop in UI wages of 50% or more from one quarter to the next, from Q1 to Q2 (March enrollees) or, for new enrollees, from the quarter prior to the quarter of their enrollment. We then predict total monthly new enrollment as a function of new unemployment claims and indicators for calendar month using 2017-2019 data from the Wisconsin Department of Workforce development (https://dwd.wisconsin.gov/uistats/). Because data on employment shocks are not observed for the second and third quarter of 2020 (the UI data are available only through Q2 2020), we need to make assumptions about the fraction of new enrollees with an employment shock on their case. We assume that it was 26% (as observed on average) through 2019 and remained elevated at 42% (as observed from Q1 to Q2 2020) throughout 2020.

Model Validation
eFigure 1 shows the fraction of enrollment predicted by the models for the six month time periods beginning April 2018, April 2019, and April 2020, where 1 means that the ratio of predicted to actual enrollment is one, or a perfect prediction. Throughout this period in 2018-2019, the models are generally off by a very small amount, with the largest absolute difference less than 3 percent. Model 1 tends to slightly overpredict total enrollment, less so in 2019 than in 2018, while Model 2 more closely matches actual enrollment in both years, slightly overpredicting in 2018 and slightly underpredicting in months 6-7 of 2019. The graph also shows the fraction of enrollment the models predict in 2020, which are immediately apparent as distinct from other years; even in month 2 when the usual margin of error is less than half a percent, Models 1-2 are already off by 1.8% and 1.2% respectively.
We also provide the Mean Absolute Percentage Error (MAPE) in addition to the Mean Absolute Deviation (MAD); both are standard forms of assessing the performance of a prediction model. The MAPE is 1.08 for Model 1 and 0.62 for Model 2. The MAD is 8,525 for Model 1 and 4,925 for Model 2.
As Model 3 is a simulation based on assumptions applied to the data only in the 2020 period using the inputs from Model 2, it has the same performance statistics as Model 2 in 2018-2019 and is excluded from eFigure1.

Method for generating prediction intervals:
We use a simulation approach to express the uncertainty in the enrollment predictions from the three models presented in Table 2 using 95% prediction intervals. These simulations incorporate two sources of uncertainty: estimation error and sampling error in the prediction. The estimation error arises because the estimates of the key model parameters, namely survival probabilities and re-enrollment probabilities, naturally contain the potential for error. The sampling error arises because even fixing a particular probability of survival or reenrollment for the population there will be randomness in how that survival/re-enrollment process plays out for the population.
To generate the prediction intervals on our estimates we create a simulation where each iteration re-calculates the projected enrollment numbers for each month (i.e., recreates the elements of Table 2). To incorporate the estimation error, the fixed survival probabilities and re-enrollment probabilities from our baseline estimates are each replaced with an independent random draw from a Normal Distribution with the mean set at the baseline probability estimate and the standard deviation set at the standard error on the mean estimated probability. In this way each of the probabilities applied in the model is allowed to vary in a range consistent with the estimated standard errors of that parameter. Then to account for sampling error, rather than simply multiplying those probability values by the starting sample sizes as we do in the baseline model, in the simulation we randomly draw enrollment numbers using the Binomial Distribution with the associate probability and sample size. Finally for the estimated number of newly enrolling members, we use the standard error on the predicted value from the regression model that generates those predictions and again draw independent random levels of new enrollment from a normal distribution with the mean set at the estimated level from the model and the standard deviation set to the standard error of the prediction model.
In each iteration the model draws random parameter levels for each monthly horizon, then draws an enrollment level from the associated binomial distribution. The final predictions are based on adding together these randomly drawn enrollment levels for different categories (e.g., continuing enrollment, re-enrolled members, etc...). We run the simulation for each of the models with 50,000 iterations. From those 50,000 iterations, we use the empirical 0.025 and 0.975 percentiles of the simulated enrollment numbers to establish our 95% prediction intervals for the model.

eFigure. Ratio of Predicted to Actual Enrollment by Model and Cohort-Month
Source: Authors' calculations from Wisconsin administrative data Notes: Figure shows ratio of model prediction to actual enrollment totals for test cohorts (2018, 2019) and for 2020 by month.
Model 1 is based only on enrollment projections. Model 2 incorporates information on recent employment shocks.