Effect of a Community Agency–Administered Nurse Home Visitation Program on Program Use and Maternal and Infant Health Outcomes

Key Points Question What is the effect of a nurse home visitation program for families with newborns implemented in a community setting on program penetration and fidelity and family outcomes? Findings This community-based randomized clinical trial found that the visitation program was implemented with 76% penetration and 90% adherence to the protocol, leading nurses to address minor problems for 52% of families and connect an additional 42% to community resources. Analyses of interviews and administrative records indicated that families assigned to the intervention had more community connections, fewer cases of maternal anxiety or depression, and fewer investigations for child abuse. Meaning This study demonstrates successful implementation and decreased child abuse investigations among participants in a postnatal nurse home visitation program implemented at scale in a community setting.


Introduction
Home visitation programs promoting healthy development in early childhood have proliferated based on promising empirical evidence. 1The federal Maternal, Infant, and Early Childhood Home-Visiting (MIECHV) Program 2 allocates $400 million annually to programs with positive impact demonstrated through randomized clinical trials (RCTs).Simultaneously, the field is challenged by failures to replicate findings when university-developed programs are implemented by community agencies, known as the scale-up penalty.The need for studies of implementation and impact of intervention in community settings is one of the highest priorities in medicine. 3e study we report here evaluated implementation and impact of the MIECHV-approved Family Connects (FC) program during its community dissemination following a university RCT. 4 The FC program is a systems-based approach to supporting families at birth by combining engagement and alignment of community resources with voluntary home visits for all families of newborns, including brief interventions, assessments of family needs, and connections to community resources to meet family-specific needs.5 Trained nurses offer mothers one to several home visits for support, clinical assessment, and referrals, as needed.They use motivational interviewing techniques to help families clarify needs in each of 12 empirically based risk domains and to facilitate referrals to community agencies.A community specialist aligns service agencies and makes available to nurses an annotated directory that includes service description, eligibility criteria, fees, and waiting lists.
Agencies include formal service providers (eg, substance abuse, maternal depression, parenting support); government and nonprofit programs (eg, housing assistance, childcare subsidies); and paraprofessional parent support groups.An integrated electronic data system documents visits, family needs, and outcomes of referrals to community services.The FC program is unique among MIECHV programs in emphasizing population impact, short-term duration, and low cost (about $500 per family).
A university-based RCT yielded 5 sets of empirical findings. 6,7First, the program penetrated a high proportion of the population.Second, the program was implemented with high fidelity.Third, the program achieved its proximal goal of connecting families with community services.Fourth, intervention-assigned families displayed more positive parent mental health and parenting behaviors at age 6 months than control parents.Fifth, hospital records at ages 6 months and 12 months showed that intervention-assigned infants accessed less emergency health care than control infants.
Following the first trial, FC began dissemination to communities across the country. 8

Participants, Intervention Assignment, and Evaluation Design
All procedures were approved by the Duke Medicine institutional review board.Written informed consent was required of intervention and control participants for outcome evaluation data collection.
Intervention families were required to provide consent to participate in the intervention and to allow access to intervention implementation data. 2 for risk that is addressed by a brief intervention; 3 for significant ongoing risk that requires a connection with a community resource to resolve; and 4 for an emergency requiring crisis intervention.For domains scored as 2, the nurse attempted prescribed brief interventions.For domains scored as 3, the nurse facilitated a connection to a community resource for ongoing intervention.The second component of intervention was alignment of community resources to serve families at birth through an electronic directory of agencies.The third component was an integrated data system that documented all services and assessments.
To examine impact of random assignment to intervention, a research assistant blind to experimental condition and unaffiliated with the community agency solicited written informed consent to participate in a research study of infant development, consisting of a single in-home interview at age 6 months (range 4-8 months).These families were blind to the goal of evaluating an intervention.Of the 456 odd-date births, 302 were located and confirmed as eligible (based on Durham residency at age 6 months), and 185 (61% of eligible) completed interviews.Of the 480 even-date births, 303 were located and confirmed as eligible residents, and 182 (60%) completed interviews.For the 15 families with twins, 1 member of each pair was randomly excluded from analysis.Administrative data indicated that 51 families had participated in FC for a prior birth (intervention and control conditions), and these families were excluded from analysis.The final sample included 316 infants (158 intervention and 158 control).
Measures of FC penetration and quality were taken from case records.Baseline covariates for consenting families were collected from hospital discharge records, and measures of intervention impact were taken from parent interviews and administrative records.

Measures of Baseline Variables
Number of infant birth risks ranged from 0 to 5 and included the following: (1)

Measures of Program Implementation
Records documented the number of families who initiated FC participation and the number who completed the program.Adherence to the manualized protocol was assessed by having an independent expert accompany the nurse on a home visit for 19 randomly selected families and check adherence to each model element.Reliability of scoring of risk between the nurse and the expert was measured by Cohen κ, 9 which corrects for chance agreement.Clinical records were inspected to score (1) the proportion of families with at least 1 identified need and referral to a community agency (scored as 3 for any of the 12 domains of need); and (2) the proportion of families with identified needs for whom community services were initiated within 1 month after referral.

Measures of the Primary Outcome
The primary outcome as listed in the pretreatment registry was child protective services reports for possible child maltreatment (abuse or neglect).State administrative records were examined for all consenting participants for the period from birth through age 24 months to code the number of documented investigations for maltreatment per infant.Reports could also be coded as "substantiation" if proof of maltreatment was documented, but no cases were so coded (primarily owing to a reform that referred cases to clinical services instead of prosecution and therefore did not code documentation as substantiation), so the substantiation variable was not tested.

Measures of Secondary Outcomes
During the impact interview conducted at infant age 6 months, mothers were asked to name all community resources they had used in the past 3 months, including professional, paraprofessional, and informal resources.The number of community agencies reported was used to score family connections to community resources.
During the interview at infant age 6 months, the mother completed the Edinburgh Postnatal Depression Scale 10 to detect possible clinical depression (the cut point for depression was a score >10) and the brief Generalized Anxiety Disorder-7 questionnaire 11 to detect possible clinical anxiety (the cut point for anxiety was a score >5).Maternal mental health problems were scored as 1 if either clinical problem was identified; otherwise, 0.
Parenting decisions and behaviors were assessed by maternal report of positive and negative parenting behaviors [12][13][14] ; the modified 21-item Infant Intentionality Questionnaire, which assesses maternal beliefs about intentional infant behaviors 15 ; a father-infant relationship quality scale 16 ; and a childcare questionnaire. 17We scored 5 variables: number of positive parenting behaviors, number of negative parenting behaviors, mother negative attribution of infant intentionality, father-infant relationship quality, and parental decision to use nonparental out-of-home childcare, which was almost always high quality (coded no = 0; yes = 1).
Mothers reported whether the family had complied with a well-baby physician appointment within the past month (coded no = 0; yes = 1

Missing Data
Nineteen participants had at least 1 missing value in a baseline variable.Tests using the Little method indicated that missing cases were not missing completely at random 18 (χ 2 = 45.1;P = .001).Multiple imputation was used to create 10 data sets in which missing values were imputed from other birth characteristics.Imputed data files were subjected to statistical tests, and estimates were merged. 18,19

Statistical Analysis
We next examined equivalence across experimental groups.In between the intervention (column 4) and control (column 5) evaluation-participating groups and found 2 of 11 variables to be significant.We accounted for pretreatment differences by conducting ordinary least-squares multivariate regression analyses predicting outcomes from random assignment to intervention, with 8 baseline covariates.We also calculated interaction terms between intervention status and the number of birth risk factors.We used linear regression models for continuous outcomes, logistic regression models for binary outcomes, and Poisson models for count variables.Results are reported in Table 2 for all variables, with exact 2-tailed P level and 95% and 90% confidence intervals.The text reports variables for which the P value is less than .10.

Program Implementation
Program Penetration Among intervention-assigned families, 76% were successfully approached, gave informed consent, and began participation in FC, compared with 80% in the university trial.Among those who began participation, 82% completed the program, compared with 86% in the first trial.

Program Quality and Family Satisfaction
Adherence to the FC protocol was 90%, which was higher than the 84% scored in the university trial.
Independent rater agreement in scoring the 12 risk factors was κ = 0.75, which was higher than κ = 0.69 achieved in the first trial.Nurses identified and addressed minor family needs for 52% of all possible cases, compared with 49% in the university trial.All 192 family members responded that they found the FC experience helpful.

Family Connections to Community Resources
Nurses identified major family needs (scored as 3) and tried to initiate community referral and service for 42% of all families, compared with 45% in the first trial.According to the 1-month follow-up inquiries, nurses were successful in getting high-need families to initiate services in 83% of all cases, compared with 79% in the university trial..03 a The P value of column 2 comes from χ 2 or t test between columns 1 and 2.
b The P value of column 3 comes from χ 2 or t test between columns 1 and 3.
c The P value of column 5 comes from χ 2 or t test between columns 4 and 5.

Program Impact Primary Outcome
Table 2 lists group means and test results for ordinary least-squares analyses of intervention main effects and interactions for all variables.The intervention group had a mean (SD) rate of child protective services investigations of 0.10 (0.30), and the control group had a mean (SD) rate of 0.18 (0.56) (b = −0.09;90% CI, −0.01 to −0.12; 95% CI, −0.18 to 0.01; P = .07).

Impact on Secondary Outcomes
Families assigned to the intervention group had a mean (SD) of The interaction effect between intervention status and the number of birth risks indicated that assignment to intervention was associated with fewer emergency department visits but only as the number of birth risks increased (b = −0.26;P = .02).This interaction effect is depicted in Figure 2.
For hospital overnights, the interaction effect indicated a higher rate for FC-assigned infants than control infants, but only as the number of birth risks increased.

Discussion
This randomized clinical trial found that when implemented by a community agency, the FC program achieved a high level of penetration to the full population of birthing families with a high degree of fidelity to the program model.It achieved its proximal goals of identifying family needs, responding to those needs with high parental satisfaction, and connecting families to community resources for major needs.Families randomly assigned to FC had 44% lower rates of investigation for child maltreatment by child protective services.The FC-assigned families had more connections to community resources and better mental health.Not all analyses confirmed program goals or met statistical significance criteria, indicating the need for caution and continued evaluation of this program as it is disseminated to new communities.
As a public health program aiming for population impact, the first goal of FC is to reach as many families giving birth as possible.The community agency was able to reach about as many families as had been reached in the university trial with similar follow-through to program completion, despite forces cited by Welsh et al 20 that would otherwise promote degradation.The second goal of FC is to deliver the program with high adherence and fidelity to the manualized protocol.The community agency was able to reach similar quality as exhibited in the university trial.
The third goal of FC is to identify and respond to clinically assessed family needs with connections to community resources.Nurses were able to identify needs and respond through brief interventions or connections to community resources, and have those connections continue for at least 6 months at rates that were similar to those in the prior trial.This function might be called primary psychosocial care, similar to the primary health care function served by pediatricians.Wellbaby pediatric health care is universally delivered, assessments are completed, modest interventions respond to clinically identified needs, and referrals are made to specialists when warranted.The FC intervention is a step toward a new system of universal primary psychosocial care for families of newborns.
The primary hypothesis guiding this study is that if the FC program can reach most families with high quality and connect them to community supports, then impact will be observed on rates of child maltreatment.We found positive impact on lowering official rates of child maltreatment investigations, by 44%.
It was secondarily hypothesized and found that the FC program had a positive impact on maternal mental health, replicating the previous trial.In contrast with the previous trial, there was no  impact on parenting behavior or nonparental care.Further scrutiny is necessary to understand whether subtle degradation in the aspect of the program related to parenting behavior occurred and could be corrected or the generalizable impact on parenting behavior is fragile.The impact of the FC program on infant and maternal health care utilization was mixed.As birth risk increased, assignment to FC was associated with fewer emergency department visits and more hospital overnights.This pattern was not hypothesized, nor was it found in any other trial, and it may not be reliable.Finally, assignment to FC was associated with higher maternal emergency department visit rates, possibly reflecting greater attentiveness by mothers to their own health needs as a function of FC.

Limitations
This study has some limitations.The findings of this study highlight both replicated and nonreplicated effects of the FC program when implemented by a community agency.One limitation of the study is that it occurred in only 1 community; thus, it will be important to conduct additional studies with new communities, contexts, and populations.Another limitation is that the active ingredients and mediational processes through which the intervention has impact were not tested and are left to further inquiry.A third limitation is that the evaluation was conducted by the same team of investigators that evaluated the first trial; independent evaluators should be identified for future trials.

Conclusions
This trial indicates that it is possible to transfer delivery of a university-based universal postnatal nurse home visitation program to community leadership while maintaining high program penetration, quality, and impact.The FC program substantially reduced child maltreatment investigations through the first 24 months of life.We recommend continued dissemination of this public health program in new communities when accompanied by ongoing implementation evaluation and, when possible, impact evaluation.
Concern about possible scale-up penalties on program implementation and impact motivated an RCT of FC when implemented by a community agency.After the first trial, the administration of FC in Durham, North Carolina, transferred to a local nonprofit entity.Funding was provided by county government, Medicaid reimbursement, and philanthropy.Hiring and supervision of nurses and implementation of the program was administered by the local community agency.This study tested 5 assertions undergirding the logic of FC: (1) FC can penetrate a large proportion of the birthing community; (2) FC JAMA Network Open | Pediatrics

Figure 2 .
Figure 2. Interaction Between Intervention Status and the Number of Birth Risks for the Cumulative Number of Emergency Department Visits by Age 12 Months 1.4 Community Agency-Administered Nurse Home Visitation Program and Family Outcomes JAMA Network Open.2019;2(11):e1914522. doi:10.1001/jamanetworkopen.2019.14522(Reprinted)November 1, 2019 2/11 Downloaded From: https://jamanetwork.com/ on 09/28/2023 can be delivered with high quality;(3) families assigned to FC will become more connected to community resources; (4) families assigned to FC will demonstrate better parent mental health and parenting; and (5) families assigned to FC will have lower rates of investigation for child maltreatment and better infant and maternal health care participation and outcomes.

456 Allocated to intervention 286 Received allocated intervention 170 Did not receive allocated intervention 83 Declined intervention 15 Unable to contact 8 Unable to schedule intervention visit 2 Moved out of county after agreeing to participate in intervention 62 Nurse unable to complete intervention visit 480 Allocated to services as usual 117 Did not participate 185 Completed interview 27 Excluded because family received FC for prior birth 158 Analyzed for program impact
Figure 1 provides the CONSORT flow diagram of participants through the trial.From January 1, 2014, through June 30, 2014, of the 977 hospital discharge records that were examined, all 936 county resident births at Duke University Hospital constituted the possible population for assignment to intervention.This sample size was determined based on minimal size needed to detect hypothesized effects.Identical to the design of the first FC RCT, 4 random assignment to intervention was based on date of birth.Intervention staff members at a community agency attempted to deliver FC to all 456 odd-date births.The 480 even-date births were not approached for FC and instead received treatment as usual (the reverse of the prior trial).

Downloaded From: https://jamanetwork.com/ on 09/28/2023
maltreatment, maternal depression and anxiety, parental substance abuse, and parental social support from others.Risk for each domain was scored on a 4-point scale: 1 for no risk;

Table 1
, the columns labeled 1 and 2 list group means for the entire population and the evaluation-eligible groups (ie, those infants still resident in the community and located at age 6 months).Of 11 baseline variables tested, none were significant.Next, we contrasted the evaluation-participating group (column 3) with the entire population (column 1) and found no significant difference across 11 tests.We then tested differencesJAMA Network Open | PediatricsCommunity Agency-Administered Nurse Home Visitation Program and Family Outcomes JAMA Network Open.2019;2(11):e1914522. doi:10.1001/jamanetworkopen.2019.14522(Reprinted) November 1, 2019 5/11 Downloaded From: https://jamanetwork.com/ on 09/28/2023

Table 2 .
Adjusted Group Means and Multivariate Regression Models Testing Main Effect of Treatment and Treatment × Birth Risk Interaction a All analyses conducted using 10 multiply imputed data sets to address missing data.All models included the baseline characteristics listed in Table1as covariates.= .04).Mothers in the intervention group had more hospital overnights but only as the number of birth risks increased (b = 0.20; P = .09).