Data represent regression-fitted point estimates for treatment and control groups for each level of pretreatment health risk defined by birth records. Risks included the following: (1) born at less than 2500 g; (2) less than 37 weeks’ gestational age; (3) birth complications, not specified; (4) substance exposure in utero; and (5) other risk noted, not specified.
Data Sharing Statement
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Dodge KA, Goodman WB, Bai Y, O’Donnell K, Murphy RA. Effect of a Community Agency–Administered Nurse Home Visitation Program on Program Use and Maternal and Infant Health Outcomes: A Randomized Clinical Trial. JAMA Netw Open. 2019;2(11):e1914522. doi:https://doi.org/10.1001/jamanetworkopen.2019.14522
What is the effect of a nurse home visitation program for families with newborns implemented in a community setting on program penetration and fidelity and family outcomes?
This community-based randomized clinical trial found that the visitation program was implemented with 76% penetration and 90% adherence to the protocol, leading nurses to address minor problems for 52% of families and connect an additional 42% to community resources. Analyses of interviews and administrative records indicated that families assigned to the intervention had more community connections, fewer cases of maternal anxiety or depression, and fewer investigations for child abuse.
This study demonstrates successful implementation and decreased child abuse investigations among participants in a postnatal nurse home visitation program implemented at scale in a community setting.
Postnatal home visitation to support parenting and infant healthy development is becoming increasingly common based on university efficacy studies, but effectiveness when disseminated by communities is not clear.
To test implementation and impact of the Family Connects (FC) program when administered by a community agency.
Design, Setting, and Participants
In this randomized clinical trial, births were randomly assigned to receive FC or treatment as usual. Independent evaluation was conducted through parent interviews and review of health and child protective services records. Interviewers were blind to the experimental condition of participants, and participants were blind about the purpose of the interview as an intervention evaluation. A total of 936 consecutive residential births at Duke University Hospital from January 1, 2014, through June 30, 2014, were included. Data were analyzed preliminarily for reporting to funders in early 2015 before all birth-record covariates were scored and were analyzed more comprehensively in mid-2019 after administrative birth and child protective service records became available.
The goals of the FC brief universal program were to assess family-specific needs, complete brief interventions, and connect families with community resources. Community agencies and families were aligned through an electronic data system.
Main Outcomes and Measures
Case records documented program penetration and quality. The primary outcome was child protective services investigations for maltreatment. Secondary outcomes were the number of sustained community connections, maternal mental health, parenting behavior, infant well-child care visits and maternal postpartum care compliance, and emergency health care utilization.
Of 936 births, 451 infants (48.2%) were female and 433 (46.3%) were from racial/ethnic minority groups. In all, 456 births (46.5%) were randomized to the intervention and 480 (53.5%) were randomized to the control. All analyses were based on intention to treat. The impact analysis included 158 intervention families and 158 control families. Intervention penetration was 76%, adherence to the protocol was 90%, and independent agreement in scoring (κ) was 0.75. Nurses identified and addressed minor problems for 52% of families and connected an additional 42% to community resources. Analysis of the primary outcome of child abuse investigations revealed a mean (SD) of 0.10 (0.30) investigations for the intervention group vs 0.18 (0.56) investigations for the control group (b = −0.09; 90% CI, −0.01 to −0.12; 95% CI, −0.18 to 0.01; P = .07). The intervention group’s rate of possible maternal anxiety or depression was 18.2% vs 25.9% for the control group (b = −7.70; 90% CI, −15.2 to −0.1; 95% CI, −16.6 to 1.3; P = .09).
Conclusions and Relevance
This study indicates that a nurse home visitation program for families of newborns can be implemented by a community agency with high penetration and quality. Other communities could benefit from wider dissemination of the program provided that quality remains strong and evaluation continues.
ClinicalTrials.gov identifier: NCT01843036
Home visitation programs promoting healthy development in early childhood have proliferated based on promising empirical evidence.1 The federal Maternal, Infant, and Early Childhood Home-Visiting (MIECHV) Program2 allocates $400 million annually to programs with positive impact demonstrated through randomized clinical trials (RCTs). Simultaneously, the field is challenged by failures to replicate findings when university-developed programs are implemented by community agencies, known as the scale-up penalty. The need for studies of implementation and impact of intervention in community settings is one of the highest priorities in medicine.3
The study we report here evaluated implementation and impact of the MIECHV-approved Family Connects (FC) program during its community dissemination following a university RCT.4 The FC program is a systems-based approach to supporting families at birth by combining engagement and alignment of community resources with voluntary home visits for all families of newborns, including brief interventions, assessments of family needs, and connections to community resources to meet family-specific needs.5 Trained nurses offer mothers one to several home visits for support, clinical assessment, and referrals, as needed. They use motivational interviewing techniques to help families clarify needs in each of 12 empirically based risk domains and to facilitate referrals to community agencies. A community specialist aligns service agencies and makes available to nurses an annotated directory that includes service description, eligibility criteria, fees, and waiting lists. Agencies include formal service providers (eg, substance abuse, maternal depression, parenting support); government and nonprofit programs (eg, housing assistance, childcare subsidies); and paraprofessional parent support groups. An integrated electronic data system documents visits, family needs, and outcomes of referrals to community services. The FC program is unique among MIECHV programs in emphasizing population impact, short-term duration, and low cost (about $500 per family).
A university-based RCT yielded 5 sets of empirical findings.6,7 First, the program penetrated a high proportion of the population. Second, the program was implemented with high fidelity. Third, the program achieved its proximal goal of connecting families with community services. Fourth, intervention-assigned families displayed more positive parent mental health and parenting behaviors at age 6 months than control parents. Fifth, hospital records at ages 6 months and 12 months showed that intervention-assigned infants accessed less emergency health care than control infants.
Following the first trial, FC began dissemination to communities across the country.8 Concern about possible scale-up penalties on program implementation and impact motivated an RCT of FC when implemented by a community agency. After the first trial, the administration of FC in Durham, North Carolina, transferred to a local nonprofit entity. Funding was provided by county government, Medicaid reimbursement, and philanthropy. Hiring and supervision of nurses and implementation of the program was administered by the local community agency. This study tested 5 assertions undergirding the logic of FC: (1) FC can penetrate a large proportion of the birthing community; (2) FC can be delivered with high quality; (3) families assigned to FC will become more connected to community resources; (4) families assigned to FC will demonstrate better parent mental health and parenting; and (5) families assigned to FC will have lower rates of investigation for child maltreatment and better infant and maternal health care participation and outcomes.
All procedures were approved by the Duke Medicine institutional review board. Written informed consent was required of intervention and control participants for outcome evaluation data collection. Intervention families were required to provide consent to participate in the intervention and to allow access to intervention implementation data. Figure 1 provides the CONSORT flow diagram of participants through the trial. From January 1, 2014, through June 30, 2014, of the 977 hospital discharge records that were examined, all 936 county resident births at Duke University Hospital constituted the possible population for assignment to intervention. This sample size was determined based on minimal size needed to detect hypothesized effects. Identical to the design of the first FC RCT,4 random assignment to intervention was based on date of birth. Intervention staff members at a community agency attempted to deliver FC to all 456 odd-date births. The 480 even-date births were not approached for FC and instead received treatment as usual (the reverse of the prior trial). Implementation performance was evaluated for the 456 odd-date births. The full trial protocol is available in Supplement 1. Reporting followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.
Implementation followed a detailed manual4 that included 3 major components. The first was direct intervention with a family beginning in the birthing hospital and continuing with 1 to 3 postpartum home visits. A nurse assessed and scored family need for intervention for each of 12 key domains assessed during home visits: parent health, infant health, medical home, child care planning, parent-infant relationship, management of infant crying, material supports, family violence, mother’s past experience of maltreatment, maternal depression and anxiety, parental substance abuse, and parental social support from others. Risk for each domain was scored on a 4-point scale: 1 for no risk; 2 for risk that is addressed by a brief intervention; 3 for significant ongoing risk that requires a connection with a community resource to resolve; and 4 for an emergency requiring crisis intervention. For domains scored as 2, the nurse attempted prescribed brief interventions. For domains scored as 3, the nurse facilitated a connection to a community resource for ongoing intervention. The second component of intervention was alignment of community resources to serve families at birth through an electronic directory of agencies. The third component was an integrated data system that documented all services and assessments.
To examine impact of random assignment to intervention, a research assistant blind to experimental condition and unaffiliated with the community agency solicited written informed consent to participate in a research study of infant development, consisting of a single in-home interview at age 6 months (range 4-8 months). These families were blind to the goal of evaluating an intervention. Of the 456 odd-date births, 302 were located and confirmed as eligible (based on Durham residency at age 6 months), and 185 (61% of eligible) completed interviews. Of the 480 even-date births, 303 were located and confirmed as eligible residents, and 182 (60%) completed interviews. For the 15 families with twins, 1 member of each pair was randomly excluded from analysis. Administrative data indicated that 51 families had participated in FC for a prior birth (intervention and control conditions), and these families were excluded from analysis. The final sample included 316 infants (158 intervention and 158 control).
Measures of FC penetration and quality were taken from case records. Baseline covariates for consenting families were collected from hospital discharge records, and measures of intervention impact were taken from parent interviews and administrative records.
Number of infant birth risks ranged from 0 to 5 and included the following: (1) born at less than 2500 g; (2) less than 37 weeks’ gestational age; (3) birth complications, not specified; (4) substance exposure in utero; and (5) other risk noted, not specified. Cesarean delivery was scored as 1 if yes; otherwise, 0. Multiple births was scored as 1 if yes; otherwise, 0. Adolescent mother was scored as 1 if the mother was aged 19 years or younger; otherwise, 0. Single parent status was scored as 1 if the mother was not married or did not have a residential partner; otherwise, 0. Family Medicaid or no insurance status was scored as 1 if yes; otherwise, 0. Infant sex was scored from the birth record as 1 if female; 0 if male. Infant race/ethnicity was scored from classifications made on the birth record as 1 if minority; otherwise, 0 (additional analyses scored 4 dichotomous variables for race/ethnicity as white, African American, Hispanic, or other).
Records documented the number of families who initiated FC participation and the number who completed the program. Adherence to the manualized protocol was assessed by having an independent expert accompany the nurse on a home visit for 19 randomly selected families and check adherence to each model element. Reliability of scoring of risk between the nurse and the expert was measured by Cohen κ,9 which corrects for chance agreement. Clinical records were inspected to score (1) the proportion of families with at least 1 identified need and referral to a community agency (scored as 3 for any of the 12 domains of need); and (2) the proportion of families with identified needs for whom community services were initiated within 1 month after referral. Family consumer satisfaction with the experience of FC was measured during a follow-up telephone call 1 month after case closure. The interviewer asked the parent whether they found the experience helpful (scored 1) or not (scored 0).
The primary outcome as listed in the pretreatment registry was child protective services reports for possible child maltreatment (abuse or neglect). State administrative records were examined for all consenting participants for the period from birth through age 24 months to code the number of documented investigations for maltreatment per infant. Reports could also be coded as “substantiation” if proof of maltreatment was documented, but no cases were so coded (primarily owing to a reform that referred cases to clinical services instead of prosecution and therefore did not code documentation as substantiation), so the substantiation variable was not tested.
During the impact interview conducted at infant age 6 months, mothers were asked to name all community resources they had used in the past 3 months, including professional, paraprofessional, and informal resources. The number of community agencies reported was used to score family connections to community resources.
During the interview at infant age 6 months, the mother completed the Edinburgh Postnatal Depression Scale10 to detect possible clinical depression (the cut point for depression was a score >10) and the brief Generalized Anxiety Disorder–7 questionnaire11 to detect possible clinical anxiety (the cut point for anxiety was a score >5). Maternal mental health problems were scored as 1 if either clinical problem was identified; otherwise, 0.
Parenting decisions and behaviors were assessed by maternal report of positive and negative parenting behaviors12-14; the modified 21-item Infant Intentionality Questionnaire, which assesses maternal beliefs about intentional infant behaviors15; a father-infant relationship quality scale16; and a childcare questionnaire.17 We scored 5 variables: number of positive parenting behaviors, number of negative parenting behaviors, mother negative attribution of infant intentionality, father-infant relationship quality, and parental decision to use nonparental out-of-home childcare, which was almost always high quality (coded no = 0; yes = 1).
Mothers reported whether the family had complied with a well-baby physician appointment within the past month (coded no = 0; yes = 1). Infant patient records were accessed to score the number of emergency department visits by age 12 months and number of overnight stays in the hospital by age 12 months.
Three variables measured mother’s health care utilization: mother’s report at infant age 6 months assessed mother’s compliance with 6-week postpartum physician visit (coded no = 0; yes = 1). Administrative records were accessed to score the mother’s number of emergency department visits by infant age 12 months and the mother’s number of overnight hospital stays by infant age 12 months.
Nineteen participants had at least 1 missing value in a baseline variable. Tests using the Little method indicated that missing cases were not missing completely at random18 (χ2 = 45.1; P = .001). Multiple imputation was used to create 10 data sets in which missing values were imputed from other birth characteristics. Imputed data files were subjected to statistical tests, and estimates were merged.18,19
We next examined equivalence across experimental groups. In Table 1, the columns labeled 1 and 2 list group means for the entire population and the evaluation-eligible groups (ie, those infants still resident in the community and located at age 6 months). Of 11 baseline variables tested, none were significant. Next, we contrasted the evaluation-participating group (column 3) with the entire population (column 1) and found no significant difference across 11 tests. We then tested differences between the intervention (column 4) and control (column 5) evaluation-participating groups and found 2 of 11 variables to be significant. We accounted for pretreatment differences by conducting ordinary least-squares multivariate regression analyses predicting outcomes from random assignment to intervention, with 8 baseline covariates. We also calculated interaction terms between intervention status and the number of birth risk factors. We used linear regression models for continuous outcomes, logistic regression models for binary outcomes, and Poisson models for count variables. Results are reported in Table 2 for all variables, with exact 2-tailed P level and 95% and 90% confidence intervals. The text reports variables for which the P value is less than .10.
Among intervention-assigned families, 76% were successfully approached, gave informed consent, and began participation in FC, compared with 80% in the university trial. Among those who began participation, 82% completed the program, compared with 86% in the first trial.
Adherence to the FC protocol was 90%, which was higher than the 84% scored in the university trial. Independent rater agreement in scoring the 12 risk factors was κ = 0.75, which was higher than κ = 0.69 achieved in the first trial. Nurses identified and addressed minor family needs for 52% of all possible cases, compared with 49% in the university trial. All 192 family members responded that they found the FC experience helpful.
Nurses identified major family needs (scored as 3) and tried to initiate community referral and service for 42% of all families, compared with 45% in the first trial. According to the 1-month follow-up inquiries, nurses were successful in getting high-need families to initiate services in 83% of all cases, compared with 79% in the university trial.
Table 2 lists group means and test results for ordinary least-squares analyses of intervention main effects and interactions for all variables. The intervention group had a mean (SD) rate of child protective services investigations of 0.10 (0.30), and the control group had a mean (SD) rate of 0.18 (0.56) (b = −0.09; 90% CI, −0.01 to −0.12; 95% CI, −0.18 to 0.01; P = .07).
Families assigned to the intervention group had a mean (SD) of 5.72 (2.37) community connections, and the control group had a mean (SD) of 5.04 (2.69) community connections (b = 0.69; 90% CI, 0.27-1.11; 95% CI, 0.19-1.19; P = .01). The rate of maternal mental health problems was 18.2% for the intervention group and 25.9% for the control group (b = −7.70; 90% CI, −15.2 to −0.1; 95% CI, −16.6 to 1.3; P = .09). The use of nonparental childcare was 54.3% for the intervention group and 44.0% for the control group (b = 10.30; 90% CI, 1.42 to 19.23; 95% CI, −0.31 to 20.92; P = .06).
The interaction effect between intervention status and the number of birth risks indicated that assignment to intervention was associated with fewer emergency department visits but only as the number of birth risks increased (b = −0.26; P = .02). This interaction effect is depicted in Figure 2. For hospital overnights, the interaction effect indicated a higher rate for FC-assigned infants than control infants, but only as the number of birth risks increased.
Mothers in the intervention group had a 6-week postpartum health care compliance rate of 90.3% vs the control group rate of 83.8% (b = 6.44; 90% CI, 0.6 to 12.3; 95% CI, −0.62 to 13.51; P = .07). Mothers in the intervention group had a mean (SD) number of emergency department visits of 0.40 (1.14) vs. 0.20 (0.64) for the control group (b = 0.21; 90% CI, 0.04-0.37; 95% CI, 0.01-0.40; P = .04). Mothers in the intervention group had more hospital overnights but only as the number of birth risks increased (b = 0.20; P = .09).
This randomized clinical trial found that when implemented by a community agency, the FC program achieved a high level of penetration to the full population of birthing families with a high degree of fidelity to the program model. It achieved its proximal goals of identifying family needs, responding to those needs with high parental satisfaction, and connecting families to community resources for major needs. Families randomly assigned to FC had 44% lower rates of investigation for child maltreatment by child protective services. The FC-assigned families had more connections to community resources and better mental health. Not all analyses confirmed program goals or met statistical significance criteria, indicating the need for caution and continued evaluation of this program as it is disseminated to new communities.
As a public health program aiming for population impact, the first goal of FC is to reach as many families giving birth as possible. The community agency was able to reach about as many families as had been reached in the university trial with similar follow-through to program completion, despite forces cited by Welsh et al20 that would otherwise promote degradation. The second goal of FC is to deliver the program with high adherence and fidelity to the manualized protocol. The community agency was able to reach similar quality as exhibited in the university trial.
The third goal of FC is to identify and respond to clinically assessed family needs with connections to community resources. Nurses were able to identify needs and respond through brief interventions or connections to community resources, and have those connections continue for at least 6 months at rates that were similar to those in the prior trial. This function might be called primary psychosocial care, similar to the primary health care function served by pediatricians. Well-baby pediatric health care is universally delivered, assessments are completed, modest interventions respond to clinically identified needs, and referrals are made to specialists when warranted. The FC intervention is a step toward a new system of universal primary psychosocial care for families of newborns.
The primary hypothesis guiding this study is that if the FC program can reach most families with high quality and connect them to community supports, then impact will be observed on rates of child maltreatment. We found positive impact on lowering official rates of child maltreatment investigations, by 44%.
It was secondarily hypothesized and found that the FC program had a positive impact on maternal mental health, replicating the previous trial. In contrast with the previous trial, there was no impact on parenting behavior or nonparental care. Further scrutiny is necessary to understand whether subtle degradation in the aspect of the program related to parenting behavior occurred and could be corrected or the generalizable impact on parenting behavior is fragile. The impact of the FC program on infant and maternal health care utilization was mixed. As birth risk increased, assignment to FC was associated with fewer emergency department visits and more hospital overnights. This pattern was not hypothesized, nor was it found in any other trial, and it may not be reliable. Finally, assignment to FC was associated with higher maternal emergency department visit rates, possibly reflecting greater attentiveness by mothers to their own health needs as a function of FC.
This study has some limitations. The findings of this study highlight both replicated and nonreplicated effects of the FC program when implemented by a community agency. One limitation of the study is that it occurred in only 1 community; thus, it will be important to conduct additional studies with new communities, contexts, and populations. Another limitation is that the active ingredients and mediational processes through which the intervention has impact were not tested and are left to further inquiry. A third limitation is that the evaluation was conducted by the same team of investigators that evaluated the first trial; independent evaluators should be identified for future trials.
This trial indicates that it is possible to transfer delivery of a university-based universal postnatal nurse home visitation program to community leadership while maintaining high program penetration, quality, and impact. The FC program substantially reduced child maltreatment investigations through the first 24 months of life. We recommend continued dissemination of this public health program in new communities when accompanied by ongoing implementation evaluation and, when possible, impact evaluation.
Accepted for Publication: September 12, 2019.
Published: November 1, 2019. doi:10.1001/jamanetworkopen.2019.14522
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Dodge KA et al. JAMA Network Open.
Corresponding Author: Kenneth A. Dodge, PhD, Sanford School of Public Policy, Duke University, 302 Towerview Rd, Durham, NC 27708 (email@example.com).
Author Contributions: Dr Dodge had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Dodge, Goodman, O'Donnell, Murphy.
Acquisition, analysis, or interpretation of data: Dodge, Goodman, Bai, O'Donnell.
Drafting of the manuscript: Dodge.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Goodman, Bai.
Obtained funding: Dodge, Goodman, Murphy.
Administrative, technical, or material support: Dodge, Goodman, O'Donnell, Murphy.
Supervision: Dodge, O'Donnell.
Conflict of Interest Disclosures: Dr Dodge reported receiving grants and stipends from The Duke Endowment outside the submitted work. Dr Goodman reported receiving grants from the National Institute of Child Health and Human Development during the conduct of the study; and an honorarium and travel reimbursement for invited presentations at conferences hosted by the Japan Society for Prevention of Child Abuse and Neglect, the Future of Children, and the Child Care and Early Education Policy Research Consortium outside the submitted work. Dr Bai reported grants from The Duke Endowment during the conduct of the study. No other disclosures were reported.
Funding/Support: Funding was provided by The Duke Endowment, grant R01HD069981 from the Eunice Kennedy Shiver National Institute for Child Health and Human Development, and the Laura and John Arnold Foundation.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The views expressed are those of the authors and do not necessarily reflect the views of the funders.
Data Sharing Statement: See Supplement 2.
Additional Contributions: We acknowledge the contributions of many staff members and Durham community leaders in implementing Family Connects and its evaluation.
Create a personal account or sign in to: