eAppendix. Statistical Methods
eTable 1. Denver Sample Composition over Time by Treatment - Through Child Ages 6 and 9
eTable 2. Background Characteristics at Intake for Families in which Children Completed Age-6 and 9 Assessments
eTable 3. Primary Outcomes: Cognitive Domain – Children Born to Mothers with High Psychological Resources
eTable 4. Secondary Outcomes: Children Born to Mothers with High Psychological Resources
Olds DL, Holmberg JR, Donelan-McCall N, Luckey DW, Knudtson MD, Robinson J. Effects of Home Visits by Paraprofessionals and by Nurses on ChildrenFollow-up of a Randomized Trial at Ages 6 and 9 Years. JAMA Pediatr. 2014;168(2):114-121. doi:10.1001/jamapediatrics.2013.3817
The Nurse-Family Partnership delivered by nurses has been found to produce long-term effects on maternal and child health in replicated randomized trials. A persistent question is whether paraprofessional home visitors might produce comparable effects.
To examine the impact of prenatal and infancy/toddler home visits by paraprofessionals and by nurses on child development at child ages 6 and 9 years.
Design, Setting, and Participants
Randomized trial in public and private care settings in Denver, Colorado, of 735 low-income women and their first-born children (85% of the mothers were unmarried; 47% were Hispanic, 35% were non-Hispanic white, 15% were African American, and 3% were American Indian/Asian).
Home visits provided from pregnancy through child age 2 years delivered in one group by paraprofessionals and in the other by nurses.
Main Outcomes and Measures
Reports of children’s internalizing, externalizing, and total emotional/behavioral problems, and tests of children’s language, intelligence, attention, attention dysfunction, visual attention/task switching, working memory, and academic achievement. We hypothesized that program effects on cognitive-related outcomes would be more pronounced among children born to mothers with low psychological resources. We report paraprofessional-control and nurse-control differences with P < .10 given similar effects in a previous trial, earlier effects in this trial, and limited statistical power.
There were no significant paraprofessional effects on emotional/behavioral problems, but paraprofessional-visited children born to mothers with low psychological resources compared with control group counterparts exhibited fewer errors in visual attention/task switching at age 9 years (effect size = −0.30, P = .08). There were no statistically significant paraprofessional effects on other primary outcomes. Nurse-visited children were less likely to be classified as having total emotional/behavioral problems at age 6 years (relative risk [RR] = 0.45, P = .08), internalizing problems at age 9 years (RR = 0.44, P = .08), and dysfunctional attention at age 9 years (RR = 0.34, P = .07). Nurse-visited children born to low-resource mothers compared with control-group counterparts had better receptive language averaged over ages 2, 4, and 6 years (effect size = 0.30, P = .01) and sustained attention averaged over ages 4, 6, and 9 years (effect size = 0.36, P = .006). There were no significant nurse effects on externalizing problems, intellectual functioning, and academic achievement.
Conclusions and Relevance
Children born to low-resource mothers visited by paraprofessionals exhibited improvement in visual attention/task switching. Nurse-visited children showed improved behavioral functioning, and those born to low-resource mothers benefited in language and attention but did not improve in intellectual functioning and academic achievement.
clinicaltrials.gov Identifier: NCT00438282 and NCT00438594.
Home visiting by nurses for low-income, at-risk families has been promoted as an evidence-based strategy for improving the health and development of first-born children from low-income families.1,2 Three randomized trials of a program known as the Nurse-Family Partnership (NFP), conducted in Elmira, New York (with a primarily white sample),3- 6 Memphis, Tennessee (with a low-income African American sample),7- 11 and Denver, Colorado (with a large portion of Latina mothers),12,13 found replicated and enduring effects in at least 2 of the 3 trials on prenatal health,3,7,12, child health and development,4,5,7,9- 13 and maternal life course.6- 9,12,13 These trials have served as the primary evidentiary foundation for the Maternal Infant and Early Childhood Home Visiting Program funded under the Patient Protection and Affordable Care Act.14
The Denver NFP trial was designed to examine the impact of the program model when delivered by paraprofessional visitors who shared many of the social characteristics of the families they visited and to estimate the impact of the program with a sample that included a large portion of Latino families. Through child age 4 years, we found that the nurse-delivered version of the NFP in Denver produced effects on child development12,13 that were essentially limited to children born to mothers with low psychological resources. The moderation of program effects by mothers’ psychological resources replicated a pattern found in Memphis.9- 11 The benefits of the paraprofessional-delivered program estimated in earlier phases of the Denver trial were smaller and more sporadic.12,13 This article reports the results of 2 waves of follow-up of children in the Denver trial at ages 6 and 9 years.
We hypothesized that nurse-visitors in the current trial would produce effects like those in earlier phases of this trial12,13 and like those in Memphis.9- 11 We hypothesized that program effects on children’s cognitive-related outcomes (including dysregulated aggression and incoherent responses to story stems) would be limited to children born to mothers with low psychological resources and that program effects on children’s emotional and behavioral problems would be present irrespective of their mothers’ psychological resources.9,11 In Memphis, program effects on emotional and behavioral problems were expressed in reductions in children’s total behavioral problems at age 6 years and in internalizing problems at age 12 years using the Child Behavior Checklist9,11; a different measure was used to assess behavioral problems at age 9 years.10 Given previous trials of paraprofessional–home visitor programs15,16 and earlier results from this trial,12,13 we expected paraprofessionals to produce fewer and smaller effects than those produced by nurses.
The current study consisted of a follow-up of children around the child’s 6th and 9th birthdays. The major features of the design were reported earlier5,7 and are summarized herein.
From March 29, 1994, through June 15, 1995, 1178 consecutive women from 21 antepartum clinics serving low-income women in Denver were invited to participate in the study by research interviewers after being referred by clinic staff. Women were recruited if they had no previous live births and either qualified for Medicaid or had no private health insurance. Medicaid eligibility in Colorado at the time was extended to pregnant women with incomes at or below 133% of the federal poverty guidelines. The numbers of women and their children invited to participate, randomized, and assessed at the 6- and 9-year follow-ups are delineated in eTable 1 and the eAppendix in the Supplement. Participants provided written informed consent. The study was approved by the Colorado Multiple Institutional Review Board.
After completion of baseline interviews, interviewers sent information on participants to a separately located data-operations office where it was entered into a computer program that randomized women to treatment conditions.17 Randomization was conducted within strata from a model with 3 classification factors: maternal race/ethnicity (Hispanic, white non-Hispanic, African American, and American Indian/Asian), maternal gestational age at enrollment (<32 vs ≥32 weeks), and geographic region of residence (4 regions). Women assigned to 1 of the 2 home visitor groups subsequently were assigned at random to home visitors responsible for their geographic region.
Women in treatment 1 (n = 255) were provided free developmental screening and referral for their child at 6, 12, 15, 21, and 24 months of age. Women in treatment 2 (n = 245) were provided the screenings offered in treatment 1 plus paraprofessional home visiting during pregnancy and the child’s first 2 years of life. Women in treatment 3 (n = 235) were provided the screening offered in treatment 1 plus nurse home visiting during pregnancy and the child’s first 2 years. At each postintervention phase of follow-up (child ages 4, 6, and 9 years), children with developmental needs were referred for further evaluation and treatment through existing community services.
The paraprofessional- and nurse-delivered programs were augmented versions of the core nurse-delivered program carried out in Elmira and updated in Memphis.17,18 The NFP has 3 goals: (1) to improve outcomes of pregnancy by helping women improve their health-related behaviors; (2) to improve children’s subsequent health and development by helping parents provide competent care of their children; and (3) to enhance mothers’ personal development by promoting planning of future pregnancies and helping women continue their educations and find work. Nurse home visitors were required to have a BSN degree with experience in community or maternal and child health nursing; paraprofessionals were required to have a high school education and no college preparation in the helping professions.19
Assessments for this follow-up were conducted at child ages 6 and 9 years, with mean (SD) child ages at assessment of 78.1 (3.2) and 118.8 (3.1) months, respectively. The assessments were carried out from March 21, 2001, through February 25, 2006, by interviewers and child evaluators masked to participants’ treatment. The assessments were based on interviews, observations, and psychological tests with the children as well as mothers’ and teachers’ reports of children’s behavior. Previous assessments were conducted by research staff at the time of registration (prior to their assignment to treatments), at the 28th and 36th weeks of pregnancy, and at the 6th, 12th, 15th, 21st, 24th, and 48th months of the child’s life.
Baseline assessments have been described in previous articles.12,13 A variable was created to form an index of mothers’ psychological resources based on the summed z scores of women’s intelligence,20 mental health,21 and sense of mastery.22 To be consistent with the psychological resource classification used in the Memphis trial, we dichotomized the sample at the exact raw score values used in Memphis to dichotomize the variable at the sample median. In Denver, this split the sample into low (40%) and high (60%) resource groups. For this article, we geocoded neighborhoods in which families lived at registration and created a neighborhood adversity score indexed in standard deviation units around the national mean for adversity.23
Outcomes in the behavioral problem domain consisted of norm-referenced measures of internalizing, externalizing, and total behavioral problems based on teacher and parent report of behavioral problems in which both reporters gave scores that put the children in the borderline or clinical ranges24,25 and children’s scores on the Conners’ Continuous Performance Test26 that placed them in the dysfunctional attention/impulsive range.27
Outcomes in the cognitive domain were derived from norm-referenced tests of receptive language28,29 and intellectual functioning30 at age 6 years; sustained attention31 at ages 6 and 9 years; reading and math achievement at ages 6 years30 and 9 years32; and executive cognitive functioning (visual attention/task switching [Trail Making Test part B]33 and working memory34) at age 9 years.
Secondary outcomes were measured to augment the interpretation of the primary outcomes. They consisted of children’s narrative responses to the MacArthur Story Stem Battery (MSSB) coded to characterize the degree of dysregulated aggression and incoherence revealed in their stories35; evaluators’ ratings of children’s behavioral regulation during testing31; and mothers’ reports of their children’s receipt of specific therapeutic services (for speech and language problems, cognitive delays, attention deficit and hyperactivity, and emotional problems—both prior to the interview at age 6 years and between ages 6 and 9 years), whether their children had been retained in school, and whether they had received special education or remedial services in the first 3 years of elementary school.
Data analyses were conducted on all cases randomized insofar as outcome data were available (intention to treat). The primary statistical model consisted of a single classification factor for treatment (3 levels), 6 baseline covariates (maternal psychological resource index, smoking status, whether mothers registered in the study after 28 weeks of gestation, housing density, maternal conflict with her mother or mother figure, and neighborhood disadvantage) to adjust for treatment nonequivalence (P < .10) among participants assessed at either the 6- or 9-year follow-ups, plus 2 additional covariates (child age at assessment and sex). This core model was examined for the sample as a whole and separately for children born to high- and low-resource mothers. Continuous dependent variables were examined in the general linear model,36 with mean differences converted to effect sizes (ESs); dichotomous outcomes were examined in a modified Poisson regression,37 with differences in rates converted to relative risks (RRs).
Estimates and tests were adjusted for all covariates, which were examined for homogeneity of regressions.38 We used pared down versions of this model when outcomes were infrequently occurring; the tables specify these reduced models. Tests focused on paraprofessional vs control and nurse vs control contrasts. Details of the statistical methods are presented in the eAppendix in the Supplement.
The tables show exact probability levels for individual contrasts of each of the visited groups with controls (for 2-tailed tests) and 95% confidence intervals for treatment contrasts. At the stage of study design, we estimated that we could detect reductions in total behavioral problems from 8.5% to 2.4% for the sample as a whole, and an ES of 0.33 on normally distributed outcomes for children born to low-resource mothers, assuming α = .05, β = 0.20, and 2-tailed tests. We did not adjust for multiple comparisons.
We discuss treatment and control differences with P < .10 (essentially 1-tailed tests with P < .05) because we have corresponding outcomes and estimates of treatment effects of similar magnitude for nurse and control contrasts that were statistically significant at P < .05 in the Memphis trial (which had a control group twice as large as the Denver trial). For outcomes in the cognitive domain and secondary outcomes, we show effects only for children born to low-resource mothers, given corresponding effects in Memphis and earlier phases of the current trial. We give results for children born to mothers with high psychological resources in the eAppendix in the Supplement and note in the text when treatment differences were present for the high-resource group at P < .05, as they were not hypothesized.
Direct assessments were conducted on 81% of the children at age 6 years and 78% at age 9 years using as a denominator all cases originally randomized (90% and 89% of those cases in which the child was alive, the child was not adopted, or the parent had not declined participation at earlier follow-up phases). Rates of retention were similar across treatment conditions (eTable 1 and eAppendix in Supplement).
Participants in the 6- and 9-year follow-ups were similar on background characteristics across treatment conditions, both for the sample overall and for the group defined by mothers having low psychological resources (eTable 2 and eAppendix in Supplement).
Table 1 shows that there were no statistically significant differences between the paraprofessional and control groups in children’s internalizing, externalizing, or total behavioral problems or dysfunctional attention.
Table 2 shows that paraprofessional-visited children born to mothers with low psychological resources compared with control group counterparts made fewer errors on the test of visual attention/task switching at age 9 years (ES = −0.30, P = .08). There were no significant paraprofessional effects on children’s working memory, receptive language, intellectual functioning, or reading and math achievement. Paraprofessional-visited children born to higher-resource mothers had lower sustained attention than their control group counterparts, both over the 4- to 9-year period and at the 9-year assessment (ES = −0.21, P = .03, and ES = −0.26, P = .04, respectively) (eTable 3 and eAppendix in Supplement).
Table 3 shows that paraprofessional-visited children born to low-resource mothers compared with control group counterparts exhibited less dysregulated aggression (ES = −0.36, P = .02) and fewer incoherent stories (ES = −0.50, P = .002) in response to the MSSB and better behavioral regulation during testing (ES = 0.32, P = .05). However, paraprofessional-visited children born to high-resource mothers had more incoherent stories than their control group counterparts (ES = 0.38, P = .004) (eTable 4 and eAppendix in Supplement). Paraprofessional-visited children born to low-resource mothers were less likely to have used therapeutic services prior to the 6-year interview (RR = 0.63, P = .07) (Table 4). There were no significant differences between the paraprofessional and control groups in children’s use of special education or remedial services or in their grade retention in the first 3 years of elementary school.
Table 1 shows that nurse-visited children were less likely to have been classified as having total emotional/behavioral problems at age 6 years (RR = 0.45, P = .08), internalizing problems at age 9 years (RR = 0.44, P = .08), and dysfunctional attention at age 9 years (RR = 0.34, P = .07). There were no statistically significant nurse effects on total behavioral problems at age 9 years, internalizing problems at age 6 years, or externalizing problems at either age.
Table 2 shows that nurse-visited children born to low-resource mothers compared with their control group counterparts had better receptive language scores averaged over ages 2, 4, and 6 years (ES = 0.30, P = .01), although the difference at age 6 years was not statistically significant (ES = 0.21, P = .16). They also had better sustained attention averaged over ages 4, 6, and 9 years (ES = 0.36, P = .006), at age 6 years (ES = 0.33, P = .048), and at age 9 years (ES = 0.33, P = .08). There were no statistically significant nurse effects on children’s visual attention/task switching, working memory, intellectual functioning, or academic achievement.
Table 3 shows that there were no significant nurse effects on dysregulated aggression in response to the MSSB or on evaluators’ ratings of behavioral regulation. Contrary to expectation, nurse-visited children born to high-resource mothers had higher rates of incoherent stories at age 6 years than their control group counterparts (ES = 0.32, P = .01) (eTable 4 and eAppendix in Supplement).
At age 6 years, nurse-visited children born to low-resource mothers were less likely to have used therapeutic services (RR = 0.46, P = .01) (Table 4). They were less likely to have been enrolled in special education or remedial services in the first 3 years of elementary school (RR = 0.57, P = .06). The difference in use of therapeutic services was not significant in the period from ages 6 to 9 years.
We examined the extent to which program effects on children were more pronounced among those with the dual risks of having mothers with low psychological resources and living in more disadvantaged neighborhoods at registration during pregnancy. The nurse (but not paraprofessional) effects on child cognition, language, and achievement were more pronounced among children born to mothers with low psychological resources and who lived in the most disadvantaged neighborhoods, but the number of cases living in the most disadvantaged neighborhoods was small, introducing instability in estimates (data not shown).
Paraprofessional-visited children born to low-resource mothers had fewer errors on the test of visual attention/task switching at age 9 years; exhibited better behavioral regulation during the assessment at age 6 years; had fewer incoherent stories and dysregulated or aggressive themes in their responses to the MSSB; and used fewer therapeutic services prior to the assessment at age 6 years. The differences between the paraprofessional and control groups in behavioral regulation, use of services, and visual attention are internally consistent, and the treatment effects in dysregulated aggression and incoherent story stem narratives align with corresponding nurse effects in the Memphis trial.9
Nurse-visited children were less likely to be classified as having total behavioral problems at age 6 years, internalizing disorders at age 9 years, and dysfunctional attention at age 9 years. These behavioral problem findings are consistent with corresponding nurse effects in Memphis at ages 6 years (for total problems)9 and 12 years (for internalizing problems)11 and with earlier nurse effects on sustained attention and executive functioning in the current trial.13 Nurse-visited children born to low-resource mothers had better language functioning aggregated through age 6 years and attention through age 9 years but exhibited no statistically significant benefits in cognition or in reading and math achievement, as found in Memphis.9- 11
In correspondence with their better language and attention scores in early childhood12,13 and at school entry, nurse-visited children born to low-resource mothers used fewer therapeutic services prior to age 6 years and were enrolled less frequently in special education or remedial services during their first 3 years of elementary school. There were no statistically significant nurse effects on children’s intellectual functioning, math and reading achievement, and dysregulated or aggressive themes in children’s response to the MSSB as we had observed in Memphis.9,10 There are 5 possible, nonexclusive explanations for the absence of nurse effects on these outcomes.
First, the nurse-delivered program simply loses impact over time. Given significant enduring effects in the cognitive domain for children born to low-resource mothers in the Memphis trial, it is unlikely that the story is as simple as this.
Second, the benefit of the nurse-delivered program for children born to low-resource mothers is most pronounced for families living in highly disadvantaged neighborhoods. Neighborhood disadvantage is a marker for adversity and toxic stress, which is associated with damaged executive functioning, language, and behavioral regulation among children unless parents are equipped to protect them.39 The mean level of neighborhood adversity in Memphis, for example, was 2.4 SDs above the national mean, substantially more disadvantaged than neighborhoods of participants in the Denver trial (0.4 SD above the national mean). The greater enduring impact of the program on cognition and achievement among children born to low-resource mothers in the Memphis trial therefore is likely a reflection, at least in part, of the greater number of stressors experienced by Memphis children that, without parental protection, damaged their development to a greater degree than their Denver counterparts who lived in lower-adversity contexts. While the differences between the nurse and control groups in language and cognition among children born to mothers with low psychological resources were most pronounced among those who lived in the most disadvantaged neighborhoods in Denver, the number of cases living in highly disadvantaged neighborhoods was too small to make stable estimates of program impacts in those contexts.
Third, the benefit of the program is greatest where there is greater room for improvement. Control group children born to low-resource mothers in the Denver trial had mean mental processing (IQ) scores of 94.85 at age 6 years compared with 87.64 for their counterparts in Memphis, about a 0.5-SD difference. It is likely that these differences in cognition across trials reflect differences in early experience that to some degree are alterable by the program through its support of parents’ efforts to protect their children. Program impact on child cognition and achievement appears to be reduced among those with normative levels of cognitive functioning.
Fourth, the relative benefit of the program for children born to low-resource mothers declines to the extent that children obtain therapeutic services that address their developmental needs. By age 6 years, control group children born to low-resource mothers were more likely to receive therapeutic services for problems with language, cognition, and behavior than their nurse-visited counterparts, and they were enrolled more frequently in special education and remedial services once they entered elementary school. It is reasonable to assume that if such services are of high quality and parents are engaged, the receipt of those services will lead to improvements in child functioning and attenuation of NFP impact.
Fifth, the absence of statistically significant effects on some cognitive and socioemotional outcomes is a reflection of limited statistical power. In Memphis, statistically significant program effects were found on language and achievement test scores at child ages 6 and 9 years that were in the 0.22- to 0.25-SD range for children born to low-resource mothers. Effects of similar magnitude in the current trial were not even statistical trends (P ≤ .10). The difference between trials is probably due in part to the Memphis trial’s having a control group that was twice as large as its counterpart in Denver and to its having a larger proportion of families defined by mothers having low psychological resources.7
The findings reported here must be interpreted in light of 2 additional limitations. First, we had lower rates of completed behavioral assessments among teachers than among parents. It is better, however, to measure behavioral problems from 2 reporters rather than 1, as this increases validity.25 Second, the pattern of statistically significant effects and trends must be interpreted in the context of our having conducted statistical tests with multiple outcomes.
As the NFP is replicated and tested in new randomized clinical trials throughout the United States and other societies, it will be important to determine whether it is particularly successful in reducing disparities in health, achievement, and economic productivity among children born to mothers who have limited psychological resources and who are living in severely disadvantaged neighborhoods, as this will enable policy makers to focus NFP resources where they produce the greatest benefit.
Finally, the findings from this trial suggest that if we are going to improve the life chances of our most vulnerable children, we must shift public policy toward investments in a range of complementary interventions early in life that have strong evidentiary foundations and capacities for quality implementation.1,40- 42
Children born to low-resource mothers visited by paraprofessionals exhibited improvement in visual attention/task switching. Consistent with an earlier trial of the NFP in Memphis, nurse-visited children showed improved behavioral functioning, and those born to low-resource mothers benefited in language and attention; unlike their counterparts in Memphis, though, they did not improve in intellectual functioning or academic achievement.
Corresponding Author: David L. Olds, PhD, Department of Pediatrics, School of Medicine, University of Colorado Denver, 13121 E 17th Ave, Mail Stop 8410, Aurora, CO 80045 (email@example.com).
Published Online: December 2, 2013. doi:10.1001/jamapediatrics.2013.3817.
Author Contributions: Mr Knudtson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Olds, Robinson.
Acquisition of data: Olds, Holmberg, Robinson.
Analysis and interpretation of data: Olds, Holmberg, Donelan-McCall, Luckey, Knudtson.
Drafting of the manuscript: Olds, Holmberg, Donelan-McCall.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Olds, Holmberg, Donelan-McCall, Luckey, Knudtson.
Obtained funding: Olds.
Administrative, technical, or material support: Olds, Robinson.
Study supervision: Olds, Donelan-McCall.
Conflict of Interest Disclosures: None of the authors have a personal financial interest in the NFP. The Prevention Research Center for Family and Child Health, directed by Dr Olds at the University of Colorado School of Medicine, has a contract with the NFP to conduct research to improve the NFP program and its implementation. All the authors were employed by this center at the time the study was conducted. Dr Olds is the founder of the NFP. Dr Donelan-McCall is conducting research and education for NFP nurses regarding the promotion of early parental caregiving. No other disclosures were reported.
Funding/Support: The current phase of this research was supported by grants 99030 from the Colorado Trust, 90PD0232 and 90XP0017 from the Administration for Children and Families, 2004-52854-CO-JS from the Office of Juvenile Justice and Delinquency Prevention, 2005-MU-MU-001 from the US Department of Justice, 1R01MH069891 and 1R01MH62485 from the National Institute of Mental Health, 2001-049 and 99012 from the Colorado Trust, and Senior Research Scientist Award 1-K05-MH01382 from the National Institute of Mental Health to Dr Olds.
Role of the Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: Bruce Pennington, PhD, University of Denver, assisted in selecting the neuropsychological measures, Jini Puma, PhD, and Norman Watt, PhD (at the University of Denver at the time this study was conducted) assisted in acquisition of the teacher data, Wendy Gehring, University of Colorado Denver, provided data management, and Cheryl Loston-Williams, University of Colorado Denver, helped with manuscript preparation. All of these contributors, except Dr Pennington, received compensation from research grants for their work.