Graphs show mean (95% CIs) for primary outcomes. ECBI indicates Early Childhood Behavior Inventory.
Improving Parenting Skills
Perrin EC, Sheldrick RC, McMenamy JM, Henson BS, Carter AS. Improving parenting skills for families of young children in pediatric settings: a randomized clinical trial. JAMA Pediatr. Published online November 4, 2013. doi:10.1001/jamapediatrics.2013.2919.
Perrin EC, Sheldrick RC, McMenamy JM, Henson BS, Carter AS. Improving Parenting Skills for Families of Young Children in Pediatric SettingsA Randomized Clinical Trial. JAMA Pediatr. 2014;168(1):16–24. doi:10.1001/jamapediatrics.2013.2919
Disruptive behavior disorders, such as attention-deficient/hyperactivity disorder and oppositional defiant disorder, are common and stable throughout childhood. These disorders cause long-term morbidity but benefit from early intervention. While symptoms are often evident before preschool, few children receive appropriate treatment during this period. Group parent training, such as the Incredible Years program, has been shown to be effective in improving parenting strategies and reducing children’s disruptive behaviors. Because they already monitor young children’s behavior and development, primary care pediatricians are in a good position to intervene early when indicated.
To investigate the feasibility and effectiveness of parent-training groups delivered to parents of toddlers in pediatric primary care settings.
Design, Setting, and Participants
This randomized clinical trial was conducted at 11 diverse pediatric practices in the Greater Boston area. A total of 273 parents of children between 2 and 4 years old who acknowledged disruptive behaviors on a 20-item checklist were included.
A 10-week Incredible Years parent-training group co-led by a research clinician and a pediatric staff member.
Main Outcomes and Measures
Self-reports and structured videotaped observations of parent and child behaviors conducted prior to, immediately after, and 12 months after the intervention.
A total of 150 parents were randomly assigned to the intervention or the waiting-list group. An additional 123 parents were assigned to receive intervention without a randomly selected comparison group. Compared with the waiting-list group, greater improvement was observed in both intervention groups (P < .05). No differences were observed between the randomized and the nonrandomized intervention groups.
Conclusions and Relevance
Self-reports and structured observations provided evidence of improvements in parenting practices and child disruptive behaviors that were attributable to participation in the Incredible Years groups. This study demonstrated the feasibility and effectiveness of parent-training groups conducted in pediatric office settings to reduce disruptive behavior in toddlers.
clinicaltrials.gov Identifier: NCT00402857
Disruptive behavior disorders, such as attention-deficit/hyperactivity disorder and oppositional defiant disorder, are among the most frequently diagnosed and stable disorders in children. Symptoms are often evident as early as 1 to 3 years of age1,2 and typically continue into later childhood and adolescence,3- 5 resulting in academic underachievement, reduced social competence, and mental health disorders.6- 8Quiz Ref IDHowever, fewer than 25% of young children identified with behavioral problems receive treatment.9,10 Because of the frequency and nature of their contact with families of young children, primary care physicians are in a unique position to affect the course of early-onset disruptive behavior.11
Extensive evidence documents the efficacy of parent-training interventions for improving child disruptive behaviors.12,13 The Incredible Years (IY) program in particular has received support in multiple randomized clinical trials,14- 18 and emerging evidence supports its efficacy for toddlers.19- 21 However, parent-training programs are not widely available and evidence of their feasibility and efficacy in primary care settings is limited.22,23
To test the efficacy of a 10-week version of the IY program for very young children in primary care settings, we conducted a randomized clinical trial in 11 pediatric practices. We hypothesized that intervention would lead to improvement in parenting practices and children’s behaviors, which would be sustained for 12 months.
Participants were parents of children receiving care at 7 private-practice groups and 4 federally qualified health centers in Eastern Massachusetts. We invited practices within 60 minutes from the medical center and with 6 or more pediatricians to participate. Of 43 practices that met criteria, 22 responded to our request and 12 agreed to participate. Practices implemented behavioral screening for all children between 22 and 42 months of age using the 20 items of the Infant-Toddler Social-Emotional Assessment Scale that best predicted disruptive behavior disorders.3,24Quiz Ref IDParents were eligible if their child scored at the 80th percentile or greater on the screener. Parents were excluded if they (1) could not speak English or Spanish well enough to participate in a parenting group or (2) reported that the child had a diagnosis of pervasive developmental disorder or global developmental delay.
All study procedures were approved by the institutional review board at Tufts University School of Medicine and informed consent was documented in writing.
Pediatric staff asked eligible parents for permission to be contacted by study staff. Research staff met with consenting parents to describe study procedures and obtain informed consent. Once enough parents were enrolled at a practice, an independent investigator with access only to study identification numbers completed randomization using a random number generator. We aimed to assign 6 to 12 parents each to a parent-training group (PTG) and a waiting-list (WL) control condition. Participants assigned to PTG who could not attend at the selected time remained assigned to the PTG condition and were invited to participate in subsequent groups. If fewer than 6 participants assigned to PTG could attend on the selected evening, additional participants from WL were chosen at random and reassigned to PTG. This procedure resulted in more parents assigned to PTG than to WL. In 3 private practices and 3 health centers, too few participants were identified within 3 months to constitute both a PTG and a control group; therefore, all participants were assigned to the PTG condition. Thus, in addition to those randomly assigned to PTG or WL, we created a third study condition consisting of those who were assigned directly to a PTG in a nonrandom fashion (NR-PTG). Parents assigned to WL were invited to participate in an intervention group after 1 year.
Following previous studies with young children,25,26 we abbreviated the IY curriculum to create a 10-week manualized protocol. The program encourages proactive, nurturing parenting, while discouraging harsh, punitive approaches using videotaped modeling, group discussion, role plays, and home practice tasks arranged across 4 modules (play, praise and rewards, effective limit setting, and handling misbehavior). Further details about IY groups are available (http://www.incredibleyears.com). Parent-training groups met for 2 hours per week for 10 weeks. These meetings generally took place within the pediatric office, but occasionally space constraints in the office required that they be held in a local public library. Both parents were encouraged to participate when possible.
To ensure fidelity to the intervention protocol, each group was facilitated by a leader and co-leader trained to run IY groups. Primary group leaders were clinical psychologists or social workers with prior experience running group interventions. Co-leaders were members of the pediatric staff (ie, nurse, nurse practitioner, social worker, or pediatrician). All intervention materials used by group leaders were standardized including the facilitator manual, video vignettes, parent home activities, and handouts. Content and delivery of group sessions were reviewed during weekly supervision meetings.
To assess fidelity, group leaders documented session activities (eg, review of homework and video vignettes discussed) after each session using the IY program’s Leader’s Process Checklist. All sessions were videotaped and 3 from each 10-week series were chosen at random and coded for content (eg, topics covered), delivery (eg, showing video vignettes, asking open-ended questions, and conducting role plays), and degree of parent participation by a research assistant trained for reliability.
Primary outcomes included 2 parent-report questionnaires: (1) the 30-item Parenting Scale,27 which assesses negative discipline styles; and (2) the 36-item Early Childhood Behavior Inventory (ECBI),28 which assesses the presence and intensity of child disruptive behaviors. In both instruments, higher scores reflect more dysfunctional behaviors. Both have displayed strong reliability, validity,29,30 and sensitivity to change21,31,32 in diverse populations.33 Raw ECBI scores were transformed to T scores using published norms for 2- to 6-year-old children to contextualize interpretation of outcomes.34
Additional primary outcomes were derived from 20-minute observations of parent-child interactions during standardized tasks including free play, problem solving, and behavioral inhibition.35 Sessions were conducted at each pediatric practice and recorded using a remote video camera controlled by a laptop over a portable and secure local area network. Videotapes were scored by independent coders who were unaware of treatment condition and assessment period. Coding followed the manual for the Dyadic Parent-Child Interactive Coding System–Revised (DPICS–R), which scores 24 parent and child behaviors in 5-minute intervals, and a 75-item Coder Impression Inventory (CII), which generates 5 parent and 5 child behavior subscale scores. The DPICS-R has demonstrated good internal reliability between CII and behavior codes, as well as concurrent validity with external criteria in previous IY studies.21
Participants were asked to complete paper-and-pencil assessments at enrollment (baseline), immediately after the intervention (posttreatment), and 6 and 12 months after the intervention (6-month follow-up and 12-month follow-up, respectively), and interactions were filmed at baseline, posttreatment, and the 12-month follow-up. Parents were compensated $40 for completing paper-and-pencil instruments and an additional $20 for each observation.
Analyses were conducted using Stata version 12. Baseline differences among conditions were tested using analyses of variance and logistic regression. Demographic variables that differed among conditions were included as covariates in subsequent analyses. We conducted an intent-to-treat analysis of the effect of the assigned experimental condition on each primary outcome using a mixed-effects regression model. Main effects included a categorical study condition variable (WL, PTG, or NR-PTG) and an ordinal point variable (baseline, posttreatment, 6-month follow-up, or 12-month follow-up). Of primary interest were tests of whether differential change between conditions led to significant standardized mean differences in outcomes at follow-up. Analyses accounted for clustering of time points within participants and for participants within pediatric practices. We also calculated adjusted means and 95% confidence intervals. A priori power analyses determined that a total of 234 participants would offer 80% power to detect a small-to-moderate effect size.
Several analyses focused on missing data.36 To explore missing data patterns, we coded loss to follow-up as a binary variable and tested baseline variables as predictors using a stepwise logistic regression. To test whether results were still significant under the assumption that data were missing at random, we conducted multiple imputation and ran our primary analyses across 10 imputed data sets,37 adjusting for additional variance across imputations. Results reported here were based on multiple imputations. These results were compared with identical analyses run on complete cases.38
Data regarding enrollment, assignment to study conditions, and follow-up assessments are presented in Figure 1. Enrollment occurred from March 2007 to April 2010. Because screening was conducted directly by pediatric practices and was not part of study protocol, data regarding the number of children screened are unavailable. Of the 830 parents who reported disruptive behavior in their toddler and consented to be contacted by research staff, 345 (41.6%) enrolled in the study. Of these, 150 were randomly assigned to PTG (n = 89) or to WL (n = 61), and an additional 123 families were assigned to NR-PTG. Of the parents in the PTG condition, 71 completed at least 3 intervention sessions (80%), 65 completed at least 7 sessions (73%), and 80 provided follow-up data (90%). Fifty parents in the WL condition (82%) provided follow-up data. Among parents in the NR-PTG condition, 73 completed at least three sessions (59%), 66 completed at least 7 sessions (54%), and 72 provided follow-up data (59%).
Table 1 presents sample characteristics. Quiz Ref IDMost children were male (62%), and 29% were of nonwhite race and/or Hispanic ethnicity (compared with 24% and 36% in Massachusetts and the United States, respectively).39 Participants were primarily mothers (96%) and married (70%), with a median age of 34 years. The sample was diverse with respect to socioeconomic status, with 26% reporting household income less than $20 000 and 33% reporting more than $100 000. No differences in demographic variables were found between PTG and WL at baseline or follow-up. However, families from practices in the NR-PTG condition were more likely to report minority race/ethnicity, lower levels of education and family income, and being a single parent. In addition, there were no differences among conditions on outcome variables at baseline. Mean T scores on the ECBI ranged from 60.1 to 62.8 on the Problem Scale and from 58.3 to 59.2 on the Intensity Scale.
High fidelity to the IY protocol was maintained throughout the study. Both session checklists and video review indicate that more than 90% of the content and delivery elements of the IY protocol were followed throughout all groups. Parents completed 81% of assigned home activities. In 100% of sessions reviewed, every parent spoke at least once, and in only 10% of sessions did 1 parent dominate discussion. These findings are consistent with other IY studies.40,41
Table 2 displays means and 95% confidence intervals for outcomes at baseline and each follow-up assessment. Tests of longitudinal change within conditions revealed that Quiz Ref IDself-reports of negative parenting behaviors on the Parenting Scale were lower than baseline at all follow-up assessments in both parent-training conditions (PTG and NR-PTG). No differences were noted for the WL group. Precisely the same pattern was noted for the ECBI’s Problem and Intensity scales with 1 exception (ECBI Intensity for PTG at the 6-month follow-up).
Table 3 displays standardized mean differences between conditions at follow-up. At all follow-up assessments, PTG and NR-PTG were superior to WL on all parent-report scales (P < .05). In addition, no differences were observed between PTG and NR-PTG with 1 exception (ECBI Intensity at the 6-month follow-up [P = .049]).
Although remote recording reliably captured video of parent-child interactions, audio quality was variable, depending on room characteristics and whether participants consistently faced the camera when speaking. Therefore, coders were unable to reliably assess individual behaviors in 5-minute segments, so we relied on the Coder Impression Inventory (CII) subscales as the primary outcomes. To minimize multiple testing, we conducted principal components analyses to create summary indices of parent and child behaviors. Across both parent and child CII scales, 1 component displayed an eigenvalue of more than 3.5 and all other eigenvalues were less than 1, supporting the use of 1 negative parenting construct and 1 child disruptive behavior construct for statistical analyses.
In assessing videotaped observations, we recognized an unanticipated problem (ie, that parents might have learned to master the challenges presented over repeated assessments and that negative behaviors would therefore become increasingly rare). In this case, floor effects would become evident as frequencies approached zero, resulting in significant skew and difficulty detecting change attributable to intervention. Analyses of CII scales lent support to this concern. Increasing skew was observed in both the negative parenting (skew = 1.2 at baseline, 2.2 at the 12-month follow-up) and child disruptive behavior components (skew = 0.8 at baseline, 2.7 at the 12-month follow-up). To minimize these effects, we conducted a post hoc analysis in which we created a new observational variable based on the 5 CII subscales with the least skew, including 2 child (child’s overall negative conduct and child bonding with parent) and 3 parent (nurturing/supportive parenting, competent parenting, parent negativity/hostility) scales. A principal components analysis of these scales yielded a single component that we labeled negative parent-child interaction, which displayed less extreme skew at baseline (skew = 0.7) and at the 12-month follow-up (skew = 1.1).
As seen in Table 2 and portrayed in Figure 2, the 3 CII components (ie, negative parenting, child disruptive behaviors, and negative parent-child interaction) were lower at posttreatment and the 12-month follow-up compared with baseline in the PTG and NR-PTG conditions. No differences were found between baseline and posttreatment in the WL condition, but both child disruptive behavior and negative parenting were lower at the 12-month follow-up. As seen in Table 3, PTG was superior to WL at posttreatment on all CII components, and at the 12-month follow-up on negative parent-child interaction; NR-PTG was superior to WL at both posttreatment and the 12-month follow-up on negative parent-child interaction. We found no differences in CII components between PTG and NR-PTG.
Consistent with a hypothesis that data are missing at random, several baseline demographic, but not outcome, variables predicted missingness including marital status (odds ratio [OR] = 3.4), parent age (OR = 0.92), child age (OR = 1.96), and non-white or Hispanic race/ethnicity (OR = 2.6). Moreover, analyses based on multiple imputed data were generally consistent with analyses of complete cases. Differences between conditions at follow-up displayed precisely the same pattern of results noted here, with the following exceptions: (1) change in ECBI Intensity Scale score from baseline to the 6-month follow-up was statistically significant between WL and PTG, but the ECBI Problem Scale score was not, and (2) change in the DPICS-CII child disruptive behavior at posttreament was significant in the NR-PTG condition.
The results provided strong support for the feasibility and effectiveness of parent training for families with young children in pediatric settings. Parents in both PTG conditions reported more change in self-reported outcomes than did parents in the WL condition. Analyses of videotaped observations followed the same pattern at posttreatment and, after omitting variables with severe skew, at 12 months after intervention.
We note several limitations to this study. Videotaped observations were technically challenging because of constraints of pediatric office space. In addition, our failure to find effects at 12 months for the primary observational variables resulted from unexpected improvement in the control condition along with maintenance of improvement in the intervention conditions. We surmise that this improvement in both intervention and control conditions was because of learning effects from repeated observations of our brief and simple protocol (leading to changes in the control as well as the experimental conditions), and that the short length of the observations might have caused floor effects in our assessments of negative behaviors (thus limiting evidence of change). It is notable that post hoc analysis correcting for skew revealed a significant effect of treatment at 12 months.
Although we enrolled more participants than expected (N = 273 vs 234 specified in our power analyses), sample sizes were attenuated by loss to follow-up and the need to create a third nonrandomized condition for practices with slower enrollment rates. While adding a third condition reduced statistical power, it allowed us to demonstrate the feasibility of PTGs in a wider range of settings including urban health care centers. The close correspondence in outcomes between the 2 parent-training conditions (one randomized and the other not) suggests that the findings are robust and, given the differences in the demographic composition across pediatric settings, demonstrates that the intervention is effective in settings with a wide range of risk. Because we targeted and enrolled children with elevated symptoms rather than diagnoses, our implementation of PTGs in primary care pediatric settings can be considered a secondary preventive intervention. Our sample can be characterized as high risk (baseline ECBI T score >55)42 or at the borderline of clinical (T score >60),34 which is typical of previous randomized clinical trials of parent training for young children.41 The results across methods in this study are impressive given that effect sizes have been shown to be associated with the magnitude of symptom severity at baseline,43 and thus it is typically more difficult to find large effects in prevention than in intervention trials.
Quiz Ref IDThe generalizability of our results is limited by the fact that many parents did not choose or were unable to participate in the parent-training intervention as offered. The reasons why parents declined enrollment or dropped out may include the relatively large research burden, the length of the program, and other unknown factors. Our conclusions apply to families who were able to complete a 10-week course of parent training when offered as a research intervention and not to the entire population of families who seek pediatric care for their children. The consistency of primary analyses with those based on multiple imputation suggests that results are robust to data that are missing at random, although of course we were unable to evaluate bias associated with unobserved variables.
This study supports the benefits of offering parent-training interventions in primary care settings. It demonstrates the feasibility of training pediatric staff (in particular nurses, nurse practitioners, and social workers) to co-lead parenting groups and the efficacy of parent training delivered in diverse pediatric settings. A growing evidence base confirms that PTGs are cost-effective in reducing children’s disruptive behaviors,44,45 and offering them in pediatric practices using trained practice staff represents a critical opportunity to provide access to effective mental health care to a wide population. Further efforts are necessary now to address fiscal barriers to making PTGs available in pediatric settings.
Corresponding Author: Ellen C. Perrin, MD, Division of Developmental-Behavioral Pediatrics, Floating Hospital for Children, Tufts Medical Center, 800 Washington St, Ste 854, Boston, MA 02111 (email@example.com).
Accepted for Publication: May 1, 2013.
Published Online: November 4, 2013. doi:10.1001/jamapediatrics.2013.2919.
Author Contributions: Dr Sheldrick had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Perrin, Sheldrick, McMenamy, Carter.
Acquisition of data: Perrin, Sheldrick, McMenamy, Henson.
Analysis and interpretation of data: All authors.
Drafting of the manuscript: Perrin, Sheldrick, Henson.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Sheldrick.
Obtained funding: Perrin, Sheldrick, McMenamy, Carter.
Administrative, technical, or material support: McMenamy, Henson, Carter.
Study supervision: Perrin, Sheldrick, McMenamy.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was funded by the National Institute of Mental Health R01 grant MH076244-01.
Role of the Sponsor: The National Institute of Mental Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentation: A previous version of this article was presented at the Academic Pediatric Association’s Presidential Plenary Session; April 30, 2012; Boston, Massachusetts.
Additional Contributions: Many people contributed to the success of this project. We are indebted to the pediatricians and their staffs who worked with us over several years. They were creative and committed in devising office systems for screening, referring parents to groups, and co-leading the groups (Codman Square Health Center; Martha Eliot Health Center; Medical Associates Pediatrics, Leominster; Massachusetts General Hospital’s Chelsea Health Care Center; Massachusetts General Hospital’s Revere Health Care Center; Pediatric Healthcare Associates, Peabody; Pediatrics West, Westford; Southboro Medical Group; Westwood-Mansfield Pediatric Associates PC; Wilmington Pediatrics; and Woburn Pediatric Associates). In addition, a dedicated team of research assistants was committed to enrolling parents, facilitating group meetings, and collecting and managing data: Archana Asundi, BA, Andrea Farnham, BA, Kathryn Mattern, BA, Maria Mayoral, BA, Emily Neger, BA, Dana Peterson, BA, and Meredith Phelps, BA. Saralyn Byler and Sara Hunt assisted with fidelity monitoring. In addition to 2 of the co-authors, Alexis Hinchey, LICSW, and Julie Weieneth, PhD, served admirably as group leaders. The staff of Carolyn Webster-Stratton, PhD, coded the videotaped assessments using its Dyadic Parent-Child Interactive Coding System. Steven Cohen, PhD, assisted with data analysis. And of course the project’s success reflects most of all on the parents who gave generously of their time and effort.