ITT indicates intention to treat; RCT, randomized clinical trial.
The themes of the 11 sessions are as follows: (1) noticing good behavior, (2) spreading attention around, (3) ignoring whining and complaining, (4) transitional warnings and when-then statements, (5) planning ahead in the home, (6) the behavior chart, (7) planning ahead outside the home, (8) working with daycare, (9) timeout, (10) problem-solving relapse prevention, and (11) putting it all together. SFSW indicates Strongest Families Smart Website.
eTable 1. Session Themes of the SFSW Intervention Program
eTable 2. Treatment Comparisons of CBCL, ICU, Parenting Scale, and DASS-21 Scores for Per Protocol Set
Customize your JAMA Network experience by selecting one or more topics from the list below.
Sourander A, McGrath PJ, Ristkari T, et al. Internet-Assisted Parent Training Intervention for Disruptive Behavior in 4-Year-Old ChildrenA Randomized Clinical Trial. JAMA Psychiatry. 2016;73(4):378–387. doi:10.1001/jamapsychiatry.2015.3411
Copyright 2016 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
There is a large gap worldwide in the provision of evidence-based early treatment of children with disruptive behavioral problems.
To determine whether an Internet-assisted intervention using whole-population screening that targets the most symptomatic 4-year-old children is effective at 6 and 12 months after the start of treatment.
Design, Setting, and Participants
This 2-parallel-group randomized clinical trial was performed from October 1, 2011, through November 30, 2013, at a primary health care clinic in Southwest Finland. Data analysis was performed from August 6, 2015, to December 11, 2015. Of a screened population of 4656 children, 730 met the screening criteria indicating a high level of disruptive behavioral problems. A total of 464 parents of 4-year-old children were randomized into the Strongest Families Smart Website (SFSW) intervention group (n = 232) or an education control (EC) group (n = 232).
The SFSW intervention, an 11-session Internet-assisted parent training program that included weekly telephone coaching.
Main Outcomes and Measures
Child Behavior Checklist version for preschool children (CBCL/1.5-5) externalizing scale (primary outcome), other CBCL/1.5-5 scales and subscores, Parenting Scale, Inventory of Callous-Unemotional Traits, and the 21-item Depression, Anxiety, and Stress Scale. All data were analyzed by intention to treat and per protocol. The assessments were made before randomization and 6 and 12 months after randomization.
Of the children randomized, 287 (61.9%) were male and 79 (17.1%) lived in other than a family with 2 biological parents. At 12-month follow-up, improvement in the SFSW intervention group was significantly greater compared with the control group on the following measures: CBCL/1.5-5 externalizing scale (effect size, 0.34; P < .001), internalizing scale (effect size, 0.35; P < .001), and total scores (effect size, 0.37; P < .001); 5 of 7 syndrome scales, including aggression (effect size, 0.36; P < .001), sleep (effect size, 0.24; P = .002), withdrawal (effect size, 0.25; P = .005), anxiety (effect size, 0.26; P = .003), and emotional problems (effect size, 0.31; P = .001); Inventory of Callous-Unemotional Traits callousness scores (effect size, 0.19; P = .03); and self-reported parenting skills (effect size, 0.53; P < .001).
Conclusions and Relevance
The study reveals the effectiveness and feasibility of an Internet-assisted parent training intervention offered for parents of preschool children with disruptive behavioral problems screened from the whole population. The strategy of population-based screening of children at an early age to offering parent training using digital technology and telephone coaching is a promising public health strategy for providing early intervention for a variety of child mental health problems.
clinicaltrials.gov Identifier: NCT01750996.
Population-based cohort studies1-12 have found that childhood disruptive behavior disorders are developmental precursors to a wide range of negative outcomes, including peer rejection, school failure, psychopathologic conditions, substance abuse, suicidality, and criminality. Approximately half of the children who have been identified as aggressive with externalizing behavior as preschoolers develop persistent problems.13,14 Parent training is the most effective approach to the psychosocial treatment of disruptive behavioral problems15-18 and one of the best-validated therapeutic techniques.19 In parent training interventions, parents typically learn to identify, define, and observe problem behaviors in new ways and acquire strategies to prevent and respond to oppositional behavior.19
Despite the significant child, parent, and societal consequences of disruptive behavioral problems and the availability of effective interventions, most parents do not receive parent training interventions. Stigma, availability, cost of services, and logistical barriers, such as child care, transportation time, work schedules, or discomfort with services provided in groups, prevent many parents from enrolling in or completing parent training programs.20-26 An optimal intervention for disruptive behavioral problems should reduce barriers that limit use, target problems emerging during the preschool years, yield meaningful outcomes, and be affordable enough to be disseminated and sustained. Internet-assisted treatment affords many benefits over traditional means, such as high fidelity, greater accessibility, convenience, and reduced cost to patients.27
This randomized clinical trial (RCT) reports 6- and 12-month follow-up results of the population-based Strongest Families Smart Website (SFSW) intervention with telephone coaching compared with an education control (EC). The target population was children with a high level of childhood disruptive behavior disorders screened from the population of 4-year-olds attending annual child health clinic checkups. The primary hypothesis was that the SFSW intervention would reduce child disruptive behavior disorder symptoms at 6 and 12 months after randomization compared with the EC. Furthermore, we expected that participants randomized to the intervention group would exhibit improvement in self-reported parenting skills and distress compared with those randomized to the control group.
Question: Is an Internet-assisted intervention using whole-population screening that targets the most symptomatic 4-year-old children effective at 6- and 12-month follow-ups?
Findings: The findings reveal effectiveness of a parent training intervention that incorporates interactive web technology to provide a personalized and sustainable intervention for the public health system.
Meaning: The strategy of population-based screening of children at an early age and offering Internet-assisted parent training that uses telephone coaching could be a promising solution for providing early prevention and intervention for childhood disruptive behavior.
The study design was a 2-parallel-group RCT stratified by sex, with 1:1 allocation comparing the SFSW intervention with an EC. Consistent with the Consolidated Standards of Reporting Trials (CONSORT) protocol, best-practice RCT guidelines were followed. The study was approved by the research ethics boards of the Hospital District of Southwest Finland and IWK Health Centre.28 All data were collected with voluntary written consent obtained online with the SFSW application and deidentified. The study protocol, which describes the study in more detail, has been previously published29 and can be found in Supplement 1. With a 30% attrition rate and 250 participants in the SFSW intervention and EC groups, we could show medium to small incremental effects of 0.30 to 0.35 standard units.29
Children with a high level of childhood disruptive behavior disorder symptoms screened from the population of 4-year-olds attending annual child health clinic checkups in the catchment area located in Southwest Finland from October 1, 2011, through November 30, 2013, were recruited. Data analysis was performed from August 6, 2015, to December 11, 2015. Before the beginning of school, practically all children visit clinics where the child’s health is comprehensively assessed. Study recruitment took place at the children's yearly health clinic visit, typically within 1 month after their fourth birthday.30
Inclusion criteria for the RCT participants were as follows: (1) the child was 4 years old; (2) at least 1 parent’s native language was Finnish; (3) families resided in the participating municipalities; (4) the child had behavioral problems for the last 6 months before screening (score of ≥5 points on the conduct problems subscale of the Strengths and Difficulties Questionnaire [SDQ] corresponding to the 80th percentile cutoff point based on a Finnish population study31 that included 4-year-olds [n = 931]); (5) on a single question the parent reported that their child had difficulties; and (6) the parent had access to a telephone, computer, and an Internet connection in their home (a computer was provided to families if needed).30
Recruitment began on October 1, 2011, in the Southwest Finnish cities of Turku, Raisio, Kaarina, and Naantali. On October 1, 2012, a total of 7 smaller municipalities were enrolled.32 Participants were identified from the Finnish National Population Register33 and included all children who had a 4-year annual health checkup in a study municipality. A study information package was mailed approximately 1 month before the child’s fourth birthday. Parents were asked to bring the completed health questionnaire to the clinic. The Finnish universal health care system has high attendance, and families participate in preschool child annual checkups to a high extent.30
Children were excluded from the study if they were not speaking in full sentences, were hearing or vision impaired, were receiving or had received behavioral treatment, or had a diagnosis of autism, pervasive development disorder, Down syndrome, fetal alcohol syndrome, or intellectual disability.
The randomization sequence was generated, stratified by sex, with a 1:1 ratio of SFSW intervention vs EC, using a computerized random permuted block sequence generator (Random Allocation Software34) with concealed block sizes. Participants were informed via email of their assignment and given a link to the relevant website. Participants were not restricted from seeking other assistance. The participant flowchart (CONSORT diagram) is presented in Figure 1.
In the EC group, participants were given access to a website that provided a brief introduction to positive parenting strategies and a 45-minute call from a coach who provided positive parenting advice, in addition to the standard care provided by their physicians or obtained by parents.
Participants in the SFSW intervention group received an Internet version of the Strongest Families telephone-based program.35-37 In this evidence-based program, participants develop skills to strengthen parent-child relationships, reinforce positive behavior, reduce conflict, manage daily transitions, plan for difficult situations, and encourage prosocial behavior (eTable 1 in Supplement 2). For the purposes of this study, the Strongest Families program was translated into Finnish, adapted to the Finnish cultural environment, and transferred to an Internet format with strategies, examples, and exercises appropriate for parents of 4-year-old children.
Parents were encouraged to complete 11 weekly online sessions (Figure 2) and to share the program’s content with their partner. Sessions included exercises, instructional videos, and audio clips demonstrating the application of new skills. Interactions within the website were personalized with the child’s name, problems, strengths, and preferred activities.29 Each week participants received an approximately 45-minute telephone call from a coach. All coaches were licensed health care professionals who received intensive training for the SWSF intervention and study protocol by an experienced Strongest Families trainer. Coaches reviewed the successful application of new skills, responded to questions, and provided encouragement. The next session was introduced if the coach determined that skill-related questions were mastered by observing website use and information collected during the telephone call. Children did not participate in coaching calls or access the website. To our knowledge, no potential adverse effects of parent training have been reported in the literature.38 Approximately 7 and 10 months after randomization, parents in the intervention group received booster coaching sessions to review the learned skills.
Parents’ demographic characteristics (native language, educational level, and family structure) were recorded. The conduct problems subscale of the SDQ was used to screen participants for the RCT.31 Parent ratings of preschoolers on the SDQ’s conduct problems subscale have yielded an internal consistency score of 0.58.39 The perceived difficulties were assessed with a single question from the SDQ: “Overall, do you think that your child has difficulties in 1 or more of the following areas: emotions, behavior or being able to get on with other people?” The alternatives were no, minor difficulties, definite difficulties, and severe difficulties. To be included in the RCT, the child had to have a high level of symptoms and at least minor perceived difficulties.
The primary outcome measure was the 24-item externalizing scale of the Child Behavior Checklist version for preschool children (CBCL/1.5-5).40 The Cronbach α for the externalizing scale among preschool children has been reported as 0.88.41,42
Secondary outcome measures (CBCL/1.5-5,43 Parenting Scale,44,45 Inventory of Callous-Unemotional Scale,46,47 21-item Depression, Anxiety, and Stress Scale,48,49 and satisfaction measures) and quality assurance are described in the eMeasures in Supplement 2. All measures were completed online by the participants at baseline and at the 6- and 12-month follow-ups.
All participants were included in the intention-to-treat analysis. Per-protocol analyses included participants with all time point measurements completed. Categorical variables are presented as number (percentage) and continuous variables as mean (SD). Pearson χ2 tests or Fisher exact tests were conducted to explore differences in categorical variables at baseline between the children and parents in the intervention and control groups. Two-tailed, 2-sample t tests were used to explore differences in continuous variables between groups at baseline. Outcome variables were analyzed with a linear mixed-effect model for repeated measurements with time (baseline and 6 and 12 months after treatment) as the within factor and treatment (SFSW intervention or EC) and sex as the between factors. Paired comparisons using linear contrasts were applied within the same model. The Bonferroni correction was applied to P values from multiple comparisons. Compound symmetry covariance structure was fitted to form the block-diagonal matrix for the random subject effect in this mixed-effect model. After examining the main treatment effect, we investigated whether the possible treatment effect on the outcome variables was modified by sex. For this purpose, a treatment group × sex interaction effect was added to the statistical model. The interaction terms were deemed to be statistically significant based on P < .05, which was then used for all statistical testing. In addition, the CBCL/1.5-5 externalizing scale score at 12 months was dichotomized using the 80th percentile as a cutoff point and analyzed with logistic regression analysis adjusting for sex. The Cohen d was calculated as a measure of effect size to complement the statistical testing. Statistical analyses were performed using SAS statistical software, version 9.4 (SAS Institute Inc).
A total of 4656 children were referred from child health clinics with a parent completed screening questionnaire. Of these, 292 parents (6.3%) declined to participate in the RCT. Of 4364 parents willing to participate in the RCT, 730 (16.7%) met the screening inclusion criteria. Of those who met the screening inclusion criteria, 186 (25.5%) declined to participate in the RCT, 21 (3.0%) did not meet the RCT inclusion criteria, and 59 (8.1%) could not be contacted. A total of 464 parents of 4-year-old children were randomized (232 in the SFSW intervention group and 232 in the EC group); 171 participants (73.7%) in the SFSW intervention group and 178 participants (76.7%) in the EC group had baseline and 6- and 12-month follow-up measures.
Of children randomized, 287 (61.9%) were male and 79 (17.1%) lived in other than a family with 2 biological parents. Table 1 gives the demographic characteristics of the SFSW intervention and EC groups. None of the children used psychotropic medication during the study. A total of 21 (12.2%) of 172 children in the SFSW intervention group and 35 (19.3%) of 181 children in the EC group had received additional treatment because of child behavioral problems between randomization and 12-month follow-up (P = .07).
Baseline and 6- and 12-month postrandomization scores of all child outcome measures are presented in Table 2 and Table 3. No significant baseline differences were found in the primary and secondary outcome variables, apart from higher overreactivity in the SFSW intervention group on the Parenting Scale (P = .04). The findings were similar at 6- and 12-month follow-ups. The CBCL/1.5-5 externalizing scale revealed significantly greater improvement in the SFSW intervention group compared with the EC group at 6- and 12-month follow-ups (effect size at 12-month follow-up, 0.34) (Table 3). Of the secondary outcome measures, CBCL/1.5-5 total score, internalizing scale score, and scores on 5 of the 7 syndrome scales revealed significantly greater improvement in the intervention group at 12-month follow-up. Five CBCL/1.5-5 DSM-IV subscores and 4 Inventory of Callous-Unemotional Scale callousness scores revealed significantly greater improvement in the SFSW intervention group. All Parenting Scale measures revealed significant improvement in the SFSW intervention group compared with the EC group. No significant difference was found in parents’ stress, anxiety, or depression when comparing the SFSW intervention group and the EC group. No significant change was found between the 6- and 12-month follow-ups except in parenting skills, which revealed further improvement. When analyzed according to the per-protocol principle, including participants with baseline and follow-up measurements, the results were similar (eTable 2 in Supplement 2).
In additional analyses, at 12-month follow-up, 34 (19.2%) of 177 children in the SFSW intervention and 64 (34.4%) of 186 children in the EC group had an CBCL/1.5-5 externalizing score over the 80th percentile based on the Finnish population sample of preschool children (odds ratio, 2.2; 95% CI, 1.4-3.6; P = .001).42 Of note, all children included in the study had externalizing problems above 80th percentile at screening phase. Parent satisfaction (defined as agree and strongly agree) ranged from 98% (the program met the needs of participants) to 84% (program reduced stress of the participant).
To our knowledge, this study is the first RCT of an Internet-assisted parent training program using a population-based screening procedure. The SFSW intervention resulted in significant improvement in child externalizing symptoms at 6 and 12 months after randomization compared with the EC. The SFSW intervention improved most psychiatric symptom domains, including parent-reported aggression, affective, anxiety, and sleep problems. Intervention reduced callousness scores, which are associated with poorer treatment outcomes.50 Self-reported parenting skills improved significantly when compared with the EC group.
Unlike RCTs on referred populations or convenience samples, the study design allows us to generalize about the effectiveness of an early parent training intervention for most symptomatic children screened from the general population. These findings have global interest in planning low-threshold early interventions and service planning for children with disruptive behavioral problems. Individuals who exhibit disruptive behavioral problems in early childhood are more likely to engage in life-course-persistent antisocial behavior that continues through adolescence into adulthood.6,10-12,51,52 The life-course-persistent pathway from childhood disruptive behavioral problems to adult criminality and violent behaviors may best be prevented early in life, when behavioral patterns are more easily modified.53 The findings suggest that population-based screening followed by Internet-assisted parent training reaches those at greatest risk who often do not obtain services to receive early evidence-based preventive intervention. Some of the effect sizes were modest compared with some parenting program studies16,18,54 that targeted clinical samples. This finding may reflect the targeted young age group with more tendency of natural improvement in behavioral problems. Furthermore, the sample was screened from the general population, and the level of problems was less severe than in clinical samples, meaning there was less room for improvement.
The findings indicate the effectiveness of a parent training intervention that incorporates interactive web technology to provide a personalized and sustainable intervention for the public health system. Internet-assisted parent training combined with support from a professional telephone coach was successful in achieving high participation rates, changing child disruptive behavior, and strengthening parental skills. Furthermore, parents reported high satisfaction with the program. These findings align with a Canadian Strongest Families parent training study35 with clinical samples using handbooks, videos, and telephone coaching. A significant percentage of parents are interested in Internet-assisted or telephone-supported parenting programs.26,55 However, to our knowledge, only a limited number of Internet-based treatments that target children’s psychiatric problems have been evaluated. The Internet-based components can be easily updated, the format is standardized and not therapist dependent, and it is more easily accessible for participants. Combining technology with remote involvement of professionals may assist in disseminating key elements of other evidence-based interventions to populations who would not otherwise be able to receive them.
The improvements in child problems were maintained until 12-month follow-up. Inclusion of 2 booster sessions after 6-month follow-up possibly had an effect on further improvement in parenting skills. Booster sessions for chronic problems, such as disruptive behavior, are often seen as an essential part of treatment but, to date, have received little study.56
Some methodologic constraints should be noted. First, only parental report of child behavior was used in the analyses. Direct observations of parenting, clinical observations, or teacher ratings would be helpful to validate the reported changes. However, the target group was 4-year-old children in Finland, where children start school at 7 years of age. Second, future studies need to determine whether the improvement in the intervention group resulted from the Internet sessions or the telephone calls (or the combination) or from the length and intensity of the intervention. Third, assessment of change in functional impairment was not conducted.
The study results are from Finland, a Nordic welfare state with a universal health care system. The educational level of Finnish parents in general is high compared with many other countries. The effects of the program are over and above the potential benefits associated with these factors. However, only a few children with mental health needs are referred to services.57 The control group clearly received more services than most preschool children with behavioral problems. Children were actively screened from the population, which is not common practice. None of screened children were taking psychotropic medication at baseline or during the study. In Finland, use of psychotropic medication in early childhood is rare, and child psychiatric medication practices for young children are different from US practices.58-60
Because of increased demands of child mental health services, long waiting lists, and the high cost of traditional approaches, our findings have important public health implications. Despite the high prevalence and significant associated burden, there is an enormous gap worldwide in the provision of treatment for mental disorders in children. Perhaps the most challenging barrier to service provision is the great shortage of skilled human resources to address child mental disorders in most regions of the world, even in countries with public health care, such as Finland. The study reveals the feasibility of a parent training intervention offered to parents of high-risk children screened from the population and referred to Internet-assisted parent training with telephone coaching. Given flexibility, anonymity, and ease of access, remote interventions have important benefits for reaching at-risk individuals. The strategy of population-based screening of children at an early age and offering Internet-assisted parent training that uses telephone coaching could be a promising solution for providing early prevention and intervention for a variety of child mental health problems.
Submitted for Publication: October 22, 2015; final revision received December 18, 2015; accepted December 21, 2015.
Corresponding Author: Andre Sourander, MD, PhD, Department of Child Psychiatry, University of Turku, Lemminkäisenkatu 3/Teutori 3.krs, 20014, Finland (email@example.com).
Published Online: February 24, 2016. doi:10.1001/jamapsychiatry.2015.3411.
Author Contributions: Drs Sourander and Hinkka-Yli-Salomäki had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Sourander, McGrath, Ristkari, Cunningham, Huttunen, Hinkka-Yli-Salomäki, Kinnunen, Vuorio, Sinokki, Fossum, Unruh.
Acquisition, analysis, or interpretation of data: Sourander, Ristkari, Cunningham, Huttunen, Lingley-Pottie, Hinkka-Yli-Salomäki, Kinnunen, Vuorio, Sinokki, Unruh.
Drafting of the manuscript: Sourander, McGrath, Ristkari, Cunningham, Huttunen, Hinkka-Yli-Salomäki, Kinnunen, Vuorio, Sinokki, Fossum, Unruh.
Critical revision of the manuscript for important intellectual content: Sourander, McGrath, Ristkari, Cunningham, Huttunen, Lingley-Pottie, Kinnunen, Vuorio, Sinokki.
Statistical analysis: Hinkka-Yli-Salomäki.
Obtained funding: Sourander, McGrath, Cunningham, Unruh.
Administrative, technical, or material support: Sourander, Ristkari, Huttunen, Lingley-Pottie, Sinokki.
Study supervision: Sourander, McGrath, Lingley-Pottie, Fossum.
Conflict of Interest Disclosures: Strongest Families Institute (SFI) is a not-for-profit organization that provides services to Canadian families. Dr McGrath reported serving as cofounder and chair of the SFI board of directors; he derives no financial benefit. Dr Lingley-Pottie reported serving as cofounder and being employed full time as the president and chief executive officer of SFI. She provides her academic and clinical consultation as an in-kind contribution to this study. Dr Lingley-Pottie may commercialize the Intelligent Research Information System platform in the future. Dr Cunningham reported receiving royalties from materials and workshops on Counseling Options and Parent Education, a large group parent training program. He receives salary support from and holds shares in Brief Child and Family Phone Interview Inc, a company that provides children’s mental health screening and outcome measurement systems. No other disclosures were reported.
Funding/Support: The research was funded by Skidi-Kids, a Finland Canada program funded by grant 135136 from the Academy of Finland (Dr Sourander) and grant 193146 from the Canadian Institutes of Health Research (Drs McGrath and Lingley-Pottie), the Jane and Aatos Erkko Foundation (Dr Sourander), the Association of Friends of the University Children's Hospitals (Dr Sourander), and Turku University Hospital Expert Responsibility Area Research Funding (Dr Sourander). Dr McGrath’s research was supported by a Canada Research Chair. Dr Cunningham’s participation was supported by the Jack Laidlaw Chair in Patient-Centered Health Care.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
Additional Contributions: We acknowledge Katja Pihlava, LicAPsych, University of Turku, for her contribution to the study design; Anne Penttinen, MHC, Turku University Hospital, Marja Guillaume, MA, Turku University Hospital, Kaisa Voutilainen, BC, Turku University Hospital, and Katja Pöllänen, BS, University of Turku, for acquisition of data; Jarna Lindroos, MA, University of Turku, for administrative work and helping to prepare the manuscript; and the public health directors and health nurses at the child health clinics in Finland who assisted in the study. Mss Pihlava, Penttinen, Guillaume, Voutilainen, Pöllänen, and Lindroos received compensation for their contribution.
Additional Information: Because of the Finnish national legislation for data privacy, the research data can only be assigned to third parties if the researchers apply for such permission from the governmental authorities and if such permission is granted.
Create a personal account or sign in to: