Effect of Clinician Training in the Modular Approach to Therapy for Children vs Usual Care on Clinical Outcomes and Use of Empirically Supported Treatments: A Randomized Clinical Trial | Adolescent Medicine | JAMA Network Open | JAMA Network
[Skip to Navigation]
Visual Abstract. Effect of Clinician Training in the Modular Approach to Therapy for Children (MATCH) vs Usual Care on Clinical Outcomes and Use of Treatments
Effect of Clinician Training in the Modular Approach to Therapy for Children (MATCH) vs Usual Care on Clinical Outcomes and Use of Treatments
Figure.  Flow Diagram of Youth Participants
Flow Diagram of Youth Participants

MATCH indicates Modular Approach to Therapy for Children.

Table 1.  Baseline Characteristics by Treatment Group
Baseline Characteristics by Treatment Group
Table 2.  Diagnoses by Category at Baseline by Treatment Group
Diagnoses by Category at Baseline by Treatment Group
Table 3.  Clinical Outcomes: Trajectories of Change by Treatment Groupa
Clinical Outcomes: Trajectories of Change by Treatment Groupa
Table 4.  Service Delivery Outcomes by Treatment Group
Service Delivery Outcomes by Treatment Group
1.
Gluckman  PD.  Improving the Transition: Reducing Social and Psychological Morbidity During Adolescence. Office of the Prime Minister’s Chief Science Advisor; 2011.
2.
Kessler  RC, Berglund  P, Demler  O, Jin  R, Merikangas  KR, Walters  EE.  Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.   Arch Gen Psychiatry. 2005;62(6):593-602. doi:10.1001/archpsyc.62.6.593PubMedGoogle ScholarCrossref
3.
Weisz  JR, Kuppens  S, Ng  MY,  et al.  What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice.   Am Psychol. 2017;72(2):79-117. doi:10.1037/a0040360PubMedGoogle ScholarCrossref
4.
Weisz  JR, Kuppens  S, Eckshtain  D, Ugueto  AM, Hawley  KM, Jensen-Doss  A.  Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis.   JAMA Psychiatry. 2013;70(7):750-761. doi:10.1001/jamapsychiatry.2013.1176PubMedGoogle ScholarCrossref
5.
Weisz  JR, Southam-Gerow  MA, Gordis  EB,  et al.  Cognitive-behavioral therapy versus usual clinical care for youth depression: an initial test of transportability to community clinics and clinicians.   J Consult Clin Psychol. 2009;77(3):383-396. doi:10.1037/a0013877PubMedGoogle ScholarCrossref
6.
Chorpita  BF, Daleiden  EL, Park  AL,  et al.  Child STEPs in California: a cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress.   J Consult Clin Psychol. 2017;85(1):13-25. doi:10.1037/ccp0000133PubMedGoogle ScholarCrossref
7.
Weisz  JR, Chorpita  BF, Palinkas  LA,  et al; Research Network on Youth Mental Health.  Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial.   Arch Gen Psychiatry. 2012;69(3):274-282. doi:10.1001/archgenpsychiatry.2011.147PubMedGoogle ScholarCrossref
8.
Peters  JL.  Undergraduate Training in Child and Adolescent Mental Health: A Review. Werry Centre for Child and Adolescent Mental Health, University of Auckland; 2003.
9.
Lucassen  MF, Stasiak  K, Crengle  S,  et al.  Modular Approach to Therapy for Anxiety, Depression, Trauma, or Conduct Problems in outpatient child and adolescent mental health services in New Zealand: study protocol for a randomized controlled trial.   Trials. 2015;16(1):457. doi:10.1186/s13063-015-0982-9PubMedGoogle ScholarCrossref
10.
Mental Health Commission.  The Funding Needed for Mental Health Services in New Zealand. Mental Health Commission; 1998.
11.
The Werry Workforce—Wharaurau.  2016 Stocktake of Infant, Child and Adolescent Mental Health and Alcohol and Other Drug Services in New Zealand. The Werry Workforce Whāraurau for Infant, Child & Adolescent Mental Health Workforce Development, The University of Auckland; 2017.
12.
Chorpita  BF, Weisz  JR.  Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct problems (MATCH-ADTC). Vol 13. PracticeWise LLC; 2009.
13.
Chorpita  BF, Daleiden  EL, Weisz  JR.  Modularity in the design and application of therapeutic interventions.   Appl Prev Psychol. 2005;11(3):141-156. doi:10.1016/j.appsy.2005.05.002Google ScholarCrossref
14.
Weisz  JR, Chorpita  BF, Frye  A,  et al; Research Network on Youth Mental Health.  Youth top problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy.   J Consult Clin Psychol. 2011;79(3):369-380. doi:10.1037/a0023307PubMedGoogle ScholarCrossref
15.
Achenbach  T, McConaughy  S, Ivanova  M, Rescorla  L.  Manual for the ASEBA Brief Problem Monitor. BPM; 2011.
16.
Bearman  SK, Herren  J, Weisz  JR.  Therapy Integrity in Evidence Based Interventions: Observational Coding System, Coding Manual. University of Texas at Austin; 2012.
17.
Goodman  R.  The Strengths and Difficulties Questionnaire: a research note.   J Child Psychol Psychiatry. 1997;38(5):581-586. doi:10.1111/j.1469-7610.1997.tb01545.xPubMedGoogle ScholarCrossref
18.
Stevens  K.  Assessing the performance of a new generic measure of health-related quality of life for children and refining it for use in health state valuation.   Appl Health Econ Health Policy. 2011;9(3):157-169. doi:10.2165/11587350-000000000-00000PubMedGoogle ScholarCrossref
19.
Goodman  R, Ford  T, Richards  H, Gatward  R, Meltzer  H.  The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology.   J Child Psychol Psychiatry. 2000;41(5):645-655. doi:10.1111/j.1469-7610.2000.tb02345.xPubMedGoogle ScholarCrossref
20.
Chorpita  BF, Park  A, Tsai  K,  et al; Research Network on Youth Mental Health.  Balancing effectiveness with responsiveness: therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial.   J Consult Clin Psychol. 2015;83(4):709-718. doi:10.1037/a0039301PubMedGoogle ScholarCrossref
21.
Weisz  JR, Bearman  SK, Ugueto  AM,  et al.  Testing Robustness of Child STEPs Effects with Children and Adolescents: a randomized controlled effectiveness trial.   J Clin Child Adolesc Psychol. Published online September 13, 2019. doi:10.1080/15374416.2019.1655757PubMedGoogle Scholar
22.
Thomassin  K, Marchette  LK, Weisz  JR.  Practitioners’ use and evaluation of transdiagnostic youth psychotherapy years after training and consultation have ended.   Adm Policy Ment Health. 2019;46(6):821-832. doi:10.1007/s10488-019-00962-xPubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Psychiatry
    August 17, 2020

    Effect of Clinician Training in the Modular Approach to Therapy for Children vs Usual Care on Clinical Outcomes and Use of Empirically Supported Treatments: A Randomized Clinical Trial

    Author Affiliations
    • 1Department of Psychological Medicine, School of Medicine, FMHS University of Auckland, Auckland, New Zealand
    • 2School of Health, Wellbeing and Social Care, The Open University, Milton Keynes, United Kingdom
    • 3Department of Psychology, Harvard University, Cambridge, Massachusetts
    • 4Department of Psychological Medicine, Christchurch School of Medicine, University of Otago, Christchurch, New Zealand
    • 5Department of Educational Psychology, The University of Texas at Austin, Austin
    • 6Department of Psychiatry and Behavioral Sciences, McGovern Medical School, The University of Texas Health Science Center at Houston, Houston
    • 7Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, Rhode Island
    • 8Le Va, Harakeke House, Manukau, Auckland, New Zealand
    • 9Infant, Child and Adolescent Mental Health Services, Waikato District Health Board, Hamilton, New Zealand
    • 10Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
    JAMA Netw Open. 2020;3(8):e2011799. doi:10.1001/jamanetworkopen.2020.11799
    Visual Abstract. Effect of Clinician Training in the Modular Approach to Therapy for Children (MATCH) vs Usual Care on Clinical Outcomes and Use of Treatments
    Effect of Clinician Training in the Modular Approach to Therapy for Children (MATCH) vs Usual Care on Clinical Outcomes and Use of Treatments
    Key Points español 中文 (chinese)

    Question  Is training in the Modular Approach to Therapy for Children (MATCH) associated with more use of empirically supported treatments, better clinical outcomes, and better service efficiency than usual care?

    Findings  This randomized clinical trial found that training clinicians in MATCH was associated with high levels of adherence to empirically supported treatments (80.0%) compared with usual care (57.0%), but it was not associated with improved clinical outcomes or efficiency.

    Meaning  These findings suggest that training in MATCH increases clinicians’ use of empirically supported treatments but does not necessarily improve clinical outcomes.

    Abstract

    Importance  The Modular Approach to Therapy for Children (MATCH) was developed to address the comorbidities common among clinically referred youth, with beneficial outcomes shown in 2 US randomized clinical trials, where it outperformed both usual clinical care and single disorder–specific treatments.

    Objective  To determine whether MATCH training of clinicians would result in more use of empirically supported treatment (EST) and better clinical outcomes than usual care (UC) in the publicly funded, multidisciplinary context of New Zealand.

    Design, Setting, and Participants  This multisite, single-blind, computer-randomized clinical effectiveness trial compared MATCH with UC in child and adolescent mental health services in 5 regions of New Zealand. Recruitment occurred from March 2014 to July 2015, and a 3-month follow-up assessment was completed by May 2016. Clinicians at participating child and adolescent mental health services were randomized (1:1) to undertake training in MATCH or to deliver UC, and young people with anxiety, depression, trauma-related symptoms, or disruptive behavior seeking treatment at child and adolescent mental health services were randomized (1:1) to receive MATCH or UC. Participants and research assistants were blind to allocation. Data analysis was performed from April 2016 to July 2017.

    Interventions  MATCH comprises EST components for flexible management of common mental health problems. UC includes case management and psychological therapies. Both can include pharmacotherapy.

    Main Outcomes and Measures  There were 3 primary outcomes: trajectory of change of clinical severity, as measured by weekly ratings on the Brief Problem Monitor (BPM); fidelity to EST content, as measured by audio recordings of therapy sessions coded using the Therapy Integrity in Evidence Based Interventions: Observational Coding System; and efficiency of service delivery, as measured by duration of therapy (days) and clinician time (minutes).

    Results  The study included 65 clinicians (mean age, 38.7 years; range, 23.0-64.0 years; 54 female [83%]; MATCH, 32 clinicians; UC, 33 clinicians) and 206 young people (mean age, 11.2 years; range 7.0-14.0 years; 122 female [61%]; MATCH, 102 patients; UC, 104 patients). For the BPM total ratings for parents, there was a mean (SE) slope of –1.04 (0.14) (1-year change, −6.12) in the MATCH group vs –1.04 (0.10) (1-year change, −6.17) in the UC group (effect size, 0.00; 95% CI, −0.27 to 0.28; P = .96). For the BPM total for youths, the mean (SE) slope was –0.74 (0.15) (1-year change, −4.35) in the MATCH group vs –0.73 (0.10) (1-year change, −4.32) in the UC group (effect size, −0.02; 95% CI, −0.30 to 0.26; P = .97). Primary analyses (intention-to-treat) showed no difference in clinical outcomes or efficiency despite significantly higher fidelity to EST content in the MATCH group (58 coded sessions; mean [SD], 80.0% [20.0%]) than the UC group (51 coded sessions; mean [SD], 57.0% [32.0%]; F(1,108) = 23.0; P < .001). With regard to efficiency of service delivery, there were no differences in total face-to-face clinician time between the MATCH group (mean [SD], 806 [527] minutes) and the UC group (mean [SD], 677 [539] minutes) or the overall duration of therapy between the MATCH group (mean [SD], 167 [107 days]) and the UC group (mean [SD], 159 [107] days).

    Conclusions and Relevance  MATCH significantly increased adherence to EST practices but did not improve outcomes or efficiency. The nonsuperiority of MATCH may be attributable to high levels of EST use in UC in New Zealand.

    Trial Registration  Australian New Zealand Clinical Trials Registry Identifier: ACTRN12614000297628

    Introduction

    Mental health problems in children and adolescents are common and persistent.1,2 There are effective therapies available3; however, delivering these therapies in clinical practice has been challenging.4 This is partly because the evidence is primarily available for single disorders or a homogeneous cluster of problems,5 whereas clinicians are faced with comorbid presentations that may change in focus during therapy. Clinicians may adopt a pragmatic but eclectic approach, unintentionally eroding the impact of carefully designed best clinical practice. The Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH-ADTC or MATCH, for brevity) has been designed to address the issues of flexibility and clinical complexity after a brief but comprehensive training program and has been shown to be more effective and efficient than usual care (UC).6,7

    In New Zealand, preregistration courses for mental health professionals do not include in-depth training in psychological therapies for children and adolescents.8 For many working in child and adolescent mental health services (CAMHS), training in empirically supported treatments (ESTs), if it occurs, takes place after qualifying, either informally on the job or in courses that are time-consuming. Although there have been efforts to roll out training in ESTs in New Zealand, this is piecemeal so that having consistent delivery of ESTs is challenging.

    We sought to determine whether training in MATCH could improve UC in New Zealand as it had been shown to do in studies in the US. Our primary hypotheses were that training CAMHS clinicians in MATCH, compared with UC, would increase the delivery of ESTs, improve clinical outcomes, and yield equal or better efficiency of service delivery.

    Methods

    The study protocol has been published elsewhere9 and is available in Supplement 1. This study was approved by New Zealand’s Health and Disability Ethics Committee and was overseen by a Data Monitoring Committee of the Health Research Council of New Zealand. Participants provided written informed consent. This study follows the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

    Trial Design

    We performed a multisite, single-blind, randomized clinical effectiveness trial comparing MATCH with UC in CAMHS in 5 District Health Boards in New Zealand. In New Zealand, 20 District Health Boards are responsible for funding or providing health services within their district or geographical region. Participating teams in the District Health Boards provided services in rural and urban settings and included 1 Kaupapa Māori and 2 Pacific teams. Kaupapa Māori teams are those in which the philosophical doctrine incorporates the knowledge, skills, attitudes, and values of Māori (ie, Indigenous) society. Pacific teams are those in which people originating from other Pacific Islands are seen in services designed to incorporate the knowledge, skills, attitudes, and values of Pasifika societies.

    Data were collected at baseline, during treatment, and 3 months after the end of treatment. The recruitment period was March 2014 to July 2015, with follow-up completed in May 2016.

    Randomization, Blinding, and Allocation Concealment

    Randomization was at 2 levels: first, clinicians were randomized in a 1:1 ratio stratified by service or team to undertake training in MATCH at the start or at the completion of the study; and second, young people and their families were randomized in a 1:1 ratio stratified by sex and ethnicity (Māori, Pacific, or an other ethnicity) to receive MATCH or UC. The major ethnic groups in New Zealand include Māori, the Indigenous people of New Zealand (14.9% of the population), New Zealand European people (74% of the population), Asian people (11.8% of the population), and non-Māori Pacific people (7.4% of the population) who have settled in New Zealand from the Pacific Islands such as Samoa. Inequities in the health and mental health for Māori and Pacific people have led to specific mental health services being set up in some regions to attempt to reduce the inequities. However, many Māori and Pacific young people are seen in the mainstream CAMHS.

    Electronically generated randomization sequences ensured allocation concealment. Young people, their families, and the research assistants collecting data were blind to allocation.

    Participants
    Inclusion and Exclusion Criteria for the Clinicians

    Clinicians were eligible if they provided clinical treatment to young people and their families at participating CAMHS and provided written, informed consent. They were invited to take part in the study by their service manager.

    Inclusion and Exclusion Criteria for the Young People

    English-speaking young people, aged 7 to 14 years, referred to CAMHS with a primary presenting problem that included anxiety, depression, trauma-related symptoms, or disruptive behavior were eligible for the study. Families were invited to take part in the study by the intake clinician and were included if parents provided written, informed consent and the young person assented. Youth were excluded if they were already being treated, the primary focus of treatment was for another disorder or problem, or a sibling had already been recruited into the study.

    Setting

    CAMHS provide mental health care for young people aged 0 to 19 years.10 CAMHS are organized into multidisciplinary teams of registered health practitioners, primarily nurses with training in mental health and social workers.11

    Interventions
    MATCH

    MATCH is a manualized program of 33 modules addressing 4 problem domains commonly encountered in clinical practice.12 MATCH combines written resources, a framework for choice of intervention, and guidance from an online system for monitoring progress and providing timely feedback to clinicians. It combines empirically supported elements of existing therapies within 1 protocol and accommodates comorbidity and changes in clinical presentation during therapy.13

    Therapists randomized to deliver MATCH were provided with 5 days’ training and then 1-hour weekly Skype-based group consultation (mean [SD] group size, 4 [1.4] individuals; range, 2-6 individuals) provided by MATCH experts (A.M.U., S.K.B., and J.H.). At the start of treatment, the young person and their family collaboratively established the top problems to be addressed.14 Clinicians used these problems to tailor treatment. Pharmacotherapy was used as part of standard CAMHS practice, as in UC.

    Usual Care

    Usual care includes case management, psychological therapies, and pharmacotherapy. UC is overseen in multidisciplinary team meetings, typically focused on brief reviews and care coordination.

    Outcomes

    Demographic information was provided by parents at enrollment in the study. Because of disparities in mental health outcomes for Māori and Pacific people in New Zealand, ethnicity data were collected.

    Research assistants who were blinded to participant treatment group collected most clinical measures by telephone. Additional data were collected from clinicians after the participants had been discharged.

    Primary Outcomes

    In keeping with the primary hypotheses, there were 3 primary outcomes. First, the trajectory of change of clinical severity was assessed using the parent-rated Brief Problem Monitor (BPM)15 administered weekly. Second, the fidelity (ie, adherence and competence) with which therapists used EST content12 was measured from audio recordings of therapy sessions using the methods and coding system adapted from those used in the initial trial of MATCH.16 Therapist competence was rated as follows: 0, not at all; 1, superficial or incomplete; 2, adequate but not optimal; 3, thorough; and 4, expert. Ten percent of recorded therapy sessions (MATCH and UC) were assessed by the research coding team (blind to treatment group). Ten percent of this sample were independently double-coded and had acceptable interrater agreement (mean intraclass correlation coefficient [ICC] on adherence, 0.70; mean ICC on competence, 0.67). Because the results of our initial coding of therapy sessions of UC were markedly different from those of previous studies,6,7 we conducted a second round of coding with a subset of 100 randomly selected sessions, coded by experienced independent coders from the US, with 20% of this sample independently assessed by a coder in New Zealand to check for systematic discrepancies between countries. The interrater agreement was acceptable across US and New Zealand coders for adherence (mean ICC, 0.74) and competence (mean ICC, 0.73). Third, the efficiency of service delivery (extracted from logs completed by clinicians) was assessed using duration of therapy (days), clinician time (minutes), and the number of therapy sessions attended and missed.

    Secondary Outcomes

    Secondary outcomes included the youth-rated BPM administered weekly,15 the parent- and youth-rated Strengths and Difficulties Questionnaire administered monthly,17 the parent- and youth-rated Top Problems Assessment administered weekly,14 the Child Health Utility (a quality of life assessment administered at baseline, discharge, and follow-up),18 and the number and type of diagnoses assessed at baseline and discharge using the Development and Well-Being Assessment.19 Prescribed medications were recorded at baseline, discharge, and follow-up. Clinician satisfaction with therapy was assessed using the Therapist Satisfaction Index.20 A treatment satisfaction questionnaire for parents and youth was developed for this study.

    Measures of Harm

    Reports of serious adverse events9 were collected and reported to the Data Monitoring Committee. At the request of the Data Monitoring Committee, we developed a measure for moderate adverse events that were reported during the study and collected from the parents through specific enquiry at follow-up.

    Sample Size and Power

    Details of the initial sample size and changes to it have been published elsewhere.9 From the results of previous studies,7 we estimated that 200 participants would be needed for 90% power to detect a significant difference in the change that was clinically important (ie, >2 units, with an effect size of approximately 0.45) on the parent BPM change (2-tailed α = .05).

    Statistical Analysis

    The primary analyses used the intention-to-treat population. Because neither MATCH nor UC has a fixed duration, testing of the primary clinical hypothesis compared the trajectory of change across time on the parent-rated BPM (total score) as per the original studies.6,7 A mixed-effects regression model was used, with outcome = a0(intercept) + a1(treatment group) + a2(time) + a3(treatment × time), with treatment and time (loge day) treated as fixed effects and the participant intercepts and slopes as random events. Sensitivity analyses were performed to test for the consistency of treatment response across medication use, clinician site, and previous evidence-based therapy training categories, by testing the appropriate interaction terms. Per-protocol analyses included participants who completed the study, received therapy as per their allocated treatment group, and had completed at least 4 therapy sessions. For the second and third primary hypotheses and all treatment satisfaction measures, between-group comparisons were made using univariable ANOVA. Trajectories of change for secondary clinical outcomes were compared for the Strengths and Difficulties Questionnaire and Top Problems Assessment as described previously in this article. Changes in the Child Health Utility were compared between groups using univariable ANOVA to assess changes between baseline, discharge, and at 3-month follow-up. Two-tailed P < .05 was considered to indicate statistical significance. Data analysis was performed using SPSS statistical software version 25 (IBM Corp) from April 2016 to July 2017.

    Results
    Group Characteristics

    Sixty-five clinicians (mean age, 38.7 years; range, 23.0-64.0 years; 54 female [83%]; 29 New Zealand European [44.6%], 11 Māori [16.9%], 6 Pacific [9.2%], 2 Asian [3.1%], and 17 other 17 [26.2%], including 5 British, 4 American, and 3 South African) were recruited and randomized to receive MATCH training (32 clinicians) or to deliver UC (33 clinicians). They were balanced with regard to age, sex, and ethnicity.

    Clinicians included 19 nurses trained in mental health (29.2%), 19 social workers (29.2%), 12 clinical psychologists (18.5%), 11 occupational therapists (16.9%), and 4 other clinicians (6.2%). They had been in their current CAMHS roles for a mean (SD) of 3.5 (3.3) years (range, <3 months to 15 years). Overall clinical practice experience ranged from less than 1 year to 40 years (mean [SD], 11.4 [9.4] years). A similar percentage between groups had previously received training in relevant ESTs, such as cognitive behavioral therapy or parent management training (MATCH, 12 clinicians [38%]; UC, 11 clinicians [33%]) and had at least 10 years postqualification practice (MATCH, 15 clinicians [47%]; UC, 15 clinicians [45%]).

    Two hundred six young people (mean age, 11.2 years; range 7.0-14.0 years; 122 female [61%]) (Table 1), were recruited and randomized, with 6 young people (5 from MATCH, 1 from UC) leaving the study before treatment began or any data were collected. Outcome data for at least 2 time points were available for the remaining 200 participants (97%), and 164 participants (80%) completed the 3-month follow-up assessment (Figure).

    The 2 groups of youth participants (MATCH and UC) were balanced with regard to age, sex, and ethnicity (Table 1). The ethnic make-up was similar to the general New Zealand population, apart from an underrepresentation of Asian people (Table 1 and Table 2; eTable 1 in Supplement 2).

    Clinical Outcomes

    There were no significant differences between groups on the primary outcome measure, the trajectory of change for total difficulties on the BPM as reported by parents (mean [SE] slope, –1.04 [0.14] and 1-year change of −6.12 in the MATCH group vs –1.04 [0.10] and 1-year change of −6.17 in the UC group; effect size, 0.00; 95% CI, – 0.27 to – 0.28; P = .96) and as reported by youths (mean [SE] slope, –0.74 [0.15] and 1-year change of −4.35 in the MATCH group vs –0.73 [0.10] and 1-year change of −4.32 in the UC group; effect size, −0.02; 95% CI, −0.30 to 0.26; P = .97) or any other clinical outcome measure (Table 3). Both groups improved significantly, with effects maintained to 3 months of follow-up (eTable 2 in Supplement 2). Sensitivity analyses on the primary clinical outcome showed that the result was not affected by medication use at baseline (F1,183 = 0.248; P = .78), prior relevant EST accreditation of the clinician (F1,183 = 0.296; P = .14), or study site (F1,184 = 0.483; P = .94). There were also no significant differences in the per-protocol analysis on the primary clinical outcome (72 participants in the MATCH group vs 84 participants in the UC group).

    There were no significant differences between groups in the number of diagnoses from before to after treatment (eTable 3 in Supplement 2). Prescription of medications for psychiatric conditions did not differ between treatment groups at baseline or during therapy (eTable 4 in Supplement 2).

    Delivery of EST

    EST content adherence was significantly higher in the MATCH group (mean [SD] level of adherence, 80.0% [20.0%] for 58 coded sessions) than in the UC group (mean [SD] level of adherence, 57.0% [32.0%] for 51 coded sessions) (F1,108 = 23.0; P < .001). Therapist competence ratings in delivery of EST content were also significantly higher in the MATCH group (mean [SD], 2.30 [0.57]) than in the UC group (mean [SD], 1.75 [0.50]) (F1,108 = 8.0; P = .001), with both adherence and competence corresponding most closely to an adequate but not optimal rating.10 The second round of coding showed almost identical findings, with greater adherence in the MATCH group (58 participants; mean [SD] level of adherence, 81.0% [22.1%]) than in the UC group (42 participants; mean [SD] level of adherence, 56.0% [30.2%]) (F1,76 = 10.7; P = .002) and greater therapist competence in the MATCH group (mean [SD] rating, 2.25 [0.62]) compared with the UC group (mean [SD] rating, 1.76 [0.64]) (F1,79 = 9.1; P = .003).

    Efficiency of Service Delivery

    Participants receiving MATCH attended significantly more therapy sessions than did participants in the UC group (mean [SD], 13.4 [8.4] sessions vs 10.7 [7.7] sessions; F1,190 = 5.6; P = .02). However, there were no significant differences in total face-to-face clinician time (mean [SD], 806 [527] minutes in the MATCH group vs 677 [539] minutes in the UC group) or the overall duration of therapy (mean [SD], 167 [107] days in the MATCH group vs 159 [107] days in the UC group) (Table 4).

    Satisfaction With Treatment

    MATCH therapists were significantly more satisfied with the treatment they had provided (mean [SD] Therapist Satisfaction Index total scores, 4.0 [0.6] for MATCH delivered to 79 participants vs 3.7 [0.6] for UC delivered to 88 participants; F1,162 = 9.567; P = .002). Parent-rated and youth-rated total satisfaction scores (maximum of 32 and 20, respectively) were high and not significantly different between treatment groups (mean [SD] scores for parents, 25.7 [6.5] for MATCH vs 24.3 [7.3] for UC; F1,170 = 1.780; P = .19; mean [SD] scores for youth, 16.4 [3.7] for MATCH vs 15.5 [4.2] for UC; F1,158 = 2.076; P = .16).

    Harms

    There were no significant group differences in terms of serious and moderate adverse events (eTable 5 in Supplement 2). Initial treatment focus by group is shown in eTable 6 in Supplement 2.

    Discussion

    Training in MATCH resulted in significantly improved delivery of ESTs by clinicians, and the trajectory of change in clinical outcomes resembled that found in other trials of MATCH.6,7 However, the increased delivery of EST did not translate into improved clinical outcomes or efficiency. Young people in both treatment groups improved similarly at discharge and maintained this improvement at 3 months’ follow-up. Effects were not moderated by individual services, clinician training, medication use, or initial focus of treatment.

    In 2 previous RCTs,6,7 MATCH improved clinical outcomes and service efficiency, increased EST use compared with UC, and increased use of standard manual-based behavioral and cognitive behavioral therapy.7 It is notable that similar clinical change was achieved in our study despite fewer therapy sessions and shorter duration than the previous studies. Our study was adequately powered and had more participants per group (200 participants across 2 groups) than the other 2 studies, with 174 participants across 3 groups7 and 138 participants across 2 groups.6 The training and weekly consultation in all 3 studies was provided by experts from the developers’ team. Despite the use of group rather than individual consultation, the delivery of EST (80.0% adherence in the present study compared with 83% in a previous study7) and the trajectory of clinical change in the MATCH group (−1.04 in the current study compared with −0.94 in a previous study7) was as good as that achieved in the first trial of MATCH.7 The populations studied, clinical severity and context, and measures used in the 3 studies were very similar. The main difference between the studies lies in the extent of EST use in the UC groups, which was 7% to 8% in both previous studies6,7 compared with 57.0% in this study. In addition, the trajectory of change in our UC group was equivalent to that in the MATCH group, in contrast to the flatter trajectory in UC in the other 2 studies.6,7 An additional RCT comparing MATCH with UC has recently been published21 and found delivery rates of EST of 67% for MATCH clinicians and 27% for UC clinicians, but there were no differences between MATCH and UC for any clinical measure. Perhaps 57.0% adherence to ESTs results in clinical outcomes similar to those for 80.0% adherence, whereas 7% to 8%, as reported in the US,6,7 does not. The most recent trial of MATCH22 also showed MATCH clinicians providing higher levels of EST than UC but no differences in clinical outcomes.

    Training in MATCH is consistently associated with improved levels of EST delivered, whereas the variation in level of EST provided in UC across the 4 RCTs is marked. We considered the possibility of contamination between MATCH and UC, with UC clinicians learning MATCH techniques through team meetings that are the norm in New Zealand. However, case discussions in team meetings are not detailed, and measures to protect against contamination in the current study were identical to those in previous studies.6,7 The intensive monitoring involved in this study that may have improved UC is also common to all studies, and so is unlikely to explain the differences. The efforts undertaken to upskill the workforce through training in individual ESTs over the last 2 decades may have been successful in improving UC in New Zealand.

    Strengths and Limitations

    Strengths of this study include the multisite clinical settings to maximize generalizability, measurement of adverse events and clinical changes, allocation concealment, and blinded assessments and analyses.9 Limitations include the underrepresentation of Asian young people, the single-blind nature of the study, and the low percentage of Development and Well-Being Assessment completion. The ICC ratings for the coding of ESTs were lower than we would have liked. There was no clear pattern of disagreement, although the sample of sessions used for coding fidelity, which was appropriate for assessing overall intercoder reliability of the system within New Zealand and between New Zealand and US coders, was not large enough to do more fine-grained analyses.

    Conclusions

    These findings suggest that a brief 5-day training in MATCH resulted in a significant increase in the delivery of ESTs. The lack of change in clinical outcome in our study despite a significant change in delivery of ESTs and in contrast to previous studies may be partly explained by the high level of EST delivered by UC clinicians. The question of how EST fidelity is related to clinical outcomes, and the part played by nonspecific factors such as warmth and empathy, warrants further attention.

    Back to top
    Article Information

    Accepted for Publication: May 18, 2020.

    Published: August 17, 2020. doi:10.1001/jamanetworkopen.2020.11799

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Merry SN et al. JAMA Network Open.

    Corresponding Author: Sally N. Merry, MD, Department of Psychological Medicine, School of Medicine, FMHS University of Auckland, Level 12 Auckland City Hospital Support Bldg, Private Bag 92019, Auckland 1142, New Zealand (s.merry@auckland.ac.nz).

    Author Contributions: Drs Merry and Frampton had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Merry, Lucassen, Stasiak, Weisz, Frampton, Bearman, Ugueto, Herren, Cribb-Su’a, Kingi-Uluave, Crengle.

    Acquisition, analysis, or interpretation of data: Merry, Hopkins, Lucassen, Weisz, Frampton, Ugueto, Herren, Cribb-Su’a, Loy, Hartdegen, Crengle.

    Drafting of the manuscript: Merry, Hopkins, Lucassen, Weisz, Bearman, Hartdegen, Crengle.

    Critical revision of the manuscript for important intellectual content: Merry, Hopkins, Lucassen, Stasiak, Weisz, Frampton, Bearman, Ugueto, Herren, Cribb-Su’a, Kingi-Uluave, Loy, Crengle.

    Statistical analysis: Frampton.

    Obtained funding: Merry, Lucassen, Stasiak, Crengle.

    Administrative, technical, or material support: Merry, Hopkins, Lucassen, Stasiak, Weisz, Bearman, Herren, Cribb-Su’a, Loy, Hartdegen.

    Supervision: Merry, Hopkins, Lucassen, Weisz, Bearman, Ugueto, Herren, Cribb-Su'a, Kingi-Uluave, Crengle.

    Conflict of Interest Disclosures: Dr Weisz reported receiving royalties from sales of Modular Approach to Treatment of Children with Anxiety, Depression, or Conduct Problems. Dr Cribb-Su’a reported receiving personal fees from Uniservices outside the submitted work and reported being an employee at one of the organizations from which participants were drawn during the course of the trial (Counties Manukau District Health Board); her role was managerial and she had no direct involvement in the running of the trial at this site. No other disclosures were reported.

    Funding/Support: The Health Research Council of New Zealand funded the study. Cure Kids and the Duke Family provide salary support for Drs Merry and Stasiak.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Data Sharing Statement: See Supplement 3.

    Additional Contributions: We thank the young people and families who participated in the study, the study clinicians and the different teams that participated in the study, the District Health Boards that supported the study, and the Data Monitoring Committee for their oversight and advice.

    References
    1.
    Gluckman  PD.  Improving the Transition: Reducing Social and Psychological Morbidity During Adolescence. Office of the Prime Minister’s Chief Science Advisor; 2011.
    2.
    Kessler  RC, Berglund  P, Demler  O, Jin  R, Merikangas  KR, Walters  EE.  Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.   Arch Gen Psychiatry. 2005;62(6):593-602. doi:10.1001/archpsyc.62.6.593PubMedGoogle ScholarCrossref
    3.
    Weisz  JR, Kuppens  S, Ng  MY,  et al.  What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice.   Am Psychol. 2017;72(2):79-117. doi:10.1037/a0040360PubMedGoogle ScholarCrossref
    4.
    Weisz  JR, Kuppens  S, Eckshtain  D, Ugueto  AM, Hawley  KM, Jensen-Doss  A.  Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis.   JAMA Psychiatry. 2013;70(7):750-761. doi:10.1001/jamapsychiatry.2013.1176PubMedGoogle ScholarCrossref
    5.
    Weisz  JR, Southam-Gerow  MA, Gordis  EB,  et al.  Cognitive-behavioral therapy versus usual clinical care for youth depression: an initial test of transportability to community clinics and clinicians.   J Consult Clin Psychol. 2009;77(3):383-396. doi:10.1037/a0013877PubMedGoogle ScholarCrossref
    6.
    Chorpita  BF, Daleiden  EL, Park  AL,  et al.  Child STEPs in California: a cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress.   J Consult Clin Psychol. 2017;85(1):13-25. doi:10.1037/ccp0000133PubMedGoogle ScholarCrossref
    7.
    Weisz  JR, Chorpita  BF, Palinkas  LA,  et al; Research Network on Youth Mental Health.  Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial.   Arch Gen Psychiatry. 2012;69(3):274-282. doi:10.1001/archgenpsychiatry.2011.147PubMedGoogle ScholarCrossref
    8.
    Peters  JL.  Undergraduate Training in Child and Adolescent Mental Health: A Review. Werry Centre for Child and Adolescent Mental Health, University of Auckland; 2003.
    9.
    Lucassen  MF, Stasiak  K, Crengle  S,  et al.  Modular Approach to Therapy for Anxiety, Depression, Trauma, or Conduct Problems in outpatient child and adolescent mental health services in New Zealand: study protocol for a randomized controlled trial.   Trials. 2015;16(1):457. doi:10.1186/s13063-015-0982-9PubMedGoogle ScholarCrossref
    10.
    Mental Health Commission.  The Funding Needed for Mental Health Services in New Zealand. Mental Health Commission; 1998.
    11.
    The Werry Workforce—Wharaurau.  2016 Stocktake of Infant, Child and Adolescent Mental Health and Alcohol and Other Drug Services in New Zealand. The Werry Workforce Whāraurau for Infant, Child & Adolescent Mental Health Workforce Development, The University of Auckland; 2017.
    12.
    Chorpita  BF, Weisz  JR.  Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct problems (MATCH-ADTC). Vol 13. PracticeWise LLC; 2009.
    13.
    Chorpita  BF, Daleiden  EL, Weisz  JR.  Modularity in the design and application of therapeutic interventions.   Appl Prev Psychol. 2005;11(3):141-156. doi:10.1016/j.appsy.2005.05.002Google ScholarCrossref
    14.
    Weisz  JR, Chorpita  BF, Frye  A,  et al; Research Network on Youth Mental Health.  Youth top problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy.   J Consult Clin Psychol. 2011;79(3):369-380. doi:10.1037/a0023307PubMedGoogle ScholarCrossref
    15.
    Achenbach  T, McConaughy  S, Ivanova  M, Rescorla  L.  Manual for the ASEBA Brief Problem Monitor. BPM; 2011.
    16.
    Bearman  SK, Herren  J, Weisz  JR.  Therapy Integrity in Evidence Based Interventions: Observational Coding System, Coding Manual. University of Texas at Austin; 2012.
    17.
    Goodman  R.  The Strengths and Difficulties Questionnaire: a research note.   J Child Psychol Psychiatry. 1997;38(5):581-586. doi:10.1111/j.1469-7610.1997.tb01545.xPubMedGoogle ScholarCrossref
    18.
    Stevens  K.  Assessing the performance of a new generic measure of health-related quality of life for children and refining it for use in health state valuation.   Appl Health Econ Health Policy. 2011;9(3):157-169. doi:10.2165/11587350-000000000-00000PubMedGoogle ScholarCrossref
    19.
    Goodman  R, Ford  T, Richards  H, Gatward  R, Meltzer  H.  The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology.   J Child Psychol Psychiatry. 2000;41(5):645-655. doi:10.1111/j.1469-7610.2000.tb02345.xPubMedGoogle ScholarCrossref
    20.
    Chorpita  BF, Park  A, Tsai  K,  et al; Research Network on Youth Mental Health.  Balancing effectiveness with responsiveness: therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial.   J Consult Clin Psychol. 2015;83(4):709-718. doi:10.1037/a0039301PubMedGoogle ScholarCrossref
    21.
    Weisz  JR, Bearman  SK, Ugueto  AM,  et al.  Testing Robustness of Child STEPs Effects with Children and Adolescents: a randomized controlled effectiveness trial.   J Clin Child Adolesc Psychol. Published online September 13, 2019. doi:10.1080/15374416.2019.1655757PubMedGoogle Scholar
    22.
    Thomassin  K, Marchette  LK, Weisz  JR.  Practitioners’ use and evaluation of transdiagnostic youth psychotherapy years after training and consultation have ended.   Adm Policy Ment Health. 2019;46(6):821-832. doi:10.1007/s10488-019-00962-xPubMedGoogle ScholarCrossref
    ×