Author Affiliations: Special Services Unit, Western Psychiatric Institute and Clinic and Departments of Psychiatry, Psychology, and Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania (Dr Kolko); College of Medicine (Dr Campo) and Center for Innovation in Pediatric Practice, The Research Institute (Dr Kelleher), Nationwide Children's Hospital (Dr Campo), The Ohio State University, Columbus; and VA Ann Arbor Center for Clinical Management Research and Department of Psychiatry, University of Michigan (Dr Kilbourne).
Objectives To evaluate the feasibility and clinical benefits of an integrated mental health intervention (doctor-office collaborative care [DOCC]) vs enhanced usual care (EUC) for children with behavioral problems.
Design Cases were assigned to DOCC and EUC using a 2:1 randomization schedule that resulted in 55 DOCC and 23 EUC cases.
Setting Preassessment was conducted in 4 pediatric primary care practices. Postassessment was conducted in the pediatric or research office. Doctor-office collaborative care was provided in the practice; EUC was initiated in the office but involved a facilitated referral to a local mental health specialist.
Participants Of 125 referrals (age range, 5-12 years), 78 children participated.
Interventions Children and their parents were assigned to receive DOCC or EUC.
Main Outcome Measures Preassessment diagnostic status was evaluated using the Schedule for Affective Disorders and Schizophrenia for School-aged Children. Preassessment and 6-month postassessment ratings of behavioral and emotional problems were collected from parents using the Vanderbilt Attention-Deficit/Hyperactivity Disorder Diagnostic Parent Rating Scale, as well as individualized goal achievement ratings forms. At discharge, care managers and a diagnostic evaluator completed the Clinical Global Impression Scale, and pediatricians and parents completed satisfaction and study feedback measures.
Results Group comparisons found significant improvements for DOCC over EUC in service use and completion, behavioral and emotional problems, individualized behavioral goals, and overall clinical response. Pediatricians and parents were highly satisfied with DOCC.
Conclusion The feasibility and clinical benefits of DOCC for behavioral problems support the integration of collaborative mental health services for common mental disorders in primary care.
The integration of evidence-based behavioral treatments in pediatric primary care and their evaluation in clinical trials represent an important service delivery development.1,2 A few outcome studies3- 5 of children with behavioral problems have reported modest benefits associated with Internet-based psychoeducation, on-site family intervention, and on-site parenting intervention. However, few sustainable models for managing child behavioral problems, the most common pediatric mental health concern in primary care, have been applied.
Collaborative care approaches based on the chronic care model (CCM) are especially relevant for pediatric settings given the flexible use of consultants, care managers (CMs) as educators, and clinicians working collaboratively with physicians.6 Research has documented enhanced outcomes for several adult disorders (eg, depression and anxiety), especially if there is access to a psychiatric consultant.7 Results of the few studies8- 10 among adolescents with depression suggest the effectiveness of using CMs to conduct treatment in pediatric settings. A major advantage of the CCM is its alignment with the medical home model. However, a primary gap is the absence of studies evaluating the feasibility and clinical benefits of collaborative care in treating children's behavioral problems.
As part of a program of treatment research in a primary care network (Services for Kids in Primary Care), our group compared a randomized trial that recently evaluated a protocol for an office-based nurse-administered intervention (PONI) in primary care, relative to enhanced usual care (EUC), in a sample of children referred for behavioral problems.10 Compared with EUC, the PONI was superior in improving service use, satisfaction, and child health and individualized behavioral targets, but both groups showed significant and comparable gains on several clinical outcomes.
To enhance the effect of the PONI condition in primary care, we adapted the CCM to develop a more collaborative approach in the management of children's behavioral problems and to enhance interdisciplinary communication and access to behavioral and psychopharmacotherapeutic interventions. This adaptation was called doctor-office collaborative care (DOCC).
Practice-based research network of local primary case practices developed with a formal administrative infrastructure and financial partnerships; lead doctors and office liaisons established; ongoing governance support with quarterly meetings and newsletters.
Expanded protocol targeting behavioral problems using specific modules: parent skills training (eg, parent management training and attention to parental functioning), child skills training (eg, anger control and social skills), parent-child psychoeducation and skills training (eg, communication and problem solving), peer/social enrichment (eg, social activities), school/teacher consultation (eg, rating scales, and behavior plans), and case or crisis management (ie, agency advocacy, calls to diffuse conflicts).10
Expanded modules on attention-deficit/hyperactivity disorder (ADHD) care management to address both medication and behavior management options (eg, use of Vanderbilt Attention-Deficit/Hyperactivity Disorder Diagnostic Parent Rating Scale, pediatrician recommendations and prescription, follow-up reviews, behavioral recommendations for homework, contingencies to target positive behaviors, and time-out) and brief anxiety management based on a recent manual11,12 (eg, self-monitoring, cognitive restructuring, relaxation skills).
Use of care managers to deliver and coordinate services with pediatricians; expanded continuum of care and delivery options (screening/assessment, psychoeducation, on-site treatment, and referral).
Clinical information system
Automated intake reports; alternative data collection formats (tablet personal computer, and Internet); shared electronic information regarding weekly clinical response, side effects, and services provided.
Psychoeducational materials for office and home; intranet Web site; handouts/worksheets.
Extensive community provider referral base and procedures; mental health resource directory by location, expertise, insurance acceptance, and referral information.
The approach integrates methods to coordinate treatment, decision making, communication, and family engagement and education using trained clinicians as CMs who collaborated with pediatricians and families.
This study describes a 2-year pilot effectiveness trial of the feasibility, fidelity, and acute effects of DOCC for the management of behavioral problems and their comorbidities in primary care. Compared with EUC, we hypothesized that DOCC would be associated with greater service use, higher consumer satisfaction, and improvements in behavioral and emotional symptoms, individualized goals, and clinical response.
Participants were 78 boys and girls (age range, 5-12 years) in 4 community-based pediatric practices affiliated with Children's Hospital of Pittsburgh, Pittsburgh, Pennsylvania, who were referred for behavioral problems and who were rated at or above the 75th percentile (score, ≥6) on the externalizing problems subscale of the 17-item Pediatric Symptom Checklist.13- 15 Cases were assigned based on a 2:1 randomization schedule to DOCC (n = 55) and EUC (n = 23). The sample was predominantly young, male, and of white race/ethnicity (Table 1). The 3 most common psychiatric diagnoses were anxiety disorder (37 children [47%]), ADHD (35 children [45%]), and opposition defiance disorder (ODD) (38 children [49%]). A comparison of the 3 conditions revealed no significant differences at intake preassessment on any of these characteristics or on any of the outcome measures (range, P = .84 to P = .17) with one exception. There were fewer cases with anxiety disorder in the DOCC (22 [40%]) than the EUC (15 [65%]) (P = .50). The study was approved by the institutional review board of the University of Pittsburgh.
Three masters-level staff served as CMs (social worker, counselor, and nurse) and provided the 2 treatment protocols, but only 2 CMs applied treatment at any one time. Care managers were trained during a 4-month period to conduct preassessments and assessments, interact with office staff and pediatricians, participate in case reviews, and complete a study database, and they received ongoing clinical supervision from a senior clinician, with input from a study child and adolescent psychiatrist (J.V.C.). Each CM was assigned to 2 practices and was available 2 days per week at each site.
Two bachelors-level research associates who were unaware of the children's condition conducted research assessments. One administered rating scales, and the other conducted the clinical and diagnostic interview and a postassessment response rating.
All 29 participating primary care clinicians (PCCs) were pediatricians from 4 large practices in Children's Community Pediatrics. This pediatric primary care network is affiliated with the Children's Hospital of Pittsburgh.
After an initial discussion of the study with numerous families, pediatricians referred 125 children who seemed appropriate for treatment to the CMs for formal screening during an 18-month period. Of 87 children who met inclusion criteria (age, externalizing problems, and study interest), 78 completed a clinical and diagnostic assessment, remained eligible (no exclusions owing to parallel treatment or emergent symptoms), and were assigned to a condition. Figure 1 shows the Consolidated Standards of Reporting Trials diagram.
Figure 1. Consolidated Standards of Reporting Trials summary showing study enrollment and participation. EUC indicates enhanced usual care; DOCC, doctor-office collaborative care; and PSC-17, Pediatric Symptom Checklist–17.
In both conditions, CMs called all the parents after the team meetings to discuss the preassessment, explain any diagnoses and recommendations, and indicate the treatment assignment, and they participated by telephone or in person in the weekly team meetings to obtain consultation. Weekly clinical supervision, periodic review of preassessment and session tapes, and conversations with pediatricians confirmed that both treatments were conducted with integrity.
The CM provided on-site direct services to the parent and child for up to 12.0 hours and over no longer than 6 months in accord with a collaborative care model that included the PCC, the family, a backup child psychiatrist, and the study team. Figure 2 shows the functions fulfilled by each of these entities. The CM first provided psychoeducation on behavioral disorders, followed by the administration of brief modules that provided skills training in cognitive behavior treatment and care management coordination10 targeting behavioral disorders and any comorbid ADHD or emotional problems.
Figure 2. Clinical staff and functions integrated into the doctor-office collaborative care model. CM indicates care manager; PCC, primary care clinician.
The treatment module content was based on adaptation of an evidence-based treatment called Alternatives for Families: A Cognitive Behavioral Therapy (www.afcbt.org), which has been evaluated in clinical trials among children referred for behavioral disorders10,16,17 and exposure to physical abuse or aggression.10,18- 20 The modules target parent skills training, child skills training, positive parent-child interactions, peer and school consultation, and brief case management.10 The program included modules for the administration of ADHD care management guidelines (American Academy of Pediatrics)21 and anxiety management techniques (eg, relaxation skills) developed for use in primary care.11,12
Each CM documented all activities performed with or on behalf of a child in 15.0-minute blocks in the following 3 domains: direct services (eg, screening and assessment, telephone sessions, and office visits), indirect services (eg, case management calls and consultation with the PCCs), and quality improvement activities (supervision and paperwork). The CM also collected individualized goal achievement ratings forms at each session.
In EUC, the CM provided the parent with psychoeducation about the child's disorders, clinical recommendations based on the evaluation, and up to 3 referral options that were tailored to the child's problem, insurance, and geographic location from a list of community providers recommended by the practice and confirmed by the CM. Based on this conversation, the CM made a referral to a local mental health provider.
Assessment used a mixed-methods approach. The CM collected a screening instrument (Pediatric Symptom Checklist–17), completed a brief clinical and diagnostic interview and the Clinical Global Impression Scale (CGI), identified treatment targets at preassessment, and completed the CGI at the 6-month postassessment. At 6 months, rating scales were collected by a research associate, and the clinical and diagnostic interview and the CGI were completed by a diagnostic evaluator for a random subset of 34 patients. All the families but 1 completed the postassessment battery. Pediatricians completed rating scales and participated in focus groups.
Preassessment diagnostic status was evaluated using the Schedule for Affective Disorders and Schizophrenia for School-aged Children. Parents completed the Vanderbilt Attention-Deficit/Hyperactivity Disorder Diagnostic Parent Rating Scale22 to evaluate the severity of all 18 symptoms of ADHD, 8 of 9 ODD symptoms, 12 of 15 conduct disorder symptoms, and 7 anxiety and depression items; an 8-item performance scale evaluated functional impairment (eg, academics and peers) using 4-point Likert-type scores (score range, 0-4). They also completed the individualized goal achievement ratings form to identify up to 4 clinical problems to be targeted by treatment (eg, oppositional behavior and anxiety).10 Items were rated as 1 at preassessment but could be rated at 4 levels of improvement at later assessments (eg, 2 indicates minimal; 3, acceptable; 4, better than expected; and 5, exceptional). Twenty-two children whose parents reported concerns about anxiety were administered the 55-item Screen for Child Anxiety Related Emotional Disorders23 to assess severity of symptoms (eg, social, generalized, and phobia) on scales ranging from 1 to 5 points.
The CGI was completed to assess overall improvement at postassessment24,25 on a 7-point scale, with scores of 1 (very much improved) or 2 (much improved) indicating acceptable treatment response. There was high agreement on the CGI between the 2 informants (r = 0.92, P < .001).
A treatment termination summary was completed at postassessment to document variables of the child's treatment by the CM (for DOCC) or by the primary outside provider (for EUC). The items reflected the receipt of the assigned treatment, dosage and duration of therapy, patient disposition, any discharge recommendations (eg, referral), and completer status (ie, case terminated on reaching treatment goals or at agreed-on time).10
The assessment of family impressions of services and the study was adapted from the prior outcome study10 to provide feedback at postassessment about treatment satisfaction and any recommendations for future studies. Pediatricians rated their attitudes about on-site mental health services on 5-point Likert-type scales (1 indicates not at all; 5, very much), including the importance of on-site mental health services, their personal competency for and effectiveness in treating behavioral problems, and interest in receiving training and in participating in collaborative care. Focus groups conducted with 2 practices discussed 5 topics (study progress, administration of treatment, benefits and disadvantages, suggestions to enhance care, and collaboration). The information was coded for thematic content and was summarized by a research associate and a study investigator.
A completed treatment termination summary was submitted by the providers for 52 of 55 DOCC children and for 15 of 23 EUC children. All 52 DOCC children and 4 EUC children had received at least some services by these providers (χ21,67 = 45.62, P = .001). Forty-three DOCC children but no EUC children completed their treatment (χ21,64 = 57.43, P < .001). Six DOCC children and 13 EUC children were referred to local providers for additional services (χ21,65 = 39.34, P < .001). Among 28 children with ADHD having a treatment termination summary, ADHD medication use was higher among DOCC children (78% [18 of 23]) than among EUC children (20% [2 of 10]), but this difference was not significant because of the small EUC sample size (P < .12, Fisher exact test).
Table 2 summarizes the care processes administered among DOCC children recorded in the study progress notes. The most common service was parent skills training, followed by child skills training. Twenty of 26 children diagnosed as having ADHD received medication, the most common of which was methylphenidate. Parents of 9 children received a referral at discharge to address new problems not targeted by DOCC (eg, depression and learning problems). On average, CMs in DOCC spent about 2.0 hours per child conducting screening and assessment, 14.4 hours providing services, 0.9 hours providing indirect services, and 1.6 hours performing quality improvement tasks. Pediatricians spent about 1.0 hour per child in direct services and another 18.0 minutes in indirect services (eg, meetings with CMs). Table 2 also gives an overview of the treatment content used in DOCC. Most parent skills training content targeted the management of family stressors and child behavior, whereas child skills training primarily targeted affect regulation and social skills.
To illustrate the overall course of change during treatment, we also calculated the mean individualized goal achievement ratings collected at the beginning of each DOCC session for those children who received 9 of 12 possible sessions. Figure 3 shows that there was a steady and substantial improvement from the first weekly session (mean score, 1.0) to about the eighth weekly session (mean score, 3.1).
Figure 3. Weekly individualized goal achievement ratings (range, 0-5) for a subset of 29 children who received doctor-office collaborative care.
All the EUC children received a comprehensive preassessment (mean, 3.2 hours). Sixty-five percent (15 of 23) received psychoeducation about the child's problem (mean, 36.0 minutes), followed by a facilitated referral to a local provider and, 2 weeks later, a follow-up call. These services were provided in about 58.0 minutes per child. The CMs spent a mean of 17.1 minutes in supervision. As noted earlier, 27% (4 of 15) of EUC children accessed their assigned community provider.
Pre-post group comparisons (analyses of variance) on standardized rating scales revealed significant improvements for both conditions on virtually all measures (Table 3). In addition, DOCC was superior to EUC in reducing oppositional behavior, inattention, hyperactivity, and functional impairment on the Vanderbilt Attention-Deficit/Hyperactivity Disorder Diagnostic Parent Rating Scale. There were no group differences in improvement for anxiety or depression or for conduct disorder symptoms. In terms of overall scores, DOCC showed significantly greater improvement than EUC in the ADHD and combined ODD and conduct disorder severity ratings.
McNemar test was used to assess subscale score changes on the Vanderbilt Attention-Deficit/Hyperactivity Disorder Diagnostic Parent Rating Scale for which at least 10 children met the clinical cutoff at preassessment. In DOCC, there was a significant reduction in the number of children with hyperactivity (P < .002), with oppositional behavior (P < .001), but not with inattention (P < .07); there were no significant changes found for the same cutoff subscale scores in EUC (P >> .99, P >> .99, and P < .69, respectively).
Because preassessment individualized goal achievement ratings targets were rated as 1, analysis of variance compared the target improvement ratings of the 2 conditions at postassessment only. The mean (SD) improvement ratings were significantly higher for DOCC (3.3 [0.8]) than for EUC (2.2 [0.8]) (F1,76 = 12.34, P < .001). As noted earlier, the midpoint rating of 3 reflects acceptable progress. At preassessment, 11 children whose parents expressed anxiety concerns demonstrated modest but highly variable scores on the Screen for Child Anxiety Related Emotional Disorders (mean [SD], 23.7 [18.1]; range, 4-72) and showed a level of improvement that approached significance (F1,9 = 4.78, P < .06), but there was no significant group difference.
On the preassessment CGIs, DOCC case ratings by the CMs of the overall severity of behavioral problems were comparable among the DOCC and EUC children. On the postassessment CGIs, DOCC children were more likely to be rated as improved or significantly improved than EUC children (66% vs 8%; P < .001, Fisher exact test). The ratings for both conditions are shown in Figure 4. Likewise, the postassessment CGIs rated by the diagnostic evaluator confirmed greater improvement for DOCC children than for EUC children (62% vs 0% [1 of 5]; P < .003, Fisher exact test).
Figure 4. Clinical Global Impression Scale ratings by clinical care managers for children in the enhanced usual care (EUC) and doctor-office collaborative care (DOCC) conditions.
Caregiver reports on the family impressions of services and the study demonstrated that 71% were satisfied with their child's evaluation. In terms of new services, 26% wanted behavioral consultation for parenting, and 21% sought school visits. More caregivers were willing to receive services in the practice (91%) than in the home (65%) or outpatient clinic (57%). Caregivers were very willing to receive on-site services to address mental health problems for their children (92%) and for themselves (71%). Ninety-one percent of parents had Internet access, and 76% were willing to complete forms on a secure Web site.
Survey responses and coded focus group data from 24 pediatricians revealed highly supportive attitudes for our DOCC model in that they acknowledged the importance of offering on-site services in the office (mean score, 4.6) and considered themselves unable to provide mental health care in a competent or effective way on their own (mean score, 2.7). They expressed high enthusiasm for being trained to participate in integrated on-site clinical services (mean score, 3.6) and for collaborating with a CM to treat behavioral problems (mean score, 4.6). The pediatricians gave several reasons for their high level of satisfaction with DOCC (eg, expanded services, increased practice capacity, and improved child outcomes), identified potential barriers to its sustainability (eg, lack of reimbursement or available clinicians), and proposed several solutions (eg, the use of the CM as a broker of services, education about prescription practices, and negotiation with insurers).
Children who were referred by their pediatricians for treatment of behavioral problems were randomized to an on-site intervention based on the CCM (DOCC) or to EUC. As in a prior clinical trial,10 the availability of on-site services improved service access, as DOCC children initiated and completed treatment at higher rates than EUC children, consistent with other studies26,27 showing greater acceptability of visits to a pediatric setting rather than to a mental health setting. Furthermore, DOCC made counseling, collaboration, consultation, and follow-up interviews available for the pediatrician and family, especially surrounding medication management, a feature designed to optimize care for pediatric mental health problems.8,21
The most common on-site service provided to virtually all the caregivers involved training parents in self-management and child management skills and helping them address individual issues. This was complemented by teaching self-control and social skills to children and by conducting parent-child sessions. The inclusion of both participants in services is consistent with the treatment literature, but the provision of services via the Internet or telephone offers an even more efficient and flexible option.5,28,29
Improvements in clinical outcomes were found for both conditions,10 but DOCC showed significantly greater reductions in oppositionality, inattention, hyperactivity, and functional impairment and improved overall treatment response ratings at postassessment. The benefits found for DOCC exceeded the outcomes for a previous intervention10 that encompassed only colocated care (PONI) and may be related to several protocol refinements, including the use of mental health clinicians as CMs, ongoing communication with the PCCs, greater attention to parental management training, and the use of automated weekly progress graphs. The improvements in ADHD may have resulted from our incorporation of American Academy of Pediatrics care management guidelines,30 which was associated with a higher rate of ADHD medication use than the 41% in a prior study10 of nurse co-location or in other studies21,31 of adherence to these guidelines. Our more modest effects on parent-reported anxiety and depression in their children may be due to the lack of an exposure component in our protocol11,12 and by the use of a parent-reported scale to screen for anxiety and depression, as few children completed the more clinically sensitive Screen for Child Anxiety Related Emotional Disorders.23
Based on follow-up interviews, parents showed strong preferences for receiving services in the pediatrician's office, consistent with the high level of consumer satisfaction found for DOCC and in a prior study.10 Likewise, the PCCs welcomed participation in DOCC given its incorporation of the CCM, mental health practitioners serving as CMs, and an expanded treatment protocol that provided ongoing collaboration with the PCC.27 This allowed us to target several child problems using focused but flexibly applied clinical content.1
The limitations of this study relate to its modest sample size, some missing data from EUC providers, and the absence of long-term follow-up assessments,10,17,18 so the effectiveness of DOCC requires replication in a more rigorous trial. Furthermore, the CGI ratings were collected by the CMs and a diagnostic evaluator who could not always be kept naive to children's treatment status, although these ratings were consistent with all self-reports. Finally, because randomization occurred at the individual participant level, we could not conduct ongoing training or programming at the practice level, which might have enhanced the clinical effects of DOCC.
In summary, the feasibility and clinical benefits of DOCC support the provision of on-site collaborative care for children's mental health problems. Further efforts are needed to develop a fully integrated model in primary care that addresses key issues related to logistics (eg, space use and scheduling), staff development (eg, credentialing and specialization), interprofessional management (eg, sharing medical records and office hierarchy), and financial reimbursement (eg, viable business models), among others, and that is guided by relevant implementation models.32,33 These developments are needed to ensure that the evidence-based practices targeting behavioral and emotional problems incorporated in the pediatric setting are effective and sustainable.8
Correspondence: David J. Kolko, PhD, ABPP, Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, 3811 O’Hara St, Pittsburgh, PA 15213 (firstname.lastname@example.org).
Accepted for Publication: August 24, 2011.
Published Online: November 7, 2011. doi:10.1001/archpediatrics.2011.201
Author Contributions:Study concept and design: Kolko, Campo, Kilbourne, and Kelleher. Acquisition of data: Kolko. Analysis and interpretation of data: Kolko, Campo, and Kilbourne. Drafting of the manuscript: Kolko. Critical revision of the manuscript for important intellectual content: Kolko, Campo, Kilbourne, and Kelleher. Statistical analysis: Kolko. Obtained funding: Kolko. Administrative, technical, and material support: Kolko, Campo, and Kilbourne. Study supervision: Kolko, Campo, and Kelleher.
Financial Disclosure: None reported.
Funding/Support: This study was supported by grant MH63272 from the National Institute of Mental Health.
Additional Contributions: David Brent, MD, provided helpful advice. We acknowledge the research and clinical staff of the Services for Kids In Primary-care (SKIP) program, especially Betsy A. Holden, MEd, CRC, LPC, for project coordination, and the clinical and administrative staff affiliated with the participating pediatric practices from Children's Community Pediatrics of Children's Hospital of Pittsburgh.
Kolko DJ, Campo JV, Kilbourne AM, Kelleher K. Doctor-Office Collaborative Care for Pediatric Behavioral ProblemsA Preliminary Clinical Trial. Arch Pediatr Adolesc Med. 2012;166(3):224-231. doi:10.1001/archpediatrics.2011.201