BHCP indicates behavioral health care practitioner; SBIRT, Screening, Brief Intervention, and Referral to Treatment; TWCQ, Teen Well Check Questionnaire.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Sterling S, Kline-Simon AH, Satre DD, et al. Implementation of Screening, Brief Intervention, and Referral to Treatment for Adolescents in Pediatric Primary Care: A Cluster Randomized Trial. JAMA Pediatr. 2015;169(11):e153145. doi:10.1001/jamapediatrics.2015.3145
Early intervention for substance use is critical to improving adolescent outcomes. Studies have found promising results for Screening, Brief Intervention, and Referral to Treatment (SBIRT), but little research has examined implementation.
To compare SBIRT implementation in pediatric primary care among trained pediatricians, pediatricians working in coordination with embedded behavioral health care practitioners (BHCPs), and usual care (UC).
Design, Setting, and Participants
The study is a 2-year (November 1, 2011, through October 31, 2013), nonblinded, cluster randomized, hybrid implementation and effectiveness trial examining SBIRT implementation outcomes across 2 modalities of implementation and UC. Fifty-two pediatricians from a large general pediatrics clinic in an integrated health care system were randomized to 1 of 3 SBIRT implementation arms; patients aged 12 to 18 years were eligible.
Two modes of SBIRT implementation, (1) pediatrician only (pediatricians trained to provide SBIRT) and (2) embedded BHCP (BHCP trained to provide SBIRT), and (3) UC.
Main Outcomes and Measures
Implementation of SBIRT (primary outcome), which included assessments, brief interventions, and referrals to specialty substance use and mental health treatment.
The final sample included 1871 eligible patients among 47 pediatricians; health care professional characteristics did not differ across study arms. Patients in the pediatrician-only (adjusted odds ratio [AOR], 10.37; 95% CI, 5.45-19.74; P < .001) and the embedded BHCP (AOR, 18.09; 95% CI, 9.69-33.77; P < .001) arms had higher odds of receiving brief interventions compared with patients in the UC arm. Patients in the embedded BHCP arm were more likely to receive brief interventions compared with those in the pediatrician-only arm (AOR, 1.74; 95% CI, 1.31-2.31; P < .001). The embedded BHCP arm had lower odds of receiving a referral compared with the pediatrician-only (AOR, 0.58; 95% CI, 0.43-0.78; P < .001) and UC (AOR, 0.65; 95% CI, 0.48-0.89; P = .006) arms; odds of referrals did not differ between the pediatrician-only and UC arms.
Conclusions and Relevance
The intervention arms had better screening, assessment, and brief intervention rates than the UC arm. Patients in the pediatrician-only and UC arms had higher odds of being referred to specialty treatment than those in the embedded BHCP arm, suggesting lingering barriers to having pediatricians fully address substance use in primary care. Findings also highlight age and ethnic groups less likely to receive these important services.
Clinicaltrials.gov Identifier: NCT02408952
Substance use is a leading cause of mortality and morbidity among adolescents1 and is frequently comorbid with mental health and medical problems,2,3 complicating prevention and treatment.4 Early intervention is critical to improving outcomes.5 Although the US Preventive Services Task Force found insufficient evidence to support behavioral interventions for substance use in pediatric primary care,6,7 more recent research reveals promising results across various settings and populations.8-16 Multiple national and international organizations endorse Screening, Brief Intervention, and Referral to Treatment (SBIRT) provided in primary care as an effective population-based approach to reducing substance use among adults.17,18 If appropriate models can be developed and implemented, SBIRT may be endorsed similarly for adolescents.
Pediatric SBIRT is not widely implemented, however, and despite calls for more research,1 to our knowledge, few studies19,20 have examined its implementation. Evidence suggests that adolescent SBIRT is effective, but we know little about contextual factors, such as settings, screening tools, staffing approaches, and use of electronic health records (EHRs).21,22 Interventions that meet the needs of patients, health care professionals, and health care systems are critical, and pragmatic trials with hybrid designs, including both effectiveness and implementation outcomes, are especially useful.23 This study examines both, with an initial focus on implementation.
The design is a cluster randomized, hybrid implementation and effectiveness trial in a large pediatric clinic at Kaiser Permanente Northern California (KPNC). The primary outcome examined is SBIRT implementation compared across 3 study arms: 2 modalities of SBIRT provision (by specially trained pediatricians or an embedded behavioral health care practitioner [BHCP]) and usual care (UC). We also compare the 2 intervention arms. Given the high rates of mental health comorbidity, we also examined mental health screening. For those at risk for substance use or mental health problems, we assess rates of brief intervention and referral to specialty treatment. We examine patient and pediatrician factors associated with SBIRT implementation for 2 years. We hypothesized that both intervention arms would have higher SBIRT rates than the UC arm because of training on SBIRT protocols and that the embedded BHCP arm would have higher rates than the pediatrician-only arm because of time pressures and competing priorities experienced by pediatricians.
Early intervention for substance use is critical to improving adolescent outcomes. We compared 2 modalities of implementing substance use and mental health screening: Screening, Brief Intervention, and Referral to Treatment in pediatric primary care, specially trained pediatricians and pediatricians working in coordination with embedded behavioral health clinicians (BHCPs), and usual care.
The pediatrician-only (adjusted odds ratio [AOR], 10.37; 95% CI, 5.45-19.74; P < .001) and the embedded BHCP (AOR, 18.09; 95% CI, 9.69-33.77; P < .001) arms had higher odds of providing brief interventions compared with usual care.
The embedded BHCP arm was more likely to provide brief interventions compared with the pediatrician-only arm (AOR, 1.74; 95% CI, 1.31-2.31; P < .001).
Patients in the embedded BHCP arm had lower odds of receiving a referral to specialty behavioral health services compared with the pediatrician-only (AOR, 0.58; 95% CI, 0.43-0.78; P < .001) and usual care (AOR, 0.65; 95% CI, 0.48-0.89; P = .006) arms; odds of referrals did not differ between the pediatrician-only and usual care arms.
Kaiser Permanente Northern California is a nonprofit integrated health care system of 3.8 million members. The study was conducted from November 1, 2011, through October 31, 2013, in the Department of Pediatrics at KPNC Oakland, which treats a racially and socioeconomically diverse population.
To have an intent-to-treat, population-based sample of health care professionals, we randomly assigned all clinic pediatricians (n = 52) to 1 of 3 study arms: (1) pediatrician only (pediatricians trained to assess substance use risk and consequences using evidence-based screening tools, provide brief interventions, and refer patients to specialty substance use or mental health treatment), (2) embedded BHCP (BHCPs trained to provide SBIRT components as above), and (3) UC (care administered as usual, no SBIRT training to pediatricians or access to the BHCP) (Figure). The trial protocol can be found in the Supplement. Many of the pediatricians are bilingual Spanish and Chinese. Blocked randomization ensured an equal number of bilingual pediatricians in each arm. Implementation differences across other pediatrician characteristics (eg, sex, experience) were of interest, so these measures were not applied to the randomization. Patients aged 12 to 18 years were eligible, and health care professional assignment to study arm was not masked. Consistent with other comparative effectiveness studies,24 we used EHR measures to examine primary outcomes, and patients were not recruited to the study or informed of which study arm included their pediatrician. The study was approved by the KPNC Institutional Review Board and the University of California, San Francisco, Committee on Human Research. Consent for health care professional participation was not required by the institutional review board or the Committee on Human Research.
The Teen Well Check Questionnaire (TWCQ) is a comprehensive health screening tool embedded in the EHR and standard clinical workflow and completed by adolescents at registration for their well-child care visit. In all arms, patients’ TWCQ responses were entered into the EHR before visiting their pediatrician. When reviewing the TWCQ responses, pediatricians talked to adolescents without parents present to encourage discussion of sensitive topics.25 Endorsement of past-year alcohol, marijuana, or other drug use and/or presence of mood symptoms or suicidality constituted a positive screening result and triggered assessment.
Assessment was conducted using the CRAFFT substance use instrument26 and questions about past 6-month use of alcohol, marijuana, opiates, tobacco, or other drugs and substance use–related school, family, social, and legal consequences (CRAFFT+).
Pediatricians and the BHCP were trained to provide brief interventions that consisted of feedback, advice, and goal-setting as adapted from the Provider Guide: Adolescent Screening, Brief Intervention, and Referral to Treatment Using the CRAFFT Screening Tool,27 which was developed for its feasibility in busy medical settings. Theoretical foundations included the stages of change model,28 motivational interviewing,29 and recognition of the links between substance use and health and mental health problems and conduct issues, such as legal and school problems. This approach encouraged patients to think through potential consequences of substance use and to choose alternatives. Pediatrician-only brief interventions were typically confined to the well-child care visit, whereas the embedded BHCP was able to spend more time with patients.
Referrals were made to substance use treatment or mental health departments within KPNC. Programs included intensive, structured outpatient treatment, supportive group therapy, psychoeducation, relapse prevention, medication management, and family and individual therapy with referral to contracted residential programs when inpatient treatment was required. Appropriate treatment intensity was determined by specialty treatment health care professionals at intake.
If substance use or mental health risk was endorsed during the TWCQ screening, the CRAFFT+ in the EHR was administered. If no severe substance use problems were present, the pediatrician was trained to conduct a brief intervention. If the CRAFFT score was 2 or higher, the pediatrician was trained to refer the patient to specialty substance use or mental health treatment for additional assessment.
If the patient endorsed substance use or mental health risk during screening, the pediatrician called the BHCP (A.J.), a licensed clinical psychologist, while the patient was in the examination room. The BHCP came to meet the patient or, if busy, spoke with the patient to set up an appointment and then followed the same protocol as in the pediatrician-only arm for assessment, brief intervention, and referral.
Pediatricians in the UC arm had the same access to the substance use and mental health screening and assessment tools and referral resources as those in the other arms but did not receive SBIRT training or have access to the BHCP for their patients. Pediatricians counseled patients with substance use or mental health symptoms as considered necessary, which they recorded in the EHR per policy.
Pediatricians and the BHCP in the intervention arms were offered on-site trainings (three 60-minute sessions for pediatricians in the pediatrician-only arm, one 60-minute session for pediatricians in the pediatrician and embedded BHCP arms) on motivational interviewing principles,29 patterns of hazardous substance use and common mental health symptoms, the manualized brief intervention protocol, educational resources, and protocols for specialty substance use and mental health treatment referral, for which they received lunch and continuing education credit. In the pediatrician-only arm, 8 of 17 pediatricians (47.1%) attended at least 2 trainings; in the embedded BHCP arm, 13 of 17 pediatricians (76.5%) attended the training. We recorded trainings for both arms, sent recordings and slides to pediatricians for viewing at their convenience, and made technical assistance and clinical consultation available as needed. Consistent with other SBIRT implementation studies,30 feedback on SBIRT rates (in the pediatrician-only arm) and rates of referral to the BHCP (in the embedded BHCP arm) were discussed with pediatricians at quarterly meetings, along with a review of the SBIRT protocol and skills, to reinforce fidelity and performance.31 Emails and staff meetings informed pediatricians equally across all arms about the screening and assessment tools in the EHR and reminded them of the requirement to document clinical activities. Among pediatricians in the pediatrician-only arm with patients eligible for assessments, brief interventions, and referrals (n=14), pediatricians who attended at least 2 trainings (7 of 14 pediatricians) administered more assessments (81 of 149 assessments administered [54.4%]; P < .001) and brief interventions (62 of 96 brief interventions provided [64.6%]; P < .001) among eligible patients compared with the pediatricians who attended fewer trainings.
Patient age, sex, and race/ethnicity (Asian, black, Hispanic, white, and other or missing) and their TWCQ responses as well as pediatrician age, sex, and years of experience were extracted from the EHR.
Patients across all arms were screened at reception using the TWCQ. The primary outcome was SBIRT implementation, which included assessments, brief interventions, and referrals to treatment among eligible patients who screened positive on at least one of the TWCQ substance use or mental health questions and/or were considered at risk based on the pediatrician’s or BHCP’s initial examination, all of which were extracted from the EHR. The first eligible encounter during the study period was included for each patient. Preexisting administrative V codes (V65.42D [counseling substance use or abuse] and V65.49ZZZZU [encounter for counseling, emotional health]) were used to document brief interventions and their content (substance use, mental health, or both). Medical record reviews for all eligible patients identified referrals to specialty treatment. For all eligible encounters, we created indicator variables of further action: CRAFFT+ assessment (1 indicating CRAFFT+ performed; 0, otherwise), brief intervention (1 indicating substance use and/or mental health brief intervention performed; 0, otherwise), or referral (1 indicating referred to substance use or mental health treatment; 0, otherwise).
All bivariate and logistic models examined differences in the primary outcomes across the 3 arms (reference group was UC) or between the intervention arms only (embedded BHCP and pediatrician only [reference]). We used standard descriptive statistics to compare patient and pediatrician characteristics and treatment outcomes across the 3 arms.
Because patients are nested within their pediatricians and observations within these clusters may be correlated, generalized estimation equation techniques were used to fit multivariable logistic regression models. Initial models controlled for patient (age, sex, and ethnicity) and pediatrician (sex and years of experience) characteristics. However, because of the study design, most assessments, brief interventions, and referrals in the embedded BHCP arm were conducted by the BHCP (pediatricians in the embedded BHCP arm administered the CRAFFT+ and/or a brief intervention on 6 occasions without referral to the BHCP), causing pediatrician characteristics to be highly correlated with the outcome measures; therefore, pediatrician characteristics were not included in final models.
Power calculations accounted for intraclass correlation among patients clustered within pediatricians (unit of randomization), which reduced effective sample size by a factor of [1 + (n − 1)] × intraclass correlation, where n is the mean cluster size.32 Our final sample size of 1871 eligible patients among 47 pediatricians (40 patients per pediatrician) with an intraclass correlation estimate of 0.02 gave us adequate power (power of 0.93) to detect a small to medium effect size of 30% in assessments, brief interventions, and referrals across treatment arms. Analyses were performed using SAS statistical software, version 9.3 (SAS Institute Inc).
During the November 1, 2011, through October 30, 2013, study period, 5183 patients (75.0% of all visits) received the TWCQ screening: 2695 (52.0%) were female, 1120 (21.6%) were white, 1659 (32.0%) were black, 1130 (21.8%) were Hispanic, 933 (18.0%) were Asian, and 342 (6.6%) were other or missing; the mean age was 15 years. Both intervention arms had significantly more girls (894 of 1558 [57.4%] in the pediatrician-only arm, 965 of the 1856 patients [52.0%] in the embedded BHCP arm, and 831 of the 1769 patients [47.0%] in the UC arm), fewer white (313 [20.0%] in the pediatrician-only arm, 356 [19.2%] in the embedded BHCP arm, and 464 [26.2%] in the UC arm), and more black patients (537 [34.5%] in the pediatrician-only arm, 629 [33.9%] in the embedded BHCP arm, and 502 [28.4%] in the UC arm) compared with those in the UC arm. Pediatrician characteristics did not differ across the study arms (Table 1).
In the pediatrician-only arm, more adolescents endorsed mental health symptoms compared with the embedded BHCP and UC arms (274 [17.6%] in the pediatrician-only arm, 244 [13.1%] in the embedded BHCP arm, and 263 [14.9%] in the UC arm). They specifically had higher prevalence of depression symptoms (248 [15.9%] in the pediatrician-only arm, 220 [11.9%] in the embedded BHCP arm, and 243 [13.7%] in the UC arm). The 3 arms did not significantly differ in prevalence of substance use symptoms.
Forty-six pediatricians (14 in the pediatrician-only arm, 16 in the embedded BHCP arm, and 16 in the UC arm) had patients (n = 1871) eligible for assessments, brief interventions, and referrals (n = 584 in the pediatrician-only army, n = 671 in the embedded BHCP arm, and n = 616 in the UC arm), which were examined across the 3 arms.
The health care professionals in the UC arm did not perform any CRAFFT+ assessments, so only the pediatrician-only and embedded BHCP arms were compared; the total number of assessments did not differ (149 [25.5%] in the pediatrician-only arm and 163 [24.3%] in the embedded BHCP arm; P = .44). Among assessments administered, more assessments occurred in response to substance use symptoms in the pediatrician-only arm (117 [78.5%] in the pediatrician-only arm and 69 [42.3%] in the embedded BHCP arm; P < .001); assessments did not differ in response to mental health symptoms. Generalized estimation equation models examined differences in the total number of assessments and found that, similar to bivariate analyses, intervention arms did not differ. Older patients and Hispanic patients had higher odds of being assessed; there were no sex differences.
Both intervention arms provided more brief interventions compared with the UC arm (96 [16.4%] in the pediatrician-only arm, 171 [25.5%] in the embedded BHCP arm, and 11 [1.8%] in the UC arm). Among the brief interventions administered, the embedded BHCP arm provided more brief interventions that contained mental health content (139 [81.3%]) than the pediatrician-only arm (11 [11.5%]); the pediatrician-only arm provided more brief interventions that contained substance use content (88 [91.7%] in the pediatrician-only arm and 95 [55.6%] in the embedded BHCP arm; P < .001).
In the adjusted models, patients in the pediatrician-only and embedded BHCP arms had higher odds of receiving brief interventions than those in the UC arm; patient age and sex were not significant, and Hispanic patients had higher odds of a brief intervention compared with white patients (Table 2). However, when comparing intervention arms, the patients in the embedded BHCP arm were more likely to receive brief interventions than those in the pediatrician-only arm; Hispanic patients had higher odds of receiving a brief intervention (Table 3).
Because the protocol required that pediatricians in the embedded BHCP arm refer patients to the BHCP, we also examined the number of assessments and brief interventions provided in the embedded BHCP arm among only those patients who were referred to the BHCP by the pediatrician (183 referred of 671 eligible [27.3%]). Among all 183 encounters with the BHCP, 161 (88.0%) received an assessment with the CRAFFT+, and 168 (91.8%) were given a brief intervention.
Those in the pediatrician-only arm made more referrals to specialty treatment (substance use only, mental health only, or substance use and mental health) than those in the other 2 arms (P < .001). In the adjusted models, the embedded BHCP arm had lower odds of referrals (by the BHCP or embedded BHCP pediatricians) compared with the UC arm; the pediatrician-only arm did not differ from the UC arm. Older patients had lower odds of receiving a referral, and Hispanic patients and those whose ethnicity was classified as other or missing had higher odds (Table 2). The embedded BHCP arm had lower odds of referrals compared with the pediatrician-only arm when treatment arms were compared. Hispanic patients had higher odds of referral compared with white patients (Table 3).
This is the first trial, to our knowledge, to compare implementation outcomes for different models of providing SBIRT in primary care to adolescents with substance use and mental health problems. It used a population-based approach to examine patient outcomes rather than recruiting individual patients, thus including many adolescents who might have been excluded in a recruitment study. The large, ethnically diverse population base and general pediatrics clinic setting allowed us to study implementation in a context that is more generalizable to common pediatric settings and populations than most randomized clinical trials.
Health care professionals in both intervention arms administered more assessments and brief interventions than those in usual care. However, overall pediatrician attention to behavioral health concerns was low.
More assessments in the pediatrician-only arm were in response to substance use alone. The content provided in the brief interventions, documented by the pediatricians and BHCP, differed similarly: more brief interventions in the pediatrician-only arm contained only substance use–related content, and more in the embedded-BHCP arm contained only mental health–related content or a combination of mental health and substance use content. This finding was surprising because research indicates that pediatricians are more comfortable addressing emotional problems, such as depression, than substance use.33 However, the BHCP in the embedded BHCP arm had more time to talk with patients and thus time to discuss a wider array of topics, which may account for the differences in brief intervention content. Given the common co-occurrence of substance use and mental health problems in adolescents, it is critical to develop approaches that address both simultaneously, despite limited visit times and competing priorities. The difference in brief intervention content speaks to the inherent strengths and weaknesses of different SBIRT modalities that might be expected if implemented as part of standard care.
Patients in the pediatrician-only and UC arms were more likely to be referred to specialty treatment than those in the embedded BHCP arm. This finding suggests lingering barriers to pediatricians fully addressing behavioral health during primary care. Once pediatricians have referred patients to an embedded BHCP, they may view responsibility to refer as having been delegated.
Research has found that nonwhite adolescents engage in behavioral health treatment less often than white patients.34 Health care systems might want to consider improving the engagement of adolescents of color, in light of our finding that Hispanic patients and patients with other or missing race/ethnicity information were more likely to be referred.
Fewer referrals to specialty substance use and mental health care and more brief interventions administered in the embedded BHCP arm suggest that integrating a BHCP into pediatric primary care could be an efficient way to provide behavioral health care to adolescents; addressing lower-level problems in pediatrics may prevent problem development and obviate the need for later referral to specialty care.35 Nevertheless, pediatrician referrals to the BHCP were low, reducing access to this potentially effective intervention. Referrals to the BHCP increased during the study period, however, and may have continued to increase during a longer period because pediatrician sensitivity to substance use and mental health problems increased.
This study was conducted in an integrated health care system with an insured population and may not be generalizable to uninsured populations. Kaiser Permanente Northern California has integrated mental health and substance use treatment programs, and clinician practices may differ from those in other settings. These limitations become less salient as more individuals are insured through health care system reform and as other health systems, such as federally qualified health centers, add BHCPs to primary care. Kaiser Permanente Northern California and the clinic in this study have ethnically and sociodemographically diverse member populations, which may increase the generalizability of these findings across populations. Contamination may have occurred between intervention and UC pediatricians, but we did not find use of the CRAFFT+ among the UC pediatricians. Although all pediatricians were randomized and health care professional characteristics did not differ across treatment arms, nonmeasured confounding variables could still have existed.
Many major medical organizations endorse regular substance use and mental health screening beginning in early adolescence,36,37 but routine SBIRT for adolescents has not been widely implemented in pediatric primary care. Consistent with the literature, we found that training pediatricians can significantly improve screening and intervention,38 but barriers to implementing SBIRT remain, including the belief among many pediatricians that other health care professionals are better situated to address behavioral health problems.33,39 Embedding nonphysician behavioral health care professionals in primary care could be a cost-effective alternative to pediatricians providing these services, and future analyses of the study data will examine patient outcomes and cost-effectiveness of the 2 SBIRT modalities. Research indicates that both physicians and nonphysicians are effective at providing SBIRT in adult primary care,40 but whether that is equally true in pediatric primary care is not known. The findings presented here represent a necessary initial step in understanding patterns of SBIRT implementation in pediatric primary care.
Submitted for Publication: April 3, 2015; final revision received August 24, 2015; accepted September 2, 2015.
Corresponding Author: Stacy Sterling, MPH, MSW, Division of Research, Kaiser Permanente Northern California, 2000 Broadway, Third Floor, Oakland, CA 94612-2403 (firstname.lastname@example.org).
Published Online: November 2, 2015. doi:10.1001/jamapediatrics.2015.3145.
Author Contributions: Mss Sterling and Kline-Simon had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Sterling, Kline-Simon, Mertens, Wong, Weisner.
Acquisition, analysis, or interpretation of data: Sterling, Kline-Simon, Satre, Jones, Mertens, Weisner.
Drafting of the manuscript: Sterling, Kline-Simon, Satre, Mertens, Wong, Weisner.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Kline-Simon, Weisner.
Obtained funding: Sterling, Weisner.
Administrative, technical, or material support: Sterling, Mertens.
Study supervision: Sterling, Wong, Weisner.
Conflict of Interest Disclosures: None reported.
Funding/Support: The Screening for Youth Alcohol and Drug Use: A Study of Primary Care Providers Study is supported by grant R01 AA016204 from the National Institute on Alcohol Abuse and Alcoholism (Dr Weisner, principal investigator).
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
Additional Contributions: Agatha Hinman, BA, provided editorial assistance and Sujaya Parthasarathy, PhD, and Felicia Chi, MPH, provided statistical consultation. The KPNC Adolescent Chemical Dependency Coordinating Committee and the KPNC Adolescent Medicine Specialists Committee provided guidance. Charles Wibbelsman, MD, and Don Mordecai, MD, provided strong leadership support of this project. We thank David Bacchus, MD, Patricia Castaneda-Davis, MD, and all the physicians, medical assistants, nurses, receptionists, managers, and especially the patients and parents of KPNC’s Oakland Pediatrics clinic for their participation in the activities related to this study.
Create a personal account or sign in to: