Effect of Computer-Based Substance Use Screening and Brief Behavioral Counseling vs Usual Care for Youths in Pediatric Primary Care: A Pilot Randomized Clinical Trial

Key Points

Question  Is a computer-facilitated system for youth substance use screening and brief intervention (CSBI) feasible and acceptable compared with usual care (UC) in primary care?

Findings  In this pilot randomized clinical trial, among 106 youths who reported cannabis use in the past 12 months at baseline, the CSBI group reported longer time to first postvisit cannabis use compared with the UC group. Among 99 youths who reported riding with an impaired driver in the past 3 months at baseline, the CSBI group reported a lower rate of having ridden with an impaired driver in the past 3 months compared with the UC group.

Meaning  The CSBI system is feasible and acceptable in clinical practice and should be further tested in larger samples.

Importance  Annual preventive health visits provide an opportunity to screen youths for unhealthy substance use and intervene before serious harm results.

Objectives  To assess the feasibility and acceptability and estimate the efficacy of a primary care computer-facilitated screening and practitioner-delivered brief intervention (CSBI) system compared with usual care (UC) for youth substance use and associated risk of riding with an impaired driver.

Design, Setting, and Participants  An intent-to-treat pilot randomized clinical trial compared CSBI with UC among 965 youths aged 12 to 18 years at 5 pediatric primary care offices and 54 practitioners. Patients were randomized to CSBI (n = 628) or usual care (n = 243) groups within practitioner with 12 months of follow-up. Data were collected from February 1, 2015, to December 31, 2017. Data analysis was performed January 1, 2018, to March 30, 2019.

Interventions  Patients self-administered a computer-facilitated substance use screening questionnaire before their annual preventive health visits. Immediately after completing the screening, they received their score and level of risk and viewed 10 pages of scientific information and true-life vignettes illustrating health risks associated with substance use. Trained practitioners received the screening results, patients’ risk levels, talking points designed to prompt brief counseling, and recommended follow-up plans.

Main Outcomes and Measures  Feasibility and acceptability were assessed using adolescents’ postvisit ratings. Days of alcohol use, cannabis use, and heavy episodic drinking were assessed at baseline and 3-, 6-, 9-, and 12-month follow-ups using Timeline Followback, and riding in the past 3 months with a driver who was impaired by use of alcohol or other drugs was assessed using 2 self-report items. The primary outcome was the intervention effect among at-risk youths who reported using alcohol or other drugs in the past 12 months or riding with an impaired driver in the past 3 months at baseline. The secondary outcome was the prevention effect among those with no prior use or risk.

Results  Among 871 youths screened, 869 completed the baseline assessment; 211 of the 869 reported alcohol or cannabis use in the past 12 months at baseline (mean [SD] age, 16.4 [1.3] years; 114 [54.1%] female; 105 [49.8%] non-Hispanic white). Of the 211 youths, 148 (70.1%) were assigned to the CSBI group and 63 (29.9%) were assigned to the UC group. Among youths in the CSBI group, 105 (70.9%) reported receiving counseling about alcohol, 122 (82.4%) reported receiving counseling about cannabis, and 129 (87.2%) reported receiving counseling about not riding with an impaired driver. Adjusted hazard ratios for time to first postvisit use of alcohol or other drugs for CSBI vs UC were as follows: alcohol use, 0.69 (95% CI, 0.47-1.02); heavy episodic drinking, 0.66 (95% CI, 0.40-1.10); and cannabis use, 0.62 (95% CI, 0.41-0.94). At 12-month follow-ups among 99 youths who reported having ridden in the past 3 months at baseline with an impaired driver (64 in the CSBI group; 35 in the UC group), adjusted relative risk ratio of riding in the past 3 months with an impaired driver for CSBI vs UC groups was 0.58 (95% CI, 0.37-0.91). No intervention effect was observed among youths who reported no prior use of alcohol or other drugs (n = 658) or not having ridden with an impaired driver (n = 769) at baseline.

Conclusions and Relevance  The CSBI system is a feasible and acceptable option for screening youths in primary care practice for use of alcohol and other drugs and for risk of riding with an impaired driver, and the estimated efficacy in this sample warrants further testing in larger samples.

Trial Registration  ClinicalTrials.gov identifier: NCT00227877

By their senior year in high school, more than 60% of students have begun to drink alcohol, 45% of students have been drunk, and 45% of students have used marijuana or hashish.¹ A 2017 survey² reported that among students in grades 9 through 12, 30% reported drinking, 13.5% reported heavy episodic drinking (HED; defined as ≥4 drinks in a row for girls and ≥5 drinks in a row for boys³), 19.8% reported using marijuana, and 16.5% reported riding in a car with a driver who had been drinking alcohol, all during the 30 days prior to the survey. Use of alcohol and drugs is strongly linked to the leading causes of death among adolescents, including motor vehicle crashes and other unintentional injuries, homicides, suicides, and an array of other serious health risks and problems.²

More than 80% of children and adolescents aged 6 to 17 years have a medical office visit annually,⁴ and pediatricians are a consistent and trusted presence in their lives,⁵ making primary care offices a logical venue for screening and early intervention. The American Academy of Pediatrics and the National Institute on Alcohol Abuse and Alcoholism both recommend annual screening and behavioral counseling for alcohol use as a routine part of adolescent care.^3,6 However, a 2013 study by Hingson et al⁷ reported that only about half of adolescents who had a visit with a physician in the past year reported being asked about alcohol use. Practitioner-reported barriers to screening include lack of time, insufficient training in how to manage positive screens, and unfamiliarity with validated screening tools.⁸ Therefore, even performed screenings can be of low quality, resulting in missed opportunities for early identification and intervention.^9-11 Primary care screening and behavioral counseling for hazardous drinking are widely recommended for adults.^12,13 However, in its most recent evidence reviews, the US Preventive Services Task Force found insufficient evidence to recommend for or against primary care screening and brief counseling of adolescents for either alcohol or drug use.^14,15

To address this evidence gap, we developed a computer-facilitated screening and brief intervention (CSBI) office system, including a self-administered screening questionnaire based on the CRAFFT (car, relax, alone, forget, family or friends, trouble) screening tool,¹⁶ immediate personalized feedback and psychoeducation, and point-of-care decision support for practitioners. The aims of our study were to assess the acceptability of the CSBI system in primary care and to generate randomized clinical trial–based estimates of effect size for youth substance use and the safety risk of riding with an impaired driver, defined by the judgment of the participating youth. Of primary interest was the intervention effect among at-risk youths, ie, those who reported prior substance use or riding risk. Secondarily, we examined the prevention effect among youths with no reported prior use of alcohol or other drugs or riding risk.

This study follows the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline. We conducted a pilot randomized clinical trial from February 1, 2015, to December 31, 2017, as part of a parent study¹⁷ whose primary purpose was to assess the psychometric properties of the National Institute on Alcohol Abuse and Alcoholism recommended screening for children and adolescents (the original study protocol and statistical analysis plan are presented in Supplement 1).³ We conducted the study in 3 community practices and 2 hospital-based practices in the greater Boston, Massachusetts, area. Data analysis was performed January 1, 2018, to March 20, 2019. The institutional review boards of all participating institutions approved the study protocol. All participating practitioners provided written informed consent. We obtained written informed assent from patients aged 12 to 17 years or written informed consent from patients aged 18 years. We received a waiver of written parental or guardian consent from the Boston Children’s Hospital and the Tufts Health Sciences institutional review boards, as requiring such consent in our 2008 study¹⁸ was associated with diminished recruitment and significant self-selection bias.

Practice leaders at each site sent emails inviting 80 practitioners to participate. Practitioners who worked fewer than 4 sessions per week or had fewer than 8 adolescent patients per week were excluded. All included practitioners attended a 1-hour orientation session that comprised a demonstration of the tablet computer program, a review of practitioner reports for various categories of risk, the study safety protocol, and a 20-minute video showing examples of brief counseling based on suggested talking points. They also completed a 1-hour online training session with video examples of practitioner counseling and attended a 1-hour motivational interviewing skills development training session. They received 3 American Medical Association Physician’s Recognition Award Category 1 continuing medical education credits for their participation and a $10 gift card after completing the end-of-study evaluation form.

We consecutively screened patients aged 12 to 18 years who presented for annual preventive health visits (Figure 1). Exclusion criteria included practitioner determination of medical or emotional instability at baseline, inability to read English at a third-grade level, or unavailability for follow-up visits during the next 12 months. Whenever possible, research associates first mailed a letter to families describing the study and providing opt-out instructions and followed up with a telephone call to patients to explain the study purpose, procedures, and confidentiality protections and to instruct interested patients to arrive 30 minutes before their appointment. On arrival, research associates privately obtained written informed assent from patients aged 12 to 17 years or written informed consent from patients 18 years or older. Parents or guardians were made aware of the study by reading the letter mailed by research associates, taking telephone messages, or accompanying their child to the clinic visit.

Patients who assented or consented were randomized within practitioner to 1 of 3 arms in a 65% CSBI, 25% usual care (UC), and 10% minimal-assessment UC allocation ratio. The minimal-assessment UC group, which completed the 12-month follow-up only, was used to address an aim in the psychometric study and was not analyzed in this study. We chose to allocate more patients to the CSBI group so that practitioners would have more experience delivering the intervention. On completion of the baseline assessment, the computerized study management system randomized patients within practitioner based on an adaptive, biased-coin minimization scheme,^19,20 in which we minimized imbalance on the following factors, ranked by priority: grade group (middle vs high school), sex, and any drug use. Because practitioners had to deliver the intervention, they were not blinded to patients assigned to the intervention effect arm. However, for patients in the UC arm, practitioners were not informed that the patient was participating in the study.

Two patients in the CSBI group subsequently withdrew from the study. Our recruitment sample size was determined to address the parent study’s aims, not for demonstrating efficacy of the CSBI intervention.

The tablet-based CSBI system started by explaining that responses to the screening questions would be kept confidential unless the practitioner identified an immediate risk to safety. In addition, practitioners asked parents or guardians to leave the room for a portion of the visit to allow a private discussion, which should be a routine part of pediatric practice for adolescents. The patient then self-administered the screening questionnaire that assessed the number of days of alcohol, cannabis, and other drug use in the past 12 months and the 6 CRAFFT questions.¹⁶ The tablet then immediately displayed to patients their CRAFFT score and level of risk, followed by 10 interactive pages of scientific information and true-life vignettes illustrating the health risks of substance use. This took on average 4 minutes to complete in a private location before the visit. Practitioners then logged into the tablet to see the patient’s screen results, risk level, talking points designed to prompt a 2- to 5-minute conversation using motivational interviewing strategies, and the recommended follow-up plan. Practitioners gave a printed Contract for Life²¹ to all patients (and parents or guardians, if present) as a prevention strategy for high- and low-risk patients. The Contract for Life asks youths to agree never to drive after substance use or accept a ride from a substance-impaired driver and instead to call a parent, guardian, or other trusted adult for a safe ride home. Parents or guardians agree to provide safe and sober transportation home and postpone discussion until the following day. Practitioners instructed patients to discuss the Contract for Life with their parent(s) or guardian(s) and to follow up if additional discussion was needed. Practitioners also gave all patients a flyer for a 20-minute family-centered online educational program, Teen-Safe,²² on preventing adolescent substance use.

The content of psychoeducational pages of CSBI was developed and tested in previous research during an iterative process of focus group discussion followed by multiple cycles of user testing with feedback and revision, and the content found favorable results in a 2012 quasi-experimental trial.²³ In the same trial, the Contract for Life had similar, favorable results for reducing risk of riding with an impaired driver 3 months after the intervention.²⁴ The content for Teen-Safe²² was developed by 2 of us (J.R.K. and S.K.H.) for educational presentations to parents and guardians of high school students. The parents and guardians rated Teen-Safe highly, but it has not been otherwise tested.

Patients in the UC group self-administered the computer screening but did not receive any other CSBI components. However, all practitioners received training in the CSBI, which may have affected their delivery of substance use–related counseling with patients in the UC group.

Patients completed a 15-minute baseline assessment battery prior to seeing their practitioner that included sociodemographic questions; Timeline Followback^25,26 (TLFB) for the previous 12 months, which uses a calendar to aid recall of the frequency of use of each substance and scales to assess perceived substance use by peers, siblings, and parents derived from the Personal Experience Inventory^27,28; and riding risk in the past 3 months assessed by 2 items (“In the past 3 months, how many times did you ride with a driver who had been drinking [using marijuana or any other drug]?” followed by 4 response options [not at all, once, twice, and 3 or more times]). To evaluate subjective reactions to the CSBI psychoeducational content, we asked CSBI participants to rate how much they felt the information was “useful,” “exaggerated,” “convincing,” and “irritating” on a scale from 1, indicating not at all, to 7, extremely. Immediately after the medical visit, patients reported on the number of previous visits they had had with the practitioner, receipt of screening and counseling, and ratings of the current visit and responded to open-ended questions about their experience during the visit and whether anything was confusing (“Was there anything you found confusing?”) or uncomfortable (“Was there anything that made you uncomfortable?”).

Trained research associates, blinded to patient screening results, administered the TLFB at baseline and reminded participants to complete it by computer self-administration at each 3-, 6-, 9-, and 12-month follow-up. To maximize data completeness for patients who missed any surveys, the TLFB asked patients to enter data for all months since their last completed survey. Participants received up to $60 in merchandise gift cards based on the number of completed assessments.

Our primary measures of feasibility and acceptability among youths were immediate postvisit reports of receiving counseling about avoiding alcohol and drug use, ratings of counseling quality, receipt of the Contract for Life, and, among those reporting receipt, whether they had discussed it with their parents or guardians by the 3-month follow-up. Among those who reported having used alcohol in the past 12 months at baseline, our primary alcohol use outcome measures were the TLFB-derived time to first postvisit alcohol use and time to first postvisit HED episode. For drug use, we examined time to first postvisit cannabis use among those who reported having used cannabis in the past 12 months at baseline. The prevalence of other drug use at baseline was too low (<1%) for analysis. Finally, we examined the intervention effect on rates of self-reported riding risk at each follow-up. We could not analyze driving risk owing to insufficient numbers (<1%).

We analyzed intent-to-treat groups, with the intervention effect cohort including youths assigned to CSBI regardless of whether they reported receiving practitioner counseling. We conducted analyses using SUDAAN statistical software version 11.0 (RTI International), with practitioner as the nest variable to account for correlated error arising from our cluster-sampling design. To assess baseline group equivalence, we used χ² tests for categorical variables and t tests for continuous variables. We dichotomized race (non-Hispanic white vs other), parents in home (2 vs other), and parent education level (college graduate vs other) to ensure adequate cell sizes. We evaluated CSBI receipt and feasibility and acceptability variables using multivariable logistic regression modeling with generalized estimating equations to compute adjusted relative risk ratios (ARRRs) for CSBI compared with UC, while controlling for baseline group differences. P values were 2-tailed and considered statistically significant at less than .05.

We stratified analyses by baseline substance use in the past 12 months to separately assess intervention effects among youths who reported substance use and prevention effects among youths who reported no substance use. To examine the effect of the CSBI intervention on the number of days to first postvisit alcohol use, HED episode, and cannabis use, we used Cox proportional hazards modeling to compute adjusted hazard ratios that controlled for any baseline group differences. Time data for participants with at least 1 follow-up were included in analyses; those with no substance use during follow-up were censored.

To evaluate riding risk in the past 3 months, we combined any riding with a driver who had been drinking alcohol and any riding with a driver who had been using drugs into a single variable and used multivariable logistic regression with generalized estimating equations to compute ARRRs at each follow-up. These analyses were stratified by report of riding risk in the past 3 months at baseline rather than use of alcohol or other drugs in the past 12 months. Because of a technical problem with riding risk data at 3-month follow-up, we present only 6-, 9-, and 12-month outcomes.

Of the 80 invited practitioners, 54 (68%) agreed to participate and completed training, including 39 pediatricians (49%) and 15 nurse practitioners (19%). Of 1200 patients initially screened, we invited 1098 (91.5%) to participate, and 965 patients (87.9%) assented or consented and were randomized to 1 of the study arms: CSBI (n = 628), UC (n = 243), or minimal-assessment UC arm (n = 94). The minimal-assessment UC arm was used only in psychometric analyses and not included in outcome analyses. Two patients in the CSBI group withdrew from the study prior to completing the baseline assessment. Patients who reported any substance use or riding risk at baseline composed the intervention effect cohort, and patients who reported no substance use or riding risk composed the prevention effect cohort.

The intervention effect cohort included 211 patients (24.3%) who reported any use of alcohol or cannabis in the past 12 months at baseline (alcohol, 192 [22.1%]; cannabis, 106 [12.2%]). Mean (SD) age was 16.4 (1.3) years, 114 (54.1%) were female, 105 (49.8%) were non-Hispanic white, and 144 (70.9%) had a college-educated parent (Table 1). Among this intervention effect cohort, most patients (178 [84.4%]) reported seeing a pediatrician, 149 patients (70.6%) saw a female practitioner, and more than half (123 [58.3%]) reported 6 or more previous visits with the practitioner. Nearly all (192 [91.4%]) reported using alcohol in the past 12 months, with a median (interquartile range [IQR]) of 3 (2-6) drinking days, and 70 patients (33.2%) reported at least 1 HED episode. Of the 211 patients who reported any use of alcohol or other drugs, 106 (50.2%) reported cannabis use in the past 12 months, with a median (IQR) of 3.5 (2-15) days of use. Fifty-nine patients (28.1%) met criteria for high risk of a substance use disorder (CRAFFT score ≥2),²⁹ and 56 patients (26.5%) reported having ridden in a car in the past 3 months with a driver who had used alcohol or other drugs at baseline. Most patients (188 [89.1%]) reported having close friends who used alcohol or drugs, and 82 patients (42.7%) had siblings who used alcohol or drugs. The CSBI and UC groups did not differ significantly on any baseline variable, except number of days of drinking during the past 12 months; we controlled for this variable in all further analyses. All patients completed the postvisit questionnaire before leaving the office, 717 patients (82.5%) completed at least 1 follow-up (CSBI, 529 of 626 patients [84.5%]; UC, 189 of 243 patients [77.8%]; P = .15), and 692 patients (79.6%) completed the 12-month follow-up (CSBI, 502 of 626 patients [80.2%]; UC, 190 of 243 patients [78.2%]; P = .09) (Figure 1).

The prevention effect cohort consisted of 658 youths (CSBI, 478; UC, 180) who reported no use of alcohol or cannabis in the past 12 months at baseline. The CSBI group was younger and less likely to report hanging out with friends who used substances (eTable 1 in Supplement 2), and these variables were entered as control variables in all further analyses.

In the intervention effect and prevention effect cohorts, youths in the CSBI group were more likely than youths in the UC group to report receiving advice about cannabis (ARRR, 1.36 [95% CI, 1.09-1.69]) and about not riding with an impaired driver (ARRR, 1.31 [95% CI, 1.09-1.57]) or driving while impaired (ARRR, 1.24 [95% CI, 1.03-1.50]) and to report receiving information about the health risks of alcohol use (ARRR, 1.22 [95% CI, 1.04-1.44]) and cannabis use (ARRR, 1.34 [95% CI, 1.09-1.65]) (Table 2; eTable 2 in Supplement 2). In the prevention effect cohort, there was significantly higher reported receipt of advice about avoiding alcohol use in the CSBI group compared with the UC group (ARRR, 1.30 [95% CI, 1.17-1.43]). In the intervention effect cohort, among 59 patients in the CSBI group with risk levels that indicated practitioners should try to bring them back for a follow-up visit, 27 (45.8%) reported being asked to return for a follow-up visit compared with 6 of 23 patients (26.1%) in the UC group, although this difference did not reach statistical significance. More than three-quarters of patients in the CSBI group in the intervention effect and prevention effect cohorts (42 of 55 [76.4%] and 141 of 178 [79.2%], respectively) reported receiving the Contract for Life. In the intervention effect cohort, the CSBI and UC groups did not differ significantly on receiving advice about alcohol use, ratings of the advice received, likelihood of following the advice, or satisfaction with the visit. In contrast, in the prevention effect cohort, patients in the CSBI group were significantly more likely than patients in the UC group to rate the information they received as excellent or good (ARRR, 1.13 [95% CI, 1.00-1.27]) and to be very likely to follow their practitioner’s advice (ARRR, 1.15 [95% CI, 1.00-1.31]) (eTable 2 in Supplement 2). However, they did not differ on level of overall satisfaction with the visit.

In the total sample, youth feedback regarding their overall visit experience was highly positive, with 808 patients (92.9%) using words such as excellent, fun, informative, and interesting. Only 12 patients reported some discomfort, giving comments such as, “being asked about riding in cars with people that drink makes me nervous now,” or “thinking about someone driving drunk is upsetting.” Patients in the CSBI group tended to rate the CSBI psychoeducational content highly; median (IQR) scores were 5.0 (4.0-6.0) for useful, 5.0 (4.8-6.0) for convincing, 2.0 (2.0-4.3) for exaggerated, and 2.0 (1.0-2.0) for irritating. Fifty of 54 participating practitioners completed a follow-up questionnaire, and 44 (88%) rated the system as moderately or very useful, 40 (80%) reported that it increased their confidence in behavioral counseling, and 31 (62%) would recommend it to other practices, while 16 (32%) were undecided and 3 (6%) would not.

In the intervention effect cohort, adjusted hazard ratios from Cox proportional hazards modeling comparing time to first postvisit use between CSBI and UC groups, with adjustment for baseline group differences, were 0.69 (95% CI, 0.47-1.02) for alcohol use, 0.66 (95% CI, 0.40-1.10) for HED, and 0.62 (95% CI, 0.41-0.94) for cannabis use (adjusted hazard ratios <1 indicate that patients in the CSBI group tended to have longer time to first use compared with patients in the UC group) (Figure 2). In the prevention effect cohort, adjusted hazard ratios comparing time to first postvisit use between CSBI and UC groups, with adjustment for baseline group differences, were 0.87 (95% CI, 0.57-1.31) for alcohol use and 0.76 (95% CI, 0.44-1.32) for cannabis use (eFigure in Supplement 2). These results did not reach statistical significance, possibly because of low levels of substance use overall in this cohort.

Among 99 patients in the total sample who reported riding risk at baseline, the CSBI group had an 18% lower risk than the UC group of reporting riding risk at the 6-month follow-up, but the difference was not statistically significant (ARRR, 0.82; 95% CI, 0.50-1.34). The effect size increased over time, with a 27% lower risk at 9 months (ARRR, 0.73; 95% CI, 0.44-1.19), although not statistically significant, and 42% lower risk at 12 months (ARRR, 0.58; 95% CI, 0.37-0.91) (Table 3). There were no meaningful differences between the CSBI and UC groups among the 769 participants who reported no riding risk at baseline.

Our results suggest that the CSBI system for youth substance use intervention and prevention is feasible and acceptable in primary care, and among patients who report substance use at baseline, effect size estimates were sufficient to warrant further testing. We also found sufficient effect size estimates for reducing youths’ risk of riding in cars with substance-impaired drivers, a behavior associated with a leading cause of youth mortality. Effect sizes among those who reported no substance use at baseline were smaller and not significant, perhaps owing in part to the low number of patients who began to use alcohol or other drugs during the 12-month follow-up.

We powered our parent study to assess the psychometric properties of the National Institute on Alcohol Abuse and Alcoholism–recommended approach to youth screening, not to demonstrate efficacy of the CSBI intervention. Nonetheless, we found a significant intervention effect on cannabis use, clinically relevant effect sizes for alcohol use and HED, and a significant decrease in riding with an impaired driver at the 12-month follow-up. However, sample sizes and effect sizes have been shown to be negatively correlated; therefore, given the moderate study sample used in these analyses, these effect sizes should be viewed cautiously.³⁰ Additionally, we found that pediatric practitioners were able to deliver the brief intervention in only a few minutes, which bodes well for widespread adoption of the approach. Other studies of brief interventions in youth primary care have relied on standalone computer programs or trained therapists to deliver the intervention,^31,32 which may be financially impractical for smaller pediatric group practices to implement.

The CSBI program was acceptable to the youth patients, who described it as excellent, fun, informative, useful, and convincing. Twelve patients reported some discomfort, largely in response to content about the dangers of riding with an impaired driver. The CSBI system’s feasibility has been supported by previous data published elsewhere.¹⁹ Of the 50 participating practitioners who completed a follow-up questionnaire, most of them rated the system as moderately or very useful, reported that it increased their confidence in behavioral counseling, and said they would recommend it to other practices. The greatest drawback reported was the time it took for the CSBI study protocol before the visit, which impeded clinic flow. Among suggestions for improvement was the need to integrate CSBI with the electronic health record.

This study had potential limitations. Using the same trained practitioners to deliver both the CSBI and UC intervention effect arms may have caused contamination of the UC arm, but we chose this design over practitioner randomization because our 2012 study²³ found that practitioner characteristics are major covariates of the outcome, and we had only a modest total number of practitioners. We also considered randomization by practice, but this design requires too large a number of practices to be feasible under prevailing research funding limits. The only group difference we could ensure was the computer-based CRAFFT screen with immediate personalized feedback and 10 psychoeducational pages. We did find high rates of substance use–related counseling in our UC group compared with a national sample of US tenth graders, in which only 40% of adolescents who visited a physician in the past 12 months reported being advised about alcohol-related health risks and only 17% reported being advised to reduce or stop drinking.⁷ In our study, more than 60% of patients in the UC group reported receiving health risk information, and more than 57% of patients who reported prior substance use received advice to reduce or stop. Other factors that may have contributed to higher rates of counseling on use of alcohol or other drugs in our study include the Hawthorne effect³³ caused by practitioners knowing they were being observed and all the participating sites being near Boston, Massachusetts, where our center has been active in substance use screening research, dissemination of results, and clinical teaching in the area for more than 15 years. Future studies should use more diverse, national samples. Additionally, we relied solely on reporting from patients of the intervention components received and did not obtain independent observations of practitioner behaviors. Despite these limitations, we still found estimates of effect size that would be clinically important and, for cannabis use and riding with impaired drivers, statistically significant.

Other study limitations include our reliance on self-reported data, although under confidential conditions, youths’ self-reports have been shown to be reliable and compare favorably with other forms of substance use detection.^34-36 We were unable to examine effects on use of drugs other than cannabis owing to low prevalence. Additionally, we did not collect any data on parents’ or guardians’ views of the confidential CSBI system, including information on discussing the Contract for Life with their teenagers or viewing the Teen-Safe educational program. These data should be included in future research.

The CSBI system showed sufficient feasibility and acceptability for implementation in busy pediatric practices to warrant further testing, and it showed promise for delaying postvisit alcohol and cannabis use, HED, and riding with impaired drivers among patients with prior risk. Future research should include larger samples, randomize by practitioner or practice, assess the contribution to the intervention’s effect by its various elements (eg, tablet computer informational pages, practitioner counseling), evaluate strategies to promote real-world dissemination, including integration of CSBI within electronic health record systems, and assess cost-effectiveness.

Accepted for Publication: May 8, 2019.

Published: June 21, 2019. doi:10.1001/jamanetworkopen.2019.6258

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Knight JR et al. JAMA Network Open.

Corresponding Author: Sion Kim Harris, PhD, Division of Adolescent and Young Adult Medicine, Boston Children's Hospital, 300 Longwood Ave, Mailstop 3189, Boston, MA 02115 (sion.harris@childrens.harvard.edu).

Author Contributions: Drs Knight and Harris had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Knight, Sherritt, Harris.

Acquisition, analysis, or interpretation of data: Knight, Sherritt, Gibson, Levinson, Grubb, Samuels, Silva, Vernacchio, Wornham, Harris.

Drafting of the manuscript: Knight, Sherritt, Gibson, Levinson, Harris.

Critical revision of the manuscript for important intellectual content: Knight, Sherritt, Gibson, Grubb, Samuels, Silva, Vernacchio, Wornham, Harris.

Statistical analysis: Sherritt, Harris.

Obtained funding: Knight, Sherritt, Harris.

Administrative, technical, or material support: Sherritt, Gibson, Levinson.

Supervision: Knight, Sherritt, Gibson, Grubb, Samuels, Silva, Vernacchio, Wornham, Harris.

Conflict of Interest Disclosures: Dr Knight reported holding the copyright for the CRAFFT screening tool. Drs Knight and Harris and Mr Sherritt reported claiming the computer-facilitated brief intervention tool as their intellectual property but have not applied for nor received a patent or copyright. No other disclosures were reported.

Funding/Support: This study was supported by grants from the National Institute on Alcohol Abuse and Alcoholism. Other support was provided by the Leadership Education in Adolescent Health Training Program and a cooperative agreement from the Maternal and Child Health Bureau, US Department of Health and Human Services (Dr Harris).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: This information, or content, and conclusions are those of the authors and should not be construed as the official position or policy of nor should any endorsements be inferred by the Health Resources and Services Administration, US Department of Health and Human Services, or the US government.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We thank East Boston Neighborhood Health Center, Lexington Pediatrics, Tufts Medical Center Floating Hospital for Children, Longwood Pediatrics, and Boston Children’s Primary Care at Longwood Children’s Hospital for providing recruitment access for this study through their clinics.