Vertical lines indicate 95% CIs.
eTable. Risk Behaviors Included in Overall Summary Outcome Measure
Data Sharing Statement
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Richardson LP, Zhou C, Gersh E, Spielvogle H, Taylor JA, McCarty CA. Effect of Electronic Screening With Personalized Feedback on Adolescent Health Risk Behaviors in a Primary Care Setting: A Randomized Clinical Trial. JAMA Netw Open. Published online May 10, 20192(5):e193581. doi:10.1001/jamanetworkopen.2019.3581
Does electronic risk screening with personalized feedback and clinical decision support increase clinician counseling and reduce risk behaviors in adolescents?
In this randomized clinical trial of 300 adolescents randomly assigned to receive electronic screening with personalized feedback vs usual care in the context of a well-child care visit, youths who received the electronic screening intervention were more likely than controls to receive risk counseling. Three months after the visit, youths who received the electronic screening also had a significantly greater reduction than controls in their risk behavior scores.
Electronic screening tools that provide risk information to clinicians and motivational feedback to teens can improve care delivery and youth outcomes.
Health risk behaviors are a leading cause of morbidity during adolescence. Screening and counseling for health risk behaviors are recommended but infrequently performed.
To test the effect of an electronic screening and feedback tool on clinician counseling and adolescent-reported health risk behaviors.
Design, Setting, and Participants
A randomized clinical trial compared electronic screening and feedback on an intention-to-treat basis with usual care among 300 youths 13 to 18 years of age at 5 pediatric clinics in the Pacific Northwest. Outcomes were assessed via electronic survey at 1 day and 3 months after the initial visit. Study data collection occurred from March 13, 2015, to November 29, 2016, and statistical analysis was conducted between February 6, 2017, and June 20, 2018.
Youths in the intervention group (n = 147) received electronic screening and personalized feedback with clinician clinical decision support. Youths in the control group (n = 153) received standard screening and counseling as provided by their clinic.
Main Outcomes and Measures
Youths’ report of receipt of counseling during the visit and risk behaviors at 3 months.
In the final study sample of 300 youths (intervention group, 75 girls and 72 boys; mean [SD] age, 14.5 [1.4 years]; and control group, 80 girls and 73 boys; mean [SD] age, 14.5 [1.4] years), 234 (78.0%) were aged 13 to 15 years. After adjusting for age, sex, and random effect of clinic, youths in the intervention group were more likely to receive counseling for each of their reported risk behaviors than were youths in the control group (adjusted rate ratio, 1.32; 95% CI, 1.07-1.63). Youths in the intervention group had a significantly greater reduction (β = –0.48; 95% CI, –0.89 to –0.02; P = .02) in their risk behavior scores at 3 months when compared with youths in the control group.
Conclusions and Relevance
Electronic screening of health risk behavior with clinical decision support and motivational feedback to teens can improve care delivery and outcomes.
ClinicalTrials.gov identifier: NCT02360410
Health-compromising behaviors, such as risky sexual behaviors and substance use, often increase in adolescence and are associated with significant short-term and long-term morbidity.1,2 In recognition of the effect of these behaviors, screening is recommended by Bright Futures3 and the Guidelines for Adolescent Preventive Services,4 2 leading US adolescent well-child care guidelines. However, despite the availability of guidelines and training materials, implementation of preventive screening remains low.5-7 Less than half of youths who are seen for well care are screened and even fewer receive counseling regarding risk behaviors.8-12
Several studies have been conducted to improve the delivery of counseling during well-adolescent care visits. The addition of standardized screening methods, including electronic screening tools, has been shown to increase detection and clinician discussion.13,14 A recent review article identified only 9 studies of screening for multiple risk behaviors and health care–based interventions with youth outcome data.15 There was considerable variation across the studies in the intensity of the interventions, the behaviors studied, and the outcomes. None of the studies incorporated electronic feedback for youths.
We conducted a study to examine the efficacy of a youth-facing electronic screening and personalized motivational feedback tool, the Check Yourself tool, which was designed to be administered prior to a well-child care visit to prime youths for discussions with their clinicians. The tool also includes clinical decision support for clinicians including a streamlined clinician summary report to encourage counseling for high-risk and moderate-risk behaviors. This study examines the efficacy of this tool in increasing youth-reported clinician counseling during the well-child care visit and improving health behavior outcomes 3 months after the visit.
This was a randomized clinical parallel-group intervention study conducted in the Pacific Northwest of the United States (trial protocol in Supplement 1). Adolescent participants (13-18 years of age) and 1 parent per teen were recruited from 5 pediatric clinics in the Puget Sound Pediatric Research Network in western Washington state between March 13, 2015, and August 8, 2016, with follow-up through November 29, 2016. Clinics were invited to participate based on the size of the practice and the number of adolescent patients and were added to the study on a rolling basis with a goal of recruiting 60 youths per site for a total recruitment goal of 300 youths. The target sample size was determined by the study statistician to give 80% power to detect a difference in the primary risk behavior measure with 95% certainty. All procedures for recruitment were approved by the Western Institutional Review Board. Parental consent and youth assent were required for participants 13 to 17 years of age, while consent was obtained directly from 18-year-old participants. Assent and consent were obtained in writing from participants who completed study procedures in person and verbally for individuals who completed study procedures via telephone. Study results are presented following the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.
Adolescents with an upcoming well visit were invited by study staff to participate first via letter with an opt-out telephone number followed by a telephone call to assess eligibility and obtain consent and assent. Exclusion criteria included being outside the study age range, lacking telephone or internet access to complete surveys, having a sibling who was previously invited to participate, having unenrolled from the clinic or cancelled the well visit, or not speaking English. Participants were told that they would receive 1 of 2 versions of an electronic tool: version 1 would have feedback and a clinician report and version 2 would have no feedback and no report to the clinician.
For random allocation of participants, a statistician-developed computer-generated list of random numbers was entered into DatStat16 with 1:1 randomization stratified by age (13-15 or 16-18 years), sex, and clinic. Randomization was completed after participants completed consent and assent procedures with study staff. The participants were not explicitly told their assignment but they could potentially intuit it based on whether they received electronic feedback. Baseline data were collected by trained study staff either in person in a private setting or online with telephone support in advance of the youths arriving in the clinic. For all youths, study staff emphasized the importance of completing screening in a confidential setting.
Participants who received the intervention completed an electronic screening tool with personalized feedback and their clinician received clinical decision support via a clinician summary. The tool screened participants for protective factors and risk behaviors following a HEADSS (home, education, activities, depression, sexual activity, safety, and substance use) framework.17 In addition, the tool screened for specific nutritional behaviors, physical activity, and sleep. The personalized feedback, delivered as part of the same screening session based on integrated algorithms, was designed to motivate healthier behaviors and to encourage discussions with the clinician during the well visit. Specific types of feedback included comparison of reported behaviors with rates reported by peers or national guidelines, education about health risks, and tips for behavior change.18 Based on internal tracking data, the tool took an average of 15 minutes to complete.
The clinician summary report consisted of a dashboard that flagged youth behavior as low risk, moderate risk, or high risk within the following 6 categories: nutrition, activity, substance use, emotions, sexual activity, and safety. Below the dashboard, individual responses were provided for questions in each category. High-risk and moderate-risk behaviors were defined a priori based on health guidelines or expert consensus (eTable in Supplement 2). Study staff coordinated with clinic administrative staff to ensure that clinicians received a printed summary report prior to the visit.
Youths in the control group completed only baseline electronic screening. Clinicians of youths in the control group did not receive results of electronic screening. Clinics were encouraged to continue their standard preexisting health risk screening procedures for all youths regardless of study group throughout the study and not to alter their standard procedures for study participants. Standard well visit screening procedures varied by site, with some clinics using paper screeners at intake and others relying solely on clinician interview to identify health risks.
As patients were randomized on an individual basis, clinicians could care for both intervention and control patients. Prior to the onset of the study in each clinic, all clinicians in the clinics were invited to participate in a single 15-minute online training module orienting them to the tool, with a very brief overview of the tenets of motivational interviewing.
Participants completed follow-up surveys 1 day and 3 months after their well visit either online or over the telephone with research assistant support. The 1-day follow-up survey asked about receipt of discussion and counseling to change behavior for each screened behavior and the youth’s level of motivation to change his or her behaviors. Questions for this section were adapted from the Adolescent Report of the Visit developed by Ozer and colleagues.19 At the 3-month follow-up survey, youths completed questionnaires assessing the same health risk behaviors as at baseline collected via an online survey tool, DatStat.16 All youths who reported suicidal ideation at the baseline screening or 3-month follow up assessment received a telephone call from a study clinician to assess risk and assist with connecting the youth with care, if needed.
Statistical analyses were conducted between February 6, 2017, and June 20, 2018. Data were uploaded into R, version 3.5.020 for statistical analysis. Bivariate analyses were conducted to examine differences between youths in the control and intervention groups in demographics and baseline risks. Race and ethnicity were gathered directly from youths as part of intake screening to better describe participants to inform generalizability. There were 2 primary outcome measures for analyses determined a priori at the time of study design: receipt of counseling during the well visit and a summary score of health risk behaviors measured 3 months after the well visit.
Receipt of counseling, measured 1 day after the well visit, was defined as youth-reported receipt of clinician counseling to change a behavior toward better health. Poisson regression models compared the youths in the intervention and control groups regarding the total number of youth-reported moderate-risk and high-risk behaviors for which they reported receiving counseling. To ensure that the coefficient of group indicator captured the differences in rates of targeted counseling between groups, the overall number of moderate-risk and high-risk behaviors reported by youths was included as an offset variable in the Poisson models. To explore potential differential effects of the intervention on high-risk vs moderate-risk behaviors, secondary analyses were conducted examining the association between intervention status and rates of counseling for each of these categories of behavior. All regression analyses controlled for age at baseline and sex, consistent with the stratified randomization, and included a clinic-specific random effect to account for clustering within clinics.
The primary outcome measure for the assessment of risk behaviors at 3 months after the intervention was a summary score of all 13 assessed behaviors. The score was weighted based on tool-defined risk levels: high-risk behaviors were assigned a score of 2, moderate-risk behaviors a score of 1, and low-risk behaviors a score of 0 (eTable in Supplement 2). A total risk score was calculated for each youth by summing all individual risk scores, with a potential total range of 0 to 21. Linear regression methods were used to compare differences in youth-reported total risk score at 3 months in youths in the intervention group vs those in the control group controlling for the baseline risk score. We also conducted exploratory analyses to examine differential effects of the tool on high-risk vs moderate-risk behaviors using separate Poisson regression analyses to examine the total number of behaviors meeting each of these categories. Finally, to examine if there were stronger effects of the intervention on specific health risk behaviors, we conducted exploratory logistic regression analyses for each of the assessed behaviors. Because of concerns about estimate instability, logistic regression analyses were not conducted for behaviors reported by fewer than 10 youths per study group.
The control group was considered the reference group for all regression analyses. For mixed-effects Poisson and logistic regression models, we assessed statistical significance of estimates based on whether 95% CI for rate ratio or odds ratio included 1. For mixed-effects linear regression models, statistical significance was based on P < .05 calculated via the Satterthwaite df method.21 In secondary analyses examining individual behaviors, we used a conservative definition of P < .01 to reduce the likelihood of identifying a spurious relationship by chance.
In total, 1272 adolescents were sent a letter inviting them to participate (Figure 1). The final study sample included 300 youths (intervention group, 75 girls and 72 boys; mean [SD] age, 14.5 [1.4 years]; and control group, 80 girls and 73 boys; mean [SD] age, 14.5 [1.4] years) who completed all consent and baseline procedures (27.0% of the eligible sample of 1113 youths); 147 individuals were randomized to the intervention group and 153 were randomized to the control group. Because of an error in implementation, 2 youths who were randomized to the intervention group did not receive feedback. They were retained in the intervention group for all analyses, consistent with an intention-to-treat protocol. The response rate was 97.7% (n = 293) for the 1-day follow-up and 97.3% (n = 292) for the 3-month follow-up.
There were no differences between the intervention and control groups in demographics, baseline risk score, or the number of baseline moderate-risk or high-risk behaviors (Table 1). A total of 155 participating youths (51.7%) were female. Youths predominantly identified as white (201 [67%), Asian (40 [13.3%]), and other or multiracial (46 [15.3%]). A total of 9 participants (3.0%) identified as Hispanic. A total of 234 youths (78.0%) were aged 13 to 15 years. A total of 125 youths (41.7%) lived in homes where at least 1 parent had a graduate or professional degree.
A total of 285 youths (95.0%) had at least 1 health risk behavior at baseline. The distribution of risk behaviors was similar across study groups. Table 2 lists the reported risk behaviors in order of baseline frequency, with the most commonly reported risk behavior in the sample being low fruit and vegetable intake (control group, 125 of 153 [81.7%] vs intervention group, 114 of 147 [77.6%]) and the least frequent being unsafe sex (control group, 1 of 153 [0.7%] vs intervention group, 3 of 147 [2.0%]). The mean (SD) number of moderate-risk behaviors at baseline was 2.18 (1.32; median, 2; interquartile range, 1-3) for youths in the intervention group and 2.17 (1.23; median, 2; interquartile range, 1-3) for youths in the control group (P = .63). The mean (SD) number of high-risk behaviors at baseline was 0.76 (1.07; median, 0; interquartile range, 0-1) for youths in the intervention group and 0.61 (0.82; median, 0; interquartile range, 0-1) for those in the control group (P = .87) (Table 1).
Youths in the control reported 319 moderate-risk behaviors and received counseling for 130 of these behaviors (40.8%) during clinician visits; those in the intervention group reported 314 moderate-risk behaviors and received counseling for 160 of these behaviors (51.0%) during clinician visits (P < .001). Youths in the control group reported 87 high-risk behaviors and received counseling for 21 of these behaviors (24.1%) during the clinician visit; those in the intervention group reported 105 high-risk behaviors and received counseling for 40 of these behaviors (38.1%) during the clinician visit (P < .001). Youths in the intervention group were significantly more likely to receive counseling for their reported moderate-risk and high-risk behaviors than were those in the control group (adjusted rate ratio [aRR], 1.32; 95% CI, 1.07-1.63). To examine differential effect of the intervention on counseling for high-risk, moderate-risk, and low-risk behaviors, we also examined rates of counseling for youths in the intervention and control groups in each of these categories (Figure 2). Youths in the intervention group were 1.28 times more likely than those in the control group to report having received counseling for moderate-risk behaviors (aRR, 1.28; 95% CI, 1.02-1.62). For high-risk behaviors, the rate of counseling was 1.61 times higher among the intervention group than the control group, but this difference was not statistically significant (aRR, 1.61; 95% CI, 0.95-2.73). There were no significant differences in counseling for no-risk or low-risk behaviors between the intervention and control groups (aRR, 1.02; 95% CI, 0.77-1.36).
The overall combined mean (SD) baseline risk score was 3.71 (2.79) in the intervention group and 3.39 (2.27) in the control group (P = .48) (Figure 3). At 3 months, the mean (SD) risk score for the intervention group was 2.89 (2.41), compared with 3.25 (2.37) for those in the control group (P = .08). On mixed-effects linear regression analysis, youths in the intervention group had a significantly greater decrease in risk behavior scores at 3 months compared with those in the control group (β = –0.48; 95% CI, –0.89 to –0.02; P = .02). When examining for effect modification by moderate-risk or high-risk behavior status, the intervention had a significant effect on reduction in the number of high-risk behaviors in the intervention group vs the control group (aRR, 0.61; 95% CI, 0.43-0.88), but not on the number of moderate-risk behaviors (aRR, 0.91; 95% CI, 0.78-1.07). In secondary analyses examining individual behaviors, significant reductions of behaviors in the intervention vs control groups were noted for only 2 behaviors: inconsistent helmet use (26 of 141 [18.4%] vs 38 of 151 [25.2%]) and inconsistent seatbelt use (7 of 141 [5.0%] vs 14 of 151 [9.3%]) (Table 2).
This study found that use of the electronic screening and personalized feedback tool increased the delivery of counseling and was associated with reduced overall rates of adolescent risk behaviors 3 months later. This finding suggests that electronic screening with integrated personalized feedback holds promise as a strategy for improving not only screening but also counseling for risk reduction and adolescent behavior change. To our knowledge, this is the only study that has examined the effect of a tool that simultaneously combines behavioral risk assessment with patient feedback and clinical decision support for clinicians on health risk behavior outcomes.
Adolescent health guidelines recommend screening across a broad range of behaviors in the context of preventive health visits. One key benefit of the tool is that it was designed to provide positive reinforcement, education, and tips for youths who were not reporting risk, thus ensuring that all youths receive some feedback on their behaviors. In addition, the clinician summary report was designed to provide clinical decision support by highlighting areas of reported risk to encourage clinicians to prioritize targeted discussion of higher-risk behaviors vs areas where behavior indicated low risk. The finding that youths in the intervention group were more likely than those in the control group to receive counseling for reported risk behaviors suggests that the tool was effective in helping clinicians prioritize counseling based on reported risk. Because clinicians do not discuss all behaviors with all patients, this approach may result in lower overall rates of counseling than interventions that target increased discussions of behaviors among all youths,19 but it may allow for more efficient use of time to address clinically relevant behaviors. Clinicians in our study completed a single 15-minute online training with basic motivational interviewing principles, compared with prior interventions that have used more intensive training22-24 and/or delivery of counseling by another clinician.22,25
This study has limitations. One key difference between our analytic approach and that of other studies is that one of our primary outcomes was a multirisk measure. The use of a combined measure allowed us to test across behaviors, which is important in a primary care sample in which the prevalence of any individual risk behavior might be low. Although there are many analytic benefits to the combined measure, one limitation of this approach is that it makes it more difficult to interpret in the context of specific health behaviors. We conducted secondary analyses of individual behaviors to allow for more ready interpretation of the intervention; however, for many behaviors the prevalence at baseline was too low to adequately assess the effect of the tool and thus we were limited in our ability to definitively draw conclusions on the effect of the tool on low-frequency behaviors. Helmet use, which is the only behavior that has shown improvements in 2 previous studies,23,26 was of adequate prevalence and showed significant improvement in our sample as well. Seatbelt use, which showed significant improvement in our sample, did not show consistent patterns of improvement in prior studies.23,26
Another limitation of this study was that few youths reported high-risk behaviors, which resulted in limited power to examine these outcomes. We believe that the low rate of behaviors is owing to the selection of youths seeking well care in pediatric primary care clinics. The rates of risk behavior prevalence in our study were similar those of to a more recent large-scale pediatric primary care screening study.27 Several factors may predispose pediatric primary care samples to lower rates of risk behavior. First, consistent with our study, in which 234 participants (78.0%) were in the age range of 13 to 15 years, prior research has shown that younger adolescents are up to 3 times more likely than older teens to have preventive visits.28 Because risk behaviors increase with age, their prevalence would be expected to be lower for a younger sample. Second, youths who are engaging in risk behaviors, such as sexual activity, may seek health care from nonpediatric settings in which there may be greater access to contraceptive care, such as school-based health clinics or family medicine settings. Using the same screening tool, another study found higher rates of high-risk behaviors in school-based health settings.29 Third, parents typically bring their children to well-child care visits, which may influence youths’ reporting of risk behaviors. Although we instructed all youths to complete the electronic screening tool in a confidential setting, it is possible that concerns about confidentiality led them to underreport some behaviors. Future studies should aim to better understand the prevalence of health risk behaviors in pediatric primary care settings, where screening is recommended, to inform the right level of interventions. In addition, regarding our tool, more work is needed in a larger sample of older adolescents and higher-risk youths to better understand its generalizability and effectiveness in these populations.
Other limitations include the fact that it is not possible to separate the effects of the feedback in the tool from those of clinician counseling because of the design of the study. As randomization occurred at the level of the patient, it is also possible that clinicians changed their practices with youths in the control group based on what they learned from working with youths in the intervention group. To the extent that this change occurred, it would introduce a conservative bias and make it less likely to detect a difference between groups. Also, this study was conducted among predominantly white low-risk youths seen for a well visit in primary care clinics in the Pacific Northwest and may not be generalizable to other, more diverse settings.
Despite these limitations, this tool has the potential to significantly improve outcomes for youths. Health risk behaviors are the leading cause of morbidity and mortality in youths and across the life span, with even modest changes conferring potentially important long-term implications for the health of youths. Electronic screening with personalized feedback requires minimal training and clinician time for implementation and may be an effective strategy for delivering preventive and risk reduction counseling to youths.
Accepted for Publication: March 22, 2019.
Published: May 10, 2019. doi:10.1001/jamanetworkopen.2019.3581
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Richardson LP et al. JAMA Network Open.
Corresponding Author: Laura P. Richardson, MD, MPH, Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, 2001 Eighth Ave, Seattle, WA 98121 (firstname.lastname@example.org).
Author Contributions: Drs Richardson and Zhou had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Richardson, Zhou, Spielvogle, McCarty.
Acquisition, analysis, or interpretation of data: Richardson, Zhou, Gersh, Taylor, McCarty.
Drafting of the manuscript: Richardson.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Zhou, Gersh.
Obtained funding: Richardson, McCarty.
Administrative, technical, or material support: Spielvogle, Taylor, McCarty.
Conflict of Interest Disclosures: Drs Richardson and McCarty reported receiving grants from Health Resources and Services Administration Maternal Child Health Bureau during the conduct of the study. Drs Richardson and McCarty reported having a license agreement with Tickit Health Inc as inventors of the Check Yourself Tool whereby they will receive royalties from the future sale of the tool to other health care companies; Seattle Children’s Hospital has a management plan in place to oversee their interests with Tickit Health Inc. No other disclosures were reported.
Funding/Support: This project was supported by the Health Resources and Services Administration of the US Department of Health and Human Services under the R40 grant program (grant R40MC26817: Promoting Adolescent Health through Personalized Feedback).
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by the Health Resources and Services Administration, US Department of Health and Human Services, or the US government.
Data Sharing Statement: See Supplement 3.
Create a personal account or sign in to: