A dose effect was observed for the effect of the intervention on depressive symptoms at 12 months. Among males, those with higher engagement in intervention services had significantly more reductions in depressive symptoms than those with lower service engagement. No significant dose effect was observed among females.
eTable 1. Intervention Engagement by Sex
eTable 2. Covariate Balance Before Matching on Dose
eTable 3. Joint Distribution of Dose Within Matched Pairs: Exposure to Mental Health Services
eTable 4. Distribution of Dose Differences Within Matched Pairs
eTable 5. Covariate Balance After Matching
eAppendix. Missingness Analysis
Tandon SD, Latimore AD, Clay E, Mitchell L, Tucker M, Sonenstein FL. Depression Outcomes Associated With an Intervention Implemented in Employment Training Programs for Low-Income Adolescents and Young Adults. JAMA Psychiatry. 2015;72(1):31-39. doi:10.1001/jamapsychiatry.2014.2022
Recent estimates indicate that 6.5 million adolescents and young adults in the United States are neither in school nor working. These youth have significant mental health concerns that require intervention.
To determine whether a mental health intervention, integrated into an employment training program that serves adolescents and young adults disconnected from school and work, can reduce depressive symptoms and improve engaged coping strategies.
Design, Setting, and Participants
A quasi-experimental study was conducted; 512 adolescents and young adults newly enrolling in one employment training program site were intervention participants, while 270 youth from a second program site were enrolled as controls. Participants were aged 16 to 23 years and not in foster care. Study recruitment took place from September 1, 2008, to May 31, 2011, with follow-up data collection occurring for 12 months after recruitment. Propensity score matching adjusted for observed baseline differences between the intervention and control groups.
Main Outcomes and Measures
Depressive symptoms measured on a Center for Epidemiologic Studies Depression Scale (CES-D) and engaged coping strategies.
The mean age of participants was 19 years, 93.7% were African American, and 49.4% were male. Six- and 12-month follow-up rates were 61.0% (n = 477) and 56.8% (n = 444), respectively. Males in the intervention group with high baseline depressive symptoms exhibited a statistically significant decrease in depressive symptoms at 12 months (5.64-point reduction in CES-D score; 95% CI, –10.30 to –0.96; P = .02) compared with similar males in the control group. A dosage effect was observed at 12 months after the intervention, whereby males with greater intervention exposure showed greater improvement in depressive symptoms compared with similar males with lower intervention doses (effect on mean change in CES-D score, −3.37; 95% CI, –6.72 to –0.09; P = .049). Males and females in the intervention group were more likely than participants in the control group to increase their engaged coping skills, with statistically significant differences found for males (effect on mean change in CES-D score, 0.32; 95% CI, 0.14-0.50; P = .001) and females (effect on mean change in CES-D score, 0.19; 95% CI, 0.01-0.37; P = .047) at 12 months.
Conclusions and Relevance
Given the growing number of adolescents and young adults using employment training programs and the mental health needs of this population, increased efforts should be made to deliver mental health interventions in these settings that usually focus primarily on academic and job skills. Ways to extend the effect of intervention for females and those with lower levels of depressive symptoms should be explored.
One-fourth of US adolescents and young adults (aged 16-24 years) will experience a depressive episode by age 24—the highest incidence rate of any age group.1,2 Depression during this period has been shown to be associated with substance abuse, interpersonal problems, delinquency, academic and workplace difficulties, and suicide attempts.3- 6 Moreover, individuals experiencing a depressive episode during adolescence or young adulthood are likely to experience another episode in later adulthood.7,8 Provision of mental health services to adolescents and young adults has proved challenging, however. National studies indicate that only 25% of African American young adults (aged 18-24 years) receive needed services for depression, with this percentage falling to 19% among unemployed African American young adults.9
The number of African American adolescents and young adults neither connected to school nor the workforce has risen dramatically in recent years, with an estimated 5.8 million youth10 now falling into this category of disconnected youth.11 There is, however, a growing public and private commitment12,13 to developing employment training programs to place these adolescents and young adults on a trajectory toward educational advancement and employment. These employment training programs can also serve as sites where mental health services and supports can be provided. This focus on addressing mental health concerns of adolescents and young adults in employment training programs is particularly salient given research showing higher rates of health problems among this population than among their school-aged counterparts14 and policy briefs highlighting the prevalence of mental health concerns among this population and the challenges faced by employment training programs in addressing these concerns.15,16
This study is the first, to our knowledge, to rigorously assess the effectiveness of a mental health intervention in employment training programs for adolescents and young adults. Our primary aim was to examine the effectiveness of a multicomponent mental health intervention aimed at reducing depressive symptoms and improving engaged coping strategies integrated into an employment training program.17 Our intervention was consistent with the Institute of Medicine’s definition of a universal preventive intervention as we targeted all adolescents and young adults within the employment training program. Similar to some previous universal interventions,18,19 we also explored the effect of the intervention on the population of adolescents and young adults exhibiting higher levels of depressive symptoms at baseline. We also examined whether intervention effects varied by sex given the exploratory nature of this study. Our intervention strategies were based on cognitive-behavioral and interpersonal approaches that have been demonstrated to be effective in preventing and treating adolescent and young adult depression.20
We conducted a quasi-experimental study with 2 employment training programs in Baltimore, Maryland. Youth Opportunities (YO) is a national employment training model, with 2 sites (Eastside YO and Westside YO) using identical program models. These centers provide comprehensive social and educational services, including General Education Development certification classes, support for college enrollment, resume building, career development resources, and job placement. Initially funded by the US Department of Labor, YO centers serve communities with pervasive poverty, high unemployment rates, and general distress characterized by high dropout rates and several other negative social, health, and economic indicators.
Eastside YO program enrollees received a multicomponent intervention aimed at improving mental health status—Healthy Minds at Work, which consisted of (1) mental health training for YO program staff; (2) audio computer-assisted self-interview (ACASI) mental health screening at the time of program enrollment to determine the level of need for mental health services; (3) psychoeducational workshops (eg, anger management, coping with stress) both integrated into the employment training curriculum and conducted as freestanding sessions; and (4) on-site mental health services provided by 2 full-time licensed clinical social workers and, as needed, a psychiatrist to provide medication management. Results from the ACASI screening were shared with the on-site clinical social worker, and an initial appointment with the clinical social worker was scheduled for each individual completing an ACASI. During this initial visit, the clinical social worker reviewed ACASI results, asked clarifying and probing questions to better assess participants’ mental health needs, and developed a preliminary case plan. All YO enrollees, regardless of baseline mental health status, were eligible for psychoeducational workshops and on-site mental health services. Enrollees with moderate (Center for Epidemiologic Studies Depression Scale21 [CES-D] score of 10-26) depressive symptoms were eligible for a peer-led depression prevention group, while enrollees with moderate and high (CES-D score >26) depressive symptoms were recommended to engage in a minimum of 8 one-on-one cognitive behavioral therapy (CBT) sessions with the on-site clinical social worker. These CBT sessions lasted 45 minutes and focused on enhancing understanding of how behavior, thoughts, and emotions act in concert and providing effective ways of behaving and thinking in response to stressful situations. Intervention group engagement in Healthy Minds at Work mental health services is reported in eTable 1 in the Supplement. Westside YO enrollees received the ACASI screening and initial visit with an on-site mental health clinical social worker. However, the Westside clinical social worker worked only 20 hours per week, limiting availability for follow-up visits.
New YO enrollees aged 16 to 23 years and not in foster care were eligible for research participation. A total of 782 youth were enrolled across the comparison (n = 270) and intervention (n = 512) samples between September 1, 2008, and May 31, 2011, representing 91.0% of total YO enrollees during the recruitment period. The mean age of participants was 19 years, 93.7% were African American, 49.4% were male, and 82.9% entered the program without a General Education Development certification or high school diploma. A total of 37.4% had moderate to high depressive symptom levels (CES-D score ≥10). Given the unstable housing and highly mobile nature of our study population, intensive efforts were undertaken to minimize participant loss to follow-up, including the use of social networking sites and in-home visits. Among the 782 participants observed at baseline, 477 (61.0%) completed follow-up assessments at 6 months and 444 (56.8%) at 12 months. A missingness analysis was also conducted to assess the characteristics of those lost to follow-up (see eAppendix in the Supplement). The final sample for this intent-to-treat analysis includes 473 participants (307 intervention and 166 control participants) at 6 months and 441 participants (275 intervention and 166 control participants) at 12 months for whom no missing values existed on the matching covariates.
Baseline ACASIs were conducted at the time of program enrollment as part of both YO programs’ standard enrollment procedure. A research assistant introduced the study to new enrollees, obtained informed written consent, and set them up to complete the baseline ACASI. For participants younger than 18 years, parental written consent was also required before conducting the ACASI. Six- and 12-month follow-up ACASIs were done at a location and time most convenient for study participants; most follow-up assessments were conducted at the YO program offices. Incarcerated youth were not eligible for follow-up assessments. Individuals not completing or not eligible for the 6-month ACASI were contacted for the 12-month assessment. Participants were given $20 cash for completing the 6- and 12-month follow-ups. The Johns Hopkins University School of Medicine institutional review board approved all study procedures.
The CES-D,21 a 20-item self-report instrument widely used in depression research with adolescents and young adults,18 was used to measure depressive symptoms. The Children’s Coping Strategies Checklist–Revision 122 assessed domains of engaged coping: active coping (eg, trying to figure out why things like this happen), support seeking (eg, telling people how you feel about a problem), and distraction (eg, listening to music).
Propensity scores were used to achieve balance between the intervention and control groups. Participants were matched on the following baseline characteristics: age (16-17, 18-19, 20-23), sex, homelessness (yes or no), self-reported employment status (employed full- or part-time or not employed), highest academic grade attained, clinically significant depressive symptoms as measured via the CES-D (score ≥16, which is the cutoff for clinically significant depressive symptoms), level of financial and emotional support received from one’s mother,23 stigmatized depression24 (yes or no), moderate to severe anxiety as measured via the Beck Anxiety Inventory25 (score ≥16), and number of life stressors experienced in the past 6 months as measured via the Life Events Scale26 (above or below median), as well as interactions between these variables and the following potential effect modifiers: age, sex, employment status, and baseline depressive symptoms.
Enrollment at the intervention or comparison site served as the treatment variable. Given the exploratory nature of this study, analyses were stratified by sex. Age group and a dichotomous variable for baseline depression were also tested as interaction terms with the treatment variable. Established cutoffs distinguished those with lower (CES-D <16) and higher (CES-D ≥16) levels of depressive symptoms.21
Intervention and comparison sites differed on some pretreatment characteristics even though the neighborhoods served by the program were thought to be similarly distressed (Table 1). Covariates for the propensity score model were required to have no missingness and be associated with both treatment assignment and either of the 2 primary outcomes. Stuart27 explains that there is a low cost to including variables not associated with the treatment but a higher cost to excluding variables associated with the outcome. Therefore, variables were also included that had no association with the treatment but a theoretical association with the outcome. Garber’s28 review enumerates 11 factors associated with adolescent depression, 6 of which were available in the current data with limited missingness and included in our propensity score model: sex, anxiety, subsyndromal depression, negative cognition, stressors (ie, life events), and interpersonal relationships (ie, social support). Analyses were conducted to assess intervention effect sensitivity to unobserved confounders. The magnitude of the bias needed to alter our conclusions was such that it is unlikely that the effect we observed was due to unmeasured confounders.29
The propensity score was estimated using a multivariable logistic regression model in which the dependent variable was a binary indicator of group assignment. Covariates predicting the probability of intervention assignment, less the interaction terms, are described in Table 2. Full matching methods were used in which all participants in the data set were retained using MatchIt (R v2.15).30,31 Standardized mean bias and propensity score distribution overlap assessed performance of the matching technique. Propensity score matching was considered successful at balancing intervention and control groups when each covariate, including interaction terms, achieved a standardized bias less than 0.25.32 Full matching on propensity scores reduced the mean standardized bias by 99.1% and 98.7% at 6 and 12 months, respectively (Table 2). Distributions of propensity scores for the intervention and control groups were also appropriately similar; therefore, matching was deemed successful in improving the balance across important covariates.
Weights for matched data were exported to Stata, version 12.1 (StataCorp LP),33 and intervention effects on depressive symptoms and coping skills were estimated using sex-specific linear regression models. Covariates used in matching were entered into the final regression models for doubly robust covariate adjustments. We used a regressor approach (Ytime 2 regressed on Ytime 1 and X) to assess scores at our 2 time points due to the causal effects of the baseline depressive symptoms on depressive symptoms at 6 and 12 months.34 To assess intervention dose effects, the CES-D score change (baseline to 6 and 12 months) was regressed on a dichotomous dose variable (high vs low) using a sample of only intervention participants matched on dose. Matching with doses was accomplished using nbpmatching in R (v2.15) and successfully minimized differences between matched pairs on key baseline variables while maximizing the difference between intervention dose.35
Intervention dose ranged from 0 to 59 intervention services and is represented by the total number of psychoeducational workshops and clinical social worker sessions attended during the 1-year intervention period among intervention participants only. Males were no different than females in their levels of engagement in the intervention, with 87.5% of males and 88.8% of females using at least 1 intervention service, averaging 4.5 (males) and 4.9 (females) services among those with any engagement. Males (mean [SD], 6.18 [6.80]) and females (mean [SD], 6.20 [7.67]) with moderate to severe baseline depressive symptoms had significantly greater levels of intervention engagement than males (mean [SD], 3.67 [4.87]) and females (mean [SD], 3.93 [4.25]) with low depressive symptoms (males, P = .001; females, P = .007).
While no effect on depressive symptoms was observed at 6 months, statistically significant effects of the intervention were observed at 12 months, which were modified by sex and baseline depressive symptoms (Table 3). Males with moderate to severe depressive symptoms at baseline (CES-D score ≥16) in the intervention group showed a 5.64-point reduction (95% CI, –10.30 to –0.96; P = .02) in depressive symptoms compared with similar males in the comparison group at 12 months. Among females with moderate to severe baseline depressive symptoms, the decrease in the control group was significantly greater than in the intervention group (–10.08, 95% CI, -6.21 to –13.96; P < .001). There were no statistically significant differences between the intervention and control groups at 12 months among those with low baseline depressive symptoms, regardless of sex. Effects of geographic clustering were examined at 6 and 12 months, and no significant geographical clustering was found by zip code (within correlation, –0.001 and –0.019, respectively) or community statistical area (within correlation, –0.025 and 0.0002, respectively).36
Given the statistically significant intervention effects observed at 12 months, additional propensity score analyses determined the dose-response relationship between level of engagement in the intervention and depressive symptoms at 12 months as represented by changes in CES-D score from baseline to 12 months. Of the 444 participants observed at 12 months, 278 were in the intervention group, 264 of whom were included in the matching-by-dose analysis, resulting in 132 matched pairs. Intervention participants with low-dose intervention service use (mean number of intervention services, 2.8; range, 0-23) were matched with intervention participants with high-dose intervention service use (mean number of intervention services, 8.0; range, 1-59), with a mean dose difference of 5.8 (range, 1-33) services. Additional details of the analysis can be found in eTables 2 through 5 in the Supplement. Males with higher doses of the intervention had a greater reduction in depressive symptoms between baseline and 12 months (effect on mean change in CES-D score, –3.37; 95% CI, –6.72 to –0.09; P = .049) compared with similar males with lower doses of the intervention (Figure).
At 12 months, females in the intervention group with lower depressive symptoms at baseline demonstrated a greater improvement in the average frequency of using active coping (95% CI, 0.08-0.54; P = .008), support-seeking (95% CI, 0.02-0.49; P = .03), and overall coping (95% CI, 0.01-0.37; P = .047) strategies than did females in the control group (Table 3). Overall coping increases were also significantly greater at 12 months for females in the intervention group with lower depressive symptoms at baseline compared with females in the control group. Intervention effects were found at 12 months for males with moderate to severe depressive symptoms for all 3 engaged coping strategies—active coping (95% CI, 0.20-0.62; P < .001), distraction coping (95% CI, 0.11-0.47; P = .01), and support seeking (95% CI, 0.04-0.46; P = .02)—as well as overall use of coping strategies (95% CI, 0.14-0.50; P = .001) (Table 3).
This is the first known study, to our knowledge, to examine the effects of a mental health intervention for impoverished African American adolescents and young adults in employment training programs—a highly vulnerable population who cannot be accessed via school- or workforce-based interventions. Male adolescents and young adults receiving our intervention who entered the employment training program with greater depressive symptoms showed greater reduction in depressive symptoms and greater use of engaged coping strategies compared with male participants not receiving mental health services. These data indicating that the effect of our intervention was sex-specific and more effective for participants with higher baseline depressive symptoms are consistent with data from previous studies demonstrating greater effectiveness for targeted interventions than for universal programs, as well as sex differences.37,38 While we did not conduct mediation analyses, we believe that changes in depressive symptoms were at least partially influenced by increased use of engaged coping strategies among study participants. Contrary to some previous psychotherapy research that shows more immediate intervention benefits on the use of engaged coping skills and depressive symptoms, this study showed benefits largely at our 12-month follow-up. We believe this outcome may be due to the amount of time it took for study participants to master skills they received during the Healthy Minds at Work intervention and the cumulative effect of using these skills over time.
Although female intervention participants entering with higher depressive symptoms also exhibited a decline in depressive symptoms, the magnitude of change among females in the intervention group entering with higher symptoms was significantly smaller than in similar females in our control group. These sex-specific findings are not due to sex differences in intervention engagement or type of services received. One possible explanation for these sex differences is that our intervention’s largely CBT focus may have been better suited for males, as there has been some debate whether male and female adolescents and young adults may respond more favorably to cognitive-behavioral or interpersonal approaches, respectively.18
A unique aspect of this study was its intentional effort to deliver a mental health intervention via different modalities—via on-site clinical social workers, psychoeducational sessions, and staff mental health training—within a setting, an approach found to improve program efficacy.39 Although our dosage analyses did not specifically identify what constellation of mental health services and supports were associated with varied outcomes, it appears that greater intervention exposure resulted in greater reductions in depressive symptoms for male participants. This finding may be particularly important for future depression prevention trials conducted outside of school settings, where it may be more difficult to deliver intervention content to a large number of individuals at the same time.
Some limitations to our study exist. Our propensity score analysis adjusted only for observed covariates. Differences between intervention and control groups may still have existed due to unobserved confounders even after balance on observed covariates was achieved, although our sensitivity analysis suggests that these unobserved variables were not likely to have an effect on the study’s internal validity. Differences in the sample were observed across time, making longitudinal analyses across all 3 time points unsuitable. Challenges in sustaining intervention participants’ contact with on-site clinical social workers may have prevented us from achieving a greater intervention effect. Although embedding mental health services in an employment training program was presumed to minimize many barriers (eg, transportation, stigma) to accessing such services in other community or clinical settings, many intervention participants still had several barriers that limited their engagement in intervention activities. Most notably, participants had difficulty regularly attending scheduled clinical social worker visits or psychoeducational workshops given the varied and sometimes unanticipated demands they faced in caretaking for younger siblings and/or older family members. We anticipated that many of these barriers might interfere with regular receipt of CBT sessions delivered by on-site clinical social workers. Thus, our program model recommended 8 CBT sessions for individuals with high baseline depressive symptoms, which is a shorter duration than many other CBT-based interventions to treat adolescent depression.40 While it would have likely been challenging to engage YO members in CBT for a longer duration given the logistical issues noted above, we may have seen a larger effect on depression and coping outcomes had we recommended a greater number of CBT sessions. Finally, caution should be used in generalizing findings to other populations of adolescents and young adults who are not in school or the workforce, both within and outside employment training programs.
Given the enormous need for mental health services and interventions among youth in employment training programs and the growing number of adolescents and young adults engaged in such programs, further efforts to meet the needs of this population are highly warranted. Subsequent work should carefully consider whether a universal approach is cost effective or if resources should be focused on youth with greater depressive symptoms upon program enrollment. While this study reports solely on mental health outcomes, our research team is also examining the effect of the intervention on education, employment, and incarceration outcomes, which may aid in making decisions about universal vs indicated approaches. Future work should also determine whether current intervention approaches are insufficient or inadequately designed to improve mental health outcomes for female adolescents and young adults.
Submitted for Publication: January 20, 2014; final revision received May 9, 2014; accepted June 17, 2014.
Corresponding Author: S. Darius Tandon, PhD, Northwestern University Feinberg School of Medicine, 750 N Lake Shore Dr, 10th Floor, Chicago, IL 60611 (email@example.com).
Published Online: November 12, 2014. doi:10.1001/jamapsychiatry.2014.2022.
Author Contributions: Dr Latimore had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Tandon, Clay, Mitchell, Sonenstein.
Acquisition, analysis, or interpretation of data: Tandon, Latimore, Tucker.
Drafting of the manuscript: Tandon, Latimore, Mitchell, Sonenstein.
Critical revision of the manuscript for important intellectual content: Clay, Tucker, Sonenstein.
Statistical analysis: Tandon, Latimore.
Obtained funding: Tandon, Mitchell, Sonenstein.
Administrative, technical, or material support: Latimore, Clay, Mitchell, Tucker.
Study supervision: Tandon, Mitchell, Tucker, Sonenstein.
Conflict of Interest Disclosures: None reported.
Funding/Support: Funding for the implementation of the Healthy Minds at Work intervention came from the Robert Wood Johnson Foundation, the Jacob and Hilda Blaustein Foundation, The Abell Foundation, the Leonard and Helen R. Stulman Foundation, The Annie E. Casey Foundation, Aaron Straus and Lillie Straus Foundation, and the France-Merrick Foundation. The research portion of Healthy Minds at Work was developed as the core research project of the Johns Hopkins Center for Adolescent Health, a prevention research center funded by the Centers for Disease Control and Prevention (grant no. 1-U48-DP-000040).
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank the Eastside and Westside YO programs for their ongoing collaboration on this study, including the dedicated mental health clinicians working at the Eastside YO site. Elizabeth Stuart, PhD, provided valuable guidance on the propensity score analyses used in the manuscript. She was not compensated for her time. Trevor Arnett, MA, Meron Solomon, BA, Danielle Feldman, MSPH, Amanda Manning, BA, Irisha Burrell, BA, and Courtney Hegadorn, BA, assisted with collection of baseline and follow-up data; they were staff members who were compensated for their work on the project.
Correction: This article was corrected online January 7, 2015, to fix a reference.