Receiver operating characteristic curves for weighted and unweighted risk scores.
Levin HS, McCauley SR, Josic CP, Boake C, Brown SA, Goodman HS, Merritt SG, Brundage SI. Predicting Depression Following Mild Traumatic Brain Injury. Arch Gen Psychiatry. 2005;62(5):523-528. doi:10.1001/archpsyc.62.5.523
Minimizing negative consequences of major depression following traumatic brain injury is an important public health objective. Identifying high-risk patients and referring them for treatment could reduce morbidity and loss of productivity.
To develop a model for early screening of patients at risk for major depressive episode at 3 months after traumatic brain injury.
Prediction model using receiver operating characteristic curve.
Level I trauma center in a major metropolitan area.
Prospective cohort of 129 adults with mild traumatic brain injury.
Main Outcome Measures
Center for Epidemiologic Studies Depression Scale score and current major depressive episode module of the Structured Clinical Interview for the DSM-IV.
A prediction model including higher 1-week Center for Epidemiologic Studies Depression Scale score, older age, and computed tomographic scans of intracranial lesions yielded 93% sensitivity and 62% specificity.
This study supports the feasibility of identifying patients with mild traumatic brain injury who are at high risk for developing major depressive episode by 3 months’ postinjury, which could facilitate selective referral for potential treatment and reduction of negative outcomes.
Mild traumatic brain injury (MTBI) accounts for 1 million hospital emergency center (EC) visits and 230 000 hospitalizations annually in the United States.1- 4 Despite a high rate of survival (100%) following MTBI,5 considerable morbidity has been documented during at least the first 3 months’ postinjury5- 8 with an estimated 18% of patients with MTBI developing a psychiatric illness by 1 year postinjury.9 Deb et al10 found that depressive illness was the most frequent psychiatric diagnosis at 1 year post-MTBI and was present in 13.9% of patients compared with 2.1% to 9.4%11- 15 of the general population. Major depressive episode (MDE) was found to develop within 3 months’ postinjury in 17% of patients with MTBI16 as compared with 33% within the first year after injury in a group with heterogeneous traumatic brain injury (TBI) severity.17 Mild TBI has also been found to be a risk factor for MDE in long-term follow-up studies18,19 and generally increased levels of depressive symptoms.20,21 Koponen et al19 evaluated the frequency of Axis I and Axis II disorders in a retrospective, 30-year follow-up study of 60 patients who had been treated for TBI. The rate of MDE (26.7%), which was the most common novel disorder after TBI, did not vary with the severity of acute injury. In a community study with a 50-year follow-up, the lifetime risk of MDE was higher after sustaining MTBI than hospitalization for extracranial injury or illness (odds ratio, 1.49 [95% confidence interval, 0.96-2.31]).18 Major depressive episode following MTBI is associated with increased levels of anxiety disorder, cognitive deficit, and disability relative to patients with MTBI who do not develop depression.16,17 Jorge et al17 found that of their patients with TBI who became depressed, 80% of them were diagnosed with MDE during the first 3 months’ postinjury, and the mean duration of untreated MDE was 5.8 months. This suggests that early identification of patients at risk for MDE could facilitate referral for treatment before symptoms progress.21 In view of the high incidence of MTBI and increased morbidity of those patients who develop MDE, triage of patients with MTBI at high risk for MDE is more feasible than follow-up of all cases, especially in view of the attrition associated with longitudinal study of this population.
Consecutive patients with head trauma were prospectively recruited (August 2001-April 2003) from Ben Taub General Hospital, a level I trauma center that is affiliated with Baylor College of Medicine and the Harris County Hospital District in Houston, Tex. The diagnosis of head injury was made by EC physicians. Criteria for enrollment included: (1) arrival to the hospital within 24 hours of injury; (2) blood alcohol level less than 200 mg/dL; (3) age 16 years and older; (4) fluent in English or Spanish; (5) residing in the hospital catchment area (Harris County, Texas); (6) not an undocumented alien, incarcerated, homeless, or on active military service; (7) no spinal cord injury; (8) no previous TBI requiring hospitalization; (9) no history of substance dependence, mental retardation, major psychiatric disorders, or other central nervous system disturbance (including patients who were clinically depressed at the time of their injury); and (10) no other preexisting condition preventing standard administration and interpretation of the outcome measures (eg, deafness, blindness). Head trauma−specific criteria included: (1) diagnosis of nonpenetrating TBI; (2) lowest postresuscitation Glasgow Coma Scale score22 higher than 8; (3) computed tomographic (CT) scan of the brain performed within 24 hours of arrival, and (4) no surgery under general anesthesia during the hospital admission. Mild TBI was defined as a closed head injury producing a Glasgow Coma Scale score of 13 to 15 on arrival to the hospital with no later deterioration below a Glasgow Coma Scale score of 13. A complete medical history (including psychiatric history and previous substance abuse) and physical examination were performed by the EC physicians at the time of admission to the EC. The study’s research assistants reviewed patients’ medical records to evaluate their eligibility for study inclusion. Participants with blood alcohol levels greater than 200 mg/dL or positive illicit drug screen results on EC admission were excluded. Patients receiving antidepressant therapy at the time of their head injury were also excluded. Once enrolled, all participants were screened at 1 week postinjury with the Alcohol Use Disorders Identification Test (AUDIT)23 and the Drug Abuse Screening Test short form (DAST-10).24,25 If participants reported an AUDIT level of 8 or higher and/or a DAST-10 score of 3 or higher, they were dropped from further participation.
Of the 239 patients with TBI who met these criteria and initially consented to participate, 129 (54.0%) were followed up at both 1 week and 3 months’ postinjury. Table 1 presents sample demographics and injury characteristics of the groups at the time of injury. Attrition biases were investigated by independent sample t tests for continuous variables and χ2 or Fisher exact tests for categorical variables at the 95% confidence level. Compared with patients who were successfully followed up at 3 months’ postinjury, those not followed up included a higher proportion of men and semiskilled to unskilled workers. Follow-up was unrelated to other demographic or injury variables.
To assess the severity of depressive symptoms, the self-report Center for Epidemiologic Studies Depression Scale27 (CES-D) was used. The CES-D is a 20-item self-report questionnaire rating depressive symptom frequency on a 4-point Likert scale from “none of the time” to “all of the time” with a score range of 0 to 60 (higher total scores indicating greater symptom frequency). The CES-D has been shown to have excellent internal reliability with total score α coefficients of .85 to .86 in general population samples,27- 29 .85 in ethnically diverse samples,28 and .90 to .91 in patient samples.27,29 Split-half reliability estimates have ranged from .76 to .85.27 Test-retest reliability estimates have been found to be lower (.32-.67), likely owing to the scale’s concentration on recent symptoms that are subject to change over long retest intervals.27 The validity of the CES-D has been demonstrated with moderate to high correlations with the Hamilton Depression Rating Scale (.49-.85)30 and the Depression Scale of the Symptom Checklist-90 (.72-.87).30 The CES-D was administered in an interview format. The CES-D total score was used in the analyses.
The DSM-IV31 criteria were used to define major depression. The diagnosis of an MDE was determined through the administration of the current MDE module of the Structured Clinical Interview for the DSM-IV32 (SCID). The SCID is an interview administered by health care professionals that includes an introductory overview followed by self-contained modules for each of the Axis I disorders. Using a decision-tree approach, the SCID guides the interviewer in testing diagnostic hypotheses as the interview is conducted. The SCID either ends when the patient meets the criteria for the disorder or sooner when required symptoms and/or thresholds are not met. The dependent measure was the presence or absence of MDE as determined by the SCID.
Computed tomographic scans of the brain were obtained within 24 hours of the injury and coded26 by a neuroradiologist blinded to the patient’s diagnostic status regarding mood disorder. Brain lesions were systematically coded for lesion type and location.
Patients were recruited in the EC or during acute hospitalization. Patients treated in the EC (but not admitted to the hospital) were recruited by study personnel according to a rotating schedule representing all shifts and days of the week. Patients admitted to the hospital were identified from the EC log or hospital census reports. Diagnosis of TBI was made by EC trauma physicians, and Glasgow Coma Scale ratings were made by EC trauma staff. Injury Severity Score ratings (derived from the Abbreviated Injury Scale33) were made by an Abbreviated Injury Scale–certified research nurse based on medical record review. The Injury Severity Score ratings did not include scores from the head region.
As part of a larger project, patients were administered a short battery of screening tests at 1 week (± 2 days) postinjury including the CES-D. Follow-up evaluations at 3 months’ (± 2 weeks) postinjury were face-to-face interviews conducted by a bachelor-level or master-level research assistant in the patient’s primary language, either English or Spanish. The battery of psychological and neuropsychological tests administered at this end point included the current MDE module of the SCID. The project psychiatrists (H.S.G. and S.G.M.) were provided with audiotape recordings and paper copies of the SCIDs for quality control and verification of the diagnosis of MDE. There was 100% agreement between the research assistants and the psychiatrists regarding diagnosis of MDE based on the patients’ responses to the SCID. Patients were reimbursed for their participation in the follow-up evaluation. Assessments were performed in the patient’s home when necessary. Informed consent was obtained from all study patients. The study protocol was approved by the institutional review boards of the participating institutions.
All data analyses were performed with SAS for Windows,34 version 8.2. Statistical significance was defined as P<.05 for all analyses. Each test score was assigned a corresponding reliability code. Only scores with codes indicating standard administration or minor deviations from standard were included in the analyses; all patients followed up at 1 week and 3 months’ postinjury had reliability codes indicating standard test administration, and no data points were deleted because of a designation of unreliability.
Logistic regression was used to generate a prediction model of MDE at 3 months’ postinjury using the CES-D score measured at 1 week postinjury as an independent variable. Major depressive episode was present in 15 subjects (11.6%) of a sample of 129 at 3 months’ postinjury. A logistic regression model was fitted to the MDE outcome variable with the following predictor variables: CES-D score at 1 week, sex,12,35 age at injury,36 years of education,37 primary language, race/ethnicity,38 marital status, CT result,17,39 and occupation level.40 A multivariate logistic regression was first fitted with all the predictor variables. Two-way interactions were also entered into the model. Backward elimination was then used to identify the significant predictors using the Wald χ2 statistic to test for significance at the P<.05 level. The final model for predicting MDE contained linear effects for age at injury, CT result, and the 1-week CES-D score. With every year increase in age, the odds of MDE at 3 months was multiplied by 1.05; similarly, for the CES-D score, for every unit increase the odds of MDE at 3 months was multiplied by 1.11 (Table 2). The odds of MDE at 3 months with an abnormal CT result were 7.68 times the odds of MDE with a normal CT result. The area under the receiver operating characteristic (ROC) curve for this model was 0.86, indicating a strong model for predicting MDE at 3 months’ postinjury.
Risk scores were developed for each subject. Each independent predictor was assigned a weight equal to the logistic regression parameter multiplied by 10. The risk score was estimated by summing the weighted risk for each predictor: risk score = 10 × (0.047 × age) + 10 × (0.1 × CES-D score) + 10 × (1.02 × CT result [1 if positive, −1 if negative]). The unweighted risk scores were also computed by multiplying each of the 3 risk factors (age, CES-D score, and CT result) by 10 and then summing the scores.
To determine the accuracy of both the weighted and unweighted risk scores, ROC curves were plotted and the area under the curves estimated. Sensitivity and specificity were calculated at all possible cutoff points for each risk score. Because an MDE has been shown to have significant adverse effects (impaired cognitive and emotional functioning, delayed return to work), it is important to minimize the number of patients who have false-negative results. To achieve this, sensitivity was set high (increasing the number of false-positive results), which resulted in decreased specificity (decreasing the number of false-negative results). Using this rationale, sensitivity was maximized while setting the specificity at no lower than 60%, which assures very high negative predictive power (the probability that a patient who has negative test results will be free from the condition of interest). Because the incidence of MDE is relatively low (estimated between 2.1% and 9.4% in the general population11- 15 and 17% in those with MTBI16), test sensitivity must be kept quite high so that no, or very few, patients are considered to have false-negative test results.
To validate the risk scores and optimal cutoff points, bootstrap was used with 1000 data sets derived from the original data with replacement. Weighted and unweighted risk scores were estimated for each data set. Sensitivity, specificity, areas under the ROC curve, and the optimal cutoff point for both risk scores were also calculated for each data set.
For the original data, weighted risk scores ranged from −0.26 to 85.24 and unweighted scores ranged from 20 to 126. The areas under the ROC curves for the weighted and unweighted risk scores (0.86 ± 0.06 and 0.84 ± 0.06, respectively) did not differ significantly (Figure). Applying a decision rule requiring that specificity meet or exceed 60%, the cutoff score for the weighted risk score was set at 31 or higher (Table 3), which resulted in a sensitivity of 93.3% and 62.3% specificity. The unweighted cutoff risk score was set at 55 or higher, resulting in a sensitivity of 86.7% and 62.3% specificity. Sensitivity and specificity are also presented for other possible cutoff scores (Table 3).
From the bootstrap samples, 95% confidence intervals for the cutoff points were calculated. For the weighted risk score, this interval was 17 to 62, and for the unweighted risk score, the interval was 52 to 60. As would be expected, the weighted risk score bootstrap samples have greater variability than the unweighted scores owing to uncertainty in the estimates of the logistic regression coefficients.
To our knowledge, this is the first prospective study including a prediction model for MDE following MTBI. The CES-D used in the present study is practical for administration by telephone or in person because it can be given in 5 minutes or less and is easily scored. Previous research established the reliability and validity of this scale for evaluating depression in community-based27,29,30,41- 44 and cross-cultural studies.28,45,46 Although replication of our findings is advisable before extrapolating our findings to other trauma centers, our study was based on prospective, consecutive admissions to the EC and neurosurgery inpatient service of the level I trauma center operated by the Harris County Hospital District. By sampling from EC admissions at various times and days of the week, we attempted to obtain a representative sample of patients with MTBI. Use of research technicians bilingual in English and Spanish also facilitated recruitment and follow-up of patients. Our findings thus confirm results of previous work by our group16 and other investigators6,10,16 that MDE is a frequent secondary condition following MTBI. Our results add to the literature by identifying older age and intracranial lesions visible on CT scans as potential risk factors for depression after MTBI. Older age has not generally emerged as a predictor of MDE following MTBI20 or in patient samples that were heterogeneous in severity of injury.17 Inclusion of patients with preinjury psychiatric disorder or substance abuse might have obscured the increased risk associated with older age in previous studies.17 Our finding that abnormalities on CT scan increased the risk of depression may be especially significant in light of the worse cognitive and functional outcomes reported when MTBI is complicated by intracranial lesions visible on CT scan.47 A sample that was heterogeneous in acute severity of TBI39 found that left anterior lesions on CT scan were related to the development of MDE. Although the recent study of a mixed-severity TBI sample by Jorge et al17 could not confirm a relation of acute CT findings to MDE, magnetic resonance images at 3 months’ postinjury showed that the volume of gray matter in left frontal subregions was reduced in patients with MDE relative to patients without depression. Abnormal CT findings in our patients with MTBI were confined to small lesions in less than 11% of the sample; of the 15 patients with MDE, 4 had lesions (all in the right hemisphere). Pending replication, an implication of our results for clinical management of MTBI is that a high 1-week CES-D total score, older age, and a brain lesion seen on CT scan are risk factors for MDE onset during the first 3 months postinjury.
A potential limitation is that by excluding patients with a history of substance dependence or major psychiatric disorder, our study sample might have been less representative of the general MTBI population. Although preinjury substance abuse has been reported to be related to postinjury onset of MDE,48 it has not been confirmed as a risk factor in more recent studies.17 Moreover, inclusion of patients with intoxication, preexisting behavioral problems, or baseline functional disability could have complicated the diagnosis of TBI49 and obscured the impact of trauma on outcome.
The present study supports the feasibility of using a brief screening measure for depression to identify patients with MTBI at high risk for MDE within 1 week after injury, a possibility that would facilitate selective referral for treatment. Although other screening measures, such as the Beck Depression Inventory and the Zung Self-Rating Depression Scale, have been shown to be sensitive to depressive symptoms following TBI20,21 and the Hospital Anxiety and Depression Scale given at 7 to 10 days after mild to moderate TBI has been shown to predict postconcussional symptoms at 3 months,50 our study is the first, to our knowledge, to demonstrate that administering a screening measure of depression at 1 week post-MTBI is useful in predicting MDE. With increased risk for MDE during the first 3 months and throughout the first year postinjury,17,48 early detection of a high-risk subgroup could facilitate treatment and mitigation of this secondary condition. With an incidence of 392 of 100 000 population,51,52 the cost of following up all patients with MTBI is prohibitive for most health care networks.52 At the same time, the indirect costs of TBI, such as time lost from work, are considerable17,53 and generally exceed the direct costs of acute medical treatment.54
Our findings raise the possibility that coordination of outpatient psychiatric services with trauma centers could improve the outcome of MTBI by mitigating secondary conditions. Pending replication of our findings, clinical trials to evaluate the efficacy of early intervention for high-risk patients could determine whether this strategy is effective in reducing and/or averting MDE following MTBI.
Correspondence: Harvey S. Levin, PhD, Cognitive Neuroscience Laboratory, Baylor College of Medicine, 6560 Fannin St, Suite 1144, Houston, TX 77030 (email@example.com).
Submitted for Publication: April 15, 2004; final revision received September 24, 2004; accepted September 24, 2004.
Funding/Support: This study was supported by grant R49/CCR612707 (“Community Based Case Management of Mild to Moderate TBI,” Dr Levin [principal investigator]) from the Centers for Disease Control and Prevention, Atlanta, Ga.