Comparison of Simulated Treatment and Cost-effectiveness of a Stepped Care Case-Finding Intervention vs Usual Care for Posttraumatic Stress Disorder After a Natural Disaster | Psychiatry | JAMA Psychiatry | JAMA Network
[Skip to Navigation]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Navigation Landing]
Figure 1.  Model Schematics
Model Schematics

A, Agent-based model overview. B, Stepped care treatment flowchart. CBT indicates cognitive behavioral therapy; PTSD, posttraumatic stress disorder; and SPR, Skills for Psychological Recovery.

Figure 2.  Map of Hurricane-Affected Areas of New York City
Map of Hurricane-Affected Areas of New York City

Affected areas consist of areas that were fully or partially inundated by flooding. Derived from Federal Emergency Management Agency Modeling Task Force data.

Figure 3.  Treatment Effectiveness of Stepped Care Compared With Usual Care
Treatment Effectiveness of Stepped Care Compared With Usual Care

A, Prevalence of posttraumatic stress disorder (PTSD) in the full population. B, Prevalence of PTSD among cases. Sensitivity = 0.80 and specificity = 0.80 (base case).

Table 1.  Comparison of Model Pseudo-Population With New York City ACS Estimates
Comparison of Model Pseudo-Population With New York City ACS Estimates
Table 2.  Cost-effectiveness Under Base Casea With Discounting Rate of 3%
Cost-effectiveness Under Base Casea With Discounting Rate of 3%
1.
Redlener  I, Reilly  MJ.  Lessons from Sandy—preparing health systems for future disasters.  N Engl J Med. 2012;367(24):2269-2271. PubMedGoogle ScholarCrossref
2.
Recovery  NYC. Community development block grant disaster recovery: the City of New York action plan incorporating amendments 1-11. http://www.nyc.gov/html/cdbg/downloads/pdf/cdbg-dr_action_plan_incorporating_amendments_1-11.pdf. Published February 1, 2016. Accessed August 23, 2017.
3.
National Oceanic and Atmospheric Administration, US Dept of Commerce. Service assessment: Hurricane/post-Tropical Cyclone Sandy, October 22-29, 2012. https://www.weather.gov/media/publications/assessments/Sandy13.pdf. Published May 2013. Accessed August 23, 2017.
4.
Centers for Disease Control and Prevention (CDC).  Deaths associated with Hurricane Sandy—October-November 2012.  MMWR Morb Mortal Wkly Rep. 2013;62(20):393-397.PubMedGoogle Scholar
5.
Brackbill  RM, Caramanica  K, Maliniak  M,  et al.  Nonfatal injuries 1 week after Hurricane Sandy—New York City metropolitan area, October 2012.  MMWR Morb Mortal Wkly Rep. 2014;63(42):950-954.PubMedGoogle Scholar
6.
Goldstein  W, Peterson  A, Zarrilli  DA. One city, rebuilding together: a report on the City of New York’s response to Hurricane Sandy and the path forward. http://www1.nyc.gov/assets/home/downloads/pdf/reports/2014/sandy_041714.pdf. Published April 2014. Accessed August 23, 2017.
7.
Goldmann  E, Galea  S.  Mental health consequences of disasters.  Annu Rev Public Health. 2014;35(1):169-183. PubMedGoogle ScholarCrossref
8.
Schreiber  MD, Yin  R, Omaish  M, Broderick  JE.  Snapshot from Superstorm Sandy: American Red Cross mental health risk surveillance in lower New York State.  Ann Emerg Med. 2014;64(1):59-65. PubMedGoogle ScholarCrossref
9.
 Crisis Counseling Assistance and Training Program Regular Services Program Final Report. Albany, NY: New York State Office of Mental Health; 2014.
10.
Berkowitz  S, Bryant  R, Brymer  M,  et al. Skills for Psychological Recovery: field operations guide, 2010. https://www.ptsd.va.gov/professional/materials/manuals/skills_psych_recovery_manual.asp. Accessed August 23, 2017.
11.
Lowe  SR, Sampson  L, Gruebner  O, Galea  S.  Mental health service need and use in the aftermath of Hurricane Sandy: findings in a population-based sample of New York City residents.  Community Ment Health J. 2016;52(1):25-31. PubMedGoogle ScholarCrossref
12.
Wang  PS, Gruber  MJ, Powers  RE,  et al.  Mental health service use among Hurricane Katrina survivors in the eight months after the disaster.  Psychiatr Serv. 2007;58(11):1403-1411. PubMedGoogle ScholarCrossref
13.
Ghuman  SJ, Brackbill  RM, Stellman  SD, Farfel  MR, Cone  JE.  Unmet mental health care need 10-11 years after the 9/11 terrorist attacks: 2011-2012 results from the World Trade Center Health Registry.  BMC Public Health. 2014;14(1):491. PubMedGoogle ScholarCrossref
14.
US Dept of Veterans Affairs. Psychological first aid: field operations guide. https://www.ptsd.va.gov/professional/materials/manuals/psych-first-aid.asp. Accessed August 23, 2017.
15.
Roy-Byrne  P, Craske  MG, Sullivan  G,  et al.  Delivery of evidence-based treatment for multiple anxiety disorders in primary care: a randomized controlled trial.  JAMA. 2010;303(19):1921-1928. PubMedGoogle ScholarCrossref
16.
Katon  W, Von Korff  M, Lin  E,  et al.  Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial.  Arch Gen Psychiatry. 1999;56(12):1109-1115. PubMedGoogle ScholarCrossref
17.
Zatzick  D, Jurkovich  G, Rivara  FP,  et al.  A randomized stepped care intervention trial targeting posttraumatic stress disorder for surgically hospitalized injury survivors.  Ann Surg. 2013;257(3):390-399. PubMedGoogle ScholarCrossref
18.
North  CS, Pfefferbaum  B.  Mental health response to community disasters: a systematic review.  JAMA. 2013;310(5):507-518. PubMedGoogle ScholarCrossref
19.
Glasgow  RE, Vogt  TM, Boles  SM.  Evaluating the public health impact of health promotion interventions: the RE-AIM framework.  Am J Public Health. 1999;89(9):1322-1327.PubMedGoogle ScholarCrossref
20.
Craske  MG, Roy-Byrne  PP, Stein  MB, Sullivan  G, Sherbourne  C, Bystritsky  A.  Treatment for anxiety disorders: efficacy to effectiveness to implementation.  Behav Res Ther. 2009;47(11):931-937. PubMedGoogle ScholarCrossref
21.
Zatzick  DF, Koepsell  T, Rivara  FP.  Using target population specification, effect size, and reach to estimate and compare the population impact of two PTSD preventive interventions.  Psychiatry. 2009;72(4):346-359. PubMedGoogle ScholarCrossref
22.
Koepsell  TD, Zatzick  DF, Rivara  FP.  Estimating the population impact of preventive interventions from randomized trials.  Am J Prev Med. 2011;40(2):191-198. PubMedGoogle ScholarCrossref
23.
Marshall  BDL, Galea  S.  Formalizing the role of agent-based modeling in causal inference and epidemiology.  Am J Epidemiol. 2015;181(2):92-99. PubMedGoogle ScholarCrossref
24.
US Census Bureau. American community survey. https://www.census.gov/programs-surveys/acs/. Accessed August 23, 2017.
25.
Infoshare. Online/Community Studies of New York, Inc. Area Comparison Table. http://www.infoshare.org/main/public.aspx. Accessed January 1, 2016.
26.
Lowe  SR, Sampson  L, Gruebner  O, Galea  S.  Psychological resilience after Hurricane Sandy: the influence of individual- and community-level factors on mental health after a large-scale natural disaster.  PLoS One. 2015;10(5):e0125761.PubMedGoogle ScholarCrossref
27.
Blanchard  EB, Jones-Alexander  J, Buckley  TC, Forneris  CA.  Psychometric properties of the PTSD checklist (PCL).  Behav Res Ther. 1996;34(8):669-673. PubMedGoogle ScholarCrossref
28.
Bradley  R, Greene  J, Russ  E, Dutra  L, Westen  D.  A multidimensional meta-analysis of psychotherapy for PTSD.  Am J Psychiatry. 2005;162(2):214-227. PubMedGoogle ScholarCrossref
29.
Liu  H, Petukhova  MV, Sampson  NA,  et al; World Health Organization World Mental Health Survey Collaborators.  Association of DSM-IV posttraumatic stress disorder with traumatic experience type and history in the World Health Organization World Mental Health Surveys.  JAMA Psychiatry. 2017;74(3):270-281. PubMedGoogle ScholarCrossref
30.
Steinert  C, Hofmann  M, Leichsenring  F, Kruse  J.  The course of PTSD in naturalistic long-term studies: high variability of outcomes: a systematic review.  Nord J Psychiatry. 2016;69(7):483-496. PubMedGoogle ScholarCrossref
31.
Norris  FH, Tracy  M, Galea  S.  Looking for resilience: understanding the longitudinal trajectories of responses to stress.  Soc Sci Med. 2009;68(12):2190-2198. PubMedGoogle ScholarCrossref
32.
Foa  E, Keane  TM, Friedman  MJ, Cohen  JA, eds.  Effective Treatments for PTSD: Practice Guidelines from the International Society for Traumatic Stress Studies. 2nd ed. New York, NY: The Guilford Press; 2009.
33.
Shalev  AY, Ankri  Y, Israeli-Shalev  Y, Peleg  T, Adessky  R, Freedman  S.  Prevention of posttraumatic stress disorder by early treatment: results from the Jerusalem Trauma Outreach and Prevention study.  Arch Gen Psychiatry. 2012;69(2):166-176. PubMedGoogle ScholarCrossref
34.
Galea  S, Ahern  J, Tracy  M,  et al.  Longitudinal determinants of posttraumatic stress in a population-based cohort study.  Epidemiology. 2008;19(1):47-54. PubMedGoogle ScholarCrossref
35.
Acierno  R. Home-based telehealth vs. in-person delivery of evidence based psychotherapy for combat-related PTSD. Presented at: Development Field-Based Meeting on Rural Health; September 2016; Washington, DC.
36.
Keyes  KM, Galea  S.  Population Health Science. New York, NY: Oxford University Press; 2016.Crossref
37.
Whiteford  HA, Degenhardt  L, Rehm  J,  et al.  Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010.  Lancet. 2013;382(9904):1575-1586. PubMedGoogle ScholarCrossref
38.
Grimm  V, Berger  U, DeAngelis  DL, Polhill  JG, Giske  J, Railsback  SF.  The ODD protocol: a review and first update.  Ecol Modell. 2010;221(23):2760-2768. doi:10.1016/j.ecolmodel.2010.08.019Google ScholarCrossref
39.
Grosse  SD.  Assessing cost-effectiveness in healthcare: history of the $50,000 per QALY threshold.  Expert Rev Pharmacoecon Outcomes Res. 2008;8(2):165-178. PubMedGoogle ScholarCrossref
40.
Jaycox  LH, Cohen  JA, Mannarino  AP,  et al.  Children’s mental health care following Hurricane Katrina: a field trial of trauma-focused psychotherapies.  J Trauma Stress. 2010;23(2):223-231.PubMedGoogle Scholar
41.
Trusz  SG, Wagner  AW, Russo  J, Love  J, Zatzick  DF.  Assessing barriers to care and readiness for cognitive behavioral therapy in early acute care PTSD interventions.  Psychiatry. 2011;74(3):207-223. PubMedGoogle ScholarCrossref
42.
Darnell  D, O’Connor  S, Wagner  A,  et al.  Enhancing the reach of cognitive-behavioral therapy targeting posttraumatic stress in acute care medical settings.  Psychiatr Serv. 2017;68(3):258-263. PubMedGoogle ScholarCrossref
43.
Joesch  JM, Sherbourne  CD, Sullivan  G, Stein  MB, Craske  MG, Roy-Byrne  P.  Incremental benefits and cost of coordinated anxiety learning and management for anxiety treatment in primary care.  Psychol Med. 2012;42(9):1937-1948.PubMedGoogle ScholarCrossref
44.
Katon  W, Russo  J, Sherbourne  C,  et al.  Incremental cost-effectiveness of a collaborative care intervention for panic disorder.  Psychol Med. 2006;36(3):353-363. PubMedGoogle ScholarCrossref
45.
Ophuis  RH, Lokkerbol  J, Heemskerk  SCM, van Balkom  AJLM, Hiligsmann  M, Evers  SMAA.  Cost-effectiveness of interventions for treating anxiety disorders: a systematic review.  J Affect Disord. 2017;210:1-13. PubMedGoogle ScholarCrossref
Original Investigation
December 2017

Comparison of Simulated Treatment and Cost-effectiveness of a Stepped Care Case-Finding Intervention vs Usual Care for Posttraumatic Stress Disorder After a Natural Disaster

Author Affiliations
  • 1Department of Epidemiology, Boston University School of Public Health, Boston, Massachusetts
  • 2Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York
  • 3Department of Psychology, Montclair State University, Montclair, New Jersey
  • 4Section of General Internal Medicine, Boston University School of Medicine, Boston, Massachusetts
  • 5College of Nursing, Medical University of South Carolina, Charleston
JAMA Psychiatry. 2017;74(12):1251-1258. doi:10.1001/jamapsychiatry.2017.3037
Key Points

Question  What is the reach, treatment effectiveness, and cost-effectiveness of a stepped care case-finding intervention for treatment of posttraumatic stress disorder, compared with usual care, after a natural disaster?

Findings  This simulation study of areas of New York City affected by Hurricane Sandy found that stepped care was associated with greater reach, was a more effective treatment, and had a favorable cost-effectiveness profile compared with usual care.

Meaning  Further testing in the field and consideration for implementation are warranted.

Abstract

Importance  Psychiatric interventions offered after natural disasters commonly address subsyndromal symptom presentations, but often remain insufficient to reduce the burden of chronic posttraumatic stress disorder (PTSD).

Objective  To simulate a comparison of a stepped care case-finding intervention (stepped care [SC]) vs a moderate-strength single-level intervention (usual care [UC]) on treatment effectiveness and incremental cost-effectiveness in the 2 years after a natural disaster.

Design, Setting, and Participants  This study, which simulated treatment scenarios that start 4 weeks after landfall of Hurricane Sandy on October 29, 2012, and ending 2 years later, created a model of 2 642 713 simulated agents living in the areas of New York City affected by Hurricane Sandy.

Interventions  Under SC, cases were referred to cognitive behavioral therapy, an evidence-based therapy that aims to improve symptoms through problem solving and by changing thoughts and behaviors; noncases were referred to Skills for Psychological Recovery, an evidence-informed therapy that aims to reduce distress and improve coping and functioning. Under UC, all patients were referred only to Skills for Psychological Recovery.

Main Outcomes and Measures  The reach of SC compared with UC for 2 years, the 2-year reduction in prevalence of PTSD among the full population, the 2-year reduction in the proportion of PTSD cases among initial cases, and 10-year incremental cost-effectiveness.

Results  This population of 2 642 713 simulated agents was initialized with a PTSD prevalence of 4.38% (115 751 cases) and distributions of sex (52.6% female and 47.4% male) and age (33.9% aged 18-34 years, 49.0% aged 35-64 years, and 17.1% aged ≥65 years) that were comparable with population estimates in the areas of New York City affected by Hurricane Sandy. Stepped care was associated with greater reach and was superior to UC in reducing the prevalence of PTSD in the full population: absolute benefit was clear at 6 months (risk difference [RD], –0.004; 95% CI, –0.004 to –0.004), improving through 1.25 years (RD, –0.015; 95% CI, –0.015 to –0.014). Relative benefits of SC were clear at 6 months (risk ratio, 0.905; 95% CI, 0.898-0.913), with continued gains through 1.75 years (risk ratio, 0.615; 95% CI, 0.609-0.662). The absolute benefit of SC among cases was much stronger, emerging at 3 months (RD, –0.006; 95% CI, –0.007 to –0.005) and increasing through 1.5 years (RD, –0.338; 95% CI, –0.342 to –0.335). Relative benefits of SC among cases were equivalent to those observed in the full population. The incremental cost-effectiveness of SC compared with UC was $3428.71 to $6857.68 per disability-adjusted life year avoided, and $0.80 to $1.61 per PTSD-free day.

Conclusions and Relevance  The results of this simulation study suggest that SC for individuals with PTSD in the aftermath of a natural disaster is associated with greater reach than UC, more effectiveness than UC, and is well within the range of acceptability for cost-effectiveness. Results should be considered in light of limitations inherent to agent-based models.

Introduction

Hurricane Sandy made landfall in New York City on October 29, 2012, resulting in disruptions of transportation, evacuation, destruction, and damage to homes, businesses, and several hospitals.1-3 In the New York City area, 44 people died4 and injuries were common among evacuees.5 Disruptions in day-to-day life were another important consequence of the storm6 that may have reduced the pace of normalization and recovery,7 while limiting access to health care services.

After the storm, surveillance efforts suggested a high risk for poor mental health outcomes in the New York City boroughs.8 In the aftermath of Hurricane Sandy, the New York State Office of Mental Health provided crisis counseling services to an estimated 250 000 individuals between mid-November 2012 and late March 2014, including group and individual crisis counseling, and group education sessions.9 This effort used a treatment modality called Skills for Psychological Recovery (SPR), a skills-building intervention aimed at reducing stress and improving coping among survivors.10

Reflecting the inadequacy of available mental health care services, the estimated prevalence of probable cases of major depression was 8.9% and of hurricane-related PTSD was 2% among areas of New York City heavily affected by Hurricane Sandy approximately 1 to 1.5 years after the storm.11 The same study found that more than half of those with a perceived mental health care service need did not receive such services. Similar patterns of high unmet need have been observed across a variety of disasters including Hurricane Katrina12 and the September 11, 2001, World Trade Center terrorist attacks.13

One potential reason for the persistence of mental health conditions, including PTSD, after disasters could be the lack of effective triage to appropriate levels of care. Although mental health care delivery efforts in the wake of disaster exposures often include psychological first aid—a universal intervention14—and SPR, a moderate-strength intervention targeting distressed individuals,10 evidence from stepped care (SC) case-finding models suggests that the most efficient and effective population approach may be screening and triage to the appropriate level of care followed by ongoing systematic reevaluation.15-17 This approach has been explicated and recommended in the context of disasters,18 but has not, to our knowledge, been tested in the field. Further investigation of SC models in the treatment of PTSD after a natural disaster is therefore warranted.

Beyond efficient screening and triage, reach, defined as an individual-level measure of participation,19 is another key feature of successful population-level interventions. Limited reach remains a major critique of specialty mental health care approaches to treatment of anxiety disorders, such as cognitive behavioral therapy (CBT).20,21 However, SC approaches, which target a broader population base, may provide greater reach than either specialty or nonspecialty mental health care approaches alone. Accordingly, in assessing the potential outcome of SC, we are informed by the population impact framework, which defines population impact as a product of the proportion of intervention candidates, proportion of candidates who received the intervention (ie, reach), and reduction in disease incidence (or prevalence in nonprimary prevention contexts) among those who received the intervention.22

To our knowledge, there are no real-world trials of SC interventions in disaster contexts, and such studies would be challenging to implement on a large scale. Accordingly, we approached this problem with the use of agent-based models to simulate in silico the effect of SC compared with usual care (UC) on hurricane-related PTSD. Agent-based models can, given appropriate assumptions, provide a useful counterfactual framework for the estimation of treatment effects.23 Therefore, we aimed to estimate the effect of an SC intervention for PTSD after Hurricane Sandy using an agent-based model. We focused our model on the following 4 questions: What is the reach of SC compared with UC? What is the effect of SC compared with UC on the prevalence of PTSD in the full population during a 2-year period? What is the effect of SC compared with UC on the proportion of PTSD cases remaining during a 2-year period? What is the incremental cost-effectiveness of SC compared with UC during a 10-year period?

Methods
Model Structure

This modelling study, which simulated treatment scenarios that start 4 weeks after landfall of Hurricane Sandy on October 29, 2012, and ending 2 years later, focuses on individuals living in hurricane-affected areas of New York City at the time of Hurricane Sandy, and simulates the effect of recovery from PTSD, comparing UC and SC treatment models during 2 years of follow-up. Individuals in the simulation have a PTSD symptom score, reflecting a mix of borough-level demographic composition and hurricane exposure. Posttraumatic stress disorder symptom score can be collapsed into positive or negative case status, which is used to generate effectiveness measures. Every time step approximates 1 week, and individuals can access or discontinue care. Over time, some people will transition out of PTSD status owing to receipt of treatment or spontaneous remission. Among those who recover, a proportion experience recurrence of case status. A schematic overview of our model is provided in Figure 1A, and an overview of SC is provided in Figure 1B. Boston University Institutional Review Board waived approval for this study as all data came from previously published work.

Model Initialization: Agent Population

Our population of 2 642 713 agents was initialized to serve as a to-scale approximation of areas of New York City affected by Hurricane Sandy. We applied parameters to our pseudopopulation using sociodemographic (ie, sex, age, and race/ethnicity) and borough characteristics derived from the American Community Survey estimates (2008-2012; 5-year mean) for New York City,24,25 as shown in Table 1. Figure 2 shows a map of New York City with hurricane-affected areas2 shaded in gray.

Posttraumatic Stress Disorder
PTSD Symptom Distribution

Initial distributions of the prevalence of and number of PTSD symptoms were estimated from an observational survey conducted among residents of affected areas after Hurricane Sandy.26 This survey used the PTSD checklist to measure hurricane-related PTSD symptoms27; probable cases of PTSD as determined by DSM-IV criteria were classified as those with a symptom score greater than 44. Specifically, we estimated prevalence and symptom counts for 24 strata composed of all possible combinations of sex (male or female), age (18-34, 35-64, or ≥65 years), and race/ethnicity (white non-Hispanic, black non-Hispanic, other non-Hispanic, or Hispanic). These estimates are provided in eTable 1 in the Supplement.

Screening of Probable PTSD

Using data from the Hurricane Sandy survey,26 a receiver operating characteristic curve was used to estimate number of symptoms (range, 0-17) corresponding to a probable diagnosis of PTSD, using a symptom score of 44 on the PTSD checklist as a criterion standard.27 Our cutpoint for case status was 6 symptoms, which maximized sensitivity (1.00) and specificity (0.96) within our observational sample. In turn, in the context of the present simulation, we set our base-case sensitivity and specificity to 0.80 each. In addition, we present estimates of treatment effectiveness and cost-effectiveness for an alternate screening case in which sensitivity is 0.80 and specificity is 1.00 (case A), and another alternate screening case in which sensitivity is 1.00 and specificity is 0.80 (case B).

Remission Without Care

Among those who do not receive care, 30% undergo remission without treatment, corresponding to the upper range of a meta-analytic estimate for remission among survivors of a broad array of traumatic events in psychotherapy trial waitlist control conditions.28 We chose the upper end of this range, as natural disasters are considered among the less potent types of traumatic events.29

Relapse

Approximately 15% of recovered individuals undergo recurrence of PTSD after resolution of symptoms. We chose 15% to split the difference between recurrence rates of approximately 30% from samples of mixed traumatic exposure,30 and lack of relapsing-remitting trajectories in studies that examine trajectories of symptoms after natural disasters.31 Among agents that underwent symptom relapse, symptoms returned to 80% of initial symptom levels, assuming that symptom levels experienced on relapse will have attenuated from their initial presentation.

Intervention Components and Strength
Cognitive Behavioral Therapy

Cognitive behavioral therapy takes place in our model for 8 sessions, per PTSD treatment guidelines.32 A course of CBT treatment reduces an agent’s symptoms by 0.36, as shown in the eAppendix in the Supplement. This parameter is a function of the reduction in symptom scale score on the PTSD Symptom Scale experienced during a 3-month period in the CBT arm of a randomized clinical trial33 testing early treatment of PTSD. Up to 8 extra sessions were allowed among agents who did not recover after the initial 8 sessions.

Skills for Psychological Recovery

As described in the SPR manual,10 the program is an intermediate-level intervention that may lower the need for formal mental health care treatment and aims to reduce distress while improving coping and functioning. In contrast to the reduction of symptoms by 0.36 associated with a course of CBT, 5 sessions of SPR will reduce an agent’s symptoms by a proportion of 0.20, as shown in the eAppendix in the Supplement.

Intervention Definitions

Usual care includes provision of SPR. Stepped care includes a triage screening step at initial entry to services, as illustrated in Figure 1B. Agents are assessed for PTSD case status, and cases are offered CBT, while noncases are offered SPR.

Treatment Seeking
SPR Treatment Enrollment

Probability of enrollment in and use of SPR was estimated from a survey of New York City residents in the aftermath of the September 11, 2001, World Trade Center terrorist attacks.34 These models included age, sex, and race/ethnicity and were estimated separately for cases and noncases, as shown in the eAppendix in the Supplement.

CBT Treatment Enrollment

Probability of enrolling in CBT treatment was estimated from World Trade Center data using age, sex, and race/ethnicity, and was estimated separately among cases and noncases, as shown in the eAppendix in the Supplement.

Outcome Measurement
Reach

We defined reach as the number of individuals per 10 000 who accessed treatment, and report reach attained at the initial time-step of treatment, which provides an optimal overall measure.

Effectiveness

We measured treatment effectiveness using risk differences (RDs) (defined in the eAppendix in the Supplement) as a measure of absolute effects and risk ratios (RRs) (eAppendix in the Supplement) as a measure of relative effects. These measures were calculated for each 3-month time increment in the 2-year simulation.

Cost

The cost of CBT was set at $60 per session, based on an assessment of in-person delivery costs.35 We did not have data on costs for SPR and made an assumption that the cost per person, per group session would be $15. We also performed sensitivity analyses to determine the effects of raising the cost of CBT to $120 per session and raising the cost of SPR to $30 per session.

Incremental Cost-effectiveness

Incremental cost-effectiveness ratios (ICERs) were computed among initial cases for each simulated comparison using disability-adjusted life years (DALYs) and PTSD-free days. Disability-adjusted life years provide a standardized method for estimating years of healthy life lost as a result of disability36; their computation is described in the eAppendix in the Supplement. Disability weights used to characterize disability are derived from the Global Burden of Disease study37 and are specific to mild (0.03), moderate (0.149), and severe (0.523) disease states. Posttraumatic stress disorder–free days are the number of days an agent spends in noncase status throughout the simulation. Calculation of ICERs is described in the eAppendix in the Supplement and applied for a 10-year time horizon, with a discounting rate of 3% on future cost and effectiveness. Our model takes a health care sector cost perspective.

Model Scenarios

Beginning 4 weeks after the hurricane, each of the 2 models (UC and SC) were run for 100 time steps, representing in total the passage of 2 years after Hurricane Sandy.

Technical Details

The model was implemented in Microsoft Visual Studio 2012 (Microsoft Corp) and developed using C++. To account for stochasticity in the modeling process, each model scenario was run 50 times, with mean statistical effect measures reported. The overview, design concept, and details protocol38 for this agent-based model is outlined in the eAppendix in the Supplement, and provides an overview of the model and submodels, pseudocode, and an elaboration of design concepts and model details. We computed the 2.5th percentile and 97.5th percentile calculated across those 50 simulations.

Results

As demonstrated in Table 1, our simulated population distributions of borough, age, sex, and race/ethnicity achieved a good representation of the 2008-2012 American Community Survey 5-year population estimates of the areas of New York City affected by Hurricane Sandy.24,25

Reach

As shown in eTable 2 in the Supplement, the reach of SC (3137.25 per 10 000) exceeded that of UC (2118.60 per 10 000) in the base-case and other simulation scenarios.

Treatment Effectiveness

Figure 3A demonstrates that SC began to yield a lower prevalence of PTSD starting at 6 months (RD, –0.004; 95% CI, –0.004 to –0.004), with the absolute benefit continuing to gradually improve through 1.25 years (RD, –0.015; 95% CI, –0.015 to –0.014), after which it reached a plateau. The relative benefit of SC, expressed as RRs, increased strongly from essentially no difference at 3 months (RR, 0.994; 95% CI, 0.914-1.081) to an approximately 30% lower risk at 1 year (RR, 0.667; 95% CI, 0.661-0.673), with benefit increasing through 1.75 years (RR, 0.615; 95% CI, 0.609-0.622).

Figure 3B demonstrates that the proportion of PTSD cases remaining under UC exceeded that among cases in the SC group at each 3-month interval. The absolute benefit increases from 3 months (RD, –0.006; 95% CI, –0.007 to –0.005) onward, reaching its maximum at 1.5 years (RD, –0.338; 95% CI, –0.342 to –0.335). The relative benefit of SC among cases was equivalent to that among the full population.

Cost-effectiveness

As shown in Table 2, the total cost of UC ranged from $8.37 to $16.75 million and the total cost of SC ranged from $50.94 to $101.87 million. Incremental cost-effectiveness ratios for SC compared with UC ranged from $3428.71 to $6857.68 per DALY averted, while the ICER per PTSD-free day ranged from $0.80 to $1.61.

Sensitivity Analyses
Screening Performance

Under alternate screening case A (eFigure 1 in the Supplement), treatment effectiveness was roughly equivalent to that observed under the base case. Under alternate screening case B (eFigure 2 in the Supplement), effectiveness was superior to that of the base case, with greater absolute and relative benefits of SC. As shown in eTable 3 in the Supplement, under alternate case A, ICERs ranged from $3192.55 per DALY averted to $6562.87 per DALY averted, and $0.75 per PTSD-free day to $1.54 per PTSD-free day. Under alternate case B (eTable 3 in the Supplement), ICERs ranged from $3394.23 per DALY averted to $6793.08 per DALY averted, and $0.80 per PTSD-free day to $1.59 per PTSD-free day.

Treatment Strength, Enrollment Probabilities, Disability Weights

To test the dependence of our results on treatment strength, we set the strength of each session of CBT and SPR to an equal value of symptom reduction (eFigure 3 in the Supplement). To test the dependence of our results on treatment enrollment probabilities we increased enrollment probabilities for SPR and decreased those for CBT (eFigure 4 in the Supplement). To test the dependence of our results on disability weighting, we decreased the severe PTSD disability weight to be the same strength as the moderate PTSD disability weight (eTable 4 in the Supplement). In all cases, SC was superior to UC and was cost-effective; under the disability weighting change, effectiveness was identical to that of the base case.

Discussion

Using a simulated counterfactual model comparing UC with an SC case-finding intervention after a natural disaster, we demonstrated that SC was superior to UC in the early treatment of PTSD. Stepped care was associated with greater reach and more effectiveness than UC in reducing the prevalence of hurricane-related PTSD in the population and among cases. These effects emerged between 3 and 6 months after the hurricane and were sustained through the study period. Although SC was more costly overall than UC, the cost per DALY saved ($3428.71-$6857.68), and the cost per PTSD-free day ($0.80-$1.61) were well below what is generally considered a reasonable cost-benefit value.39

These results suggest that the SC scenario we present provides greater population impact than UC and offers excellent value. Our sensitivity analyses suggest acceptable reach, effectiveness, and cost-effectiveness profiles that are robust to a range of screener performance characteristics, an equalization of per-session treatment effectiveness, changes to enrollment probabilities, and strength of disability weighting. Nonetheless, despite the excellent value afforded by SC, deployment of this model requires substantial resources, per our estimated absolute costs of $50.94 million to $101.87 million for deployment in New York City after Hurricane Sandy.

This simulation provides proof of concept for SC in the treatment of PTSD after a natural disaster, and further extends the rationale for this approach. Given the lack of randomized clinical trials or natural experiments to inform this question, these simulation results represent the best evidence to date supporting this approach after disasters. Future studies should consider internet-based interventions, which may extend the reach of SC and improve population outcomes. In addition, given limited uptake of CBT in postdisaster contexts40 and other nonspecialty mental health care contexts,41,42 if CBT reach after a disaster were enhanced, SC performance, as documented here, would likely improve.

The findings of this study are consistent with those of prior studies examining SC care approaches to treating mental health disorders. Stepped care was superior in the treatment of PTSD among injury survivors hospitalized for surgical care, although no cost-effectiveness estimates were provided.17 A prior study of SC for multiple anxiety disorders in primary care found an RD of –0.143 at the 18-month follow-up,15 and an ICER of about $4 per anxiety-free day.43 Similarly, a collaborative care intervention for panic disorder, featuring many elements of SC, found an ICER of $8.40 per anxiety-free day.44 The higher observed costs in those studies may be explained by the primary care and hospital settings, along with the inclusion of pharmacologic interventions, which are, on average, less cost-effective than psychotherapy.45

Limitations

This study should be considered in light of several limitations. First, and most importantly, this was a simulation, and the findings are based on the assumptions and design of our model. Second, geography is incorporated into our model only at symptom initialization, and not as part of the sub-models on mental health care treatment seeking that were run at each time step. Third, our cost-effectiveness estimates do not include administrative costs.

Working within these limitations, however, our models were initialized with strong empirical data,26,33,34 providing confidence that our agent-based model provided reasonable estimates of mean treatment effect. Furthermore, we conducted sensitivity analyses to check model robustness under varying screening and cost assumptions and found further support for our main results. In addition, geography beyond the level of symptom initialization is not necessary in attempting to model mean treatment effects in New York City overall. Finally, instead of including administrative costs, we provide a range of cost-effectiveness profiles to allay concerns about underestimation.

Conclusions

This study has shown in a simulation that SC for early treatment of PTSD after a disaster was associated with greater reach, treatment effectiveness, and cost-effectiveness relative to the more universally applied interventions that resemble our UC condition. Accordingly, these results provide further proof of concept for the SC approach to treating PTSD after a disaster, and they warrant further study and application in real-world settings.

Back to top
Article Information

Corresponding Author: Gregory H. Cohen, MPhil, MSW, Department of Epidemiology, Boston University School of Public Health, 715 Albany St, Boston, MA 02118 (ghcohen@bu.edu).

Accepted for Publication: August 9, 2017.

Published Online: October 4, 2017. doi:10.1001/jamapsychiatry.2017.3037

Author Contributions: Mr Cohen and Dr Galea had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Cohen, Tamrakar, Lowe, Sampson, Galea.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Cohen, Tamrakar, Linas, Galea.

Critical revision of the manuscript for important intellectual content: Cohen, Tamrakar, Lowe, Sampson, Ettman, Ruggiero.

Statistical analysis: Cohen, Tamrakar, Sampson.

Obtained funding: Ruggiero, Galea.

Administrative, technical, or material support: Tamrakar, Sampson, Ettman, Linas, Galea.

Study supervision: Linas, Galea.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding for this work was provided by grant HITEP140015 from the US Department of Health and Human Services.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: Oliver Gruebner, PhD, Humboldt-Universität zu Berlin, created the map for this article. He was not compensated for his contribution.

References
1.
Redlener  I, Reilly  MJ.  Lessons from Sandy—preparing health systems for future disasters.  N Engl J Med. 2012;367(24):2269-2271. PubMedGoogle ScholarCrossref
2.
Recovery  NYC. Community development block grant disaster recovery: the City of New York action plan incorporating amendments 1-11. http://www.nyc.gov/html/cdbg/downloads/pdf/cdbg-dr_action_plan_incorporating_amendments_1-11.pdf. Published February 1, 2016. Accessed August 23, 2017.
3.
National Oceanic and Atmospheric Administration, US Dept of Commerce. Service assessment: Hurricane/post-Tropical Cyclone Sandy, October 22-29, 2012. https://www.weather.gov/media/publications/assessments/Sandy13.pdf. Published May 2013. Accessed August 23, 2017.
4.
Centers for Disease Control and Prevention (CDC).  Deaths associated with Hurricane Sandy—October-November 2012.  MMWR Morb Mortal Wkly Rep. 2013;62(20):393-397.PubMedGoogle Scholar
5.
Brackbill  RM, Caramanica  K, Maliniak  M,  et al.  Nonfatal injuries 1 week after Hurricane Sandy—New York City metropolitan area, October 2012.  MMWR Morb Mortal Wkly Rep. 2014;63(42):950-954.PubMedGoogle Scholar
6.
Goldstein  W, Peterson  A, Zarrilli  DA. One city, rebuilding together: a report on the City of New York’s response to Hurricane Sandy and the path forward. http://www1.nyc.gov/assets/home/downloads/pdf/reports/2014/sandy_041714.pdf. Published April 2014. Accessed August 23, 2017.
7.
Goldmann  E, Galea  S.  Mental health consequences of disasters.  Annu Rev Public Health. 2014;35(1):169-183. PubMedGoogle ScholarCrossref
8.
Schreiber  MD, Yin  R, Omaish  M, Broderick  JE.  Snapshot from Superstorm Sandy: American Red Cross mental health risk surveillance in lower New York State.  Ann Emerg Med. 2014;64(1):59-65. PubMedGoogle ScholarCrossref
9.
 Crisis Counseling Assistance and Training Program Regular Services Program Final Report. Albany, NY: New York State Office of Mental Health; 2014.
10.
Berkowitz  S, Bryant  R, Brymer  M,  et al. Skills for Psychological Recovery: field operations guide, 2010. https://www.ptsd.va.gov/professional/materials/manuals/skills_psych_recovery_manual.asp. Accessed August 23, 2017.
11.
Lowe  SR, Sampson  L, Gruebner  O, Galea  S.  Mental health service need and use in the aftermath of Hurricane Sandy: findings in a population-based sample of New York City residents.  Community Ment Health J. 2016;52(1):25-31. PubMedGoogle ScholarCrossref
12.
Wang  PS, Gruber  MJ, Powers  RE,  et al.  Mental health service use among Hurricane Katrina survivors in the eight months after the disaster.  Psychiatr Serv. 2007;58(11):1403-1411. PubMedGoogle ScholarCrossref
13.
Ghuman  SJ, Brackbill  RM, Stellman  SD, Farfel  MR, Cone  JE.  Unmet mental health care need 10-11 years after the 9/11 terrorist attacks: 2011-2012 results from the World Trade Center Health Registry.  BMC Public Health. 2014;14(1):491. PubMedGoogle ScholarCrossref
14.
US Dept of Veterans Affairs. Psychological first aid: field operations guide. https://www.ptsd.va.gov/professional/materials/manuals/psych-first-aid.asp. Accessed August 23, 2017.
15.
Roy-Byrne  P, Craske  MG, Sullivan  G,  et al.  Delivery of evidence-based treatment for multiple anxiety disorders in primary care: a randomized controlled trial.  JAMA. 2010;303(19):1921-1928. PubMedGoogle ScholarCrossref
16.
Katon  W, Von Korff  M, Lin  E,  et al.  Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial.  Arch Gen Psychiatry. 1999;56(12):1109-1115. PubMedGoogle ScholarCrossref
17.
Zatzick  D, Jurkovich  G, Rivara  FP,  et al.  A randomized stepped care intervention trial targeting posttraumatic stress disorder for surgically hospitalized injury survivors.  Ann Surg. 2013;257(3):390-399. PubMedGoogle ScholarCrossref
18.
North  CS, Pfefferbaum  B.  Mental health response to community disasters: a systematic review.  JAMA. 2013;310(5):507-518. PubMedGoogle ScholarCrossref
19.
Glasgow  RE, Vogt  TM, Boles  SM.  Evaluating the public health impact of health promotion interventions: the RE-AIM framework.  Am J Public Health. 1999;89(9):1322-1327.PubMedGoogle ScholarCrossref
20.
Craske  MG, Roy-Byrne  PP, Stein  MB, Sullivan  G, Sherbourne  C, Bystritsky  A.  Treatment for anxiety disorders: efficacy to effectiveness to implementation.  Behav Res Ther. 2009;47(11):931-937. PubMedGoogle ScholarCrossref
21.
Zatzick  DF, Koepsell  T, Rivara  FP.  Using target population specification, effect size, and reach to estimate and compare the population impact of two PTSD preventive interventions.  Psychiatry. 2009;72(4):346-359. PubMedGoogle ScholarCrossref
22.
Koepsell  TD, Zatzick  DF, Rivara  FP.  Estimating the population impact of preventive interventions from randomized trials.  Am J Prev Med. 2011;40(2):191-198. PubMedGoogle ScholarCrossref
23.
Marshall  BDL, Galea  S.  Formalizing the role of agent-based modeling in causal inference and epidemiology.  Am J Epidemiol. 2015;181(2):92-99. PubMedGoogle ScholarCrossref
24.
US Census Bureau. American community survey. https://www.census.gov/programs-surveys/acs/. Accessed August 23, 2017.
25.
Infoshare. Online/Community Studies of New York, Inc. Area Comparison Table. http://www.infoshare.org/main/public.aspx. Accessed January 1, 2016.
26.
Lowe  SR, Sampson  L, Gruebner  O, Galea  S.  Psychological resilience after Hurricane Sandy: the influence of individual- and community-level factors on mental health after a large-scale natural disaster.  PLoS One. 2015;10(5):e0125761.PubMedGoogle ScholarCrossref
27.
Blanchard  EB, Jones-Alexander  J, Buckley  TC, Forneris  CA.  Psychometric properties of the PTSD checklist (PCL).  Behav Res Ther. 1996;34(8):669-673. PubMedGoogle ScholarCrossref
28.
Bradley  R, Greene  J, Russ  E, Dutra  L, Westen  D.  A multidimensional meta-analysis of psychotherapy for PTSD.  Am J Psychiatry. 2005;162(2):214-227. PubMedGoogle ScholarCrossref
29.
Liu  H, Petukhova  MV, Sampson  NA,  et al; World Health Organization World Mental Health Survey Collaborators.  Association of DSM-IV posttraumatic stress disorder with traumatic experience type and history in the World Health Organization World Mental Health Surveys.  JAMA Psychiatry. 2017;74(3):270-281. PubMedGoogle ScholarCrossref
30.
Steinert  C, Hofmann  M, Leichsenring  F, Kruse  J.  The course of PTSD in naturalistic long-term studies: high variability of outcomes: a systematic review.  Nord J Psychiatry. 2016;69(7):483-496. PubMedGoogle ScholarCrossref
31.
Norris  FH, Tracy  M, Galea  S.  Looking for resilience: understanding the longitudinal trajectories of responses to stress.  Soc Sci Med. 2009;68(12):2190-2198. PubMedGoogle ScholarCrossref
32.
Foa  E, Keane  TM, Friedman  MJ, Cohen  JA, eds.  Effective Treatments for PTSD: Practice Guidelines from the International Society for Traumatic Stress Studies. 2nd ed. New York, NY: The Guilford Press; 2009.
33.
Shalev  AY, Ankri  Y, Israeli-Shalev  Y, Peleg  T, Adessky  R, Freedman  S.  Prevention of posttraumatic stress disorder by early treatment: results from the Jerusalem Trauma Outreach and Prevention study.  Arch Gen Psychiatry. 2012;69(2):166-176. PubMedGoogle ScholarCrossref
34.
Galea  S, Ahern  J, Tracy  M,  et al.  Longitudinal determinants of posttraumatic stress in a population-based cohort study.  Epidemiology. 2008;19(1):47-54. PubMedGoogle ScholarCrossref
35.
Acierno  R. Home-based telehealth vs. in-person delivery of evidence based psychotherapy for combat-related PTSD. Presented at: Development Field-Based Meeting on Rural Health; September 2016; Washington, DC.
36.
Keyes  KM, Galea  S.  Population Health Science. New York, NY: Oxford University Press; 2016.Crossref
37.
Whiteford  HA, Degenhardt  L, Rehm  J,  et al.  Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010.  Lancet. 2013;382(9904):1575-1586. PubMedGoogle ScholarCrossref
38.
Grimm  V, Berger  U, DeAngelis  DL, Polhill  JG, Giske  J, Railsback  SF.  The ODD protocol: a review and first update.  Ecol Modell. 2010;221(23):2760-2768. doi:10.1016/j.ecolmodel.2010.08.019Google ScholarCrossref
39.
Grosse  SD.  Assessing cost-effectiveness in healthcare: history of the $50,000 per QALY threshold.  Expert Rev Pharmacoecon Outcomes Res. 2008;8(2):165-178. PubMedGoogle ScholarCrossref
40.
Jaycox  LH, Cohen  JA, Mannarino  AP,  et al.  Children’s mental health care following Hurricane Katrina: a field trial of trauma-focused psychotherapies.  J Trauma Stress. 2010;23(2):223-231.PubMedGoogle Scholar
41.
Trusz  SG, Wagner  AW, Russo  J, Love  J, Zatzick  DF.  Assessing barriers to care and readiness for cognitive behavioral therapy in early acute care PTSD interventions.  Psychiatry. 2011;74(3):207-223. PubMedGoogle ScholarCrossref
42.
Darnell  D, O’Connor  S, Wagner  A,  et al.  Enhancing the reach of cognitive-behavioral therapy targeting posttraumatic stress in acute care medical settings.  Psychiatr Serv. 2017;68(3):258-263. PubMedGoogle ScholarCrossref
43.
Joesch  JM, Sherbourne  CD, Sullivan  G, Stein  MB, Craske  MG, Roy-Byrne  P.  Incremental benefits and cost of coordinated anxiety learning and management for anxiety treatment in primary care.  Psychol Med. 2012;42(9):1937-1948.PubMedGoogle ScholarCrossref
44.
Katon  W, Russo  J, Sherbourne  C,  et al.  Incremental cost-effectiveness of a collaborative care intervention for panic disorder.  Psychol Med. 2006;36(3):353-363. PubMedGoogle ScholarCrossref
45.
Ophuis  RH, Lokkerbol  J, Heemskerk  SCM, van Balkom  AJLM, Hiligsmann  M, Evers  SMAA.  Cost-effectiveness of interventions for treating anxiety disorders: a systematic review.  J Affect Disord. 2017;210:1-13. PubMedGoogle ScholarCrossref
×