[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.
Cost-effectiveness Acceptability Curve
Cost-effectiveness Acceptability Curve

The probability that the Housing First (HF) intervention is cost-effective compared with treatment as usual as a function of the value that the decision-maker attributes to an additional day of stable housing.

Table 1.  
Baseline Characteristics of Participants, Stratified by Randomization Group
Baseline Characteristics of Participants, Stratified by Randomization Group
Table 2.  
Mean (Unadjusted) Costs per Person per Year by Time and Cost Categorya
Mean (Unadjusted) Costs per Person per Year by Time and Cost Categorya
Table 3.  
Total Unadjusted Mean Costs per Person per Year by Site
Total Unadjusted Mean Costs per Person per Year by Site
Table 4.  
Net Benefit Regression Results Assigning Different Values to an Additional Day of Stable Housing With Interaction Termsa
Net Benefit Regression Results Assigning Different Values to an Additional Day of Stable Housing With Interaction Termsa
1.
Fazel  S, Khosla  V, Doll  H, Geddes  J.  The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis.  PLoS Med. 2008;5(12):e225. doi:10.1371/journal.pmed.0050225PubMedGoogle ScholarCrossref
2.
Aubry  T, Nelson  G, Tsemberis  S.  Housing First for people with severe mental illness who are homeless: a review of the research and findings from the At Home–Chez Soi demonstration project.  Can J Psychiatry. 2015;60(11):467-474. doi:10.1177/070674371506001102PubMedGoogle ScholarCrossref
3.
Tsemberis  S.  Housing First: The Pathways Model to End Homelessness for People with Mental Illness and Addiction Manual. Center City, MN: Hazelden; 2010.
4.
Ly  A, Latimer  E.  Housing first impacts on costs and associated cost offsets: a review of the literature.  Can J Psychiatry. 2015;60(11):475-487. doi:10.1177/070674371506001103PubMedGoogle ScholarCrossref
5.
Rosenheck  R, Kasprow  W, Frisman  L, Liu-Mares  W.  Cost-effectiveness of supported housing for homeless persons with mental illness.  Arch Gen Psychiatry. 2003;60(9):940-951. doi:10.1001/archpsyc.60.9.940PubMedGoogle ScholarCrossref
6.
Goering  PN, Streiner  DL, Adair  C,  et al.  The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomised controlled trial of a Housing First intervention for homeless individuals with mental illness in five Canadian cities.  BMJ Open. 2011;1(2):e000323. doi:10.1136/bmjopen-2011-000323PubMedGoogle ScholarCrossref
7.
Stergiopoulos  V, Hwang  SW, Gozdzik  A,  et al; At Home/Chez Soi Investigators.  Effect of scattered-site housing using rent supplements and intensive case management on housing stability among homeless adults with mental illness: a randomized trial.  JAMA. 2015;313(9):905-915. doi:10.1001/jama.2015.1163PubMedGoogle ScholarCrossref
8.
Aubry  T, Goering  P, Veldhuizen  S,  et al.  A randomized controlled trial in five Canadian cities of the effectiveness of Housing First with assertive community treatment for persons with serious mental illness and a history of homelessness.  Psychiatr Serv. 2016;67(3):275-281. doi:10.1176/appi.ps.201400587PubMedGoogle ScholarCrossref
9.
Husereau  D, Drummond  M, Petrou  S,  et al; CHEERS Task Force.  Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.  BMJ. 2013;346:f1049. doi:10.1136/bmj.f1049PubMedGoogle ScholarCrossref
10.
Dickerson  FB, Origoni  AE, Pater  A, Friedman  BK, Kordonski  WM.  An expanded version of the Multnomah Community Ability Scale: anchors and interview probes for the assessment of adults with serious mental illness.  Community Ment Health J. 2003;39(2):131-137. doi:10.1023/A:1022610620391PubMedGoogle ScholarCrossref
11.
Frane  JW.  A method of biased coin randomization, its implementation, and its validation.  Drug Inf J. 1998;32:423-432. doi:10.1177/009286159803200213Google ScholarCrossref
12.
Macnaughton  E, Stefancic  A, Nelson  G,  et al.  Implementing Housing First across sites and over time: later fidelity and implementation evaluation of a pan-Canadian multi-site Housing First program for homeless people with mental illness.  Am J Community Psychol. 2015;55(3-4):279-291. doi:10.1007/s10464-015-9709-zPubMedGoogle ScholarCrossref
13.
Latimer  E, Rabouin  D, Cao  Z,  et al; At Home/Chez Soi Investigators.  The costs of usual services for homeless people with mental illness in five Canadian cities: results from a large prospective follow-up study.  Canadian Medical Association Journal Open. 2017;5(3):E576-E85.PubMedGoogle Scholar
14.
Lecrubier  Y, Weiller  E, Amorim  P, Bonora  I, Sheehan  KH, Janavs  J.  The Mini International Neuropsychiatric Interview (MINI): a short diagnostic structured interview: reliability and validity according to the CIDI.  Eur Psychiatry. 1997;12(5):224-231. doi:10.1016/S0924-9338(97)83296-8Google ScholarCrossref
15.
Frisman  L, Rosenheck  R.  How transfer payments are treated in cost-effectiveness and cost-benefit analyses.  Adm Policy Ment Health. 1996;23(6):533-546. doi:10.1007/BF02108689Google ScholarCrossref
16.
Weisbrod  BA, Test  MA, Stein  LI.  Alternative to mental hospital treatment, II: economic benefit-cost analysis.  Arch Gen Psychiatry. 1980;37(4):400-405. doi:10.1001/archpsyc.1980.01780170042004PubMedGoogle ScholarCrossref
17.
Salkever  D.  Social costs of expanding access to evidence-based supported employment: concepts and interpretive review of evidence.  Psychiatr Serv. 2013;64(2):111-119. doi:10.1176/appi.ps.201100499PubMedGoogle ScholarCrossref
18.
Meltzer  D.  Accounting for future costs in medical cost-effectiveness analysis.  J Health Econ. 1997;16(1):33-64. doi:10.1016/S0167-6296(96)00507-3PubMedGoogle ScholarCrossref
19.
Drummond  MF, Sculpher  MJ, Claxton  K, Stoddart  GL, Torrance  GW.  Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford, UK: Oxford University Press; 2015.
20.
Statistics Canada. Consumer Price Index, monthly, not seasonally adjusted. Table 326-0020. http://www5.statcan.gc.ca/cansim/a26?lang=eng&retrLang=eng&id=3260020&&pattern=&stByVal=1&p1=1&p2=37&tabMode=dataTable&csid=#customizeTab. Modified July 14, 2019. Accessed November 17, 2016.
21.
Azur  MJ, Stuart  EA, Frangakis  C, Leaf  PJ.  Multiple imputation by chained equations: what is it and how does it work?  Int J Methods Psychiatr Res. 2011;20(1):40-49. doi:10.1002/mpr.329PubMedGoogle ScholarCrossref
22.
Shao  J, Sitter  RR.  Bootstrap for imputed survey data.  J Am Stat Assoc. 1996;91(435):1278-1288. doi:10.1080/01621459.1996.10476997Google ScholarCrossref
23.
Hoch  JS, Briggs  AH, Willan  AR.  Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis.  Health Econ. 2002;11(5):415-430. doi:10.1002/hec.678PubMedGoogle ScholarCrossref
24.
van Asselt  AD, van Mastrigt  GA, Dirksen  CD, Arntz  A, Severens  JL, Kessels  AG.  How to deal with cost differences at baseline.  Pharmacoeconomics. 2009;27(6):519-528. doi:10.2165/00019053-200927060-00007PubMedGoogle ScholarCrossref
25.
Goering  P, Veldhuizen  S, Watson  A,  et al.  National At Home/Chez Soi Final Report. Calgary, AB: Mental Health Commission of Canada; 2014.
26.
Nord  E, Daniels  N, Kamlet  M.  QALYs: some challenges.  Value Health. 2009;12(1)(suppl 1):S10-S15. doi:10.1111/j.1524-4733.2009.00516.xPubMedGoogle ScholarCrossref
27.
Euroqol Group. EQ-5D. https://euroqol.org/. Accessed January 5, 2018.
28.
Nelson  G, Patterson  M, Kirst  M,  et al.  Life changes among homeless persons with mental illness: a longitudinal study of housing first and usual treatment.  Psychiatr Serv. 2015;66(6):592-597. doi:10.1176/appi.ps.201400201PubMedGoogle ScholarCrossref
29.
Bureau of Labor Statistics. Databases, Tables & Calculators by Subject: CPI Inflation Calculator. https://www.bls.gov/data/inflation_calculator.htm. Accessed November 17, 2016.
30.
Statistics Canada. Purchasing power parities, 2016. https://www150.statcan.gc.ca/n1/daily-quotidien/180424/dq180424b-eng.htm. Posted April 24, 2018. Accessed July 9, 2018.
31.
Goering  P, Veldhuizen  S, Nelson  GB,  et al.  Further validation of the pathways Housing First Fidelity Scale.  Psychiatr Serv. 2016;67(1):111-114. doi:10.1176/appi.ps.201400359PubMedGoogle ScholarCrossref
32.
Davidson  C, Neighbors  C, Hall  G,  et al.  Association of Housing First implementation and key outcomes among homeless persons with problematic substance use.  Psychiatr Serv. 2014;65(11):1318-1324. doi:10.1176/appi.ps.201300195PubMedGoogle ScholarCrossref
33.
Gilmer  TP, Stefancic  A, Henwood  BF, Ettner  SL.  Fidelity to the Housing First model and variation in health service use within permanent supportive housing.  Psychiatr Serv. 2015;66(12):1283-1289. doi:10.1176/appi.ps.201400564PubMedGoogle ScholarCrossref
34.
Bhandari  A, Wagner  T.  Self-reported utilization of health care services: improving measurement and accuracy.  Med Care Res Rev. 2006;63(2):217-235. doi:10.1177/1077558705285298PubMedGoogle ScholarCrossref
35.
Somers  JM, Moniruzzaman  A, Currie  L, Rezansoff  SN, Russolillo  A, Parpouchi  M.  Accuracy of reported service use in a cohort of people who are chronically homeless and seriously mentally ill.  BMC Psychiatry. 2016;16:41. doi:10.1186/s12888-016-0758-0PubMedGoogle ScholarCrossref
36.
Lemieux  AJ, Roy  L, Martin  MS, Latimer  EA, Crocker  AG.  Justice involvement among homeless individuals with mental illnesses: are self-report and administrative measures comparable?  Eval Program Plann. 2017;61(April):86-95. doi:10.1016/j.evalprogplan.2016.12.007PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    EXPAND ALL
    The Capability Trap
    Paul Nelson, M.D., M.S. | Family Health Care, P.C.
    For a person with serious emotional health problems who has been chronically homeless, their decision to become "housed" is difficult and usually requires a unique outreach process. As the homeless crisis seems to have nationally exploded, the needs within each community for solutions will continue to benefit from broadly collaborative commitments for resolving the institutional barriers that impede progressive improvement. Lost in the national angst about homelessness, we are in danger of losing our commitment to resolve the generational, social determinants underlying not only homelessness but also maternal/infant mortality, child neglect/abuse, childhood obesity, adolescent suicide/homicide, substance abuse, mass shootings, mid-life depression/disability, and decreasing longevity at birth (now four years in a row).

    For a concept to re-energize our commitment to primary prevention, I offer a citation to a presentation by Erik Landry and John Sterman given initially at the 34th UAlbany-MIT-WPI System Dynamics Colloquium on May 24, 2017. Its title is: "The Capability Trap: Prevalence in Human Systems." If you sense that homelessness and mass shootings represent a metastisizing malady underlying our nation's social cohesion and its associated decline in each community's social capital, I offer a website for the Landry-Sterman paper.

    https://www.systemdynamics.org/assets/conferences/2017/proceed/papers/P1325.pdf

    I cite the first sentence of its abstract: "What do abused children, the fatal collapse if the I-35 bridge connecting Minneapolis and St. Paul, and the disproportionately high costs of the United States healthcare system all have in common?" Yes, you really do need to download this report!
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Original Investigation
    Psychiatry
    August 21, 2019

    Cost-effectiveness of Housing First Intervention With Intensive Case Management Compared With Treatment as Usual for Homeless Adults With Mental Illness: Secondary Analysis of a Randomized Clinical Trial

    Author Affiliations
    • 1Department of Psychiatry, McGill University, Montreal, Quebec, Canada
    • 2Douglas Research Centre, Montreal, Quebec, Canada
    • 3Montreal West Island Integrated University Health and Social Services Centre, Montreal, Quebec, Canada
    • 4Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada
    • 5Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
    • 6Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
    • 7Department of Psychiatry, Faculty of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
    • 8Faculty of Health Sciences, Simon Fraser University, Vancouver, British Columbia, Canada
    • 9Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
    • 10Centre for Addiction and Mental Health, University of Toronto, Toronto, Ontario, Canada
    • 11Department of Family and Community Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
    • 12MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, Ontario, Canada
    JAMA Netw Open. 2019;2(8):e199782. doi:10.1001/jamanetworkopen.2019.9782
    Key Points español 中文 (chinese)

    Question  Is a Housing First intervention with Intensive Case Management for homeless people with mental illness cost-effective compared with treatment as usual?

    Findings  In this economic evaluation study of data from the At Home/Chez Soi randomized clinical trial with 1198 initially homeless participants, the incremental cost-effectiveness ratio was $56.08 per additional day of stable housing. At $67 per day of stable housing, there was an 80% chance that the Housing First intervention with Intensive Case Management was cost-effective compared with treatment as usual.

    Meaning  Expanding access to Housing First with Intensive Case Management appears to be warranted from an economic point of view.

    Abstract

    Importance  In the At Home/Chez Soi trial for homeless individuals with mental illness, the scattered-site Housing First (HF) with Intensive Case Management (ICM) intervention proved more effective than treatment as usual (TAU).

    Objective  To evaluate the cost-effectiveness of the HF plus ICM intervention compared with TAU.

    Design, Setting, and Participants  This is an economic evaluation study of data from the At Home/Chez Soi randomized clinical trial. From October 2009 through July 2011, 1198 individuals were randomized to the intervention (n = 689) or TAU (n = 509) and followed up for as long as 24 months. Participants were recruited in the Canadian cities of Vancouver, Winnipeg, Toronto, and Montreal. Participants with a current mental disorder who were homeless and had a moderate level of need were included. Data were analyzed from 2013 through 2019, per protocol.

    Interventions  Scattered-site HF (using rent supplements) with off-site ICM services was compared with usual housing and support services in each city.

    Main Outcomes and Measures  The analysis was performed from the perspective of society, with days of stable housing as the outcome. Service use was ascertained using questionnaires. Unit costs were estimated in 2016 Canadian dollars.

    Results  Of 1198 randomized individuals, 795 (66.4%) were men and 696 (58.1%) were aged 30 to 49 years. Almost all (1160 participants, including 677 in the HF group and 483 in the TAU group) contributed data to the economic analysis. Days of stable housing were higher by 140.34 days (95% CI, 128.14-153.31 days) in the HF group. The intervention cost $14 496 per person per year; reductions in costs of other services brought the net cost down by 46% to $7868 (95% CI, $4409-$11 405). The incremental cost-effectiveness ratio was $56.08 (95% CI, $29.55-$84.78) per additional day of stable housing. In sensitivity analyses, adjusting for baseline differences using a regression-based method, without altering the discount rate, caused the largest change in the incremental cost-effectiveness ratio with an increase to $60.18 (95% CI, $35.27-$86.95). At $67 per day of stable housing, there was an 80% chance that HF was cost-effective compared with TAU. The cost-effectiveness of HF appeared to be similar for all participants, although possibly less for those with a higher number of previous psychiatric hospitalizations.

    Conclusions and Relevance  In this study, the cost per additional day of stable housing was similar to that of many interventions for homeless individuals. Based on these results, expanding access to HF with ICM appears to be warranted from an economic standpoint.

    Trial Registration  isrctn.org Identifier: ISRCTN42520374

    Introduction

    A significant proportion of homeless individuals experience mental illness.1 Housing First (HF), which provides immediate access to subsidized housing together with support services, has proven to be the most effective approach at helping such individuals access and maintain permanent housing.2,3 Previous analyses,4 using mostly before-and-after comparisons or quasi-experimental designs, have reported significant cost offsets associated with the provision of HF. To our knowledge, only 1 cost-effectiveness analysis,5 conducted alongside a randomized clinical trial, has been published.

    The multiple-site At Home/Chez Soi trial compared outcomes of the scattered-site variant of HF, in which participants receive income-related rent supplements for private market apartments of their choice, with those of treatment as usual (TAU). The trial tested, in parallel, HF with Assertive Community Treatment for people who had more severe mental illness and functional difficulties and HF with Intensive Case Management (ICM) for those whose needs were less acute.6 Summary results of cost analyses, but not cost-effectiveness analyses, were included in the main trial reports.7,8 Herein we report on the cost-effectiveness of HF with ICM compared with TAU.

    Methods

    The cost-effectiveness analysis was performed in conformity with the published protocol of the At Home/Chez Soi study.6 The analysis followed the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) reporting guideline.9 The analysis began in 2013 and underwent successive refinements until 2019. The study protocol is available elsewhere.6 The trial was conducted in the Canadian cities of Vancouver, British Columbia; Winnipeg, Manitoba; Toronto, Ontario; Montreal, Québec; and Moncton, New Brunswick. Ethics approval was obtained from the local ethics review board at each data-collection site and from the Centre for Addiction and Mental Health, where the coordinating center was based.6 Participants provided written informed consent after the screening interview.

    Participants

    Details on sample recruitment are available elsewhere.6,7 Briefly, potential participants were referred from various sources or found through street outreach. Individuals were eligible if they were adults with legal status in their province of residence; had at least 1 of 6 current mental disorders, including psychotic disorder, major depressive disorder, or posttraumatic stress disorder; and were absolutely homeless or precariously housed with previous episodes of absolute homelessness. Individuals who were currently receiving services from an Assertive Community Treatment or ICM team (similar to what the experimental interventions offered, minus the access to rent supplements and dedicated housing staff) were not eligible.

    Initial Assessment and Randomization

    Using data collected online during the baseline interview, a computerized algorithm classified individuals as high or moderate need. To be classified as high need, individuals needed to (1) have a lower level of functioning (score of ≤62 on the Multnomah Community Ability Scale)10; (2) have a diagnosis of current psychotic disorder or bipolar disorder; and (3) meet at least 1 of the following criteria: 2 or more hospitalizations for mental illness during a 1-year period within the previous 5 years; comorbid substance use; or 1 or more arrests or incarcerations in the past 6 months. Others were classified as moderate need. (In Moncton, high-need and moderate-need individuals were classified as high need, because of the relatively small pool of individuals eligible for the study.6 Accordingly, data from Moncton are not included in this analysis.) Moderate-need individuals were randomized to receive HF plus ICM or other services normally available to them (TAU).7 An adaptive randomization algorithm11 with allocation concealment was used. From October 2009 through June 2011, 1198 moderate-need individuals were recruited.

    Interventions

    Participants in the HF plus ICM group received recovery-oriented supports from an ICM team with about 17 participants per case manager. Each ICM team worked in collaboration with housing specialists, also paid by the project, to help participants find housing of their choice, usually an apartment on the private rental market, and respond to housing issues as they arose. Participants were required to pay 25% or 30% of their income toward the rent, depending on whether it covered heating costs. The project paid the remainder of the rent, with this supplement ranging from a mean of $375 in Montreal to $600 in Vancouver. Periodic evaluation of the fidelity of the interventions to the program model, combined with feedback and ongoing coaching, aided standardization of the interventions across sites.12

    Although their intervention did not include HF, participants assigned to the TAU group had access to substantial supports, especially in the larger cities.13 These supports included emergency response services, such as shelters and hospital emergency departments, and some rehabilitative services, such as drug and alcohol rehabilitation centers and transitional housing. A small number of participants also were able to access ICM or Assertive Community Treatment services after they were recruited into the study.

    Data Collection

    Study participants were followed up for as long as 24 months. At baseline, interviewers used the Mini International Neuropsychiatric Interview14 and other sources such as medical records to ascertain diagnosis, assess whether abuse or dependence of alcohol or other substances was present, and document homelessness, hospitalization, and arrest history, among others. At baseline and every 6 months thereafter, a battery of standardized questionnaires was administered. The Multnomah Community Ability Scale, the only interviewer-rated measure, was completed at the end of the interview. Measures also included 3 questionnaires adapted for this study and designed to assess use of services.6,13 Participants completed the Health Services and Justice Services Use questionnaire at baseline and every 6 months thereafter. The Health Services and Justice Services Use questionnaire documented all non–overnight health- and justice-related services.6 The Residential Time-Line Follow-Back instrument was administered every 3 months starting 3 months after baseline and asked participants where they had spent every night since the previous interview (or since 3 months before baseline).6 To enable estimation of costs associated with service use, the Residential Time-Line Follow-Back instrument allowed coding of simultaneous places of residence; for example, if a participant had a subsidized apartment and was hospitalized, costs were associated with both places concurrently. Finally, the Vocational Time-Line Follow-Back questionnaire asked about income received month by month and any regular or casual work obtained during the previous 3 months.6 Due to the nature of the intervention and the inclusion of measures on service use and housing, participants and interviewers could not be blinded.

    Choice of Outcome Measure

    Days of stable housing (as assessed by the Residential Time-Line Follow-Back instrument) served as the outcome measure. Places where people stayed were classified as stable (own apartment, social housing, or staying with one’s family if this could be maintained for ≥6 months) or unstable. Thus, all other housing situations, for this purpose, were deemed unstable.

    Perspective of the Economic Analysis

    As discussed in detail elsewhere,13 cost elements were collected and analyzed from the perspective of society.15 We modified this perspective slightly, following Weisbrod et al,16 in that we included social assistance and disability benefits as costs. This modified societal perspective may be viewed as consistent with a social cost impact analysis.17,18

    Calculation of Costs per Individual

    We calculated many unit costs at a high level of specificity, distinguishing, for example, among supportive housing providers with different staffing levels. Whenever possible, we used financial statements and activity reports to estimate a fully allocated average cost of each service.19 The unit costs that we used and the methods that we used to derive them have been published.13

    Unit costs for the intervention were based on reported expenses of each clinical team and housing provider. Program expenses were distributed among participants based on their own time receiving services from their clinical team, as estimated using the Health Services and Justice Services Use questionnaire, and on the number of nights that they had a subsidized apartment or housing unit provided by the project.

    All unit costs were originally in 2011 Canadian dollars or adjusted to 2011 Canadian dollars. For this analysis, we used the city-specific Consumer Price Index to convert costs into 2016 Canadian dollars.20 We calculated costs per individual by multiplying frequencies by the corresponding unit cost, including the intervention cost for experimental group participants, adding to that social assistance and other contributions by society to their income, and finally subtracting income earned.13

    Discounting

    For each participant, costs as well as days of stable housing were estimated for a 2-year period. Costs and days of stable housing in the second year were discounted at a 3% rate, a common rate for a base case analysis.19

    Statistical Analysis

    All analyses used multiple imputation with chained equations (20 imputations) to account for missing data.21 Mean costs per year after randomization, aggregated across sites but grouped into different categories,13 were compared between the HF and TAU groups at baseline and during each of the 2 years after baseline. Mean total costs per year were then compared site-by-site between the HF plus ICM and TAU groups.

    Confidence intervals for incremental cost-effectiveness ratios were computed via bootstrapping, with 500 bootstrap resamples.21,22 We plotted the bootstrap resamples on the cost-effectiveness plane.

    We used the net-benefit approach to describe further the effect of sampling uncertainty.19 The intervention is deemed cost-effective if λμΔE – μΔC > 0, where λ is the threshold ratio (in dollars per additional day of stable housing) above which the decision-maker no longer finds the intervention cost-effective; μΔE, the mean difference in effectiveness between the 2 groups; and μΔC, the mean difference in costs. Using the bootstrap resamples, we plotted the cost-effectiveness acceptability curve, showing the estimated probability that the intervention is cost-effective as a function of λ.

    We then regressed, using values of λ ranging from $0 to $100, each individual’s net monetary benefit on several variables selected a priori as potentially relevant, including group assignment, site, age, sex, presence of psychotic disorder, Multnomah Community Ability Scale score, duration of longest previous episode of homelessness, and number of hospital days in the year before study entry. Linear regression was used in each case. To evaluate how participant characteristics might mediate the cost-effectiveness of HF,19,23 and in the absence of any strong a priori hypotheses about which characteristics might be relevant, we then tested, one by one, interactions between all these variables and the group assignment variable. Interaction terms with 2-sided P < .10 were retained for a final model with interactions. Fitted models were checked for misspecification by plotting the residuals against the fitted value of the dependent variable as well as continuous covariates. The Rubin rule was used to derive 95% CIs.21 Statistical analyses were performed using Stata, version 15 (StataCorp).

    Sensitivity Analyses

    We tested the robustness of the results to the choice of discount rate by using 0% and 5% instead of 3%. We also checked the effects of adjusting for baseline differences in costs using a regression-based method24 and performed a 2-way sensitivity analysis on these factors.

    Results

    Of 1198 individuals originally randomized (795 [66.4%] male, 390 [32.6%] female, and 13 [1.1%] other; 696 [58.1%] aged 30 to 49 years), 1160 (96.8%) provided usable data for this analysis. eFigure 1 in the Supplement describes the flow of participants into and through the trial and shows the available sample size by group and by site.

    Table 1 provides descriptive statistics for the sample at baseline. Values for other variables not used in this analysis have been reported elsewhere.7 Their mean (SD) longest period of homelessness was 29.0 (42.6) months (median, 12 months [interquartile range, 5-36 months]).

    Table 2 shows baseline and first- and second-year costs by type of cost for the HF and TAU groups. During the 2-year follow-up period, meaningful cost offsets (mean reductions in costs attributable to the intervention) were observed for shelters (−$2627; 95% CI, −$3232 to −$2079), substance use treatment (−$1148; 95% CI, −$1658 to −$638), supportive housing (−$1861; 95% CI, −$2540 to −$1222), and ambulatory visits (−$2375; 95% CI, −$3226 to −$1523). For other cost categories, the 95% CIs for offsets for other cost categories included zero, or the point estimate was less than $1000. Excluding the intervention cost, the total mean cost offset was −$6629 (95% CI, −$10 199 to −$2969). However, after including the mean cost of the intervention ($14 496, inferred from Table 2), the mean total cost for HF participants exceeded that for TAU by $7868 (95% CI, $4409-$11 405). Thus, 46% of the cost of the intervention was offset. For most services as well as in total, the cost difference was less favorable to HF in the second year than in the first.

    Table 3 disaggregates costs by site rather than by cost category. The magnitude of the net cost, including the cost of the intervention, ranged from $5218 (95% CI, −$983 to $11 385) per person per year in Montreal to $11 702 (95% CI, $5196-$18 873) per person per year in Toronto.

    Days with stable housing were higher by 140.34 (95% CI, 128.14-153.31) days in the HF group, with a cost difference of $7867.73 (95% CI, $4408.81-$11 404.79). Thus, the incremental cost-effectiveness ratio was $56.08 (95% CI, $29.55-$84.78) per day of stable housing.

    eFigure 2 in the Supplement shows 500 bootstrap replicates of mean incremental cost and the corresponding mean incremental number of days of stable housing on the cost-effectiveness plane. All the points lie in the quadrant corresponding to higher effectiveness and higher costs, indicating that taking all sites together, the intervention unambiguously increases days of stable housing and costs.

    The cost-effectiveness acceptability curve shown in the Figure indicates that if the decision-maker is willing to pay $67 per night of stable housing, there is an 80% chance that HF is cost-effective compared with TAU. If the decision-maker is willing to pay approximately $100 per day of stable housing, then the probability that the intervention is cost-effective increases to 100%.

    eTable 1 in the Supplement shows the results of net benefit regressions that do not include interactions. As the decision-maker’s willingness to pay for an additional day of stable housing (represented by λ) rises from $0 to $100, the adjusted net benefit of receiving HF is initially negative (net cost of −$8604 per person per year; 95% CI, −$12 027 to −$5181) but increases quickly so that at $100 the net benefit is positive, reaching $5269 (95% CI, $1352-$9186). Only 1 other variable appeared to be associated with a meaningful difference in net benefit: people who had been arrested or incarcerated in the 6 months before baseline had a lower net benefit. Age, sex, alcohol or substance abuse or dependence, and level of functioning were not associated with net benefit, regardless of λ, after adjusting for site and the other factors.

    Table 4 shows the results of adding interactions between group assignment and the variables identified using the procedure described above. None of the site variables or interactions with site were meaningfully different from zero, suggesting that the cost-effectiveness of HF with ICM did not vary by site. Costs appeared to vary with arrest history; the addition of the interaction terms had little effect on estimated coefficients. A higher level of functioning was associated with a higher net benefit at higher levels of λ: at λ = $100, a (clinically meaningful) 10-point increase in the Multnomah Community Ability Scale score was associated with an increase in net benefit of $6901 (95% CI, $1839-$11 962) per person per year, indicating that a higher level of functioning was associated with more days of stable housing. The results suggest that HF yielded a net benefit lower by approximately $7000 for people who had had 2 or more hospitalizations for mental illness during a 1-year period during the previous 5 years; the amount varied from −$6820 (95% CI, −$12 673 to −$967) at λ = 0 to −$7456 (95% CI, −$14 065 to −$847) at λ = 100. None of the other interaction terms appeared to meaningfully alter costs at any value of λ. Thus, no individual-level baseline variable, except possibly hospitalization history, and no site appeared to make HF with ICM more or less cost-effective.

    Sensitivity analyses shown in eTable 2 in the Supplement indicate that our results are robust to changes in the discount rate and only somewhat sensitive to the adjustment for baseline differences or a combination of both. Adjusting for baseline differences increases the incremental cost-effectiveness ratio from $56.08 to $60.18. Changes in the discount rate have a minimal effect. The largest change is obtained by adjusting for baseline differences, without altering the discount rate: the incremental cost-effectiveness ratio becomes $60.18 (95% CI, $35.27-$86.95).

    Discussion

    This cost-effectiveness analysis relies on data from the largest trial of scattered-site HF with ICM for people with mental illness and moderate needs, to our knowledge, conducted to date. The intervention costs a mean of $14 496 per person per year. Cost offsets on a wide range of other services reduced the net cost to $7868, a 46% reduction. An additional day of stable housing cost $56.08. Cost-effectiveness seemed to be about the same regardless of participant characteristics, with the possible exception of hospitalization history.

    In previous reports of the At Home/Chez Soi study,7,25 the intervention cost was reported as Can$14 177 per participant annually (in 2011 dollars), and the mean net cost offset was Can $4849, or 34% of the cost of the intervention. Although qualitatively similar, the numerical estimates presented herein differ from the earlier ones for several reasons. Most important, in the present study, we did not adjust for baseline differences in costs, whereas the earlier reports used a relatively simple difference-in-differences method, applied to mean costs per person per site. Second, we allocated the cost of the intervention to individual study participants. Third, we refined several unit cost calculations compared with the earlier report.

    Cost-effectiveness analyses, including for the treatment of mental health conditions, often use quality-adjusted life-years as an outcome measure,19 despite their known limitations.26 In the present report, consistent with the only published cost-effectiveness analysis of a related intervention published to date,5 we used days of stable housing instead. Hierarchical linear models estimated on the EQ-5D, a common instrument from which to derive quality-adjusted life-years,27 showed no significant difference between the experimental and TAU groups.7 This finding is not surprising because HF is in significant measure a social care intervention, with health effects that may only become apparent during a longer period. Qualitative interviews conducted on a 10% sample of study participants showed that participants in the HF group were much more likely to experience positive changes in their life trajectories than those in the TAU group during the 2 years of the study.28

    The only comparable cost-effectiveness analysis in the literature5 was performed at Veterans Affairs medical centers in 4 US cities using data collected in the early to middle 1990s. The most similar intervention in that study was a less intensive form of case management than herein, with as many as 25 participants per case manager. This intervention was combined with Section 8 vouchers (rent subsidies). Fully half of participants in that study5 had no psychiatric diagnosis other than alcohol or drug use disorder. The study found that, from a societal perspective, the intervention cost $45 more per additional day of housing, compared with standard care. Adjusting for the effects of inflation29 and using purchasing power parity in 2016 to convert US into Canadian dollars,30 that amount is equivalent to about Can$79. Our somewhat lower estimate may reflect the greater potential for cost offsets with a group of participants with greater illness.

    In this trial, careful attention was paid to implementation fidelity.3 Fidelity was found to be fair to excellent across sites.12 Other work has reported an association between higher fidelity and housing stability, quality of life, and community functioning31 as well as other positive outcomes.32,33 Not all HF implementations are as careful to follow the Pathways model as those of the At Home/Chez Soi trial; for instance, caseloads may be increased or rent supplements may be reduced. Although such changes will reduce the cost of the intervention itself, the magnitude of the cost offsets observed herein may not remain the same.

    The fact that HF with ICM did not dominate the intervention—that it did not prove more effective and less costly—does not mean that it should not be implemented. Most health and social interventions do not pay for themselves. Rather, they yield benefits judged sufficient to merit their cost. The cost of the intervention itself, which was approximately $40 per participant per day, is well within the range of the costs of many currently funded forms of emergency shelter and supported housing.13 Allowing for some budgetary reallocations and reflecting the reductions in costs of these and other services in which the delivery of HF with ICM results, the overall budgetary impact of a significant expansion of HF with ICM capacity is likely to be quite reasonable.

    Our study found little evidence of HF being more or less cost-effective for different subgroups. This finding may be attributable to the fact that clinical teams adjust the nature and intensity of their interventions according to each participant’s needs and preferences. Cost-effectiveness may have been lower for people with more previous hospitalizations for mental illness, possibly because the teams expended more effort on clients more prone to prior hospitalizations but could not sufficiently alter participants’ propensity to be hospitalized more often. Our findings give no ground for any selection of participants on characteristics such as current alcohol or substance abuse or previous justice services involvement.

    Strengths and Limitations

    Our study presents several strengths. The sample size was large and drawn from several sites. It evaluated a carefully defined and implemented intervention. Attrition, and in particular differential attrition, was modest. Service use was measured in a much more comprehensive way than is typical in cost-effectiveness studies. Unit costs were carefully estimated.

    Several limitations need to be noted as well. Service use data were based on participant self-reports. Although these reports are subject to recall biases, the most costly components of costs are based on self-reports gathered at 3-month intervals, which have been shown to have good validity.34 The validity of self-reports in the At Home/Chez Soi study has also been corroborated directly.35,36 The costs of medications were not included, owing to the difficulty of obtaining reliable and sufficiently detailed information on medications taken by participants by questionnaire and restrictions on sharing of participant-level administrative data, including on medication use, across provinces. The ICM teams, however, do not typically focus on increasing medication adherence. The follow-up period was 2 years, and we do not know how cost-effectiveness would evolve for a longer duration. We did not estimate the administrative costs of transfer payments,15 but because most participants in both groups received social assistance payments, doing so would have had little effect.

    Conclusions

    In this large, multiple-site trial, we found that about half of the costs of an HF with ICM intervention were offset by reductions in the costs of shelters, ambulatory visits, emergency department visits, and other services. Even if these cost offsets do not result in reductions in budgetary outlays, they translate into greater availability of resources for others in need. The net cost was modest in relation to current expenditures on individuals who are homeless. Furthermore, we found little evidence of cost-effectiveness varying according to participant characteristics. These results support more widespread implementation of scattered-site HF with ICM programs for homeless people with characteristics comparable to those of the participants included in this trial.

    Back to top
    Article Information

    Accepted for Publication: June 28, 2019.

    Published: August 21, 2019. doi:10.1001/jamanetworkopen.2019.9782

    Correction: This article was corrected on September 11, 2019, to fix 2 incorrect numbers in eFigure 1 in the Supplement.

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Latimer EA et al. JAMA Network Open.

    Corresponding Author: Eric A. Latimer, PhD, Perry 3C, Douglas Research Centre, 6875 LaSalle Blvd, Montreal, QC H4H 1R3, Canada (eric.latimer@mcgill.ca).

    Author Contributions: Dr Latimer and Ms Cao had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Latimer, Ly, Adair, Sareen, Somers, Stergiopoulos, Pinto.

    Acquisition, analysis, or interpretation of data: Latimer, Rabouin, Cao, Powell, Adair, Sareen, Somers, Stergiopoulos, Moodie, Veldhuizen.

    Drafting of the manuscript: Latimer.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Latimer, Rabouin, Cao, Powell, Moodie.

    Obtained funding: Latimer, Sareen, Somers, Stergiopoulos.

    Administrative, technical, or material support: Latimer, Ly, Adair, Sareen, Somers, Stergiopoulos, Pinto, Veldhuizen.

    Supervision: Latimer, Adair, Somers, Moodie.

    Conflict of Interest Disclosures: Dr Adair reported being engaged as a dedicated researcher at the national level for the duration and being paid by the study, not her academic institution, to devote time to the study; her role was the same as the academic researchers on the team and she retained academic freedom in all aspects. Dr Pinto reported support as a clinician scientist by the Department of Family and Community Medicine, Faculty of Medicine at the University of Toronto, St Michael’s Hospital, and the Li Ka Shing Knowledge Institute, St Michael’s Hospital; support by a fellowship from the Physicians’ Services Incorporated Foundation; and serving as the Associate Director for Clinical Research at the University of Toronto Practice-Based Research Network. No other disclosures were reported.

    Funding/Support: This study was supported by a financial contribution from Health Canada to the Mental Health Commission of Canada.

    Role of the Funder/Sponsor: The Mental Health Commission of Canada oversaw the conduct of the study, including ensuring that data collection timelines were met, and helped organize the management of the data set until it was complete. It also provided training and technical support to the service teams and research staff throughout the project. However, the sponsor had no role in the analysis and interpretation of the data and preparation, review, or approval of the manuscript, or in the decision to submit it for publication.

    Disclaimer: The views expressed herein solely represent those of the authors.

    Group Members: The following investigators participated in At Home/Chez Soi: National Team: Carol E. Adair, PhD; Tim Aubry, PhD; Paula N. Goering, RN, PhD; Geoffrey Nelson, PhD; Myra Piat, PhD; Sanjeev Sridharan, PhD; David Streiner, PhD; and Sam Tsemberis, PhD. Moncton: Saïd Bergheul, PhD; Jimmy Bourque, PhD; Paul-Émile Bourque, PhD; Pierre-Marcel Desjardins, PhD; Stefanie R. LeBlanc, MA; Danielle Nolin, PhD; Sarah Pakzad, PhD; and John Sylvestre, PhD. Montreal: Jean-Pierre Bonin, PhD; Anne G. Crocker, PhD; Henri Dorvil, PhD; Marie-Josée Fleury, PhD; Roch Hurtubise, PhD; Eric A. Latimer, PhD; Alain Lesage, MD; Christopher McAll, PhD; Catherine Vallée, PhD; and Helen-Maria Vasiliadis, PhD. Toronto: Ahmed M. Bayoumi, MD; Stephen W. Hwang, MD; Bonnie Kirsh, PhD; Maritt Kirst, PhD; Kwame McKenzie, MD; Rosane Nisenbaum, PhD; Patricia O’Campo, PhD; and Vicky Stergiopoulos, MD. Winnipeg: James M. Bolton, MD; Daniel G. Chateau, PhD; Jino Distasio, PhD; Murray W. Enns, MD; Laurence Y. Katz, MD; Patricia J. Martens, PhD; Jitender Sareen, MD; and Mark J. Smith, MSc. Vancouver: James Frankish, PhD; Michael R. Krausz, MD, PhD; Lawrence McCandless, PhD; Akm Moniruzzaman, PhD; Tonia L. Nicholls, PhD; Anita Palepu, MD; Michelle L. Patterson, PhD; Christian G. Schutz, MD, PhD; Julian M. Somers, PhD; Verena Strehlau, MD; and Denise Zabkiewicz, PhD.

    Additional Contributions: Jayne Barker, PhD (2008-2011), Cameron Keller, MC (2011-2012), and Catharine Hume, MHSc (2012-2014), served as the Mental Health Commission of Canada At Home/Chez Soi National Project Leads. We thank the National Research Team, the 5 site research teams, the site coordinators, and the numerous service and housing providers, as well as persons with lived experience, who contributed to this project and the research. The individuals named above received compensation from the Mental Health Commission of Canada for their role in the study. Paula N. Goering, RN, PhD, was the lead investigator for the At Home/Chez Soi project and as such led the design of the overall study and the process of data collection and contributed to the design of the economic analysis before her death May 26, 2016.

    Additional Information: The data set, code book and Stata syntax used for analysis are available to other investigators, on certain conditions related to protection of confidentiality and acknowledgment of contributions of those who prepared the data set, as well as with the acceptance of technical support provided at cost. (We judge such technical support essential to correct interpretation and analysis of the data.) Contact the corresponding author for details.

    References
    1.
    Fazel  S, Khosla  V, Doll  H, Geddes  J.  The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis.  PLoS Med. 2008;5(12):e225. doi:10.1371/journal.pmed.0050225PubMedGoogle ScholarCrossref
    2.
    Aubry  T, Nelson  G, Tsemberis  S.  Housing First for people with severe mental illness who are homeless: a review of the research and findings from the At Home–Chez Soi demonstration project.  Can J Psychiatry. 2015;60(11):467-474. doi:10.1177/070674371506001102PubMedGoogle ScholarCrossref
    3.
    Tsemberis  S.  Housing First: The Pathways Model to End Homelessness for People with Mental Illness and Addiction Manual. Center City, MN: Hazelden; 2010.
    4.
    Ly  A, Latimer  E.  Housing first impacts on costs and associated cost offsets: a review of the literature.  Can J Psychiatry. 2015;60(11):475-487. doi:10.1177/070674371506001103PubMedGoogle ScholarCrossref
    5.
    Rosenheck  R, Kasprow  W, Frisman  L, Liu-Mares  W.  Cost-effectiveness of supported housing for homeless persons with mental illness.  Arch Gen Psychiatry. 2003;60(9):940-951. doi:10.1001/archpsyc.60.9.940PubMedGoogle ScholarCrossref
    6.
    Goering  PN, Streiner  DL, Adair  C,  et al.  The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomised controlled trial of a Housing First intervention for homeless individuals with mental illness in five Canadian cities.  BMJ Open. 2011;1(2):e000323. doi:10.1136/bmjopen-2011-000323PubMedGoogle ScholarCrossref
    7.
    Stergiopoulos  V, Hwang  SW, Gozdzik  A,  et al; At Home/Chez Soi Investigators.  Effect of scattered-site housing using rent supplements and intensive case management on housing stability among homeless adults with mental illness: a randomized trial.  JAMA. 2015;313(9):905-915. doi:10.1001/jama.2015.1163PubMedGoogle ScholarCrossref
    8.
    Aubry  T, Goering  P, Veldhuizen  S,  et al.  A randomized controlled trial in five Canadian cities of the effectiveness of Housing First with assertive community treatment for persons with serious mental illness and a history of homelessness.  Psychiatr Serv. 2016;67(3):275-281. doi:10.1176/appi.ps.201400587PubMedGoogle ScholarCrossref
    9.
    Husereau  D, Drummond  M, Petrou  S,  et al; CHEERS Task Force.  Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.  BMJ. 2013;346:f1049. doi:10.1136/bmj.f1049PubMedGoogle ScholarCrossref
    10.
    Dickerson  FB, Origoni  AE, Pater  A, Friedman  BK, Kordonski  WM.  An expanded version of the Multnomah Community Ability Scale: anchors and interview probes for the assessment of adults with serious mental illness.  Community Ment Health J. 2003;39(2):131-137. doi:10.1023/A:1022610620391PubMedGoogle ScholarCrossref
    11.
    Frane  JW.  A method of biased coin randomization, its implementation, and its validation.  Drug Inf J. 1998;32:423-432. doi:10.1177/009286159803200213Google ScholarCrossref
    12.
    Macnaughton  E, Stefancic  A, Nelson  G,  et al.  Implementing Housing First across sites and over time: later fidelity and implementation evaluation of a pan-Canadian multi-site Housing First program for homeless people with mental illness.  Am J Community Psychol. 2015;55(3-4):279-291. doi:10.1007/s10464-015-9709-zPubMedGoogle ScholarCrossref
    13.
    Latimer  E, Rabouin  D, Cao  Z,  et al; At Home/Chez Soi Investigators.  The costs of usual services for homeless people with mental illness in five Canadian cities: results from a large prospective follow-up study.  Canadian Medical Association Journal Open. 2017;5(3):E576-E85.PubMedGoogle Scholar
    14.
    Lecrubier  Y, Weiller  E, Amorim  P, Bonora  I, Sheehan  KH, Janavs  J.  The Mini International Neuropsychiatric Interview (MINI): a short diagnostic structured interview: reliability and validity according to the CIDI.  Eur Psychiatry. 1997;12(5):224-231. doi:10.1016/S0924-9338(97)83296-8Google ScholarCrossref
    15.
    Frisman  L, Rosenheck  R.  How transfer payments are treated in cost-effectiveness and cost-benefit analyses.  Adm Policy Ment Health. 1996;23(6):533-546. doi:10.1007/BF02108689Google ScholarCrossref
    16.
    Weisbrod  BA, Test  MA, Stein  LI.  Alternative to mental hospital treatment, II: economic benefit-cost analysis.  Arch Gen Psychiatry. 1980;37(4):400-405. doi:10.1001/archpsyc.1980.01780170042004PubMedGoogle ScholarCrossref
    17.
    Salkever  D.  Social costs of expanding access to evidence-based supported employment: concepts and interpretive review of evidence.  Psychiatr Serv. 2013;64(2):111-119. doi:10.1176/appi.ps.201100499PubMedGoogle ScholarCrossref
    18.
    Meltzer  D.  Accounting for future costs in medical cost-effectiveness analysis.  J Health Econ. 1997;16(1):33-64. doi:10.1016/S0167-6296(96)00507-3PubMedGoogle ScholarCrossref
    19.
    Drummond  MF, Sculpher  MJ, Claxton  K, Stoddart  GL, Torrance  GW.  Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford, UK: Oxford University Press; 2015.
    20.
    Statistics Canada. Consumer Price Index, monthly, not seasonally adjusted. Table 326-0020. http://www5.statcan.gc.ca/cansim/a26?lang=eng&retrLang=eng&id=3260020&&pattern=&stByVal=1&p1=1&p2=37&tabMode=dataTable&csid=#customizeTab. Modified July 14, 2019. Accessed November 17, 2016.
    21.
    Azur  MJ, Stuart  EA, Frangakis  C, Leaf  PJ.  Multiple imputation by chained equations: what is it and how does it work?  Int J Methods Psychiatr Res. 2011;20(1):40-49. doi:10.1002/mpr.329PubMedGoogle ScholarCrossref
    22.
    Shao  J, Sitter  RR.  Bootstrap for imputed survey data.  J Am Stat Assoc. 1996;91(435):1278-1288. doi:10.1080/01621459.1996.10476997Google ScholarCrossref
    23.
    Hoch  JS, Briggs  AH, Willan  AR.  Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis.  Health Econ. 2002;11(5):415-430. doi:10.1002/hec.678PubMedGoogle ScholarCrossref
    24.
    van Asselt  AD, van Mastrigt  GA, Dirksen  CD, Arntz  A, Severens  JL, Kessels  AG.  How to deal with cost differences at baseline.  Pharmacoeconomics. 2009;27(6):519-528. doi:10.2165/00019053-200927060-00007PubMedGoogle ScholarCrossref
    25.
    Goering  P, Veldhuizen  S, Watson  A,  et al.  National At Home/Chez Soi Final Report. Calgary, AB: Mental Health Commission of Canada; 2014.
    26.
    Nord  E, Daniels  N, Kamlet  M.  QALYs: some challenges.  Value Health. 2009;12(1)(suppl 1):S10-S15. doi:10.1111/j.1524-4733.2009.00516.xPubMedGoogle ScholarCrossref
    27.
    Euroqol Group. EQ-5D. https://euroqol.org/. Accessed January 5, 2018.
    28.
    Nelson  G, Patterson  M, Kirst  M,  et al.  Life changes among homeless persons with mental illness: a longitudinal study of housing first and usual treatment.  Psychiatr Serv. 2015;66(6):592-597. doi:10.1176/appi.ps.201400201PubMedGoogle ScholarCrossref
    29.
    Bureau of Labor Statistics. Databases, Tables & Calculators by Subject: CPI Inflation Calculator. https://www.bls.gov/data/inflation_calculator.htm. Accessed November 17, 2016.
    30.
    Statistics Canada. Purchasing power parities, 2016. https://www150.statcan.gc.ca/n1/daily-quotidien/180424/dq180424b-eng.htm. Posted April 24, 2018. Accessed July 9, 2018.
    31.
    Goering  P, Veldhuizen  S, Nelson  GB,  et al.  Further validation of the pathways Housing First Fidelity Scale.  Psychiatr Serv. 2016;67(1):111-114. doi:10.1176/appi.ps.201400359PubMedGoogle ScholarCrossref
    32.
    Davidson  C, Neighbors  C, Hall  G,  et al.  Association of Housing First implementation and key outcomes among homeless persons with problematic substance use.  Psychiatr Serv. 2014;65(11):1318-1324. doi:10.1176/appi.ps.201300195PubMedGoogle ScholarCrossref
    33.
    Gilmer  TP, Stefancic  A, Henwood  BF, Ettner  SL.  Fidelity to the Housing First model and variation in health service use within permanent supportive housing.  Psychiatr Serv. 2015;66(12):1283-1289. doi:10.1176/appi.ps.201400564PubMedGoogle ScholarCrossref
    34.
    Bhandari  A, Wagner  T.  Self-reported utilization of health care services: improving measurement and accuracy.  Med Care Res Rev. 2006;63(2):217-235. doi:10.1177/1077558705285298PubMedGoogle ScholarCrossref
    35.
    Somers  JM, Moniruzzaman  A, Currie  L, Rezansoff  SN, Russolillo  A, Parpouchi  M.  Accuracy of reported service use in a cohort of people who are chronically homeless and seriously mentally ill.  BMC Psychiatry. 2016;16:41. doi:10.1186/s12888-016-0758-0PubMedGoogle ScholarCrossref
    36.
    Lemieux  AJ, Roy  L, Martin  MS, Latimer  EA, Crocker  AG.  Justice involvement among homeless individuals with mental illnesses: are self-report and administrative measures comparable?  Eval Program Plann. 2017;61(April):86-95. doi:10.1016/j.evalprogplan.2016.12.007PubMedGoogle ScholarCrossref
    ×