The probability associated with each branch is displayed underneath the branch. Clone 1 indicates branches emanating from an IQ of 70 or greater. Clone 2 indicates branches emanating from an IQ less than 70. Clone 3 indicates branches emanating from intensive behavioral intervention (IBI) starting age of younger than 4 years. Clone 4 indicates branches emanating from IBI starting age of 4 years or older. DFLYs indicates dependency-free life-years.
Cost (Can$1=US$0.78) and effectiveness (dependence-free life-years [DFLYs]) associated with current wait time (CWT), reduced wait time (RWT) (by half), and eliminated wait time (EWT) from the provincial perspective. Both CWT and RWT were dominated by EWT.
Tornado diagram ranking the expected value of uncertainty on the ICER for eliminated wait time (EWT) vs current wait time (CWT). Bar width represents variation in the ICER over the range of possible values for each variable shown in parentheses beside the variable name. The blue dashed line indicates a willingness to pay of Can$0, indicating that possible values for all parameters were still within the range of cost savings. IBI indicates intensive behavioral intervention; RWT indicates reduced wait time. Can$1=US$0.78.
Each of the 10 000 iterations of the probabilistic sensitivity analyses is represented by a blue dot. The 95% CIs of incremental cost and effectiveness results of eliminated wait time vs current wait time are indicated by the black elliptical line. The intersection of the dashed lines represents a willingness to pay (WTP) of Can$0 (Can$1=US$0.78). The percentages of the 10 000 simulations in each quadrant are displayed.
eFigure 1. Cost-effectiveness acceptability curve for the provincial perspective.
eFigure 2. Cost-effectiveness frontier for the societal perspective.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Piccininni C, Bisnaire L, Penner M. Cost-effectiveness of Wait Time Reduction for Intensive Behavioral Intervention Services in Ontario, Canada. JAMA Pediatr. 2017;171(1):23–30. doi:10.1001/jamapediatrics.2016.2695
What are the costs and effects on independence of decreasing the wait time for intensive behavioral intervention (IBI) for autism spectrum disorder (ASD) in Ontario, Canada?
A cost-effectiveness analysis modeled the lifetime costs and independence per person with ASD based on the current IBI wait time of 32 months, a wait time reduced by half, and no wait time. Eliminating the wait time resulted in more independence and substantial savings to the government (Can$53 000 per person with ASD) and society (Can$267 000 per person with ASD).
Improved access to IBI for ASD results in long-term substantial savings to governments and society.
Earlier access to intensive behavioral intervention (IBI) is associated with improved outcomes for children with severe autism spectrum disorder (ASD); however, there are long waiting times for this program. No analyses have been performed modeling the cost-effectiveness of wait time reduction for IBI.
To model the starting age for IBI with reduced wait time (RWT) (by half) and eliminated wait time (EWT), and perform a cost-effectiveness analysis comparing RWT and EWT with current wait time (CWT) from government and societal perspectives.
Design, Setting, and Participants
Published waiting times were used to model the mean starting age for IBI for CWT, RWT, and EWT in children diagnosed with severe ASD who were treated at Ontario’s Autism Intervention Program. Inputs were loaded into a decision analytic model, with an annual discount rate of 3% applied. Incremental cost-effectiveness ratios (ICERs) were determined. One-way and probabilistic sensitivity analyses were performed to assess the effect of model uncertainty. We used data from the year 2012 (January 1 through December 31) provided from the Children’s Hospital of Eastern Ontario IBI center for the starting ages. Data analysis was done from May through July 2015.
Main Outcomes and Measures
The outcome was independence measured in dependency-free life-years (DFLYs) to 65 years of age. To derive this, expected IQ was modeled based on probability of early (age <4 years) or late (age ≥4 years) access to IBI. Probabilities of having an IQ in the normal (≥70) or intellectual disability (<70) range were calculated. The IQ strata were assigned probabilities of achieving an independent (60 DFLYs), semidependent (30 DFLYs), or dependent (0 DFLYs) outcome. Costs were calculated for provincial government and societal perspectives in Canadian dollars (Can$1 = US$0.78).
The mean starting ages for IBI were 5.24 years for CWT, 3.89 years for RWT, and 2.71 years for EWT. From the provincial government perspective, EWT was the dominant strategy, generating the most DFLYs for Can$53 000 less per individual to 65 years of age than CWT. From the societal perspective, EWT produced lifetime savings of Can$267 000 per individual compared with CWT. The ICERs were most sensitive to uncertainty in the starting age for IBI and in achieving a normal IQ based on starting age.
Conclusions and Relevance
This study predicts the long-term effect of the current disparity between IBI service needs and the amount of IBI being delivered in the province of Ontario. The results suggest that providing timely access optimizes IBI outcomes, improves future independence, and lessens costs from provincial and societal perspectives.
Autism spectrum disorder (ASD) is a neurodevelopmental disorder with core symptoms of impairment in social interaction and restricted interests and behaviors.1 During the past 2 decades, increasing rates of ASD diagnosis have been reported, causing the estimated prevalence in North America to increase to 1 in 68 children.2 As this population continues to increase, more children with ASD are placed on waiting lists, hindering access to ASD interventions.
The delivery of intensive behavioral intervention (IBI) has been affected by the increase in the population with ASD. Intensive behavioral intervention is an ASD therapy based on the principles of applied behavior analysis and is delivered one on one by a trained therapist for 20 hours or more weekly.3 The goals of IBI are to facilitate skill acquisition, remove barriers to learning, and increase quality of life.4 Early intervention with IBI during the preschool years is effective in children with ASD for improvement of developmental trajectory and maximization of individual potential.3,5 Accessing IBI and other behavioral therapies at younger ages has been associated with significantly better treatment outcomes compared with access at older ages.6-9
The Canadian province of Ontario offers the publicly funded Autism Intervention Program for children with ASD “toward the severe end of the spectrum,”10(p 8) with each site determining participant eligibility based on clinical impressions of ASD severity. The system in Ontario is relevant in a broader context as an example of a publicly funded system that provides community-based IBI services. As part of this program, children receive IBI for a mean of 23 hours each week for a mean of 2 years.10 In an evaluation of this program, children who began IBI before 4 years of age had lower autism symptom severity scores, higher scores on measures of adaptive skills, and higher IQ gains at the end of treatment compared with children who began IBI at older than 4 years.6
The mean wait time for IBI across Ontario is 2.7 years, delaying access to intervention for thousands of children.11 The waiting lists are maintained in chronological order based on date of referral to the program.10 Increases in waiting lists continually surpass increases in program funding, such that most eligible children start IBI months or years after it is considered most effective, with approximately 75% of eligible children beginning IBI at older than 6 years.12
In addition to costs directly associated with intervention, ASD comes with high costs over the lifetime, particularly in educational and social supports.13 The per capita lifetime cost to society for an individual with ASD has been estimated to exceed $3 million.13-15 Improving outcomes via increased early access to treatment is hypothesized to reduce the need for services and supports later in life. In 2006, a cost-effectiveness analysis16 evaluated the cost-effectiveness of expanding Ontario’s Autism Intervention Program to all children with ASD in the province and reported cost savings with program expansion. A more recent cost-effectiveness analysis compared prediagnosis intervention with Ontario’s Autism Intervention Program and found the potential for lifetime cost savings with the addition of prediagnosis interventions.17 To our knowledge, no studies have evaluated the lifetime cost-effectiveness of reducing wait times for IBI to yield earlier access for the eligible population of children with more severe ASD. Such evidence is urgently needed to inform policy decisions for governments considering investing in this population.
This analysis had 2 objectives. The first was to determine the change in mean starting age for IBI in Ontario with wait time reduction (by half) and elimination. The second was to perform a cost-effectiveness analysis comparing the lifetime costs and projected dependency-free life-years (DFLYs) for reduced wait time (RWT) and eliminated wait time (EWT) compared with current wait time (CWT) from provincial and societal perspectives.
We used data from the year 2012 (January 1 through December 31) provided from the Children’s Hospital of Eastern Ontario IBI center for the starting ages. Data analysis was done from May through July 2015. Study investigators consulted with the research ethics board at the Children's Hospital of Eastern Ontario about the use of age distribution data from their IBI program. The board confirmed that research ethics approval was not necessary for the use of these data.
Reported wait time statistics for each of the 9 ASD service delivery regions (Surrey Place Centre, Toronto; Kinark Child and Family Services, Markham; McMaster Children’s Hospital, Hamilton; Children’s Hospital of Eastern Ontario, Ottawa; Child and Community Resources, Sudbury; Thames Valley Children’s Centre, London; Hands–The Family Help Network, North Bay; Pathways for Children and Youth, Kingston; ErinoakKids Centre for Treatment and Development, Mississauga [taken from https://www.thestar.com/news/gta/2012/11/23/the_autism_project_wait_times.html]) in Ontario, Canada, were used to calculate the mean CWT for IBI across the province.11 This mean was weighted by the proportion of the eligible population in each region using the total numbers of children waiting for and receiving IBI in each region. For RWT, CWT was halved. For EWT, the wait time was reduced to 2 months, which was a conservative allowance for the period between determination of eligibility and initiation of IBI.
Data for the starting age for IBI were obtained from the Eastern Ontario region and were used to determine the mean starting age for IBI under each of the 3 wait-time scenarios. A normal distribution was fit to the frequency distribution of ages of the 73 children beginning IBI in 2012 (mean [SD] age, 5.2 [0.9] years), which corresponded to the year of the most recently published data on provincial wait times.11 For subsequent calculations, it was assumed that the ages of children on the waiting list would remain normally distributed as the wait time was reduced. Mean age at referral was calculated as the difference between the reported mean IBI starting age in 2012 and CWT. Assuming that the mean age at referral would remain constant with changes in wait time, the mean starting ages for IBI with RWT and EWT were determined by adding this age to the length of the reduced mean wait time and 2 months, respectively.
This study used a cost-effectiveness analysis to compare the relative costs and effects of 3 possible wait-time lengths (CWT, RWT, and EWT) in the provincial and societal context.
The target population for this study was children with severe ASD determined eligible to receive IBI through Ontario’s Autism Intervention Program. Children with mild and moderate ASD were excluded from the models.
Costs were determined from the perspectives of the provincial government and society. Provincial government costs included public funding for education, health, and social services. The societal perspective is a broader perspective that adds costs to families, including costs associated with caregiving and lost productivity.
The outcome modeled was independence measured in DFLYs, previously defined as years of life with a similar level of independence as would be expected from a typically developing individual.17 Predictions for stratification into the final outcome categories were based on posttreatment IQ scores for the younger and older age groups.6 Specifically, the probability of achieving an IQ score greater than 70 was assigned for each age group and then a probability of achieving an independent (60 DFLYs), semidependent (30 DFLYs), or dependent outcome (0 DFLYs) was assigned for each IQ group.16,17 IQ was used because it is the only available post-IBI outcome measure that has been reported to predict future dependence in the literature.18,19
The time horizon over which costs and DFLYs were evaluated was from the mean age at determination of eligibility for IBI to 65 years of age as reported previously in the literature.16,17,20
A complete breakdown of sources used to determine resource use and costs attributable to ASD is given in the Table. All costs are in Canadian dollars, which is the currency to be used throughout this article (Can$1 = $US0.78). The cost of IBI was valued at $56 000 per child per year, with regional variation from $50 800 to $67 000, which was the cost reported in 2013.12 Intensive behavioral intervention was assumed to have a mean duration of 2 years. Other costs to the provincial government included ASD-related health care costs to 65 years of age for all individuals and additional costs of transition services, special education, supported employment, the Special Services at Home program (which provides funding for children with developmental needs), the Passport program for semidependent and dependent individuals (which provides funding for adults with developmental disabilities), and respite care for dependent individuals.12,16,21,23 Only those with a semidependent or dependent outcome were assumed to participate in special education and receive adult developmental disability supports.
The societal perspective also factored in the estimated costs of productivity loss and caregiving. The major source of lost productivity for individuals in the target population is inability to participate in the workforce for dependent individuals and limited ability for semidependent individuals.27,28 As in previous analyses,16,17 productivity loss was obtained based on the mean national wage and projected working hours per week for each outcome with dependent individuals assumed not to work, semidependent individuals assumed to be employed for 20 hours per week, and independent individuals assumed to be employed for 40 hours per week. Caregiver costs were valued based on the mean hourly salary of a social assistant and total estimated hours of caregiving for each outcome.17,20,25 The societal perspective excluded funds that were directly transferred from the government to families, namely, the costs of the Special Services at Home program and supported employment.
An annual discount rate of 3% was applied to costs and DFLYs occurring in the future. Results were also analyzed using discount rates ranging from 0% to 5% in 1-way sensitivity analyses.
Model inputs (Table) were gathered based on best available evidence and loaded into decision-analytic models (Figure 1) comparing cost-effectiveness of RWT and EWT from both perspectives using TreeAge Pro software, version 2015 (TreeAge Software Inc). Probabilities of accessing IBI before 4 years of age were derived from normal distributions, with the mean starting ages for each wait time scenario and an SD of 11.2 months taken from the normal distribution of ages of children beginning IBI in 2012. Cost-effectiveness frontiers were created for each analysis, and incremental cost-effectiveness ratios (ICERs) were calculated for each strategy from both perspectives.
The robustness of model outputs to uncertainty in key parameters was assessed using 1-way and probabilistic sensitivity analyses (PSAs). One-way sensitivity analyses evaluated model outputs in response to high and low parameter values, which were derived from 95% CIs for all parameters (unless otherwise indicated) (Table). The influence of each input in determining variation in model outputs is shown in Tornado diagram. The PSAs were performed using 10 000 Monte Carlo simulations to demonstrate the effects of joint parameter uncertainty under the assumptions of normal distributions for IBI starting age and post-IBI IQ and β-distributions for other probabilities related to effectiveness of the intervention. The PSA results are presented as incremental cost-effectiveness scatterplots.
The mean starting ages were determined to be 5.24 years for the CWT scenario, 3.89 years for the RWT scenario, and 2.71 years for the EWT scenario. Assuming a normal distribution for probability of accessing IBI before 4 years of age, the mean ages for each scenario corresponded to 9.6% accessing IBI before 4 years of age with CWT, 54.8% with RWT, and 91.8% with EWT.
From the perspective of the provincial government, the overall findings revealed decreasing total costs and increasing effectiveness as the probability of early access to IBI increased. These results are presented on a cost-effectiveness frontier in Figure 2. The mean discounted cost of CWT was $742 488 per individual with 3.75 DFLYs generated to 65 years of age. Halving the wait time (RWT) decreased the cost to $713 182 and increased the number of DFLYs generated to 5.03. The EWT scenario had the lowest cost and highest effectiveness, costing $689 512 and generating 6.17 DFLYs. Thus, the EWT scenario was the dominant strategy. The negative ICERs are indicative of absolute dominance, which occurs when a strategy is more effective and less costly than the others. The EWT scenario resulted in savings of $52 976 per individual over a lifetime compared with the CWT scenario.
In 1-way sensitivity analyses, the calculated ICERs were most influenced by uncertainty relating to the probability of achieving a posttreatment IQ score greater than 70 based on IBI starting age, the probability of achieving an independent outcome when IQ was greater than 70, and IBI starting age (Figure 3). The calculated ICERs remained negative for the range of possible values for all variables. When analyzed using PSA, there was a large spread of incremental cost and effectiveness results, reflecting the degree of uncertainty in the effectiveness inputs. The results of the multiway PSA for EWT compared with CWT are presented in Figure 4, along with a cost-effectiveness acceptability curve (eFigure 1 in the Supplement).
The DFLYs generated for each of the 3 wait time scenarios were identical to the provincial perspective. Costs were notably higher, with CWT at $3 267 500, RWT at $3 120 700, and EWT at $3 000 200. The results of the cost-effectiveness analysis from the societal perspective are shown in eFigure 2 in the Supplement. The EWT scenario again dominated the other strategies, resulting in cost savings of more than $267 000 per individual over a lifetime compared with CWT.
Similar to the provincial perspective, the societal ICER outputs were most sensitive to uncertainty surrounding the probability of achieving a post-IBI IQ score greater than 70 based on age in 1-way sensitivity analyses. In addition, the ICERs calculated from the societal perspective were influenced by uncertainty in the cost associated with caregiving for adults with a dependent outcome. The results of the PSAs were also similar to the provincial perspective. The cost-effectiveness acceptability curve revealed eliminated wait time to be the preferred strategy at all levels of willingness to pay.
Cost-effectiveness analyses are undertaken as an aid to determine how to use limited resources efficiently. Intervention strategies are recommended when additional effectiveness is achieved at a reasonable additional cost compared with the status quo.29 This analysis, the first to model the economic effect of wait times, revealed that eliminating wait times for IBI was the dominant strategy over the CWT and halving the wait time from both provincial and societal perspectives. Incremental reductions in wait time were associated with gains in independence and lower projected lifetime costs.
Evidence consistently indicates that children who receive IBI at younger ages make greater gains than their older counterparts.6-9 This analysis incorporates the substantial costs associated with ASD, indicating that the long-term cost burden on the provincial government and society is also decreased when children begin IBI at younger ages. The decreased costs result from minimizing the need for social supports later in life, including special education, family services, and adult disability services. Although this model was constructed using parameters from the Ontario context, the results are relevant to any jurisdiction providing publicly funded early IBI.
Although the model is based on assumptions that are consistent with the current literature on IBI, there are several limitations. The number of children included in the starting age data sample was relatively small; however, variability around this parameter was incorporated into the sensitivity analyses. Projections of adult outcome are limited by substantial variability in post-IBI IQ and limited predictors of future independence (beyond IQ) in the published literature. The presented model captures this variability by using large ranges for the associated probabilities in the sensitivity analysis, resulting in wide 95% CIs around the ICERs. Better predictors of adult independence in the future will decrease uncertainty in the model. These studies are critical to future economic decision making because most ASD-related costs are in adult services.30
The aim of this analysis was to compare incremental cost differences among scenarios; the reported absolute costs for each of the scenarios are not intended to represent all ASD costs to governments or society, which have been previously reported.20,23,30 The model does not fully account for differences in cost between the direct service option (wherein the child receives publicly delivered IBI) and the direct funding option (wherein the family receives funding to contract an IBI practitioner) for IBI in Ontario, and it does not account for the differences in proportions of families who choose each option across regions, both of which are known to exist. Reported costs are higher when more families choose the direct service option over the direct funding option.12 The range of values used in our model attempted to capture this variation by using the regional minimum ($50 800 per year) and maximum ($67 000 per year) costs reported in 2013.12 There also may be differences between the direct service option and the direct funding option relating to the amount of hours of service received, duration of service, quality of service, and out-of-pocket costs to families.12
This study does not analyze the effect of increasing the absolute number of children with ASD who receive IBI but rather the proportion of eligible children who receive IBI at younger ages. The model assumed that all infrastructure costs were covered within the median cost of IBI, although there may be some additional start-up costs associated with program expansion, such as staff hiring, training, and infrastructure requirements. An important recent development in Ontario is significant restructuring of the Autism Intervention Program, with children older than 4 years no longer offered an eligibility assessment for the IBI intervention. As such, increased program capacity may not be necessary to achieve wait time reduction. The government also pledged an additional $330 million during 3 years to increase treatment spaces and provide flexible ASD services at an individualized intensity level.31 On the basis of the calculation in this model (which does include children older than 4 years), the upfront investment required for the program expansion implicated in the EWT scenario is estimated at $93 million (calculated as the product of IBI cost and number of children currently waiting). The model demonstrates that the return on this investment would occur during a long period with EWT for the entire cohort of children on waiting lists for IBI at the time of analysis (n = 1700), resulting in cost savings of $90 059 200 for the provincial government and societal savings of $454 437 200.
This study predicts the long-term effect of the current disparity between IBI service needs and the amount of IBI being delivered in the province of Ontario. With waiting lists increasing much faster than the number of children receiving IBI, reducing and ideally eliminating wait times for IBI in Ontario has the potential to result in better treatment and adult outcomes for many children and substantial cost savings from the perspectives of the provincial government and society.
Corresponding Author: Melanie Penner, MSc, MD, FRCPC, Autism Research Centre, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, 150 Kilgour Rd, Toronto, ON M4G 1R8, Canada (firstname.lastname@example.org).
Accepted for Publication: July 27, 2016.
Published Online: November 14, 2016. doi:10.1001/jamapediatrics.2016.2695
Author Contributions: Ms Piccininni and Dr Penner had full access to the data and take responsibility for the integrity of the data and accuracy of the data analysis.
Study concept and design: Piccininni, Penner.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Piccininni, Penner.
Critical revision of the manuscript for important intellectual content: Bisnaire, Penner.
Statistical analysis: Piccininni, Penner.
Obtained funding: Penner.
Administrative, technical, or material support: Penner.
Study supervision: Penner.
Conflict of Interest Disclosures: Dr Piccininni reported receiving financial support from the Bloorview Research Institute Ward Summer Student Program. Dr Bisnaire reported working as the director of the Autism Program of the Children’s Hospital of Eastern Ontario, a regional provider of the Ontario Autism Intervention Program. No other disclosures were reported.
Additional Contributions: Meera Rayar, MD, FRCPC, and Naazish Bashir, MSc, MBA, The Hospital for Sick Children, Toronto, Ontario, Canada, collaborated on a previous cost-effectiveness analysis17; some model inputs were derived from their previous work.
Create a personal account or sign in to: