[Skip to Navigation]
Sign In
Figure 1.  CONSORT Flow Diagram
CONSORT Flow Diagram

PCP indicates primary care physician or practitioner; ECHO, Extension for Community Health Outcomes; WCV, well-child visit.

Figure 2.  Planned Data Collection Times Across Cohorts
Planned Data Collection Times Across Cohorts

ECHO indicates Extension for Community Health Outcomes; T, time.

Table 1.  Demographics for mITT Sample
Demographics for mITT Sample
Table 2.  Model Estimates at Each Time
Model Estimates at Each Time
Table 3.  Model-Based Estimated Treatment Effects
Model-Based Estimated Treatment Effects
1.
Cook  NL, Hicks  LS, O’Malley  AJ, Keegan  T, Guadagnoli  E, Landon  BE.  Access to specialty care and medical services in community health centers.   Health Aff (Millwood). 2007;26(5):1459-1468. doi:10.1377/hlthaff.26.5.1459PubMedGoogle ScholarCrossref
2.
Arora  S, Thornton  K, Murata  G,  et al.  Outcomes of treatment for hepatitis C virus infection by primary care providers.   N Engl J Med. 2011;364(23):2199-2207. doi:10.1056/NEJMoa1009370PubMedGoogle ScholarCrossref
3.
Arora  S, Kalishman  S, Thornton  K,  et al.  Expanding access to hepatitis C virus treatment: Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care.   Hepatology. 2010;52(3):1124-1133. doi:10.1002/hep.23802PubMedGoogle ScholarCrossref
4.
Arora  S, Kalishman  S, Dion  D,  et al.  Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care.   Health Aff (Millwood). 2011;30(6):1176-1184. doi:10.1377/hlthaff.2011.0278PubMedGoogle ScholarCrossref
5.
Office of Health Policy; Office of the Assistant Secretary for Planning and Evaluation (ASPE); US Department of Health and Human Services. Report to Congress: current state of technology-enabled collaborative learning and capacity building models. report to congress. https://aspe.hhs.gov/pdf-report/report-congress-current-state-technology-enabled-collaborative-learning-and-capacity-building-models. Published March 2019. Accessed February 6, 2020.
6.
Project  ECHO. https://echo.unm.edu/. Accessed January 8, 2019.
7.
Simonoff  E, Pickles  A, Charman  T, Chandler  S, Loucas  T, Baird  G.  Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample.   J Am Acad Child Adolesc Psychiatry. 2008;47(8):921-929. doi:10.1097/CHI.0b013e318179964fPubMedGoogle ScholarCrossref
8.
Myers  SM, Johnson  CP; American Academy of Pediatrics Council on Children With Disabilities.  Management of children with autism spectrum disorders.   Pediatrics. 2007;120(5):1162-1182. doi:10.1542/peds.2007-2362PubMedGoogle ScholarCrossref
9.
American Psychiatric Association.  Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: American Psychiatric Association; 2013.
10.
Baio  J, Wiggins  L, Christensen  DL,  et al.  Prevalence of autism spectrum disorder among children aged 8 years: autism and developmental disabilities monitoring network, 11 sites, United States, 2014.   MMWR Surveill Summ. 2018;67(6):1-23. doi:10.15585/mmwr.ss6706a1PubMedGoogle ScholarCrossref
11.
Bisgaier  J, Levinson  D, Cutts  DB, Rhodes  KV.  Access to autism evaluation appointments with developmental-behavioral and neurodevelopmental subspecialists.   Arch Pediatr Adolesc Med. 2011;165(7):673-674. doi:10.1001/archpediatrics.2011.90PubMedGoogle ScholarCrossref
12.
Kogan  MD, Strickland  BB, Blumberg  SJ, Singh  GK, Perrin  JM, van Dyck  PC.  A national profile of the health care experiences and family impact of autism spectrum disorder among children in the United States, 2005-2006.   Pediatrics. 2008;122(6):e1149-e1158. doi:10.1542/peds.2008-1057PubMedGoogle ScholarCrossref
13.
Croen  LA, Najjar  DV, Ray  GT, Lotspeich  L, Bernal  P.  A comparison of health care utilization and costs of children with and without autism spectrum disorders in a large group-model health plan.   Pediatrics. 2006;118(4):e1203-e1211. doi:10.1542/peds.2006-0127PubMedGoogle ScholarCrossref
14.
Liptak  GS, Stuart  T, Auinger  P.  Health care utilization and expenditures for children with autism: data from U.S. national samples.   J Autism Dev Disord. 2006;36(7):871-879. doi:10.1007/s10803-006-0119-9PubMedGoogle ScholarCrossref
15.
Krauss  MW, Gulley  S, Sciegaj  M, Wells  N.  Access to specialty medical care for children with mental retardation, autism, and other special health care needs.   Ment Retard. 2003;41(5):329-339. doi:10.1352/0047-6765(2003)41<329:ATSMCF>2.0.CO;2PubMedGoogle ScholarCrossref
16.
Sheldrick  RC, Perrin  EC.  Medical home services for children with behavioral health conditions.   J Dev Behav Pediatr. 2010;31(2):92-99. doi:10.1097/DBP.0b013e3181cdabdaPubMedGoogle ScholarCrossref
17.
Newacheck  PW, Hung  YY, Wright  KK.  Racial and ethnic disparities in access to care for children with special health care needs.   Ambul Pediatr. 2002;2(4):247-254. doi:10.1367/1539-4409(2002)002<0247:RAEDIA>2.0.CO;2PubMedGoogle ScholarCrossref
18.
Skinner  AC, Slifkin  RT.  Rural/urban differences in barriers to and burden of care for children with special health care needs.   J Rural Health. 2007;23(2):150-157. doi:10.1111/j.1748-0361.2007.00082.xPubMedGoogle ScholarCrossref
19.
Gresenz  CR, Rogowski  J, Escarce  JJ.  Dimensions of the local health care environment and use of care by uninsured children in rural and urban areas.   Pediatrics. 2006;117(3):e509-e517. doi:10.1542/peds.2005-0733PubMedGoogle ScholarCrossref
20.
Mandell  DS, Novak  MM, Zubritsky  CD.  Factors associated with age of diagnosis among children with autism spectrum disorders.   Pediatrics. 2005;116(6):1480-1486. doi:10.1542/peds.2005-0185PubMedGoogle ScholarCrossref
21.
Daniels  AM, Mandell  DS.  Explaining differences in age at autism spectrum disorder diagnosis: a critical review.   Autism. 2014;18(5):583-597. doi:10.1177/1362361313480277PubMedGoogle ScholarCrossref
22.
Kuhlthau  KA, McDonnell  E, Coury  DL, Payakachat  N, Macklin  E.  Associations of quality of life with health-related characteristics among children with autism.   Autism. 2018;22(7):804-813.PubMedGoogle ScholarCrossref
23.
Institute of Medicine.  Defining Primary Care: An Interim Report. Washington, DC: The National Academies Press; 1994.
24.
Golnik  A, Scal  P, Wey  A, Gaillard  P.  Autism-specific primary care medical home intervention.   J Autism Dev Disord. 2012;42(6):1087-1093. doi:10.1007/s10803-011-1351-5PubMedGoogle ScholarCrossref
25.
Carbone  PS.  Moving from research to practice in the primary care of children with autism spectrum disorders.   Acad Pediatr. 2013;13(5):390-399. doi:10.1016/j.acap.2013.04.003PubMedGoogle ScholarCrossref
26.
Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee.  Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening.   Pediatrics. 2006;118(1):405-420. doi:10.1542/peds.2006-1231PubMedGoogle ScholarCrossref
27.
Johnson  CP, Myers  SM; American Academy of Pediatrics Council on Children With Disabilities.  Identification and evaluation of children with autism spectrum disorders.   Pediatrics. 2007;120(5):1183-1215. doi:10.1542/peds.2007-2361PubMedGoogle ScholarCrossref
28.
Dosreis  S, Weiner  CL, Johnson  L, Newschaffer  CJ.  Autism spectrum disorder screening and management practices among general pediatric providers.   J Dev Behav Pediatr. 2006;27(2)(suppl):S88-S94. doi:10.1097/00004703-200604002-00006PubMedGoogle ScholarCrossref
29.
King  TM, Tandon  SD, Macias  MM,  et al.  Implementing developmental screening and referrals: lessons learned from a national project.   Pediatrics. 2010;125(2):350-360. doi:10.1542/peds.2009-0388PubMedGoogle ScholarCrossref
30.
Self  TL, Parham  DF, Rajagopalan  J.  Autism spectrum disorder early screening practices: a survey of physicians.   Comm Disord Q. 2015;36:195-207. doi:10.1177/1525740114560060Google ScholarCrossref
31.
Arunyanart  W, Fenick  A, Ukritchon  S, Imjaijitt  W, Northrup  V, Weitzman  C.  Developmental and autism screening: a survey across six states.   Infants Young Child. 2012;25:175-187. doi:10.1097/IYC.0b013e31825a5a42Google ScholarCrossref
32.
Fenikilé  TS, Ellerbeck  K, Filippi  MK, Daley  CM.  Barriers to autism screening in family medicine practice: a qualitative study.   Prim Health Care Res Dev. 2015;16(4):356-366. doi:10.1017/S1463423614000449PubMedGoogle ScholarCrossref
33.
Carbone  PS, Murphy  NA, Norlin  C, Azor  V, Sheng  X, Young  PC.  Parent and pediatrician perspectives regarding the primary care of children with autism spectrum disorders.   J Autism Dev Disord. 2013;43(4):964-972. doi:10.1007/s10803-012-1640-7PubMedGoogle ScholarCrossref
34.
Mazurek  MO, Brown  R, Curran  A, Sohl  K.  ECHO autism.   Clin Pediatr (Phila). 2017;56(3):247-256. doi:10.1177/0009922816648288PubMedGoogle ScholarCrossref
35.
Sohl  K, Mazurek  MO, Brown  R.  ECHO autism: using technology and mentorship to bridge gaps, increase access to care, and bring best practice autism care to primary care.   Clin Pediatr (Phila). 2017;56(6):509-511. doi:10.1177/0009922817691825PubMedGoogle ScholarCrossref
36.
Malow  BA, Byars  K, Johnson  K,  et al; Sleep Committee of the Autism Treatment Network.  A practice pathway for the identification, evaluation, and management of insomnia in children and adolescents with autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S106-S124. doi:10.1542/peds.2012-0900IPubMedGoogle ScholarCrossref
37.
Mahajan  R, Bernal  MP, Panzer  R,  et al; Autism Speaks Autism Treatment Network Psychopharmacology Committee.  Clinical practice pathways for evaluation and medication choice for attention-deficit/hyperactivity disorder symptoms in autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S125-S138. doi:10.1542/peds.2012-0900JPubMedGoogle ScholarCrossref
38.
Vasa  RA, Mazurek  MO, Mahajan  R,  et al.  Assessment and treatment of anxiety in youth with autism spectrum disorders.   Pediatrics. 2016;137(suppl 2):S115-S123. doi:10.1542/peds.2015-2851JPubMedGoogle ScholarCrossref
39.
Furuta  GT, Williams  K, Kooros  K,  et al.  Management of constipation in children and adolescents with autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S98-S105. doi:10.1542/peds.2012-0900HPubMedGoogle ScholarCrossref
40.
Buie  T, Campbell  DB, Fuchs  GJ  III,  et al.  Evaluation, diagnosis, and treatment of gastrointestinal disorders in individuals with ASDs: a consensus report.   Pediatrics. 2010;125(suppl 1):S1-S18. doi:10.1542/peds.2009-1878CPubMedGoogle ScholarCrossref
41.
Maglione  MA, Gans  D, Das  L, Timbie  J, Kasari  C; Technical Expert Panel; HRSA Autism Intervention Research – Behavioral (AIR-B) Network.  Nonmedical interventions for children with ASD: recommended guidelines and further research needs.   Pediatrics. 2012;130(suppl 2):S169-S178. doi:10.1542/peds.2012-0900OPubMedGoogle ScholarCrossref
42.
Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. doi:10.1016/j.cct.2006.05.007PubMedGoogle ScholarCrossref
43.
Moore  C, Zamora  I, Patel Gera  M, Williams  ME.  Developmental screening and referrals: assessing the influence of provider specialty, training, and interagency communication.   Clin Pediatr (Phila). 2017;56(11):1040-1047. doi:10.1177/0009922817701174PubMedGoogle ScholarCrossref
44.
Porter  S, Qureshi  R, Caldwell  BA, Echevarria  M, Dubbs  WB, Sullivan  MW.  Developmental surveillance and screening practices by pediatric primary care providers.   Infants Young Child. 2016;29:91-101. doi:10.1097/IYC.0000000000000057Google ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    March 9, 2020

    Effectiveness of the Extension for Community Health Outcomes Model as Applied to Primary Care for Autism: A Partial Stepped-Wedge Randomized Clinical Trial

    Author Affiliations
    • 1Curry School of Education and Human Development, Department of Human Services, University of Virginia, Charlottesville
    • 2Biostatistics Center, Department of Medicine, Massachusetts General Hospital, Boston
    • 3Department of Medicine, Harvard Medical School, Boston, Massachusetts
    • 4Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
    • 5Division of General Academic Pediatrics, Massachusetts General Hospital, Boston, Massachusetts
    • 6Department of Child Health, University of Missouri, Columbia
    JAMA Pediatr. 2020;174(5):e196306. doi:10.1001/jamapediatrics.2019.6306
    Key Points

    Question  Is the Extension for Community Health Outcomes (ECHO) telementoring model effective in improving clinical practice among primary care clinicians caring for children with autism?

    Findings  In this large-scale, multisite partial stepped-wedge study of the ECHO model as applied to autism, significant changes in autism screening or treatment of comorbidities were not demonstrated. However, primary care clinicians demonstrated significant improvements in knowledge and self-efficacy immediately following and 3 months after completing the ECHO program.

    Meaning  The results provide support for the ECHO model in improving clinician knowledge and confidence in caring for patients with autism in primary care practices, but measurable changes in practice were not observed.

    Abstract

    Importance  The Extension for Community Health Outcomes (ECHO) model is a widely adopted technology-based model for training primary care physicians and practitioners (PCPs) to care for patients with complex conditions. Despite its popularity, to our knowledge, direct effects of ECHO on clinical practice have not been tested in a large-scale study.

    Objective  To test the effectiveness of the ECHO model as applied to primary care for autism and whether it resulted in improved clinical practice, knowledge, and self-efficacy regarding autism screening and comorbidity management.

    Design, Setting, and Participants  Primary care physicians and practitioners were recruited to participate in a 6-month ECHO Autism program delivered by 1 of 10 academic medical center sites. A sequential, staggered rollout of ECHO Autism was delivered to 5 cohorts of participants (15 per site; 2 sites per cohort). Sites were randomized after recruitment to cohort/start time. Cohorts launched every 3 months. The ECHO Autism program used videoconferencing technology to connect community-based PCPs with interdisciplinary expert teams at academic medical centers. There were 148 participants (PCPs [family practice physicians, pediatricians, nurse practitioners, and physician assistants] providing outpatient services to underserved children) studied between December 2016 and November 2018.

    Interventions  The 6-month ECHO Autism program included twelve 2-hour sessions connecting PCP participants with an interdisciplinary expert team. Sessions included didactics, case-based learning, guided practice, and discussion.

    Main Outcomes and Measures  Coprimary outcomes were autism screening practices and comorbidity management (assessed by medical record review). Secondary outcomes were knowledge (assessed by direct testing) and self-efficacy (assessed by self-report survey). Assessments were conducted at baseline, mid-ECHO, post-ECHO, and follow-up (3 months after ECHO).

    Results  Ten sites were randomized to 1 of 5 cohorts. Participants were 82% female (n = 108), 76% white (n = 100), and 6% Hispanic or Latino (n = 8); the median age was 46 years (interquartile range, 37-55 years). Significant changes in autism screening and treatment of comorbidities in children with autism were not observed. Participants demonstrated significant improvements in knowledge (9%; 95% CI, 4-13; P < .001) and self-efficacy (29%; 95% CI, 25-32; P < .001).

    Conclusions and Relevance  The ECHO model was developed to increase access to high-quality health care for underserved patients with complex conditions. Study results provide support for the model in improving clinician knowledge and confidence but little support for achieving practice change.

    Trial Registration  ClinicalTrials.gov Identifier: NCT03677089

    Introduction

    Despite rapid advancements in medical knowledge, patients with complex conditions from rural and underserved areas face significant barriers in accessing specialty care.1 The Extension for Community Health Outcomes (ECHO) model was developed to address this gap by infusing specialty knowledge into local primary care practices.2 Through ECHO, groups of community-based primary care physicians and practitioners (PCPs) connect with a team of experts using secure videoconferencing and receive best-practice knowledge through didactics, case-based learning, comanagement, and a virtual community of practice.3,4 Since 2003, the framework has been adopted by more than 500 programs globally. Originally developed to improve care of hepatitis C, the model has now been applied to more than 100 complex conditions (ranging from diabetes to addiction).5,6 Despite widespread adoption of the model, evidence of effectiveness remains preliminary. Evidence to date derives from tests of single-project ECHO clinics rather than from more robust multisite designs and primarily from self-reported outcomes rather than direct measures of practice change, as was extensively summarized for Congress.5 More rigorous tests of the model are needed to assess its effectiveness in improving the quality of health care for patients with complex conditions. Autism represents one such increasingly common and complex condition.

    Autism is a neurodevelopmental disorder associated with high rates of medical and psychiatric comorbidities.7-9 Increased autism prevalence over the past 2 decades10 has resulted in health care demands that greatly exceed the capacity of specialty centers.11 Children with autism experience greater unmet health care needs, higher health care costs, and worse access to medical homes and specialty care than children with other special health care needs.12-16 Children with autism from underserved populations face disproportionate health care barriers owing to clinician shortages, high costs of services, and language or cultural barriers.17-19 Poor access to care results in diagnostic and intervention delays, unmanaged comorbid conditions, poor health, and reduced quality of life for children with autism.20-22

    Community-based PCPs are ideally positioned to address these disparities. Primary care, by definition, consists of “integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”23 Unfortunately, many PCPs lack training in autism and feel unprepared to provide best-practice care.24,25 Although the American Academy of Pediatrics issued clear recommendations for developmental surveillance and screening for autism,26,27 physician compliance has been strikingly poor.28-31 Many PCPs lack knowledge about autism20,28,32 and lack confidence in managing common co-occurring conditions in children with autism.33 Equipping PCPs with the skills to provide effective community-based screening and management of autism has the potential to enhance outcomes for children with autism.

    A 2017 pilot study34,35 was conducted to examine the application of the ECHO model to enhance primary care for autism. The study consisted of a small sample of 14 PCPs who attended 12 biweekly ECHO Autism clinics focused on best-practice medical care for children with autism. The results were promising, showing significant improvements in participant self-efficacy and self-reported practice change. However, objective measures of improvement were not collected.34 Thus, a more rigorous examination of the model is needed.

    This study was conducted to test the effectiveness of ECHO Autism in a large and geographically diverse randomized sample using objective measures of improvement in clinical practice, while delivering a potentially important educational intervention to all participants. The specific hypotheses were that participating in ECHO Autism would result in significant improvements among PCPs in (1) clinical practice, (2) knowledge, and (3) self-efficacy regarding autism screening and management of comorbidities.

    Methods
    Study Design

    Ten academic medical centers (eAppendix in Supplement 1) participating in the Autism Treatment Network and the Autism Intervention Research Network on Physical Health served as individual ECHO Autism clinic sites (each training a target sample of 15 PCP participants). A sequential, staggered rollout of the ECHO Autism program was studied in 5 cohorts of participants (2 sites every 3 months) between December 2016 and November 2018. Sites were randomized to startup order after recruitment into study. This design allowed us to distribute limited resources to assess the effect of the ECHO Autism program across a broad range of sites. Assessments were completed at 4 points separated by 3-month intervals: Baseline/pre-ECHO, mid-ECHO, post-ECHO, and a post-ECHO follow-up to assess persistence of effect. Figure 1 shows a CONSORT diagram of participation in the study. Figure 2 describes the timeline for data collection. The study was registered at ClinicalTrials.gov (NCT03677089), and the formal trial protocols are in Supplement 2.

    Participants

    Each of the 10 ECHO Autism sites (9 US and 1 Canadian site) planned to recruit a sample of 15 PCPs from their respective geographic region. Inclusion criteria were current practice as a PCP (ie, family practice physician, pediatrician, nurse practitioner, or physician assistant), providing outpatient services to children, and providing care to underserved populations (at least 50% of patients were underserved). Only 1 PCP from a single-practice location was allowed to participate in the study. Recruitment methods included professional distribution lists, professional meetings, and face-to-face and word-of-mouth recruitment. Participants received no-cost continuing medical education credit for attendance at each ECHO Autism clinic, and modest financial incentives were provided for completion of study measures at each assessment point. Informed consent followed procedures dictated by the institutional review board at the specific site; generally verbal consent was considered adequate because the study involved no more than minimal risk. The study was approved by the institutional review board at each site and at the Network Coordinating Center.

    ECHO Autism Implementation and Procedures

    ECHO Autism expert teams were formed at each of the 10 participating sites, consisting of at least 5 autism specialists (physician/autism medical specialist, psychologist, family resource specialist, dietician, and parent expert). One site included an additional Family Resource Specialist because participants were practicing in 2 different states. Teams were trained to fidelity in the ECHO model and in the ECHO Autism curriculum.34 Training included in-person training and orientation at the ECHO Institute, regular learning network videoconference meetings, attendance and observation of at least 3 ECHO Autism clinics facilitated by the leadership team, and facilitation of at least 2 mock clinics with live coaching and feedback prior to clinic launch.

    Each ECHO autism clinic site provided twice-monthly 2-hour ECHO Autism clinics during a 6-month period. Clinics were conducted using high-quality secure Health Insurance Portability and Accountability Act–compliant videoconferencing technology (Zoom), enabling screen-sharing (for document viewing) and real-time interactions among participants and ECHO Autism expert team members. All clinics followed the established ECHO Autism curriculum, which included a standard set of didactic presentation slides and content, case presentation forms, and materials.34 Each clinic consisted of a brief 15-minute didactic presentation, 2 PCP-generated case presentations, and collaborative discussion among experts and participants. Didactic content focused on guidelines and algorithms for evidence-based best-practice medical care for children with autism, with particular emphasis on autism screening and identification and on medical management of comorbid conditions27,36-41 (didactic materials are available on request). The intent of the program was for participants to develop new clinical skills through guided practice and collaborative case-based learning while maintaining responsibility for care of their patients. Implementation fidelity was assessed by study team members at 2 randomly selected sessions per site using a form developed by the University of New Mexico Project ECHO Team to assess fidelity; adequate fidelity was defined as “agree” or “strongly agree” for at least 80% of the relevant items.

    Unconstrained random allocation was generated in SAS (SAS Institute Inc) by the senior study statistician after sites were enrolled by principal investigator. Sites were allocated to a cluster based on the random allocation.

    Some operational problems occurred during the study. Owing to scheduling issues, 2 sites started 1 month after the scheduled start of their cohort. This was ignored for the analysis. Because Canadian guidelines for screening for autism were not consistent with US guidelines, autism screening for the Canadian site was excluded from the primary analysis. Full details of all operational problems are provided in section 12 of the statistical analysis plan in Supplement 3.

    Outcome Measures

    Clinical practice/behavior was assessed through medical record review conducted by study staff in each PCP practice. Appropriate autism screening was assessed at 18-month and 24-month well-child visits for the 30 days prior to the scheduled date of medical record review. Medical record reviews were done at baseline (before the start of ECHO, but on rare occasions after the first but before the second ECHO session), at mid-ECHO (between the sixth and seventh ECHO session), post-ECHO (within 30 days after the end of the ECHO program), and for long-term effect (between 3 and 4 months after the end of the ECHO program). Appropriate autism screening was defined as administering an autism screening tool at 18 and 24 months, as recommended by the American Academy of Pediatrics27 for the 9 US sites (all other measures included both US and Canadian sites). Treatment of 4 common autism comorbidities (constipation, sleep disturbance, behavior problems, and anxiety) was assessed for all visits for children with autism seen within the 60 days prior to the scheduled date of medical record review. We considered any treatment addressing a recognized problem (from a predefined list of appropriate treatment options for each condition, which are shown in eTable 1 in Supplement 1) as appropriately addressing the problem. Secondary outcome measures (knowledge, self-efficacy, and barriers to care) are described further in the trial protocol in Supplement 2. Self-reported demographic information (including age, sex, and race/ethnicity) was collected to characterize the sample (Table 1).

    Power Calculations

    Given the complexity of the proposed analysis, power calculations were based on simulations. The data generation process allowed for random effects for site, PCP within site, and nominal period. There was no time trend in the data, although a potential time trend as a fixed effect was included in the model. Simulations were done for 10 randomly selected seeds (from several different websites and different random number tables) at 1000 simulations per seed. The data-generating process allowed for approximately a 50% intraclass correlation for the PCP within-group effect, reflecting the possibility that the effect of ECHO would be correlated within each site, even with good fidelity to the intervention program. Simulations allowed for varying numbers of patients per PCP practice.

    If PCPs on average had 15 patients with well-child visits seen in the past month, we would have more than 90% power to detect an increase from 50% to 60% in autism screening (α = .025; all tests were 2-sided). If PCPs on average had seen 5 patients with autism in the last 60 days, we would have more than 90% power to detect an increase from 50% to 65% in appropriate comorbidity management (α = .025; all tests were 2-sided).

    Statistical Methods

    Data are summarized as median (interquartile range [IQR]) or counts. The primary analysis used a generalized linear mixed model with data from baseline, mid-ECHO, post-ECHO, and 9-month follow-up to assess program effect, with fixed effects for period (time since start of study) and time in study (categorical time since PCP participant entered study) including site-specific random intercept for each PCP participant. We accounted for repeated measures within PCP participants with unstructured covariance. This is a modified version of the standard model for a stepped-wedge design put forward by Hussey and Hughes.42 Model estimates for baseline, month 3, month 6, and month 9 are presented are presented with the primary analysis test of change over 6 months of study. Additional results test changes from month 6 to month 9 to assess any regression after ECHO is completed. Results are reported for a modified intent-to-treat population that includes all participants who consented and provided basic demographics at baseline. Although we initially planned an analysis restricted to participants who more closely followed protocol, we agreed with a reviewer to present mITT analyses because this was better suited to the study design; results were almost identical between the 2 analyses. The SAP called for baseline data to be imputed if more than 5% of data were missing for a primary outcome. Because less than 5% of baseline data were missing, no baseline data were imputed. Missing follow-up data had the baseline carried forward as a conservative estimate (5 PCPs had autism screening follow-up information imputed, and 15 PCPs had follow-up information imputed for the proportion of comorbidities addressed outcome). Because we had 2 coprimary end points, we used P < .025 to determine statistical significance for proportion screened appropriately and proportion of comorbidities addressed appropriately at 6 months; otherwise P < .05 was considered statistically significant. All tests were 2-sided.

    Role of the ECHO Institute

    The study team received training and consultation from ECHO Institute members on ECHO implementation. Members of the ECHO Institute were not involved in study design, analyses, interpretation of results, or decision to publish.

    Results
    Population, Participation, and Implementation

    A total of 148 participants were enrolled across the 10 ECHO Autism clinic sites, ranging from 11 to 22 per site. See Table 1 for demographic data by recruitment cohort. Of the 129 participants (87%) who attended at least 1 session, the median attendance was 10 of 12 sessions (interquartile range [IQR], 8-11). Regarding implementation, all sessions assessed during random fidelity checks demonstrated fidelity to the ECHO model. Median fidelity at individual clinics was 96% (range, 82%-100%).

    Some demographic information was provided by 132 participants (89%), and baseline medical record review was completed for 118 participants (80%). The number of medical records available for review at each point varied across participants and visit type. For the baseline medical record review window, participants had a median of 5 well-child visits (IQR, 2-9) reviewed for autism screening (18-month or 24-month visits) and 3 visits (IQR, 1-6) with children with autism. Of the 118 participants having a medical record review attempted at baseline, 25 had no relevant well-child visits within the 30-day review period, 18 had no visits with a child with autism within the 60-day review period, and 24 provided a medical record for a child with autism but did not identify any of the listed problems. Baseline outcome measures by cohort are presented in eTable 2 in Supplement 1.

    Efficacy of ECHO Autism

    Model-based estimates of the outcomes are presented in Table 2, while effect estimates and statistical significance are shown in Table 3. Our primary analysis models estimated a small 6-month increase in autism screening rates from 65% (95% CI, 40%-84%) to 71% (95% CI, 52%-84%) (P = .63) and a small 6-month decrease in mean percentage of comorbidities addressed from 81% (95% CI, 67%-90%) to 77% (95% CI, 66%-86%) (P = .59). There was substantial improvement in knowledge (9% increase; 95% CI, 4%-13%; P < .001) and overall self-efficacy (29% increase; 95% CI, 25%-32%; P < .001) and a substantial decrease in the number of barriers identified by participants (2.2 decrease; 95% CI, −2.7 to −1.7; P < .001) (Table 3).

    Long-term Effects of ECHO Autism

    The significant effects seen in knowledge, overall self-efficacy, and barriers identified did not subside by month 9, and there were no significant differences observed from the 6-month post-ECHO visit to the 9-month follow-up visit (Table 3).

    Discussion

    This study was designed to evaluate the effectiveness of a widely adopted technology-based continuing education model (project ECHO) in improving direct clinical care. In what is to our knowledge the first large-scale prospective multisite study of this model, we directly tested the effects of ECHO as applied to primary care for autism. Contrary to predictions, participants did not demonstrate statistically significant improvements in either autism screening or in treatment of comorbidities in children with autism after participation in the 6-month ECHO autism program. However, consistent with previous studies,5 participants did demonstrate significant increases in both knowledge and self-efficacy as well as significant reductions in perceived barriers to caring for children with autism in their practices.

    The ECHO model was developed to increase access to high-quality health care for underserved patients with complex conditions. The results of this study do not provide strong support for the model in achieving direct practice change among PCPs, at least as it relates to the autism screening and comorbidity management. However, marked improvements in clinician knowledge and confidence in treating children with autism were observed. Although less direct than the specific practice behaviors we assessed through medical record review, these outcomes may ultimately enhance overall care and patient-clinician interactions for a population at high risk for poor health care experiences and outcomes.11-15

    Strengths and Limitations

    This study benefitted from a large, multisite, and geographically diverse sample of PCPs. Additional strengths included randomization at the level of cohort, staggered implementation of the program across cohorts, and tests of effectiveness under baseline, intervention, and follow-up conditions. There are also several limitations that should be noted. Participants in this study had some motivation to improve their care of children with autism and may not be representative of the larger population of PCPs. However, their screening rates at baseline were generally consistent with 2016/2017 clinician-reported estimates,43,44 suggesting that they did not differ substantially from their peers with regard to screening practices prior to participation in the program.

    The ECHO Autism curriculum was delivered for a predefined period (6 months) for the purposes of the research study. However, typical ECHO implementation is ongoing, with no clearly defined stopping point. We also prohibited multiple participants from the same practice to minimize confounds; however, traditional ECHO models encourage participation from multiple clinicians from within a single practice. We also observed a relatively high rate of dropout, which may indicate issues with feasibility or acceptability of the model for some practicing PCPs.

    Finally, our measure of comorbidity management may have been too simplistic. We were unable to assess whether PCPs were actively screening for each condition or whether they were noted only when concerns were raised by caregivers. As such, we were unable to determine the extent to which comorbid conditions were accurately screened and identified. Because the medical record review process did not allow for determination of best treatment approach for an individual patient (because this would have required thorough clinical review of each case), our measure simply noted whether the PCP administered 1 or more potentially appropriate treatments when a comorbid condition was observed. In future studies, a more fine-grained analysis of treatment strategies would be more helpful in determining whether a clinician followed a best-practice treatment approach or algorithm.

    Conclusions

    In conclusion, the results of this large-scale study provide some support for the effectiveness of the ECHO model in improving primary care knowledge and self-efficacy regarding treatment of children with autism but little evidence of practice change in specific screening and management domains. Future research is needed to further evaluate the cost, benefit, and effectiveness of ECHO. A more comprehensive assessment of screening and management practices, patient-clinician interactions, and patient outcomes would be particularly informative in future studies.

    Back to top
    Article Information

    Accepted for Publication: December 12, 2018.

    Corresponding Author: Micah O. Mazurek, PhD, Curry School of Education and Human Development, University of Virginia, 417 Emmet St S, PO Box 400267, Charlottesville, VA 22904 (mazurekm@virginia.edu).

    Published Online: March 9, 2020. doi:10.1001/jamapediatrics.2019.6306

    Author Contributions: Dr Parker and Mr Chan had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Mazurek, Parker, Kuhlthau, Sohl.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Mazurek, Parker, Chan.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Parker, Chan.

    Obtained funding: Kuhlthau, Sohl.

    Administrative, technical, or material support: Mazurek, Parker, Kuhlthau, Sohl.

    Supervision: Mazurek, Parker, Kuhlthau.

    Conflict of Interest Disclosures: Dr Mazurek reported grants from Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services during the conduct of the study. Dr Parker reported other support from HRSA and grants from Autism Speaks during the conduct of the study. Since completing the study, Dr Parker has become a consultant to the ECHO Institute, University of New Mexico. Dr Kuhlthau reported grants from Autism Intervention Research Network on Physical Health grant from MCHB HRSA and funding from the Autism Speaks Foundation during the conduct of the study. Dr Sohl reported other support from Quadrant Biosciences, Autism Navigator, the National Institutdes of Health/National Institute of Neurological Disorders and Stroke; and Cognoa and personal fees from university-based grand rounds; in addition, Dr Sohl had a patent to Autism Ready Communities pending. No other disclosures were reported.

    Funding/Support: This project was supported by the Health Resources and Services Administration of the US Department of Health and Human Services under cooperative agreement UA3 MC11054–Autism Intervention Research Network on Physical Health.

    Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Group Information: The members of the ECHO Autism Collaborative are Evdokia Anagnostou, MD, University of Toronto, Toronto, Ontario, Canada; Amanda Bennett, MD, MPH, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Kristin Dalope, MD, University of Pittsburgh, Pittsburgh, Pennsylvania; Susan Hyman, MD, University of Rochester, Rochester, New York; Heather Johnson, PsyD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Maya Lopez, MD, University of Arkansas for Medical Sciences, Little Rock; Beth Malow, MD, MS, Vanderbilt University Medical Center, Nashville, Tennessee; Ann Neumeyer, MD, Lurie Center for Autism, Lexington, Massachusetts; Karen Ratliff-Schaub, MD, Nationwide Children’s Hospital, Columbus, Ohio; Robin Steinberg-Epstein, MD, University of California, Irvine.

    Disclaimer: This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by Health Resources and Services Administration of the US Department of Health and Human Services or the US government.

    Data Sharing Statement: See Supplement 4.

    Additional Contributions: We thank the members of the Clinical Coordinating Center and Data Coordinating Center at Massachusetts General Hospital, and the project staff and hub team members at each ECHO Autism Collaborative site. We also gratefully acknowledge all the primary care clinicians who participated in this study.

    Additional Information: This work was conducted through the Autism Speaks Autism Treatment Network serving as the Autism Intervention Research Network on Physical Health.

    References
    1.
    Cook  NL, Hicks  LS, O’Malley  AJ, Keegan  T, Guadagnoli  E, Landon  BE.  Access to specialty care and medical services in community health centers.   Health Aff (Millwood). 2007;26(5):1459-1468. doi:10.1377/hlthaff.26.5.1459PubMedGoogle ScholarCrossref
    2.
    Arora  S, Thornton  K, Murata  G,  et al.  Outcomes of treatment for hepatitis C virus infection by primary care providers.   N Engl J Med. 2011;364(23):2199-2207. doi:10.1056/NEJMoa1009370PubMedGoogle ScholarCrossref
    3.
    Arora  S, Kalishman  S, Thornton  K,  et al.  Expanding access to hepatitis C virus treatment: Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care.   Hepatology. 2010;52(3):1124-1133. doi:10.1002/hep.23802PubMedGoogle ScholarCrossref
    4.
    Arora  S, Kalishman  S, Dion  D,  et al.  Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care.   Health Aff (Millwood). 2011;30(6):1176-1184. doi:10.1377/hlthaff.2011.0278PubMedGoogle ScholarCrossref
    5.
    Office of Health Policy; Office of the Assistant Secretary for Planning and Evaluation (ASPE); US Department of Health and Human Services. Report to Congress: current state of technology-enabled collaborative learning and capacity building models. report to congress. https://aspe.hhs.gov/pdf-report/report-congress-current-state-technology-enabled-collaborative-learning-and-capacity-building-models. Published March 2019. Accessed February 6, 2020.
    6.
    Project  ECHO. https://echo.unm.edu/. Accessed January 8, 2019.
    7.
    Simonoff  E, Pickles  A, Charman  T, Chandler  S, Loucas  T, Baird  G.  Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample.   J Am Acad Child Adolesc Psychiatry. 2008;47(8):921-929. doi:10.1097/CHI.0b013e318179964fPubMedGoogle ScholarCrossref
    8.
    Myers  SM, Johnson  CP; American Academy of Pediatrics Council on Children With Disabilities.  Management of children with autism spectrum disorders.   Pediatrics. 2007;120(5):1162-1182. doi:10.1542/peds.2007-2362PubMedGoogle ScholarCrossref
    9.
    American Psychiatric Association.  Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: American Psychiatric Association; 2013.
    10.
    Baio  J, Wiggins  L, Christensen  DL,  et al.  Prevalence of autism spectrum disorder among children aged 8 years: autism and developmental disabilities monitoring network, 11 sites, United States, 2014.   MMWR Surveill Summ. 2018;67(6):1-23. doi:10.15585/mmwr.ss6706a1PubMedGoogle ScholarCrossref
    11.
    Bisgaier  J, Levinson  D, Cutts  DB, Rhodes  KV.  Access to autism evaluation appointments with developmental-behavioral and neurodevelopmental subspecialists.   Arch Pediatr Adolesc Med. 2011;165(7):673-674. doi:10.1001/archpediatrics.2011.90PubMedGoogle ScholarCrossref
    12.
    Kogan  MD, Strickland  BB, Blumberg  SJ, Singh  GK, Perrin  JM, van Dyck  PC.  A national profile of the health care experiences and family impact of autism spectrum disorder among children in the United States, 2005-2006.   Pediatrics. 2008;122(6):e1149-e1158. doi:10.1542/peds.2008-1057PubMedGoogle ScholarCrossref
    13.
    Croen  LA, Najjar  DV, Ray  GT, Lotspeich  L, Bernal  P.  A comparison of health care utilization and costs of children with and without autism spectrum disorders in a large group-model health plan.   Pediatrics. 2006;118(4):e1203-e1211. doi:10.1542/peds.2006-0127PubMedGoogle ScholarCrossref
    14.
    Liptak  GS, Stuart  T, Auinger  P.  Health care utilization and expenditures for children with autism: data from U.S. national samples.   J Autism Dev Disord. 2006;36(7):871-879. doi:10.1007/s10803-006-0119-9PubMedGoogle ScholarCrossref
    15.
    Krauss  MW, Gulley  S, Sciegaj  M, Wells  N.  Access to specialty medical care for children with mental retardation, autism, and other special health care needs.   Ment Retard. 2003;41(5):329-339. doi:10.1352/0047-6765(2003)41<329:ATSMCF>2.0.CO;2PubMedGoogle ScholarCrossref
    16.
    Sheldrick  RC, Perrin  EC.  Medical home services for children with behavioral health conditions.   J Dev Behav Pediatr. 2010;31(2):92-99. doi:10.1097/DBP.0b013e3181cdabdaPubMedGoogle ScholarCrossref
    17.
    Newacheck  PW, Hung  YY, Wright  KK.  Racial and ethnic disparities in access to care for children with special health care needs.   Ambul Pediatr. 2002;2(4):247-254. doi:10.1367/1539-4409(2002)002<0247:RAEDIA>2.0.CO;2PubMedGoogle ScholarCrossref
    18.
    Skinner  AC, Slifkin  RT.  Rural/urban differences in barriers to and burden of care for children with special health care needs.   J Rural Health. 2007;23(2):150-157. doi:10.1111/j.1748-0361.2007.00082.xPubMedGoogle ScholarCrossref
    19.
    Gresenz  CR, Rogowski  J, Escarce  JJ.  Dimensions of the local health care environment and use of care by uninsured children in rural and urban areas.   Pediatrics. 2006;117(3):e509-e517. doi:10.1542/peds.2005-0733PubMedGoogle ScholarCrossref
    20.
    Mandell  DS, Novak  MM, Zubritsky  CD.  Factors associated with age of diagnosis among children with autism spectrum disorders.   Pediatrics. 2005;116(6):1480-1486. doi:10.1542/peds.2005-0185PubMedGoogle ScholarCrossref
    21.
    Daniels  AM, Mandell  DS.  Explaining differences in age at autism spectrum disorder diagnosis: a critical review.   Autism. 2014;18(5):583-597. doi:10.1177/1362361313480277PubMedGoogle ScholarCrossref
    22.
    Kuhlthau  KA, McDonnell  E, Coury  DL, Payakachat  N, Macklin  E.  Associations of quality of life with health-related characteristics among children with autism.   Autism. 2018;22(7):804-813.PubMedGoogle ScholarCrossref
    23.
    Institute of Medicine.  Defining Primary Care: An Interim Report. Washington, DC: The National Academies Press; 1994.
    24.
    Golnik  A, Scal  P, Wey  A, Gaillard  P.  Autism-specific primary care medical home intervention.   J Autism Dev Disord. 2012;42(6):1087-1093. doi:10.1007/s10803-011-1351-5PubMedGoogle ScholarCrossref
    25.
    Carbone  PS.  Moving from research to practice in the primary care of children with autism spectrum disorders.   Acad Pediatr. 2013;13(5):390-399. doi:10.1016/j.acap.2013.04.003PubMedGoogle ScholarCrossref
    26.
    Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee.  Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening.   Pediatrics. 2006;118(1):405-420. doi:10.1542/peds.2006-1231PubMedGoogle ScholarCrossref
    27.
    Johnson  CP, Myers  SM; American Academy of Pediatrics Council on Children With Disabilities.  Identification and evaluation of children with autism spectrum disorders.   Pediatrics. 2007;120(5):1183-1215. doi:10.1542/peds.2007-2361PubMedGoogle ScholarCrossref
    28.
    Dosreis  S, Weiner  CL, Johnson  L, Newschaffer  CJ.  Autism spectrum disorder screening and management practices among general pediatric providers.   J Dev Behav Pediatr. 2006;27(2)(suppl):S88-S94. doi:10.1097/00004703-200604002-00006PubMedGoogle ScholarCrossref
    29.
    King  TM, Tandon  SD, Macias  MM,  et al.  Implementing developmental screening and referrals: lessons learned from a national project.   Pediatrics. 2010;125(2):350-360. doi:10.1542/peds.2009-0388PubMedGoogle ScholarCrossref
    30.
    Self  TL, Parham  DF, Rajagopalan  J.  Autism spectrum disorder early screening practices: a survey of physicians.   Comm Disord Q. 2015;36:195-207. doi:10.1177/1525740114560060Google ScholarCrossref
    31.
    Arunyanart  W, Fenick  A, Ukritchon  S, Imjaijitt  W, Northrup  V, Weitzman  C.  Developmental and autism screening: a survey across six states.   Infants Young Child. 2012;25:175-187. doi:10.1097/IYC.0b013e31825a5a42Google ScholarCrossref
    32.
    Fenikilé  TS, Ellerbeck  K, Filippi  MK, Daley  CM.  Barriers to autism screening in family medicine practice: a qualitative study.   Prim Health Care Res Dev. 2015;16(4):356-366. doi:10.1017/S1463423614000449PubMedGoogle ScholarCrossref
    33.
    Carbone  PS, Murphy  NA, Norlin  C, Azor  V, Sheng  X, Young  PC.  Parent and pediatrician perspectives regarding the primary care of children with autism spectrum disorders.   J Autism Dev Disord. 2013;43(4):964-972. doi:10.1007/s10803-012-1640-7PubMedGoogle ScholarCrossref
    34.
    Mazurek  MO, Brown  R, Curran  A, Sohl  K.  ECHO autism.   Clin Pediatr (Phila). 2017;56(3):247-256. doi:10.1177/0009922816648288PubMedGoogle ScholarCrossref
    35.
    Sohl  K, Mazurek  MO, Brown  R.  ECHO autism: using technology and mentorship to bridge gaps, increase access to care, and bring best practice autism care to primary care.   Clin Pediatr (Phila). 2017;56(6):509-511. doi:10.1177/0009922817691825PubMedGoogle ScholarCrossref
    36.
    Malow  BA, Byars  K, Johnson  K,  et al; Sleep Committee of the Autism Treatment Network.  A practice pathway for the identification, evaluation, and management of insomnia in children and adolescents with autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S106-S124. doi:10.1542/peds.2012-0900IPubMedGoogle ScholarCrossref
    37.
    Mahajan  R, Bernal  MP, Panzer  R,  et al; Autism Speaks Autism Treatment Network Psychopharmacology Committee.  Clinical practice pathways for evaluation and medication choice for attention-deficit/hyperactivity disorder symptoms in autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S125-S138. doi:10.1542/peds.2012-0900JPubMedGoogle ScholarCrossref
    38.
    Vasa  RA, Mazurek  MO, Mahajan  R,  et al.  Assessment and treatment of anxiety in youth with autism spectrum disorders.   Pediatrics. 2016;137(suppl 2):S115-S123. doi:10.1542/peds.2015-2851JPubMedGoogle ScholarCrossref
    39.
    Furuta  GT, Williams  K, Kooros  K,  et al.  Management of constipation in children and adolescents with autism spectrum disorders.   Pediatrics. 2012;130(suppl 2):S98-S105. doi:10.1542/peds.2012-0900HPubMedGoogle ScholarCrossref
    40.
    Buie  T, Campbell  DB, Fuchs  GJ  III,  et al.  Evaluation, diagnosis, and treatment of gastrointestinal disorders in individuals with ASDs: a consensus report.   Pediatrics. 2010;125(suppl 1):S1-S18. doi:10.1542/peds.2009-1878CPubMedGoogle ScholarCrossref
    41.
    Maglione  MA, Gans  D, Das  L, Timbie  J, Kasari  C; Technical Expert Panel; HRSA Autism Intervention Research – Behavioral (AIR-B) Network.  Nonmedical interventions for children with ASD: recommended guidelines and further research needs.   Pediatrics. 2012;130(suppl 2):S169-S178. doi:10.1542/peds.2012-0900OPubMedGoogle ScholarCrossref
    42.
    Hussey  MA, Hughes  JP.  Design and analysis of stepped wedge cluster randomized trials.   Contemp Clin Trials. 2007;28(2):182-191. doi:10.1016/j.cct.2006.05.007PubMedGoogle ScholarCrossref
    43.
    Moore  C, Zamora  I, Patel Gera  M, Williams  ME.  Developmental screening and referrals: assessing the influence of provider specialty, training, and interagency communication.   Clin Pediatr (Phila). 2017;56(11):1040-1047. doi:10.1177/0009922817701174PubMedGoogle ScholarCrossref
    44.
    Porter  S, Qureshi  R, Caldwell  BA, Echevarria  M, Dubbs  WB, Sullivan  MW.  Developmental surveillance and screening practices by pediatric primary care providers.   Infants Young Child. 2016;29:91-101. doi:10.1097/IYC.0000000000000057Google ScholarCrossref
    ×