[Skip to Navigation]
Sign In
Figure 1.  Participant Flow Diagram
Participant Flow Diagram

aChild was enrolled from a primary care site not participating in the study.

Figure 2.  Time to Diagnostic Ascertainment, by Treatment Group, All Sites
Time to Diagnostic Ascertainment, by Treatment Group, All Sites

CCM indicates conventional care management; FN, family navigation; and HR, hazard ratio.

Figure 3.  Time to Diagnostic Ascertainment, by Site and Treatment Group
Time to Diagnostic Ascertainment, by Site and Treatment Group

CCM indicates conventional care management; FN, family navigation; and HR, hazard ratio.

Figure 4.  Time to Diagnostic Ascertainment, by Ethnicity
Time to Diagnostic Ascertainment, by Ethnicity

This analysis included only the Boston and New Haven sites, which accounts for the loss of sample. CCM indicates conventional care management; FN, family navigation; and HR, hazard ratio.

Table.  Baseline Characteristics of Participants Enrolled in Project Early, by Treatment Group
Baseline Characteristics of Participants Enrolled in Project Early, by Treatment Group
1.
Durkin  MS, Maenner  MJ, Baio  J,  et al.  Autism spectrum disorder among US children (2002-2010): socioeconomic, racial, and ethnic disparities.   Am J Public Health. 2017;107(11):1818-1826. doi:10.2105/AJPH.2017.304032 PubMedGoogle ScholarCrossref
2.
Lu  W.  Child and adolescent mental disorders and health care disparities: results from the National Survey of Children’s Health, 2011–2012.   J Health Care Poor Underserved. 2017;28(3):988-1011. doi:10.1353/hpu.2017.0092 PubMedGoogle ScholarCrossref
3.
Mandell  DS, Listerud  J, Levy  SE, Pinto-Martin  JA.  Race differences in the age at diagnosis among Medicaid-eligible children with autism.   J Am Acad Child Adolesc Psychiatry. 2002;41(12):1447-1453. doi:10.1097/00004583-200212000-00016 PubMedGoogle ScholarCrossref
4.
McGuire  TG, Miranda  J.  New evidence regarding racial and ethnic disparities in mental health: policy implications.   Health Aff (Millwood). 2008;27(2):393-403. doi:10.1377/hlthaff.27.2.393PubMedGoogle ScholarCrossref
5.
Baio  J, Wiggins  L, Christensen  DL,  et al.  Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2014.   MMWR Surveill Summ. 2018;67(6):1-23. doi:10.15585/mmwr.ss6706a1PubMedGoogle ScholarCrossref
6.
Maenner  MJ, Shaw  KA, Baio  J,  et al; EdS1; PhD-7.  Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2016.   MMWR Surveill Summ. 2020;69(4):1-12. doi:10.15585/mmwr.ss6904a1 PubMedGoogle ScholarCrossref
7.
Dawson  G, Rogers  S, Munson  J,  et al.  Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model.   Pediatrics. 2010;125(1):e17-e23. doi:10.1542/peds.2009-0958 PubMedGoogle ScholarCrossref
8.
Landa  RJ, Kalb  LG.  Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention.   Pediatrics. 2012;130(suppl 2):S186-S190. doi:10.1542/peds.2012-0900Q PubMedGoogle ScholarCrossref
9.
Kasari  C, Gulsrud  AC, Wong  C, Kwon  S, Locke  J.  Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism.   J Autism Dev Disord. 2010;40(9):1045-1056. doi:10.1007/s10803-010-0955-5 PubMedGoogle ScholarCrossref
10.
National Research Council.  Educating Children with Autism. National Academies Press; 2001.
11.
Dawson  G, Jones  EJH, Merkle  K,  et al.  Early behavioral intervention is associated with normalized brain activity in young children with autism.   J Am Acad Child Adolesc Psychiatry. 2012;51(11):1150-1159. doi:10.1016/j.jaac.2012.08.018 PubMedGoogle ScholarCrossref
12.
Huttenlocher  PR.  Neural Plasticity: The Effects of Environment on the Development of the Cerebral Cortex. Harvard University Press; 2002.
13.
Sullivan  K, Stone  WL, Dawson  G.  Potential neural mechanisms underlying the effectiveness of early intervention for children with autism spectrum disorder.   Res Dev Disabil. 2014;35(11):2921-2932. doi:10.1016/j.ridd.2014.07.027 PubMedGoogle ScholarCrossref
14.
Feinberg  E, Abufhele  M, Sandler  J,  et al.  Reducing disparities in timely autism diagnosis through family navigation: results from a randomized pilot trial.   Psychiatr Serv. 2016;67(8):912-915. doi:10.1176/appi.ps.201500162 PubMedGoogle ScholarCrossref
15.
Feinberg  E, Kuhn  J, Sandler Eilenberg  J,  et al.  Improving family navigation for children with autism: a comparison of two pilot randomized controlled trials.   Acad Pediatr. Published online April 24, 2020. doi:10.1016/j.acap.2020.04.007PubMedGoogle Scholar
16.
Broder-Fingert  S, Walls  M, Augustyn  M,  et al.  A hybrid type I randomized effectiveness-implementation trial of patient navigation to improve access to services for children with autism spectrum disorder.   BMC Psychiatry. 2018;18(1):79. doi:10.1186/s12888-018-1661-7 PubMedGoogle ScholarCrossref
17.
Broder-Fingert  S, Stadnick  NA, Hickey  E, Goupil  J, Diaz Lindhart  Y, Feinberg  E.  Defining the core components of family navigation for autism spectrum disorder.   Autism. 2020;24(2):526-530. doi:10.1177/1362361319864079PubMedGoogle ScholarCrossref
18.
Glasgow  RE, Orleans  CT, Wagner  EH.  Does the chronic care model serve also as a template for improving prevention?   Milbank Q. 2001;79(4):579-612, iv-v. doi:10.1111/1468-0009.00222 PubMedGoogle ScholarCrossref
19.
Freeman  HP, Rodriguez  RL.  History and principles of patient navigation.   Cancer. 2011;117(15)(suppl):3539-3542. doi:10.1002/cncr.26262PubMedGoogle Scholar
20.
Marshall  JK, Mbah  OM, Ford  JG,  et al.  Effect of patient navigation on breast cancer screening among African American Medicare beneficiaries: a randomized controlled trial.   J Gen Intern Med. 2016;31(1):68-76. doi:10.1007/s11606-015-3484-2 PubMedGoogle ScholarCrossref
21.
McKevitt  E, Dingee  C, Warburton  R,  et al.  Patient navigation reduces time to care for patients with breast symptoms and abnormal screening mammograms.   Am J Surg. 2018;215(5):805-811. doi:10.1016/j.amjsurg.2017.12.016 PubMedGoogle ScholarCrossref
22.
McBrien  KA, Ivers  N, Barnieh  L,  et al.  Patient navigators for people with chronic disease: a systematic review.   PLoS One. 2018;13(2):e0191980. doi:10.1371/journal.pone.0191980 PubMedGoogle Scholar
23.
Mizuno  Y, Higa  DH, Leighton  CA, Roland  KB, Deluca  JB, Koenig  LJ.  Is HIV patient navigation associated with HIV care continuum outcomes?   AIDS. 2018;32(17):2557-2571. doi:10.1097/QAD.0000000000001987 PubMedGoogle ScholarCrossref
24.
Kelley  L, Capp  R, Carmona  JF,  et al.  Patient navigation to reduce emergency department (ED) utilization among Medicaid insured, frequent ED users: a randomized controlled trial.   J Emerg Med. 2020;58(6):967-977. doi:10.1016/j.jemermed.2019.12.001 PubMedGoogle ScholarCrossref
25.
Gordon  J.  Towards Interventions Across the Autism Spectrum. National Institute of Mental Health; 2017.
26.
StudyTRAX, version 3.73.0020. ScienceTRAX LLC; 2020.
27.
Robins  DL, Casagrande  K, Barton  M, Chen  C-MA, Dumont-Mathieu  T, Fein  D.  Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up (M-CHAT-R/F).   Pediatrics. 2014;133(1):37-45. doi:10.1542/peds.2013-1813 PubMedGoogle ScholarCrossref
28.
Borrelli  B.  The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials.   J Public Health Dent. 2011;71(s1)(suppl 1):S52-S63. doi:10.1111/j.1752-7325.2011.00233.x PubMedGoogle ScholarCrossref
29.
Madson  MB, Campbell  TC, Barrett  DE, Brondino  MJ, Melchert  TP.  Development of the Motivational Interviewing Supervision and Training Scale.   Psychol Addict Behav. 2005;19(3):303-310. doi:10.1037/0893-164X.19.3.303 PubMedGoogle ScholarCrossref
30.
Lord  C, Rutter  M, DiLavore  P, Risi  S, Gotham  K, Bishop  S.  Autism Diagnostic Observation Schedule, 2nd ed. Western Psychological Services; 2012.
31.
American Psychiatric Association.  Diagnostic and Statistical Manual of Mental Disorders. 5th ed. American Psychiatric Association; 2013.
32.
Zuckerman  KE, Sinche  B, Mejia  A, Cobian  M, Becker  T, Nicolaidis  C.  Latino parents’ perspectives on barriers to autism diagnosis.   Acad Pediatr. 2014;14(3):301-308. doi:10.1016/j.acap.2013.12.004 PubMedGoogle ScholarCrossref
33.
Magaña  S, Lopez  K, Aguinaga  A, Morton  H.  Access to diagnosis and treatment services among Latino children with autism spectrum disorders.   Intellect Dev Disabil. 2013;51(3):141-153. doi:10.1352/1934-9556-51.3.141 PubMedGoogle ScholarCrossref
34.
Lopez  K.  Sociocultural perspectives of Latino children with autism spectrum disorder.   Best Practices Ment Health. 2014;10(2):15-31.Google Scholar
35.
Blanche  EI, Diaz  J, Barretto  T, Cermak  SA.  Caregiving experiences of Latino families with children with autism spectrum disorder.   Am J Occup Ther. 2015;69(5):5010p1-11. doi:10.5014/ajot.2015.017848PubMedGoogle Scholar
36.
Kraemer  HC, Mintz  J, Noda  A, Tinklenberg  J, Yesavage  JA.  Caution regarding the use of pilot studies to guide power calculations for study proposals.   Arch Gen Psychiatry. 2006;63(5):484-489. doi:10.1001/archpsyc.63.5.484 PubMedGoogle ScholarCrossref
37.
Wei  LJ, Lin  DY, Weissfeld  L.  Regression analysis of multivariate incomplete failure time data by modeling marginal distributions.   J Am Stat Assoc. 1989;84(408):1065-1073. doi:10.1080/01621459.1989.10478873 Google ScholarCrossref
38.
Lin  DY, Wei  LJ, Ying  Z.  Checking the Cox model with cumulative sums of martingale-based residuals.   Biometrika. 1993;80(3):557-575. doi:10.1093/biomet/80.3.557 Google ScholarCrossref
39.
US Department of Education. Individuals with Disabilities Education Act (IDEA) Part C—Infants and Toddlers with Disabilities, Pub L No. 108–446, §631 (2004).
Original Investigation
January 11, 2021

Effect of Family Navigation on Diagnostic Ascertainment Among Children at Risk for Autism: A Randomized Clinical Trial From DBPNet

Author Affiliations
  • 1Division of General Academic Pediatrics, Department of Pediatrics, Boston University School of Medicine, Boston, Massachusetts
  • 2Department of Community Health Sciences, Boston University School of Public Health, Boston, Massachusetts
  • 3Division of Developmental and Behavioral Pediatrics, Department of Pediatrics, Boston University School of Medicine, Boston, Massachusetts
  • 4Department of Pediatrics, Boston Medical Center, Boston, Massachusetts
  • 5Division of Developmental and Behavioral Pediatrics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
  • 6Developmental and Behavioral Pediatrics, Division of General Pediatrics, Department of Pediatrics, Yale School of Medicine, New Haven, Connecticut
  • 7Now with Division of Developmental Medicine, Boston Children’s Hospital, Boston, Massachusetts
  • 8Now with Waisman Center, University of Wisconsin–Madison, Madison
  • 9Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts
  • 10Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts
  • 11Biostatistics, Epidemiology, and Data Analytics Center, Boston University School of Public Health, Boston, Massachusetts
  • 12The Heller School for Social Policy and Management, Brandeis University, Waltham, Massachusetts
  • 13Division of Developmental and Behavioral Pediatrics, Department of Pediatrics, The Perelman School of Medicine at the University of Pennsylvania, Philadelphia
  • 14Department of Pediatrics, The Perelman School of Medicine at the University of Pennsylvania, Philadelphia
  • 15Department of Biostatistics, Epidemiology, and Informatics, The Perelman School of Medicine at the University of Pennsylvania, Philadelphia
JAMA Pediatr. 2021;175(3):243-250. doi:10.1001/jamapediatrics.2020.5218
Key Points

Question  Does family navigation (FN), an individually tailored, culturally informed care management strategy, increase the likelihood of achieving diagnostic ascertainment among children from low-income, racial/ethnic minority families who have positive screening results for autism spectrum disorder (ASD)?

Findings  In this multisite randomized clinical trial of 249 children aged 15 to 27 months, FN decreased the time to diagnostic ascertainment and increased the likelihood of diagnostic ascertainment over the course of 1 year. However, Hispanic ethnicity and site moderated the effect of FN.

Meaning  Family navigation is a promising approach to improve early ASD diagnosis among children from low-income, racial/ethnic minority families; its effects may be contextually dependent.

Abstract

Importance  Early identification of autism spectrum disorder (ASD) is associated with improved cognitive and behavioral outcomes. Targeted strategies are needed to support equitable access to diagnostic services to ensure that children from low-income and racial/ethnic minority families receive the benefits of early ASD identification and treatment.

Objective  To test the efficacy of family navigation (FN), an individually tailored, culturally informed care management strategy, to increase the likelihood of achieving diagnostic ascertainment among young children at risk for ASD.

Design, Setting, and Participants  This randomized clinical trial of 249 families of children aged 15 to 27 months who had positive screening results for possible ASD was conducted in 11 urban primary care sites in 3 cities. Data collection occurred from February 24, 2015, through November 5, 2018. Statistical analysis was performed on an intent-to-treat basis from November 5, 2018, to July 27, 2020.

Interventions  Families were randomized to FN or conventional care management (CCM). Families receiving FN were assigned a navigator who conducted community-based outreach to families to address structural barriers to care and support engagement in recommended services. Families receiving CCM were assigned to a care manager, who did limited telephone outreach. Families received FN or CCM after positive initial screening results and for 100 days after diagnostic ascertainment.

Main Outcomes and Measures  The primary outcome, diagnostic ascertainment, was measured as the number of days from randomization to completion of the child’s clinical developmental evaluation, when a diagnosis of ASD or other developmental disorder was determined.

Results  Among 250 families randomized, 249 were included in the primary analysis (174 boys [69.9%]; mean [SD] age, 22.0 [3.5] months; 205 [82.3%] publicly insured; 233 [93.6%] non-White). Children who received FN had a greater likelihood of reaching diagnostic ascertainment over the course of 1 year (FN, 108 of 126 [85.7%]; CCM, 94 of 123 [76.4%]; unadjusted hazard ratio [HR], 1.39 [95% CI, 1.05-1.84]). Site (Boston, New Haven, and Philadelphia) and ethnicity (Hispanic vs non-Hispanic) moderated the effect of FN (treatment × site interaction; P = .03; Boston: HR, 2.07 [95% CI, 1.31-3.26]; New Haven: HR, 1.91 [95% CI, 0.94-3.89]; and Philadelphia: HR, 0.91 [95% CI, 0.60-1.37]) (treatment × ethnicity interaction; P < .001; Hispanic families: HR, 2.81 [95% CI, 2.23-3.54] vs non-Hispanic families: HR, 1.49 [95% CI, 1.45-1.53]). The magnitude of FN’s effect was significantly greater among Hispanic families than among non-Hispanic families (diagnostic ascertainment among Hispanic families: FN, 90.9% [30 of 33], and CCM, 53.3% [16 of 30]; vs non-Hispanic families: FN, 89.7% [35 of 39], and CCM, 77.5% [31 of 40]).

Conclusions and Relevance  Family navigation improved the likelihood of diagnostic ascertainment among children from racial/ethnic minority, low-income families who were detected as at risk for ASD in primary care. Results suggest differential effects of FN by site and ethnicity.

Trial Registration  ClinicalTrials.gov Identifier: NCT02359084

Introduction

Disparities exist in access to autism spectrum disorder (ASD) diagnostic and treatment services for low-income and racial/ethnic minority families.1-4 The consequences of existing disparities have become evident as both the prevalence of ASD and evidence of effectiveness of ASD-specific treatments have increased.5-9 Autism spectrum disorder now affects 1 in 54 children in the United States.6 High-quality studies demonstrate the effectiveness of early intensive behavioral intervention to improve skills and reduce ASD-related impairments.7-11 Such services are more efficacious when they are initiated at younger ages.12,13 Because a formal ASD diagnosis is typically required to qualify for ASD-specific services, decreasing disparities in the age at ASD diagnosis is a critical step to ensuring equitable early access to evidence-based services, and plausibly to better outcomes. Without explicit strategies to promote early identification and engagement with diagnostic and treatment services among low-income and racial/ethnic minority children, disparities in ASD outcomes are likely to persist.

There is growing interest and investment in evidence-based approaches to reduce disparities and address barriers to accessing ASD-specific diagnostic and treatment services.14,15 Family navigation (FN) is 1 strategy to address barriers to care.14-16 The goal of FN is to support families in overcoming structural and psychological obstacles (eg, transportation, language, fear, and stigma) to achieve diagnostic ascertainment and engagement in recommended services.17 Rooted in the chronic care model,18 FN shares basic tenets of patient navigation.19 Patient navigation originally targeted individual patients with cancer risk.20,21 It focused on the period from a suspicious screening result through diagnostic ascertainment and engagement in recommended services, addressing the “discovery to diagnosis disconnect.”19(p5) Its use has expanded to other conditions and currently addresses a range of barriers to care.22-24 Similar to the original navigation model, FN focuses on the period beginning with a suspicious screening result but expands the model to engage the entire family unit rather than just the individual. Family navigation uses community health workers, referred to as navigators, who are trained in motivational interviewing, collaborative problem solving, and psychoeducation, to accomplish family goals, navigate barriers to care, and provide cross-sector care coordination.17

As 1 of 5 studies of the National Institute of Mental Health’s ASD Pediatric Early Detection, Engagement, and Services Research Network,25 this multisite randomized clinical trial is the first, to our knowledge, to systematically assess the effect of FN on time to diagnostic ascertainment among children with an initial positive screening result for ASD.

Methods

We conducted a parallel-group, randomized trial of families of children aged 15 to 27 months identified as at risk for ASD based on screening and surveillance in primary care. The trial took place in 11 urban pediatric primary care practices, which were part of 3 integrated care networks in Boston, Massachusetts; New Haven, Connecticut; and Philadelphia, Pennsylvania, and their developmental and behavioral pediatrics (DBP) specialty clinics. Each clinic was a member of the Health Resources and Services Administration–funded DBP Research Network (DBPNet). The full trial protocol is provided in Supplement 1.16 Study procedures were approved by the Boston University Medical Center Institutional Review Board. Parents and/or guardians provided written informed consent. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

Families were recruited from February 12, 2015, to October 23, 2017; data collection occurred through November 5, 2018. After completing the baseline assessment, participants (parent-child dyads) were randomized 1:1 to FN or conventional care management (CCM) using randomly permuted blocks of 2 and 4. Participants were stratified by primary care site and receipt of prescreening educational materials. Randomization lists were generated for each site by a secure web-based data management system.26 Investigators and staff responsible for data collection were masked to study allocation.

Participants

Children were identified as being at risk for ASD based on results from the Modified Checklist for Autism in Toddlers, Revised with Follow-up (MCHAT-R/F),27 which was administered at the 18-month and 24-month well-child visits as standard care.23,25 Research staff confirmed ASD risk by readministering the MCHAT-R/F using recommended administration and scoring procedures. Readministration ensured consistency across sites and that all children received the MCHAT-R Follow-up Interview, which decreases the number of children requiring further evaluation.27 To emulate conditions in real-world practices, primary care clinicians who had high levels of concern about a child were allowed to override negative screening results and recommend study enrollment. Children who had a previous ASD diagnosis or who were in custody of child protective services were excluded. No families were excluded based on language.

Interventions
Family Navigation

Family navigation is a structured, manualized intervention consisting of 11 core components.17 Navigators were predominantly bilingual, bicultural members of the community, who received training in ASD management, community resources, motivational interviewing, and principles of patient navigation.19 Family navigation was designed to support families from the time of a positive ASD screening result through engagement in recommended services; it included a minimum of 3 standardized visits that aligned with critical time points in the child’s diagnostic process. In this report, we present only the FN components that supported families through diagnostic ascertainment. These components were a minimum of 1 structured visit conducted prior to the diagnostic evaluation to prepare the family and identify barriers to engagement. Additional in-person and remote contacts occurred based on families’ needs (eg, evaluation-specific service outreach, housing, and transportation). Navigators had cellular telephones to facilitate communication via telephone, text, or email; they also had a subscription to a car service for travel to families’ homes. Contacts were initiated by navigators or families; there was no limit to the amount of contacts.

Navigators worked closely with existing staff at primary care sites but were supervised centrally by the project team. They participated in monthly group and individual supervision conducted by Boston’s lead navigator. To assess fidelity, 20% of FN sessions between each navigator and family were randomly selected and audiotaped. A blinded assessor evaluated FN visit content fidelity using a standardized visit checklist; visits in which 80% of the items were present were considered to have met fidelity.28 Fidelity to motivational interviewing principles was assessed using the Motivational Interviewing Supervision and Training Scale.29 The threshold for motivational interviewing fidelity was set at 70% because the varied nature of FN interactions did not support the use of all motivational interviewing tenets in all visits. A second assessor cross-checked 13% of FN fidelity recordings and 15% of Motivational Interviewing Supervision and Training Scale recordings.

Conventional Care Management

Conventional care management exceeded usual care at all sites and was provided in addition to existing procedures. Usual care was enhanced by low-cost, easily implemented procedures that could be integrated into existing workflows, including a manualized protocol of outreach to families and the child’s primary care clinician and a designated direct line to reach the care manager. Outreach to families consisted of an introductory telephone call, during which care managers reminded families about their child’s DBP intake appointment, offered to answer questions about the developmental assessment and developmental services, and provided resources to community services to address social needs. If the family was not reached by telephone after 3 attempts, a letter was mailed to the family introducing the care manager and the date of the child’s appointment. Outreach to the child’s primary care clinician included a letter to introduce the role of the care manager and provide contact information. Conventional care management was delivered by designated care managers. They were existing staff who had access to all resources at their site, including interpreter services. At 2 of the 3 sites, designated care managers’ race/ethnicity mirrored that of participants. All other contacts with care managers after the initial telephone call were initiated by the family or other members of the child’s care team. All study participants, regardless of randomization group, were offered the next available DBP appointment for ASD diagnostic evaluation.

Primary Outcome

The primary outcome was time to completion of a clinical diagnostic evaluation, during which a diagnosis of ASD or other developmental condition was made. This time period was measured as the number of days from randomization to the date of diagnostic ascertainment, obtained from the child’s medical record. As a pragmatic trial, the diagnostic evaluations were completed within the context of DBP standard clinical care, which typically included a detailed history and administration of the Autism Diagnostic Observation Schedule, 2nd edition,30 to inform Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) ASD diagnostic criteria.31

Other Subgroup Analyses

We identified 2 a priori, theory-based, potential effect modifiers: site and ethnicity. Site was modeled as a state-level variable and hypothesized to be a potential effect modifier based on state differences in available services for children with ASD; Hispanic ethnicity was examined as an effect modifier based on the large body of research describing substantial barriers to care experienced by Hispanic families.32-35 We also conducted analyses comparing children enrolled based on screening results with those enrolled based on clinical concerns alone.

Sample Size

Using the threshold of clinical significance from Kraemer et al,36 we considered a 25% absolute difference in diagnostic ascertainment rates between treatment groups to be clinically meaningful and to provide evidence of the efficacy of FN compared with CCM. Based on pilot studies,14,15 assuming that 65% of children in the CCM group would achieve diagnostic ascertainment and a moderate design effect resulting from site clustering (intraclass correlation of 0.01), a sample size of 250 was estimated to detect a 25% difference in diagnostic ascertainment with 80% power at a 2-sided α of .05.

Statistical Analysis

Statistical analysis was performed on an intent-to-treat basis from November 5, 2018, to July 27, 2020. Analysis of the primary outcome included all randomized participants, with the exception of 1 postrandomization exclusion of a child who received care from a nonparticipating primary care site. Time to diagnostic ascertainment was analyzed descriptively using Kaplan-Meier curves. Children who did not complete a diagnostic evaluation within 1 year of study enrollment were censored. Cox proportional hazards regression was performed to estimate the hazard ratio (HR) and 95% CI for the difference in the likelihood of achieving diagnostic ascertainment. We assessed potential clustering by site using the method of Wei et al.37 We performed formal testing of interaction terms to assess effect modification followed by stratified analyses. We tested the proportional hazards assumption for the intervention effect using a Kolmogorov-type, simulation-based supremum test as well as graphically examining plots of cumulative Martingale residuals.38

Results

A total of 616 families were referred to the study: 211 did not meet inclusion criteria; 86 could not be contacted; 69 declined participation. A total of 250 families were randomly assigned to an intervention; 249 received the allocated intervention (126 families received FN and 123 families received CCM) (Figure 1).

Baseline Data

There were no clinically significant differences in sociodemographic variables by treatment group at baseline. The sample was racially and ethnically diverse, with 72 Hispanic participants (28.9%), 131 non-Hispanic Black participants (52.6%), 16 non-Hispanic White participants (6.4%), and 30 participants of other races/ethnicities (12.0%) (Table). The mean (SD) parent age was 31.4 (7.3) years. Ninety-eight parents (39.7%) were born outside the United States; 45 (18.1%) preferred to communicate in a language other than English. The mean (SD) child age was 22.0 (3.5) months, 174 children (69.9%) were male, 205 (82.3%) received public insurance, and 109 (43.8%) were receiving IDEA Part C39 Birth to Three developmental services at enrollment. Seven children (2.8%) were enrolled based on clinical concerns alone.

Protocol Adherence

Prior to diagnostic ascertainment, 92.9% of families (117 of 126) assigned to FN had at least 1 face-to-face navigation encounter. The mean (SD) number of contacts with the navigator was 19.6 (12.7): 2.6 (1.3) in-person visits, 9.2 (6.8) telephone calls, 6.9 (4.8) texts, and 2.7 (4.8) emails. Seventeen of 126 families (13.5%) disengaged from the intervention, defined as no navigator contact for more than 10 weeks. Of reviewed visits, 80.0% (40 of 50) met criteria for visit content fidelity and 72.3% (34 of 47) for motivational interviewing fidelity. Within the CCM group, 95.9% of families (118 of 123) received the protocol as described; 26.8% (33 of 123) initiated outreach to the care manager. The mean (SD) number of contacts with the care manager beyond the introductory telephone call was 2.4 (2.1): 1.8 (1.8) telephone calls, 0.07 (0.3) texts, and 0.4 (0.9) emails. No associations between intervention fidelity and study outcomes were observed.

Diagnostic Ascertainment

Time to diagnostic ascertainment is shown in Figure 2. The proportion who reached diagnostic ascertainment within 1 year was 85.7% (108 of 126) in the FN group and 76.4% (94 of 123) in the CCM group (unadjusted HR, 1.39 [95% CI, 1.05-1.84]; P = .02). Among 202 families who achieved diagnostic ascertainment, 118 children (58.4%) received an ASD diagnosis; differences between groups in receipt of an ASD diagnosis were not significant. All 84 children who did not receive an ASD diagnosis received a diagnosis of other developmental conditions.

Subgroup Analyses

When site was included as a clustering variable in the model, the HR remained the same, but the 95% CI widened and the association was no longer statistically significant (HR, 1.39 [95% CI, 0.88-2.21]; P = .16), likely owing to loss of power related to nonindependence of events. We tested for effect modification by site and found a significant treatment × site interaction (P = .03). Site-specific Kaplan-Meier plots are shown in Figure 3. In Cox proportional hazards regression models stratified by site, children in Boston and New Haven who received FN were about twice as likely as families who received CCM to achieve diagnostic ascertainment (Boston: HR, 2.07 [95% CI, 1.31-3.26]; P = .002; and New Haven: HR, 1.91 [95% CI, 0.94-3.89]; P = .08). In Philadelphia, there was no difference in the likelihood of diagnostic ascertainment between CCM and FN (HR, 0.91 [95% CI, 0.60-1.37]; P = .65), likely a result of Philadelphia’s high ascertainment rate among CCM families compared with that of other sites.

The effect of ethnicity was assessed among participants from Boston and New Haven only, as Philadelphia enrolled very few Hispanic families. Given the difference in intervention effect between Philadelphia and the other sites, its inclusion necessitated a 3-way site × ethnicity × treatment interaction term, which the sample size did not support. Figure 4 displays Kaplan-Meier plots by ethnicity for Boston and New Haven families. The difference between the proportions who achieved diagnostic resolution in FN and CCM groups was greater among Hispanic families (90.9% [30 of 33] vs 53.3% [16 of 30]) than among non-Hispanic families (89.7% [35 of 39] vs 77.5% [31 of 40]). In the Cox proportional hazards regression models, the treatment × ethnicity interaction term was significant (P < .001). In models stratified by ethnicity, the magnitude of the effect of FN was greater for Hispanic than non-Hispanic families (Hispanic families: HR, 2.81 [95% CI, 2.23-3.54]; and non-Hispanic families: HR, 1.49 [95% CI, 1.45-1.53]).

To assess the effect of including participants based on clinician concerns alone, we compared sociodemographic characteristics and diagnostic outcomes between these 7 children and the 242 children enrolled because of screening results. We did not find differences between the 2 groups.

Discussion

To our knowledge, this is the first multisite randomized clinical trial of a primary care–based FN intervention designed specifically to improve early evaluation of low-income, racial/ethnic minority children at risk for ASD. Compared with CCM, FN increased the likelihood of achieving diagnostic ascertainment within a 1-year period. The effects varied by site and ethnicity, however, suggesting that the positive effects of FN are contextually dependent. At 2 of the 3 research sites (Boston and New Haven), children who received FN were about twice as likely to achieve diagnostic ascertainment; no difference was observed at the third site (Philadelphia), where both FN and CCM families had high rates of diagnostic ascertainment. Hispanic families benefited more from FN compared with their non-Hispanic counterparts. Although the FN main effect was below the 25% threshold that we set a priori as clinically significant, the difference between FN and CCM participants who achieved diagnostic ascertainment approached or exceeded this threshold at 2 of the 3 sites (Boston, 23% difference; and New Haven, 32% difference) and among Hispanic participants (41% difference). These findings suggest the potential of FN to affect disparities in ASD diagnostic ascertainment in many, but not all, health service settings and populations experiencing disparities.

Initially, we hypothesized that any observed effect modification would result from differences in FN efficacy. However, we found that FN outcomes were fairly consistent across sites (85.7% [108 of 126] achieved diagnostic ascertainment). Instead, the observed effect modification by site and ethnicity was better explained by variations in diagnostic ascertainment among CCM families.

A possible explanation for this finding relates to site racial/ethnic demographic characteristics. Sociodemographic characteristics were similar across sites, with the exception of race/ethnicity and variables associated with immigration status. The Philadelphia primary care sites served predominantly English-speaking Black families (76.6% [82 of 107]); the Boston and New Haven sites served more Hispanic (44.4% [63 of 142]) and ethnically diverse non-US–born families (59.6% [84 of 141]). It is possible that Hispanic and other ethnic minority participants experienced greater difficulty accessing health services than the English-speaking Black families owing to language and cultural barriers. Thus, they accrued greater benefits of FN. Common experiences, values, and other cultural influences among Hispanic families may make FN a particularly effective intervention for this subgroup. Family navigation is rooted in tenets of culturally informed care; thus, navigators were selected based on their racial/ethnic background and knowledge of the local community. The cultural concordance between families and the navigators may have had greater saliency for Hispanic than Black families. What remains unexplained is why Black families, who have historically experienced delays in ASD diagnosis,3 had a higher than expected rate of diagnostic ascertainment in Philadelphia, despite consistent implementation of CCM and a relatively similar standard of care across sites. Findings suggest that FN may be most valuable when baseline diagnostic ascertainment rates are low and may be relevant to other conditions for which disparities in diagnostic ascertainment exist.

Limitations

This study has some limitations. Analyses of moderation by site and ethnicity provided initial explanations regarding contextual factors and their effects on FN. However, we were unable to identify specific factors contributing to differential site effects, particularly those that supported engagement of Black families. Study findings may not be generalizable to other sites, as urban care networks have site-specific screening practices and exist within state-specific service systems. Regarding potential selection bias, we were unable to assess differences between study participants and the 155 parent-child dyads who could not be contacted or declined to participate. We also did not assess associations between early ASD diagnosis and functional outcomes.

Conclusions

Study findings provide evidence that a replicable model of FN increased the likelihood of ASD diagnostic ascertainment. Because diagnostic ascertainment is the first step in the pathway to receipt of evidence-based services, earlier diagnostic ascertainment has the potential to improve outcomes for vulnerable ethnically and racially diverse young children with ASD and other developmental disabilities. Future analyses of the contextual factors that affect the effectiveness of FN in various settings and populations are necessary to determine the conditions under which implementing FN is most likely to yield the strongest benefits.

Back to top
Article Information

Accepted for Publication: July 28, 2020.

Published Online: January 11, 2021. doi:10.1001/jamapediatrics.2020.5218

Correction: This article was corrected on March 8, 2021, to add a name to the byline and update the Author Contributions.

Corresponding Author: Emily Feinberg, ScD, CPNP, Department of Pediatrics, Boston University School of Medicine, 801 Albany St, Second Floor, Boston, MA 02118 (emfeinbe@bu.edu).

Author Contributions: Dr Feinberg had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Feinberg, Augustyn, Broder-Fingert, Bennett, Weitzman, Silverstein, Cabral, Diaz-Linhart, Fernandez-Pastrana, Fenick, Blum.

Acquisition, analysis, or interpretation of data: Feinberg, Augustyn, Broder-Fingert, Bennett, Weitzman, Kuhn, Hickey, Chu, Levinson, Sandler Eilenberg, Silverstein, Cabral, Patts, Fernandez-Pastrana, Rosenberg, Miller, Guevara, Fenick, Blum.

Drafting of the manuscript: Feinberg, Augustyn, Broder-Fingert, Weitzman, Kuhn, Hickey, Levinson, Cabral, Patts, Rosenberg.

Critical revision of the manuscript for important intellectual content: Feinberg, Broder-Fingert, Bennett, Weitzman, Kuhn, Chu, Sandler Eilenberg, Silverstein, Cabral, Patts, Diaz-Linhart, Fernandez-Pastrana, Rosenberg, Miller, Guevara, Fenick, Blum.

Statistical analysis: Feinberg, Kuhn, Hickey, Levinson, Cabral, Patts, Rosenberg.

Obtained funding: Feinberg, Weitzman.

Administrative, technical, or material support: Feinberg, Augustyn, Broder-Fingert, Bennett, Weitzman, Kuhn, Chu, Levinson, Sandler Eilenberg, Silverstein, Diaz-Linhart, Fernandez-Pastrana, Miller, Guevara, Blum.

Supervision: Feinberg, Augustyn, Broder-Fingert, Bennett, Chu, Silverstein, Fernandez-Pastrana, Blum.

Conflict of Interest Disclosures: Dr Bennett reported receving grants from the National Institute of Mental Health (NIMH) during the conduct of the study; grants from Autism Speaks, Roche Pharmaceuticals, Stemina Biomarker Discovery, and Neurim Pharmaceuticals outside the submitted work; and spouse is employed at Pfizer but not in a field relevant to this research. Dr Weitzman reported receiving honoraria from the Journal of Developmental Behavioral Pediatrics for being a section editor; and receiving royalties from Up to Date for writing articles. Dr Silverstein reported serving as a member of the United States Preventive Services Task Force. Ms Diaz-Linhart reported receiving grants from Boston Medical Center during the conduct of the study. Dr. Miller reported providing legal consultation related to autism spectrum disorders; and receiving grant funding from Lumos Pharma and Ultragenyx Pharmaceuticals. Dr Guevara reported receiving grants from the National Institutes of Health (NIH) and the Patient Centered Outcomes Research Institute during the conduct of the study. Dr Fenick reported receiving grants from the NIH during the conduct of the study. Dr Blum reported receiving grants from NIMH during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was funded by the National Institute of Mental Health (grant R01MH104355), NIMH’s national ASD Pediatric Early Detection, Engagement, and Service (PEDS) Network. It was conducted in collaboration with the Developmental and Behavioral Pediatrics Research Network (DBPNet). DBPNet is supported by cooperative agreement UA3MC20218 from the Maternal and Child Health Bureau, Health Resources and Services Administration (HRSA), US Department of Health and Human Services. Fellowship funding came from an HRSA Institutional Training Grant (T32HS10038) and an Agency for Healthcare Research and Quality Training in Health Services Research for Vulnerable Populations Grant (2T32HS022242). Additional funding was provided by a NIMH Mentored Patient-Oriented Research Career Development Award (5K23MH109673) and a NIMH Research Supplement to Promote Diversity In Health-Related Research (3R01MH104355-02S1).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, Health Resources and Services Administration, US Department of Health and Human Services, or the US government.

Meeting Presentation: This study was presented at the Annual Meeting of the Pediatric Academic Society; April 29, 2019; Baltimore, Maryland; and the Annual Meeting of the International Society for Autism Research; May 3, 2019; Montreal, Québec, Canada.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We would like to acknowledge the participating families, without whom this study would not have been possible. We would also like to acknowledge the family navigators, study coordinators, and research staff who played a critical role in study implementation: Manju Abraham, MS, Syneos Health, Jenny Acevedo-Usuga, MS, BCBA, LBA, Behavioral Consulting Associates, Marisol Credle, MA, Yale School of Medicine, Sonia Erlich, MFA, MS, LMHC, Boston Medical Center, Mitsouka Exantus, BA, Boston Medical Center, Ivys Fernandez-Pastrana, JD, Boston Medical Center, Julianna Gardener, MA, Trinity School of Medicine, Nadia Martinez, BA, CommuniCare, Plyce Fuchu, BS, Boston University School of Public Health, Nicole O’Dea, MA, Clark University and Boston Children’s Hospital, and Samia Omer, BS, Children’s Hospital of Philadelphia. They were all compensated for their contributions.

References
1.
Durkin  MS, Maenner  MJ, Baio  J,  et al.  Autism spectrum disorder among US children (2002-2010): socioeconomic, racial, and ethnic disparities.   Am J Public Health. 2017;107(11):1818-1826. doi:10.2105/AJPH.2017.304032 PubMedGoogle ScholarCrossref
2.
Lu  W.  Child and adolescent mental disorders and health care disparities: results from the National Survey of Children’s Health, 2011–2012.   J Health Care Poor Underserved. 2017;28(3):988-1011. doi:10.1353/hpu.2017.0092 PubMedGoogle ScholarCrossref
3.
Mandell  DS, Listerud  J, Levy  SE, Pinto-Martin  JA.  Race differences in the age at diagnosis among Medicaid-eligible children with autism.   J Am Acad Child Adolesc Psychiatry. 2002;41(12):1447-1453. doi:10.1097/00004583-200212000-00016 PubMedGoogle ScholarCrossref
4.
McGuire  TG, Miranda  J.  New evidence regarding racial and ethnic disparities in mental health: policy implications.   Health Aff (Millwood). 2008;27(2):393-403. doi:10.1377/hlthaff.27.2.393PubMedGoogle ScholarCrossref
5.
Baio  J, Wiggins  L, Christensen  DL,  et al.  Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2014.   MMWR Surveill Summ. 2018;67(6):1-23. doi:10.15585/mmwr.ss6706a1PubMedGoogle ScholarCrossref
6.
Maenner  MJ, Shaw  KA, Baio  J,  et al; EdS1; PhD-7.  Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2016.   MMWR Surveill Summ. 2020;69(4):1-12. doi:10.15585/mmwr.ss6904a1 PubMedGoogle ScholarCrossref
7.
Dawson  G, Rogers  S, Munson  J,  et al.  Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model.   Pediatrics. 2010;125(1):e17-e23. doi:10.1542/peds.2009-0958 PubMedGoogle ScholarCrossref
8.
Landa  RJ, Kalb  LG.  Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention.   Pediatrics. 2012;130(suppl 2):S186-S190. doi:10.1542/peds.2012-0900Q PubMedGoogle ScholarCrossref
9.
Kasari  C, Gulsrud  AC, Wong  C, Kwon  S, Locke  J.  Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism.   J Autism Dev Disord. 2010;40(9):1045-1056. doi:10.1007/s10803-010-0955-5 PubMedGoogle ScholarCrossref
10.
National Research Council.  Educating Children with Autism. National Academies Press; 2001.
11.
Dawson  G, Jones  EJH, Merkle  K,  et al.  Early behavioral intervention is associated with normalized brain activity in young children with autism.   J Am Acad Child Adolesc Psychiatry. 2012;51(11):1150-1159. doi:10.1016/j.jaac.2012.08.018 PubMedGoogle ScholarCrossref
12.
Huttenlocher  PR.  Neural Plasticity: The Effects of Environment on the Development of the Cerebral Cortex. Harvard University Press; 2002.
13.
Sullivan  K, Stone  WL, Dawson  G.  Potential neural mechanisms underlying the effectiveness of early intervention for children with autism spectrum disorder.   Res Dev Disabil. 2014;35(11):2921-2932. doi:10.1016/j.ridd.2014.07.027 PubMedGoogle ScholarCrossref
14.
Feinberg  E, Abufhele  M, Sandler  J,  et al.  Reducing disparities in timely autism diagnosis through family navigation: results from a randomized pilot trial.   Psychiatr Serv. 2016;67(8):912-915. doi:10.1176/appi.ps.201500162 PubMedGoogle ScholarCrossref
15.
Feinberg  E, Kuhn  J, Sandler Eilenberg  J,  et al.  Improving family navigation for children with autism: a comparison of two pilot randomized controlled trials.   Acad Pediatr. Published online April 24, 2020. doi:10.1016/j.acap.2020.04.007PubMedGoogle Scholar
16.
Broder-Fingert  S, Walls  M, Augustyn  M,  et al.  A hybrid type I randomized effectiveness-implementation trial of patient navigation to improve access to services for children with autism spectrum disorder.   BMC Psychiatry. 2018;18(1):79. doi:10.1186/s12888-018-1661-7 PubMedGoogle ScholarCrossref
17.
Broder-Fingert  S, Stadnick  NA, Hickey  E, Goupil  J, Diaz Lindhart  Y, Feinberg  E.  Defining the core components of family navigation for autism spectrum disorder.   Autism. 2020;24(2):526-530. doi:10.1177/1362361319864079PubMedGoogle ScholarCrossref
18.
Glasgow  RE, Orleans  CT, Wagner  EH.  Does the chronic care model serve also as a template for improving prevention?   Milbank Q. 2001;79(4):579-612, iv-v. doi:10.1111/1468-0009.00222 PubMedGoogle ScholarCrossref
19.
Freeman  HP, Rodriguez  RL.  History and principles of patient navigation.   Cancer. 2011;117(15)(suppl):3539-3542. doi:10.1002/cncr.26262PubMedGoogle Scholar
20.
Marshall  JK, Mbah  OM, Ford  JG,  et al.  Effect of patient navigation on breast cancer screening among African American Medicare beneficiaries: a randomized controlled trial.   J Gen Intern Med. 2016;31(1):68-76. doi:10.1007/s11606-015-3484-2 PubMedGoogle ScholarCrossref
21.
McKevitt  E, Dingee  C, Warburton  R,  et al.  Patient navigation reduces time to care for patients with breast symptoms and abnormal screening mammograms.   Am J Surg. 2018;215(5):805-811. doi:10.1016/j.amjsurg.2017.12.016 PubMedGoogle ScholarCrossref
22.
McBrien  KA, Ivers  N, Barnieh  L,  et al.  Patient navigators for people with chronic disease: a systematic review.   PLoS One. 2018;13(2):e0191980. doi:10.1371/journal.pone.0191980 PubMedGoogle Scholar
23.
Mizuno  Y, Higa  DH, Leighton  CA, Roland  KB, Deluca  JB, Koenig  LJ.  Is HIV patient navigation associated with HIV care continuum outcomes?   AIDS. 2018;32(17):2557-2571. doi:10.1097/QAD.0000000000001987 PubMedGoogle ScholarCrossref
24.
Kelley  L, Capp  R, Carmona  JF,  et al.  Patient navigation to reduce emergency department (ED) utilization among Medicaid insured, frequent ED users: a randomized controlled trial.   J Emerg Med. 2020;58(6):967-977. doi:10.1016/j.jemermed.2019.12.001 PubMedGoogle ScholarCrossref
25.
Gordon  J.  Towards Interventions Across the Autism Spectrum. National Institute of Mental Health; 2017.
26.
StudyTRAX, version 3.73.0020. ScienceTRAX LLC; 2020.
27.
Robins  DL, Casagrande  K, Barton  M, Chen  C-MA, Dumont-Mathieu  T, Fein  D.  Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up (M-CHAT-R/F).   Pediatrics. 2014;133(1):37-45. doi:10.1542/peds.2013-1813 PubMedGoogle ScholarCrossref
28.
Borrelli  B.  The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials.   J Public Health Dent. 2011;71(s1)(suppl 1):S52-S63. doi:10.1111/j.1752-7325.2011.00233.x PubMedGoogle ScholarCrossref
29.
Madson  MB, Campbell  TC, Barrett  DE, Brondino  MJ, Melchert  TP.  Development of the Motivational Interviewing Supervision and Training Scale.   Psychol Addict Behav. 2005;19(3):303-310. doi:10.1037/0893-164X.19.3.303 PubMedGoogle ScholarCrossref
30.
Lord  C, Rutter  M, DiLavore  P, Risi  S, Gotham  K, Bishop  S.  Autism Diagnostic Observation Schedule, 2nd ed. Western Psychological Services; 2012.
31.
American Psychiatric Association.  Diagnostic and Statistical Manual of Mental Disorders. 5th ed. American Psychiatric Association; 2013.
32.
Zuckerman  KE, Sinche  B, Mejia  A, Cobian  M, Becker  T, Nicolaidis  C.  Latino parents’ perspectives on barriers to autism diagnosis.   Acad Pediatr. 2014;14(3):301-308. doi:10.1016/j.acap.2013.12.004 PubMedGoogle ScholarCrossref
33.
Magaña  S, Lopez  K, Aguinaga  A, Morton  H.  Access to diagnosis and treatment services among Latino children with autism spectrum disorders.   Intellect Dev Disabil. 2013;51(3):141-153. doi:10.1352/1934-9556-51.3.141 PubMedGoogle ScholarCrossref
34.
Lopez  K.  Sociocultural perspectives of Latino children with autism spectrum disorder.   Best Practices Ment Health. 2014;10(2):15-31.Google Scholar
35.
Blanche  EI, Diaz  J, Barretto  T, Cermak  SA.  Caregiving experiences of Latino families with children with autism spectrum disorder.   Am J Occup Ther. 2015;69(5):5010p1-11. doi:10.5014/ajot.2015.017848PubMedGoogle Scholar
36.
Kraemer  HC, Mintz  J, Noda  A, Tinklenberg  J, Yesavage  JA.  Caution regarding the use of pilot studies to guide power calculations for study proposals.   Arch Gen Psychiatry. 2006;63(5):484-489. doi:10.1001/archpsyc.63.5.484 PubMedGoogle ScholarCrossref
37.
Wei  LJ, Lin  DY, Weissfeld  L.  Regression analysis of multivariate incomplete failure time data by modeling marginal distributions.   J Am Stat Assoc. 1989;84(408):1065-1073. doi:10.1080/01621459.1989.10478873 Google ScholarCrossref
38.
Lin  DY, Wei  LJ, Ying  Z.  Checking the Cox model with cumulative sums of martingale-based residuals.   Biometrika. 1993;80(3):557-575. doi:10.1093/biomet/80.3.557 Google ScholarCrossref
39.
US Department of Education. Individuals with Disabilities Education Act (IDEA) Part C—Infants and Toddlers with Disabilities, Pub L No. 108–446, §631 (2004).
×