Assessment of Emergency Department and Inpatient Use and Costs in Adult and Pediatric Functional Neurological Disorders | Emergency Medicine | JAMA Neurology | JAMA Network
[Skip to Navigation]
Sign In
Table 1.  Clinical Characteristics of FNDs vs Other Neurological Disorders in Adult and Pediatric ED Visits Comparing FNDs With Other Neurological Disordersa
Clinical Characteristics of FNDs vs Other Neurological Disorders in Adult and Pediatric ED Visits Comparing FNDs With Other Neurological Disordersa
Table 2.  Clinical Characteristics of Admissions for FNDs vs Other Neurological Disorders in Adult and Pediatric Populations and Inpatient Health Care Resource Usea
Clinical Characteristics of Admissions for FNDs vs Other Neurological Disorders in Adult and Pediatric Populations and Inpatient Health Care Resource Usea
Table 3.  NIS 2008-2017 Data for Inpatient Charges in Adults 18 Years or Older Comparing FNDs With Other Neurological Disordersa
NIS 2008-2017 Data for Inpatient Charges in Adults 18 Years or Older Comparing FNDs With Other Neurological Disordersa
Table 4.  KID (2003, 2006, 2009, 2012, and 2016) Data for Inpatient Charges, Ages 5 to 17 Years, Comparing FNDs With Other Neurological Disordersa
KID (2003, 2006, 2009, 2012, and 2016) Data for Inpatient Charges, Ages 5 to 17 Years, Comparing FNDs With Other Neurological Disordersa
Table 5.  NEDS (2008-2017) Data for Charges in Adults and Pediatric Patients Comparing FNDs With Other Neurological Disordersa
NEDS (2008-2017) Data for Charges in Adults and Pediatric Patients Comparing FNDs With Other Neurological Disordersa
1.
Espay  AJ, Aybek  S, Carson  A,  et al.  Current concepts in diagnosis and treatment of functional neurological disorders.   JAMA Neurol. 2018;75(9):1132-1141. doi:10.1001/jamaneurol.2018.1264 PubMedGoogle Scholar
2.
Nimnuan  C, Hotopf  M, Wessely  S.  Medically unexplained symptoms: an epidemiological study in seven specialities.   J Psychosom Res. 2001;51(1):361-367. doi:10.1016/S0022-3999(01)00223-9 PubMedGoogle Scholar
3.
Leary  PM.  Conversion disorder in childhood: diagnosed too late, investigated too much?   J R Soc Med. 2003;96(9):436-438. doi:10.1258/jrsm.96.9.436PubMedGoogle Scholar
4.
Stone  J, Carson  A, Duncan  R,  et al.  Symptoms “unexplained by organic disease” in 1144 new neurology out-patients: how often does the diagnosis change at follow-up?   Brain. 2009;132(pt 10):2878-2888. doi:10.1093/brain/awp220 PubMedGoogle Scholar
5.
Hallett  M.  Psychogenic movement disorders: a crisis for neurology.   Curr Neurol Neurosci Rep. 2006;6(4):269-271. doi:10.1007/s11910-006-0015-x PubMedGoogle Scholar
6.
Lempert  T, Dieterich  M, Huppert  D, Brandt  T.  Psychogenic disorders in neurology: frequency and clinical spectrum.   Acta Neurol Scand. 1990;82(5):335-340. doi:10.1111/j.1600-0404.1990.tb03312.x PubMedGoogle Scholar
7.
Benbadis  SR, O’Neill  E, Tatum  WO, Heriaud  L.  Outcome of prolonged video-EEG monitoring at a typical referral epilepsy center.   Epilepsia. 2004;45(9):1150-1153. doi:10.1111/j.0013-9580.2004.14504.x PubMedGoogle Scholar
8.
Stone  J, Carson  A.  Functional neurologic disorders.   Continuum (Minneap Minn). 2015;21(3 Behavioral Neurology and Neuropsychiatry):818-837. doi:10.1212/01.CON.0000466669.02477.45PubMedGoogle Scholar
9.
Espay  AJ, Goldenhar  LM, Voon  V, Schrag  A, Burton  N, Lang  AE.  Opinions and clinical practices related to diagnosing and managing patients with psychogenic movement disorders: an international survey of Movement Disorder Society members.   Mov Disord. 2009;24(9):1366-1374. doi:10.1002/mds.22618 PubMedGoogle Scholar
10.
Konnopka  A, Schaefert  R, Heinrich  S,  et al.  Economics of medically unexplained symptoms: a systematic review of the literature.   Psychother Psychosom. 2012;81(5):265-275. doi:10.1159/000337349 PubMedGoogle Scholar
11.
Bermingham  SL, Cohen  A, Hague  J, Parsonage  M.  The cost of somatisation among the working-age population in England for the year 2008-2009.   Ment Health Fam Med. 2010;7(2):71-84.PubMedGoogle Scholar
12.
Barsky  AJ, Orav  EJ, Bates  DW.  Somatization increases medical utilization and costs independent of psychiatric and medical comorbidity.   Arch Gen Psychiatry. 2005;62(8):903-910. doi:10.1001/archpsyc.62.8.903 PubMedGoogle Scholar
13.
Magee  JA, Burke  T, Delanty  N, Pender  N, Fortune  GM.  The economic cost of nonepileptic attack disorder in Ireland.   Epilepsy Behav. 2014;33:45-48. doi:10.1016/j.yebeh.2014.02.010 PubMedGoogle Scholar
14.
Carson  A, Lehn  A.  Epidemiology.   Handb Clin Neurol. 2016;139:47-60. doi:10.1016/B978-0-12-801772-2.00005-9 PubMedGoogle Scholar
15.
Healthcare Cost and Utilization Project. Overview of the National (Nationwide) Inpatient Sample (NIS). Accessed December 17, 2019. https://www.hcup-us.ahrq.gov/nisoverview.jsp
16.
Healthcare Cost and Utilization Project. Overview of the Kids’ Inpatient Database (KID). Accessed September 18, 2019. https://www.hcup-us.ahrq.gov/kidoverview.jsp
17.
Healthcare Cost and Utilization Project. Overview of the Nationwide Emergency Department Sample (NEDS). Accessed December 14, 2017. https://www.hcup-us.ahrq.gov/nedsoverview.jsp
18.
Grattan-Smith  PJ, Dale  RC.  Pediatric functional neurologic symptoms.   Handb Clin Neurol. 2016;139:489-498. doi:10.1016/B978-0-12-801772-2.00040-0 PubMedGoogle Scholar
19.
Matin  N, Young  SS, Williams  B,  et al.  Neuropsychiatric associations with gender, illness duration, work disability, and motor subtype in a U.S. functional neurological disorders clinic population.   J Neuropsychiatry Clin Neurosci. 2017;29(4):375-382. doi:10.1176/appi.neuropsych.16110302 PubMedGoogle Scholar
20.
Cubo  E, Hinson  VK, Goetz  CG,  et al.  Transcultural comparison of psychogenic movement disorders.   Mov Disord. 2005;20(10):1343-1345. doi:10.1002/mds.20561 PubMedGoogle Scholar
21.
Anderson  JR, Nakhate  V, Stephen  CD, Perez  DL.  Functional (psychogenic) neurological disorders: assessment and acute management in the emergency department.   Semin Neurol. 2019;39(1):102-114. doi:10.1055/s-0038-1676844 PubMedGoogle Scholar
22.
McKee  K, Glass  S, Adams  C,  et al.  The inpatient assessment and management of motor functional neurological disorders: an interdisciplinary perspective.   Psychosomatics. 2018;59(4):358-368. doi:10.1016/j.psym.2017.12.006 PubMedGoogle Scholar
23.
Duncan  R, Oto  M, Wainman-Lefley  J.  Mortality in a cohort of patients with psychogenic non-epileptic seizures.   J Neurol Neurosurg Psychiatry. 2012;83(7):761-762. doi:10.1136/jnnp-2012-302900 PubMedGoogle Scholar
24.
Galarraga  JE, Pines  JM.  Costs of ED episodes of care in the United States.   Am J Emerg Med. 2016;34(3):357-365. doi:10.1016/j.ajem.2015.06.001PubMedGoogle Scholar
25.
Bai  G, Anderson  GF.  Extreme markup: the fifty US hospitals with the highest charge-to-cost ratios.   Health Aff (Millwood). 2015;34(6):922-928. doi:10.1377/hlthaff.2014.1414 PubMedGoogle Scholar
26.
Kutlubaev  MA, Xu  Y, Hackett  ML, Stone  J.  Dual diagnosis of epilepsy and psychogenic nonepileptic seizures: systematic review and meta-analysis of frequency, correlates, and outcomes.   Epilepsy Behav. 2018;89:70-78. doi:10.1016/j.yebeh.2018.10.010 PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    EXPAND ALL
    Functional Neurological Disorders-Awareness, Cost, Quality and Management
    Khichar Shubhakaran, MD (Med), D.M.(Neurology) | Senior Professor and Head of Department of Neurology, M D M Hospital, Dr. S.N Medical College, Jodhpur, (Rajasthan)-India
    The functional neurological dysfunctions may be chronic but usually present in an emergency setting which increases panic and more frequently results in inpatient admissions, so smooth handling, ideal workup and appropriate guidelines for management and follow up may help in this direction.
    Since there is limited information about awareness, health care use, and costs in patients with functional neurological disorders (FNDs), it should be taken into mainstream.
    There is also dire need for attention to inpatient physical therapy, occupational therapy, speech and language pathology, and psychiatric consultation as most of these patients may also have an attention seeking behavior. /> Because of overlap with refractory neurological disorders like headache, seizure, focal deficit etc..., they should be subcategorized in such cases. Guidelines also need to be framed for such subcategories with respect to appropriate line of management as is with other disorders so as improve management and avoid unwanted costs.
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Original Investigation
    October 26, 2020

    Assessment of Emergency Department and Inpatient Use and Costs in Adult and Pediatric Functional Neurological Disorders

    Author Affiliations
    • 1Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston
    • 2Health Policy Research Center, Mongan Institute, Massachusetts General Hospital, Harvard Medical School, Boston
    • 3Division of Clinical Research, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, Maryland
    • 4James J. and Joan A. Gardner Center for Parkinson’s Disease and Movement Disorders, Department of Neurology, University of Cincinnati, Cincinnati, Ohio
    JAMA Neurol. 2021;78(1):88-101. doi:10.1001/jamaneurol.2020.3753
    Key Points

    Question  What are emergency department and inpatient health care use and costs in patients with functional neurological disorders?

    Findings  In this economic evaluation using large US health care databases of more than 40 000 emergency department visits and 20 000 functional neurological disorder admissions annually from 2008 to 2017, the health care use and annual charges for emergency department and inpatient care were estimated to be more than $1.2 billion per year and rising, comparable to other investigation-intensive and pharmacologically demanding neurological disorders.

    Meaning  Functional neurological disorders are associated with a high level of health care use, with treatment costs comparable to those of care-intensive neurological disorders.

    Abstract

    Importance  There is limited information about health care use and costs in patients with functional neurological disorders (FNDs).

    Objective  To assess US emergency department (ED) and inpatient use and charges for FNDs.

    Design, Setting, and Participants  This economic evaluation used Healthcare Cost and Utilization Project data to assess all-payer (1) adult (age, ≥18 years) hospitalizations (2008-2017), (2) pediatric (age, 5-17 years) hospitalizations (2003, 2006, 2009, 2012, and 2016), and (3) adult and pediatric ED evaluations (2008-2017). International Classification of Diseases, Ninth Revision, Clinical Modification code 300.11 (conversion disorder) or 306.0 (musculoskeletal malfunction arising from mental factors) and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Clinical Modification codes for conversion disorder/functional neurological symptom disorder (F44.4 to F44.7) were used to conservatively define FNDs and to compare them with other neurological disorders that are associated with high levels of health care use. Analysis was performed between January 2019 and July 2020.

    Main Outcomes and Measures  Admission traits (eg, demographic characteristics of patients, length of stay, and discharge disposition) and hospital charges.

    Results  Compared with other neurological disorders in 2017, emergency FND evaluations of 36 359 adults (25 807 women [71.0%] and 3800 children (2733 girls [71.9%]) more frequently resulted in inpatient admissions (22 895 adult admissions [69.2% female] and 1264 pediatric admissions [73.4% ]). These FND admissions had a shorter mean (SEM) hospital length of stay (5.21 [0.15] days vs 6.03 [0.03] days, P < .001) but higher workup rates than admissions for comparable neurological diagnoses. Admissions for FNDs had low rates of inpatient physical therapy, occupational therapy, speech and language pathology, and psychiatric consultation. The total annual costs (a proxy for total costs in 2017 US dollars) were $1066 million (95% CI, $971-$1160 million) for adult FND inpatient charges in 2017 compared with $1241 million (95% CI, $1132-$1351 million) for anterior horn cell disease; $75 million (95% CI, $57-$92 million) for pediatric FND inpatient charges in 2012 compared with $86 million (95% CI, $63-$108 million) for demyelinating diseases; and $163 million (95% CI, $144-$182 million) for adult and pediatric ED visits in 2017 compared with $135 million (95% CI $111-$159 million) for refractory epilepsy. Total charges per admission for ED care of FNDs were higher than the other comparison groups in adults. Total costs and costs per admission for FNDs increased from 2008 to 2017 at a higher rate than that of other neurological disorders.

    Conclusions and Relevance  This economic evaluation found that the more than $1.2 billion and increasing annual costs for ED and inpatient care of FNDs were similar to other investigation-intensive and pharmacologically demanding neurological disorders. Unnecessary investigations and iatrogenic harm inflate costs at the expense of necessary but neglected psychiatric and rehabilitative treatments.

    Introduction

    Functional neurological disorders (FNDs) are a collection of neurological symptoms, such as gait difficulties, limb weakness, sensory and cognitive complaints, and abnormal movements that include dissociative or seizurelike episodes that are inconsistent and incongruent with the range of manifestations of other organic neurological disorders.1 As a result, patients with FNDs require an expert neurological evaluation, precluding an exclusionary approach to the diagnosis. Frequently, associated functional somatic disorders include irritable bowel syndrome, unexplained pain, atypical chest pain, multiple chemical sensitivity, and chronic fatigue syndrome.2 Functional neurological disorders are more common in women than in men and affect all age groups, including children (2%-10% of patients treated at pediatric neurology clinics).3

    Up to one-third of adult outpatients with neurological disorders are thought to have symptoms that are not at all or only somewhat explained by disease, most of whom subsequently are found to have an FND in isolation or in combination with another neurological disease.4 Functional neurological disorders represent up to 20% of visits to movement disorders clinics5 and 9% of inpatient admissions for neurological disorders.6 Psychogenic nonepileptic seizure (PNES), a subtype of FNDs, accounts for 20% of epilepsy center referrals and 30% to 50% of epilepsy monitoring unit admissions and are a common reason for emergency department (ED) evaluations.7 Appropriate treatment of FNDs starts with diagnostic debriefing, avoidance of unnecessary investigations, avoidance of pharmacotherapy (iatrogenic harm), and coordination of cognitive behavior therapy and motor retraining programs.8 Instead, patients often undergo extensive evaluations by several clinicians that result in multiple unnecessary or potentially misleading investigations, delaying treatment and increasing disability.9

    The health care costs associated with FNDs are difficult to calculate, particularly for a complex disorder with a substantial delay to diagnosis. For example, several studies have assessed the costs of conditions associated with or overlapping with FNDs, such as medically unexplained symptoms or somatization (functional somatic symptoms involving multiple organ systems).10 Somatization is estimated to cost the UK economy £18 billion (US $24 billion at the 2020 exchange rate) per year,11 of which inpatient stays account for more than £600 million (US $800 million at the 2020 exchange rate) per year.11 In the United States, Barsky et al12 provided an estimate of $256 billion annually for the total cost of somatization. However, few studies provide specific estimates and focus solely on adult FNDs; these single-center studies13,14 do not provide reliable national estimates and have no data for the pediatric population. We sought to address this gap by assessing resource use and spending for FNDs in adults and children using national data on ED and inpatient visits.

    Methods

    This economic evaluation used data from 3 Healthcare Cost and Utilization Project databases sponsored by the Agency for Healthcare Research and Quality to assess all-payer (1) adult (age, ≥18 years) hospitalizations (2008-2017), (2) pediatric (age, 5-17 years) hospitalizations (2003, 2006, 2009, 2012, and 2016), and (3) adult and pediatric ED evaluations (2008-2017). These databases are the largest publicly available source of all-payer ED and inpatient data. Limited data sets were used, with an exemption obtained from the Partners Institutional Review Board owing to the use of deidentified data such that this was not deemed to meet the definition of human participants research.

    Databases

    For the adult sample, we used 2008 to 2017 discharge data from the National Inpatient Sample (NIS, formerly Nationwide Inpatient Sample).15 The 2017 NIS includes 48 states representing more than 96% of the US population and provides a random sampling of 20% of community hospital (nonrehabilitation and non–long-term care) discharges, comprising more than 7 million (providing a national weighted estimate >35 million) inpatient discharges.15 For the pediatric sample, we used 2003, 2006, 2009, 2012, and 2016 discharge data from the Kids’ Inpatient Database (KID), which is released every 3 years (with a single year of data).16 The 2016 KID includes community-based, nonrehabilitation hospitals across 47 states, with more than 3 million (estimating >6.2 million) discharges for patients younger than 21 years.16 The KID sampling rate is 10% for healthy newborn discharges and 80% for complicated newborn and other pediatric discharges across 4200 US community hospitals.16 We examined adult and pediatric ED visits using data from 2008 to 2017 from the Nationwide Emergency Department Sample (NEDS), which sampled all discharges from 984 hospitals across 36 states and the District of Columbia and approximately 20% of discharges from hospital-owned EDs in 2017, comprising 3.5 million (estimating >144 million) ED visits.17 Among inpatients, adults in the NIS were compared with the enriched KID pediatric sample (given the higher pediatric sampling rates) owing to limited pediatric representation within the NIS. Adults included individuals 18 years or older; in children (age 5-17 years), those younger than 5 years were excluded because FNDs invariably present after age 5 years (generally at age 7 years or older) and diagnoses made before this age are unreliable.18

    There was a change in the design of the NIS databases after 2012; therefore, we used a corrective discharge trend weight for the 2008 to 2011 NIS to allow comparison of cost data across years.15 All of the databases include a sample of discharges, and individual patients cannot be linked across hospitalizations, which prevents identification of readmissions. Hence, it is not possible to ascertain whether certain discharges were associated with readmissions. In addition, given the change in coding in October 2015 from International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) to International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Clinical Modification (ICD-10-CM) and ICD-10 Procedure Coding System (ICD-10-PCS), we analyzed data up to 2014 using ICD-9-CM and then used ICD-10-CM and ICD-10-PCS for data from 2016 to 2017 for consistency. The ICD-9-CM and ICD-10-CM diagnosis and ICD-10-PCS procedure code definitions are listed in eTable 1 in the Supplement.

    Patient Population

    Emergency department visits and hospitalizations were not mutually exclusive, with ED data representing all patients whether admitted or not. The patients with FNDs had an ED or inpatient discharge diagnosis defined using ICD-9-CM code 300.11 (conversion disorder) (>96%) or 306.0 (musculoskeletal malfunction arising from mental factors) (eTable 2 in the Supplement). There are inherent difficulties in coding FNDs (often associated with diagnostic uncertainty), with frequent use of nonspecific codes that may also designate organic illnesses (including 781.0 [abnormal involuntary movements] or 780.39 [other convulsions]) or purely symptom coding. For ICD-10-CM data, we used codes F44.4 to F44.7 (specific conversion disorder/functional neurological symptom disorder codes) and excluded F44.8 and F44.9 because these codes may encompass nonspecific dissociative disorders. We used a deliberately conservative FND definition to avoid potential misclassification and to err on the side of providing a lower bound estimate. Data analysis was performed between January 2019 and July 2020. Main outcomes and measures were admission traits (eg, demographic characteristics of patients, length of stay, and discharge disposition) and hospital charges.

    Comparisons

    Health care resource use and costs associated with FNDs were compared with those of other neurological disorders, particularly neurological disorders with high levels of health care use. We assessed differences in health care use by examining the most recent data before the October 2015 coding change (2014 NIS, 2012 KID, and 2014 NEDS) and the latest ICD-10-CM and ICD-10-PCS data (2017 NIS, 2016 KID, and 2017 NEDS).

    A generic neurological comparison group included common diagnoses for neurological admission, excluding ICD-9-CM and ICD-10-CM codes for typical neurosurgery indications (primary or secondary central nervous system tumors and intracranial hemorrhage), which incur higher costs associated with required neurosurgical procedures and intensive care unit stays. Functional neurological disorders were compared with non-FND neurological disorders and with the following specific neurological disorders with high levels of health care use: neuroinflammatory or demyelinating disease (including multiple sclerosis [MS]), refractory epilepsy (excluding admissions with status epilepticus), and anterior horn cell disease (AHCD) (including amyotrophic lateral sclerosis and spinal muscular atrophy). Functional neurological disorders can coexist with organic diagnoses, but we found that the overlap with these diagnoses was 1% or less (eTable 2 in the Supplement). When forming these comparison groups, all discharge diagnosis codes were assessed, not just the primary diagnosis, to estimate total costs of ED and inpatient care stemming from any comorbidities within each group. Rates of psychiatric diagnoses and functional somatic disorders were compared between the FND and non-FND groups.

    Variables Assessed and Data Reporting

    To examine differences between groups for ED and inpatient visits, we assessed patient demographic characteristics, including age, sex, race/ethnicity, and zip code income quartiles (a proxy for socioeconomic status). To investigate health care use and other factors altering inpatient costs, we assessed admission type (elective vs nonelective or emergent), length of stay, procedures performed (neuroimaging, lumbar puncture, and rehabilitation or psychiatric treatment), discharge disposition, and primary insurance payer.

    Health care costs were calculated using the weighted estimate total charges for ED or hospitalization visits as a proxy, which included billed services but not professional (physician) fees. Cost-to-charge ratios were available for the NIS and KID, allowing estimation of the expense of the ED or admission presentation to the hospital but not actual societal health care spending by payers or patients, which varies across insurance segments and plans. Because this information was not available for NEDS, we assessed total charges in the analyses for consistency. Adult inpatient costs were assessed using the 2008 to 2017 NIS, pediatric admissions were assessed using KID data every 3 years from 2003 to 2016, and adult and pediatric ED visits were assessed using the 2008 to 2017 NEDS, excluding 2015 data given the midyear coding change. Costs were adjusted for inflation using the US general Consumer Price Index to 2017 US dollars. We did not adjust for age or sex because the goal was to estimate overall costs across the age range.

    We do not present cell sizes of 10 or less; hence, some of the data are given without the absolute values or percentages. If there were 0 patients, this number is reported. There were generally low levels of missing data (eTable 3 in the Supplement) other than some ED charge data, and analysis was performed on nonmissing data.

    Statistical Analysis

    Analyses were conducted using SAS, version 9.4 (SAS Institute Inc) with SUDAAN and complex survey weights for national projections. Additional details on the sampling frame and weighting methods for each database are available from the Healthcare Cost and Utilization Project.13-15 We performed descriptive statistics, t tests for continuous variables, and χ2 tests for categorical or ordinal variables comparing FNDs with other organic neurological disorders. Two-sided P < .05 indicated statistical significance.

    Results
    Demographic Comparisons of Patients With FNDs vs Other Neurological Disorders

    The ED sample included 36 359 adult ED visits (25 807 women [71.0%]) and 3800 pediatric visits (2733 girls [71.9%]) for FNDs among 133 557 275 adult and pediatric combined discharges (2017 NEDS) (Table 1). The inpatient study sample included 22 895 adult admissions (15 830 [69.2%] female) for FNDs among 30 420 907 total hospitalizations (2017 NIS) and 1264 pediatric admissions (928 [73.4%] female) for FNDs among 965 132 total hospitalizations (2016 KID) (Table 2).

    Functional neurological disorders accounted for a small proportion of overall neurology admissions at 0.5% for adult and 1% for pediatric neurology admissions. Adult and pediatric inpatient admissions for FNDs were more common in female patients than in male patients compared with admissions for other neurological disorders (69.2% vs 55.0% for adults and 73.4% vs 50.1% for children, P < .001 for both) (Table 2). There were fewer admissions for FNDs in patients of White race compared with admissions for other neurological disorders (66.2% vs 70.5%, P < .001), but the proportions of admissions for FNDs vs other neurological disorders were similar in White children (58.4% vs 53.7%, P = .07). Most admissions for FNDs were nonelective or emergent (urgent, trauma, or emergency). In adults, the proportion of emergent admissions for FNDs (85.1%) vs other neurological disorders (85.0%) was similar. However, in children, admissions for FNDs were more frequently nonelective or emergent compared with those for other neurological disorders (87.0% vs 71.6%, P < .001). In adults, the proportion of admissions for FND vs that of admissions for other neurological disorders was higher in patients living in zip codes with the lowest quartile of median household income (34.4% vs 30.8%, P < .001), but this finding was not observed in children living in those zip codes.

    Functional neurological disorders were infrequently comorbid with a diagnosis of refractory epilepsy (0.9% of FND in adults and 1.3% of FND in children), but the concurrent presence of any epilepsy diagnosis was more common, being present in 14.2% of adult and 11.0% of pediatric FND admissions (eTable 2 in the Supplement). In adults and children, there were higher rates of psychiatric diagnoses with FND admissions compared with admissions for other neurological disorders, with mood and anxiety disorders more common in adults and children (Table 2). The rate of functional somatic disorders was higher in admissions for FNDs than in admissions for other neurological disorders (10.5% vs 4.1% for adults and 6.7% vs 1.2% for children, P < .001 for both). Adult and pediatric patterns were reasonably similar in the ED sample, with rates of 7.1% in FNDs vs 3.3% in other neurological disorders in adults and 2.5% vs 0.5%, respectively, in children.

    In the NIS, Medicaid was often the primary insurance payer in inpatient discharges for FNDs vs other neurological disorders (28.3% vs 14.0%), and FND discharges were less likely to be covered by Medicare (Table 2). In KID, primary insurance payer was reasonably similar but still statistically significantly different (P < .001), with 48.6% of FNDs vs 50.9% of other neurological disorders covered by Medicaid and 44.9% vs 42.3%, respectively, covered by private insurance. Adults with ED visits for FNDs were more likely than those with visits for other neurological disorders to have Medicaid insurance (33.3% vs 19.8%, P < .001), and children were more likely to have private insurance (39.9% vs 37.0%, respectively; P = .006) (Table 1).

    Inpatient Health Care Use for Adults and Children With FNDs

    Differences in resource use in adult and pediatric inpatient admissions are listed in Table 2. Differences in discharge disposition for adult and pediatric ED visits are shown in Table 1. Results using ICD-9-CM data from the 2014 NIS and the 2012 KID were similar to the findings presented herein (eTable 4 in the Supplement).

    The mean (SEM) length of stay was shorter in FNDs than in other neurological disorders among both adults (5.21 [0.15] vs 6.03 [0.03] days, P < .001) and children (4.01 [0.31] vs 5.84 [0.10], P < .001) (Table 2). Both adults and children with FNDs had fewer inpatient rehabilitation consultations (physical therapy, occupational therapy, or speech and language pathology) compared with patients with other neurological disorders. However, there was a considerable change between the 2014 NIS and the 2012 KID ICD-9-CM data (in which rates in adults with FNDs were 3.2% vs 5.1% in those with other neurological disorders, and rates in children with FNDs were 4.2% vs 5.7%; P < .001 for both as summarized in eTable 4 in the Supplement) and the 2017 NIS and the 2016 KID ICD-10-CM and ICD-10-PCS data (in which the rates were lower at 0.6% vs 1.4% in adults [P < .001], and the rates in children were 0.7% vs 0.8% [P = .51]). This change may be associated with ICD-9-CM to ICD-10-CM and ICD-10-PCS coding issues, with rehabilitation services being improperly coded. Rates of psychiatric management were low in both groups, with rates of 0.7% vs 0.4% in adults with FNDs vs other neurological disorders (P = .02) and rates of 0.9% vs 0.4%, respectively, in children (P = .23) using ICD-10-CM and ICD-10-PCS data.

    Regarding inpatient workup, patients with FNDs vs patients with other neurological disorders had more frequent electroencephalographic monitoring, including both standard electroencephalogram and long-term monitoring and continuous video monitoring (11.8% vs 1.7%, respectively, for adults and 20.9% vs 11.6%, respectively, for children; P < .001 for both), likely associated with evaluation of PNES or other paroxysmal FNDs (Table 2). When assessing rates for common imaging using the ICD-9-CM data, patients with FNDs were also more likely to have neuroimaging (computed tomography or magnetic resonance imaging of the head or spine) than patients with other neurological disorders (5.0% vs 3.4% for adults and 8.5% vs 5.2% for children; P < .001 for both as summarized in eTable 4 in the Supplement). In contrast, when assessing rates using the ICD-10-CM and ICD-10-PCS data, patients with FNDs had similar rates of neuroimaging compared with patients with other neurological disorders in 2017 (1.1% vs 1.1% in adults, P = .75 and 3.1% vs 3.6% in children, P = .47). Although this finding could reflect a change in practice, the substantial difference in rates may be associated with coding transition issues given the large change in both the FND group and the neurological comparator group. Lumbar punctures were more common in FND admissions than in admissions for other neurological disorders, but there was again a similar decrease in rates from the ICD-9-CM data (with rates in adult FNDs of 4.6% vs 2.2%; P < .001 and rates in pediatric FNDs of 7.9% vs 6.5%; P = .04 as summarized in eTable 4 in the Supplement) and in the ICD-10-CM data (with rates in adult FNDs of 2.8% vs 2.2%; P = .03 and rates in pediatric FNDs of 4.6% vs 6.0%; P = .08 [2014 data in eTable 4 in the Supplement]). Discharge disposition was not subject to coding differences; using ICD-10-CM data in inpatient admissions, patients with FNDs were generally discharged home, with or without home health care, which was more common in adults (75.8% [17 335of 22 880] vs 62.3% [3 061 230 of 4 915 724]; P < .001) and less common in children (91.0% [1150 of 1264] vs 93.4% [115 719 of 123 915]; P = .03).

    In adults, 0.9% of patients with FNDs vs 3.1% of patients with other neurological disorders (P < .001) died of any cause during their hospitalization, and death was very rare in pediatric FNDs (Table 2). In ED visits, adult patients with FNDs had similar rates of admission vs patients with other neurological disorders (45.3% vs 42.6%, P = .07), and children with FNDs were more commonly admitted than children with other neurological disorders (25.1% vs 15.4%, P < .001) (Table 1). Death during ED visits was extremely rare in adult FND, and there were no deaths noted in pediatric FNDs.

    ED and Inpatient Cost Estimates for FNDs

    The annual charge data comparing FNDs with other neurological disorders for adult and pediatric admissions (Table 3 and Table 4) and for adult and pediatric ED presentations (Table 5) show steep yearly increases. Few cost data were missing in the inpatient databases, but in some cases, more than 30% of data were missing in NEDS (eTable 3 in the Supplement). There were large decreases in FND numbers in 2016 across all databases, which may be owing to unfamiliarity with the new ICD-10-CM coding, but by 2017 numbers increased to similar pre-2015 levels; hence, 2012 (and not 2016) data were used to estimate pediatric inpatient FND costs.

    Total adult inpatient charges (2017 NIS) showed that the annual total charges (a proxy for total costs in 2017 US dollars) were $1066 million (95% CI, $971-$1160 million) for adult FND inpatient charges in 2017, comparable to those for AHCD ($1241 million; 95% CI, $1132-$1351 million) (Table 3). From 2008 to 2017, there was a 162% increase in total annual inpatient FND charges vs 87% in charges for other neurological disorders, 96% in MS, and 81% in AHCD. The mean charges per patient for FNDs increased by 88% vs 61% or less for other neurological disorders.

    Total pediatric inpatient charges (2012 KID) showed that FNDs ($75 million; 95% CI, $57-$92 million) were similar to demyelinating diseases ($86 million; 95% CI, $63-$108 million) and AHCD ($96 million; 95% CI, $72-$120 million) (Table 4). From 2006 to 2012, there was a 113% increase in total FND charges compared with 76% in MS, 72% in refractory epilepsy, and 59% in AHCD. The mean charges per patient for FNDs increased by 41%, similar to refractory epilepsy, vs 34% in other neurological disorders.

    Overall, charges for adult and pediatric ED visits for FNDs (2017 NEDS) were $163 million (95% CI, $144-$182 million), comparable to refractory epilepsy ($135 million; 95% CI, $111-$159 million). The ED charges in adults with FNDs ($150 million; 95% CI, $135-$165 million) were higher than in those with refractory epilepsy and AHCD; in children with FNDs, the ED charges ($13 million; 95% CI, $9-$17 million) were substantially higher in those with FNDs than in those with MS and AHCD, likely owing to larger numbers of ED visits for FNDs (Table 5). The mean ED visit charges for FNDs per patient were higher than those for other neurological disorders in adults. From 2008 to 2017, there was a 256% increase in charges in adults and a 249% increase in children, higher than for all comparator groups other than refractory epilepsy. The mean charges per patient also increased (92% in adults and 97% in pediatric FND), slightly lower than other groups in adults and higher than other general neurological disorders in children. Using the most recent annual cost estimates for adult and pediatric ED presentations and hospitalizations ($1216 million in adults using 2017 data and a likely low estimate of $88 million in children based on 2017 ED and older 2012 inpatient charge data), the annual expenditures attributable to FNDs yield a conservative estimate of more than $1.2 billion and are increasing (eTable 5 in the Supplement).

    Discussion

    Overall, our estimates suggest that ED and inpatient care of both adults and children with FNDs incurred total charges to the US health care system of more than $1.2 billion annually ($1.2 billion in adult FNDs and $88 million in pediatric FNDs), similar to those of neurological conditions that require high health care resource use. Adult inpatient FND costs are comparable to those of AHCD, with pediatric inpatient FND costs similar to those of demyelinating or neuroinflammatory diseases like MS. Total ED charges for FNDs were higher than those of several of the comparator groups, and total charges per patient for FNDs were often higher than those of other neurological disorders. Furthermore, these cost estimates likely represent an underestimate because of undercoding or inaccurate coding and our conservative diagnostic definition of FNDs; however, yearly total costs are increasing faster than those of other neurological disorders.

    Besides the anticipated greater prevalence of FNDs among women,14,19 adult patients were more frequently racial/ethnic minorities compared with other neurological disorders. This finding contrasts with other published data,20 which may be associated with the regional differences in racial/ethnic composition of the investigating centers or could reflect a higher likelihood of readmission among racial/ethnic minority patients. Adult patients with FNDs also lived in lower-income neighborhoods compared with those with other neurological disorders. In comparison, children with FNDs had a similar race/ethnicity and socioeconomic status as their non-FND neurological counterparts.

    Admissions were more often emergent given the general acuity of worsening or newly presenting FND symptoms. Inpatient admissions for FNDs had shorter hospital stays but often higher workup rates than those of other neurological disorders. The ED21 and inpatient22 management of FNDs require multidisciplinary management, including neurological assessment, trained physical therapy, occupational therapy, and speech and language pathology management and, when necessary, psychiatric care. However, data in the present study show a low level of these needed inpatient rehabilitation services and psychiatric management for patients with FNDs despite high levels of psychiatric diagnoses, particularly in children. This observation suggests that delivery of the optimum FND care may not be occurring appropriately in the inpatient setting.21,22 Of concern, approximately 1% of adult patients with FNDs (vs 3% with other neurological disorders) died during their hospitalization. Although the cause of death cannot be ascertained from the data, this finding is surprising and requires further investigation to assess if it may be associated with other medical comorbidities or iatrogenic harm.23

    Strengths and Limitations

    Strengths of this analysis include capturing ED and inpatient costs, which constitute the largest single costs among health care expenditure.24 To do so, we used 3 nationally representative databases assessing health care spending for both adults and children given a dearth of published US cost estimates in FNDs.10,12 The large sample size with weighted estimates to the US population increases the generalizability of our conclusions and the validity of epidemiological data.

    There are limitations to this analysis. The databases assess a sample of ED and inpatient discharges; therefore, individuals cannot be linked across hospitalizations or ED visits, and some patients may have been counted multiple times. We also present only hospital charge data, which likely overestimates actual spending. Available cost-to-charge ratios give an estimation of hospital expenses for the ED visit or admission but do not provide information on reimbursement amounts or the cost borne by the patient; hence, we used charges as a proxy for ED and inpatient costs and include comparison groups for context. On a per-hospital level, the average cost-to-charge ratio for the 2017 NIS was 0.47 (range, 0.06-1.87).25

    We chose to assess all ED visits or hospitalizations for which a discharge diagnosis of an FND was included, regardless of the main reason for the hospital visit, to give a fuller accounting of total costs associated with FND diagnoses. A patient with an FND may be admitted for a variety of reasons (eg, headaches and other forms of pain, involuntary movements, strokelike symptoms, or seizurelike episodes, among others); hence, only including FND as the primary diagnosis would have undercounted the true cost of this population, particularly because ED or hospital physicians may often not code FND as a primary diagnosis if the main complaint was any of the many associated symptoms with FND. However, using this approach, we could also include patients with FNDs who present with an unrelated problem. Nevertheless, we found that a small proportion of neurology admissions were for FNDs (0.5% for adults and 1.0% for children), which could reflect undercoding of FNDs overall (particularly given the sharp contrast to previous estimates of FNDs that accounted for 9% of inpatient admissions to the neurology service6).

    It is also notable that our diagnostic definitions substantially underestimate the true FND population presenting to an ED or admitted to the hospital. There is a lack of unambiguous diagnostic codes in cases of suspected FNDs, and uncertainty on the part of the treating physician often leads to the use of non–FND-related or symptomatic codes alone (eg, tremor, weakness, or loss of consciousness). In addition, complexities in reimbursement may sway clinicians toward using neurological as opposed to psychiatric diagnosis codes, potentially resulting in a further underestimate of FND prevalence based on coding data. There was also a small proportion of patients with FNDs who had comorbid refractory epilepsy; however, in this cohort, generic FND was comorbid with any kind of epilepsy in 14.2% of adults and 11.0% of children, suggesting skewing toward PNES because about 20% of patients with the PNES subtype of FNDs also have comorbid epilepsy.26 Finally, coding frequency differences between ICD-9-CM and ICD-10-CM and ICD-10-PCS data may have altered the comparability of data before vs after 2015.

    In general, ED and inpatient care is reserved for the most serious and acute cases, suggesting that we may have captured data on the most severe FND population because many patients with FNDs do not require hospital care. This analysis could also reflect disparities in access to care. In certain areas, patients with FNDs might have no option but to present to the ED given inadequate outpatient neurological resources. This dilemma should be explored in further studies. Future studies would benefit from a more comprehensive assessment of health care spending, including outpatient costs, such as using Medicare, Medicaid, and other insurance claims data and readmission data for patients with FNDs with recurrent hospital presentations, although none of these data sets are sufficient alone to estimate population-level resource use or spending for FNDs.

    Conclusions

    Emergency department and inpatient health care use by patients with FNDs is high and increasing compared with other investigation-intensive and pharmacologically demanding neurological disorders. The paradoxically low rates of inpatient rehabilitation and psychiatric management suggest that the high cost comes with low returns for this patient population. Although an underestimate given the exclusion of outpatient costs, the annual charges to the health care system are considerable and increasing at a higher rate than those of other neurological disorders. These data warrant medical education and a health care policy overhaul to improve the service to this vulnerable population while preventing investigation-intensive hospital admissions. Despite similar or higher expenditures than those of other neurological disorders, funding for FND research and appropriate clinical management remains scant. Therefore, investment into FND research and health care delivery stand to yield much larger returns given the cost savings arising from early diagnosis and rational deployment of cognitive behavior therapy and physical therapy while preventing unnecessary investigations and iatrogenic harm.

    Back to top
    Article Information

    Accepted for Publication: August 15, 2020.

    Corresponding Author: Christopher D. Stephen, MB ChB, MRCP(UK), MS, Department of Neurology, Massachusetts General Hospital, 100 Cambridge St, Ste 2000, Boston, MA 02114 (cstephen@mgh.harvard.edu).

    Published Online: October 26, 2020. doi:10.1001/jamaneurol.2020.3753

    Author Contributions: Dr Stephen had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Stephen, Lungu, Espay.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Stephen.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Stephen, Fung.

    Administrative, technical, or material support: Lungu.

    Supervision: Lungu, Espay.

    Conflict of Interest Disclosures: Dr Stephen reported providing scientific advisory for SwanBio Therapeutics and Xenon Pharmaceuticals, receiving research funding from Sanofi Genzyme for a study of videooculography in late-onset GM2 gangliosidosis, and receiving financial support from Biogen, Biohaven, and Sanofi Genzyme for the conduct of clinical trials. Dr Espay reported receiving grant support from The Michael J. Fox Foundation and the National Institutes of Health; receiving personal compensation as a consultant or scientific advisory board member for AbbVie, Acadia, Acorda, Adamas, Amneal, InTrance, Lundbeck, Neurocrine, NeuroDerm, Sunovion, and US WorldMeds; receiving publishing royalties from Cambridge University Press, Lippincott Williams & Wilkins, and Springer; and receiving honoraria from Acadia, Sunovion, and US WorldMeds. No other disclosures were reported.

    References
    1.
    Espay  AJ, Aybek  S, Carson  A,  et al.  Current concepts in diagnosis and treatment of functional neurological disorders.   JAMA Neurol. 2018;75(9):1132-1141. doi:10.1001/jamaneurol.2018.1264 PubMedGoogle Scholar
    2.
    Nimnuan  C, Hotopf  M, Wessely  S.  Medically unexplained symptoms: an epidemiological study in seven specialities.   J Psychosom Res. 2001;51(1):361-367. doi:10.1016/S0022-3999(01)00223-9 PubMedGoogle Scholar
    3.
    Leary  PM.  Conversion disorder in childhood: diagnosed too late, investigated too much?   J R Soc Med. 2003;96(9):436-438. doi:10.1258/jrsm.96.9.436PubMedGoogle Scholar
    4.
    Stone  J, Carson  A, Duncan  R,  et al.  Symptoms “unexplained by organic disease” in 1144 new neurology out-patients: how often does the diagnosis change at follow-up?   Brain. 2009;132(pt 10):2878-2888. doi:10.1093/brain/awp220 PubMedGoogle Scholar
    5.
    Hallett  M.  Psychogenic movement disorders: a crisis for neurology.   Curr Neurol Neurosci Rep. 2006;6(4):269-271. doi:10.1007/s11910-006-0015-x PubMedGoogle Scholar
    6.
    Lempert  T, Dieterich  M, Huppert  D, Brandt  T.  Psychogenic disorders in neurology: frequency and clinical spectrum.   Acta Neurol Scand. 1990;82(5):335-340. doi:10.1111/j.1600-0404.1990.tb03312.x PubMedGoogle Scholar
    7.
    Benbadis  SR, O’Neill  E, Tatum  WO, Heriaud  L.  Outcome of prolonged video-EEG monitoring at a typical referral epilepsy center.   Epilepsia. 2004;45(9):1150-1153. doi:10.1111/j.0013-9580.2004.14504.x PubMedGoogle Scholar
    8.
    Stone  J, Carson  A.  Functional neurologic disorders.   Continuum (Minneap Minn). 2015;21(3 Behavioral Neurology and Neuropsychiatry):818-837. doi:10.1212/01.CON.0000466669.02477.45PubMedGoogle Scholar
    9.
    Espay  AJ, Goldenhar  LM, Voon  V, Schrag  A, Burton  N, Lang  AE.  Opinions and clinical practices related to diagnosing and managing patients with psychogenic movement disorders: an international survey of Movement Disorder Society members.   Mov Disord. 2009;24(9):1366-1374. doi:10.1002/mds.22618 PubMedGoogle Scholar
    10.
    Konnopka  A, Schaefert  R, Heinrich  S,  et al.  Economics of medically unexplained symptoms: a systematic review of the literature.   Psychother Psychosom. 2012;81(5):265-275. doi:10.1159/000337349 PubMedGoogle Scholar
    11.
    Bermingham  SL, Cohen  A, Hague  J, Parsonage  M.  The cost of somatisation among the working-age population in England for the year 2008-2009.   Ment Health Fam Med. 2010;7(2):71-84.PubMedGoogle Scholar
    12.
    Barsky  AJ, Orav  EJ, Bates  DW.  Somatization increases medical utilization and costs independent of psychiatric and medical comorbidity.   Arch Gen Psychiatry. 2005;62(8):903-910. doi:10.1001/archpsyc.62.8.903 PubMedGoogle Scholar
    13.
    Magee  JA, Burke  T, Delanty  N, Pender  N, Fortune  GM.  The economic cost of nonepileptic attack disorder in Ireland.   Epilepsy Behav. 2014;33:45-48. doi:10.1016/j.yebeh.2014.02.010 PubMedGoogle Scholar
    14.
    Carson  A, Lehn  A.  Epidemiology.   Handb Clin Neurol. 2016;139:47-60. doi:10.1016/B978-0-12-801772-2.00005-9 PubMedGoogle Scholar
    15.
    Healthcare Cost and Utilization Project. Overview of the National (Nationwide) Inpatient Sample (NIS). Accessed December 17, 2019. https://www.hcup-us.ahrq.gov/nisoverview.jsp
    16.
    Healthcare Cost and Utilization Project. Overview of the Kids’ Inpatient Database (KID). Accessed September 18, 2019. https://www.hcup-us.ahrq.gov/kidoverview.jsp
    17.
    Healthcare Cost and Utilization Project. Overview of the Nationwide Emergency Department Sample (NEDS). Accessed December 14, 2017. https://www.hcup-us.ahrq.gov/nedsoverview.jsp
    18.
    Grattan-Smith  PJ, Dale  RC.  Pediatric functional neurologic symptoms.   Handb Clin Neurol. 2016;139:489-498. doi:10.1016/B978-0-12-801772-2.00040-0 PubMedGoogle Scholar
    19.
    Matin  N, Young  SS, Williams  B,  et al.  Neuropsychiatric associations with gender, illness duration, work disability, and motor subtype in a U.S. functional neurological disorders clinic population.   J Neuropsychiatry Clin Neurosci. 2017;29(4):375-382. doi:10.1176/appi.neuropsych.16110302 PubMedGoogle Scholar
    20.
    Cubo  E, Hinson  VK, Goetz  CG,  et al.  Transcultural comparison of psychogenic movement disorders.   Mov Disord. 2005;20(10):1343-1345. doi:10.1002/mds.20561 PubMedGoogle Scholar
    21.
    Anderson  JR, Nakhate  V, Stephen  CD, Perez  DL.  Functional (psychogenic) neurological disorders: assessment and acute management in the emergency department.   Semin Neurol. 2019;39(1):102-114. doi:10.1055/s-0038-1676844 PubMedGoogle Scholar
    22.
    McKee  K, Glass  S, Adams  C,  et al.  The inpatient assessment and management of motor functional neurological disorders: an interdisciplinary perspective.   Psychosomatics. 2018;59(4):358-368. doi:10.1016/j.psym.2017.12.006 PubMedGoogle Scholar
    23.
    Duncan  R, Oto  M, Wainman-Lefley  J.  Mortality in a cohort of patients with psychogenic non-epileptic seizures.   J Neurol Neurosurg Psychiatry. 2012;83(7):761-762. doi:10.1136/jnnp-2012-302900 PubMedGoogle Scholar
    24.
    Galarraga  JE, Pines  JM.  Costs of ED episodes of care in the United States.   Am J Emerg Med. 2016;34(3):357-365. doi:10.1016/j.ajem.2015.06.001PubMedGoogle Scholar
    25.
    Bai  G, Anderson  GF.  Extreme markup: the fifty US hospitals with the highest charge-to-cost ratios.   Health Aff (Millwood). 2015;34(6):922-928. doi:10.1377/hlthaff.2014.1414 PubMedGoogle Scholar
    26.
    Kutlubaev  MA, Xu  Y, Hackett  ML, Stone  J.  Dual diagnosis of epilepsy and psychogenic nonepileptic seizures: systematic review and meta-analysis of frequency, correlates, and outcomes.   Epilepsy Behav. 2018;89:70-78. doi:10.1016/j.yebeh.2018.10.010 PubMedGoogle Scholar
    ×