Development and Validation of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) | Amyotrophic Lateral Sclerosis | JAMA Neurology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Category Probability Curves
Category Probability Curves

A, Using a knife and fork was included in the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) with appropriately ordered category probability curves. Category 2 is the most probable answer choice for patients with the least amount of overall disability. As overall disability increases, category 1 sequentially becomes the most probable answer choice for patients with moderate overall disability level. As overall disability increases to the greatest level, 0 sequentially becomes the most probable answer choice. B, Driving a car was removed from ROADS owing to disordered thresholds. Category 2 is the most probable answer choice for patients with the least amount of overall disability, but category 0 appears out of order as the most probable answer choice for patients with a higher level of overall disability, and category 1 is never the most probable answer choice for any overall disability level. C, An example of disordered thresholds from the revised Amyotrophic Lateral Sclerosis Functional Rating Scale. Item responses 1, 2, and 3 are never the most probable answer choice for any overall functional rating ability.

Figure 2.  Threshold Map and Rasch Measures
Threshold Map and Rasch Measures

The Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale threshold map shows the difficulty order and item targeting for each question, with the most difficult task to perform (get heavy objects off a high shelf) appearing on the top and the easiest task to perform (nod yes or no) appearing on the bottom. The x-axis represents the logit measure for overall disability level. The placement of the 0, 1, and 2 responses for each item demonstrate the most probable answer at any particular overall disability level, with the open circles representing the transition point where a 0 and 1 response or a 1 and 2 response are equally probable. The threshold map shows a gradual shift from mostly 2 answers on the right for patients with the least disabilities to more 1 and 0 answers as overall disability level declines. Each item was scored as follows: 0, unable to perform; 1, abnormal: able to perform but with difficulty; or 2, normal: able to perform without difficulty.

Figure 3.  Improved Item Targeting on the ROADS Compared With the ALSFRS-R
Improved Item Targeting on the ROADS Compared With the ALSFRS-R

The Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) targets a broader range of ability levels compared with the revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R). Each questionnaire item is placed in difficulty order with the most difficult items on top and the easiest items at the bottom, and the measure column indicates the logit measure where the middle item response choice is the most probable answer. Item difficulties spanned 5.9 logits for the ROADS (range, −2.97 to 2.93 logits) but only 2.27 logits (range, −1.32 to 0.95 logits) for the ALSFRS-R.

Table 1.  Characteristics of Questionnaire Study Participants (N = 243)
Characteristics of Questionnaire Study Participants (N = 243)
Table 2.  Linearly Weighted Normed Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) Scores
Linearly Weighted Normed Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) Scores
1.
Mora  JS, Genge  A, Chio  A,  et al; AB10015 STUDY GROUP.  Masitinib as an add-on therapy to riluzole in patients with amyotrophic lateral sclerosis: a randomized clinical trial.   Amyotroph Lateral Scler Frontotemporal Degener. 2019;1-10. doi:10.1080/21678421.2019.1632346PubMedGoogle Scholar
2.
Statland  JM, Moore  D, Wang  Y,  et al; Rasagiline Investigators of the Muscle Study Group and Western ALS Consortium.  Rasagiline for amyotrophic lateral sclerosis: a randomized, controlled trial.   Muscle Nerve. 2019;59(2):201-207. doi:10.1002/mus.26335PubMedGoogle ScholarCrossref
3.
Writing  G, Edaravone  ALSSG; Writing Group; Edaravone (MCI-186) ALS 19 Study Group.  Safety and efficacy of edaravone in well defined patients with amyotrophic lateral sclerosis: a randomised, double-blind, placebo-controlled trial.   Lancet Neurol. 2017;16(7):505-512. doi:10.1016/S1474-4422(17)30115-1PubMedGoogle ScholarCrossref
4.
Shefner  JM, Wolff  AA, Meng  L,  et al; BENEFIT-ALS Study Group.  A randomized, placebo-controlled, double-blind phase IIb trial evaluating the safety and efficacy of tirasemtiv in patients with amyotrophic lateral sclerosis.   Amyotroph Lateral Scler Frontotemporal Degener. 2016;17(5-6):426-435. doi:10.3109/21678421.2016.1148169PubMedGoogle ScholarCrossref
5.
Shibuya  K, Misawa  S, Kimura  H,  et al.  A single blind randomized controlled clinical trial of mexiletine in amyotrophic lateral sclerosis: efficacy and safety of sodium channel blocker phase II trial.   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(5-6):353-358. doi:10.3109/21678421.2015.1038277PubMedGoogle ScholarCrossref
6.
Fournier  CN, Schoenfeld  D, Berry  JD,  et al.  An open label study of a novel immunosuppression intervention for the treatment of amyotrophic lateral sclerosis.   Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(3-4):242-249. doi:10.1080/21678421.2017.1421666PubMedGoogle ScholarCrossref
7.
Cedarbaum  JM, Stambler  N, Malta  E,  et al; BDNF ALS Study Group (Phase III).  The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function.   J Neurol Sci. 1999;169(1-2):13-21. doi:10.1016/S0022-510X(99)00210-5PubMedGoogle ScholarCrossref
8.
Rutkove  SB, Caress  JB, Cartwright  MS,  et al.  Electrical impedance myography as a biomarker to assess ALS progression.   Amyotroph Lateral Scler. 2012;13(5):439-445. doi:10.3109/17482968.2012.688837PubMedGoogle ScholarCrossref
9.
Kollewe  K, Mauss  U, Krampfl  K, Petri  S, Dengler  R, Mohammadi  B.  ALSFRS-R score and its ratio: a useful predictor for ALS-progression.   J Neurol Sci. 2008;275(1-2):69-73. doi:10.1016/j.jns.2008.07.016PubMedGoogle ScholarCrossref
10.
Shefner  JM, Liu  D, Leitner  ML,  et al.  Quantitative strength testing in ALS clinical trials.   Neurology. 2016;87(6):617-624. doi:10.1212/WNL.0000000000002941PubMedGoogle Scholar
11.
Kaufmann  P, Levy  G, Montes  J,  et al; QALS study group.  Excellent inter-rater, intra-rater, and telephone-administered reliability of the ALSFRS-R in a multicenter clinical trial.   Amyotroph Lateral Scler. 2007;8(1):42-46. doi:10.1080/17482960600888156PubMedGoogle ScholarCrossref
12.
Maier  A, Holm  T, Wicks  P,  et al.  Online assessment of ALS functional rating scale compares well to in-clinic evaluation: a prospective trial.   Amyotroph Lateral Scler. 2012;13(2):210-216. doi:10.3109/17482968.2011.633268PubMedGoogle ScholarCrossref
13.
Atassi  N, Berry  J, Shui  A,  et al.  The PRO-ACT database: design, initial analyses, and predictive features.   Neurology. 2014;83(19):1719-1725. doi:10.1212/WNL.0000000000000951PubMedGoogle ScholarCrossref
14.
Bedlack  RS, Vaughan  T, Wicks  P,  et al.  How common are ALS plateaus and reversals?   Neurology. 2016;86(9):808-812. doi:10.1212/WNL.0000000000002251PubMedGoogle ScholarCrossref
15.
Jackson  CE, Gronseth  G, Rosenfeld  J,  et al; Muscle Study Group.  Randomized double-blind study of botulinum toxin type B for sialorrhea in ALS patients.   Muscle Nerve. 2009;39(2):137-143. doi:10.1002/mus.21213PubMedGoogle ScholarCrossref
16.
Shefner  JM, Rutkove  SB, Caress  JB,  et al.  Reducing sample size requirements for future ALS clinical trials with a dedicated electrical impedance myography system.   Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(7-8):555-561. doi:10.1080/21678421.2018.1510008PubMedGoogle ScholarCrossref
17.
Andres  PL, Allred  MP, Stephens  HE,  et al.  Fixed dynamometry is more sensitive than vital capacity or ALS rating scale.   Muscle Nerve. 2017;56(4):710-715. doi:10.1002/mus.25586PubMedGoogle ScholarCrossref
18.
Vanhoutte  EK, Hermans  MC, Faber  CG, Gorson  KC, Merkies  IS, Thonnard  JL; PeriNomS Study Group.  Rasch-ionale for neurologists.   J Peripher Nerv Syst. 2015;20(3):260-268. doi:10.1111/jns.12122PubMedGoogle ScholarCrossref
19.
Tennant  A, Conaghan  PG.  The Rasch measurement model in rheumatology: what is it and why use it? when should it be applied, and what should one look for in a Rasch paper?   Arthritis Rheum. 2007;57(8):1358-1362. doi:10.1002/art.23108PubMedGoogle ScholarCrossref
20.
van Nes  SI, Vanhoutte  EK, van Doorn  PA,  et al.  Rasch-built Overall Disability Scale (R-ODS) for immune-mediated peripheral neuropathies.   Neurology. 2011;76(4):337-345. doi:10.1212/WNL.0b013e318208824bPubMedGoogle ScholarCrossref
21.
Vanhoutte  EK, Faber  CG, van Nes  SI,  et al; PeriNomS Study Group.  Rasch-built overall disability scale for multifocal motor neuropathy (MMN-RODS(c)).   J Peripher Nerv Syst. 2015;20(3):296-305. doi:10.1111/jns.12141PubMedGoogle ScholarCrossref
22.
Pinto  S, de Carvalho  M.  The R of ALSFRS-R: does it really mirror functional respiratory involvement in amyotrophic lateral sclerosis?   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(1-2):120-123. doi:10.3109/21678421.2014.952641PubMedGoogle ScholarCrossref
23.
Franchignoni  F, Mandrioli  J, Giordano  A, Ferro  S; ERRALS Group.  A further Rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements.   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(5-6):331-337. doi:10.3109/21678421.2015.1026829PubMedGoogle ScholarCrossref
24.
Franchignoni  F, Mora  G, Giordano  A, Volanti  P, Chiò  A.  Evidence of multidimensionality in the ALSFRS-R Scale: a critical appraisal on its measurement properties using Rasch analysis.   J Neurol Neurosurg Psychiatry. 2013;84(12):1340-1345. doi:10.1136/jnnp-2012-304701PubMedGoogle ScholarCrossref
25.
Vanhoutte  EK, Draak  TH, Gorson  KC,  et al; PeriNomS Study Group.  Impairment measures versus inflammatory RODS in GBS and CIDP: a responsiveness comparison.   J Peripher Nerv Syst. 2015;20(3):289-295. doi:10.1111/jns.12118PubMedGoogle ScholarCrossref
26.
Hobart  JC, Riazi  A, Lamping  DL, Fitzpatrick  R, Thompson  AJ.  How responsive is the Multiple Sclerosis Impact Scale (MSIS-29)? a comparison with some other self report scales.   J Neurol Neurosurg Psychiatry. 2005;76(11):1539-1543. doi:10.1136/jnnp.2005.064584PubMedGoogle ScholarCrossref
27.
Emory University School of Medicine. Emory ALS center. https://med.emory.edu/departments/neurology/programs_centers/emory_als_center/. Accessed November 19, 2019.
28.
Knudson  RJ, Lebowitz  MD, Holberg  CJ, Burrows  B.  Changes in the normal maximal expiratory flow-volume curve with growth and aging.   Am Rev Respir Dis. 1983;127(6):725-734.PubMedGoogle Scholar
29.
Jenkinson  C, Fitzpatrick  R, Brennan  C, Bromberg  M, Swash  M.  Development and validation of a short measure of health status for individuals with amyotrophic lateral sclerosis/motor neurone disease: the ALSAQ-40.   J Neurol. 1999;246(suppl 3):III16-III21. doi:10.1007/BF03161085PubMedGoogle ScholarCrossref
30.
Jenkinson  C, Hobart  J, Chandola  T, Fitzpatrick  R, Peto  V, Swash  M; ALS-HPS Steering Group. Amyotrophic Lateral Sclerosis Health Profile Study.  Use of the short form health survey (SF-36) in patients with amyotrophic lateral sclerosis: tests of data quality, score reliability, response rate and scaling assumptions.   J Neurol. 2002;249(2):178-183. doi:10.1007/PL00007861PubMedGoogle ScholarCrossref
31.
Kidney  D, Alexander  M, Corr  B, O’Toole  O, Hardiman  O.  Oropharyngeal dysphagia in amyotrophic lateral sclerosis: neurological and dysphagia specific rating scales.   Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5(3):150-153. doi:10.1080/14660820410019675PubMedGoogle ScholarCrossref
32.
Barrett  L, Cano  S, Zajicek  J, Hobart  J.  Lending a hand: can DASH items help ABILHAND improve manual ability measurement in multiple sclerosis?   Mult Scler. 2015;21(5):612-621. doi:10.1177/1352458514549396PubMedGoogle ScholarCrossref
33.
Chen  CC, Bode  RK.  Psychometric validation of the Manual Ability Measure-36 (MAM-36) in patients with neurologic and musculoskeletal disorders.   Arch Phys Med Rehabil. 2010;91(3):414-420. doi:10.1016/j.apmr.2009.11.012PubMedGoogle ScholarCrossref
34.
Bacci  ED, Wyrwich  KW, Phillips  GA, Vollmer  T, Guo  S.  Analysis of the psychometric properties of the Multiple Sclerosis Impact Scale-29 (MSIS-29) in relapsing-remitting multiple sclerosis using classical and modern test theory.   Mult Scler J Exp Transl Clin. 2016;2:2055217316673235. doi:10.1177/2055217316673235PubMedGoogle Scholar
35.
Linacre  JM.  Rasch analysis of rank-ordered data.   J Appl Meas. 2006;7(1):129-139.PubMedGoogle Scholar
36.
Linacre  JM.  Sample size and item calibration stability.   Rasch Meas Trans. 1994;7:328.Google Scholar
37.
Wyse  AE.  Scale anchoring with the Rasch Model.   J Appl Meas. 2017;18(1):43-53.PubMedGoogle Scholar
38.
Wolfe  EW, Smith  EV  Jr.  Instrument development tools and activities for measure validation using Rasch models: part II: validation activities.   J Appl Meas. 2007;8(2):204-234.PubMedGoogle Scholar
39.
Wolfe  EW, Smith  EV  Jr.  Instrument development tools and activities for measure validation using Rasch models: part I: instrument development tools.   J Appl Meas. 2007;8(1):97-123.PubMedGoogle Scholar
40.
European Medicines Agency. Radicava: withdrawal of the marketing authorisation application. https://www.ema.europa.eu/en/medicines/human/withdrawn-applications/radicava. Accessed November 18, 2019.
41.
Food and Drug Administration.  Guidance for Industry on Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims; Availability. Published December 9, 2009. https://www.federalregister.gov/documents/2009/12/09/E9-29273/guidance-for-industry-on-patient-reported-outcome-measures-use-in-medical-product-development-to. Accessed November 18, 2019.
42.
Northeast Amyotrophic Lateral Sclerosis Consortium. NEALS clinical trials capabilities. https://www.neals.org/for-als-researchers/neals-clinical-trial-capabilities/. Accessed November 20, 2019.
43.
Al-Chalabi  A,. ALSFRS-R training: ENCALS: SOP v1.2. Published June 2015. https://www.encals.eu/wp-content/uploads/2016/09/ALSFRS-SOP-ENCALS-presentation.pdf. Accessed November 18, 2019.
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    December 30, 2019

    Development and Validation of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS)

    Author Affiliations
    • 1Atlanta VA Medical Center, Atlanta, Georgia
    • 2Emory University, Atlanta, Georgia
    • 3Duke University, Durham, North Carolina
    • 4University of Pennsylvania, Philadelphia
    • 5Lahey Clinic, Burlington, Massachusetts
    JAMA Neurol. 2020;77(4):480-488. doi:10.1001/jamaneurol.2019.4490
    Key Points

    Question  By combining clinical expertise with mathematically rigorous Rasch methodology, can a new self-reported amyotrophic lateral sclerosis disability scale be developed with improved reliability and responsiveness compared with the ordinal scale in current use?

    Findings  In this study of 243 individuals, the 28-question Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale was created and had improved item targeting and high test-retest reliability.

    Meaning  This new Rasch-Built disability scale may serve as an easily accessible outcome measure that could improve the efficiency of amyotrophic lateral sclerosis clinical trials and serve as an informative tool in the clinic setting.

    Abstract

    Importance  A new outcome measure for overall disability level with improved responsiveness is needed for amyotrophic lateral sclerosis (ALS) clinical trials.

    Objective  To describe the creation and development of a new self-reported ALS disability scale with improved item targeting and psychometric properties that used a mathematically rigorous Rasch methodology.

    Design, Setting, and Participants  A preliminary ALS disability questionnaire with 119 questions was created based on literature review, clinical judgement of an expert panel, and patient input. Patients with ALS were recruited from January 2017 to June 2019 from the Emory University and Atlanta VA Medical Center ALS clinics, both in Atlanta, Georgia, during regularly scheduled clinic appointments to complete the draft questionnaire and standard ALS outcome measures. All consecutive patients seen at the Emory University and Atlanta VA Medical Center ALS clinics during the recruitment period with a diagnosis of ALS who were able to provide informed consent were invited to participate in the study. Rasch analyses were performed, and items were systematically removed based on missing data, model fit, disordered thresholds, item bias, and clinical judgment. A total of 509 patients with ALS were seen at the 2 sites during the recruitment period, and 264 patients provided informed consent.

    Interventions  Participants completed the draft Rasch questionnaire and the revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R).

    Main Outcomes and Measures  Rasch analyses and standard scale metrics were performed to create the new scale, and Rasch analyses were performed on the ALSFRS-R for comparison.

    Results  Overall, 243 participants with ALS completed the draft questionnaire, and 230 participants were included for Rasch analyses. The mean (SD) age for study participants was 61.9 (11.1) years, 146 (60.1%) were men, and site of onset was 23.0% bulbar (n = 56), 36.2% upper extremity (n = 88), and 39.5% lower extremity (n = 96). A 28-question Rasch-Built Overall ALS Disability Scale (ROADS) was constructed with each item scored 0, 1, or 2. The ROADS fulfilled Rasch model requirements, demonstrated improved item targeting compared with the ALSFRS-R, and had test-retest reliability of 0.97. Individual question fit statistics demonstrated infit values from 0.68 to 1.37 and outfit values from 0.66 to 1.43. The difference between the empirical variance explained by the measures and the modeled variance was 0.1%. The ALSFRS-R violated Rasch model expectations and demonstrated disordered thresholds for 9 of 12 questions; 13 of 48 answer choices on the ALSFRS-R were never the most probable answer choice for any overall disability level.

    Conclusions and Relevance  In this study, the 28-question, self-reported ROADS, which is linearly weighted, had improved item targeting compared with the ALSFRS-R, had high test-retest reliability, and was validated. ROADS may serve as a valuable and easily accessible outcome measure for use in ALS trials and in the clinic with improved responsiveness compared with the ALSFRS-R.

    Introduction

    The revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R) is commonly used as the primary outcome measure for contemporary clinical trials aiming to slow disease progression in patients with amyotrophic lateral sclerosis (ALS).1-6 The ALSFRS-R is an ordinal rating scale that includes 12 questions, each rated 0 through 4, that assesses the overall functional status of the patient with ALS.7 Strengths of this scale include questions with obvious face validity and a reasonable, although incomplete, correlation of the ALSFRS-R sum score with other more difficult and costly outcome measures such as measures of strength, electrophysiologic measures, or survival.8-10 The ALSFRS-R is accessible for widespread use and can be administered over the telephone or online.11,12

    However, the ALSFRS-R also has problems that hinder its clinical and research utility. Analysis of an ALS pooled clinical trial database (PRO-ACT13) has shown that 25% of patients receiving placebo or a failed therapeutic agent showed no decline on the ALSFRS-R over a 6-month period.14 This striking lack of scale responsiveness dilutes the ability to detect a treatment effect in clinical trials if one is in fact present, particularly when many ALS treatment trials test agents designed to slow disease progression and use a 6-month treatment period. The ALSFRS-R also contains questions that show improvement due to changes in symptom management or due to changes in behavior. For example, if a patient receives symptomatic treatment to reduce sialorrhea, score improvements can occur.15 As another example, patients lose points on the ALSFRS-R as they increase use of noninvasive positive pressure ventilation, and score improvements would occur if a patient chooses to discontinue noninvasive ventilation even if ventilatory muscle strength is declining. Despite the fact that ALS is an almost universally progressive disease, 14% of patients in a pooled ALS clinical trial database experienced a 180-day period of improved ALSFRS-R slope,14 which does not match the biologic reality of ALS disease course or the findings on objective outcome measures.10,16,17

    Modern test theory techniques, such as Rasch methodology, can be used to create scales that outperform traditional ordinal scales such as the ALSFRS-R, which are created with classic theory techniques.18-21 While both classic and modern test theory approaches rely on assessment of construct validity, reliability, and internal consistency, classic methods rank data based on whether a value is greater or less compared with another value, but the distance between each number cannot be quantified and is expected to be unequal between categories.18 Rasch-built scales are linearly weighted, meaning that a 1-point change is a consistent measurable unit across the scale, an important feature when a scale sum score is being used as an outcome measure. The ALSFRS-R is not linearly weighted. As an example, the score of a patient with ALS will decline by 3 points on the ALSFRS-R if there is a change from climbing stairs normally to holding the handrail while climbing stairs, and a 3-point loss also occurs if a patient goes from normal dressing and hygiene to needing an attendant for self-care. This discrepancy shows that a 3-point loss in the total score can indicate a slight decline in function or a drastic loss of function; 1 point is not a measurable unit of function on an ordinal scale such as the ALSFRS-R.

    Rasch-built scales are also unidimensional, meaning that all questions are measuring the same domain, and thus the sum score is a meaningful measure of overall status. The ALSFRS-R purports to measure functional ability, but it also includes respiratory questions that are not a measure of function.22 For example, a patient who is compliant with noninvasive ventilation will have a lower score on the ALSFRS-R compared with a patient with identical vital capacity who does not comply with noninvasive ventilation. Prior Rasch analyses of the ALSFRS-R have objectively demonstrated the lack of unidimensionality on the ALSFRS-R.23,24

    Rasch-built scales also allow for detailed assessment of item targeting, meaning that the difficulty level of each item is quantified to allow careful selection of items capturing a broad range of ability levels. Improved item targeting is likely one of the reasons that Rasch-built scales have been shown to be more responsive than classically built ordinal scales when assessing outcomes in other neurologic diseases,25,26 something that is direly needed to conduct more efficient ALS clinical trials. The goal of this study was to create and validate a Rasch-Built Overall ALS Disability Scale (ROADS), a publicly available tool that is available at no charge,27 for use as a clinical outcome measure in patients with ALS.

    Methods
    Participants

    Patients with a diagnosis of ALS at all stages of disease who were able to provide informed consent were recruited consecutively from the Emory University and the Atlanta VA Medical Center ALS clinics, both in Atlanta, Georgia, from January 2017 to June 2019 during regularly scheduled clinic appointments. The institutional review boards of both institutions approved the study. Written informed consent was obtained from all participants. Questionnaire responses were provided by the patient, but caregivers or staff members were allowed to mark answer responses for patients needing assistance.

    Study Procedures

    At the time of the clinic visit, study participants completed the first questionnaire containing 119 questions along with standard ALS outcome measures, the ALSFRS-R and vital capacity spirometry testing. The ALSFRS-R was completed by trained research coordinators with certification for scale administration, and spirometry was performed either by a trained research coordinator with spirometry certification or by a licensed respiratory therapist (V.J., M.P, and nonauthors). Vital capacity percentage of predicted value was calculated automatically by the spirometry devices using Knudson criteria.28 On completion of the clinic visit, participants were given another blank copy of the 119-question questionnaire to complete at home within 2 to 7 days to return to study team by mail for assessment of test-retest reliability.

    Questionnaire Development

    A preliminary ALS disability questionnaire was created and modified based on literature review, clinical judgment of an expert panel, and patient input. Literature review included existing scales that assess function or disability specifically in an ALS population,7,29 scales assessing function or disability that have been validated in an ALS population30,31 and relevant scales of function or disability for other neurologic diseases that were built or modified with Rasch methodology.20,21,32-34 The expert panel included 5 neurologists (C.N.F., R.B., C.Q., J.R., and J.D.G), 1 physical therapist (D.B.), and 1 speech-language pathologist (K.H.K.), all with ALS subspecialty expertise and in-depth experience working with patients with ALS in tertiary academic referral centers. The goal of expert panel and patient review in developing the initial question bank was to ensure creation of a comprehensive question bank assessing the full range of ALS disability levels and phenotypes. The initial questionnaire contained 119 questions related to daily activities for a broad-based ALS population. For each question, item response options were 0, unable to perform; 1, abnormal: able to perform but with difficulty; 2, normal: able to perform without difficulty; or 3, not applicable. The written instructions provided additional clarification that answers should be scored based on how the task is usually performed; items should be scored as 2 (normal) if the task was performed as easily and as quickly as it was before having ALS symptoms, and items should be scored as 1 (abnormal) if it was harder to perform the task, the task takes more time or effort, or the task is performed with the assistance of a device or another person.

    Statistical Analyses

    Questionnaire items with more than 10% missing values were removed, and participants were excluded from analysis if more than 10% of items were missing from their questionnaire. Items marked as 3 (not applicable) were considered as missing. Rasch analyses were performed on remaining responses with Winsteps software (version 4.4.4) using the partial credit model, allowing each item to have its own response structure.35 Category probability curves should demonstrate that for the most severe overall disability level, the lowest item response selection is the most probable, and as overall disability level improves, each item response answer sequentially becomes the most probable response (Figure 1). When this does not occur, an item demonstrates disordered thresholds. Category probability curves were examined for each question, and any question with disordered thresholds was removed.

    Differential item functioning occurs when 2 or more groups of respondents of the same ability level respond differently to an item based on a factor other than disability. Differential item functioning was examined on the basis of age and sex, and any item demonstrating significant bias was removed.

    Analyses of dimensionality, individual fit statistics, and overall model fit statistics were assessed as questions were systematically removed from the questionnaire to create the final scale. Principal components analysis on the standardized residuals was used to assess dimensionality, and variance explained by the measured construct of greater than 50% was considered sufficient for unidimensionality.35 Fit statistics examined include infit or the inlier-sensitive fit, outfit or the outlier-sensitive fit, and mean-square statistics showing the size of the randomness.35 A correlation matrix was examined to assess local dependency, or the association of each question to other questions on the scale, to avoid interdependent questions.

    The overall disability score was compared among patients of different sites of onset to attain the clinically based goal of balancing assessment of bulbar, upper extremity, and lower extremity function. The expert panel reviewed the final questionnaire to evaluate item difficulty order, content validity, and overall clinical utility of the complete scale. The expert panel removed any questions with insufficient face validity or that were expected to demonstrate improvement in score due to factors other than ALS reversal.

    Per prior Rasch literature, a sample size of 250 participants provides 99% confidence with stable item calibration with ±0.5 logits, allowing for a stable model.20,36 External construct validity was evaluated by assessing correlations of the total ROADS score to the total ALSFRS-R score and to vital capacity percentage of predicted value completed on the same day. Rasch analyses were also performed on the ALSFRS-R using an anchoring approach37 with common ROADS items to allow direct comparison of scale metrics and item targeting. Anchoring sets the Rasch measure for common questions seen on both scales to the same value, allowing direct comparison of item difficulty measures along the same linear ruler. Specifically, for items that appear on both the ROADS and the ALSFRS-R, the logit difficulty measures that were calculated through Rasch analyses of the ROADS were preset to identical values when performing Rasch analyses on the ALSFRS-R. Test-retest reliability was assessed by examining correlation coefficients for the overall ROADS 28-question sum score for 2 questionnaires completed 2 to 7 days apart. Analysis began July 2019.

    Results
    Demographics

    A total of 509 patients with ALS were seen at the 2 sites during the recruitment period. An unselected group of 264 patients with ALS provided informed consent for study participation, and 243 completed at least one 119-item questionnaire. Characteristics of these 243 patients are included in Table 1. Thirteen study participants were removed from the data set prior to Rasch analyses owing to more than 10% missing answers, and 7 questions were removed owing to more than 10% missing responses. A total of 67 participants completed a second 119-item questionnaire within 2 to 7 days for the test-retest analyses.

    Rasch Analyses

    Category probability curves were examined for each remaining question, and 4 questions (drive a car, lick an envelope or stamp, run, and whisper quietly) were removed owing to disordered thresholds, meaning that at least 1 of the answer choices was never the most probable response for any overall disability level (Figure 1B). No items demonstrated significant differential item functioning on the basis of age when comparing participants above and below the mean age for this data set. Two items (read a newspaper or book and watch television) demonstrated differential item functioning on the basis of sex and were removed.

    For the remaining questions, the panel of clinical experts removed any items with insufficient face validity or items that were expected to improve owing to factors other than ALS reversal. Additional items were systematically removed based on individual item fit statistics, overall model fit, and local dependency based on the item correlation matrix, leaving the final 28-item ROADS (Figure 2) with each item scored as 0, 1, or 2. This final scale met Rasch model expectations. Item-measure correlations were all positive, ranging from 0.53 to 0.74, providing evidence of content validity. A negative item–correlation measure would suggest a negative correlation between disability and item responses, which can happen with reverse coding or improperly worded questions.

    Infit and outfit statistics indicate how accurately the data for each question fit the Rasch model, with a value of 1 indicating a perfect fit, and values between 0.5 to 1.5 considered productive for measurement.38,39 Individual question fit statistics for the ROADS demonstrated that infit values on each item ranged from 0.68 to 1.37, and outfit values ranged from 0.66 to 1.43, indicating appropriate item fit for all selected questions.38,39 Principal components analysis on the standardized residuals was used to assess dimensionality, which is an assessment of whether questions are measuring the same domain, in this case disability. A variance explained by the measured construct of greater than 50% is considered sufficient for unidimensionality,35 and for the ROADS variance explained by the measured construct was 58.2%. The strength of the first residual contrast showed insufficient association with the total measurement structure, meaning that this group of questions may not have segregated from the others enough to warrant their classification as a different dimension. The difference between the empirical variance explained by the measures and the modeled variance was 0.1%, indicating a close match between what was observed in the ROADS and what was expected based on the Rasch model.

    Test-Retest Reliability and External Construct Validity

    Test-retest reliability on the 28 ROADS questions for the 67 participants who completed 2 questionnaires was excellent with an intraclass correlation coefficient of 0.97. In a prior study, interrater reliability for the ALSFRS-R has been reported as 0.93, while intrarater reliability has been reported as 0.95.11 External construct validity was good between the ROADS and the ALSFRS-R with a correlation of 0.82. Vital capacity percentage of predicted value and ROADS had a moderate correlation of 0.57, matching the expected association between disability level and ventilatory status in patients with ALS as shown in prior clinical studies.7 The correlation between the ALSFRS-R sum score and the vital capacity percentage of predicted value was 0.65.

    ALSFRS-R Analyses

    Rasch analyses were performed on the ALSFRS-R data set from the same patient cohort (Figure 3). ROADS items use a knife and fork (measure = 0.16) and roll over in bed (measure = −0.03) were used as anchors for ALSFRS-R questions 5 (cutting food) and 7 (turning in bed), meaning that the logit measures for the ROADS questions were preset to 0.16 and −0.03 for the corresponding ALSFRS-R questions when Rasch analyses were performed on the ALSFRS-R. The ROADS targeted a broader range of disability levels compared with the ALSFRS-R; item difficulties spanned 5.9 logits for the ROADS (range, −2.97 to 2.93 logits) but only 2.27 logits (range, −1.32 to 0.95 logits) for the ALSFRS-R.

    Examination of category probability curves of the ALSFRS-R (Figure 1C) showed that 9 of 12 questions demonstrated disordered thresholds. Thirteen of 48 answer responses on the ALSFRS-R were never the most probable answer choice for any overall functional rating ability.

    Overall disability or functional rating score was not different between patients with bulbar vs upper extremity vs lower extremity symptom onset either on the ROADS or the ALSFRS-R. For the ROADS, the mean (SD) raw sum score was 28.5 (11.8) for patients with bulbar onset, 26.5 (8.6) for patients with lower-extremity onset, and 25.5 (10.0) for patients with upper-extremity onset. For the ALSFRS-R, the mean (SD) sum score was 26.7 (9.0) for patients with bulbar onset, 28.5 (8.3) for patients with lower-extremity onset, and 27.8 (9.8) for patients with upper-extremity onset.

    ROADS Scoring

    As logit units are not intuitive, Table 2 provides a conversion from logit units to normed whole numbers provided by WINSTEPS software, ranging from 0 to 146. While the raw sum score is not linearly weighted, normed whole number scores are linearly weighted, meaning that a 1-point change is a measurable and consistent unit of disability across the entire scale, and a 2-point change reflects twice the disability level compared with a 1-point change.

    Discussion

    We successfully created and validated ROADS, which is expected to overcome many of the limitations seen with the currently used ordinal ALSFRS-R. While there are always limitations to subjective self-reported outcome measures, using Rasch methodology brings a mathematical rigor to this type of scale that is not present in standard ordinal scales. The ROADS demonstrates a wider range of item targeting compared with the ALSFRS-R, meaning that while both scales contain questions targeting respondents of average disability level, only the ROADS contains questions targeting respondents with higher and lower levels of overall disability, and thus the ROADS is expected to be able to better differentiate levels of overall disability for respondents with very high or very low levels of overall disability. The breadth of item targeting is expected to improve scale responsiveness, meaning that the scale is more likely to detect change when change has actually occurred. The performance of the ROADS in terms of responsiveness will need to be tested in future longitudinal studies for confirmation.

    Unlike the ALSFRS-R, the ROADS is linearly weighted, meaning that a 1-point change in the overall normed score captures a measurable unit of disability that is consistent across the entire scale. A 1-point change in the ordinal ALSFRS-R is not a quantifiable measure of function and can represent a small or large loss of functional ability depending on the question. Dimensionality analyses of the ROADS confirm that each question is measuring the same domain (disability), while the ALSFRS-R violates unidimensionality,23,24 meaning that changes in the ALSFRS-R might not be associated with functional status. Additionally, ROADS questions were carefully selected by the panel of clinical experts to avoid questions that may improve in the absence of disease reversal14 or owing to factors not associated with ALS. These limitations are highly relevant given that drugs are approved or disapproved for slowing disease progression in ALS based on ALSFRS differences of only 2 to 3 points.3,40

    Selection of an appropriate primary outcome measure is vital for the success and efficiency of clinical trials. Creation of an improved outcome measure to quantify overall disability level, a highly relevant and important clinical outcome, is particularly important for ALS where there is no criterion standard to measure disease progression, there is no objective measure of overall disability or functional status, and current candidate biomarkers are still in exploratory stages. Even if biomarkers of disease activity are validated in the future, clinically meaningful outcome measures will still remain relevant for patients and clinicians and are strongly encouraged by the US Food and Drug Administration for use in clinical trials.41

    Future studies are planned to examine the longitudinal performance of the ROADS, assess correlation of ROADS and survival, and examine predictive features of the scale. Ongoing studies are also planned to determine test-retest reliability for telephone-administered scales and scales completed by live-in caregivers.

    Limitations

    In the ROADS instructions, patients are asked to score a task as 2 (normal) if the task is performed as easily and as quickly as it was before having ALS symptoms. These instructions introduce the risk of recall errors if patients are uncertain of prior task performance. Although the ALSFRS-R was not initially designed to compare current performance with pre-ALS performance, the scale as it is currently administered for ALS clinical trials does include these explicit instructions,42,43 and thus the ALSFRS-R also has this same risk of recall errors.

    Conclusions

    The 28-question, self-reported ROADS was successfully created and validated. This scale is linearly weighted, has improved item targeting compared with the ALSFRS-R, and has high test-retest reliability. The ROADS can serve as a valuable and easily accessible outcome measure for use in ALS trials and in the clinic.

    Back to top
    Article Information

    Corresponding Author: Christina N. Fournier, MD, MSc, Emory University, Department of Neurology, 101 Woodruff Cir, Ste 6000, Atlanta, GA 30022 (cfourni@emory.edu).

    Accepted for Publication: November 1, 2019.

    Published Online: December 30, 2019. doi:10.1001/jamaneurol.2019.4490

    Author Contributions: Dr Fournier had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Fournier, Bedlack, Russell, Beckwith, Kaminski, Tyor, Hertzberg, Glass.

    Acquisition, analysis, or interpretation of data: Fournier, Bedlack, Quinn, Hertzberg, James, Polak, Glass.

    Drafting of the manuscript: Fournier, Quinn, Russell, Tyor, Polak, Glass.

    Critical revision of the manuscript for important intellectual content: Fournier, Bedlack, Russell, Beckwith, Kaminski, Tyor, Hertzberg, James, Glass.

    Statistical analysis: Fournier, Hertzberg.

    Obtained funding: Fournier.

    Administrative, technical, or material support: Bedlack, Kaminski, James, Polak.

    Supervision: Fournier, Bedlack, Quinn, Tyor, Hertzberg, Polak, Glass.

    Conflict of Interest Disclosures: Dr Bedlack reports grants from ALS Association, Motor Neuron Disease Association, Cytokinetics, Orion, and Ultragenyx and personal fees from ALS Association, Mallinkrodt, Biohaven Pharmaceutical, ITF Pharma, Brainstorm Cell Therapeutics, Biogen, New Biotic, Cytokinetics, and Woolsey Pharmaceuticals outside the submitted work. Dr Quinn reports personal fees for serving on the advisory boards of Acceleron Pharma and Amicus Therapeutics outside the submitted work. Dr Tyor reports grants from Atlanta VA Medical Center during the conduct of the study. No other disclosures were reported.

    Funding/Support: This study was funded by the US Department of Veterans Affairs Office of Research and Development (grant IK2CX001595-02; Dr Fournier).

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    References
    1.
    Mora  JS, Genge  A, Chio  A,  et al; AB10015 STUDY GROUP.  Masitinib as an add-on therapy to riluzole in patients with amyotrophic lateral sclerosis: a randomized clinical trial.   Amyotroph Lateral Scler Frontotemporal Degener. 2019;1-10. doi:10.1080/21678421.2019.1632346PubMedGoogle Scholar
    2.
    Statland  JM, Moore  D, Wang  Y,  et al; Rasagiline Investigators of the Muscle Study Group and Western ALS Consortium.  Rasagiline for amyotrophic lateral sclerosis: a randomized, controlled trial.   Muscle Nerve. 2019;59(2):201-207. doi:10.1002/mus.26335PubMedGoogle ScholarCrossref
    3.
    Writing  G, Edaravone  ALSSG; Writing Group; Edaravone (MCI-186) ALS 19 Study Group.  Safety and efficacy of edaravone in well defined patients with amyotrophic lateral sclerosis: a randomised, double-blind, placebo-controlled trial.   Lancet Neurol. 2017;16(7):505-512. doi:10.1016/S1474-4422(17)30115-1PubMedGoogle ScholarCrossref
    4.
    Shefner  JM, Wolff  AA, Meng  L,  et al; BENEFIT-ALS Study Group.  A randomized, placebo-controlled, double-blind phase IIb trial evaluating the safety and efficacy of tirasemtiv in patients with amyotrophic lateral sclerosis.   Amyotroph Lateral Scler Frontotemporal Degener. 2016;17(5-6):426-435. doi:10.3109/21678421.2016.1148169PubMedGoogle ScholarCrossref
    5.
    Shibuya  K, Misawa  S, Kimura  H,  et al.  A single blind randomized controlled clinical trial of mexiletine in amyotrophic lateral sclerosis: efficacy and safety of sodium channel blocker phase II trial.   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(5-6):353-358. doi:10.3109/21678421.2015.1038277PubMedGoogle ScholarCrossref
    6.
    Fournier  CN, Schoenfeld  D, Berry  JD,  et al.  An open label study of a novel immunosuppression intervention for the treatment of amyotrophic lateral sclerosis.   Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(3-4):242-249. doi:10.1080/21678421.2017.1421666PubMedGoogle ScholarCrossref
    7.
    Cedarbaum  JM, Stambler  N, Malta  E,  et al; BDNF ALS Study Group (Phase III).  The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function.   J Neurol Sci. 1999;169(1-2):13-21. doi:10.1016/S0022-510X(99)00210-5PubMedGoogle ScholarCrossref
    8.
    Rutkove  SB, Caress  JB, Cartwright  MS,  et al.  Electrical impedance myography as a biomarker to assess ALS progression.   Amyotroph Lateral Scler. 2012;13(5):439-445. doi:10.3109/17482968.2012.688837PubMedGoogle ScholarCrossref
    9.
    Kollewe  K, Mauss  U, Krampfl  K, Petri  S, Dengler  R, Mohammadi  B.  ALSFRS-R score and its ratio: a useful predictor for ALS-progression.   J Neurol Sci. 2008;275(1-2):69-73. doi:10.1016/j.jns.2008.07.016PubMedGoogle ScholarCrossref
    10.
    Shefner  JM, Liu  D, Leitner  ML,  et al.  Quantitative strength testing in ALS clinical trials.   Neurology. 2016;87(6):617-624. doi:10.1212/WNL.0000000000002941PubMedGoogle Scholar
    11.
    Kaufmann  P, Levy  G, Montes  J,  et al; QALS study group.  Excellent inter-rater, intra-rater, and telephone-administered reliability of the ALSFRS-R in a multicenter clinical trial.   Amyotroph Lateral Scler. 2007;8(1):42-46. doi:10.1080/17482960600888156PubMedGoogle ScholarCrossref
    12.
    Maier  A, Holm  T, Wicks  P,  et al.  Online assessment of ALS functional rating scale compares well to in-clinic evaluation: a prospective trial.   Amyotroph Lateral Scler. 2012;13(2):210-216. doi:10.3109/17482968.2011.633268PubMedGoogle ScholarCrossref
    13.
    Atassi  N, Berry  J, Shui  A,  et al.  The PRO-ACT database: design, initial analyses, and predictive features.   Neurology. 2014;83(19):1719-1725. doi:10.1212/WNL.0000000000000951PubMedGoogle ScholarCrossref
    14.
    Bedlack  RS, Vaughan  T, Wicks  P,  et al.  How common are ALS plateaus and reversals?   Neurology. 2016;86(9):808-812. doi:10.1212/WNL.0000000000002251PubMedGoogle ScholarCrossref
    15.
    Jackson  CE, Gronseth  G, Rosenfeld  J,  et al; Muscle Study Group.  Randomized double-blind study of botulinum toxin type B for sialorrhea in ALS patients.   Muscle Nerve. 2009;39(2):137-143. doi:10.1002/mus.21213PubMedGoogle ScholarCrossref
    16.
    Shefner  JM, Rutkove  SB, Caress  JB,  et al.  Reducing sample size requirements for future ALS clinical trials with a dedicated electrical impedance myography system.   Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(7-8):555-561. doi:10.1080/21678421.2018.1510008PubMedGoogle ScholarCrossref
    17.
    Andres  PL, Allred  MP, Stephens  HE,  et al.  Fixed dynamometry is more sensitive than vital capacity or ALS rating scale.   Muscle Nerve. 2017;56(4):710-715. doi:10.1002/mus.25586PubMedGoogle ScholarCrossref
    18.
    Vanhoutte  EK, Hermans  MC, Faber  CG, Gorson  KC, Merkies  IS, Thonnard  JL; PeriNomS Study Group.  Rasch-ionale for neurologists.   J Peripher Nerv Syst. 2015;20(3):260-268. doi:10.1111/jns.12122PubMedGoogle ScholarCrossref
    19.
    Tennant  A, Conaghan  PG.  The Rasch measurement model in rheumatology: what is it and why use it? when should it be applied, and what should one look for in a Rasch paper?   Arthritis Rheum. 2007;57(8):1358-1362. doi:10.1002/art.23108PubMedGoogle ScholarCrossref
    20.
    van Nes  SI, Vanhoutte  EK, van Doorn  PA,  et al.  Rasch-built Overall Disability Scale (R-ODS) for immune-mediated peripheral neuropathies.   Neurology. 2011;76(4):337-345. doi:10.1212/WNL.0b013e318208824bPubMedGoogle ScholarCrossref
    21.
    Vanhoutte  EK, Faber  CG, van Nes  SI,  et al; PeriNomS Study Group.  Rasch-built overall disability scale for multifocal motor neuropathy (MMN-RODS(c)).   J Peripher Nerv Syst. 2015;20(3):296-305. doi:10.1111/jns.12141PubMedGoogle ScholarCrossref
    22.
    Pinto  S, de Carvalho  M.  The R of ALSFRS-R: does it really mirror functional respiratory involvement in amyotrophic lateral sclerosis?   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(1-2):120-123. doi:10.3109/21678421.2014.952641PubMedGoogle ScholarCrossref
    23.
    Franchignoni  F, Mandrioli  J, Giordano  A, Ferro  S; ERRALS Group.  A further Rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements.   Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(5-6):331-337. doi:10.3109/21678421.2015.1026829PubMedGoogle ScholarCrossref
    24.
    Franchignoni  F, Mora  G, Giordano  A, Volanti  P, Chiò  A.  Evidence of multidimensionality in the ALSFRS-R Scale: a critical appraisal on its measurement properties using Rasch analysis.   J Neurol Neurosurg Psychiatry. 2013;84(12):1340-1345. doi:10.1136/jnnp-2012-304701PubMedGoogle ScholarCrossref
    25.
    Vanhoutte  EK, Draak  TH, Gorson  KC,  et al; PeriNomS Study Group.  Impairment measures versus inflammatory RODS in GBS and CIDP: a responsiveness comparison.   J Peripher Nerv Syst. 2015;20(3):289-295. doi:10.1111/jns.12118PubMedGoogle ScholarCrossref
    26.
    Hobart  JC, Riazi  A, Lamping  DL, Fitzpatrick  R, Thompson  AJ.  How responsive is the Multiple Sclerosis Impact Scale (MSIS-29)? a comparison with some other self report scales.   J Neurol Neurosurg Psychiatry. 2005;76(11):1539-1543. doi:10.1136/jnnp.2005.064584PubMedGoogle ScholarCrossref
    27.
    Emory University School of Medicine. Emory ALS center. https://med.emory.edu/departments/neurology/programs_centers/emory_als_center/. Accessed November 19, 2019.
    28.
    Knudson  RJ, Lebowitz  MD, Holberg  CJ, Burrows  B.  Changes in the normal maximal expiratory flow-volume curve with growth and aging.   Am Rev Respir Dis. 1983;127(6):725-734.PubMedGoogle Scholar
    29.
    Jenkinson  C, Fitzpatrick  R, Brennan  C, Bromberg  M, Swash  M.  Development and validation of a short measure of health status for individuals with amyotrophic lateral sclerosis/motor neurone disease: the ALSAQ-40.   J Neurol. 1999;246(suppl 3):III16-III21. doi:10.1007/BF03161085PubMedGoogle ScholarCrossref
    30.
    Jenkinson  C, Hobart  J, Chandola  T, Fitzpatrick  R, Peto  V, Swash  M; ALS-HPS Steering Group. Amyotrophic Lateral Sclerosis Health Profile Study.  Use of the short form health survey (SF-36) in patients with amyotrophic lateral sclerosis: tests of data quality, score reliability, response rate and scaling assumptions.   J Neurol. 2002;249(2):178-183. doi:10.1007/PL00007861PubMedGoogle ScholarCrossref
    31.
    Kidney  D, Alexander  M, Corr  B, O’Toole  O, Hardiman  O.  Oropharyngeal dysphagia in amyotrophic lateral sclerosis: neurological and dysphagia specific rating scales.   Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5(3):150-153. doi:10.1080/14660820410019675PubMedGoogle ScholarCrossref
    32.
    Barrett  L, Cano  S, Zajicek  J, Hobart  J.  Lending a hand: can DASH items help ABILHAND improve manual ability measurement in multiple sclerosis?   Mult Scler. 2015;21(5):612-621. doi:10.1177/1352458514549396PubMedGoogle ScholarCrossref
    33.
    Chen  CC, Bode  RK.  Psychometric validation of the Manual Ability Measure-36 (MAM-36) in patients with neurologic and musculoskeletal disorders.   Arch Phys Med Rehabil. 2010;91(3):414-420. doi:10.1016/j.apmr.2009.11.012PubMedGoogle ScholarCrossref
    34.
    Bacci  ED, Wyrwich  KW, Phillips  GA, Vollmer  T, Guo  S.  Analysis of the psychometric properties of the Multiple Sclerosis Impact Scale-29 (MSIS-29) in relapsing-remitting multiple sclerosis using classical and modern test theory.   Mult Scler J Exp Transl Clin. 2016;2:2055217316673235. doi:10.1177/2055217316673235PubMedGoogle Scholar
    35.
    Linacre  JM.  Rasch analysis of rank-ordered data.   J Appl Meas. 2006;7(1):129-139.PubMedGoogle Scholar
    36.
    Linacre  JM.  Sample size and item calibration stability.   Rasch Meas Trans. 1994;7:328.Google Scholar
    37.
    Wyse  AE.  Scale anchoring with the Rasch Model.   J Appl Meas. 2017;18(1):43-53.PubMedGoogle Scholar
    38.
    Wolfe  EW, Smith  EV  Jr.  Instrument development tools and activities for measure validation using Rasch models: part II: validation activities.   J Appl Meas. 2007;8(2):204-234.PubMedGoogle Scholar
    39.
    Wolfe  EW, Smith  EV  Jr.  Instrument development tools and activities for measure validation using Rasch models: part I: instrument development tools.   J Appl Meas. 2007;8(1):97-123.PubMedGoogle Scholar
    40.
    European Medicines Agency. Radicava: withdrawal of the marketing authorisation application. https://www.ema.europa.eu/en/medicines/human/withdrawn-applications/radicava. Accessed November 18, 2019.
    41.
    Food and Drug Administration.  Guidance for Industry on Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims; Availability. Published December 9, 2009. https://www.federalregister.gov/documents/2009/12/09/E9-29273/guidance-for-industry-on-patient-reported-outcome-measures-use-in-medical-product-development-to. Accessed November 18, 2019.
    42.
    Northeast Amyotrophic Lateral Sclerosis Consortium. NEALS clinical trials capabilities. https://www.neals.org/for-als-researchers/neals-clinical-trial-capabilities/. Accessed November 20, 2019.
    43.
    Al-Chalabi  A,. ALSFRS-R training: ENCALS: SOP v1.2. Published June 2015. https://www.encals.eu/wp-content/uploads/2016/09/ALSFRS-SOP-ENCALS-presentation.pdf. Accessed November 18, 2019.
    ×