Consensus-Based Attributes for Identifying Patients With Spasmodic Dysphonia and Other Voice Disorders

Key Points

Question  Are guidelines needed for patient classification for research on adductor spasmodic dysphonia (ADSD), abductor SD (ABSD), voice tremor (VT), and muscular tension dysphonia (MTD)?

Findings  In this observational study of 178 patients with spasmodic dysphonia and other voice disorders, experts in SD examined uniform video recordings of speech and nasolaryngoscopy examinations of patients with these voice disorders without classification guidelines and had poor interrater agreement that did not relate to clinical classifications. Consequently, a Delphi panel and 46 community experts reached consensus on the Spasmodic Dysphonia Attributes Inventory (SDAI).

Meaning  The SDAI may be useful for selecting patients with SD, VT, or MTD for research on pathophysiology and treatment outcomes.

Importance  A roadblock for research on adductor spasmodic dysphonia (ADSD), abductor SD (ABSD), voice tremor (VT), and muscular tension dysphonia (MTD) is the lack of criteria for selecting patients with these disorders.

Objective  To determine the agreement among experts not using standard guidelines to classify patients with ABSD, ADSD, VT, and MTD, and develop expert consensus attributes for classifying patients for research.

Design, Setting and Participants  From 2011 to 2016, a multicenter observational study examined agreement among blinded experts when classifying patients with ADSD, ABSD, VT or MTD (first study). Subsequently, a 4-stage Delphi method study used reiterative stages of review by an expert panel and 46 community experts to develop consensus on attributes to be used for classifying patients with the 4 disorders (second study). The study used a convenience sample of 178 patients clinically diagnosed with ADSD, ABSD, VT MTD, vocal fold paresis/paralysis, psychogenic voice disorders, or hypophonia secondary to Parkinson disease. Participants were aged 18 years or older, without laryngeal structural disease or surgery for ADSD and underwent speech and nasolaryngoscopy video recordings following a standard protocol.

Exposures  Speech and nasolaryngoscopy video recordings following a standard protocol.

Main Outcomes and Measures  Specialists at 4 sites classified 178 patients into 11 categories. Four international experts independently classified 75 patients using the same categories without guidelines after viewing speech and nasolaryngoscopy video recordings. Each member from the 4 sites also classified 50 patients from other sites after viewing video clips of voice/laryngeal tasks. Interrater κ less than 0.40 indicated poor classification agreement among rater pairs and across recruiting sites. Consequently, a Delphi panel of 13 experts identified and ranked speech and laryngeal movement attributes for classifying ADSD, ABSD, VT, and MTD, which were reviewed by 46 community specialists. Based on the median attribute rankings, a final attribute list was created for each disorder.

Results  When classifying patients without guidelines, raters differed in their classification distributions (likelihood ratio, χ² = 107.66), had poor interrater agreement, and poor agreement with site categories. For 11 categories, the highest agreement was 34%, with no κ values greater than 0.26. In external rater pairs, the highest κ was 0.23 and the highest agreement was 38.5%. Using 6 categories, the highest percent agreement was 73.3% and the highest κ was 0.40. The Delphi method yielded 18 attributes for classifying disorders from speech and nasolaryngoscopic examinations.

Conclusions and Relevance  Specialists without guidelines had poor agreement when classifying patients for research, leading to a Delphi-based development of the Spasmodic Dysphonia Attributes Inventory for classifying patients with ADSD, ABSD, VT, and MTD for research.

Spasmodic dysphonia (SD) is a rare task-specific disorder¹ affecting voice production during speech.² Signs include voice breaks owing to spasmodic vocal fold hyperadduction during vowels in the adductor (ADSD) type^3,4 and prolonged voiceless consonants owing to spasmodic hyperabduction prior to vowels in the abductor (ABSD) type.⁵ No diagnostic test has been developed for SD and clinical diagnosis depends on specialists in voice care teams including otolaryngologists, speech-language pathologists (SLP), and neurologists experienced in the identification and treatment of SD,⁶ voice tremor (VT),⁷ and muscular tension dysphonia (MTD) considered a functional voice disorder.⁸ Many patients have limited access to specialized voice centers and often see multiple physicians before diagnosis, thereby delaying treatment by an average of 4.4 years.⁹

The lack of a method for classification of SD type, not only leads to delays in diagnosis and treatment,¹⁰ but impedes research for studying these disorders.¹¹ Although experienced members in a voice care team agree on classification of patients with SD,^12-15 voice care teams from different centers may not agree. Visualization of the larynx by laryngoscopy is needed to exclude other laryngeal abnormalities that interfere with voice production such as vocal fold paralysis (VFP) or laryngeal lesions. Excluding other laryngeal disorders is required before identifying SD,^10,11,13,16 VT,¹³ or MTD.^8,17 Although these disorders have similarities, the optimal treatment differs for each. The symptoms of ADSD are reduced by thyroarytenoid muscle botulinum toxin injections at regular intervals^18,19 or selective adductor denervation with reinnervation surgery.^20,21 Voice tremor is less responsive to botulinum toxin injections than ADSD,^7,22 and MTD commonly responds to voice therapy.^23,24

Measures conducted in SD include perceptual voice ratings,²⁵ acoustic measures,^4,5 and laryngeal electromyography.²⁶ None have demonstrated validity for differentiating SD from other voice disorders. Approaches suggested for differentiating ADSD from MTD include identifying phonatory breaks in speech,¹⁴ using sentence-level material,²⁷ and analysis of high-speed vocal fold vibration regularity.²⁸

We conducted 2 studies. First, we measured experts’ agreement on SD classification without using standardized guidelines when they reviewed speech and nasolaryngoscopic video recordings of unfamiliar patients. Second, voice experts employed the Delphi Method^29-31 with 4 reiterative stages to reach consensus on attributes for classifying patients.^32-37 Our aim was to develop classification methods to support research on ADSD, ABSD, VT, and MTD.

In the first study we determined classification agreement among experts without standardized classification criteria. This study involved participant recruitment, video recording and site classification; internationally recognized external rater classification without standardized criteria of patients from masked video recordings; and team rater classification of patients from other centers. In the subsequent study we used a 4-stage Delphi Process to reach consensus on attributes for classifying patients as having ADSD, ABSD, MTD, or VT (eFigure in the Supplement).

Voice care teams who regularly saw patients with SD from Emory University, Washington University in St Louis (WashU), The Medical College of Wisconsin (MCW), and Mayo Clinic Arizona (Mayo-Arizona) recruited participants. Each followed the same protocol and used the same consent form for informed written consent after local institutional review board and National Institute of Neurological Disorders and Stroke–National Institutes of Health approval.

At initial evaluation or on return for reinjection of botulinum toxin at least 2 months after previous injection, patients were approached for participation. Selection criteria were: aged 18 years or older, no known cause for the voice disorder, no medical condition precluding nasolaryngoscopy, no laryngeal structural disease (eg, lesions), no surgery for SD, and diagnosis of ADSD, ABSD, VT, MTD, vocal fold paralysis or paresis (VFP), or other voice disorders (psychogenic dysphonia or hypophonia with Parkinson disease).

After informed written consent, a laryngologist used flexible nasolaryngoscopy to exclude other laryngeal diseases/disorders and diagnose any patient who had vocal fold paralysis or paresis. Nasal topicalization was only used in 2 centers, MCW and Emory. Only participants classified by the laryngologist and SLP as having either ADSD, ABSD, or VT received a medical history and neurological examination. The participant’s clinical site classification was entered by the voice team based on their patient knowledge and nasolaryngoscopy by the laryngologist from 11 alternatives: ADSD, ABSD, VT, MTD, ADSD with VT, ABSD with VT, ADSD with MTD, ADSD with VT and MTD, VT with MTD, VFP, or other.

Each center used the same equipment and procedures. A head-held microphone was placed at a 45° angle 2 inches off the midline of the mouth to reduce aspiration noise and maintain a constant amplification level during digital video recording. After the examiner stated the date, patient number, and center, the participant repeated 10 ADSD sentences containing glottal stops in vowels to elicit breaks (eMethods in the Supplement). If the participant missed words in a sentence, the examiner repeated it until the sentence was said correctly while continuing to record. Participants also repeated 10 ABSD sentences containing voiceless consonants (p, t, k, f, h, s, th) before vowels, followed by 6 whispered sentences, prolonged vowels, and 2 shouted phrases (eMethods in the Supplement).

The nasolaryngoscope was first placed above the epiglottis to examine supraglottic movement abnormalities by recording posterior tongue and pharyngeal wall movement during repetition of a sniff followed by vowel /i/ repeated 3 times. Two cycles of quiet respiration were recorded. The scope then was moved downward to view the vocal folds during 3 sniff and /i/ repetitions. Next, 5 repetitions of a glottal stop and /i/ were repeated twice, followed by 5 repetitions of the word “see” twice. The participant whistled “Happy Birthday,” and twice prolonged /i/ for 10 seconds. Four sentences were repeated and 2 phrases were shouted (eMethods in the Supplement).

Stroboscopy during a prolonged vowel was not required during nasolaryngoscopy because vibratory tracking was often disrupted by instabilities owing to tremor, spasms, or severe aperiodicity. However, all centers included stroboscopy in the recorded examination whenever possible.

The deidentified digital speech and nasolaryngoscopy recordings were mailed on an encrypted drive to James Madison University, checked for sound, video quality, and task inclusion. The speech and nasolaryngoscopy tasks were extracted into separate video clips for review.

Two sets of ratings were conducted to assess interrater classification agreement from viewing the speech and nasolaryngoscopy recordings. First, 4 internationally recognized SD experts (C.L.L., C.S., G.B., and M.E.S.) not from the 4 voice centers, served as external raters for some of the first 75 participants. Raters 1 and 2 were SLPs, and raters 3 and 4 were laryngologists. Each reviewed the entire speech and nasolaryngoscopy unclipped video recordings without knowledge of the patient’s diagnosis. They were instructed to select the best classification from the same 11 categories used for site classification (eTable 1 in the Supplement).

Second, 12 team members from the 4 voice centers independently reviewed video recordings of 50 participants seen at other centers. The laryngologists and SLPs reviewed clips of tasks from the speech and nasolaryngoscopy recordings whereas the neurologists only reviewed speech tasks. Each team member independently selected the best classification from the same 11 categories after viewing the speech recording. The laryngologists and SLPs again selected the best classification after viewing nasolaryngoscopy recordings.

Interrater agreement was determined using Cohen’s κ coefficient and percent agreement using SAS statistical software (version 9.4, SAS Institute). The distribution of classifications among the external raters was compared using the likelihood ratio χ². Adequate agreement among the 4 external raters and the site classification, among the 4 external raters, and among pairs of the blinded site team raters was defined as κ≥0.60 and a percent agreement of 70% or more.

Interrater reliability was determined only for rater pairs who had reviewed 10 or more of the same participants’ video recordings. The agreement using 11 categories was assessed and then collapsed into the following 6 categories: ADSD for participants classified as ADSD regardless of accompanying disorders, ABSD for participants with ABSD regardless of accompanying disorders, VT only for those classified with VT alone, and MTD only for those with MTD alone. Those with both VT and MTD were classified as VT. Classifications of VFP and other (psychogenic or Parkinson disease) were retained.

A 4-stage Delphi method developed consensus on groups of speech and laryngeal attributes for each of 4 main disorders: ADSD, ABSD, VT, and MTD. The aim was to identify groups of attributes considered essential for classification of each disorder rather than defining combinations of disorders. The Delphi Consensus Group (DCG) included 3 team members from each center and an additional otolaryngologist as voting members and the principal investigator and a statistician.

First, each voice team jointly listed an unlimited number of voice and laryngeal attributes for each of the 4 disorders in an Excel file. Second, the DCG convened by conference call and deleted redundancies from the team attribute lists. Next each DCG member independently ranked attributes for each disorder from high to low importance and sent their ranking to the principal investigator. The results were collated and sent to the DCG which convened and eliminated poorly ranked attributes that did not differentiate among disorders.

Third, community SD experts were recruited to independently rank the attributes for each disorder from: the Neurolaryngology Study Section of the AAO-HNS, the Voice Disorders Interest Group of the American Speech-Language-Hearing Association and neurologists from the Movement Disorder Society. They were sent an email with a Qualtrics survey link for ranking the attributes for each disorder. They were asked to reorder the attributes based on their perceived order of importance and record their rankings by moving a bar from the top to the bottom of the screen for each disorder. The median rank of the community experts for each attribute per disorder was computed and the percent of experts determined who gave an attribute the same rank as the median. Finally, the DCG reviewed the ranked attributes from the community experts and finalized the attributes for each disorder creating the Spasmodic Dysphonia Attributes Inventory (SDAI).

Of 197 patients admitted for study, 19 were excluded owing to an unacceptable video recording, lack of a voice disorder, missing sound or tasks, leaving 178 participants; 66 from Emory; 55 from Mayo-Arizona; 38 from WashU; and 19 from MCW. The site classifications identified 102 (57.2%) with ADSD either alone or combined with VT and/or MTD (eTable 1 in the Supplement).

After collapsing the 11 categories into 6, the percentage classified by voice disorder varied across centers (Figure 1A). For example, 12 (20%) or more were diagnosed as MTD at WashU and Mayo-Arizona, none as MTD at MCW, and less than 3 (10%) at Emory. Thus, only 2 of 4 centers identified 20% or more of their participants as having MTD.

Percent agreement and Cohen’s κ values between external raters and site classification and was low for 11 and 6 categories (Table 1). For 11 categories, the highest agreement was 34%, with no κ values greater than 0.26. Using 6 categories, the highest percent agreement was 73.3% and the highest κ was 0.40 (Table 1). Thus, none of the external raters had acceptable agreement with the recruiting site classification.

A confusion matrix (eTable 2 in the Supplement) compared the 6 category classifications of the external raters and the sites on 160 patients. Of the 128 patients classified as ADSD by sites, only 74 (57.8%) were classified as ADSD by external raters, whereas 20 (15.6%) were classified as MTD only and 24 (18.75%) were classified as VT only by raters. Thus, the external raters differentiated between categories more than the sites (Figure 1).

The external raters differed in their distributions of classifications (Figure 1B). Raters 1 and 2 classified 37 and 12 (45% and 48%, respectively) as VT, whereas raters 3 and 4 classified 36 and 9 (80% and 64.3%, respectively) as ADSD.

Interrater agreement was low in external rater pairs (Table 1). Using 11 categories, the highest κ was 0.23 and the highest agreement was 38.5% for raters 3 and 4. Most external rater classifications were ADSD whereas other categories had zero cases (Figure 1B), likely adversely affecting the κ values. Using 6 categories, the highest agreement was 64% and the highest κ was 0.35 for raters 1 and 3 (Table 1). Although these values increased when using 6 categories, interrater agreement remained unsatisfactory.

Pairs of voice team members classifications using the 6 collapsed categories were examined based on whether they were either: different professionals at the same center; the same profession but different centers; or different centers and professions. Twenty-four rater pairs performed classifications on 10 or more of the same participants’ speech recordings; only 7 rater pairs completed classifications on 10 or more of the same participants after reviewing speech and nasolaryngoscopy recordings. Two team members’ ratings were discarded because they were not conducted independently.

The κ values among voice team members were also low (<0.60) (Figure 2). Only 2 of 24 κ values were above 0.43 for classification based on viewing speech and 3 of 7 based on viewing both speech and nasolaryngoscopy recordings (Figure 2). This indicates that although still unsatisfactory, interrater agreement improved after reviewing both speech and nasolaryngoscopy recordings.

First, the initial lists of attributes from each center yielded 52 attributes: 16 for ADSD, 15 for ABSD, 11 for MTD, and 10 for VT (Table 2). Second, the DCG reduced these to 42 by deleting 10 attributes that were: not specific to disorders, based on patient reporting, not required for identification, or required electromyography which is unavailable in many voice clinics. The median attribute ranks for each disorder by DCG members are in Table 2. Several nasolaryngoscopy items were retained including AD7 and AD11 to exclude other laryngeal disorders by otolaryngologists by noting, “normal structure and function of the vocal folds” and “intermittent vocal fold/arytenoid hyper adduction” respectively. Items AB2, AB3, MTD2, TR4, TR5, and TR7 were retained for similar reasons.

Third, the 81 community experts contacted included: 35 otolaryngologists, 34 SLPs, and 12 neurologists. Forty-nine (60.5%) responded including 17 otolaryngologists, 26 SLPs, and 6 neurologists. Experts were asked to rate attributes only on disorders familiar to them. Forty-five ranked ADSD attributes, 38 ABSD, 41 MTD, and 33 VT. The median attribute ranks by community experts for each disorder and the percent of community experts who gave an attribute the median ranking are in Table 2. Overall the ordering of attributes by the community experts agreed with the DCG. The highest-ranked items for each disorder had the highest agreement among experts.

Fourth, the DCG reviewed the community experts’ results and concluded that a smaller number of attributes could identify each disorder. They agreed that speech attributes could only identify a probable disorder whereas nasolaryngoscopy items were required to exclude other laryngeal disorders. The DCG identified 18 attributes for the SDAI: 11 speech attributes and 7 nasolaryngoscopy items across the 4 disorders (Figure 3).

In the first study, experienced specialists without standardized criteria had poor agreement classifying patients with SD and related voice disorders. Thus, research studies reporting patients with either ADSD, ABSD, VT, or MTD may have groups with dissimilar characteristics. Further, the distribution of patients differed across centers (Figure 1A), which may reflect both referral and classification differences among centers. For example, some centers had 30% patients with MTD whereas others had none or very few with MTD (Figure 1A). Of 102 patients with some form of ADSD, only 35 (19.7%) had ADSD alone; 27 (15.1%) also had VT, 31 (17.4%) also had MTD, and 9 (5.0%) had all 3 disorders (eTable 1 in the Supplement). Thus, patients with ADSD varied in their characteristics. Also, external raters tended to classify patients differently from the recruiting sites; the external raters classified 74 of 160 (46.25%) as ADSD whereas the sites classified 128 of 160 (80%) as ADSD (eTable 2 in the Supplement). This suggests that some experts tend to group all patients with ADSD as similar whereas others differentiate tremor and MTD from ADSD.

The second study used the Delphi process involving the DCG panel and community experts to establish consensus for identifying and ranking attributes characteristic of these different disorders. Agreement was high on the top attributes for ABSD, 97% for “intermittent breathy breaks” AB1 and 76% for “symptoms most evident in speech” AB7 and for VT, 97% for “pitch and amplitude modulation on prolonged vowels” TR1, and 88% on “tremor on multiple sites on nasolaryngoscopy” TR4 (Table 2). However, for MTD, the top attributes had lower agreement at 78% on “continuous effortful strained voice” MTD1 and 68% on “similar voice abnormalities on all types of sounds” MTD3. Agreement was lowest for ADSD at 53% for “intermittent glottal stops” AD1 and 47% for “patient report of speaking effort” AD14. Thus, the characteristics of ABSD and VT were better delineated by experts than those for ADSD and MTD.

The poor classification agreement among experts on patient characteristics found in the first study is problematic for research. The lack of agreement among experts on ADSD characteristics can impair clinical trials owing to participant cohort heterogeneity. Evaluation of whether using the attributes from the SDAI to classify patients improves the agreement of patient classification across experts and centers is the next step needed for future research on SD and related disorders.

This study had limitations. Most important was that the raters used video recordings of speech and nasolaryngoscopic examinations including stroboscopy whenever feasible but did not have extensive medical background on the patients. Therefore, this study did not assess the agreement between centers for clinical diagnosis of ADSD, ABSD, VT, and MTD. Rather we only examined agreement when raters were asked to classify patients for research. Also, the initial site classification for many patients with ADSD was prior to botulinum toxin treatment, which alters clinical manifestations. Further, some participants may have had injections as recently as 2 months earlier, which may not be enough time for full manifestations to recur. We excluded participants that had no symptom manifestations at time of recruitment.

Our findings demonstrate that classification by experts without benefit of standardized criteria failed to be consistent across independent experts and different recruiting sites. There was poor agreement among independent experts based on viewing extensive speech and nasolaryngoscopy recordings. It was unexpected that members of the same voice care team had unacceptable interrater agreement when conducting ratings independently both for speech and nasolaryngoscopy. Further, agreement was no better when raters were members of different voice care teams but from the same specialty. The 4-stage Delphi method permitted experts to reach consensus on attributes for identifying types of SD and related disorders. This checklist of attributes may allow investigators to develop homogenous patient cohorts for multicenter studies of ADSD, ABSD, VT, or MTD.

Corresponding Author: Christy L. Ludlow, PhD, Department of Communication Sciences and Disorders, James Madison University, PO Box 449, Washington, VA 22747 (christyll@gmail.com).

Accepted for Publication: March 20, 2018.

Published Online: June 21, 2018. doi:10.1001/jamaoto.2018.0644

Author Contributions: Dr Ludlow had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Ludlow, Domangue, Jinnah, Perlmutter, Sapienza, Blumin, Johns, Adler, Lott, Stebbins.

Acquisition, analysis, or interpretation of data: Ludlow, Domangue, Sharma, Jinnah, Perlmutter, Berke, Smith, Blumin, Kalata, Blindauer, Johns, Hapner, Harmon, Paniello, Adler, Crujido, Bansberg, Barone, Drulia, Stebbins.

Drafting of the manuscript: Ludlow, Domangue, Sharma, Sapienza, Barone, Stebbins.

Critical revision of the manuscript for important intellectual content: Ludlow, Sharma, Jinnah, Perlmutter, Berke, Smith, Blumin, Kalata, Blindauer, Johns, Hapner, Harmon, Paniello, Adler, Crujido, Lott, Bansberg, Barone, Drulia, Stebbins.

Statistical analysis: Ludlow, Domangue, Sharma, Stebbins.

Obtained funding: Ludlow, Jinnah, Blumin.

Administrative, technical, or material support: Ludlow, Jinnah, Perlmutter, Berke, Sapienza, Blumin, Johns, Crujido, Lott, Bansberg, Barone.

Study supervision: Ludlow, Jinnah, Lott.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: This study was supported by the National Spasmodic Dysphonia Association and a grant to the Dystonia Coalition from the Office of Rare Diseases Research in the National Institutes of Health (NIH) National Institute of Translational Sciences (grant number U54 TR001456) and the National Institute of Neurological Disorders and Stroke (grant number U45 NS065701) from the NIH.

Role of the Funder/Sponsor: the National Spasmodic Dysphonia Association and the National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.