Figure 1. Sample of an nasendoscopic view of the velopharynx and use of the Golding-Kushner scale. Velopharynx at rest (A), during speech production (B), at rest with superimposed grid (C), and during speech production with superimposed grid (D). In this sample, the palate movement is rated as 0.2; the posterior pharyngeal wall is rated 0.25; and the right and left lateral pharyngeal walls are rated 0.15 and 0.2, respectively. (For an explanation of the rating system, see the introductory section of the article.) The gap size is estimated to be 60%. Reproduced with permission from Sie and Chen.8
Figure 2. Sample of individual and consensus ratings for palatal and lateral wall motion. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).
Figure 3. Individual ratings for each of the 11 video segments using the Golding-Kushner scale in comparison to the consensus ratings. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).
Figure 4. Example of velopharyngeal closure with various features from the American Society of Pediatric Otolaryngology Velopharyngeal Insufficiency Study Group teaching tool. A and B, Velopharyngeal complex during rest. C, The presence of the Passavant ridge during closure. D, Presence of the notch on the nasal surface of soft palate during closure.
Tieu DD, Gerber ME, Milczuk HA, Parikh SR, Perkins JA, Yoon PJ, Sie KCY. Generation of Consensus in the Application of a Rating Scale to Nasendoscopic Assessment of Velopharyngeal Function. Arch Otolaryngol Head Neck Surg. 2012;138(10):923-928. doi:10.1001/archotol.2013.203
Author Affiliations: Division of Pediatric Otolaryngology (Drs Tieu, Parikh, Perkins, and Sie) and Childhood Communication Center (Dr Sie), Seattle Children's Hospital, University of Washington School of Medicine; Pediatric Otolaryngology–Head and Neck Surgery, Northshore University Health System, Chicago, Illinois (Dr Gerber); Department of Otolaryngology–Head and Neck Surgery, Oregon Health and Sciences University, Portland (Dr Milczuk); and Department of Pediatric Otolaryngology, Children's Hospital Colorado, University of Colorado, Aurora (Dr Yoon).
Objective To generate consensus ratings of velopharyngeal function on nasendoscopy (NE) with the goal of creating a video instruction tool.
Methods The American Society of Pediatric Otolaryngology Velopharyngeal Insufficiency Study Group convened to identify NE segments to be included in an instructional video. Of 24 segments reviewed, 11 were selected based on the quality of the examinations and spectrum of closure patterns. Participating otolaryngologists independently rated NE segments using the Golding-Kushner scale. The participants then convened and rated each of the NE segments as a group. Thirty-nine members of the American Society of Pediatric Otolaryngology met and agreed with the group ratings, creating a consensus standard.
Results Individual scores for palate and lateral wall motion showed high variability, ranging from 0 to 6 points difference from the consensus. Variability was also seen for the following qualitative findings: the Passavant ridge, aberrant pulsations, and dorsal palatal notch. The individual ratings are presented graphically to demonstrate the range of individual responses as well as to compare responses to the consensus ratings. No further changes were made to the proposed consensus ratings when reviewed by the larger group.
Conclusions Rating of NE evaluations of velopharyngeal function was variable among a group of pediatric otolaryngologists experienced in treating velopharyngeal insufficiency. These results highlight the need to develop a standardized method of reporting NE findings for velopharyngeal insufficiency. Despite this, consensus ratings were achieved that will facilitate development of a video instruction tool.
Velopharyngeal (VP) closure has an important role in normal speech production. Incomplete closure of the velopharyngeal complex, known as velopharyngeal insufficiency (VPI), is characterized by hypernasal resonance and nasal air emissions or turbulence during speech production. This dysfunction may be a result of anatomical abnormalities, cleft palate, or neuromuscular dysfunction.
The diagnosis of VPI remains a subjective determination on the basis of perceptual speech assessment.1,2 Perceptual speech assessment involves the evaluation of all the components of speech production (including voice), articulation (including compensatory misarticulations, motor speech, oral motor sequencing), velopharyngeal function, and overall speech intelligibility. Description of VP function includes ratings of resonance, nasal emissions or turbulence, and overall VPI severity. Features, such as hypernasal resonance and nasal rustle, were shown to be predictive of relative VP gap size in the absence of instrumental assessment.3 Although perceptual evaluation is performed by trained speech language pathologists, it is subjective. Furthermore, the degree of VPI does not reliably predict the degree of VP closure.
Estimating the amount of VP closure is important when surgical intervention is being considered. Alteration of the VP complex has the potential to compromise upper airway patency, leading to hyponasal resonance, nasal obstruction, or obstructive sleep apnea. Because the degree of VP closure varies among individual patients, other authors have advocated for surgical approaches that are tailored to each clinical situation.4- 6 For example, individuals who have a larger VP gap require more tissue augmentation to effectively treat the VPI.
Nasendoscopy provides a useful tool for direct visualization of the VP during speech, allowing for a 2-dimensional view of the VP complex. This allows the endoscopist to evaluate lateral wall and palatal motion during speech; determine the presence of a notch on the nasal surface of the palate; see the Passavant ridge and aberrant pulsations; and estimate gap size and shape. For example, the presence of a notch on the nasal palate surface suggests sagittal orientation of the levator veli palatini, which can be corrected with palatal procedures. Patients with evidence of transverse orientation of the levator veli palatini musculature may be candidates for pharyngeal or palatopharyngeal speech procedures. The nasendoscopic (NE) findings may be helpful in selecting the type of surgical procedure in addition to providing the surgeon with an estimate of degree of VP augmentation required.
An international working group, which was convened to address the description of VP function, published its recommendations in 1990.7 The group described a rating system in which the maximal VP movement during speech is rated relative to the resting VP gap. The scale subjectively rates right and left lateral wall movement on a scale from 0 to 0.5, with 0 denoting no movement and 0.5 denoting movement to the midline; right and left palate movement on a scale from 0 to 1.0, with 0 denoting no movement and 1.0 denoting movement to the posterior pharyngeal wall; and posterior wall movement on a scale of 0 to 1.0, with 0 denoting no movement and 1.0 denoting movement to meet the soft palate (Figure 1).8 Lateral wall motion and palatal motion are rated separately, and other characteristics of the closure pattern, such as gap shape and notch on the nasal surface of the palate, are indicated. This rating scale is intended to be applied to nasendoscopic and fluoroscopic examinations. Although instrumental assessment of VP function is more objective than perceptual speech assessment, there continues to be wide variation in the description of nasendoscopic assessment of VP function. Previous studies examining the reliability of NE evaluation have suggested that providing feedback during the evaluation or allowing group participation statistically improves reliability.9 Studies of intrarater and interrater reliability have generally shown that intrarater reliability is moderately good regardless of the level of training or amount of experience in managing patients with VPI. The interrater reliability was greatest among 2 experienced attending otolaryngologists at a single institution.10 In a study comparing the use of the Golding-Kushner scale among 16 otolaryngologists at 8 institutions, the interrater reliability was weaker.11
On the basis of the findings of previous studies and the need to have a reliable method of communicating the nasendoscopic findings of VP function for patient care, clinical training, and research, the American Society of Pediatric Otolaryngology (ASPO) VPI Study Group set out to create a video instruction tool. The ASPO VPI Study Group proposed a series of NE segments with consensus ratings of lateral pharyngeal, palatal, and posterior pharyngeal wall movement, and descriptions of the nasal surface of the palate, Passavant ridge, aberrant pulsations, and pattern of closure. A minor modification of the Golding-Kushner scale was used. Specifically, instead of assigning a decimal rating between 0 and 1 to the palate and lateral walls, whole numbers between 0 and 10 were used to simplify calculations. The goal of the NE video instruction tool is to facilitate a standardized approach to describing VP function. This may become an important tool in patient care, clinical training, and clinical research.
Institutional review board exemption was obtained at Seattle Children's Hospital, Seattle, Washington. The ASPO VPI Study Group convened at the 2009 Annual ASPO meeting to identify NE segments to be included in a teaching video. Twenty-four endoscopic examinations were reviewed for possible inclusion. Eleven segments were selected on the quality of the examination and characteristics of VP closure pattern. After the 11 segments were identified, the participating otolaryngologists independently rated each segment using the Golding-Kushner scale. The group then reconvened and rated each of the segments as a group to generate consensus ratings. Items rated included right and left lateral pharyngeal wall movement, right and left palate movement, posterior pharyngeal wall movement, and an estimate of gap size. Raters were asked to judge the presence or absence of the Passavant ridge, midline palatal notch, aberrant pulsations, and shape of VP closure.
Finally, members of ASPO were invited to attend a meeting of the ASPO VPI Study Group at that 2010 Annual ASPO meeting. Thirty-nine ASPO members participated. The 11 segments and the consensus ratings generated by the 2009 participants were presented. The items are presented graphically to demonstrate individual responses as well as the consensus ratings. All 39 participants agreed with the consensus ratings proposed, thus creating a consensus standard. Descriptive statistics were used to describe individual ratings for each characteristic of the Golding-Kushner scale used.
Using the 11 selected nasendoscopy segments, 5 otolaryngologists (M.E.G., H.A.M., S.R.P., P.J.Y., and K.C.Y.S.) rated each component of VP function using the Golding-Kushner scale. Prior to generation of the consensus ratings, these ratings appeared to have a broad distribution. For example, ratings of palatal and lateral wall motion were variable among each rater (Figure 2). There was a range of up to 7 points when compared with the consensus rating (Figure 3). This variability was also present when rating qualitative factors, such as posterior pharyngeal wall motion and gap size. Using descriptive statistics to compare individual ratings with consensus ratings, there continues to be a broad range of ratings for each component of the Golding-Kushner scale (Table 1 and Table 2).
Ratings of the presence of carotid pulsations, the Passavant ridge, and a notch on the nasal surface of the soft palate showed high agreement with the consensus ratings, with 85%, 76%, and 78% in agreement with the consensus, respectively (Table 3). Interestingly, the characterization of the shape of the gap closure demonstrated the lowest percentage agreement with consensus ratings with 54% in agreement with the consensus. This variability may be due to the more subjective nature of this rating. As a result of this variability among otolaryngologists, we showed a series of video segments to highlight the qualitative components of the Golding-Kushner scale and to propose a teaching video to facilitate these evaluations (Figure 4).
When assessing the range of VP function included in the 11 segments, descriptive statistics of the consensus ratings were evaluated. When examining right and left palatal motion and gap size, there appears to be a broad range of ratings from 2% to 10% and 5% to 80%, respectively. Specifically, when examining the gap size, consensus ratings ranged from 5% to 80%, suggesting that the selected video segments are collectively reflective of a broad range of degree of closure.
There is no objective clinical measure of VP function that closely correlates with perceptual speech assessment. It is important to understand the reliability of the clinical tools commonly used in the evaluation of patients with VPI. Because the evaluation of VP function is relevant for clinical and surgical planning as well as outcomes assessment, otolaryngologists who routinely treat patients with VPI emphasize the importance of developing a standardized reporting system.7
A 2008 multicenter study, consisting of a large group of otolaryngologists who routinely treat children with VPI, used this standardized reporting system to evaluate the interrater and intrarater reliability.11 Results demonstrate good intrarater reliability when using the Golding-Kushner scale for rating VP function on the basis of NE. Moreover, the reliability of rating qualitative characteristics, such as the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate, was variable.
The Golding-Kushner scale is the most detailed scale described for the assessment of the multidimensional mobility of the VP complex. The scale is appealing because it can be applied to both nasendoscopic and radiographic tests of VP function, but interrater reliability needs improvement as seen in both single10 and multi-institutional studies.11
The results of our study suggest that rating of NE evaluations of VP function was variable even among a group of experienced otolaryngologists who treat VPI. During the group discussion, participants expressed their previous misconceptions of the Golding-Kushner rating scale. The results highlight the need to develop a standardized method of describing the nasendoscopic assessment of VP function. The fact that the larger ASPO VPI Study Group in 2010 reviewed the consensus ratings without any modifications suggests that the ratings were broadly accepted.
We present ratings for 11 video segments, generated from a group of experts, that were then reviewed and agreed on by a larger group of pediatric otolaryngologists. These ratings will be used to produce an ASPO VPI video instruction tool to demonstrate application of the Golding-Kushner scale to NE examinations. This could be used as a self-administered tutorial available online.
Follow-up studies are required to demonstrate the effect of this video instruction tool on the variability of this rating system. Overall, standardized reporting of VPI is important to determine disease impact, VPI severity, and the effectiveness of therapy. Studying the effect of this video instruction tool will allow us to determine whether the Golding-Kushner scale can be a reliable rating system.
In conclusion, there was wide variability in application of the Golding-Kushner scale to nasendoscopic examination of VP function among a small group of pediatric otolaryngologists. This group proposed consensus ratings for 11 videoendoscopic segments that were reviewed and accepted by a larger group of pediatric otolaryngologists, thereby generating consensus standard ratings. Generation of the consensus standard is the first step in the development of a video instruction tool for evaluation of VP function on NE.
Correspondence: Kathleen C. Y. Sie, MD, Division of Pediatric Otolaryngology, Seattle Children's Hospital, University of Washington School of Medicine, 4800 Sandpoint Way NE, Seattle, WA 98105 (firstname.lastname@example.org).
Submitted for Publication: April 25, 2012; final revision received June 14, 2012; accepted August 12, 2012.
Author Contributions: Drs Tieu, Yoon, and Sie had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Tieu, Milczuk, Perkins, and Sie. Acquisition of data: Tieu, Gerber, Parikh, Yoon, and Sie. Analysis and interpretation of data: Tieu, Parikh, Perkins, and Sie. Drafting of the manuscript: Tieu and Parikh. Critical revision of the manuscript for important intellectual content: Tieu, Gerber, Milczuk, Parikh, Perkins, and Yoon. Statistical analysis: Sie. Administrative, technical, and material support: Tieu, Parikh, Perkins, and Sie. Study supervision: Yoon and Sie.
Financial Disclosure: None reported.
Previous Presentation: This article was presented at the American Society of Pediatric Otolaryngology Annual Meeting; April 20, 2012; San Diego, California.
Additional Contributions: Rose Mary S. Stocks, MD, provided input during generation of consensus ratings; Babette Saltzman, PhD, performed the statistical analysis and compiled descriptive statistics; and Carrie Wachob, BS, generated the figures and multimedia.