Flowchart showing eligible patients and reasons for not participating in study.
Receiver operating characteristics (ROC) curve for age with area under the curve (0.54), standard error (0.098), and optimal threshold for sensitivity and specificity (45% and 82%, respectively).
Receiver operating characteristics (ROC) curve for Sino-Nasal Outcome Test (SNOT-22) with area under the curve (0.68), standard error (0.091), and optimal threshold for sensitivity and specificity (75% and 59%, respectively).
Receiver operating characteristics (ROC) curve for University of Pennsylvania Smell Identification Test (UPSIT) with area under the curve (0.68), standard error (0.089), and optimal threshold for sensitivity and specificity (25% and 94%, respectively).
Thamboo A, Santos RCD, Naidoo L, Rahmanian R, Chilvers MA, Chadha NK. Use of the SNOT-22 and UPSIT to Appropriately Select Pediatric Patients With Cystic Fibrosis Who Should Be Referred to an OtolaryngologistCross-sectional Study. JAMA Otolaryngol Head Neck Surg. 2014;140(10):934-939. doi:10.1001/jamaoto.2014.1650
Sinonasal disease and, specifically, nasal polyps, occur frequently in children with cystic fibrosis (CF). As survival rates have improved, it has become imperative that otolaryngologists become involved in the care of patients with CF to provide appropriate medical and surgical interventions for sinonasal disease. Despite significant variability in the subjective reporting of clinical symptoms, previous work has suggested there may be a relationship between clinical indicators and sinonasal disease in this population.
To determine whether the 22-item Sino-Nasal Outcome Test (SNOT-22), the University of Pennsylvania Smell Identification Test (UPSIT), and other measures of sinonasal disease could be used to predict the presence of subclinical nasal polyps in children with CF.
Design, Setting, and Participants
This was a cross-sectional study performed from May 2012 through April 2013 at a cystic fibrosis clinic at BC Children’s Hospital in Vancouver, British Columbia, Canada. There were 72 eligible children with CF for this study (with a confirmed diagnosis of CF based on genetic testing; their ages ranged from 6 to 18 years, and they were not actively being treated by an otolaryngologist). Thirty-seven of these patients (23 males, 14 females) consented to participate in this study. Twenty-three declined participation, and 12 could not be contacted.
Main Outcomes and Measures
Potential clinical predictors for the presence of subclinical nasal polyps were determined a priori. All 37 recruited participants completed a full study assessment. Nasal endoscopy (the gold standard) was performed to determine the presence of nasal polyps. Potential predictors that were assessed included age, sex, genotype, pancreatic function, SNOT-22 and UPSIT scores, oral culture swab result, and severity of forced expiratory volume in 1 second (FEV1).
A SNOT-22 score of greater than 11 was the only statistically significant predictor of nasal polyps (P = .04). The positive predictive value was 68.1%, the negative predictive value was 66.7%, and the positive likelihood ratio was 1.82.
Conclusions and Relevance
Given that the SNOT-22 is easy to administer and inexpensive, this sinus disease–specific questionnaire seems to be an appropriate tool for routine use by respirologists when assessing patients with CF to help predict subclinical nasal polyps.
Sinonasal disease is developed to some degree in nearly all children with cystic fibrosis (CF).1 Patients with CF have a defect in chloride ion transport owing to an impaired CF transmembrane regulator, which plays a key role in hydrating the mucus.2 Viscous mucus results in mechanical obstruction of the sinus ostia, predisposing the sinuses to local infection and inflammation resulting in chronic rhinosinusitis (CRS) and frequently in the development of nasal polyps. As survival rates have improved, it has become imperative that otolaryngologists become involved in the care of patients with CF to provide appropriate medical and surgical interventions for nasal polyps and CRS.
Immunocompetent patients with CRS often seek medical and surgical intervention based on their symptoms. However, only 10% of patients with CF may complain of sinus-related symptoms, although this significantly underestimates the severity of their sinus disease on clinical examination.3 Theoretic explanations for patients with CF underreporting their sinus symptoms include the early onset of CRS in childhood, causing a lack of awareness of a symptom-free state; or masking of their sinus symptoms by the severity of their other medical issues.3,4 Despite limited subjective reporting of clinical symptoms, some studies5- 8 have demonstrated relationships between clinical indicators and the severity of sinus disease in these children.
There has historically been limited interdisciplinary coordination between physicians who primarily treat patients with CF and otolaryngologists at our institution, owing to the relative underreporting of symptoms. The advantage of an otolaryngologist assessing patients with CF with sinonasal symptoms is the opportunity to provide medical and surgical treatment options while correlating treatment to a complete nasal endoscopic examination. This not only improves sinonasal symptoms but can result in clinically significant improvement in lung function.9 Nonotolaryngologists who assess patients with CF are limited to an anterior rhinoscopy examination with which it is difficult to assess the presence or absence of nasal polyps unless they occupy a large proportion of the sinonasal cavity. Although providing nonotolaryngologists with the skill set to do their own endoscopy would eliminate the potential for missed diagnoses, the costs of buying, cleaning, and maintaining endoscopes is not cost-efficient for clinics that do not use them that often. Nonetheless, patients with CF may not recognize that they have nasal polyps, but it may be advantageous to treat subclinical nasal polyps in their early stages because as they grow larger, they may obstruct the natural drainage pathway of the sinuses, leading to persistent sinusitis. The gold standard for the diagnosis of nasal polyps is nasal endoscopy, which requires specialist expertise and equipment.
We hypothesized there are other clinical tests beyond anterior rhinoscopy that could be used to help physicians determine if patients with CF should be referred to an otolaryngologist for further assessment of subclinical nasal polyps. One potential test is the well-validated sinus-specific quality-of-life measure known as the Sino-Nasal Outcome Test (SNOT), which has been used to quantify severity of sinus-related symptoms. The most recently validated SNOT is the SNOT-22, which contains 22 disease-specific questions.10 In 2003, the Sinus and Nasal Quality of Life Survey (SN-5) proved to be a valid and reliable test for children when compared with other tests.11 However, a systematic review of 15 different tests of sinonasal outcome demonstrated that the SNOT-22 test was the most reliable, valid, and easy-to-use test, even when compared with the SN-5 test12; therefore, we felt that using the SNOT-22 with the guidance of a research coordinator could provide the most reliable result. Another available test is the validated sense of smell test known as University of Pennsylvania Smell Identification Test (UPSIT).13 The UPSIT was developed for patients 4 to 99 years old. These 2 tests are easy to administer. Therefore, we sought to determine whether the utility of these 2 tests, and clinical indicators of sinus disease, could be used to predict the presence of subclinical nasal polyps in patients with CF.
A cross-sectional study was conducted at the cystic fibrosis clinic at BC Children’s Hospital in Vancouver, British Columbia, Canada. This clinic provides ongoing care for a large population of children with CF in mainland British Columbia. This study was conducted with the approval of the University of British Columbia Children’s and Women’s Research Ethics Board (H10-03132). The study took place from May 2012 through April 2013. Eligible patients were those with a confirmed diagnosis of CF based on genetic testing, ages 6 to 18 years, and not actively being treated by the pediatric otolaryngology department. Eligible patients did not received compensation for participating in this study. Genetic mutation subtype and pancreatic function data were extracted from the patient medical charts. During a single clinic visit, a number of clinical data points were obtained.
With the assistance of a research coordinator, all children with CF completed 2 surveys: the SNOT-22 and the UPSIT. The same research coordinator was present throughout the study to ensure that the tests were conducted in the same manner each time. Scores for the SNOT-22 range from 0 to 100.10 The UPSIT typically has 4 booklets consisting a total of 40 multiple choice questions.13 In adults, administration time is approximately 12 minutes with dedicated focused attention on the task.14 Although the UPSIT was developed for ages ranging from 4 to 99 years old, asking children who are in a clinical setting for an average of 4 hours to complete 4 “scratch-and-sniff” booklets was not practical. A shortened version of the UPSIT was made available to our research group and consisted of 2 of the 4 booklets (booklets 1 and 3). In the interest of time and efficacy, we opted to use this shortened version of the UPSIT as validated by Whitlock et al.15 The score for the UPSIT in this modified version ranged from 0 to 20.
Once the surveys were complete, the pediatric respirologist was brought into the room to perform a throat swab, which was sent for culture, followed by the otolaryngologist, who decongested each nostril with oxymetazoline prior to flexible endoscopy. A 2.4-mm flexible endoscope was used to identify the presence of nasal polyps. The pediatric respirologist (M.A.C.) and otolaryngologists (A.T., L.N., and R.R.) were blinded to the survey results.
Consistent with standard respirology follow-up, all the children with CF underwent lung function tests at each of their visits. The forced expiratory volume in 1 second (FEV1), which is expressed as a percentage of the predicated value for sex and height of the patient, was recorded for each patient. Patients were grouped as having severe (FEV1<50), moderate (FEV1, 50-70), mild (FEV1, >70-100), and normal (FEV1>100) results.
The predictive value for the presence of nasal polyps was calculated using the clinical information obtained during the patient’s clinic visit. Assessed variables included age, sex, genotype, pancreatic function, SNOT-22 and UPSIT scores, results from the culture of the oral swab, and severity of FEV1 results. Continuous variables, such as age and SNOT-22 and UPSIT scores, were further dichotomized by establishing receiver operating characteristic (ROC) curves for each variable (GraphPad Prism 6) to provide the most optimal sensitivity and specificity at a certain threshold. Univariate logistic regression analysis was used to determine the association between nasal polyps and each clinical variable. A potential predictor was defined by evaluating the odds ratios. Results with probability values less than 0.15 were then carried forward into the multivariate logistic regression model. Prior to modeling, the variables were assessed for collinearity with Fisher exact test. In a step-wise fashion, potential predictors were placed in the model. Likelihood ratio test was used to compare between models to determine if the addition of a clinical variable was significant to the predictive model. The positive and negative predictive values, as well as the positive likelihood ratio of the final model, were calculated.
There were 72 children with CF eligible for participation in this study. Thirty-seven of these patients (23 males and 14 females) consented to participate (Figure 1). The median (SD) age was 12 (3.01) years. Polyps were present in 20 patients with CF (54%).
The thresholds used for age and SNOT-22 and UPSIT scores were established using ROC curves (Figures 2, 3, and 4). The ROC curves are usually used to measure the accuracy of a test. Accuracy is measured by the area under the ROC curve. An area of 1 represents a perfect test, and an area of 0.5 or less represents a worthless test. The areas under the curve for age and SNOT-22 and UPSIT scores were 0.54, 0.68, and 0.68, respectively. Age would be considered a poor test marker, while SNOT-22 and UPSIT scores based on the ROC curve would be considered average test markers. The ROC curve was used in this study to establish optimal thresholds for age and SNOT-22 and UPSIT scores so they could be dichotomized to provide the best sensitivity and specificity for each marker. For age, the threshold was age older than 13 years, which gave a sensitivity of 45% and a specificity of 82%. For the SNOT-22, the threshold was a score higher than 11, which gave a sensitivity of 75% and specificity of 59%. For the UPSIT, the threshold was a score lower than 12, with a sensitivity of 25% and a specificity of 94%. Univariate logistic regression analyses of the clinical variables mentioned herein for the presence of nasal polyps were calculated (Table). Potential clinical variables as indicators of nasal polyps included a SNOT-22 score greater than 11 and an UPSIT score less than 12 (P < .15). Collinearity was assessed between these 2 variables and showed no significant correlation (P = .37). Consequently, both variables were carried forward into the model.
After the completion of a stepwise multivariate logistic regression analysis, a SNOT-22 score greater than 11 proved to be the only statistically significant individual predictor of nasal polyps (P = .04). The positive predictive value was 68.1%, the negative predictive value was 66.7%, and the positive likelihood ratio was 1.82. If the SNOT-22 and the UPSIT were performed by sequential testing, the sensitivity and specificity of combining the tests would be 18.5% and 97.8%, respectively. The positive predictive value of the test increased to 90.9% with a positive likelihood ratio of 8.52, but the negative predictive value decreased to 50.6%. If the SNOT-22 and the UPSIT were performed simultaneously, the sensitivity and specificity of combining the tests were 79.6% and 55.4%, respectively. The positive predictive value of combining test did not change (68.3%), but there was a slight increase in the negative predictive value (74.3% compared with 66.7%).
With life expectancy of individuals with CF improving,16 there is widespread interest in improving their quality of life. The reported prevalence of sinus disease among the pediatric population with CF is as high as 100% with radiology.5 Nasal polyposis has been estimated to affect as many as 6% to 57% of children with CF,5,6 whereas the estimated incidence of nasal polyps in the general pediatric population is reported to be 0.1% to 0.2%.17,18 In our cohort, 54% presented with nasal polyps. With an increased understanding that lung function is connected to the upper airways,19,20 specifically the sinuses, it is believed that the overall quality of life of patients with CF could be improved with increased medical attention toward any sinus-related disease. Hence, a number of studies that have investigated possible clinical predictors that may help physicians determine if children with CF should be referred to an otolaryngologist.5- 7,21 In our institution, like many, the otolaryngologist is not a part of the medical team that routinely assesses patients with CF during their scheduled visits.
Several studies5- 8 have strived to identify clinical symptoms and factors that can be used as predictors of nasal polyposis and the severity of disease. This, however, has proven to be a challenge because not all patients with CF and with sinonasal polyposis experience clinical symptoms. A comprehensive review of the literature performed by Feuillet-Fieux et al6 explored the presence of various correlations between clinical severity of nasal polyposis and the CF genotype, frequency of Pseudomonas aeruginosa lung infection, pulmonary function, frequency of allergy, and nutritional status. Based on this review and other previous studies, patients with CF with nasal polyps were found to have a milder phenotype associated with less severe lung impairment and better nutritional status.5- 7,21 Studies5,6 have also suggested an association between the severity of nasal polyposis in patients with CF and ΔF508 homozygosity, suggesting that this genotype could be a potential risk factor for serious sinus disease.
The unique aspect of the present study was the investigation of 2 common surveys, the SNOT-22 and UPSIT, as surrogate markers of nasal polyps. The SNOT-22, which is self-administered, has been validated within an adult population.10 It contains 22 questions for which the patient gives a score out of 5 for each symptom. Despite it being a self-administered test, in our study a research coordinator administered the SNOT-22 test to each child to ensure that they understood each question. A score higher than 11 on the SNOT-22 within our population with CF was shown to be a significant predictor of the presence of nasal polyps. Using the SNOT-22 as a test for nasal polyps in pediatric patients with CF demonstrated moderate reliability with 75% sensitivity and 59% specificity. A report10 on the validation of the SNOT-22 was able to differentiate patients without CF with polyp disease who had CRS, but the threshold of a score higher than 11 could be considered low for self-reported sinus symptoms in the adult population. However, this is likely consistent with children with CF underreporting their symptoms. In addition, this study did not include patients being treated by the otolaryngology department. It is assumed that those being treated by the otolaryngology department are patients with more severe cases of polyps who are more likely to be symptomatic and with higher SNOT-22 scores leading to a potentially higher thresholds. Given that the SNOT-22 was the only significantly predictive variable for sinonasal disease in our study, it provides support that this inexpensive test, involving just the cost of photocopier paper, is potentially reliable and valid for the pediatric patients. In our experience, children were able to complete the SNOT-22 questionnaire in 5 to 8 minutes with the help of the research coordinator. To enhance the ease of use of the SNOT-22 test in the pediatric population, the test can be modified to make the test easier by using a global faces scale, with each face assigned a numeric value, allowing younger children to assign their own scores with adult assistance.
The addition of further tests is a common practice to rule in or rule out disease. In this case, the addition of the UPSIT to the model did not significantly improve the predictive model. Simultaneous testing of the SNOT-22 and UPSIT improved the sensitivity from 75% to 80% and decreased the specificity to 55% from 59%, therefore having very little effect on the positive predictive and negative predictive value of this test and no tangible utility. The UPSIT is arguably the most common smell test used in North America, but there are other smell tests available, which may have provided different results. “Sniffin’ Sticks” have been compared with the UPSIT and have been shown to have a better coefficient of correlation between age and olfactory performance. Although the initial investment in Sniffin’ Sticks can be quite costly ($1175), they are relatively inexpensive compared with the UPSIT over time given that the sticks are reusable.22 The cost of using the UPSIT scratch-and-sniff booklets is $26 per person for the 4-booklet set. We used a 2-booklet set for half the cost. Given that there were 72 eligible study candidates, it was determined that the UPSIT was the most appropriate initial investment for this study. Despite the UPSIT scores not being significant in our final model, initial univariate logistic regression showed an odds ratio of 5.33 (P = .15), which suggested that a more sensitive smell test like Sniffin’ Sticks may yield statistically significant results. Nonetheless, based on this study, a clinician should not be influenced by the UPSIT and should base his or her decision to perform endoscopy solely on the SNOT-22 score.
This study is unique compared with other studies investigating clinical variables associated with nasal polyps. By excluding patients being treated by an otolaryngologist, we captured a group of individuals who clinicians felt did not require involvement by the otolaryngology department with the aim of finding clinical variables associated with subclinical nasal polyposis. Consequently, the number of events (nasal polyps) is limited in our sample population. With 20 individuals identified as having nasal polyps, the multivariate logistic regression model can only incorporate up to 2 clinical variables. In this study, using P < .15 for the initial univariate logistic regression analysis yielded only 2 potential clinical variables; however, if the sample population had been much larger, it would have been appropriate to increase the P value threshold of potential clinical variables so more variables could be included in the model. Moreover, increasing the number of patients with subclinical nasal polyps would have improved the area under the curve of the ROC curves (Figures 2, 3, and 4), resulting in improved positive and negative predictive values of each marker. A multicenter study would be required to achieve this. This would be a worthwhile endeavor in establishing a stronger model that will allow for patients with CF with subclinical nasal polyposis to be appropriately referred to otolaryngologists.
This study showed the potential effectiveness of using a sinus-specific questionnaire in children with CF to predict the presence of subclinical nasal polyps. The questionnaire would be inexpensive and could be provided to patients on arrival at the physician’s office to be filled in while awaiting for their appointment with little or no effect to clinic flow. The addition of the UPSIT did not significantly increase the positive predictive value of the diagnosis of nasal polyps. Other clinical indicators, such as age, sex, genotype, pancreatic function, oral swab results, and severity of FEV1, also proved to be nonsignificant as predictors of subclinical nasal polyposis.
Submitted for Publication: February 7, 2014; final revision received June 23, 2014; accepted June 30, 2014.
Corresponding Author: Neil K. Chadha, MBChB(Hons), MPHe, BSc(Hons), FRCS(ORL-HNS), Division of Pediatric Otolaryngology–Head and Neck Surgery, BC Children's Hospital, K2-184, 4480 Oak St, Vancouver, BC V6H 3V4, Canada (firstname.lastname@example.org).
Published Online: August 21, 2014. doi:10.1001/jamaoto.2014.1650.
Author Contributions: Dr Thamboo had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Thamboo, Chilvers, Chadha.
Acquisition, analysis, or interpretation of data: Thamboo, Dar Santos, Naidoo, Rahmanian, Chadha.
Drafting of the manuscript: Thamboo, Dar Santos, Naidoo, Rahmanian.
Critical revision of the manuscript for important intellectual content: Thamboo, Naidoo, Chilvers, Chadha.
Statistical analysis: Thamboo.
Obtained funding: Thamboo, Chadha.
Administrative, technical, or material support: Thamboo, Dar Santos, Rahmanian, Chilvers, Chadha.
Study supervision: Dar Santos, Chilvers, Chadha.
Conflict of Interest Disclosures: None reported.
Funding/Support: External funding was provided by the Rare Diseases Foundation and a Child and Family Research Institute Innovations in Acute Care & Technology (iACT) grant.
Role of the Sponsor: The Rare Diseases Foundation and Child and Family Research Institute had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentation: This study was a podium presentation at the Annual Meeting of the American Society of Pediatric Otolaryngology; May 18, 2014; Las Vegas, Nevada.
Additional Contributions: We thank Amy Schellenberg, cystic fibrosis nurse clinician, for assistance in identifying the eligible patient population; Julie Pauwels, ENT clinical research coordinator, for her coordination; and BC Children’s Hospital pediatric cystic fibrosis clinic for accommodating our study during clinic hours. None of these individuals received compensation for their contributions.