Correlation of Odor Stick Identification Test for Japanese (OSIT-J) test scores with the Connecticut Chemosensory Clinical Research Center (CCCRC) composite (A), identification (B), and detection threshold (C) scores. The OSIT-J test scores correlated strongly with CCCRC composite Spearman rank correlation (rs = 0.80) and identification scores (rs = 0.86).
Correlation between patients' self-reports of olfactory function and olfactory test scores. Both the Odor Stick Identification Test for Japanese (OSIT-J) (A) and Connecticut Chemosensory Clinical Research Center (CCCRC) (B) test scores show strong correlations with patients' self-reports of olfactory function levels.
Kobayashi M, Reiter ER, DiNardo LJ, Costanzo RM. A New Clinical Olfactory Function TestCross-Cultural Influence. Arch Otolaryngol Head Neck Surg. 2007;133(4):331-336. doi:10.1001/archotol.133.4.331
To investigate whether a new clinical olfactory test, the Odor Stick Identification Test for Japanese (OSIT-J), can be used to assess olfactory function cross-culturally in a US patient population.
Cross-sectional prospective study.
A university medical center otolaryngology clinic.
Fifty US patients presenting with complaints of olfactory dysfunction from December 2004 to January 2006.
Olfactory testing and patient interview.
Main Outcome Measures
Comparison of test results obtained with the OSIT-J, the Connecticut Chemosensory Clinical Research Center (CCCRC) olfactory function test, and patients' self-reported level of olfactory function. Patients' opinions regarding the 2 test methods were also recorded.
The mean ± SD time required to administer the OSIT-J (8 ± 1 minutes) was shorter than that required for the standard CCCRC test (21 ± 6 minutes). Significant Spearman rank correlations were found between the OSIT-J and CCCRC test scores (rs = 0.80, P<.001, n = 50), and patients' self-reported level of olfactory function (rs = 0.73, P<.001, n = 50). Although 3 of the 13 odors used in the OSIT-J were not familiar to US subjects, patients reported that the OSIT-J was easier, more interesting, and the odors used more pleasant than the CCCRC test.
Olfactory function tests developed in different countries should be evaluated to determine if a cross-cultural bias exists among test odorants. Although a cultural bias was detected for a few odorants, this study demonstrates that a modified version of the OSIT-J can be used to assess olfactory function in US patients.
In smell and taste clinics, objective assessment of patients' olfactory function is critical for the diagnosis and treatment of olfactory dysfunction. For both clinical and research testing, validity and reliability of results are of paramount importance. However, practical concerns such as simplicity and brevity of administration, cost, and patient preference must be weighed against the available resources and testing needs of the test center. As such, numerous olfactory tests that take these issues into account have been developed at smell and taste centers throughout the world. The Connecticut Chemosensory Clinical Research Center (CCCRC) test1 and University of Pennsylvania Smell Identification Test (UPSIT)2 are frequently employed in the United States. Sniffin’ Sticks (Burghart, Wedel, Germany) are recognized by the German Society of Otorhinolaryngology as a valid test of olfactory function and are used in European countries.3,4 The standard olfactory test used in Japan is T&T olfactometry.5 Each of these tests has its own advantages and is intended to suit the needs of the smell and taste center where the test was developed. In some cases, odorant selection may be specific to the cultural demographics of the patients evaluated in the center.
The independent development of olfactory tests at multiple smell and taste centers worldwide has hindered widespread recognition of any one testing technique as the universal gold standard. However, it remains important to be able to translate olfactory test results between centers using disparate test methods to allow for exchange of research data and longitudinal tracking of patients' olfactory function. In the absence of a single universally accepted test, direct quantitative comparison of test techniques and, in some cases, modification of existing tests allow administration to patients from diverse cultural backgrounds.6 Recently, a new olfactory test, the Odor Stick Identification Test for Japanese (OSIT-J), was developed in Japan.7,8 This test is simple, can be administered quickly, and has been validated for olfactory testing of Japanese patients with olfactory dysfunction.9,10 We have previously shown6 that the OSIT-J can be administered to US subjects with a normal sense of smell, despite the unfamiliarity of these subjects with some of the odors used in the test. However, cross-cultural clinical applicability of this test in evaluating patients with olfactory dysfunction has not been demonstrated. In this study, we investigated the suitability of the OSIT-J for clinical use in testing US patients with olfactory dysfunction.
A cross-sectional study was performed with 50 consecutive patients who presented to the Smell and Taste Clinic of Virginia Commonwealth University, Richmond, with complaints of olfactory dysfunction. None of the patients declined participation in the study. Informed consent was obtained from all subjects prior to study participation. The protocol for this study was approved by the Virginia Commonwealth University Office of Research Subjects Protection.
The OSIT-J is composed of 13 different odorants familiar to the Japanese population.8 These odorants are described as condensed milk, cooking gas, curry, hinoki (Japanese cypress wood), India ink, Japanese orange, menthol, perfume, putrid smell, roasted garlic, rose, sweaty smelling clothes/natto (fermented soybeans), and wood. Test odorants are microencapsulated in a melamine resin and contained within an odorless solid cream that is dispensed in a lipstick container. The cream is applied to 1 side of a 5 × 10-cm strip of paraffin paper within a circle 2 cm in diameter. The paper strip is folded in two and rubbed together to release the odorant. Subjects receive the paper, open it in front of both nostrils, and sniff. For each odorant, subjects are presented with a card showing 4 odor names and associated pictures and are asked to select the odor presented. For 1 of the odor items, both of the answers “sweaty smelling clothes” and “fermented soybeans” are considered correct answers. If the subject cannot select 1 of the 4 odor choices, they must then respond by selecting 1 of 2 alternative answers: “detectable but not recognizable” or “no smell detected.” The total number of correct answers for the 13 odorants presented, expressed as a percentage, is the OSIT-J score.
The components and technique for administration of the CCCRC test have been described in detail elsewhere.1 The CCCRC test kit consists of odor detection and identification tests. Detection threshold is measured using 9 serial dilutions of butanol in nanopure-deionized water. Each concentration is presented along with a water control in a double-blind, forced-choice paradigm. Threshold is defined as the dilution at which the butanol bottle is correctly identified in 4 consecutive trials. If the water bottle is incorrectly selected in less than 4 trials, the next higher concentration step is tested in a similar fashion (standard CCCRC test method). For 11 of the 50 patients who presented to our clinic for testing for litigation cases, we completed the 4 consecutive trials for each dilution step until threshold was defined, even if the patients selected a water bottle in less than 4 consecutive trials (extended trial CCCRC test method).
The CCCRC identification test is composed of 7 olfactory stimuli (baby powder, chocolate, cinnamon, coffee, mothballs, peanut butter, and soap). Three stimuli (ammonia, Vicks VapoRub [Procter & Gamble, Cincinnati, Ohio], and wintergreen) are also presented to test trigeminal nerve nasal sensation but are not included in calculating the olfactory function test score. Ten jars, each containing 1 of the 7 odor stimuli or 1 of the 3 trigeminal stimuli, are presented, and the subject is asked to select the stimulus name from a list of odors. The number of olfactory stimuli correctly identified determines the identification score (Table 1). The composite CCCRC test score (a maximum of 100 points) is calculated by adding the threshold score (a maximum of 50 points) and identification score (a maximum of 50 points).
In the CCCRC test method, each nostril is tested and scored separately.1 However, in this study, to appropriately compare the data with the bilateral sampling method used in the OSIT-J (odor is sampled through both nostrils), we used a modified scoring method for the CCCRC test. First, we tested each nostril separately. For detection threshold, we selected the nostril with the best score. For the identification test, if patients identified a test odorant correctly with either the left or right nostril, they were given credit for the odorant. The identification score was calculated using the total number of points. Finally, the sum of the detection threshold and identification scores was determined to obtain a composite CCCRC test score. The composite CCCRC score is assigned to 1 of 5 diagnostic categories: normal (100-90 points); mild (80-70), moderate (60-50), or severe hyposmia (40-20); and anosmia (10-0).
Prior to testing, patients were required to describe the level of their olfactory function using 1 of 5 categories (normal, mild hyposmia, moderate hyposmia, severe hyposmia, or anosmia). For each subject, olfactory testing was performed by 1 of 2 of us (M.K. or R.M.C.), both of whom have extensive training and experience with the OSIT-J and CCCRC test methods. Each patient was tested using both the OSIT-J and CCCRC test, with patients randomized as to which test was administered first. The order in which test odorants were presented was also randomized in both the OSIT-J and CCCRC identification tests. The time required to administer each test was recorded. After testing was completed, each patient was asked whether the odorants used for each test item in the OSIT-J were either familiar or unfamiliar. Patients were also asked for their opinions of each test, specifically whether they found each test (1) easy or difficult, (2) short or long in duration, or (3) interesting or boring, and (4) whether the odors used were pleasant or unpleasant.
All numerical data are expressed as mean ± SD. The Spearman rank correlation coefficient was used to assess correlations among patients' test scores and self-reports of olfactory function. The χ2 test for independence was used to test for differences in patients' opinions regarding the olfactory tests. The Wilcoxon signed rank test was used to determine differences in average time between the OSIT-J and CCCRC tests. Differences were regarded as significant at P<.05 (2-tailed test).
Study participants were 23 men and 27 women with a mean ± SD age of 55 ± 16 years (range, 19-82 years). A total of 39 individuals were white; 10, black; and 1, Native American. Etiologies of olfactory loss, as determined by clinical evaluation, chemosensory testing, and review of imaging (when available) were head trauma (12 patients), upper respiratory tract infection (9), chronic rhinosinusitis (5), adverse effects of medication (2), sinonasal irradiation (2), congenital abnormality (1), sarcoidosis (1), psychogenic dysfunction (1), aging (1), and unknown (16). No patients had known cancer at the time of study participation, although 2 patients had previously undergone surgery and radiotherapy for maxillary sinus cancers.
Table 2 shows the US patients' reported familiarity with odors used in the OSIT-J. Eleven of the odors were familiar to at least 80% of subjects; however, over 50% reported unfamiliarity with the smell of fermented soybeans, India ink, or Japanese cypress wood.
Significant Spearman rank correlations (rs = 0.80, P<.001) were found between scores from the standard 13-item OSIT-J test and composite scores from the CCCRC test (Figure 1A). The correlation between the OSIT-J score and the CCCRC identification test score was stronger (rs = 0.86, P<.001; Figure 1B) than that between the OSIT-J score and CCCRC threshold test score (rs = 0.74, P<.001; Figure 1C). Both the OSIT-J score and the CCCRC composite score correlated strongly with patients' self-reported levels of olfactory function (rs = 0.73, P<.001 and rs = 0.74, P<.001, respectively) (Figure 2).
Given US subjects' reported lack of familiarity with some of the odorants in the OSIT-J, we retrospectively analyzed the impact of omitting selected odor items from the OSIT-J. “Modified” OSIT-J scores were calculated first using only the 11 odors that were reported in the present study as familiar to more than 80% of subjects and then using only the 8 odors that were correctly identified by more than 80% of US subjects with a normal sense of smell in our previous study.6 When compared with the CCCRC test scores, the test scores for each of the 3 versions of the OSIT-J test (standard 13-item, modified-11 item, and modified 8-item versions) all showed significant correlations (Table 3).
Most patients reported that both the OSIT-J and CCCRC tests were easy, short in duration, and interesting and the odorants used pleasant (Table 4). In addition, the number of patients who reported that the OSIT-J was easy, short in duration, and interesting was significantly more than that reported for the CCCRC test. The mean ± SD time to administer the OSIT-J (8 ± 1 minutes; n = 50) was shorter than for both the standard CCCRC test (21 ± 6 minutes; n = 39) and extended trial CCCRC tests (26 ± 5 minutes; n = 11). Neither patients' opinions about these tests nor the measured test times were affected by order of test administration (ie, OSIT-J first compared with CCCRC first). In addition, there did not seem to be a relationship between patient demographics (age, sex, race, or etiology of olfactory loss) and opinions regarding the 2 tests.
The OSIT-J was developed in Japan to provide a method of testing odor identification and to supplement existing threshold testing techniques, such as T&T olfactometry.7,8 However, because other widely used tests of odor identification have proven effective in assessing the overall level of olfactory function,2- 4 we sought to apply the OSIT-J to the assessment of olfactory function in a cohort of US patients with olfactory complaints. A previous study6 shows that the OSIT-J, although developed specifically for use with Japanese subjects, is applicable to US subjects with a normal sense of smell. However, some of the 13 test odorants used in the OSIT-J were unfamiliar to US subjects and were correctly identified by less than 80% of the subjects. These data raised questions about the cultural specificity of the test and thus its usefulness in testing US or other non-Japanese patients with olfactory dysfunction. The present study revealed that in US patients with olfactory complaints, results from the OSIT-J correlated well with patient-reported level of olfactory function, as well as with results obtained with the CCCRC test, which has been widely used in US smell and taste clinics. Results from the OSIT-J showed strong correlations with the composite, detection threshold, and odor identification scores obtained with the CCCRC test. Presumably, because the OSIT-J tests only suprathreshold odor identification, correlation was, however, the highest with the CCCRC odor identification score.
As noted previously,6 in subjects with a normal sense of smell, some odors used in the OSIT-J were unfamiliar to US patients. In the present study, over 50% of US patients reported that they were unfamiliar with the smells of fermented soybean, India ink, and Japanese cypress. Smaller percentages of subjects reported they were unfamiliar with the smells of condensed milk, curry, and cooking gas. These results are likely owing to differences in cultural experiences between US and Japanese populations. Our previous study6 demonstrated that the average OSIT-J test score from a US subject with a normal sense of smell was 77%, although the average Japanese score was 94%. This could explain why the maximum OSIT-J scores recorded in the current study were approximately 80%, even for patients whose CCCRC test scores were at or near 100% (Figure 1). However, on recalculating OSIT-J scores based only on the test odorants that were familiar to both US and Japanese subjects, we found that the correlations between the modified OSIT-J scores and CCCRC test scores or patients' self-reported levels of olfactory dysfunction were similar to those associated with the standard OSIT-J scores. Although this result suggests that the OSIT-J can be used for assessment of olfactory function in US patients even if culture-specific odor items are not removed, a ceiling effect clearly exists. That is, US patients with a normal sense of smell may achieve scores on the OSIT-J comparable to those of Japanese subjects with mild hyposmia. This may complicate comparison of test data from patients with different cultural backgrounds.
In selecting a test of olfactory function to be used in a clinical setting, the practicalities of test administration must be considered. First, patient tolerance and acceptance of the test administration should be evaluated. Our test population reported the OSIT-J to be easier, shorter in duration, and more interesting and the odors used more pleasant than the CCCRC test. In addition, the average time required for administration of the OSIT-J was shorter than that for the CCCRC test. Some of this difference can be attributed to the fact that the OSIT-J as devised is administered to both nasal passages simultaneously, whereas the CCCRC test is administered to each side independently. However, the time required for the bilateral OSIT-J (approximately 8 minutes) is about the same as that required to administer the standard CCCRC test to 1 nostril (1 nostril, 10.5 minutes; 2 nostrils, 21 minutes). In contrast to the CCCRC test, which tests both odor identification and threshold, the OSIT-J is a test of odor identification alone. Nevertheless, the OSIT-J test scores strongly correlated with the CCCRC composite test scores. These findings help to validate the OSIT-J as effective and practical for use in US clinics. The UPSIT, used widely in the United States, also tests odor identification alone.2 Both the UPSIT and OSIT-J use microencapsulated odorants. The paper-based UPSIT lends itself well to patient self-administration and can easily be distributed through the mail to allow testing of large patient populations. The OSIT-J is best administered by a trained tester, although the compact testing kit allows for low-cost testing of up to 250 subjects and has a long shelf life.6,8,11
One unique feature of the OSIT-J is its method of odor selection. In some smell identification tests, a forced-choice paradigm is used. Patients must select from 1 correct answer and 3 distractors.2- 4 In this paradigm, for example, patients with anosmia are likely to select the correct choice by chance, with a probability of 1 in 4 receiving a score of 25% even though they cannot detect any odor. This standard forced-choice technique is helpful in detecting malingerers in litigation cases in which the subject can generate a score as low as 0%.2 On one hand, with the standard forced-choice method it is difficult to determine if a score of 25% represents anosmia or severe hyposmia. Furthermore, the standard forced-choice constraint may be stressful for patients with anosmia because they must select 1 of the odors even though they have no smell sensation. On the other hand, some smell identification tests allow for additional choices.1,6,8 The OSIT-J, for example, includes 2 optional selections, “detectable but not recognizable” and “no smell detected,” along with 4 odor selections. Although this paradigm is not effective in detecting malingerers, it may be better at distinguishing severe hyposmia from anosmia and decreasing patients' stress. In addition, the sensitivity of test results using this method may be higher than that of test results using the standard forced-choice method. In the OSIT-J, the correct choice is selected only by patients who can identify the odorant, whereas in the forced-choice method, the correct choice can be made by patients who can identify the odor as well as by those who cannot but select it by chance.
Although there are universally accepted standard tests used throughout the world for sensory systems such as audition and vision, this is not the case for olfaction. The development of a single gold standard olfactory function test would be ideal for comparing clinical results obtained from different smell and taste centers around the world. However, this may be difficult, if not impossible, owing to the cultural differences that exist for odors and fragrances used in different countries. In the absence of a gold standard test for olfactory function, identifying cultural differences and adjusting odorants used in existing smell tests such as the OSIT-J may provide an alternative approach that would permit comparison of clinical data from different smell and taste centers. Results from the present study demonstrate that the OSIT-J is an easily administered and effective test of olfactory function for US patients. Replacement of the culturally specific odorants identified in this study may enhance the future applicability of the OSIT-J for use in US and other patient populations. This study underscores the need to validate olfactory function tests prior to use in patient populations who are culturally distinct from those used in test development.
Correspondence: Masayoshi Kobayashi, MD, PhD, Department of Otorhinolaryngology–Head and Neck Surgery, Mie University, Graduate School of Medicine, 2-174 Edobashi, Tsu, Mie 514-8507, Japan (firstname.lastname@example.org).
Submitted for Publication: May 25, 2006; final revision received November 8, 2006; accepted December 11, 2006.
Author Contributions: Drs Kobayashi and Costanzo had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Kobayashi, Reiter, DiNardo, and Costanzo. Acquisition of data: Kobayashi, Reiter, and Costanzo. Analysis and interpretation of data: Kobayashi, Reiter, DiNardo, and Costanzo. Drafting of the manuscript: Kobayashi, Reiter, and Costanzo. Critical revision of the manuscript for important intellectual content: Kobayashi, Reiter, DiNardo, and Costanzo. Statistical analysis: Kobayashi. Administrative, technical, and material support: Kobayashi and Reiter. Study supervision: DiNardo and Costanzo.
Financial Disclosure: None reported.
Previous Presentations: This study was presented in part at the Annual Meeting of the American Academy of Otolaryngology–Head and Neck Surgery Foundation (AAO-HNSF); September 25-28, 2005; Los Angeles, Calif; and the 28th Annual Meeting of Association for Chemoreception Sciences (AChemS); April 27, 2006; Sarasota, Fla.
Acknowledgment: We thank Sachiko Saito, PhD, of the National Institute of Advanced Industrial Science and Technology and the Saito Sachiko Taste and Smell Institute and Tatsu Kobayakawa, PhD, of the National Institute of Advanced Industrial Science and Technology for their generous gift of the OSIT-J test kit and technical advice regarding the administration of the test odorants.