In the final model, noise was not included on the within-individuals factor. Single-headed arrows represent standardized factor loadings. Double-headed arrows represent correlations. All standardized estimates are significant at P < .001.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Goldberg RL, Piccirillo ML, Nicklaus J, et al. Evaluation of Ecological Momentary Assessment for Tinnitus Severity. JAMA Otolaryngol Head Neck Surg. 2017;143(7):700–706. doi:10.1001/jamaoto.2017.0020
Can responses to ecological momentary assessment questions about tinnitus be used to characterize a common underlying latent factor of tinnitus distress?
In this longitudinal observational study, the identified latent factor demonstrates that within-individual tinnitus bother, loudness, and stress vary together over time. In addition, tinnitus bother, feeling, and stress symptoms all vary together across individuals.
Momentary variability in tinnitus bother is strongly associated with levels of perceived stress.
Existing patient-reported outcome measures of tinnitus assess the severity and disability retrospectively, which may result in adequate reliability, but cannot capture the fluctuating and individualized nature of tinnitus. Experience sampling may provide an alternative.
To use an ecological momentary assessment (EMA) to measure tinnitus disability and associated constructs.
Design, Setting, and Participants
Forty adults with tinnitus provided self-report of their tinnitus bother using 5 questions measured by EMA, as well as standard retrospective outcome measures. In this 6-week longitudinal observational study conducted from July 15 to December 22, 2014, participants provided EMA data for 2 weeks (part 1); then after a 2-week break, they provided EMA data for an additional 2 weeks (part 2). A text message with a link to the EMA survey was sent for a total of 56 assessments during each 2-week assessment period. Ecological momentary assessment responses were evaluated using multilevel confirmatory factor analysis to assess the fluctuating nature of bothersome tinnitus across the group and within the pool of individuals over time.
Main Outcomes and Measures
Ecological momentary assessment questions measured tinnitus disability and associated constructs. Compliance in each study part was assessed based on response rates. The Tinnitus Functional Index and the Overall Global Rating of Bother Scale were assessed at the beginning and end of each 2-week assessment period to explore the effect of the frequent EMAs on the perceived level of bother from tinnitus.
Of the 40 participants in the study (10 women and 30 men; mean [SD] age, 60.0 [10.5] years), the median survey response rate was high (49 responses to 56 surveys sent [88%] for part 1 and 47 responses of 56 surveys sent [84%] for part 2). The latent factor identified by the 2-level confirmatory factor analysis models demonstrates that within-individual tinnitus bother, loudness, and stress vary together over time. In addition, tinnitus bother, feeling, and stress symptoms all vary together across individuals, which means that bother and stress covary strongly both across time and across individuals.
Conclusions and Relevance
Ecological momentary assessment evaluates the moment-to-moment perception of tinnitus and the effect of emotional and environmental factors, which suggests that it is a superior tool to measure tinnitus outcomes compared with standard retrospective self-reports. Taken together, information from emotional and environmental factors can be summarized in an underlying (latent) factor that represents a vulnerability to bothersome tinnitus and that can be used to comprehensively describe the tinnitus experience. Momentary variability in tinnitus bother is strongly associated with levels of perceived stress.
Idiopathic, nonpulsatile tinnitus is defined as the perceived sensation of sound without acoustic stimulation.1,2 Chronic tinnitus affects approximately 40 million individuals in the United States, of which 10 million are extremely bothered by their condition.1 Patients with tinnitus experience a wide range of associated symptoms and conditions, including insomnia, cognitive inefficiency, difficulty with attentional focus, stress, anxiety, concentration disturbances, and depression.3-7 At present, patients with tinnitus are treated as a homogenous group,8,9 despite literature supporting the heterogeneous nature of tinnitus and patient comorbidity.10 Neglecting to account for the complex nature of tinnitus distress represents an important limitation of the field, and further study is required to evaluate the fluctuating severity of the condition, its associated functional impairments, and the emotional consequences of tinnitus that complicate the care of patients and scientific progress.
One of the current limitations in the field is the method in which tinnitus distress is measured. Tinnitus distress is frequently measured using retrospective self-report measures, including the Tinnitus Questionnaire,11 the Tinnitus Handicap Inventory,12 and the Tinnitus Functional Index (TFI),13 which are considered to be criterion standard measures of tinnitus outcomes. Yet, these assessments are based on a static retrospective self-report14 requiring patients to recall and summarize their tinnitus experience during a period of 1 to 2 weeks. Although these measures have high reliability, they cannot capture the moment-to-moment fluctuations of tinnitus distress that likely complicate and affect successful treatment,15 resulting in low ecological validity owing to these inherent biases.16 Furthermore, previous studies have shown that these retrospective reports are prone to recall biases and errors in summarizing prior events and that they often emphasize the patient’s current state and environment,14,17 as well as the most recent and extreme events,14 resulting in a biased summary of the patient’s tinnitus distress.
Innovative methods, including an ecological momentary assessment (EMA), can provide an alternative to retrospective sampling. An EMA samples a patient’s subjective states via in-the-moment self-report questionnaires administered multiple times daily.18 In contrast to retrospective self-report questionnaires, an EMA typically focuses on the current moment, minimizing the need for patients to recall or summarize their experience and limiting the potential for retrospective biases.19 Furthermore, an EMA can be completed via smartphones or mobile devices, serving as a convenient way to collect near-instantaneous data using a commonplace medium in a participant’s natural setting.20-22 Ecological momentary assessments have been used for a relatively long time in psychological research, and EMA methods have been used to study a variety of conditions,23 including anxiety,18 mood,24-26 depression,22 eating disorders,27 and addictive disorders.28 As an alternative to retrospective sampling, an EMA could allow for a more reliable and comprehensive description of the population of patients with tinnitus.
Recently, a pilot study29 was completed using an EMA to assess levels of tinnitus bother, tinnitus loudness, overall feeling, stress, environmental noise, and current activity among 20 patients with tinnitus. The study reported that the degree of bother from tinnitus had a large range of variability even within a given day and provided novel results reflecting the fluctuations in tinnitus distress not previously captured with retrospective self-report. This study demonstrated that an EMA can improve the assessment of patients with tinnitus, providing important details about the patients’ experience of bothersome tinnitus on a day-to-day basis, which likely affects their response to treatment.
The present study replicates and expands on these preliminary findings, using an EMA to assess tinnitus in a larger sample and analyzing EMA data with advanced statistical methods, including latent variable and confirmatory-factor modeling. Primarily, we aim to explore how responses to several EMA questions could be strategically synthesized to characterize a common underlying latent factor of tinnitus distress. Latent factors represent traits or variables that cannot be measured directly, yet likely underlie and cause an individual’s response on observable indicators. A common example of a latent factor is intelligence. Although we cannot measure intelligence directly, it can be evaluated by measuring scores on observable indicators, such as math and reading comprehension. We then make the assumption that an individual’s level of intelligence causes or leads to his or her scores on these indicators of intelligence. Statistical findings that items on a survey are well characterized as arising from a latent factor thus constitute strong evidence that those items “belong together” because they fundamentally measure the same thing.
Here, we used 2-level confirmatory factor analysis (CFA), which is appropriate for EMA data. Two-level CFA was used to create a model that clustered data by participant so that associations between tinnitus EMA variables can be modeled both across participants and across time. In particular, we tested which of the EMA questions tended to vary together across time and across participants. Testing both levels is essential because examining EMA data at only 1 level conflates the tendency for items to move together over time with their tendency to vary across participants.
This study was conducted from July 15 to December 22, 2014. The study population consisted of participants between 21 and 80 years of age who had been bothered by nonpulsatile tinnitus for more than 6 months and who had access to a smartphone device with texting and internet capabilities. All participants were required to read, write, and understand English. Study flyers were posted in the adult otolaryngology and audiology clinics, were emailed to members of the American Tinnitus Association, and were sent to the offices of local otolaryngologists. In addition, the Washington University Volunteers for Health Research Participant Registry and Otolaryngology Research Participant Registry were queried to identify potential participants with tinnitus. Those who consented to participate in the study (written informed consent was sent to all individuals interested in the study that stated, “Returning of the completed survey or questionnaire will indicate your willingness to be considered for participation in the study”) agreed to respond to a test text message that would establish the route of communication for the study. Participants who did not respond to the test text message were excluded from the study. All communication with interested study participants was performed online or over the telephone. All assessments were completed online using the RedCap, version 6.4.4, survey tool (Vanderbilt University), which is a web-based application that stores digital information on a Washington University secure server. This study was approved by Washington University’s Human Research Protection Office prior to study initiation (NCT02191592).
Participants completed the baseline assessments, which included the TFI,13 a 25-item, self-report retrospective questionnaire and the Overall Global Rating of Bother Scale (GBS) from tinnitus (the scale included extremely bothered; bothered a lot; bothered more than a little but not a lot; bothered a little, but not much; and not bothered), a modification of the Clinical Global Impressions Scale30; participants who reported to be bothered a little, but not much or more by their tinnitus were invited to participate in the study. On agreement to participate in the study, a message was sent to each participant 4 times a day for 2 consecutive weeks, for a total of 56 assessments (part 1). The text messages were delivered at random times each day between the hours of 8 am and 9 pm, based on a predefined electronic schedule of delivery. Each text message contained a hyperlink to a RedCap survey assessing 2 questions about their tinnitus (“In the last five minutes, how bothered have you been by your tinnitus?” and “How loud is your tinnitus?”), 2 questions about emotional status (“How are you feeling right now?” and “How stressed do you feel right now?”), and 1 question about the loudness of sound in the environment (“How loud is the environment you are currently in?”), all reported on a range of 0 to 100. The final question asked the participant to provide a description of his or her current activity (from a list of 23 different options).
On completion of part 1, participants completed the TFI and received no messages or contact from the research team for the next 2 weeks. At the end of this 2-week hiatus from messages, the participants again received 4 messages a day for 2 consecutive weeks for a total of 56 assessments (part 2). The GBS and the TFI were assessed at the beginning and at the end of each study part.
At study end, the effect of repeated EMA smartphone messages on the participant’s perception, bother, and awareness of tinnitus was assessed. In addition, participants were asked if they thought the EMA was a good way for researchers to measure tinnitus and associated bother.
Standard descriptive statistics, using measures of central tendency and variability for the continuous variables, and frequency and relative frequency for the categorical variables, were used to describe the study population, tinnitus symptom severity, and results on all the assessments. Response rates were calculated as a proxy for each participant’s compliance in each study part. Line and spider graphs were used to describe and explore the within-individual pattern of change in response to each of the 2 tinnitus and 2 emotional questions. Averages of bother, loudness, feeling, and stress level scores were calculated as aggregate scores for the possible total of 56 assessments of each study part for each individual. Pearson correlation coefficients were used to explore the correlation among responses of different questions, and the association of the mean bother score with the TFI scores at the beginning (concurrent validity) and at the end of each study part. Repeated-measures analysis of variance was used to test for agreement in TFI scores at the beginning and end of each study part. All statistical tests were 2-sided and tested at a level of α = .05. The analyses were performed with IBM SPSS Statistics for Windows, version 20.0 (IBM Corp).
Multilevel modeling was used to test a 2-level CFA model measuring how responses to the tinnitus questions moved together over time both between (group-level) and within (individual-level) people. In other words, this model explores the association between EMA items, such that if 1 item changes from 1 time point to the next, other items are also likely to change, and if, when 1 person has a higher score on an EMA item than another person, whether the other EMA items for this individual are likely to be higher as well. Item loadings provide information about how strongly each item is associated with the underlying latent variable. A full information maximum likelihood estimation, which handles the missing data within the analysis by using all the information available to estimate the model parameters, was used to account for missing data. Variables with nonsignificant loadings were removed because they did not contribute substantially to the model. Although we planned to use the first arm of the data to develop the model and the second arm of the data to demonstrate that the model was reliable, problems with convergence led us to instead develop the final model in the second arm of the data, which was then found to be reliable in the first arm of the data. Mplus 7.0 was used for all multilevel modeling31 and we present 2 standard fit statistics, the comparative fit index32 and the root mean square error of approximation,33 both of which we judged by standard fit statistic recommendations.34 Both indices evaluate how well a model balances complete explanation of the data and parsimony, with values above 0.95 being ideal for the comparative fit index and values below 0.06 ideal for the root mean square error of approximation.34
A total of 717 individuals responded to study advertisements. From the pool of 402 individuals who completed the prescreening survey, 47 were invited to participate and, on the successful completion of the test text message, 41 individuals with tinnitus were enrolled in the study. The median response rate of surveys successfully completed in part 1 was 88% (49 responses of 56 surveys sent; range, 82%-95%), and the median response rate in part 2 was 84% (47 responses of 56 surveys sent; range, 64%-94%). One participant completed only a portion of the part 1 assessments and was excluded from the analysis.
The participants were predominantly white men (30 [75%]), with a mean (SD) age of 60.0 (10.5) years, primarily college educated (15 with a bachelor’s degree, 11 with a master’s degree, and 2 with a doctorate), and lived in 4 different United States time zones. The description of tinnitus characteristics is displayed in Table 1. Most participants reported on the GBS that they were bothered more than a little but not a lot or worse by their tinnitus (32 [80%]) and only 8 participants (20%) reported that they were bothered a little, but not much, by their tinnitus. A total of 32 participants (80%) reported hearing their predominant tinnitus sound all the time. The median loudness of the reported tinnitus on a scale of 0 to 10, with 10 being the loudest, was 5.5 (range, 2-9) at the start of the study. The median tinnitus duration was 15 years (range, 7 months to 50 years). The distribution of scores on the TFI and GBS assessed at the beginning and end of each study part is displayed in Table 2. There was no significant difference in TFI or GBS scores assessed at the 4 time points.
To explore the effect of the frequent EMAs on the perceived level of bother from tinnitus, we compared the change in GBS score between the start and end of each study part (Table 2). At the end of part 1, a total of 22 participants (55%) reported no change in bother by tinnitus, 13 (33%) reported a decrease in bother by 1 point, and 5 (12%) reported a 1-point increase in bother. At the end of part 2, 4 participants (10%) reported no change in bother by tinnitus, 26 (65%) reported a 1-point increase in bother by tinnitus, 9 (23%) reported a 2-point increase in bother, and 1 (2%) reported a 3-point increase. When asked how the assessment changed their awareness of their tinnitus, half of the participants reported that they were more aware of it or were thinking and focusing on their tinnitus more, but 31 of 39 participants (80%) identified EMAs as a good method for assessment of tinnitus by health care professionals (Table 3).
Our initial plans to develop a model based on the first arm and replicate that model in the second arm led to nonconvergence problems, which indicates a lack of fit but is otherwise not remarkable in a data set of moderate size. We thus tested a 2-level CFA model in the second arm of the data and attempted to replicate that model in the first arm of the data. The initial model tested on the second arm included all items (bother, loudness, noise, stress, and overall feeling) on both the within-individual and between-individual factors. The stress and overall feeling variables and the loudness and bother variables were also allowed to correlate on both the within-individual and between-individual levels because of overlap in item content (ie, 2 items asking about feelings and 2 asking about tinnitus). Ultimately, all correlations between loudness and bother were removed because they were nonsignificant. In addition, the noise item had no significant loading on the within-individual factor and was thus excluded from the within-individual factor in the model.
Because this model did not have an acceptable fit, we examined successive models in which weaker loadings were removed. The model converged with excellent fit when the overall feeling variable was excluded from the within-individual factor but was allowed to correlate with the between-individual factor. Such a model indicates that the overall feeling variable is merely associated with the underlying latent factor, rather than changing over time in concert with the items on that factor. However, because all items loaded on the between-individual level factor, all 5 variables appear to be affected by the same causal factor that accounts for differences between individuals in the sample. This final model had excellent fit by standard indices in both the first arm (comparative fit index, 0.99; root mean square error of approximation, 0.02) and second arm (comparative fit index, 1.00; root mean square error of approximation, 0.00). This model is displayed in the Figure.
Overall, this model suggests that the tinnitus bother, loudness, and stress variables vary together over time and are all caused by the same underlying factor that varies across time in the pool of individuals (within-individual factor). Thus, when stress increased across time, so did tinnitus loudness and bother. The feeling variable is associated with this underlying within-individual factor but is not plausibly caused by this factor. In contrast, all 5 of the tinnitus EMA items varied across individuals; in a participant who reports a higher level of stress compared with a second participant, the tinnitus bother is also likely to be higher. The same is true of all other items, with the caveat that the overall feeling item had an inverse association with the other items because higher ratings indicated more positive feeling. These results suggest that the tinnitus bother, loudness, and stress variables all vary together over time and, to a large extent, are caused by the same underlying factor. Finally, this model suggests that there is an underlying factor associated with stress that initiates change both over time in the pool of individuals and across the sample of individuals.
In this study, we found an EMA to provide novel and useful data on an individual’s experience of tinnitus that can be used to enhance assessment of tinnitus, minimizing the limitations of previous work using retrospective self-report measures. Frequent EMA sampling can characterize the variability in patients’ perceptions of their bothersome tinnitus, allowing for a contextual understanding of factors that affect tinnitus, including the effect of emotion (eg, stress) and environment (eg, level of noise). Using latent factor modeling and a 2-level confirmatory analysis, we identified a latent factor that reflects an underlying vulnerability to bothersome tinnitus. We demonstrated that the bother, loudness, and stress EMA items varied across the pool of individuals across time, suggesting that as individuals become more stressed, they may report more bothersome tinnitus. Taken together, these variables may capture and represent an underlying vulnerability to bothersome tinnitus. We believe this latent factor can be used to comprehensively describe the tinnitus experience and provide a deeper understanding of the population of patients with tinnitus.
A challenge in accurately assessing chronic tinnitus is the lack of instruments designed to capture the patient’s entire experience of tinnitus over time. Retrospective sampling simultaneously demonstrates low validity and high reliability. As mentioned in previous work,18 an EMA offers an alternative to retrospective self-report questionnaires, potentially minimizing biases introduced by summarizing and recalling tinnitus experiences over long periods of time. An EMA allows researchers and physicians to come as close as currently possible to capture the real-time experience of patients with tinnitus, accounting for variability over time and between patients. As previously demonstrated,29 an EMA distributed by personal smartphones and other mobile devices provides a convenient form of assessment, resulting in high rates of response.
This study focused on expanding and validating a previous pilot study and demonstrated that an EMA is a reliable and stable tool for use in future clinical trials. Responses in parts 1 and 2 of the study were highly correlated and not statistically significantly different from each other. In accordance with our previous research,29 this study confirmed the importance of multiple variables to the description of the tinnitus experience. We found that the responses to the tinnitus bother, tinnitus loudness, overall feeling, and stress questions vary together between individuals, with tinnitus bother, tinnitus loudness, and stress also varying across time. This finding suggests that a weighted fusion of the latter 3 variables can contribute to accurately quantifying the tinnitus experience. The identification of a holistic model of bothersome tinnitus, through the use of 2-level CFA analyses, represents an important advancement in the tinnitus literature because it allows us to comprehensively describe the EMA responses and, ultimately, the tinnitus experience of each individual over time.
The limitations of this study include the study population, which consisted mostly of white, educated men who were either employed or were retired and owned smartphones. The results from this study may not therefore be generalizable to a broader population. Other limitations include the possibility that repeatedly asking patients to respond to the EMA questions could affect the perception of tinnitus, although in a previous study,35 patients did not report that EMA protocol increased distress associated with their tinnitus. Finally, for many patients, tinnitus disruption of sleep can be a major problem. Because we chose not to contact participants during sleep, we were unable to assess the association between sleep disturbance and tinnitus bother.
We believe that an EMA of the moment-to-moment perception of tinnitus and the association with emotional and environmental factors, especially stress, serves as a superior tool to measure the tinnitus experience compared with standard retrospective self-report questionnaires. Many individuals demonstrate large moment-to-moment variability in tinnitus bother and loudness and a close association between bother and perceived stress. The close association between tinnitus bother and stress, as identified in the 2-level CFA model, supports the inclusion of treatments focusing on stress relief, such as mindfulness-based stress reduction. Use of the 2-level CFA helps to develop a more thorough model of risk factors that may constitute an underlying vulnerability to bothersome tinnitus. Furthermore, latent models could be used to compare patient responses before and after intervention targeting stress. If the intervention is successful, we may find that there are meaningful changes between the risk factors that denote risk for bothersome tinnitus. Further use of structural equation modeling may help advance efforts to define the sensitivity or responsiveness of risk factors to clinical change and can advance our understanding of how bothersome tinnitus arises in a heterogeneous patient population.
Corresponding Author: Jay F. Piccirillo, MD, Department of Otolaryngology–Head and Neck Surgery, Washington University School of Medicine in St Louis, 660 S Euclid Ave, PO Box 8115, St Louis, MO 63110 (firstname.lastname@example.org).
Accepted for Publication: December 24, 2016.
Published Online: April 27, 2017. doi:10.1001/jamaoto.2017.0020
Author Contributions: Dr Piccirillo had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Goldberg, Nicklaus, Kallogjeri, J.F. Piccirillo.
Acquisition, analysis, or interpretation of data: Goldberg, M.L. Piccirillo, Nicklaus, Skillington, Lenze, Rodebaugh, Kallogjeri.
Drafting of the manuscript: Goldberg, Lenze, Rodebaugh, Kallogjeri.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Goldberg, M.L. Piccirillo, Skillington, Rodebaugh, Kallogjeri, J.F. Piccirillo.
Obtained funding: J.F. Piccirillo.
Administrative, technical, or material support: Nicklaus.
Study supervision: Nicklaus, J.F. Piccirillo.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Funding/Support: This study was supported by the Washington University Institute of Clinical and Translational Sciences grants UL1 TR000448 and TL1 TR000449 from the National Center for Advancing Translational Sciences, National Institutes of Health and grant T32DC000022 from the National Institute on Deafness and Other Communication Disorders.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr Piccirillo is the Editor of JAMA Otolaryngology–Head & Neck Surgery, but he was not involved in any of the decisions regarding review of the manuscript or its acceptance.