eTable. Examples of the items and domains from the literature search considered by the neuro-otologists and audiologists to be of high importance to adult CI users
Customize your JAMA Network experience by selecting one or more topics from the list below.
McRackan TR, Velozo CA, Holcomb MA, et al. Use of Adult Patient Focus Groups to Develop the Initial Item Bank for a Cochlear Implant Quality-of-Life Instrument. JAMA Otolaryngol Head Neck Surg. 2017;143(10):975–982. doi:10.1001/jamaoto.2017.1182
What is the value of using patient focus groups to develop a quality-of-life (QOL) instrument in the adult cochlear implant population?
Focus group participants identified communication, emotion, environmental sounds, independence and work function, listening effort, social isolation and ability to socialize, and sound clarity as the central themes important in defining QOL in adult cochlear implant users. Several of these areas have not been emphasized in prior instruments used to assess QOL in this population.
This work helps develop the hierarchical QOL construct that serves as the foundation for the development of a new QOL instrument.
No instrument exists to assess quality of life (QOL) in adult cochlear implant (CI) users that has been developed and validated using accepted scientific standards.
To develop a CI-specific QOL instrument for adults in accordance with the Patient Reported Outcomes Measurement Information System (PROMIS) guidelines.
Design, Setting, and Participants
As required in the PROMIS guidelines, patient focus groups participated in creation of the initial item bank. Twenty-three adult CI users were divided into 1 of 3 focus groups stratified by word recognition ability. Three moderator-led focus groups were conducted based on grounded theory on December 3, 2016. Two reviewers independently analyzed focus group recordings and transcripts, with a third reviewer available to resolve discrepancies. All data were reviewed and reported according to the Consolidated Criteria for Reporting Qualitative Research. The setting was a tertiary referral center.
Main Outcomes and Measures
Coded focus group data.
The 23 focus group participants (10 [43%] female; mean [range] age, 68.1 [46.2-84.2] years) represented a wide range of income levels, education levels, listening modalities, CI device manufacturers, duration of CI use, and age at implantation. Data saturation was determined to be reached before the conclusion of each of the focus groups. After analysis of the transcripts, the central themes identified were communication, emotion, environmental sounds, independence and work function, listening effort, social isolation and ability to socialize, and sound clarity. Cognitive interviews were carried out on 20 adult CI patients who did not participate in the focus groups to ensure item clarity. Based on these results, the initial QOL item bank and prototype were developed.
Conclusions and Relevance
Patient focus groups drawn from the target population are the preferred method of identifying content areas and domains for developing the item bank for a CI-specific QOL instrument. Compared with previously used methods, the use of patient-centered item development for a CI-specific QOL instrument will more accurately reflect patient experience and increase our understanding of how CI use affects QOL.
Cochlear implantation is the standard of care for treatment of bilateral severe to profound sensorineural hearing loss. Cochlear implant (CI) outcomes are traditionally reported using standard word and sentence recognition metrics.1 Although these metrics are important, they do not capture the diverse listening and communication experiences of CI users. Moreover, outcomes reported using these test batteries are characterized by large and unexplained individual differences2 and often poorly correlate with CI user self-report of real-world communication abilities and CI benefits.3-7
Health-related quality-of-life (HRQOL) instruments have become increasingly important and accepted for understanding the impact of a medical intervention on a patient’s life. In fact, the Centers for Medicare & Medicaid Services has targeted QOL improvement as a primary outcome measure in their Quality Strategy Report.8 The US Department of Health and Human Services defines HRQOL as a “multi-dimensional concept that includes domains related to physical, mental, emotional, and social function.”9 Patient-reported outcomes measures (PROMs) are instruments (typically questionnaires) devised to capture a patient’s perspective about overall health or treatment. Patient-reported outcome measures are the primary metrics used to assess HRQOL.
Numerous studies have shown the positive effects of CI on patients’ HRQOL,3,7,10-14 but there is no universally accepted and validated QOL PROM that is focused on the specific benefits of CIs. Most studies have used generic HRQOL measures, such as the Short Form-36 (SF-36), Health Utility Index (HUI-3), or Glasgow Benefit Inventory, which include domains, such as mobility and bodily pain, unrelated to CIs, or hearing handicap or hearing-aid PROMs that have not been validated for CIs. Several CI-specific QOL measures have been developed, but these do not meet modern development or reporting standards.15-17
The National Institutes of Health established the Patient Reported Outcomes Measurement Information System (PROMIS) in 2004 to develop and evaluate PROMs. Since that time, rigorous and clear guidelines have been established for how PROMs should be developed and reported.18 This process includes 5 steps, with the first being a comprehensive literature search of existing measures.19,20 Focus groups are then used so that the population being studied can identify important topics that affect their lives, confirm or deny areas that have been investigated in prior measures, and identify new areas of importance that have not been previously recognized.21 Thematic analysis is then performed, which is a qualitative analysis that involves reviewing transcripts of focus group meetings and linking participants’ statements that represent common themes or domains. The consolidated criteria for reporting qualitative research (COREQ) is a 32-item checklist that has been established to standardize how focus groups are designed and how data are collected and reported.22
Using data from the comprehensive literature review and focus groups, the initial item (question) bank is developed. Next, cognitive interviews are performed in the population of interest to review the item bank. This allows patients to provide feedback regarding item clarity as described by DeWalt et al.21 The initial item bank is then ready for psychometric testing using item response theory. Item response theory is used to identify those items with the greatest ability to discriminate among high- and low-performing subjects. Additionally, factor analysis confirms that individual items are placed into the proper domain.23 Finally, validity testing is performed, which determines how well the instrument measures the outcome that it was intended to evaluate.24 To date, no CI-specific QOL instrument has been developed using these rigorous methods, which are now considered the gold standard for development of PROMs.
As noted by the Minimal Reporting Standards for Cochlear Implantation of the American Academy of Otolaryngology–Head and Neck Surgery and the 2017-2021 Strategic Plan of the National Institute on Deafness and Other Communication Disorders,25 there is a need for a universally accepted, validated QOL instrument targeted to CI users. Our team is developing such an instrument using the methods established by the National Institutes of Health’s PROMIS. By developing such an instrument, we aim to change the manner in which we evaluate CI outcomes and better understand the impact of this intervention on patients with CIs. This report includes the results from our group’s work leading up to and including the CI patient focus groups, which has led to the development of the first prototype that is ready for validation. By reporting these data, we aim to demonstrate the importance of using patient focus groups when developing PROMs in our field.
The CI-QOL instrument is being developed in accordance with the PROMIS guidelines described herein (PROMIS) and using COREQ-32.18,21
PubMed, Scopus, and OVID/Medline databases were searched using the following search terms: “cochlear implant” and “quality of life”; “hearing” and “quality of life”; and “hearing aid” and “quality of life.” Methods search was performed following the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement.26 No date range limitations were used. Studies with patients in the cohort younger than 18 years old were excluded. Generic HRQOL instruments, such as the SF-36 or HUI-3, were excluded.
From the literature search, we developed a comprehensive list of 273 items from existing validated hearing-related QOL PROMs: Hearing Handicap Inventory for Adults/Elderly (HHIE/HHIA),27,28 Abbreviated Profile of Hearing Aid Benefit (APHAB),29 Glasgow Benefit Inventory,30 Hearing Implant Sound Quality Index Questionnaire (HISQUI),17 Cochlear Implant Function Index (CIFI),16 Speech, Spatial and Qualities of Hearing Scale (SSQ),31,32 Satisfaction With Amplification in Daily Life (SADL),33,34 International Outcome Inventory for Hearing Aids (IOI-HA),35 Comprehensive Cochlear Implant Questionnaire (CCIQ),36 Attitudes Toward Loss of Hearing Questionnaire (ALHQ),37 and Nijmegen Cochlear Implant Questionnaire (NCIQ).15 Items referring directly to hearing aids were changed to reflect CI use. We then eliminated 128 items due to redundancy to yield 145 items. Next, 2 senior neurotologists and 2 CI audiologists from our institution ranked these items from 1 to 5 (least to most important) on a Likert scale based on their perception of importance to patients with CIs. Any item with a mean score of 4.0 or higher was included (yielding a total of 62 items). These items were then grouped on the basis of their domain (examples are provided in the eTable in the Supplement). Within each domain, the individual items formed the basis for items used in the focus group protocol. Prior to the focus group, these items were reviewed by the research team to ensure clarity.
As discussed herein, focus groups allow the population of interest to directly respond to what does or does not affect their health or well-being. Focus group participants were randomly selected from the Medical University of South Carolina’s registry of adult CI users who received implants during the past 15 years and agreed to be contacted for research studies. Inclusion criteria included age at least 18 years old, CI activated at least 12 months before the focus group, and use of CI on a daily basis for at least 3 weeks of the last month. Exclusion criteria included patients whose implant surgery had been performed by the moderator, patients who had had prior research or clinical contact with the moderator, patients with vision impairment who would be unable to follow live transcription, and individuals with neurological or cognitive impairment prohibiting engagement in discussion. Approval from the institutional review board of Medical University of South Carolina was received prior to formation of the focus groups, and written consent was obtained from participants prior to the focus groups. Potential participants were contacted by email to ask whether they were interested in participating in the focus group. Participants were stratified into 3 focus groups based on communication abilities with their implant as measured by word scores on the consonant–vowel nucleus–consonant (CNC) test in quiet presented at 60 dB sound pressure level (group 1, 0%-34%; group 2, 36%-66%; group 3, 68%-100%). Reasons for stratification were 2-fold. First, we hypothesized that communication would be a substantial component of patient QOL and discussed at length during the focus groups. We wanted to ensure that individuals with a broad range of communication abilities were able to vocalize their experiences. Second, we were interested in seeing how individuals’ word recognition affected domains other than communication. In accordance with PROMIS guidelines,18 focus group sample size is not determined a priori, but rather based on data saturation—when no further unique themes are introduced by the focus group participants. However, PROMIS guidelines suggest the use of at least 3 focus groups of 6 to 12 participants each, which can be adjusted based on the needs of the participants. Due to the communication difficulties of the group with the lowest word recognition scores, a smaller group of 4 was used to facilitate group discussion. We initially scheduled 3 focus groups with plans to have more if needed. Additional focus groups were not needed, however, because data saturation was reached for each session within the allotted time. Prior to the focus group, participants completed an online demographic characteristics questionnaire and a visual analog scale (VAS) asking them “What impact has your cochlear implant had on your quality of life?” Participant’s responses were reported on a scale from 0 (no impact) to 10 (most impact). A 1-way analysis of variance was used followed by a Tukey post hoc comparison test to compare VAS scores among the focus groups.
The development, execution, and analysis of the focus group protocol was based on grounded theory. Grounded theory involves the collection and analysis of patient data to develop a theoretical framework that describes an individual’s experience.38 A protocol with open-ended questions was developed by the authors and used to facilitate discussion during the focus group. Questions narrowed to more specific topics as the focus group progressed. However, to avoid bias, no specific topic was addressed until it had been brought up by a focus group participant during discussion of a more general question. For example, the question “How has your cochlear implant affected your ability to socialize?” was not asked until a participant independently mentioned this topic when answering a previous question.
A neurotologist who was trained in community engagement research and routinely treats patients with CIs served as the moderator for the focus groups. The 3 focus groups were held in the Medical University of South Carolina’s otolaryngology–head and neck surgery conference room. Before any questions were asked, the participants were given a verbal summary of the moderator’s research goals and purpose for performing the focus group. Each focus group session was terminated when data saturation was reached. The moderator kept field notes throughout the focus group to keep track of this. To ensure comprehension of the discussion by all participants, professional Communication Access Real-time Translation (CART) services were used; this transcription was used later for thematic coding. Each focus group discussion was also audiorecorded as a backup, in the event of transcription errors. The moderator, CART transcriptionist, and the participants were the only individuals in the room during focus group discussions. Race and sex, as defined by the participants, were used in reporting the data because these are important descriptors of the focus group population.
After instructions were given, 2 individuals independently manually coded the transcriptions of the 3 focus groups. After reviewing the transcripts, all coders verified that data saturation was reached before the conclusion of each focus group. Each coder developed a coding tree by identifying minor themes that were associated with overall central themes. These themes were independently derived by the coders based on the data and not selected in advance. After the themes were identified, the 2 coders and the moderator met to discuss any discrepancies. Based on this discussion, the final themes generated from the 3 focus groups were established. These themes were then emailed to the focus group participants for verification and to see whether there were any additional comments.
Fifty-seven patients were contacted for participation in the focus groups. Of these, 23 (40%) agreed to participate. Patient focus group demographic information is included in Table 1. The CI-related characteristics for focus group participants are included in Table 2. The CNC scores of focus group participants were similar to the distribution of scores in the adult CI population.39 Based on the VAS scores, group 1 reported a lower CI-related QOL than group 2 or 3 (P = .047), but QOL VAS ratings for groups 2 and 3 were equivalent (P = .68).
The central themes that described the impact of cochlear implantation on patients’ QOL were communication, emotion, environmental sounds, independence, listening effort, social isolation/socializing, and sound clarity. The following sections include further discussion of these themes. Table 3 includes quotes from focus group participants related to these themes.
Improvement in communication was identified during the focus groups as a central theme that affected QOL. Participants described a wide range of communication abilities in all environments, but especially when background noise was present. Minor themes related to communication included improvement in one-on-one conversation, improvement in group discussions, understanding speech in noise, being able to talk on the telephone, and ability to hear their own voice.
Focus group participants consistently stated that cochlear implantation improved their emotional well-being. Prior to cochlear implantation, the majority of focus group participants noted feeling depressed and anxious and having low self-esteem or self-worth. A clear overlap between the themes of emotion and social isolation was expressed. Several participants noted that they were embarrassed to engage in social interaction due to their hearing loss prior to receiving their CI. All participants agreed that their CI improved their emotional state. Minor themes included anxiety, depression, self-esteem, and confidence.
Transcript review led to the combination of 3 topics into 1 common theme of “environmental sounds”: hearing and recognizing sounds from nature (birds chirping), urban settings (car horns, traffic), and the household (microwave, vacuum). All participants commented that the ability to hear and recognize these sounds greatly improved their QOL. Many reported that by hearing these sounds, they once again felt part of society.
The ability of participants to function without the help of others after cochlear implantation was discussed as a significant way in which CI use improved QOL. Related minor themes were less dependence on relatives and loved ones and resumption or improvement in ability at work. Nine (39%) focus group participants were retired, but all participants in the work force noted that the CI was vital to them maintaining employment. Several reported fear of being terminated from work prior to implantation.
Focus group participants reported mixed degrees of effortful listening. Overall, participants in the low-performance group reported expending the most mental energy when communicating, stating that they often felt fatigued after a full day of listening. The middle- and high-performing groups reported similar fatigue, but primarily when discussing communicating in complex listening environments (eg, large crowds, background noise). Numerous participants also noted that subtle CI programming changes made by their audiologist decreased their listening effort.
The ability to interact with and feel included in groups and to attend and enjoy social functions, as well as increased confidence when socializing, made focus group participants feel that CI use greatly decreased their social isolation and improved their ability to socialize. Prior to implantation, many participants reported that they had secluded themselves from friends and family due to a decreased ability to effectively communicate and embarrassment that they were unable to follow conversations. This theme overlapped with the listening effort theme, as many participants reported that prior to implantation it had been too tiring to socialize, leading them to withdraw from such situations. This finding overlapped with the emotion theme as participants reported that increased isolation and decreased confidence in communication made them feel depressed and experience low self-esteem. Ability to listen to the radio and television were related to interacting with the world, so these items were included in the social isolation/socialization theme.
As participants became more familiar with their CI, they reported improvements in sound clarity and QOL. Additionally, numerous participants noted that subtle programming changes made by their CI audiologists led to substantial improvements in sound clarity.
The initial QOL prototype was developed from the central and minor themes identified from the focus groups. The initial item bank for the prototype includes 101 questions, many of which were derived from direct quotes from focus group participants (examples provided in Table 3). We then performed cognitive interviews with 20 patients with CIs who were in clinic for routine CI visits and did not participate in the focus groups. No items required revision based on these interviews.
The use of PROMs to assess QOL allows direct input from the affected population about how disease processes and interventions affect patients’ lives. This direct measure avoids the need to interpret clinical outcome measures, such as CNC scores, and gives patients a means to express their feelings about their outcomes using a validated tool. These instruments allow measurement of patients’ own perspectives on their ability or functional level that go beyond clinical metrics that may or may not match patient self-report. The use of such instruments is especially important when performing an intervention in which survival is not the most relevant outcome and the intervention directly affects functional abilities and patient QOL. The importance of PROMs is best highlighted by the US Food and Drug Administration requirement that they be included in all clinical trials seeking approval for an intervention.40
Because PROMs are attempting to quantify the patients’ opinions of their health or functional outcomes, the use of focus groups to better understand patient perspective is vital. The importance of these focus groups is 4-fold. First, they allow individuals affected by the topic of interest to confirm or refute the importance of the themes or domains obtained from the comprehensive literature search. Second, they permit researchers to better identify content gaps from existing PROMs. Third, researchers can gain a better understanding of how patients view their condition and the vocabulary they use to describe it. Fourth, they allow expansion of the PROMs into unique content areas that may not have been discussed without patient involvement.
To our knowledge, the instrument under development is the first CI-specific QOL instrument using focus groups to develop the item bank. In doing so, we identified several areas of importance to CI users that have not been emphasized in previous PROMs. These included, but were not limited to, increase in functional independence, sound clarity, listening effort, and sense of work function ability. In addition, we were directly able to address the issue of instrument responsiveness with the focus group participants. Several focus group participants had been involved in CI programming studies, and all had routine visits with CI audiologists where more nuanced programming changes had been made. Participants noted improvements in sound clarity, decreased listening effort/fatigue, and improvements in ignoring background sounds after these subtle programming changes, which may not have affected speech recognition abilities. Interestingly, these themes were discussed more than improvements in speech recognition (as would be indicated by increased CNC scores).
The limitations of this study are similar to those common to all qualitative research. The data obtained are limited by the responses of the focus group participants and may not be generalizable to the adult CI population. By following the strictest standards and guidelines available18,22 and recruiting patients representative of the general CI population in terms of communication ability, age, sex, all CI manufacturers, and listening modalities, this was minimized to the best of our ability. To maximize the generalizability of the final QOL instrument, the validation process will be carried out with study participants drawn from a national sample.
Using experienced clinicians, patient focus groups, and the National Institutes of Health’s PROMIS guidelines, our goal is to develop a comprehensive and responsive QOL instrument for use in research and as a routine clinical outcome measure. The patient focus groups provided additional information about the factors affecting adult CI users’ daily life that could not be gained by methods used by existing instruments. This information is being used to develop the initial QOL item bank and prototype, which are currently undergoing analysis and validation. In developing and validating this instrument, we hope to fundamentally change how we measure CI outcomes to gain a better understanding of the extent to which cochlear implantation, electric/acoustic configurations, listening modality, enhanced signal-processing strategies, and adjustments to CI programming affect CI user listening, communication, and other experiences that contribute to QOL.
Accepted for Publication: May 21, 2017.
Corresponding Author: Theodore R. McRackan, MD, Department of Otolaryngology–Head and Neck Surgery, Medical University of South Carolina, 135 Rutledge Ave, MSC 550, Office Ste 1120, Rutledge Tower, Charleston, SC 29425-5500 (email@example.com).
Published Online: August 3, 2017. doi:10.1001/jamaoto.2017.1182
Author Contributions: Dr McRackan had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: McRackan, Velozo, Camposeo, Meyer, Melvin, Dubno.
Acquisition, analysis, or interpretation of data: McRackan, Holcomb, Camposeo, Hatch, Meyer, Lambert, Melvin, Dubno.
Drafting of the manuscript: McRackan, Dubno.
Critical revision of the manuscript for important intellectual content: McRackan, Velozo, Holcomb, Camposeo, Hatch, Meyer, Lambert, Melvin, Dubno.
Statistical analysis: McRackan.
Obtained funding: McRackan.
Administrative, technical, or material support: McRackan, Holcomb, Camposeo, Hatch, Lambert, Melvin, Dubno.
Supervision: McRackan, Velozo, Meyer, Dubno.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr McRackan has a patent copyright pending for final quality-of-life instrument. Dr Holcomb is on the medical advisory board for and has received personal fees from Advanced Bionics and Cochlear Americas and has received grants from Med El Corporation. Dr Meyer is on the medical advisory board for Advanced Bionics. Dr Lambert is on the medical advisory board for Cochlear Americas. No other disclosures are reported.
Funding/Support: This research was made possible by funding from a K12 award through the South Carolina Clinical & Translational Research Institute, with an academic home at the Medical University of South Carolina, National Institutes of Health/National Center for Advancing Translational Sciences grant UL1TR001450, and a grant from the Doris Duke Charitable Foundation.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: The authors would like to thank Evan Abdullah, BS, and Michael Bauschard, MD, MS, Department of Otolaryngology–Head and Neck Surgery, Medical University of South Carolina, for their efforts in the organization of the focus groups. They received no compensation from funding sponsors.