A schematic presentation of the data-sampling periods using the 6-item quality-of-life survey, relative to the treatment of otitis media with effusion (OME).
The phenomenon of response-shift bias. Mean values and SDs are depicted at the group level, at different times during the study: baseline test, at study entry; pretest, at outpatient surgery; posttest, 6 to 8 weeks after surgery; and retrospective pretest, 6 to 8 weeks after surgery. OM-6 indicates 6-item quality-of-life survey.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Timmerman AA, Anteunis LJC, Meesters CMG. Response-Shift Bias and Parent-Reported Quality of Life in Children With Otitis Media. Arch Otolaryngol Head Neck Surg. 2003;129(9):987–991. doi:10.1001/archotol.129.9.987
To validate the 6-item quality-of-life survey (OM-6) and to investigate response-shift bias regarding children with otitis media.
Otorhinolaryngology department of a university hospital that serves the southernmost part of the Netherlands.
Seventy-seven children (age range, 12-38 months) experiencing persistent otitis media with effusion and scheduled for placement of tympanostomy tubes.
The OM-6 measures health-related quality of life in 6 domains: physical suffering, hearing loss, speech impairment, emotional distress, activity limitations, and caregiver concerns.
Parents completed the OM-6 before surgery (pretest) and 6 weeks after surgery (posttest). At the posttest, parents also completed a retrospective version of the pretest (retrospective pretest).
For most items, the test-retest reliability was good (R>0.8). The internal consistency of the OM-6 was satisfactory (α = .79). The construct validity, determined by correlating the ear-related global quality-of-life measure and the OM-6 summary score, was fair (R = −0.77, P<.01). Prospective change in quality of life on the OM-6 ranged from moderate (standardized response mean ≥0.5) to large (standardized response mean ≥0.8). Response-shift bias was present at the group level (t = −3.3, P<.01). Retrospective change was significant for hearing loss (z = −3.3, P<.05) and ear-related global quality of life (z = −3.6, P<.05).
The validity of the OM-6 has been proved in a Dutch population. The data suggest that parents underestimate the seriousness of hearing loss and overestimate the quality of life of their child before surgery, indicating a response shift. Treatment results could lead parents to realize that the situation before surgery had been worse than they thought.
IN MEDICAL PRACTICE concerning infants, a caregiver, in most cases the parent, makes decisions about seeking medical consultation on the basis of perceived symptoms of the child. This situation resembles that of other care-dependent groups, such as developmentally disabled or psychiatric patients.1
This could be applied to otitis media with effusion (OME) as well. Otitis media with effusion has a high prevalence in childhood; almost 90% of children experience at least 1 episode in the first few years of life, and up to one third have recurrent problems with OME.2-4 Although OME is mainly known as a silent disease, it has been associated with symptoms such as common cold, mouth breathing, ear infection, and hearing loss.5 Except for children detected by an ongoing hearing screening program, parental suspicion usually leads to consultation of the general practitioner and often referral to an ears, nose, and throat surgeon.6 Caregiver concern usually results from a noticed change in behavior related to a child's hearing loss, because expression is assumed to be predominantly nonverbal during the first years of life.7 The developmental rate of the young child may not meet parental expectations and gives rise to concerns about the seriousness and duration of hearing-related symptoms.
The general practitioner or ears, nose, and throat surgeon has to specify and objectify these parental concerns through further questioning, inspection of the tympanic membrane, and, if possible, estimation of hearing thresholds. The situation becomes complicated for the surgeon when there seems to be no objective indication for surgical intervention (such as grommet insertion or adenoidectomy), yet parental concern and desire for surgical intervention remain.
The basis of parental concern is generally subjective symptoms, such as hearing loss, poor appetite, irritability, and loss of energy, which may have negative consequences on the quality of life in children with OME. One attempt to objectify the presence of subjective symptoms has resulted in the development, validation, and application of the 6-item quality-of-life survey (OM-6) in the United States. The aim of this instrument, developed by Rosenfeld and colleagues,8 is to measure health-related quality of life (HRQL) in the presence of chronic OME or recurrent acute otitis media. In this context, HRQL reflects the parents' subjective perception of the current health status of the child and refers to the possible impairment in functioning and well-being of the child as a consequence of OME. In addition, a "real change" in HRQL after surgery is increasingly needed to justify any intervention for OME in the absence of objective symptoms.9,10
As an extension of the earlier OM-6 study,8 the possible effect of parental perceptions on their report about the child's HRQL is examined herein. Assessment of response shift on HRQL changes constitutes the main goal of the present study. For example, the placement of tympanostomy tubes in children with OME, often a chronic condition, creates a sudden resolution, which may elicit an adaptation in parental perception. It is hypothesized that, after treatment of OME, parents realize that the previous HRQL of the child was worse than they had recognized, as they are now able to make a comparison between the situation before and after surgery. This phenomenon has been described as scale recalibration, a variant of response-shift bias, which indicates that after intervention a change has taken place in the respondent's internal standard of measurement.11 When parents overestimate the HRQL before surgery, not taking response-shift bias into consideration may result in an underestimation of the reported treatment effects.12
This study is also an attempt to validate the OM-6 in a Dutch population of children with chronic OME. Validation of the OM-6 is a necessary requisite for the assessment of hypothesized changes in HRQL resulting from surgical intervention.
The participating children were seen in the Department of Otorhinolaryngology–Head and Neck Surgery at the University Hospital of Maastricht, which also serves a regional function in the southernmost part of the Netherlands. This population was part of an ongoing study7 concerning OME and specifically related behaviors that had been approved by the University Hospital Medical Ethics Committee. The following eligibility criteria were met: (1) age at entry between 12 and 36 months; (2) chronic or recurrent OME in both ears for at least 3 months as diagnosed by an ears, nose, and throat surgeon through otoscopy; (3) hearing loss of at least 20 dB in the better ear; (4) tympanostomy tube insertion scheduled as outpatient surgery within 2 to 4 weeks after diagnosis of OME; and (5) ability of the accompanying parent or other caregiver to read and understand the Dutch language. These criteria were comparable to those used in the first OM-6 study,8 except for our inclusion of only young children at study entry. The first OM-6 study included children aged 6 months to 12 years. Exclusion criteria consisted of (1) tympanic membrane perforation, (2) tympanostomy tubes at study entry, (3) middle ear pathologic condition other than OME, and (4) any known neurological disorder.
The parents of participating children received a letter with information about the goals and procedures of the OM-6 study, and written informed consent was obtained.
The OM-6 was translated from English into Dutch, in consultation with a native English speaker.8 The OM-6 contains 6 items representing HRQL associated with OME: (1) physical suffering, (2) hearing loss, (3) speech impairment, (4) emotional distress, (5) activity limitations, and (6) caregiver concerns. The scoring format consisted of a 7-point scale for the items, ranging from 1 (not present or no problem) to 7 (extreme problem), with a midpoint of 4 (moderate problem). A global measure, also present in the original English version, summarizing ear-related global quality of life was included as well. The global measure was scored by a 10-point visual analog scale, which runs from 0 (worst possible quality of life) to 10 (best possible quality of life), with a midpoint of 5 (halfway between worst and best).
The OM-6 data sampling relative to the treatment of OME over time is shown in Figure 1. At each OM-6 measurement, the situation at that time was assessed. Caregivers were asked to what degree the symptoms, as described in each item, had been a problem for their child. Answers were based on either individual symptoms or problems or on global effects.
During the study, the OM-6 was completed by parents without assistance. Possible bias resulting from interactions with an interviewer is eliminated in this way (Richard M. Rosenfeld, MD, written communication, March 1998). A subgroup completed the OM-6 at study entry (baseline test) after the diagnosis of OME; all parents completed a pretest on the day of surgery. At follow-up, which included otoscopy and tympanometry 6 to 8 weeks after the intervention, 2 surveys (posttest and retrospective pretest) were completed. The retrospective pretest reassessed the parental perception of the child's condition at the time of the pretest, to evaluate the presence of a response shift.
Before performing statistical tests on a group level, some calculations of individual scores were necessary to evaluate HRQL change scores. Test-retest reliability was assessed using the Pearson correlation coefficient (R) between the scores on the baseline test and pretest administered within 7 days before surgery. Internal consistency was measured by examining the statistical relations between items with similar content. Construct validity was assessed by the relationship between item or survey summary scores and the ear-related global quality of life. Responsiveness to clinical change after placement of tympanostomy tubes was measured by the standardized response mean (SRM), which is the mean change score per item divided by its SD.13
In the presence of a response shift between the pretest and the retrospective pretest scores, the latter test is a better reference point than the posttest for calculating responsiveness to change.14 Confirmation of significant differences at the group level was determined by a paired-samples t test. The Wilcoxon signed rank test was applied to indicate the direction of response shift for individual responses.14 Finally, retrospective responsiveness to clinical change was measured by means of the SRM.
The sample consisted of 77 children, 48 boys and 29 girls, with a mean ± SD age of 24.6 ± 7.5 months (range, 12-38 months). The OM-6 surveys were completed by the mother (79%), father (16%), or both (5%). The number of surveys evaluated was 68 for the pretest and 69 each for the posttest and retrospective pretest. The baseline survey was completed in a subgroup of 19 only.
Among the pretest scores, caregiver concerns, speech impairment, and physical suffering had the highest median values of almost 4; followed by hearing loss and emotional distress, with median values of 3; while activity limitations had the lowest median value of 2 (Table 1).
No parent had experienced a month without any concerns at all. Parents reported most often the presence of speech impairment (mean ± SD, 3.6 ± 1.8), while the least often reported was activity limitations (mean ± SD, 2.5 ± 1.7). The pretest was used as a reference point.
For each item, the test-retest reliability was good (R>0.8), including caregiver concerns (R = 0.74). Ear-related global quality of life had the highest reliability (R = 0.98).
Internal consistency was satisfactory (α = .79). The most striking correlation was found between emotional distress and activity limitations (R = 0.69). The correlation between hearing loss and speech impairment (R = 0.46) was lower than expected, but in line with the results of Rosenfeld and colleagues.8
Construct validity, as determined by the correlation between the ear-related global quality of life measure and the OM-6 summary score, was fair (R = −0.77, P<.01).
Prospective responsiveness to clinical change as indicated by the SRM ranged from moderate (SRM ≥0.5) to large (SRM ≥0.8) for this population. The exact values are given in Table 2, with the survey summary score (SRM = 1.13) showing the largest sensitivity to change, followed by caregiver concerns (SRM = 0.92) and hearing loss (SRM = 0.87).
The median difference between the pretest and the retrospective pretest scores was significant at the group level (t = −3.3, P<.01), indicating that a response-shift bias is present. This phenomenon is visualized in Figure 2, reflecting that a greater change after surgery is observed if the retrospective pretest and posttest are compared, instead of the pretest and the posttest.
Two findings confirm this statement. First is the larger retrospective responsiveness to clinical change for most items of the OM-6, as seen in Table 3. A comparison of the prospective (Table 2) and retrospective (Table 3) SRMs confirms the tendency of a positive response shift for most scales of the OM-6. The differences, calculated as the retrospective SRM minus the prospective SRM, were as follows: emotional distress, 0.04; caregiver concerns, 0.10; speech impairment, 0.20; physical suffering, 0.40; and hearing loss, 0.70; with the largest retrospective shift in parental response. There were 2 exceptions with a negative response shift: the survey summary score (SRM = −0.04) and activity limitations (SRM = −0.03).
Second, retrospective pretest vs posttest analyses revealed a higher number of positive changes for individual items compared with pretest vs posttest analyses, as reflected in significant z scores for each of the items (Table 4). This indicates that the OM-6 scores at the pretest had generally been underestimations of HRQL problems. Comparing the retrospective pretest with the pretest scores, differences in hearing loss were significant (z = −3.3, P<.05). In contrast, the results for ear-related global quality of life (z = −3.6, P<.05) suggest an overestimation of HRQL before surgery. This may be indicative of changing internal standards of parents during the study.
Caregiver perceptions influence their reports on quality of life in young children with OME. The results of this study are comparable to those of the original study8 of the OM-6; however, the results were obtained from different age groups. The young age of our study population, 12 to 38 months vs 6 months to 12 years in the first OM-6 study, necessitated the exclusive use of parental report. Validation of the OM-6 is necessary and has been shown in a Dutch population in this study. However, parents are subject to internal and external factors influencing their perception. Confirmation of this is found in 64 surveys in which there was minimal correlation between audiometric (objective) hearing loss and prospective (subjective) hearing loss before surgery (R = 0.06, P = .65). It would seem that parents are in general not sensitive to the degree of hearing loss before surgery.
In recent studies,6,8,15 parental perceptions have been found to be poor predictors of childhood hearing loss, an important symptom of OME and an item of the OM-6. Although the accuracy of parental prediction was not an explicit study hypothesis herein, some comments should be made. It is not realistic to expect from parents a similar accuracy in predicting the child's hearing compared with audiometric testing in a sound-free environment. The parent and child are subject to constant distractions in their daily life, personal and environmental. The OM-6 is a convenient method of assessing the consequences of OME and hearing loss, separate from otoscopy and audiometry. Parental concern about their child's hearing, as measured by the OM-6, can serve as a supplement to standard diagnostic procedures.
To our knowledge, reported quality-of-life studies in children with OME have not considered the effects of response-shift bias on parental reporting. Our results suggest there were changes in parental perception during the study, confirming the presence of response shift and questioning pretest and posttest differences as indicators of change.14 This is clear in the comparison of the prospective and retrospective SRMs (Table 2 and Table 4), which show small to large positive differences at the retrospective measurement for most of the scales. Not using the retrospective pretest would include a risk of missing substantial changes and result in missed treatment indications, such as insertion of tympanostomy tubes. In this way, response shift can be viewed as an important component of treatment effect.11 The significant differences between the pretest and retrospective pretest scores for ear-related global quality of life and hearing loss support this proposition (Table 3).
As evidenced by parents' underestimating the seriousness of hearing loss and overestimating the quality of life of their child before surgery, an adaptation to the presence of symptoms appears to have taken place, with a positive effect on perceived HRQL.16 Caregivers have difficulty in accurately identifying the extent of hearing loss present before surgery for multiple reasons, which are not mutually exclusive.17 In chronic conditions, such as OME often is during childhood, this response-shift bias can be understood as an attempt by parents to internally normalize the experienced level of HRQL. The underlying assumption is that parents want to think and feel positive about themselves and their child, even in the presence of symptoms.9,18 Anticipatory effects may also be present,14,19 as parental expectations about future HRQL could be heightened at the pretest by already having scheduled the child for tympanostomy tube placement. At the retrospective pretest, parents are able to compare HRQL before and after surgery. The results of treatment may lead parents to realize that the situation before surgery was worse than they had thought. This is especially true for a child experiencing OME for the first time, as parents become knowledgeable about the disorder and its possible consequences,12 affecting their self-reported test scores. This uncertainty is compounded by the fluctuating nature of the hearing loss resulting from OME and by differing developmental rates among children. The first issue has been dealt with by evaluation of the child's middle ear status at each completion of the OM-6.8 The problem of differing developmental rates has been addressed by structuring each domain of the OM-6 as a single item, listing several ways in which OME may effect the function within that domain, with the possibility of assessing age-appropriate symptoms.8
In conclusion, response-shift bias is an important factor in assessing the effects of surgery in children with OME. Our results suggest that the dynamic nature of a quality-of-life measurement, manifested as a change in parental frame of reference, needs to be considered. Otherwise, there is a risk of underestimating the effects of tympanostomy tube placement on subjective measures, such as the HRQL of young children and parental concerns.
Corresponding author and reprints: Angelique A. Timmerman, MA, Department of Otorhinolaryngology–Head and Neck Surgery, University Hospital of Maastricht, PO Box 5800, 6202 AZ Maastricht, the Netherlands (e-mail: ATI@skno.azm.nl).
Submitted for publication September 20, 2001; final revision received April 22, 2002; accepted January 14, 2003.
This study was supported by the Heinsius-Houbolt Foundation, Bussum, the Netherlands, and the ENT Research Foundation of the University Hospital of Maastricht.
We thank all the parent-child pairs for their willingness to participate in this study. Richard M. Rosenfeld, MD, is acknowledged for his permission to use the OM-6 in our study and for the valuable comments on the manuscript. We also thank Johannes J. Manni, MD, for his useful comments on the manuscript and Angélique Oligschläger for her efforts in data collection.
Create a personal account or sign in to: