Figure. Questionnaire scenarios A and B. Median estimates and interquartile ranges (IQRs) for the probability of a significant reduction in apnea-hypopnea index for the 5 preferred treatment options a patient with severe obstructive sleep apnea–hypopnea syndrome (OSAHS) (A) and one with moderate OSAHS. CPAP indicates continuous positive airway pressure; LAUP, laser-assisted uvulopalatoplasty; MRA, mandibular repositioning appliance; UPPP, uvulopalatopharyngoplasty.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Aarts MCJ, van der Heijden GJMG, Siegers C, Grolman W, Rovers MM. Awareness of, Opinions About, and Adherence to Evidence-Based Guidelines in Otorhinolaryngology. Arch Otolaryngol Head Neck Surg. 2012;138(2):148–152. doi:10.1001/archoto.2011.1166
Author Affiliations: Department of Otorhinolaryngology, University Medical Center Utrecht (Drs Aarts, van der Heijden, Grolman, and Rovers and Ms Siegers), and Julius Center for Health Sciences and Primary Care (Drs van der Heijden and Rovers), Utrecht, the Netherlands.
Background Guidelines may assist physicians and patients in decisions about effective and safe care. Little is known about the awareness of, opinions about, and adherence to evidence-based guidelines in otolaryngology.
Methods We performed a survey among 440 otorhinolaryngologists of the Dutch Society of Otolaryngology–Head and Neck Surgery. The questionnaire consisted of questions about the characteristics of the respondents, their knowledge and opinions of available evidence-based guidelines, and their adherence to them. Furthermore, 2 clinical scenarios were included to test their knowledge regarding the guideline for diagnosis and treatment of obstructive sleep apnea–hypopnea syndrome.
Results The daily practice of most otorhinolaryngologists (70%) was influenced by evidence based guidelines: 62% stated that evidence-based guidelines supported their clinical practice; 32% stated that guidelines directed their clinical practice. The mean confidence in the evidence of recommendations stated in the guidelines was 77%. The mean percentage of nonadherence to guideline recommendations was 45%. The guideline adherence was higher in younger otorhinolaryngologists. Sex, type of hospital, and PhD grade did not affect the preferences of the responders. In general, surveyed otorhinolaryngologists treated patients in accordance with the guidelines. However, when disease characteristics were less distinct, on the one hand, the guidelines included a wider range of treatment options, and on the other hand, variation in chosen treatment by otorhinolaryngologists increased.
Conclusions Dutch otorhinolaryngologists are well aware of the available evidence-based guidelines, and many use these to support their clinical practice. The treatment by Dutch otorhinolaryngologists is in accordance with the Dutch guidelines. When guidelines, however, do not provide strict recommendations and allow flexibility in treatment, larger variations in chosen treatment occur. This may reflect that otorhinolaryngologists still may encounter difficulties when applying the current guidelines to an individual patient.
During the last decade, the number of published guidelines has rapidly increased. Awareness of and adherence to evidence-based clinical guidelines is considered vital for improving effectiveness, quality, and safety of patient care. Clinical guidelines are considered valid if they are developed in a rigorous way, independently of vested interests of their developers, and if they support decision making in practice and affect actual care. National health care improvement institutes (eg, the National Institute for Health and Clinical Excellence in the United Kingdom, the Agency for Healthcare Research and Quality in the United States, and the Institute for Healthcare Improvement in the Netherlands) follow validated systematic approaches for guideline development such as GRADE1 and AGREE.2 Guideline statements may assist physicians and patients in decisions about appropriate clinical and health care for specific circumstances. Guidelines have limited impact on clinical practice unless they are successfully integrated in the clinical settings.3 Several otolaryngology guidelines have been developed worldwide, including for adult sinusitis, surgical management of otitis media with effusion, allergic and nonallergic rhinitis, respiratory tract infections, head and neck cancer, disease of the adenoids and tonsils (DAT) and obstructive sleep apnea–hypopnea syndrome (OSAHS).4-9 One survey about guideline awareness among pediatricians has been published before.10 However, little is known about the dissemination of evidence-based guidelines by otorhinolaryngologists.
We performed a survey among otorhinolaryngologists to study their current awareness, knowledge, and opinions of evidence based-otolaryngology guidelines. In addition, we used 2 clinical scenarios to assess their adherence to a set of guidelines.
We performed a survey among Dutch otorhinolaryngologists between September and December 2010. We contacted 440 Dutch otorhinolaryngologists of the Dutch Society of Otolaryngology–Head and Neck Surgery by mail and asked them to complete a structured postal questionnaire. To maximize the response rate, we posted a reminder 5 weeks later. All questionnaires were processed anonymously.
The first part of the questionnaire consisted 6 questions concerning the characteristics of the respondents, ie, sex, age, PhD grade, type of hospital, area of interest, and time registered as an otorhinolaryngologist. Furthermore, we asked if they were aware of evidence-based guidelines, which guidelines they knew, how often they used them, and what their general opinion about evidence-based guidelines was. We also asked them which of the current guidelines they knew by heart. The current existing guidelines used in the Netherlands are for the treatment of chronic rhinosinusitis, DAT, OSAHS, facial nerve paralysis, laryngeal carcinoma, oral cavity/oropharyngeal carcinoma, and hypopharyngeal carcinoma.4,10-16 The respondents also rated their agreement with the following 3 statements on a visual analog scale ranging from 0% to 100%: (1) To what extent do you have confidence in the correctness of evidence-based guidelines?; (2) To what extent do you deviate from evidence-based guidelines in your daily practice?, and (3) To what extent do the guidelines influence your daily practice?
In the second part of the questionnaire, 2 clinical scenarios were described to test the knowledge of the otorhinolaryngologists regarding the available evidence-based guideline for the diagnosis and treatment of OSAHS (Table 1).4
Data from completed and returned questionnaires were analyzed using SPSS, version 17 (IBM). To test for differences in percentages, we used a χ2 test, and for continuous data, we used a t test or a Wilcoxon test.
Of the 440 otorhinolaryngologists, 187 returned a completed questionnaire (43.0%); 7 indicated that they did not want to participate (2%); and 246 did not respond at all (55%). The baseline characteristics of the respondents did not differ from the characteristics of all Dutch otorhinolaryngologists. Data for this comparison were provided by the Dutch Society of Otolaryngology–Head and Neck Surgery (Table 2).
The guidelines were used daily by 114 respondents (61%); 2 to 3 times a week by 38 respondents (20%); and once a week or less by 35 respondents (19%). Most otorhinolaryngologists stated that evidence-based guidelines supported their clinical practice (62%); 32% stated that the guidelines even guided their clinical practice; and 1% stated that the guidelines impeded their clinical practice. The remaining 5% of the otorhinolaryngologists had another opinion. The percentage of otorhinolaryngologists whose practice was guided by guidelines was higher among academic than among general otorhinolaryngologists: 59% vs 29% (difference, 30% [95% CI, 11%-49%]).
The respondents had the best knowledge of the guidelines on DAT, chronic rhinosinusitis, and OSAHS and the least knowledge of the guidelines on oral cavity/oropharynx and hypopharynx carcinoma (percentages for exact knowledge of the respective guidelines, 68%, 61%, 60%, 18%, and 17%). Accurate knowledge of the guideline on laryngeal carcinoma was reported by 57% and 14% of the academic and general otorhinolaryngologists, respectively (difference, 43% [95% CI, 25%-62%]). Similar differences were found regarding the guideline on hypopharynx carcinoma (44% [95% CI, 25%-63%]) and oral cavity/oropharynx carcinoma (47% [95% CI, 29%-65%]). Regarding the guidelines on DAT, OSAHS, chronic rhinosinusitis, and facial palsy, these respective differences were reversed, ie, 75%, 71%, 69%, and 60% of the general otorhinolaryngologists reported accurate knowledge of these guidelines vs 37%, 13%, 27%, and 40% of the academic otorhinolaryngologists (differences, 42% [95% CI, 25%-60%]; 58% [95% CI, 44%-72%]; 38% [95% CI, 19%-57%]; and 20% [95% CI, 1%-39%]).
The mean confidence of the respondents in the evidence of recommendations stated in the guidelines was 77% (median, 80% [interquartile range (IQR), 75%-85%]). The mean percentage of nonadherence to guideline recommendations was 45% (median, 50% [IQR, 25%-75%]). The mean percentage of the influence of evidence-based guidelines on daily practice was 70% (median, 75% [IQR, 61%-85%]). There was more guideline adherence among younger otorhinolaryngologists (age group, 30-39 years) than among older ones. Gender, type of hospital, and PhD grade did not affect the preferences of the responders.
Twenty-eight of the respondents who returned a questionnaire indicated that OSAHS was not their area of interest (15%). In addition, 35 did not answer the questions about scenario A (19%), and 42 did not answer the questions about scenario B (22%).
Concerning scenario A (patient with severe OSAHS), the current Dutch OSAHS guidelines4 recommend continuous positive airway pressure (CPAP) as first-line treatment and weight reduction in obese patients with OSAHS as an additional treatment goal. Of the 152 respondents, 92% preferred CPAP; 2% preferred uvulopalatopharyngoplasty (UPPP); and 6% preferred treatment technique other than those listed (eg, laser-assisted uvulopalatoplasty [LAUP], mandibular repositioning appliance [MRA], weight reduction, or other surgery). A quarter of the otorhinolaryngologists chose weight reduction as the only treatment or as a part of it. Sex, type of hospital, and PhD grade did not affect the preference of the responders. The treatment preferences were in line with the current guideline, which recommends CPAP as first-line treatment in patients with severe OSAS. No large differences in preferred treatment were found between the OSAHS and non-OSAHS specialists. Figure, A shows the estimated reduction in apnea-hypopnea index (AHI) for each of the 5 preferred treatment options. The probability of an AHI reduction varied from 31% for LAUP to 85% for CPAP.
Concerning scenario B (moderate OSAHS), the current guidelines recommend MRA as well as CPAP or surgery (UPPP or LAUP) as primary treatment. Of the 145 respondents, 42% preferred MRA; 20%, UPPP; 13%, CPAP; 7%, weight reduction; and 2% preferred LAUP. The remaining 16% preferred a combination of treatments (Table 3). Nineteen percent of the otorhinolaryngologists chose weight reduction as the only treatment or as part of it. No large differences in preferred treatment were found between OSAHS and non-OSAHS specialists. Respondents aged 30 to 39 years more often preferred a surgical policy than did those aged 50 years or older. Sex, type of hospital, and PhD grade did not affect the preferences of the responders.
The mean probability of an AHI reduction varied from 34% for weight reduction to 79% for CPAP (Figure, B). We also stratified the estimates for treatment effect of CPAP, MRA, and UPPP according to respondents who preferred this treatment and those who did not. For scenario A (severe OSAHS), a reduction of AHI after CPAP, MRA, and UPPP was estimated by 85%, 50%, and 45% of the respondents who preferred each of these respective treatments compared with 85%, 35%, and 35% of the respondents, respectively, who preferred another treatment. For scenario B (moderate OSAHS), a reduction of AHI with CPAP, MRA, and UPPP treatment was estimated by 85%, 75%, and 75% of the respondents who preferred these respective treatments compared with 85%, 40%, and 50% of the respondents, respectively, who preferred another treatment (Table 4). The estimated probability of an improvement in quality of life for patients with OSAHS treated with CPAP was 76% (95% CI, 74%-78%).
Our results show that the Dutch otorhinolaryngologists are familiar with the currently available evidence-based guidelines. Their confidence in the guidelines is high, and most otorhinolaryngologists reported that they use evidence-based guidelines in their daily clinical practice. Still, 45% of the respondents also reported nonadherence to these guidelines. In the case of severe OSAHS (scenario A) the treatment by Dutch otorhinolaryngologists showed a homogeneous pattern, in accordance with the strict recommendations of the guidelines. When the disease characteristics were less distinct (scenario B), a much larger variation in treatments was found, which reflects the wide range of treatment options described in the guidelines.
Our results corroborate previous studies that also showed that younger, less experienced professionals used guidelines more often than older or more experienced professionals.17 Medical training programs may explain the increased guideline adherence over the last decade.18 In contrast to previous studies, we found no differences between academic and general otorhinolaryngologists regarding their use of guidelines in daily practice.18 Incorporating graded recommendations from evidence-based guidelines as learning goals in the current medical training programs may diminish practice variations within a medical specialty.
The relatively high proportion of otorhinolaryngologists who were found not to adhere to guidelines is in line with earlier reported adherence rates.19-21 If the guidelines are nonspecific, controversial, and/or not compatible with current values and do recommend a change to existing routines, poor adherence is more likely. When guidelines have been mentioned in the media, have a firm scientific base, are explicit and precise, and are in accordance with existing routines, poor adherence is less likely.22
Some limitations of this study should be mentioned. First, our response rate was moderate, which may influence the validity of our results. Since the general characteristics of our respondents are in agreement with data of the Dutch Society of Otolaryngology–Head and Neck Surgery, we consider our sample of respondents to be representative enough for all registered otorhinolaryngologists in the Netherlands (Table 2).
Second, we used clinical vignettes to assess physician practice variation. Clinical scenarios have proven to be effective tools for the evaluation of medical decision behavior.23-25 Moreover, research has shown that, when used to study differential diagnosis, selection of tests, and treatment decisions, validity of data on quality of care derived from vignette-based surveys is higher than those from medical record reviews.23-25 Nevertheless, many respondents reported that they would like to know more about the patient in both scenarios for their judgment and decisions; eg, information about dentition was missing, and there was no record of the chin-lift. We, however, consider it unlikely that more detailed information on diagnostic criteria for the case scenarios would have markedly changed our results.
Third, we conducted our study among Dutch otorhinolaryngologists. This may restrict the generalizability of our results to other medical specialties and countries. We do not believe, however, that the characteristics that may influence knowledge of guidelines and adherence to them are typically related to the Dutch setting. Hence, our results on the knowledge and use of guidelines may be relevant for other medical specialties and countries.
Concerning the clinical scenarios, a high level of guideline adherence was reported in this survey. This is quite remarkable because impaired dissemination of practice guidelines and clinical research into daily practice has been reported repeatedly.17,19-21,26-28 Since the patients with OSAHS in the scenarios were well treated according to the guidelines, the implementation of the current OSAHS guidelines in clinical practice can be considered successful. The authors of the Dutch OSAHS guideline4 reported several actions to promote awareness and implementation of the guidelines: they were actively distributed among all relevant associations and hospitals; an abstract of the guidelines was published in the Dutch Journal of Medicine and several related clinical journals; the content of the guidelines was published on the Dutch Institute for Health Care Improvement (Centraal Begeleidings Orgaan) (CBO) Web site,4 and the guidelines were discussed at several scientific conventions and audits of the related societies. These actions can be considered sufficient to promote awareness of the guidelines.
However, the increasing heterogeneity of treatment that we found when indications were less explicit deserves further attention when a revision is considered. For this, we recommend avoiding guidelines produced and written in medical textbook style. This may make implementation ineffective, particularly when it requires new behavior and organizational change. Instead, we advise that a limited number of graded practice recommendations be created and presented in a very concise manner. In an additional source document, each recommendation should be accompanied by an explanatory text that should cover the explicitly filtered best evidence. By using graded recommendations, guidelines may become more explicit about where the best evidence leads to strong recommendations and where the best evidence leaves more room for personal judgments and preferences with weaker recommendations. In the source document, the best evidence should also be placed in the context of norms and values of target users.
In conclusion, Dutch otorhinolaryngologists are well aware of and use the available evidence-based guidelines to support their clinical practice. The treatment by Dutch otorhinolaryngologists is in accordance with the Dutch guidelines. If the guidelines, however, do not provide strict recommendations and allow flexibility in treatment, larger treatment variations were found. This may reflect that otorhinolaryngologists still may encounter difficulties when applying the current guidelines in individual patient care.
Correspondence: Dr Aarts, Department of Otolaryngology, University Medical Center Utrecht, PO Box 85500, 3508 GA, Utrecht, the Netherlands (ENT-Research@umcutrecht.nl).
Submitted for Publication: June 7, 2011; final revision received August 10, 2011; accepted November 5, 2011.
Author Contributions: Dr Aarts had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Aarts, van der Heijden, Siegers, Grolman, and Rovers. Acquisition of data: Aarts, van der Heijden, Siegers, and Grolman. Analysis and interpretation of data: Aarts, van der Heijden, Siegers, Grolman, and Rovers. Drafting of the manuscript: Aarts, van der Heijden, Siegers, and Grolman. Critical revision of the manuscript for important intellectual content: Aarts, van der Heijden, Siegers, Grolman, and Rovers. Statistical analysis: Aarts, van der Heijden, Grolman, and Rovers. Administrative, technical, and material support: Aarts. Study supervision: van der Heijden, Siegers, Grolman, and Rovers.
Financial Disclosure: None reported.