Contingent Valuation Analysis of an Otolaryngology and Ophthalmology Emergency Department: The Value of Acute Specialty Care

Key Points

Question  What is the value of acute specialty care, as assessed by a contingent valuation analysis of a specialty emergency department?

Findings  In this survey study, patients presenting to the emergency department with eye and ear, nose, and throat complaints placed an increased explicit value on specialty emergency services, relative to general emergency care.

Meaning  Contingent valuation data are useful in valuing patient preferences in monetary terms and can help inform state-wide resource allocation and the availability of direct-to-specialist care.

Importance  Specialty emergency departments (EDs) provide a unique mechanism of health care delivery, but the value that they add to the medical system is not known. Evaluation of patient preferences to determine value can have a direct impact on resource allocation and direct-to-specialist care.

Objective  To assess the feasibility of contingent valuation (CV) methodology using a willingness-to-pay (WTP) survey to evaluate specialty emergency services, in the context of an ophthalmology- and otolaryngology-specific ED.

Design, Setting, and Participants  Contingent valuation analysis of a standalone otolaryngology and ophthalmology ED. Participants were English-speaking adults presenting to a dedicated otolaryngology and ophthalmology ED. The WTP questions were assessed using a payment card format, with reference to an alternative modality of treatment (ie, general ED), and were analyzed with multivariate regression.

Intervention  Validated WTP survey administered from October 14, 2014, through October 1, 2015.

Main Outcomes and Measures  Sociodemographic data, level of distress, referral data, income, and WTP.

Results  A total of 327 of 423 (77.3%) ED patients responded to the WTP survey, with 116 ophthalmology and 211 otolaryngology patients included (52.3% female; mean [range] age, 46 [18-90] years). The most common reason for seeking care at this facility was a reputation for specialty care for both ear, nose, and throat (80 [37.9%]) and ophthalmology (43 [37.1%]). Mean WTP for specialty-specific ED services was $377 for ophthalmology patients, and $321 for otolaryngology patients ($340 overall; 95% CI, $294 to $386), without significant difference between groups (absolute difference, $56; 95% CI, $−156 to $43). Self-reported level of distress was higher among ear, nose, and throat vs ophthalmology patients (absolute difference, 0.47 on a Likert scale of 1-7; 95% CI, 0.10 to 0.84). Neither level of distress, income, nor demographic characteristics influenced WTP, but patients with higher estimates of total visit cost were more likely to have higher WTP (β coefficient, 0.27; SE, 0.05; adjusted R² = 0.17 for model).

Conclusions and Relevance  Patients with eye and ear, nose, and throat complaints place a mean explicit value on specialty emergency services of $340 per visit, relative to general emergency care. Ultimately, CV data using WTP methodology are useful in valuing patient preferences in monetary terms and can help inform state-wide resource allocation and the availability of direct-to-specialist care.

Physicians and hospitals are increasingly under pressure to justify medical and surgical interventions in terms of quality, cost, and benefits to the patient. The Patient Protection and Affordable Care Act of 2010 has made the rationalization of health care a priority within American health care policy.^1-3 And while there has been increasing focus on cost reduction at the physician, hospital, and health care system level, measuring the value of care (defined as health care outcome per unit of cost) has proven more difficult. Unfortunately, measurement of health outcomes does not always easily translate into dollars and cents. This is especially germane to the field of otolaryngology, which has a high proportion of quality-of-life interventions for which standard outcome measures such as survival and mortality rates often do not apply.

This study explores a way of defining value in otolaryngology, referred to as contingent valuation (CV).^4,5 Contingent valuation encompasses a holistic assessment of benefits from a patient perspective, including quality of life. This methodology uses willingness-to-pay (WTP) questions to assign an explicit dollar value to the perceived patient benefit of medical products and services, allowing researchers to obtain valuations for hypothetical scenarios that would otherwise be difficult to obtain.⁶ For example, CV has been used to assess WTP for spine surgery,⁷ polysomnography,⁸ depression treatment,⁹ and acute otolaryngology care.⁵ Contingent valuation data can then be applied to cost-benefit analysis, a form of economic evaluation that compares the benefits of a health program (in monetary terms) to its costs, allowing rationalization of services based on their added value to society. Herein, we aimed to assess the feasibility of CV methodology using WTP surveys to compare 2 different specialty emergency services, an ophthalmology- and otolaryngology-specific emergency department (ED).

This study was approved by the institutional review board of the Massachusetts Eye and Ear Infirmary. Patients gave verbal consent before taking the survey; written consent was waived given minimal risk to participants. All patients were enrolled through a specialty eye and ear, nose, and throat (ENT) ED in Boston, Massachusetts, which provides otolaryngology and ophthalmology care 24 hours per day on a walk-in basis or to patients who transfer from another facility. Personnel include ophthalmologists, otolaryngologists, and nurses available at all times, and the facility supports laboratory, radiology, and pathology services, as well as specialty-specific services (eg, audiology, videolaryngoscopy, slitlamp examinations, and minor procedure rooms). The patient population has previously been defined.¹⁰

An anonymous CV survey was first constructed. The theoretical framework for CV studies outlined by O’Brien and Gafni¹¹ was used to design the study. The survey first asked 21 questions that assessed demographic data, income, and patients’ estimates of cost per visit. A Likert scale was used to assess self-reported patient distress levels, from 1 (no distress) to 7 (extreme distress). Income was averaged using the midpoint values of the provided income ranges. Willingness to pay was then assessed using CV questions, using a standard payment card format (eAppendix in the Supplement). The WTP values were explicitly created in terms of out-of-pocket costs to the patient. For reference, the mean cost of an ED visit in our state ($575) was provided in writing.¹²

For validation, the survey was then reviewed in reference to the National Oceanographic and Atmospheric Administration recommendations on CV research.⁶ The survey first underwent review by several rounds of revisions based on expert opinion, followed by pilot testing with 15 patients in the ED, with face-to-face interviews. Improvements were made based on participants’ comments. Last, the final survey was piloted on an additional 15 patients in our ED in an unobserved fashion.

Patients presenting for otolaryngology ED care were enrolled. Ophthalmology patients were also included, as a comparator group. Patients were required to be 18 years or older, speak English, and present to the ED for an eye, ear, nose, throat, head, or neck complaint. Children, adults with decisional impairment, non–English-speaking and -reading individuals, and prisoners were excluded. Patients who could not fill out surveys because of life-threatening problems (eg, acute bleeding) were also excluded.

Data were analyzed with Microsoft Excel and JMP 12 Pro software. Basic demographic data and WTP distributions were analyzed. Univariate and multivariate analyses were performed to determine the variables most likely to influence reported value metrics. Effect sizes and the confidence intervals around these values were calculated. Significance was defined at a P value of .05.

From October 14, 2014, through October 1, 2015, a total of 423 surveys were offered. Three surveys were not returned, 77 patients elected not to participate, and 16 patients were excluded after taking the survey (Figure 1). No patients took the survey twice. Thus, 327 surveys (116 ophthalmology, 211 ENT) were included in the study, for a total response rate of 77.3%.

Cohort demographic characteristics are listed in the Table. Ophthalmology patients were significantly more likely than ENT patients to be presenting at the institution for the first time (76 [65.5%] and 99 [46.9%], respectively). A total of 171 (52.3%) survey respondents were female, and the mean age was 46 years (range, 18-90 years). Most patients identified as white (265 [81.0%]). The average income was $92 386 (ENT patients, $93 295; ophthalmology patients, $90 776) and was similar between ENT and ophthalmology patients (absolute difference, $2519; 95% CI, $−16 643 to $21 682). Patients traveled a mean of 33 minutes (95% CI, 30 to 36 minutes) to seek care (ENT patients, 32 minutes [95% CI, 29 to 35 minutes]; ophthalmology patients, 35 minutes [95% CI, 29 to 40 minutes]), with no significant difference between groups (absolute difference, 3 minutes; 95% CI, −9 to 3 minutes). The most common reason for seeking care at this facility was a reputation for specialty care for both ENT (80 [37.9%]) and ophthalmology (43 [37.1%]), followed by referrals from other area physicians (Figure 2). Ninety-one (27.8%) patients reported seeing their primary physician for their presenting complaint before being seen in the specialty ED.

Self-reported level of distress seemed to be higher in those presenting with ENT complaints compared with ophthalmology complaints (Figure 3). On a Likert scale of 1 to 7, with 7 representing maximal distress, the modal response was 4 among eye patients and 7 among ENT patients; median was 5 in both groups. Distress was higher among ENT vs ophthalmology patients (absolute difference, 0.47; 95% CI, 0.10-0.84).

Mean WTP for a specialty-specific ED was $377 for ophthalmology patients and $321 for ENT patients ($340 overall; 95% CI, $294 to $386), without significant difference between groups (absolute difference, $56; 95% CI, $−156 to $43). The median response for both ENT and ophthalmology patients was $200, with a modal response of $100 for ENT patients and $200 for ophthalmology patients (Figure 4). Notably, 53 patients (16.2%) did not answer the question regarding WTP. Overall, 299 patients (91.4%) reported that they were able to understand the WTP question, while 5 (1.5%) reported that they did not; 23 (7.0%) left this question blank. Regression analysis demonstrated that the WTP was not influenced by income level, level of distress, age, sex, or race/ethnicity; patients with higher estimates of total visit cost were more likely to have higher WTP (β coefficient, 0.27; SE, 0.05; adjusted R² = 0.17 for entire model).

In this study, we explored the relative WTP for specialty emergency services in a dedicated otolaryngology and ophthalmology ED. Using CV surveys, we show that patients assigned a mean value of $340 to a visit to a specialty-specific ED, without difference between the ENT and ophthalmology patients. Level of distress on presentation, although different between the ENT and ophthalmology groups, did not seem to influence WTP, nor did demographic factors. Although income has been shown to be related to WTP in several other studies,^7,9 income was not related to WTP in our cohort; we hypothesize that for acute specialty care, patients with high levels of distress (as was seen in our patient group) will be less constrained by their overall income level when making medical decisions. Only the patient’s initial guess of how much an ED visit costs was correlated with WTP.

From this, several conclusions may be drawn. First, patients in this study exhibited a quantifiable preference for acute specialty care, as opposed to general acute care. While this may be intuitive in a qualitative sense, demonstrating this preference in a quantitative way is essential to economic evaluation such as cost-benefit analysis. Notably, only 27.8% of patients presenting for care had seen a primary care physician before presenting to the ED. This may reflect the perceived urgency of their complaints, but it also suggests that patients may desire “direct-to-specialist” care. Patient self-referral to specialists has been suggested in several studies to be both beneficial and cost-effective.^13-15 Second, CV using WTP methodology is a feasible method within the surgical subspecialties for assessing patient preferences. Given the novelty of this study, there are no WTP comparisons for other services in otolaryngology, although CV studies have been performed for treatments of tinnitus and reflux disease.^16,17 Health care goods and services are often difficult to value¹⁸ because the health care market is imperfect: prices and costs are not transparent, the benefits of interventions are often uncertain, and insurance coverage incentivizes patients to seek more care than they would if paying out of pocket.¹⁹ Thus, CV can be valuable in obtaining patient-provided information on value that would otherwise be obscured. Although the use of CV is increasing, it is still relatively underused in health care.⁴

This study also highlights several of the criticisms of CV methodology. One can see in Figure 3 patients’ preferences to pick round numbers—for instance, $100 and $500 were common responses to the WTP questions. Does this then lead to a fabricated valuation of emergency services? Gregory et al²⁰ argue that “designers of a CV study should function not as archaeologists, carefully uncovering what is there, but as architects, working to build a defensible expression of value.” In this way, although participants may find it novel and difficult to value services that they do not usually pay for, they are able to do so with a valid questionnaire. Notably, our respondents had mild difficulty answering WTP questions (with 1.5% indicating that they did not understand the questionnaire and 16.2% leaving WTP estimates blank). These criticisms all suggest that CV analyses are useful in resource allocation only when they are methodologically sound, and that further research should be done in the surgical subspecialties to enhance CV’s reliability. Finally, several authors have noted that stated WTP does not always correlate with actual WTP.^6,21 This study was not designed to explicitly address this question.

There are several limitations to this study. We surveyed patients who were already presenting for specialty care—this represents a sample at a single institution. Ideally, a cross-section of the entire population would be surveyed to obtain WTP data,¹¹ while at the same time questions must not be too general or inconsequential to respondents.²² We also excluded non–English-speaking patients, which may skew results, but this was done in an attempt to mitigate the inconsistencies that have been demonstrated when surveys are administered in various languages.^23,24 Last, our surveys were not administered to patients who were in substantial distress—specifically, those with severe bleeding, airway distress, or need for urgent operative intervention. This was done for safety, and has an uncertain effect on the mean WTP values elicited. Correlating distress levels with specific presenting signs and symptoms would help to further stratify WTP valuations.

Specialty-specific ED services were valued by patients at a mean of $340 per visit in this patient population. These results are consistent between ophthalmology and otolaryngology. More importantly, this study illustrates how CV analysis using WTP can be used for the economic evaluation of surgical services. In the current health care climate, analysis of patient preferences should be considered when resources are allocated on an institutional and national level. Furthermore, it is incumbent on otolaryngologists to demonstrate the value of surgical services in a meaningful, quantifiable, and comparable fashion.

Accepted for Publication: September 10, 2016.

Corresponding Author: Matthew R. Naunheim, MD, MBA, Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA 02114 (matthew_naunheim@meei.harvard.edu).

Published Online: November 23, 2016. doi:10.1001/jamaoto.2016.3267

Author Contributions: Dr Naunheim had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Naunheim, Kozin, Sethi, Gray, Shrime.

Acquisition, analysis, or interpretation of data: Naunheim, Kozin, Ota, Shrime.

Drafting of the manuscript: Naunheim, Kozin, Ota.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Naunheim, Sethi.

Administrative, technical, or material support: Naunheim, Kozin, Sethi, Ota, Gray.

Study supervision: Naunheim, Kozin, Ota, Shrime.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Shrime receives funding from the GE Foundation for work unrelated to this manuscript and has received personal fees from Ethicon unrelated to this work. No other disclosures were reported.