Patients' Ratings of Quality and Satisfaction With Care at the En[a-z] of Life

Objective  To elicit ratings of quality and satisfaction with care from medical inpatients, especially those near the end of life.

Methods  We conducted a cross-sectional survey of 84 seriously ill medical inpatients at 2 academic medical centers. Forty-five had do-not-resuscitate orders. Patients were interviewed using a valid and reliable instrument, the Quality of End-of-Life Care and Satisfaction With Treatment scale (scored from 1.0-5.0) and standard measures of symptoms, anxiety, depression, and delirium.

Results  Mean patient ratings of quality of care were higher regarding physicians than nurses (4.39 vs 4.24; P = .01). Mean patient ratings of satisfaction with physicians were also higher, but not significantly (4.53 vs 4.43; P = .32). In analysis of variance models, patient ratings of physician quality were lowest for patients with do-not-resuscitate orders who were treated by a house-staff service compared with other patients (P = .01). These patients were also least satisfied with their physicians (P = .03). Nondepressed patients with private attending physicians rated nursing quality the highest (P = .16). These patients also reported the highest satisfaction with nurses (P = .002). Quality and satisfaction were not related to severity of illness, and pain was only weakly associated with satisfaction with physicians.

Conclusions  Patients with do-not-resuscitate orders who were treated by a house-staff service gave the lowest ratings of physician quality and satisfaction. Only private patients who were not depressed were highly satisfied with their nursing care. Further study is required to better understand these findings and whether they are amenable to quality improvement.

SEVERAL STUDIES^1-4 have reported on the quality of care patients receive at the end of life, but little has been reported regarding dying patients' own assessments of quality or satisfaction with their care. Assessing patients' views seems especially important at the end of life, since morbidity, mortality, functional status, and other standard measures of quality become less helpful, whereas other measures such as symptom burden,⁵ quality of life, satisfaction, and the interpersonal aspects of quality assume greater significance.⁶ Because of alterations in alertness, dyspnea, and other factors, interviewing patients at the end of life can be difficult. However, the direct reports of patients are extremely valuable, since several studies have reported divergence between the reports of patients regarding their care and the reports of their family members or other surrogates.^7,8

We have drawn on this previous work and that of Matthews et al⁹ and Matthews and Feinstein¹⁰ by adapting some of their patient survey questions to the end-of-life setting and to the rating by patients of nurses as well as physicians. We have called this instrument QUEST (the Quality of End-of-Life Care and Satisfaction With Treatment scale), and have shown in preliminary studies that it is valid and reliable.⁸ We herein report on the administration of this instrument, with a battery of other standardized questionnaires, to hospitalized medical patients with serious and complex illness,¹¹ and assess the clinical and sociodemographic factors associated with patients' responses. In particular, we focused on patients with do-not-resuscitate (DNR) orders, because these patients are very likely to die in the hospital,^12,13 and compared their responses with those of other seriously ill patients who are less likely to be imminently close to death.

Study participants included 84 medical inpatients from 2 urban teaching hospitals (Georgetown University Medical Center, Washington, DC, and St Vincent Catholic Medical Centers, St Vincent's Manhattan, New York, NY). This sample was drawn from a larger study of 257 hospitalized patients. We selected only those patients who gave interviews, since we have shown that surrogate reports do not match those of patients.⁸ Just more than half of the patients had DNR orders (n = 45). Hospitalized patients without DNR orders were included if they had had any serious (usually fatal) chronic illness with complex care needs,¹¹ including malignancy, human immunodeficiency virus/acquired immunodeficiency syndrome, coronary heart disease, congestive heart failure, chronic obstructive pulmonary disease, cerebrovascular disease, neurodegenerative disease, lupus, cirrhosis, and others. We interviewed English-speaking patients at Georgetown, and English- or Spanish-speaking patients at St Vincent's. The Spanish-language version was verified by means of back-translation. Patients were not interviewed if the staff deemed that they lacked decision-making capacity or were younger than 18 years. Patient interviews were also excluded from the analysis for low Reduced-Set Mini-Mental State Examination scores¹⁴ or excluded due to the presence of delirium according to the Confusion Assessment Method.¹⁵ The study was approved by the institutional review boards of both institutions, and all participating patients and surrogates gave informed consent.

Face-to-face structured interviews were conducted with patients to assess the quality of and their satisfaction with the care received from physicians and nurses and their symptoms, mental status, depression, anxiety, and sociodemographic information. Medical chart reviews were performed to assess severity of illness and to obtain other clinical information.

The previously validated and reliable 30-item QUEST scale⁸ was used to measure patients' perceptions of quality of care and satisfaction with their physicians and nurses. The QUEST scales were based on items derived from the previous work of Matthews et al⁹ and Matthews and Feinstein,¹⁰ who had developed instruments to elicit patients' appraisals of physician performance. The QUEST scale transformed these items into 4 subscales, including quality of care from physicians, quality of care from nurses, satisfaction with physicians, and satisfaction with nurses. The quality-of-care subscales consist of 9 items, rated on a 5-point Likert scale ranging from never to always, with possible mean scale scores from 1.0 to 5.0. Patients rated the physicians as a whole and the nurses as a whole rather than individual physicians or nurses. Patients were asked how frequently their physicians and nurses performed a variety of behaviors including "spent enough time with you," "arrived late," "been hard to reach," "seemed distracted," "willing to listen," "treated you as a disease," "showed personal concern," "ignored your feelings," and "responded quickly." The satisfaction subscales consist of 6 items, rated on a 5-point Likert scale ranging from very dissatisfied to very satisfied, with possible mean scale scores from 1.0 to 5.0. Patients were asked how satisfied they were with the behavior of their physicians and nurses, including "bedside manner," "common courtesy," "way of talking," "clinical and technical skills," "concern," and "overall" level of satisfaction. The QUEST scales demonstrated good internal reliability, with Cronbach α values ranging from 0.83 to 0.95 for the 4 subscales.

We used a 9-item symptom severity scale to assess a variety of symptoms including shortness of breath, pain, restlessness, nausea, constipation, feverishness, fecal incontinence, urinary incontinence, and dry mouth. Items were rated on a 4-point scale ranging from absent to severe.

We used the 14-item Hospital Anxiety and Depression Scale to assess emotional distress.¹⁶ Responses were on a 4-point scale. Anxiety and depression scores can each range from 0 to 21.

Other measures were used, including the Reduced-Set Mini-Mental State Examination¹⁴ and the interviewer's rating of the patient's degree of delirium using the Confusion Assessment Method.¹⁵ We reviewed medical charts to obtain demographic information, APACHE (Acute Physiology and Chronic Health Evaluation) III severity of illness scores,¹⁷ and information regarding insurance, type of physician, and advance directives.

The QUEST scores were positively skewed, and therefore the data were rank transformed to perform parametric testing.¹⁸ Bivariate associations were examined between the 4 QUEST subscales and the following patient characteristics and clinical variables: sex, ethnicity, diagnosis, DNR status, anxiety, depression, severity of illness, symptoms, religion, religious participation, type of insurance, type of attending physician, and the presence of a living will or a health-care power of attorney (proxy). We examined these bivariate associations using t tests, correlations, and 1-way analyses of variance (ANOVAs). We selected any variables found to be at least marginally significant (P<.10) for testing in multivariate analyses. Because we found interactions between DNR status and the type of attending physician as well as depression and type of attending physician, we constructed ANOVA models as the most simple and straightforward means of presenting these data. To do so, we dichotomized depression scores into depressed vs nondepressed according to the prescribed cutoff score of at least 11 on the Hospital Anxiety and Depression Scale.¹⁶

Of the 367 cases in which we could approach a patient or a surrogate, 110 (30%) refused. Of the 257 interviews with a patient or a surrogate, 88 were with patients. Four of these were excluded because of low Reduced-Set Mini-Mental State Examination scores, yielding 84 patients for this analysis.

Table 1 lists the characteristics of these 84 study patients. Fifty (60%) were men. Race was not recorded for 2 patients; of the remainder, 58 (71%) were white and 24 (29%) were African American, Latino, or Asian. Patients ranged in age from 27 to 101 years, and the average age was 59.8 years. Patients had a wide variety of diagnoses, including advanced cardiac, pulmonary, and neurological conditions; malignancy; and human immunodeficiency virus/acquired immunodeficiency syndrome.

As is commonly the case with scales of satisfaction and quality, we found a substantial skew toward more positive ratings of physicians and nurses. On a scale of 1.0 to 5.0, the mean QUEST rating for quality of physician care was 4.39, whereas the mean QUEST rating for quality of nursing care was 4.24 (P = .01). The mean QUEST rating for satisfaction with physicians was 4.53, whereas the mean QUEST rating for satisfaction with nurses was 4.44. This difference in satisfaction ratings was not significant (P = .32).

In exploring bivariate associations between the QUEST scores and multiple clinical and sociodemographic characteristics of patients, consistent associations or trends were apparent for only a few variables, as presented in Table 2. Having a DNR order and being treated by a house-staff service rather than a private attending physician were each significantly associated with lower ratings of quality and satisfaction for physicians and nurses. Higher anxiety and depression scores were also associated with lower quality and satisfaction ratings, although statistical significance was reached in only 3 of the 4 QUEST scales for depression and only 2 of the 4 QUEST scales for anxiety. Higher pain scores showed a trend toward association with lower QUEST scales, but this factor only reached statistical significance for the physician satisfaction QUEST score. Severity of illness, as measured by the APACHE III acute physiology score, was not associated with QUEST scores.

Table 3 shows ANOVA models of factors associated with QUEST scores. Patient ratings of physician quality of care were significantly lower for patients with DNR orders who were treated by a house-staff service compared with all other patients. Having a DNR order per se and being treated by a house-staff service per se were not associated with lower QUEST scores for physician quality.

In the multivariate ANOVA model for patient ratings of their satisfaction with physicians, greater pain tended to be associated with lower satisfaction with physicians, but this association was not statistically significant. However, the interaction between DNR status and treatment by a house-staff service vs a private attending physician was once again significant. Private patients without DNR orders reported the highest satisfaction, whereas service patients with DNR orders reported the lowest.

Although the findings of the ANOVA model for factors associated with patient QUEST scores for nursing quality were not significant, we found a trend for ratings to be highest for private patients who were not depressed, as measured by the Hospital Anxiety and Depression Scale.

The ANOVA model for factors associated with patient QUEST scores for satisfaction with nurses showed very high ratings of satisfaction with nurses by private patients who were not depressed, compared with all other patients.

Although our validity studies showed that surrogate QUEST ratings did not accurately reflect patient QUEST ratings,⁸ we nonetheless calculated surrogate QUEST scores for general comparison with the 84 patients who were interviewed in the present study. Interviewed surrogates (n = 195) were mostly family members (21% spouses, 32% children, 33% other relatives, and 15% nonfamily surrogates). The mean surrogate QUEST scores were 4.31 for physician quality, 4.37 for nursing quality, 4.43 for satisfaction with physicians, and 4.42 for satisfaction with nurses.

This study contributes to understanding care at the end of life by asking seriously ill hospitalized patients to rate the quality of their health care and their satisfaction with that care using instruments modified and refined to capture features that patients facing the end of life consider significant. Most available instruments to measure patients' perceptions of quality and satisfaction with care have been developed for contexts other than the end of life.¹⁹ However, some progress is being made. For example, Wenger at al²⁰ are developing measures that can be used in large administrative databases. The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments interviewed family members about satisfaction after the patient's death using 2 unvalidated 4-item scales.²¹ Tolle at al²² interviewed surviving family members many months after patients' deaths. However, the retrospective views of the family are quite limited. We do not know how grief, recall effects, and the need for closure may affect the surrogates' after-death responses.²³ Furthermore, the satisfaction of the family cannot be assumed to represent the satisfaction of the patient.^24,25 Other investigators have concentrated on patients in hospice settings, not in hospitals.²⁶ None have attempted to distinguish patient ratings of satisfaction from patient ratings of quality. Our instrument is also unique in that it emphasizes the interpersonal aspects of care, an element that is especially important to patients at the end of life.²⁷

Our finding that patients rated physicians more highly than nurses is unusual. Although comparisons of patient ratings of physicians and nurses in the same clinical episode are rare, studies in outpatient settings and in emergency departments have suggested higher ratings for nurses.^28-30 It is unclear whether staffing levels³¹ or other factors more specific to end-of-life care in the hospital might play a role. In a study of a surgical ward, a lack of registered nurses during night shifts was associated with lower overall patient satisfaction.³²

Although DNR orders and being cared for by the house-staff service were independently associated with lower ratings of physician quality and less satisfaction with physician care, we found an important interaction. Patients who had DNR orders and were treated by the house-staff service gave their physicians especially low quality ratings compared with all other patients. Why the combination of being a patient of the house-staff service and having a DNR order is associated with lower ratings of quality is uncertain. Patients with DNR orders are very likely to die in the hospital.^12,13 Some observers have suggested that physicians simply lose interest in dying patients,³³ and patients with DNR orders are likely to be the victims of this lack of interest. Others apparently believe that patients with DNR orders are abandoned by the staff,³⁴ and this belief might explain these differences. However, a recent study failed to demonstrate any decrease in the time spent by residents or other staff in the rooms of patients with DNR orders compared with patients without DNR orders.³⁵ These findings do not preclude more subtle differences in quality of care that patients might be able to observe. However, our data suggest that if differences in quality of care between patients with and without DNR orders exist, these differences are only apparent in the house-staff service, since private patients with and without DNR orders gave equivalent QUEST scores to their physicians.

Results of previous studies comparing the satisfaction of patients treated by house-staff services compared with those treated by private services have been mixed. Wells et al³⁶ failed to show any differences between patients cared for by private attending physicians and those cared for by a house-staff service. However Yarnold et a³⁷ found more negative ratings of satisfaction with care at an academic hospital compared with a community hospital. Perhaps a long-term relationship with a private physician mitigates the loss of interest or other factors that decrease the quality of interpersonal care patients with DNR orders otherwise receive.

Although depression and treatment by the house-staff service were associated with lower ratings of the quality of nursing care and satisfaction with nursing care, we also found an important interaction between depression and type of attending physician. Only findings for the model for satisfaction with care were significant, but both models show the same trend, ie, patients with private attending physicians who were not depressed showed high levels of satisfaction with nursing care compared with all other patients.

Satisfaction is a highly complex phenomenon, involving patient expectations and staff behavior.³⁸ The mode of delivery of patient care (house-staff service vs private attending physician) was important in our nursing and physician models. Perhaps the best explanation might be that having a private attending physician was associated with higher ratings of quality and satisfaction with nursing services, but that this effect was eradicated if the patient was depressed. Further explanation is not possible, given these data.

Previous studies have suggested that most variance in patient satisfaction actually depends on the patient, not the practitioner or the health care environment.³⁹ Patients with private attending physicians tend to have a higher socioeconomic status, but, ironically, this makes them harder to please.³⁸ Depression might also change a patient's outlook and expectations, but no consistent correlations between depression and satisfaction in the acute^39,40 or the chronic⁴¹ care settings have been made.

Although we found a trend for quality and satisfaction ratings to correlate negatively with the level of pain, this association was only significant for satisfaction with physicians on the QUEST scales. This association was no longer significant in a multivariate ANOVA that also controlled for DNR status and house-staff vs private care. Our sample size was too small to detect a possible weak association between symptoms and satisfaction. One might speculate that, if it exists, this association should hold only for satisfaction with physicians, not nurses, since physicians write the orders. However, it has been difficult to show a consistent correlation between pain and satisfaction.⁴² Some have speculated that patients have low expectations for pain control, or that satisfaction has far more to do with the quality of interpersonal interactions with staff than the technical correctness of the care rendered.

Severity of illness was also uncorrelated with quality or satisfaction in our study. Quality of life has been shown in acute care settings to correlate with severity of illness,^43,44 but this has not been true in end-of life care, where quality of life can be high despite severe illness.⁴⁵ We are unaware of studies that have directly measured satisfaction with care and severity of illness at the end of life, but can only speculate that patient ratings of quality and satisfaction with care might similarly be high once it is clear that the medical condition cannot be reversed and that death is drawing near.

Overall, surrogate ratings were not very different from patient ratings, with the possible exception that patients tended to give somewhat lower ratings of the quality of nursing care. Since we have previously demonstrated that surrogate QUEST ratings do not validly represent the ratings of the patients,⁸ one must be very cautious in drawing any conclusions from the QUEST scores of the surrogates.

Our findings are observational and should be considered preliminary. Our sample size is relatively small, but our data are drawn from 2 institutions. The QUEST instrument is newly adapted for measuring quality and satisfaction, but it has undergone substantial testing of reliability and validity. Our ability to detect differences in QUEST scores for patients with and without DNR orders and between patients treated by house-staff services and private physicians suggests that the scale is sensitive to differences in quality and satisfaction. However, the instrument's sensitivity has not yet been established in prospective interventional studies. Also, patient ratings of quality and satisfaction, although extremely important, are meant to complement other quality measures such as symptom burden and process measures. The QUEST scale should not be misconstrued as a global quality measure.

Patients' ratings of quality and satisfaction using the QUEST scale vary according to multiple factors. Overall, physicians received higher ratings than nurses. Both physicians and nurses received higher ratings by patients with private attending physicians than by those treated by the house-staff service. Depressed patients expressed more dissatisfaction with nursing care, even those with private attending physicians. Finally, patients who were likely to be very close to death, ie, those with DNR orders, were more likely to be dissatisfied with physician care and to rate the quality of their physicians as low, particularly if treated by the house-staff service. Further study will be required to determine whether these findings are generalizable and, if so, whether they might be amenable to efforts to improve the quality of end-of-life care.

Accepted for publication March 20, 2002.

This study was supported by a Faculty Scholar's Award of the Open Society Institute's Project on Death in America, New York (Dr Sulmasy), and by a generous grant from the Altman Foundation, New York.

We thank Vic Tolentino, MPH, JD, Sr Grace Henke, SC, EdD, and Maike Rahn, MA, for their assistance with this project.

Corresponding author: Daniel P. Sulmasy, OFM, MD, PhD, The John J. Conley Department of Ethics, St Vincent Catholic Medical Centers, St Vincent's Manhattan, 153 W 11th St, New York, NY 10011 (e-mail: daniel_sulmasy@nymc.edu).