Lyu H, Wick EC, Housman M, Freischlag JA, Makary MA. Patient Satisfaction as a Possible Indicator of Quality Surgical Care. JAMA Surg. 2013;148(4):362-367. doi:10.1001/2013.jamasurg.270
Author Affiliations: Department of Surgery, The Johns Hopkins University School of Medicine (Ms Lyu and Drs Wick, Freischlag, and Makary), and Department of Health Policy and Management, The Johns Hopkins School of Public Health (Dr Makary), Baltimore, Maryland; and Pascal Metrics Inc, Washington, DC (Dr Housman).
Importance In 2010, national payers announced they would begin using patient satisfaction scores to adjust reimbursements for surgical care.
Objective To determine whether patient satisfaction is independent from surgical process measures and hospital safety.
Design We compared the performance of hospitals that participated in the Patient Satisfaction Survey, the Centers for Medicare & Medicaid Services Surgical Care Improvement Program, and the employee Safety Attitudes Questionnaire.
Setting Thirty-one US hospitals.
Participants Patients and hospital employees.
Interventions There were no interventions for this study.
Main Outcomes and Measures Hospital patient satisfaction scores were compared with hospital Surgical Care Improvement Program compliance and hospital employee safety attitudes (safety culture) scores during a 2-year period (2009-2010). Secondary outcomes were individual domains of the safety culture survey.
Results Patient satisfaction was not associated with performance on process measures (antibiotic prophylaxis, R = −0.216 [P = .24]; appropriate hair removal, R = −0.012 [P = .95]; Foley catheter removal, R = −0.089 [P = .63]; deep vein thrombosis prophylaxis, R = 0.101 [P = .59]). In addition, patient satisfaction was not associated with a hospital's overall safety culture score (R = 0.295 [P = .11]). We found no association between patient satisfaction and the individual culture domains of job satisfaction (R = 0.327 [P = .07]), working conditions (R = 0.191 [P = .30]), or perceptions of management (R = 0.223 [P = .23]); however, patient satisfaction was associated with the individual culture domains of employee teamwork climate (R = 0.439 [P = .01]), safety climate (R = 0.395 [P = .03]), and stress recognition (R = −0.462 [P = .008]).
Conclusions and Relevance Patient satisfaction was independent of hospital compliance with surgical processes of quality care and with overall hospital employee safety culture, although a few individual domains of culture were associated. Patient satisfaction may provide information about a hospital's ability to provide good service as a part of the patient experience; however, further study is needed before it is applied widely to surgeons as a quality indicator.
Quiz Ref IDIn 2011, the Centers for Medicare & Medicaid Services finalized details for a new reimbursement method that would adjust payments based on patient satisfaction scores—a trend that is also being adopted by private insurers.1 This new policy reflects the perception that patient satisfaction is an indicator of health care quality. Although this metric is easy to apply, making it highly attractive to payers, it may not be a comprehensive or even a reliable metric of quality, particularly for procedure-based medical care. Despite the widespread popularity of consumer satisfaction in other industries, patient satisfaction has not been evaluated as a metric of quality medical care. In fact, the ability of patient satisfaction scores to evaluate technical quality, particularly in the operating room, has been questioned for many reasons.2
Quiz Ref IDSafety culture is increasingly recognized as a marker of safe practices on a local level. It measures the perception by health care workers of employee teamwork and satisfaction in the setting in which they work.3 A preliminary multicenter study has demonstrated that certain safety attitude scores were associated with patient outcomes.4 Specifically, poor perceptions of management were associated with higher hospital mortality, and low employee scores with regard to safety climate, perceptions of management, and job satisfaction were associated with increased hospital length of stay.4 Other studies that examined the relationship between the climate of patient safety and hospital outcomes have demonstrated associations between individual domains of safety attitudes and coordination of care, communication, and quality of care.5- 7Quiz Ref IDWe designed a study to evaluate the association between patient satisfaction and quality of care as defined by compliance with evidence-based processes of surgical care and employee attitudes of safety.
We conducted a cohort study using Hospital Consumer Assessment of Healthcare Providers and Systems Survey (HCAHPS), the Centers for Medicare & Medicaid Surgical Care Improvement Program (SCIP), and the employee Safety Attitudes Questionnaire (SAQ). Primary outcomes were process measure compliance and safety attitudes at the hospital level.
Patient satisfaction was collected from the HCAHPS (http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalHCAHPS.html). The HCAHPS includes 27 questions focused on 18 patient perspectives regarding care and rating items. The questions are focused in the following 8 areas: communication with physicians, communication with nurses, responsiveness of hospital staff, pain management, communication about medicines, discharge information, hospital cleanliness, and noise control. The data in this study are based on the results of the question “Using any number from 0 to 10, where 0 is the worst hospital possible and 10 is the best hospital possible, what number would you use to rate this hospital during your stay?” Scores are expressed as the percentage of respondents reporting a favorable score (9 or 10) at a hospital level. As a secondary analysis, we also evaluated the specific question “Would you recommend this hospital to your friends and family?” We used a global score for patient satisfaction; however, we found that the individual survey item scores correlated with the hospital's global score. The correlation scores between the patient satisfaction score that we used (patients who gave their hospital a rating of 9 or 10 on a scale from 0 [lowest] to 10 [highest]) and all other patient satisfaction scores from the HCAHPS ranged from 0.63 to 0.90. All questions from the HCAHPS are listed in Table 1.
The 2009 and 2010 process-of-care measures included in the study were outpatient and inpatient antibiotic prophylaxis, hair removal, Foley catheter removal, and deep vein thrombosis prophylaxis. Compliance rates were abstracted from the Centers for Medicare & Medicaid Services public reporting website (http://www.medicare.gov/hospitalcompare/About/WhatIs/What-Is-HOS.aspx).
We obtained data on hospital safety culture from US hospitals that participated in the publicly reported metrics and the SAQ, a standardized survey of safety culture (Table 2). The SAQ was adapted from the Flight Management Attitudes Questionnaire and has been validated and widely adopted.8- 10 The SAQ measures 6 domains, including teamwork climate, safety climate, job satisfaction, perceptions of facility and local management, stress recognition, and working conditions. The SAQ was administered to all caregivers in each hospital's Department of Surgery, including nurses, physicians, and technicians. To ensure a high response rate, surveys were made available through multiple different methods. First, paper surveys were packaged and shipped to each hospital and then distributed to care providers by unit. Second, unit members were given a website along with a web token to take the survey online. Third, respondents were assigned personalized tokens using the last 4 digits of their social security number to access the online surveys. These surveys were assigned to ensure respondent anonymity and accuracy regarding unit assignments. A protocol for close follow-up was administered by Pascal Metrics, Inc to confirm participation. All hospitals included in this study had a response rate of 60% or greater. The safety culture score, overall and for individual domains, is expressed as a percentage of respondents reporting a mean score of 4 or 5 at a hospital level (ie, positive score). An overall mean safety culture score for each hospital was calculated by averaging all the individual domain scores.
We estimated relationships between process measures and patient satisfaction percentages using log-regression models in which patient satisfaction was the independent variable and process-of-care measures were the dependent variables. We used the same method with patient satisfaction and safety culture values, with patient satisfaction serving as the independent variable. We extended the model to include individual culture domains, including teamwork climate, safety climate, job satisfaction, stress recognition, working conditions, and perceptions of management. The relationship between patient satisfaction and process-of-care measures was assessed using the Pearson coefficient extracted from log regression models. The R value indicates the relationship between the independent and dependent variables, with 1 indicating a linear, direct relationship. All statistical analyses were performed using commercially available software (STATA; StataCorp), assuming statistical significance at P < .05.
Thirty-one hospitals in 10 states were identified as having all data sources available. Equal proportions of small (<250 beds), medium (250-500 beds), and large (>500 beds) hospitals were observed. All were urban hospitals and most were academic centers (Table 3).
Patient satisfaction scores varied widely across the surveyed hospitals. Patient overall satisfaction scores ranged from 47% to 83%, with a mean of 67.5% (Table 4).
Process-of-care measures did not vary as much as patient satisfaction scores among the hospitals studied. Outpatient prophylactic antibiotic use varied the most, with scores ranging from 61% to 99% (mean, 87.1%). Scores for inpatient prophylactic antibiotic selection varied the least, with a range of 93% to 99% (mean, 97.9%). Means and ranges for process-of-care measures are listed in Table 5. Safety culture scores varied widely across the hospitals (range of positive scores, 15.8%-86.7%). The mean for overall safety culture was 52.3% (range of positive scores, 33.1%-74.9%). Of the 6 individual domains, perceptions of local management and teamwork climate varied the most across hospitals (range of positive scores, 24.2%-86.7%), whereas stress recognition varied the least (25.9%-71.0%). The mean score for safety climate was 60.5% (range of positive scores, 25.4%-85.0%); for job satisfaction, 61.6% (28.1%-80.8%); for working conditions, 43.3% (24.3%-74.2%); for perceptions of local management, 56.8% (36.5%-86.7%); and for perceptions of facility management, 33.5% (15.8%-67.5%).
Quiz Ref IDPatient satisfaction was not associated with hospital compliance with process-of-care measures. The Pearson coefficient value for patient satisfaction and antibiotic prophylaxis was −0.216 (P = .24); for hair removal, −0.012 (P = .95); for Foley catheter removal, −0.089 (P = .63), and for deep vein thrombosis prophylaxis, 0.101 (P = .59).
Quiz Ref IDThe Pearson coefficient value between overall culture scores and patient satisfaction scores was 0.295 (P = .11). Teamwork climate and safety climate had mild but positive correlation with patient satisfaction scores, with R values of 0.439 (P = .01) and 0.395 (P = .03), respectively. The correlation trend continued for job satisfaction (R = 0.327 [P = .07]), working conditions (R = 0.191 [P = .30]), and perceptions of management (R = 0.223 [P = 23]). Only stress recognition had an inverse relationship with patient satisfaction scores (R = −0.462 [P = .008]).
In January of 2012, the US Department of Health and Human Services announced its final list of core health care quality measures for Medicare-eligible adults. The list was constructed using the following criteria: importance, scientific evidence, scientific soundness, alignment with existing programs, and feasibility for state reporting. “Family experiences of care” is included as 1 of the 26 quality measures in the final list, measured by the patient satisfaction version of the Consumer Assessment of Healthcare Providers and Systems Survey. In 2007, the Joint Commission for Accreditation of Healthcare Organizations determined that an annual assessment of safety culture should be included in its patient safety goals.11,12
We applaud the introduction of patient satisfaction as a metric of health care, recognizing that satisfaction measures the important customer service component of a patient's experience.13 Accordingly, patient satisfaction inquiries are limited to particular aspects of a patient's interface with a health care institution, that is, the physical plant, scheduling ease, and responsiveness of the hospital employees. These important services may be surrogates of a well-coordinated and patient-centered level of service.14 However, the use of patient satisfaction as a comprehensive measure of quality to determine hospital reimbursement must be questioned in the absence of other reliable outcome metrics. Based on our findings, we specifically challenge the notion that a patient's level of satisfaction reported in isolation of other surgical outcome metrics reflects the quality of a surgeon's procedure or the perioperative expertise associated with their care.
Previous studies on the relationship between patients' experiences and the quality of clinical care have had mixed results, and many have failed to show any relationship.15 Several studies have found a positive relationship between patients' experiences and the quality of clinical care in US hospitals using data from the patient satisfaction version of the Consumer Assessment of Healthcare Providers and Systems Survey and national hospital ratings.15,16 Another study found that patient satisfaction may be correlated with poor health outcomes.17 Specifically, patients who are most satisfied with their health care receive more prescription medications, visit a physician more often, have extended hospital stays, and subsequently have increased mortality.18 Although this study may have been influenced by a selection bias toward patients seeking medical care, it speaks to the challenge in using patient satisfaction as a marker of overall medical quality. Conversely, patient satisfaction is an important metric of a hospital service and the administrative processes required to deliver care. We conclude that elevating patient satisfaction as a comprehensive surrogate of quality medical care can be misleading if used in isolation of other important quality metrics. At the time we completed this study, patient satisfaction was the only outcome measure publicly available for many hospital services on the Hospital Compare website.
Our analysis suggests that patient satisfaction is not related to standard process-of-care measures that have long been used as markers of surgical quality. Process-of-care measures have been accepted widely by clinicians and surgeons because they demonstrate clearly how providers can improve outcomes.19 Various explanations for this illustrated lack of relationship between patient satisfaction and process-of-care measures have been elucidated. Patient satisfaction has been defined as a “subjective” measure by Ware et al20 and as “a health care recipient's reaction to salient aspects of the context, process, and results of their service experience” by Pascoe.21 Despite various attempts and studies to quantify and objectify patient satisfaction, the vast number of potential dimensions underlying these measures and the limited availability of universal and standardized instruments in the field make it difficult to use patient satisfaction as an “outcome” measure.14,22 Moreover, patient satisfaction at times may not be driven by assessment of the entire experience but rather by the patients' feelings and emotions at the moment of surveillance.2 This failure of patient satisfaction measures to appreciate improvement may reward institutions with less severe cases, whereas larger hospitals with patients who have worse health at baseline are likely to lose compensation. Patient satisfaction may even serve as a marker for illness because those patients who rely more on their physicians for support relative to other, possibly healthier, patients may report higher satisfaction.18 Furthermore, Fenton et al18 reported an inverse association between patient satisfaction scores and mortality. Several studies concluded that this finding may have been a result of an emphasis on patient satisfaction negatively influencing physicians to alter decision making about health care to please patients rather than adhere to evidence-based care.18,23 We do not mean that the measurement of patient satisfaction is unnecessary or irrelevant; rather, it is not a reliable assessment of total health care quality in surgery.
Our study showed a correlation between patient satisfaction scores and the following 2 domains of culture: safety climate and teamwork climate. Although the correlation values for both were weak, further study may support a role for satisfaction as a measure of safety via increased communication and teamwork. The perception of these domains by the patient may be a valuable surrogate of other good safety culture characteristics at a hospital.
Although patient satisfaction should be measured to add accountability to hospital services, its reliability as a safety measure awaits further study. At minimum, patient satisfaction should be considered in the context of other proven metrics of hospital outcomes, such as the American College of Surgeons National Surgical Quality Improvement Program, which will be added to the public reporting website in 2012. We caution the use of patient satisfaction as a large component of hospital reimbursement formulas before its contribution to the determination of quality in surgery is better understood. At present, little evidence supports its ability to predict the quality of surgical care. This finding is similar in the airline industry, where the quality of a passenger's experience may not correlate with the safety of the flight or the safety of the pilots. Although patient satisfaction might play a role in reimbursement, it would be best labeled as a service quality indicator.
This study has several important limitations. First, we used a global score for hospital-wide patient satisfaction because surgery-specific satisfaction scores were not available. The Agency for Healthcare Research and Quality has encouraged a new surgery-specific patient satisfaction survey; when those results are available, they will provide more surgery-specific data for future research. Second, the present study reports findings from 31 urban hospitals that may not be representative of other hospitals. Sampling bias is possible because the hospitals that are included in this study chose to participate in the SAQ. Third, process measures are not a definitive criterion standard for quality of care. The SCIP measures have been shown to be flawed and likely represent only a small fraction of overall quality of care in surgery.24,25 The SCIP measures may be a surrogate marker of quality; however, some studies have shown that they do not correlate with patient outcomes. Despite those findings, variation in SCIP compliance may represent varying prioritization of quality at the hospital level. Finally, this study is a preliminary analysis of patient satisfaction and its role in surgical outcomes. Future metrics are needed to better understand the interaction between patient satisfaction and actual outcomes.
Based on our findings, we suggest that patient satisfaction be measured in the context of other metrics of surgical quality. We have identified the importance of patient satisfaction as well as the limitations of this metric. This study may inform new reimbursement policies being widely adopted and lead to more studies to further define the relationship between patient satisfaction and surgical outcomes. More reliable metrics of surgical outcome soon may be publicly available for hospitals. Studies defining the relationship between patient satisfaction and future outcomes data may confirm the findings of our study. This transparency will better inform patients seeking care and provide an important context for patient satisfaction scores—currently the only outcome listed for surgical services at the Hospital Compare website.
Correspondence: Martin A. Makary, MD, MPH, Department of Surgery, The Johns Hopkins University School of Medicine, Osler 624, 600 N Wolfe St, Baltimore, MD 21287 (firstname.lastname@example.org).
Accepted for Publication: August 2, 2012.
Author Contributions:Study concept and design: Lyu, Wick, and Makary. Acquisition of data: Lyu, Wick, Housman, and Makary. Analysis and interpretation of data: Lyu, Wick, and Freischlag. Drafting of the manuscript: Lyu, Wick, Housman, and Makary. Critical revision of the manuscript for important intellectual content: Lyu, Wick, and Freischlag. Statistical analysis: Lyu and Housman. Obtained funding: Makary. Administrative, technical, and material support: Wick. Study supervision: Wick and Makary.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by Mr and Mrs Chad and Nissa Richison (Dr Makary).