Associations Between Web-Based Patient Ratings and Objective Measures of Hospital Quality

Patients are increasingly using the Internet to rate their experience of health care.¹ Although controversial, particularly among health care professionals,² Web-based hospital or physician rating sites represent a potentially important development in public reporting.³ Patients rarely use conventional publicly reported metrics when choosing their health care provider and often find these difficult to understand.⁴

In 2008, the English National Health Service (NHS) established a Web site called NHS Choices that allows patients to rate their experiences with health care providers. Patients can rate several dimensions of quality on simple scales and leave comments on particular aspects of care in free text. As of October 2011, Medicare has not chosen to include patient comments and ratings, despite recent calls for their introduction.⁵

Although some prior research has shown an association between results on formal patient satisfaction surveys and clinical measures of quality,⁶ to our knowledge, no previous study has examined the relationship between unsolicited ratings by patients and objective measures of quality. The degree to which such ratings are associated with objective clinical outcomes and conventional patient surveys may well determine their place in the landscape of transparency measures. In addition, decisions by policy makers and payers to support such Web sites might well hinge on whether such associations exist. This study examines hospital-level associations between Web-based patient ratings on the NHS Choices Web site and objective measures of quality.

We performed a cross-sectional observational study of all (n = 166) NHS acute hospital trusts in England, using data from 10 274 patient Web-based ratings posted on the NHS Choices Web site from January 1, 2009, to December 31, 2010, obtained from the English Department of Health. Hospital trust–level indicators of clinical outcomes (5 measures) and health care-acquired infections (2 measures) were compared with Web-based patient ratings using the Spearman rank correlation coefficient. When comparing patient ratings with infection rates, we used all 166 acute care hospital trusts in England. When comparing ratings with other clinical outcomes, a reduced list of 146 hospital trusts was used that excluded trusts that only care for children or specific specialties (eg, oncology). Data on clinical outcomes used were for financial year 2009/2010, and obtained from Dr Foster (a health intelligence company) and NHS Comparators (the NHS's comparative health system performance service). We obtained data on methicillin-resistant Staphylococcus aureus (MRSA) bacteremia and Clostridium difficile infection rates from the Health Protection Agency compulsory national surveillance system.

The mean and median number of ratings per hospital was 62 and 46, respectively (range, 1-290). Of those who offered a view about recommending the hospital (9349 ratings), 68.0% (n = 6354) would recommend the hospital to a friend. Positive recommendations of hospitals were significantly associated with lower hospital standardized mortality ratios (Spearman ρ = −0.20; P = .01), lower mortality from high-risk conditions (ρ = −0.23; P = .01), and lower readmission rates (ρ = −0.31; P < .001) (Table). Positive recommendations were not associated with mortality rates among surgical inpatients with serious treatable complications (ρ = 0.00; P = .99) or mortality from low-risk conditions (ρ = 0.03; P = .70). Mean and median ratings of hospital cleanliness were 3.6 (range, 2.6-5.0) on a scale from 1 (dirty) to 5 (exceptionally clean). Better ratings of hospital cleanliness were associated with lower MRSA (ρ = −0.30, P < .001) and C difficile (ρ = −0.16; P = .04) infection rates.

Our results demonstrate a relationship between patients' Web site ratings of hospitals and some objective measures of clinical quality, including mortality and infection rates.

Arguments in favor of the use of patient Web site reporting include that physicians can be poor judges of their patients' experience, that feedback changes physicians' performance, and people will inevitably use the Internet to voice opinions, so why not capture this information in a useful form.³ Arguments against using this data include the selection bias by those leaving reviews, the lack of meaningful data on technical quality of health care, and straining of physician-patient relationships.^2,5 Although our results do not counter all of these arguments against, they suggest that discretionary patient ratings, obtained through a Web site, may be a more useful tool than previously considered for both patients and health care workers. If patients are making choices based on this information, they can be reassured that the ratings are not entirely misleading and may be providing relevant information about health care quality. In his book The Wisdom of Crowds, James Surowiecki⁷ argues that a diverse collection of “independently deciding individuals” is likely to make better predictions and decisions than single individuals or even experts. At least to an extent, the self-selecting crowd of patients appears to be wise.

The use of Web-based patient ratings has become common in other industries such as hotels and restaurants, and consumers value these rankings in making choices. We believe that the information provided by these Web sites, although flawed, represents a potentially important development in the measurement of health care quality.

Correspondence: Dr Greaves, Department of Primary Care and Public Health, Reynolds Building, Charing Cross Campus, Imperial College London, London W6 8RP, England (felix.greaves08@imperial.ac.uk).

Published Online: February 13, 2012. doi:10.1001/archinternmed.2011.1675

Author Contributions: All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Greaves and Millett. Acquisition of data: Greaves. Analysis and interpretation of data: Greaves, Pape, King, Darzi, Majeed, Wachter, and Millett. Drafting of the manuscript: Greaves, King, and Millett. Critical revision of the manuscript for important intellectual content: Pape, Darzi, Majeed, and Wachter. Statistical analysis: Greaves and Pape. Obtained funding: Majeed. Administrative, technical, and material support: King, Darzi, and Millett. Study supervision: Darzi, Majeed, and Millett.

Funding/Support: Dr Wachter is a 2011 recipient of a Fulbright Award for study in the United Kingdom. Dr Millett is funded by the Higher Education Funding Council for England and the National Institute for Health Research.

Additional Information: Dr Darzi was Parliamentary Under-Secretary of State (Lords) in the United Kingdom Department of Health from 2007 to 2009.

Additional Contributions: We thank the team at NHS Choices, and John Robinson in particular, for providing access to their data. The Department of Primary Care & Public Health at Imperial College is grateful for support from the National Institute for Health Research Biomedical Research Centre Funding scheme, the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care scheme, and the Imperial Centre for Patient Safety and Service Quality.