[Skip to Navigation]
Sign In
Table 1.  Sample Quotes of Racist Experiences and Associated Coded Themes
Sample Quotes of Racist Experiences and Associated Coded Themes
Table 2.  Themes Associated With Interpersonal Racism by Identified Perpetrator Categories
Themes Associated With Interpersonal Racism by Identified Perpetrator Categories
1.
Moy  E, Dayton  E, Clancy  CM.  Compiling the evidence: the National Healthcare Disparities Reports.   Health Aff (Millwood). 2005;24(2):376-387. doi:10.1377/hlthaff.24.2.376PubMedGoogle ScholarCrossref
2.
Adkins-Jackson  PB, Legha  RK, Jones  KA.  How to measure racism in academic health centers.   AMA J Ethics. 2021;23(2):E140-E145. doi:10.1001/amajethics.2021.140PubMedGoogle Scholar
3.
Ranard  BL, Werner  RM, Antanavicius  T,  et al.  Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.   Health Aff (Millwood). 2016;35(4):697-705. doi:10.1377/hlthaff.2015.1030PubMedGoogle ScholarCrossref
4.
Ryskina  KL, Andy  AU, Manges  KA, Foley  KA, Werner  RM, Merchant  RM.  Association of online consumer reviews of skilled nursing facilities with patient rehospitalization rates.   JAMA Netw Open. 2020;3(5):e204682. doi:10.1001/jamanetworkopen.2020.4682PubMedGoogle Scholar
5.
Jones  CP.  Levels of racism: a theoretic framework and a gardener’s tale.   Am J Public Health. 2000;90(8):1212-1215. doi:10.2105/AJPH.90.8.1212PubMedGoogle Scholar
Research Letter
Diversity, Equity, and Inclusion
September 22, 2021

Evaluation of Online Consumer Reviews of Hospitals and Experiences of Racism Using Qualitative Methods

Author Affiliations
  • 1National Clinician Scholars, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
  • 2Center for Digital Health, University of Pennsylvania, Philadelphia
  • 3Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
  • 4Department of Surgery, Center for Surgery and Health Economics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
JAMA Netw Open. 2021;4(9):e2126118. doi:10.1001/jamanetworkopen.2021.26118
Introduction

Racial disparities, the result of structural and interpersonal racism, represent a complex phenomenon present in all domains of health care.1,2 Although techniques to measure the quantitative impact of structural racism on racial disparities exist, the measurement of interpersonal racism is limited in health care because of its subjective nature. The inability to measure interpersonal racism at the local level has limited the ability to improve racist patient experiences within health care. Prior work has demonstrated consumer reviews’ unique ability to highlight novel concepts not captured in traditional performance metrics and to impact consumer hospital selection.3 Consumer reviews offer an opportunity to understand subjective perceptions of racism in health care in an unstructured and anonymous format. To demonstrate the potential role of consumer reviews in studying interpersonal racism in health care, we explored reviews of hospitals to better understand how consumers perceive and report racism.

Methods

This qualitative study was deemed exempt by the University of Pennsylvania institutional review board and informed consent was not required because of the public and retrospective nature of the consumer reviews. This study followed the Strengthening the Standards for Reporting Qualitative Research (SRQR) reporting guideline.

A study of Yelp consumer reviews on United States hospitals published between January 2010 to January 2020 was performed. Yelp was selected because it is the most widely used referral website in the world, is frequently updated, and screens out potentially falsified reviews to prevent skewed ratings.4 Natural language processing was used to identify all reviews containing the terms “racist” or “racism.” All collected data were publicly available and no attempts at contacting reviewers or ascertaining reviewer gender, race, or ethnicity were made. Prior work has offered a conceptual framework delineating racism into distinct levels including institutional, interpersonal, and internalized.5 A random sample of reviews were analyzed to understand the content, followed by a formal qualitative content analysis to code recurrent themes and unique levels of racism. Episodes of racism were further classified into clinical and nonclinical environments. Content code queries were used to better understand the associations between identified codes.

Results

During the study period, 90 786 online consumer reviews of US hospitals were obtained. Reviewer demographics were intentionally left anonymous for the purposes of this study. Of all reviews obtained, 260 reviews explicitly cited racism in 190 hospitals spread across 33 states. See sample reviews in Table 1. Among these, 179 reviews cited individual perpetrators of interpersonal racism, of which physicians (31% [56 of 179]) and nurses (53% [94 of 179]) were mentioned most. Interpersonal racism was associated with recurring themes including disrespect and unprofessionalism, incompetence, and acts of commission or omission. The most common themes included disrespect and unprofessionalism and acts of omission. See Table 2 for relative frequency of themes associated with each perpetrator category. There were 68 mentions of institutional racism, which were most frequently described as omitting standard care (26%; 18 of 68) and disrespectful (24%; 16 of 68). Episodes of racism were 2.5 times more frequently associated with clinical encounters (143 of 260) compared with nonclinical environments (58 of 260). Racism within nonclinical spaces occurred most commonly during interactions with receptionists and security guards. Nine reviews demonstrated internalized racist behavior by the consumer, primarily directed at nurses.

Discussion

This qualitative study found that (1) it is feasible to identify acts of interpersonal racism in health care using qualitative methods on consumer reviews, (2) racism in health care can involve critical organization personnel outside of the clinical staff, and (3) racism can be bidirectional, affecting both patients and hospital employees. These detailed reviews likely represent the tip of the iceberg, and future efforts to supplement this data with existing hospital-based reporting measures, such as human resources reports, may help to further explore concepts such as patient-to-employee racism. Additionally, experiences of interpersonal racism are just one aspect of racism in health care, and future work is needed to couple the subjective measures of interpersonal racism with objective measures of structural racism to establish a hospital quality composite metric of racism. This study’s findings are limited by the low frequency of reviews citing racism, limited information on reviewer race, and additional context about reported experiences.

Back to top
Article Information

Accepted for Publication: July 19, 2021.

Published: September 22, 2021. doi:10.1001/jamanetworkopen.2021.26118

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Tong J et al. JAMA Network Open.

Corresponding Author: Jason Tong, MD, National Clinician Scholars, Corporal Michael J. Crescenz Veterans Affairs Medical Center, 3400 Spruce St, 4 Maloney, Philadelphia, PA 19104 (jason.tong@pennmedicine.upenn.edu).

Author Contributions: Dr Tong had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: Tong, Andy, Merchant.

Drafting of the manuscript: Tong, Andy, Merchant.

Critical revision of the manuscript for important intellectual content: Tong, Merchant, Kelz.

Statistical analysis: Tong, Andy.

Supervision: Merchant, Kelz.

Conflict of Interest Disclosures: None reported.

References
1.
Moy  E, Dayton  E, Clancy  CM.  Compiling the evidence: the National Healthcare Disparities Reports.   Health Aff (Millwood). 2005;24(2):376-387. doi:10.1377/hlthaff.24.2.376PubMedGoogle ScholarCrossref
2.
Adkins-Jackson  PB, Legha  RK, Jones  KA.  How to measure racism in academic health centers.   AMA J Ethics. 2021;23(2):E140-E145. doi:10.1001/amajethics.2021.140PubMedGoogle Scholar
3.
Ranard  BL, Werner  RM, Antanavicius  T,  et al.  Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.   Health Aff (Millwood). 2016;35(4):697-705. doi:10.1377/hlthaff.2015.1030PubMedGoogle ScholarCrossref
4.
Ryskina  KL, Andy  AU, Manges  KA, Foley  KA, Werner  RM, Merchant  RM.  Association of online consumer reviews of skilled nursing facilities with patient rehospitalization rates.   JAMA Netw Open. 2020;3(5):e204682. doi:10.1001/jamanetworkopen.2020.4682PubMedGoogle Scholar
5.
Jones  CP.  Levels of racism: a theoretic framework and a gardener’s tale.   Am J Public Health. 2000;90(8):1212-1215. doi:10.2105/AJPH.90.8.1212PubMedGoogle Scholar
×