Evaluation of Online Consumer Reviews of Hospitals and Experiences of Racism Using Qualitative Methods | Diversity, Equity, and Inclusion | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Table 1.  Sample Quotes of Racist Experiences and Associated Coded Themes
Sample Quotes of Racist Experiences and Associated Coded Themes
Table 2.  Themes Associated With Interpersonal Racism by Identified Perpetrator Categories
Themes Associated With Interpersonal Racism by Identified Perpetrator Categories
1.
Moy  E, Dayton  E, Clancy  CM.  Compiling the evidence: the National Healthcare Disparities Reports.   Health Aff (Millwood). 2005;24(2):376-387. doi:10.1377/hlthaff.24.2.376PubMedGoogle ScholarCrossref
2.
Adkins-Jackson  PB, Legha  RK, Jones  KA.  How to measure racism in academic health centers.   AMA J Ethics. 2021;23(2):E140-E145. doi:10.1001/amajethics.2021.140PubMedGoogle Scholar
3.
Ranard  BL, Werner  RM, Antanavicius  T,  et al.  Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.   Health Aff (Millwood). 2016;35(4):697-705. doi:10.1377/hlthaff.2015.1030PubMedGoogle ScholarCrossref
4.
Ryskina  KL, Andy  AU, Manges  KA, Foley  KA, Werner  RM, Merchant  RM.  Association of online consumer reviews of skilled nursing facilities with patient rehospitalization rates.   JAMA Netw Open. 2020;3(5):e204682. doi:10.1001/jamanetworkopen.2020.4682PubMedGoogle Scholar
5.
Jones  CP.  Levels of racism: a theoretic framework and a gardener’s tale.   Am J Public Health. 2000;90(8):1212-1215. doi:10.2105/AJPH.90.8.1212PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,354
    Citations 0
    Research Letter
    Diversity, Equity, and Inclusion
    September 22, 2021

    Evaluation of Online Consumer Reviews of Hospitals and Experiences of Racism Using Qualitative Methods

    Author Affiliations
    • 1National Clinician Scholars, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
    • 2Center for Digital Health, University of Pennsylvania, Philadelphia
    • 3Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
    • 4Department of Surgery, Center for Surgery and Health Economics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
    JAMA Netw Open. 2021;4(9):e2126118. doi:10.1001/jamanetworkopen.2021.26118
    Introduction

    Racial disparities, the result of structural and interpersonal racism, represent a complex phenomenon present in all domains of health care.1,2 Although techniques to measure the quantitative impact of structural racism on racial disparities exist, the measurement of interpersonal racism is limited in health care because of its subjective nature. The inability to measure interpersonal racism at the local level has limited the ability to improve racist patient experiences within health care. Prior work has demonstrated consumer reviews’ unique ability to highlight novel concepts not captured in traditional performance metrics and to impact consumer hospital selection.3 Consumer reviews offer an opportunity to understand subjective perceptions of racism in health care in an unstructured and anonymous format. To demonstrate the potential role of consumer reviews in studying interpersonal racism in health care, we explored reviews of hospitals to better understand how consumers perceive and report racism.

    Methods

    This qualitative study was deemed exempt by the University of Pennsylvania institutional review board and informed consent was not required because of the public and retrospective nature of the consumer reviews. This study followed the Strengthening the Standards for Reporting Qualitative Research (SRQR) reporting guideline.

    A study of Yelp consumer reviews on United States hospitals published between January 2010 to January 2020 was performed. Yelp was selected because it is the most widely used referral website in the world, is frequently updated, and screens out potentially falsified reviews to prevent skewed ratings.4 Natural language processing was used to identify all reviews containing the terms “racist” or “racism.” All collected data were publicly available and no attempts at contacting reviewers or ascertaining reviewer gender, race, or ethnicity were made. Prior work has offered a conceptual framework delineating racism into distinct levels including institutional, interpersonal, and internalized.5 A random sample of reviews were analyzed to understand the content, followed by a formal qualitative content analysis to code recurrent themes and unique levels of racism. Episodes of racism were further classified into clinical and nonclinical environments. Content code queries were used to better understand the associations between identified codes.

    Results

    During the study period, 90 786 online consumer reviews of US hospitals were obtained. Reviewer demographics were intentionally left anonymous for the purposes of this study. Of all reviews obtained, 260 reviews explicitly cited racism in 190 hospitals spread across 33 states. See sample reviews in Table 1. Among these, 179 reviews cited individual perpetrators of interpersonal racism, of which physicians (31% [56 of 179]) and nurses (53% [94 of 179]) were mentioned most. Interpersonal racism was associated with recurring themes including disrespect and unprofessionalism, incompetence, and acts of commission or omission. The most common themes included disrespect and unprofessionalism and acts of omission. See Table 2 for relative frequency of themes associated with each perpetrator category. There were 68 mentions of institutional racism, which were most frequently described as omitting standard care (26%; 18 of 68) and disrespectful (24%; 16 of 68). Episodes of racism were 2.5 times more frequently associated with clinical encounters (143 of 260) compared with nonclinical environments (58 of 260). Racism within nonclinical spaces occurred most commonly during interactions with receptionists and security guards. Nine reviews demonstrated internalized racist behavior by the consumer, primarily directed at nurses.

    Discussion

    This qualitative study found that (1) it is feasible to identify acts of interpersonal racism in health care using qualitative methods on consumer reviews, (2) racism in health care can involve critical organization personnel outside of the clinical staff, and (3) racism can be bidirectional, affecting both patients and hospital employees. These detailed reviews likely represent the tip of the iceberg, and future efforts to supplement this data with existing hospital-based reporting measures, such as human resources reports, may help to further explore concepts such as patient-to-employee racism. Additionally, experiences of interpersonal racism are just one aspect of racism in health care, and future work is needed to couple the subjective measures of interpersonal racism with objective measures of structural racism to establish a hospital quality composite metric of racism. This study’s findings are limited by the low frequency of reviews citing racism, limited information on reviewer race, and additional context about reported experiences.

    Back to top
    Article Information

    Accepted for Publication: July 19, 2021.

    Published: September 22, 2021. doi:10.1001/jamanetworkopen.2021.26118

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Tong J et al. JAMA Network Open.

    Corresponding Author: Jason Tong, MD, National Clinician Scholars, Corporal Michael J. Crescenz Veterans Affairs Medical Center, 3400 Spruce St, 4 Maloney, Philadelphia, PA 19104 (jason.tong@pennmedicine.upenn.edu).

    Author Contributions: Dr Tong had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: All authors.

    Acquisition, analysis, or interpretation of data: Tong, Andy, Merchant.

    Drafting of the manuscript: Tong, Andy, Merchant.

    Critical revision of the manuscript for important intellectual content: Tong, Merchant, Kelz.

    Statistical analysis: Tong, Andy.

    Supervision: Merchant, Kelz.

    Conflict of Interest Disclosures: None reported.

    References
    1.
    Moy  E, Dayton  E, Clancy  CM.  Compiling the evidence: the National Healthcare Disparities Reports.   Health Aff (Millwood). 2005;24(2):376-387. doi:10.1377/hlthaff.24.2.376PubMedGoogle ScholarCrossref
    2.
    Adkins-Jackson  PB, Legha  RK, Jones  KA.  How to measure racism in academic health centers.   AMA J Ethics. 2021;23(2):E140-E145. doi:10.1001/amajethics.2021.140PubMedGoogle Scholar
    3.
    Ranard  BL, Werner  RM, Antanavicius  T,  et al.  Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care.   Health Aff (Millwood). 2016;35(4):697-705. doi:10.1377/hlthaff.2015.1030PubMedGoogle ScholarCrossref
    4.
    Ryskina  KL, Andy  AU, Manges  KA, Foley  KA, Werner  RM, Merchant  RM.  Association of online consumer reviews of skilled nursing facilities with patient rehospitalization rates.   JAMA Netw Open. 2020;3(5):e204682. doi:10.1001/jamanetworkopen.2020.4682PubMedGoogle Scholar
    5.
    Jones  CP.  Levels of racism: a theoretic framework and a gardener’s tale.   Am J Public Health. 2000;90(8):1212-1215. doi:10.2105/AJPH.90.8.1212PubMedGoogle Scholar
    ×