Comparison of Community-Level and Patient-Level Social Risk Data in a Network of Community Health Centers

Key Points

Question  Can community-level data (eg, social deprivation index score of the census tract where patients live) accurately identify patient-level social risks?

Findings  In this cross-sectional study including 36 578 patients, 10 858 (29.7%) screened positive for 1 or more social risks; 42% of patients with at least 1 social risk lived in neighborhoods not defined as disadvantaged.

Meaning  Using community-level data to guide patient-level activities may result in missing some patients who can benefit from social risk–targeted or social risk–informed care.

Importance  Responding to the substantial research on the relationship between social risk factors and health, enthusiasm has grown around social risk screening in health care settings, and numerous US health systems are experimenting with social risk screening initiatives. In the absence of standard social risk screening recommendations, some health systems are exploring using publicly available community-level data to identify patients who live in the most vulnerable communities as a way to characterize patient social and economic contexts, identify patients with potential social risks, and/or to target social risk screening efforts.

Objective  To explore the utility of community-level data for accurately identifying patients with social risks by comparing the social deprivation index score for the census tract where a patient lives with patient-level social risk screening data.

Design, Setting, and Participants  Cross-sectional study using patient-level social risk screening data from the electronic health records of a national network of community health centers between June 24, 2016, and November 15, 2018, linked to geocoded community-level data from publicly available sources. Eligible patients were those with a recorded response to social risk screening questions about food, housing, and/or financial resource strain, and a valid address of sufficient quality for geocoding.

Exposures  Social risk screening documented in the electronic health record.

Main Outcomes and Measures  Community-level social risk was assessed using census tract–level social deprivation index score stratified by quartile. Patient-level social risks were identified using food insecurity, housing insecurity, and financial resource strain screening responses.

Results  The final study sample included 36 578 patients from 13 US states; 22 113 (60.5%) received public insurance, 21 181 (57.9%) were female, 17 578 (48.1%) were White, and 10 918 (29.8%) were Black. Although 6516 (60.0%) of those with at least 1 social risk factor were in the most deprived quartile of census tracts, patients with social risk factors lived in all census tracts. Overall, the accuracy of the community-level data for identifying patients with and without social risks was 48.0%.

Conclusions and Relevance  Although there is overlap, patient-level and community-level approaches for assessing patient social risks are not equivalent. Using community-level data to guide patient-level activities may mean that some patients who could benefit from targeted interventions or care adjustments would not be identified.

Responding to the substantial research on the association between social risk factors and health, enthusiasm has grown around social risk screening in health care settings, and numerous health systems in the US are experimenting with social risk screening initiatives.^1-4 Recent studies have demonstrated the feasibility of implementing clinic-based screening and documenting social risks in electronic health records (EHRs),^5,6 yet no clear standard has emerged on how to implement social risk screening, nor how clinicians can or should use social risk information to adjust patient care or make referrals to community resources.^6,7 Moreover, some have questioned the benefit of integrating social risk screening into primary care, raising concerns about the additional burden of adding more required data collection to already busy primary care practices and the limited resources available to address identified social risk factors.^8-10

In the absence of standard social risk screening recommendations, some health systems are exploring obtaining social risk information without screening patients directly.^11,12 Community and neighborhood-level data characterizing the “conditions in which people are born, grow, live, work and age”¹³ are readily available from public sources, such as the US Census or American Community Survey, and can be geocoded and linked to patients’ addresses. Researchers have demonstrated the association between community-level measures of socioeconomic status and health.¹⁴ Theoretically, such data could provide an alternative way to identify patients with social risks or to target patients for whom self-reported screening should be prioritized.¹¹ For example, community-level data could be used for “cold-spotting” to identify patients living in geographic areas that lack certain characteristics that support health, such as access to grocery stores and parks, higher rates of education and employment, clean air, and adequate housing.¹⁵ Identifying patients who live in the most vulnerable communities, or “cold spots,” could help clinics characterize and understand patients’ social and economic contexts and/or target social risk screening efforts.

However, relying solely on community-level data to understand the social context of an individual patient and/or to guide patient-level interventions poses a risk of ecological fallacy, or making erroneous assumptions about individuals based on aggregate information.^16,17 Using patient-level social risk screening data from a national network of community health centers (CHCs),¹⁸ linked to geocoded data from the American Community Survey, we explored the utility of community-level data as a mechanism for identifying patients with social risks. Specifically, we compared the social deprivation index (SDI) score for the census tract where a patient lives with patient-level social risk screening data documented in the EHR to assess whether patients who live in “cold spots” have more patient-level social risks.^11,15

For this cross-sectional study, we analyzed data from the OCHIN network of CHCs. OCHIN Inc is a nonprofit health center–controlled network that hosts a centrally managed instance of the Epic EHR (Epic Systems Corporation) for 645 CHC clinics across 21 US states. OCHIN CHCs provide care for the nation’s most vulnerable patients, the majority of whom are publicly insured or uninsured. Like most CHCs in the US, compared with the general population, patients who receive care at OCHIN CHCs are disproportionately poor, members of racial and ethnic minorities, and living with multiple chronic conditions. OCHIN hosts a research data warehouse that includes EHR data on more than 4.9 million patients, making it, to our knowledge, the largest single research-ready data source on US safety net patients. The research data warehouse also includes neighborhood and community-level data—also called community vital signs—from publicly available sources (eg, US Census, American Community Survey) that provide information about each patient’s community context. Patient addresses, collected by OCHIN network clinics, are geocoded to identify the census tract of each patient’s residence, then linked to the community vital signs data for that tract. The geocoding and linkage process and selection of measures are described in detail elsewhere.¹²

In 2016, OCHIN released a suite of EHR tools to help clinics document and review patient-reported social risk screening results.¹⁹ To meet the needs of OCHIN’s diverse members, these tools include several nationally recognized screening instruments (eg, PRAPARE, Centers for Medicare and Medicaid Services Accountable Health Communities, National Academy of Medicine [formerly Institute of Medicine] recommendations) encompassing 7 social risk domains: financial resource strain, food insecurity, housing instability, relationship safety, inadequate physical activity, social connection/isolation, and stress. In May 2017, a question was added asking patients if they would like help addressing any identified social risks. The CHCs have the option to choose an entire screening questionnaire (eg, PRAPARE) or individual social risk domains. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

This study was approved by the Western States Institutional Review Board. The study sample (N = 36 578) included patients screened for social risks in an OCHIN clinic between June 24, 2016, and November 15, 2018, who responded to at least 1 of the food, housing, or financial resource strain questions. A complete description of the exclusion criteria is provided in the Figure. This study uses social risk screening data collected during clinical care and documented in the EHR. As reported elsewhere, CHCs use a variety of different criteria in selecting patients for social risk screening, and there are significant differences between patients who are screened and not screened.^5,18 Patients were excluded if they did not have a residential address in the same state as where the screening occurred (to enable meaningful comparison of screening responses and census-tract level information) or if they lacked an address of adequate quality to enable geocoding. Patients who recently moved and/or those who traveled across state lines to receive care may have had a higher likelihood of being excluded. Finally, patient address data were documented by the individual CHC, and we were unable to determine what factors influenced address quality. This could have introduced bias if address data were not missing at random.

To quantify community-level social risk, all patients with a valid address in the OCHIN research data warehouse were assigned a census-tract level SDI score using information on their last available address recorded in the EHR. Originally developed by Butler et al²⁰ and updated in 2015,²¹ the SDI is a composite measure of 7 demographic characteristics from the American Community Survey, including percentage living in poverty, percentage with less than 12 years of education, percentage of single-parent households, percentage living in a rented housing unit, percentage living in an overcrowded housing unit, percentage of households without a car, and percentage of nonemployed adults younger than 65 years. Census tracts with higher percentile scores have higher levels of social deprivation relative to other census tracts nationwide. For example, a census tract with an SDI score of 75 is considered “worse off” than 75% of census tracts in the US. On the other hand, a census tract with an SDI score of 5 is considered “better off” than 95% of census tracts in the US. This multidimensional measure has been shown to be more strongly associated with health outcomes than poverty alone.¹⁸ Moreover, prior studies suggest that patients living in cold spots—defined as those census tracts with an SDI score in the highest quartile nationally (≥75)—have worse health outcomes relative to those in more resource-rich tracts.¹¹

Patient-level measures of food insecurity, housing insecurity, and financial resource strain were included in our analysis. We classified patients as having social risks if they screened positive for 1 or more of these domains (see eTable 1 in the Supplement for a description of screening questions for each of these domains). These 3 domains were selected for several reasons. First, given the lack of standardized screening recommendations, CHCs are implementing screening in a variety of ways. Instead of using an established tool (eg, PRAPARE, Accountable Health Communities), many CHCs have opted to focus on screening for specific social risk domains. To date, food insecurity, housing insecurity, and financial resource strain are among the most frequently documented social risk domains in OCHIN CHCs, in part because they are actionable (eg, through referrals to local resources).¹⁸ Second, relative to other patient-level measures that focus on psychosocial domains (eg, relationship safety, social connection/isolation, stress), these domains are most aligned with the socioeconomic factors integrated into the SDI. Third, despite the limited consensus on standardized SDH screening recommendations, there is emerging consensus around standardized questions to assess food insecurity, housing insecurity, and financial resource strain.

OCHIN’s EHR-based screening tools include standardized question and response categories for each of these domains. Food insecurity is assessed using a validated 2-item screen called the Hunger Vital Sign.²² Housing insecurity is assessed differently depending on when the patient received the screening tool. Before May 17, 2018, patients were asked 2 items from the HealthBegins Upstream Risk screening tool about their living situation in the past month.²³ This has since transitioned to a 2-item question from the Accountable Health Communities screening tool about stability and quality of their housing.² Financial resource strain was assessed using the National Academy of Medicine–recommended screening question.^5,19,24 Finally, based on user feedback in pilot CHCs, screening questions were added in May 2017 asking whether patients desired CHC assistance in addressing identified risk factors.¹⁹ When available, data on whether patients wanted help addressing their social risks were also extracted (see eTable 1 in the Supplement for a list of measures and response options).

Descriptive statistics (counts and proportions) were generated to characterize the study sample. First, each patient in the study sample was categorized into national quartiles based on the SDI of their census tract of residence (Q1: 1 to <25; Q2: 25 to <50; Q3: 50 to <75; Q4: 75-100). Second, the numbers and percentages of patients who reported food insecurity, housing insecurity, or financial resource strain were calculated, overall and within each SDI quartile. We then compared the percentage of patients who screened positive for 1 or more social risk factors in each SDI quartile as well as between cold spots (Q4 census tracts) and non–cold spots (Q1-Q3 census tracts). Given the exploratory nature of this study, no hypothesis testing was performed. Data analysis was performed between January 2019 and July 2019.

Of the 36 578 patients included in the final study sample, 60.5% (n = 22 113) received public insurance, 57.9% (n = 21 181) were female, 19.2% (n = 7013) were 19 years or younger at the time of screening, and 20.3% (n = 7422) were 60 years or older (Table 1). The majority of study patients were White (n = 17 578; 48.1%), with 29.8% (10 918) identifying as Black/African American, and 23.0% (8431) as Hispanic. A small percentage (n = 1725; 4.7%) were recorded as homeless, and 30% spoke a primary language other than English. Screening occurred at 275 clinics (43% of all OCHIN clinics) across 13 states, with the majority residing in Massachusetts (55.7%), Oregon (18.5%), or California (12.4%) (see eTable 2 in the Supplement).

A total of 58.0% of patients in the study sample (n = 21 197) lived in cold spot census tracts with an SDI score in the highest quartile (≥75 or Q4; most deprived), and 42.0% lived in non–cold spot census tracts (23.5% [n = 8585] lived in a Q3 census tract; 13.5% [n = 4926] in a Q2 census tract; and 5.1% [n = 1869] in a Q1 census tract) (Table 1). In total, 88.4% of sample patients (n = 32 337) had documented responses to questions about housing insecurity (n = 3880; 12.0% screened positive), 66.8% (n = 24 426) had documented responses to questions about food insecurity (n = 6839; 28.0% screened positive), and 39.0% (14 276) had documented responses to questions about financial resource strain (7145; 50.0% screened positive) (Table 2). Because CHC programs and incentives differ regionally, patterns of screening differed by state (see eTable 2 in the Supplement).

Overall, approximately 29.7% of sample patients (n = 10 858) screened positive for housing insecurity, food insecurity, and/or financial resource strain (Table 3). Of these, 60.0% (n = 6516) resided in a Q4 census tract, 22.9% in Q3 (n = 2491), 13.0% in Q2 (n = 1415), and 4.0% (n = 436) in Q1. The percentage of patients who screened positive within each quartile was relatively stable across Q2 to Q4, with 6516 of 21 197 patients (30.7%) screening positive in the highest SDI quartile (Q4 or cold spot), 2491 of 8585 (29.0%) in the third quartile, and 1415 of 4926 (28.7%) in the second quartile. Although the first-quartile SDI had the fewest number of patients screened (1869), almost a quarter (436, 23.3%) screened positive for at least 1 risk factor (Table 3).

Of the patients who screened positive for 1 or more social risks (n = 10 858), 23.6% (n = 2561) were asked whether they wanted help from clinic staff to address identified risks. Of those, 35.5% (n = 908) said they wanted help. Overall, 63.3% (n = 575) of those who wanted help lived in a cold spot census tract (Q4), with 19.2% (n = 174) in Q3; 12.4% (n = 113) in Q2; and 5.1% (n = 46) in Q1 (Table 3). Interestingly, when looking at the percentage who wanted help within each quartile, Q1 had the highest percentage of patients who wanted help to address identified risks (n = 46; 48.0%). Conversely, Q4 had the highest percentage of patients who said they did not want help (n = 1046; 64.5%) (Table 3).

Of those who screened positive for at least 1 social risk factor, 60.0% (n = 6516) resided in a cold spot census tract and would have been correctly identified as having a risk using a cold-spotting approach. However, 40.0% (n = 4342) of patients reporting 1 or more social risks would not be correctly identified (Table 4). Of the 25 719 patients who did not screen positive for any social risk, 57.1% (n = 14 681) resided in a cold spot and thus would be incorrectly identified as having social risk using a cold-spotting approach (Table 4). Overall, the accuracy of the community-level data for identifying patients with and without social risks was 48.0% (n = 17 544).

There is growing recognition of the integral role of the health care sector in identifying and addressing social factors and the importance of considering social context to improve health equity.^2-4,7,25 Yet critical knowledge gaps remain, and there is a limited evidence base regarding best practices for integrating social and medical care.⁷ In the midst of this environment, numerous approaches have emerged, including both patient-level screening and efforts that leverage community-level social risk data, sometimes in combination.^{1,5,6,10,18,26-31} Recognizing the challenge, cost, and time involved with implementing patient-level social risk screening initiatives, some health care systems are exploring strategies for using publicly available community-level data as proxies of patient-level social and economic information and/or as a way to target social risk–targeted (directly addressing social risk factors) or social risk–informed care (adapting care to accommodate social risks).^6-10,27,32 In this study, we assessed concordance between patient-reported social risks and community-level social deprivation among patients receiving care in a national network of CHCs. Building on previous work,^10,11 we defined cold spots as census tracts with SDI scores within the highest quartile nationally. Although there was some overlap between cold spot status and the presence of patient-level social risks, with 60.0% of those who reported at least 1 social need living in a cold spot census tract, 40.0% of patients who screened positive for at least 1 social risk did not live in a cold spot. Overall, the accuracy of the community-level data for identifying patients with and without social risks was 48.0%.

These findings have several implications for the use of patient-level and community-level social risk data in clinical settings. First, using community-level data as a proxy for patient-level social risk screening or to refine targets for patient-level data collection^10,29 may heighten the risk of ecologic fallacy, wherein incorrect assumptions are made about an individual based on aggregate-level information about a group.^6,16 Despite the potential utility of community-level data for a variety of purposes, including in cases when universal screening is not feasible, findings within our study population suggest that ecologic fallacy may in fact be an issue when using community-level data to identify patient-level social needs. Indeed, based on data from the OCHIN network, if a CHC were to use the cold spot approach to identify patients for screening or to implement a social care intervention, they would risk missing 40.0% (n = 4342) of patients with social risks who live in more affluent areas and/or incorrectly target 57.1% (n = 14 681) of patients without social risks who live in a cold spot.

Second, our results suggest that, when asked, a larger percentage of patients said that they did not want help addressing identified risks (64.5% [n = 1653] said that they did not want help vs 35.5% [n = 908] who did). This finding is supported by a 2019 qualitative study,³³ which found that although patients and caregivers believed social risk screening was important and acceptable, they did not expect their health care teams to address the social challenges they faced. Moreover, despite the low numbers, the counterintuitive finding that a higher percentage of patients within the most affluent quartile (Q1) said they wanted help could be indicative of a greater availability of resources in these areas, but this is unclear. Overall, these findings underscore the need for additional research to explore patient perspectives on social care screening and referrals, including whether and how health care teams should address identified risk factors. They also raise questions about whether and how the availability of resources in a patient’s neighborhood or community—either perceived or actual—might influence their desire for help.

Third, our findings underscore the importance of identifying and exploring the potential for other less intensive strategies for understanding patient-level social risks. Medical professionals, especially those in primary care, have articulated the range of challenges involved in identifying and addressing social risks, including the increased burden of integrating these activities into an already burgeoning workoad.^9-12 One strategy would be to lower the burden of patient-level screening. There are several multiquestion screening instruments,^2-4 variability in measures, and limited understanding of their psychometric properties (ie, reliability and validity).³⁴ Potential next steps could include identifying a smaller set of risk factors to screen for in clinical settings, or alternatively, developing a valid and reliable single-question screening for social risks.

Finally, although our findings suggest that census tract-level SDI is not always accurate in identifying patient-level social risks, community-level data still have enormous value if targeted appropriately. As others have articulated, community-level social determinants of health or “community vital signs”—such as the availability of green spaces for walking, or environmental factors that might affect health—provide important contextual information about a patient’s environment that could inform care.^12,35 Beyond their utility in contextualizing patient care, community-level data are a vital source of information for community-level interventions (eg, advocacy, alignment) and could be used to inform the development value-based payment structures or approaches to risk adjustment.⁶ Indeed, other countries have demonstrated the value of using area-based measures of socioeconomic variation to assess community needs, inform research, adjust clinical funding, allocate community resources, and determine policy impact.³⁶ More research is needed to understand how patient-level and community-level data can be used in concert to most effectively and efficiently invest limited resources.

This study has several limitations. First, a primary limitation of our findings is the generalizability of the study sample. Patients who completed social risk assessments were a nonrandom sample of OCHIN patients and represented less than 1% of OCHIN patients seen during the study period. There was significant variation among OCHIN CHCs in the number of screenings conducted, the types of patients selected for screening, and the social risk questions that were asked.¹⁸ Because of the limitations of these EHR data, we cannot determine if differing response rates across SDH domains can be attributed to patient refusal to answer or clinics’ decisions not to ask them. Bias resulting from missing, inaccurate, or inadequate address data are also unknown. Second, this was an exploratory study using data from a relatively limited period and at the very beginning of social risk screening implementation in CHCs. Preliminary estimates from OCHIN CHCs suggest that the number of documented screenings is continuing to increase steadily. Future research should seek to replicate these results using a larger sample of patients and over an extended time frame. Third, we were unable to ascertain whether the SDI performed better in different geographic regions and/or in urban vs rural areas. Future research should explore whether the risk of misclassification varies between these geographic classifications and/or whether the SDI has more utility in some circumstances than in others. Finally, there are numerous community-level measures, beyond the SDI, that are available for public use. Although our findings suggest that relying on SDI in place of patient-level social risk screening may not be sufficient, future research should explore the utility of other measures of community-level social context for identifying high-risk individuals.

Our findings suggest that patient-level and community-level approaches to incorporating social risk data in clinical contexts are not equivalent. Although there is overlap in patients identified using each method, 48.0% of patients are misclassified, and many patients in less disadvantaged neighborhoods desired assistance. Using community-level social risk data to guide patient-level activities may mean that some patients who could benefit from programs targeting social conditions or care adjustments would not be identified.

Accepted for Publication: July 6, 2020.

Published: October 29, 2020. doi:10.1001/jamanetworkopen.2020.16852

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Cottrell EK et al. JAMA Network Open.

Corresponding Author: Erika K. Cottrell, PhD, MPP, OCHIN Inc, 1881 SW Naito Pkwy, Portland, OR 97201 (cottrelle@ochin.org).

Author Contributions: Dr Cottrell had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Cottrell, Dambrun, Gottlieb.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Cottrell, Hendricks, Dambrun, Gottlieb.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Pantell, Gottlieb.

Obtained funding: Cottrell, Dambrun, Gold.

Administrative, technical, or material support: Cottrell, Hendricks, Dambrun, Cowburn.

Supervision: Cottrell, Hendricks, Dambrun, Gottlieb.

Conflict of Interest Disclosures: None reported.

Funding/Support: The study was funded by the Patient-Centered Outcomes Research Institute (PCORI) through a PCORI Award (HSD-1603-34987).

Role of the Funder/Sponsor: PCORI had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The views presented in this work are solely the responsibility of the authors and do not necessarily represent the views of organizations participating in, collaborating with, or funding the National Patient-Centered Clinical Research Network (PCORnet) or of PCORI.

Additional Information: The research reported in this publication was conducted using PCORnet, the National Patient-Centered Clinical Research Network. PCORnet has been developed with funding from PCORI.