Patient-Reported Social and Behavioral Determinants of Health and Estimated Risk of Hospitalization in High-Risk Veterans Affairs Patients | Electronic Health Records | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Table 1.  Characteristics of Survey Respondents and Nonrespondentsa
Characteristics of Survey Respondents and Nonrespondentsa
Table 2.  Patient-Reported Survey Responses (4685 Patients Responded)
Patient-Reported Survey Responses (4685 Patients Responded)
Table 3.  Logistic Regression Associations Between Covariates of Interest and 90-Day Hospital Admission, by Model Type (N = 4685)
Logistic Regression Associations Between Covariates of Interest and 90-Day Hospital Admission, by Model Type (N = 4685)
Table 4.  Summary of SDH Variables Included in Final EHR Plus Survey–Based, 90-Day and 180-Day Primary and Post Hoc Logistic Regression Modelsa
Summary of SDH Variables Included in Final EHR Plus Survey–Based, 90-Day and 180-Day Primary and Post Hoc Logistic Regression Modelsa
Table 5.  Logistic Regression Associations Between Covariates of Interest and 180-Day Hospital Admission, by Model Type (N = 4685)
Logistic Regression Associations Between Covariates of Interest and 180-Day Hospital Admission, by Model Type (N = 4685)
1.
Pantell  M, Rehkopf  D, Jutte  D, Syme  SL, Balmes  J, Adler  N.  Social isolation: a predictor of mortality comparable to traditional clinical risk factors.   Am J Public Health. 2013;103(11):2056-2062. doi:10.2105/AJPH.2013.301261 PubMedGoogle ScholarCrossref
2.
Wolff  JL, Nicholas  LH, Willink  A, Mulcahy  J, Davis  K, Kasper  JD.  Medicare spending and the adequacy of support with daily activities in community-living older adults with disability: an observational study.   Ann Intern Med. 2019;170(12):837-844. doi:10.7326/M18-2467 PubMedGoogle ScholarCrossref
3.
Hatef  E, Searle  KM, Predmore  Z,  et al.  The impact of social determinants of health on hospitalization in the Veterans Health Administration.   Am J Prev Med. 2019;56(6):811-818. doi:10.1016/j.amepre.2018.12.012 PubMedGoogle ScholarCrossref
4.
Bradley  EH, Taylor  LA.  The American Health Care Paradox: Why Spending More Is Getting Us Less. New York, NY: Public Affairs Press; 2013.
5.
Commonwealth Fund. Review of evidence for health related social needs interventions. Accessed September 12, 2020. https://www.commonwealthfund.org/sites/default/files/2019-07/ROI-EVIDENCE-REVIEW-FINAL-VERSION.pdf
6.
Mathematica Policy Research. Population segmentation and targeting of health care resources: findings from a literature review: working paper 58. Published December 7, 2017. Accessed September 12, 2020. https://www.mathematica.org/our-publications-and-findings/publications/population-segmentation-and-targeting-of-health-care-resources-findings-from-a-literature-review
7.
Behforouz  HL, Drain  PK, Rhatigan  JJ.  Rethinking the social history.   N Engl J Med. 2014;371(14):1277-1279. doi:10.1056/NEJMp1404846 PubMedGoogle ScholarCrossref
8.
Buitron de la Vega  P, Losi  S, Sprague Martinez  L,  et al.  Implementing an EHR-based screening and referral system to address social determinants of health in primary care.   Med Care. 2019;57(6)(suppl 2):S133-S139. doi:10.1097/MLR.0000000000001029 PubMedGoogle ScholarCrossref
9.
Institute of Medicine.  Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 2. Washington, DC: National Academies Press; 2014.
10.
Institute of Medicine.  Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 1. Washington, DC: National Academies Press; 2014.
11.
Joynt  KE, De Lew  N, Sheingold  SH, Conway  PH, Goodrich  K, Epstein  AM.  Should Medicare value-based purchasing take social risk into account?   N Engl J Med. 2017;376(6):510-513. doi:10.1056/NEJMp1616278 PubMedGoogle ScholarCrossref
12.
Buntin  MB, Ayanian  JZ.  Social risk factors and equity in medicare payment.   N Engl J Med. 2017;376(6):507-510. doi:10.1056/NEJMp1700081 PubMedGoogle ScholarCrossref
13.
Walker  RJ, Smalls  BL, Campbell  JA, Strom Williams  JL, Egede  LE.  Impact of social determinants of health on outcomes for type 2 diabetes: a systematic review.   Endocrine. 2014;47(1):29-48. doi:10.1007/s12020-014-0195-0 PubMedGoogle ScholarCrossref
14.
Berkowitz  SA, Seligman  HK, Meigs  JB, Basu  S.  Food insecurity, healthcare utilization, and high cost: a longitudinal cohort study.   Am J Manag Care. 2018;24(9):399-404.PubMedGoogle Scholar
15.
Gundersen  C, Ziliak  JP.  Food insecurity and health outcomes.   Health Aff (Millwood). 2015;34(11):1830-1839. doi:10.1377/hlthaff.2015.0645 PubMedGoogle ScholarCrossref
16.
US Department of Veterans Affairs. Office of Health Equity. Accessed June 7, 2018. https://www.va.gov/healthequity/
17.
Agha  Z, Lofgren  RP, VanRuiswyk  JV, Layde  PM.  Are patients at Veterans Affairs medical centers sicker? a comparative analysis of health status and medical resource use.   Arch Intern Med. 2000;160(21):3252-3257. doi:10.1001/archinte.160.21.3252 PubMedGoogle ScholarCrossref
18.
Selim  AJ, Berlowitz  DR, Fincke  G,  et al.  The health status of elderly veteran enrollees in the Veterans Health Administration.   J Am Geriatr Soc. 2004;52(8):1271-1276. doi:10.1111/j.1532-5415.2004.52355.x PubMedGoogle ScholarCrossref
19.
Zullig  LL, Whitson  HE, Hastings  SN,  et al.  A systematic review of conceptual frameworks of medical complexity and new model development.   J Gen Intern Med. 2016;31(3):329-337. doi:10.1007/s11606-015-3512-2 PubMedGoogle ScholarCrossref
20.
Global Adult Tobacco Survey Collaborative Group.  Tobacco Questions for Surveys: A Subset of Key Questions from the Global Adult Tobacco Survey (GATS). 2nd ed. Atlanta, GA: Centers for Disease Control and Prevention; 2011.
21.
Zajacova  A, Dowd  JB.  Reliability of self-rated health in US adults.   Am J Epidemiol. 2011;174(8):977-983. doi:10.1093/aje/kwr204 PubMedGoogle ScholarCrossref
22.
DeSalvo  KB, Bloser  N, Reynolds  K, He  J, Muntner  P.  Mortality prediction with a single general self-rated health question: a meta-analysis.   J Gen Intern Med. 2006;21(3):267-275. doi:10.1111/j.1525-1497.2005.00291.x PubMedGoogle ScholarCrossref
23.
Fillenbaum  GG, Smyer  MA.  The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire.   J Gerontol. 1981;36(4):428-434. doi:10.1093/geronj/36.4.428 PubMedGoogle ScholarCrossref
24.
Yu  L, Buysse  DJ, Germain  A,  et al.  Development of short forms from the PROMIS™ sleep disturbance and Sleep-Related Impairment item banks.   Behav Sleep Med. 2011;10(1):6-24. doi:10.1080/15402002.2012.636266 PubMedGoogle ScholarCrossref
25.
Chew  LD, Griffin  JM, Partin  MR,  et al.  Validation of screening questions for limited health literacy in a large VA outpatient population.   J Gen Intern Med. 2008;23(5):561-566. doi:10.1007/s11606-008-0520-5 PubMedGoogle ScholarCrossref
26.
Wolf  MS, Smith  SG, Pandit  AU,  et al.  Development and validation of the Consumer Health Activation Index.   Med Decis Making. 2018;38(3):334-343. doi:10.1177/0272989X17753392 PubMedGoogle ScholarCrossref
27.
Moser  A, Stuck  AE, Silliman  RA, Ganz  PA, Clough-Gorr  KM.  The eight-item modified Medical Outcomes Study Social Support Survey: psychometric evaluation showed excellent performance.   J Clin Epidemiol. 2012;65(10):1107-1116. doi:10.1016/j.jclinepi.2012.04.007 PubMedGoogle ScholarCrossref
28.
Hughes  ME, Waite  LJ, Hawkley  LC, Cacioppo  JT.  A short scale for measuring loneliness in large surveys: results from two population-based studies.   Res Aging. 2004;26(6):655-672. doi:10.1177/0164027504268574 PubMedGoogle ScholarCrossref
29.
Berkowitz  SA, Meigs  JB, DeWalt  D,  et al.  Material need insecurities, control of diabetes mellitus, and use of health care resources: results of the Measuring Economic Insecurity in Diabetes study.   JAMA Intern Med. 2015;175(2):257-265. doi:10.1001/jamainternmed.2014.6888 PubMedGoogle ScholarCrossref
30.
Kroenke  K, Spitzer  RL, Williams  JB.  The Patient Health Questionnaire-2: validity of a two-item depression screener.   Med Care. 2003;41(11):1284-1292. doi:10.1097/01.MLR.0000093487.78664.3C PubMedGoogle ScholarCrossref
31.
Smith  BW, Dalen  J, Wiggins  K, Tooley  E, Christopher  P, Bernard  J.  The brief resilience scale: assessing the ability to bounce back.   Int J Behav Med. 2008;15(3):194-200. doi:10.1080/10705500802222972 PubMedGoogle ScholarCrossref
32.
Kelly  PA, Kallen  MA, Suárez-Almazor  ME.  A combined-method psychometric analysis recommended modification of the multidimensional health locus of control scales.   J Clin Epidemiol. 2007;60(5):440-447. doi:10.1016/j.jclinepi.2006.08.005 PubMedGoogle ScholarCrossref
33.
Matheny  AP  Jr, Wachs  TD, Ludwig  JL, Phillips  K.  Bringing order out of chaos: psychometric characteristics of the confusion, hubbub, and order scale.   J Appl Developmental Psychology. 1995;16(3):429-444. doi:10.1016/0193-3973(95)90028-4 Google ScholarCrossref
34.
Cornoni-Huntley  J, Ostfeld  AM, Taylor  JO,  et al.  Established populations for epidemiologic studies of the elderly: study design and methodology.   Aging (Milano). 1993;5(1):27-37.PubMedGoogle Scholar
35.
Duckworth  AL, Quinn  PD.  Development and validation of the short grit scale (grit-s).   J Pers Assess. 2009;91(2):166-174. doi:10.1080/00223890802634290 PubMedGoogle ScholarCrossref
36.
Fihn S, Box T. Care Assessment Need (CAN) score and the Patient Care Assessment System (PCAS): tools for care management. Published June 27, 2013. Accessed September 14, 2020. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/713-notes.pdf
37.
Wang  L, Porter  B, Maynard  C,  et al.  Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration.   Med Care. 2013;51(4):368-373. doi:10.1097/MLR.0b013e31827da95a PubMedGoogle ScholarCrossref
38.
Dillman  DA.  Mail and Telephone Surveys: The Total Design Method. New York, NY: John Wiley; 1978.
39.
Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.   J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004 PubMedGoogle ScholarCrossref
40.
Kinosian  B, Wieland  D, Gu  X, Stallard  E, Phibbs  CS, Intrator  O.  Validation of the JEN frailty index in the National Long-Term Care Survey community population: identifying functionally impaired older adults from claims data.   BMC Health Serv Res. 2018;18(1):908. doi:10.1186/s12913-018-3689-2 PubMedGoogle ScholarCrossref
41.
Bradley  KA, DeBenedetti  AF, Volk  RJ, Williams  EC, Frank  D, Kivlahan  DR.  AUDIT-C as a brief screen for alcohol misuse in primary care.   Alcohol Clin Exp Res. 2007;31(7):1208-1217. doi:10.1111/j.1530-0277.2007.00403.x PubMedGoogle ScholarCrossref
42.
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Published 2019. Accessed September 12, 2020. https://www.R-project.org/
43.
Yoshida K, Bohn J. tableone: Create 'Table 1' to describe baseline characteristics with or without propensity score weights. R package version 0.9.3. Published 2018. Accessed September 12, 2020. https://CRAN.R-project.org/package=tableone
44.
Austin  PC.  Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples.   Stat Med. 2009;28(25):3083-3107. doi:10.1002/sim.3697 PubMedGoogle ScholarCrossref
45.
Wasserstein  RL, Lazar  NA.  The ASA statement on P values: context, process, and purpose.   Am Statistician. 2016;70:129-133. doi:10.1080/00031305.2016.1154108 Google ScholarCrossref
46.
Sanchez-Pinto  LN, Venable  LR, Fahrenbach  J, Churpek  MM.  Comparison of variable selection methods for clinical predictive modeling.   Int J Med Inform. 2018;116:10-17. doi:10.1016/j.ijmedinf.2018.05.006 PubMedGoogle ScholarCrossref
47.
Heinze  G, Wallisch  C, Dunkler  D.  Variable selection—a review and recommendations for the practicing statistician.   Biom J. 2018;60(3):431-449. doi:10.1002/bimj.201700067 PubMedGoogle ScholarCrossref
48.
Friedman  J, Hastie  T, Tibshirani  R. The Elements of Statistical Learning. Vol 1, No 10. Springer; 2001.
49.
SAS. SAS macros and data step programs useful in survival analysis and logistic regression. Accessed March 17, 2020. http://biostat.mc.vanderbilt.edu/wiki/pub/Main/SasMacros/survrisk.txt
50.
Xu  H, Covinsky  KE, Stallard  E, Thomas  J  III, Sands  LP.  Insufficient help for activity of daily living disabilities and risk of all-cause hospitalization.   J Am Geriatr Soc. 2012;60(5):927-933. doi:10.1111/j.1532-5415.2012.03926.x PubMedGoogle ScholarCrossref
51.
Wolff  JL, Nicholas  LH, Willink  A, Mulcahy  J, Davis  K, Kasper  JD.  Medicare spending and the adequacy of support with daily activities in community-living older adults with disability: an observational study.   Ann Intern Med. 2019;170(12):837-844. doi:10.7326/M18-2467 PubMedGoogle ScholarCrossref
52.
Castrucci B, Auerbach J. Health Affairs blog. Meeting individual social needs falls short of addressing social determinants of health. Posted January 16, 2019. Accessed September 12, 2020. https://www.healthaffairs.org/do/10.1377/hblog20190115.234942/full/
53.
Selby  JV, Beal  AC, Frank  L.  The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda.   JAMA. 2012;307(15):1583-1584. doi:10.1001/jama.2012.500 PubMedGoogle ScholarCrossref
54.
Glasgow  R, Emmons  KM.  The public health need for patient-reported measures and health behaviors in electronic health records: a policy statement of the Society of Behavioral Medicine.   Transl Behav Med. 2011;1(1):108-109. doi:10.1007/s13142-011-0017-3 PubMedGoogle ScholarCrossref
55.
Adler  NE, Stead  WW.  Patients in context—EHR capture of social and behavioral determinants of health.   N Engl J Med. 2015;372(8):698-701. doi:10.1056/NEJMp1413945 PubMedGoogle ScholarCrossref
56.
Gold  R, Cottrell  E, Bunce  A,  et al.  Developing electronic health record (EHR) strategies related to health center patients’ social determinants of health.   J Am Board Fam Med. 2017;30(4):428-447. doi:10.3122/jabfm.2017.04.170046 PubMedGoogle ScholarCrossref
57.
Trinacty  CM, LaWall  E, Ashton  M, Taira  D, Seto  TB, Sentell  T.  Adding social determinants in the electronic health record in clinical care in Hawai’i: supporting community-clinical linkages in patient care.   Hawaii J Med Public Health. 2019;78(6)(suppl 1):46-51.PubMedGoogle Scholar
58.
Proser  M. Collecting social determinants of health data using PRAPARE to reduce disparities, improve outcomes, and transform care. Published 2016. Accessed September 12, 2020. http://www.nachc.org/wp-content/uploads/2016/07/SDH-Academy-Presentation-10-19-16.pdf
59.
National Association of Community Health Centers. PRAPARE Implementation and Action Toolkit. Published 2019. Accessed September 12, 2020. https://www.nachc.org/research-and-data/prapare/toolkit/
60.
Center for Medicare and Medicaid Services. The Accountable Health Communities Health-Related Social Needs Screening Tool. Accessed September 16, 2020. https://innovation.cms.gov/files/worksheets/ahcm-screeningtool.pdf
61.
Fuchs  VR.  Social determinants of health: caveats and nuances.   JAMA. 2017;317(1):25-26. doi:10.1001/jama.2016.17335 PubMedGoogle ScholarCrossref
62.
Garg  A, Boynton-Jarrett  R, Dworkin  PH.  Avoiding the unintended consequences of screening for social determinants of health.   JAMA. 2016;316(8):813-814. doi:10.1001/jama.2016.9282 PubMedGoogle ScholarCrossref
63.
Healthcare IT Today. Downsides of incorporating behavioral and social data into an EHR. Published 2015. Accessed June 3, 2020. http://www.emrandhipaa.com/emr-and-hipaa/2015/06/19/downsides-of-incorporating-behavioral-and-social-data-into-an-ehr/
64.
Schickedanz  A, Hamity  C, Rogers  A, Sharp  AL, Jackson  A.  Clinician experiences and attitudes regarding screening for social determinants of health in a large integrated health system.   Med Care. 2019;57(6)(suppl 2):S197-S201. doi:10.1097/MLR.0000000000001051 PubMedGoogle ScholarCrossref
65.
Davidson  KW, McGinn  T.  Screening for social determinants of health: the known and unknown.   JAMA. 2019;322(11):1037-1038. doi:10.1001/jama.2019.10915 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Statistics and Research Methods
    October 20, 2020

    Patient-Reported Social and Behavioral Determinants of Health and Estimated Risk of Hospitalization in High-Risk Veterans Affairs Patients

    Author Affiliations
    • 1Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, California
    • 2Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, California
    • 3Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina
    • 4Department of Population Health Sciences, Duke University, Durham, North Carolina
    • 5Division of General Internal Medicine, Department of Medicine, Duke University, Durham, North Carolina
    • 6Department of Medicine, Duke University School of Medicine, Durham, North Carolina
    • 7Geriatric Research, Education, and Clinical Center, Durham Veterans Affairs Health Care System, Durham, North Carolina
    • 8Center for the Study of Human Aging and Development, Duke University, Durham, North Carolina
    JAMA Netw Open. 2020;3(10):e2021457. doi:10.1001/jamanetworkopen.2020.21457
    Key Points

    Question  Can the estimation of risk of future hospitalization be improved by adding patient-reported social determinants of health to a model that is based on electronic health record data?

    Findings  In this survey study of 4685 respondent Veterans Affairs patients with 1-year risk in the 75th percentile or higher for hospitalization or death, a logistic regression model that included patient-reported social determinants of health outperformed a logistic model that was solely based on electronic health record variables.

    Meaning  Collecting social information from patients could improve health system algorithms that identify individuals at risk for poor outcomes.

    Abstract

    Importance  Despite recognition of the association between individual social and behavioral determinants of health (SDH) and patient outcomes, little is known regarding the value of SDH in explaining variation in outcomes for high-risk patients.

    Objective  To describe SDH factors among veterans who are at high risk for hospitalization, and to determine whether adding patient-reported SDH measures to electronic health record (EHR) measures improves estimation of 90-day and 180-day all-cause hospital admission.

    Design, Setting, and Participants  A survey was mailed between April 16 and June 29, 2018, to a nationally representative sample of 10 000 Veterans Affairs (VA) patients whose 1-year risk of hospitalization or death was in the 75th percentile or higher based on a VA EHR-derived risk score. The survey included multiple SDH measures, such as resilience, social support, health literacy, smoking status, transportation barriers, and recent life stressors.

    Main Outcomes and Measures  The EHR-based characteristics of survey respondents and nonrespondents were compared using standardized differences. Estimation of 90-day and 180-day hospital admission risk was assessed for 3 logistic regression models: (1) a base model of all prespecified EHR-based covariates, (2) a restricted model of EHR-based covariates chosen via forward selection based on minimizing Akaike information criterion (AIC), and (3) a model of EHR- and survey-based covariates chosen via forward selection based on AIC minimization.

    Results  In total, 4685 individuals (response rate 46.9%) responded to the survey. Respondents were comparable to nonrespondents in most characteristics, but survey respondents were older (eg, >80 years old, 881 [18.8%] vs 800 [15.1%]), comprised a higher percentage of men (4391 [93.7%] vs 4794 [90.2%]), and were composed of more White non-Hispanic individuals (3366 [71.8%] vs 3259 [61.3%]). Based on AIC, the regression model with survey-based covariates and EHR-based covariates better estimated hospital admission at 90 days (AIC, 1947.7) and 180 days (AIC, 2951.9) than restricted models with only EHR-based covariates (AIC, 1980.2 at 90 days; AIC, 2981.9 at 180 days). This result was due to inclusion of self-reported measures such as marital or partner status, health-related locus of control, resilience, smoking status, health literacy, and medication insecurity.

    Conclusions and Relevance  Augmenting EHR data with patient-reported social information improved estimation of 90-day and 180-day hospitalization risk, highlighting specific SDH factors that might identify individuals who are at high risk for hospitalization.

    Introduction

    There is increasing recognition that social and behavioral determinants of health (SDH)—the personal circumstances and environmental factors that shape individuals’ conditions of daily life—explain meaningful variation in clinical and economic outcomes.1-3 From a societal perspective, addressing SDH factors could increase the effectiveness of prevention and treatment efforts, health care use, and health expenditures.4,5 Clinician recognition of patients’ SDH vulnerabilities creates opportunities to offer social service referrals and targeted interventions for high-risk patients.6-8

    In 2014, the Health and Medicine Division of the Institute of Medicine proposed that a battery of SDH measures be universally incorporated into electronic health records (EHRs) in an “interoperable fashion.”9,10 The Office of the Assistant Secretary for Planning and Evaluation of the Department of Health and Human Services reaffirmed this recommendation, based on an analysis that found that Medicare beneficiaries with social risk factors had worse outcomes than beneficiaries without them.11 Concurrently, a National Academies of Sciences, Engineering, and Medicine panel recommended identification and incorporation of a core set of SDH factors into federal payment reforms.12 To be clinically useful and merit integration into EHRs and payment reform, those measures must be brief, validated, and predictive of outcomes important to patients and health systems.

    Because SDH measures are not routinely captured in most EHRs in general, little is known about which specific factors are most predictive of acute care use, such as hospitalization. Studies have found associations between hospitalizations and factors such as neighborhood socioeconomic status, race/ethnicity, food insecurity, and social isolation,1,3,13-15 but little is known about other social and behavioral characteristics, such as resilience and health-related locus of control.

    The Veterans Affairs (VA) health care system has a long-standing mission to address SDH factors in its patient population,16 and VA users typically experience greater medical, social, and behavioral complexity than nonveterans or veterans not engaged in Veterans Administration care.17,18 With 2 decades of experience with a national EHR for millions of patients in the nation’s largest integrated health system, the VA is well positioned to inform selection and integration of SDH measures into EHRs. We leveraged this long-standing EHR by fielding a mailed survey with an array of SDH measures to a nationally representative sample of 10 000 VA patients who were at high risk for hospitalization. We linked the survey data to EHR data to describe variation in SDH factors and to assess whether adding patient-reported SDH measures to models with EHR-based covariates improves estimation of 90-day and 180-day all-cause hospital admission risk.

    Methods
    Conceptual Model and Measure Selection

    Survey measures were informed by the Cycle of Complexity model,19 which posits that patient complexity is a multifactorial construct composed of workload, acute shocks and medical events, and capacity or resilience, all of which influence access or use. Study team members (D.M.Z., M.L.M., J.M.G., H.J.W., D.V.B., L.L.Z., H.E.W., and S.N.H.) used an iterative consensus process to generate a draft list of survey measures, prioritizing measures that were not available in administrative data, had a known association with hospitalization, explained the maximum amount of variation within the construct, shared minimal overlap with other scales, and could help primary care teams identify patients who would benefit from clinical intervention. This study followed the American Association for Public Opinion Research (AAPOR) reporting guideline for survey studies. This evaluation was reviewed and designated as nonresearch quality improvement by the supporting Veterans Health Administration program office and the Durham VA Medical Center institutional review board and therefore did not require approval or informed consent. Per VA protocol, individuals who were invited to participate were sent a letter describing the survey's purpose and inviting them to opt out by mail or telephone if they did not wish to participate or to be contacted again. Returning the survey constituted consent.

    The final survey included measures of educational attainment, employment, economic circumstances, smoking status,20 global health status (SF-121,22), functional status (activities of daily living [ADL], instrumental ADL [IADL]23), and sleep quality.24 Measures of health literacy25 and patient activation from the Consumer Health Activation Index26 were included because these characteristics may help determine the level of assistance that patients need with health care navigation. Social circumstances could be critical in helping a patient avoid acute care use; thus, the survey also queried patients about their marital or partner status (“Are you currently married or living with a partner?”), social support (Medical Outcomes Study Social Support Survey 827), and loneliness.28 Factors that reflect socioeconomic status included transportation availability and medication and food insecurity.29 Depression symptoms (2-item Patient Health Questionnaire30), resilience,31 Multidimensional Health Locus of Control,32 chaotic lifestyle (CHAOS33), recent life stressors (adapted from Duke Established Populations for Epidemiologic Studies of the Elderly [EPESE]34), and grit35 were assessed because these factors may be important risk factors for acute health deterioration and health care needs.

    Analytic Sample

    The sampling frame included veterans with a 1-year risk of hospitalization or death on March 16, 2018, that was in the 75th or higher percentile based on the VA’s Care Assessment Need score,36,37 the lower end of which reflects approximately 10% to 11% probability of hospitalization or death in the upcoming year. The 1-year risk measure is more stable and should fluctuate over time more slowly than a 90-day risk measure. A sample of 10 000 high-risk veterans within US VA medical centers was selected, using PROC SURVEYSELECT (SAS, version 9.4; SAS Institute Inc), via stratified random sampling proportional to the percentage of veterans with Care Assessment Need score 75 or higher in each US VA medical center to obtain a nationally representative sample of veterans who had at least 1 VA outpatient visit from March 20, 2017, to March 18, 2018, were alive, and had a valid home address at the time of cohort generation.

    The survey was submitted to the US Office of Management and Budget for review on June 13, 2016, and approved on October 25, 2017. A mail survey was fielded by Westat on April 16, 2018, to 10 000 veterans. Using the Dillman method,38 the survey was sent with a cover letter, a $2 bill incentive, a prepaid return envelope, and information for opting out from further mailings via a toll-free telephone number or return postcard. The cover letter described the purpose of the survey and included a statement that survey completion was voluntary and indicated consent to link survey responses to health record information. Veterans who did not opt out and did not respond within a 6-week period were mailed a second survey with a prepaid envelope and cover letter.

    VA Electronic Health Record Data

    We merged survey respondents’ survey data with their EHR data from the VA’s Corporate Data Warehouse. Demographic characteristics included age, sex, race and ethnicity as reported in the EHR, VA copay status (a marker of socioeconomic status), and rural vs urban status of residence. Clinical characteristics included chronic conditions in the Gagne comorbidity score,39 and 7 conditions not included in the Gagne comorbidity score (anxiety, bipolar disorder, dyslipidemia, posttraumatic stress disorder, schizophrenia, traumatic brain injury, and traumatic spinal cord injury) that are common in veteran populations. These chronic conditions were identified based on at least 1 outpatient encounter with an International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code for the condition in the year before the data pull. Other clinical characteristics extracted from the EHR included body mass index, frailty (JEN Frailty Index),40 and alcohol use based on the 3-item Alcohol Use Disorders Identification Test–Consumption (AUDIT-C).41

    Outcomes and Statistical Analysis

    The primary outcomes of interest—all-cause VA hospitalization 90 and 180 days after survey completion (operationalized as 7 days before receipt by survey vendor)—were obtained from the VA EHR. To examine whether survey respondents differed from nonrespondents, we compared EHR-based characteristics (eg, age, sex, race/ethnicity, VA copay status, rural residence, comorbidities) between survey respondents and nonrespondents using standardized mean differences (SMDs), calculated using tableone in R.42,43 The use of SMDs are recommended by the American Statistical Association because, unlike P values, SMDs are insensitive to sample size,44 thereby protecting against overstatement of significance.45

    Among respondents, we conducted 3 sets of logistic regressions for each of 2 dependent variables (90-day and 180-day all-cause hospitalizations) to identify specific patient-reported SDH measures that improved estimation of patient risk of hospital admission. Our goal was to evaluate whether including survey-based SDH measures improved estimates of hospitalization risk by comparing 3 models: (1) a base model of all prespecified EHR-based covariates (ie, all variables in Table 1, except individual Gagne conditions), (2) a restricted model of EHR-based covariates chosen via forward selection based on minimizing Akaike information criterion (AIC), and (3) a model of EHR- and survey-based covariates chosen via forward selection (including base model variables and all survey-based variables in Table 2, except number of stressful life events) based on AIC minimization.46 The AIC was used as the primary criteria for estimate accuracy, based on its ability to reflect model performance and because the rare outcome examined did not allow for withholding a validation sample.46 The AIC relates to P values, but with a penalty for model complexity, prioritizing a more parsimonious model that often leads to improved out-of-sample estimation, particularly when there are a small number of events (eg, hospitalizations) compared with the number of variables considered.46,47 Differences of AIC greater than 2 suggest considerable improvement; differences greater than 10 suggest substantial improvement.47 Although backward selection is generally preferred, forward selection was used because the sparseness of the binary outcome precluded entering all covariates into the model simultaneously.48 Because the goal of the analysis was to identify the model that yielded the most accurate estimates (not a model limited to statistically significant covariates), the final models could include covariates that were or were not statistically significant at a .05 threshold. We also report C statistics to facilitate interpretation of findings, although C statistics reflect in-sample classification, sorting those on which we already have data into one outcome or the other, rather than the ability of a model to predict an outcome on a patient not in the current sample. All continuous covariates were assessed for linearity using restricted cubic splines, as computed with the %PSPLINET SAS macro.49 Analyses other than calculation of standardized mean differences were conducted using SAS, version 9.4.

    Most survey and some EHR covariates had at least a small proportion (1%-6%) of missing data, but listwise deletion of respondents with missing values would have resulted in the loss of approximately 70% of respondents. Therefore, we multiply (n = 10) the imputed data using the MCMC option in PROC MI. The PROC HPLogistic procedure with forward selection based on minimizing AIC was used to run logistic regressions for each of the 10 imputed data sets and each of the 3 specifications for the 90-day and 180-day hospitalization outcomes. Logistic results for the second and third models (those using forward selection) were examined to determine the number of times that each covariate was chosen for inclusion in the final model. We identified covariates as important for improving estimates if they were chosen in at least 6 of 10 imputed data set models (eFigure 1 and eFigure 2 in the Supplement). After these covariates were identified, we entered them into a logistic regression model for each of the imputed data sets for both the 90-day and 180-day hospitalization outcomes using PROC Logistic (SAS, version 9.4). The regression results from the 10 imputations were combined using PROC MIANALYZE to appropriately estimate odds ratios and 95% CIs.

    We also conducted post hoc analyses, repeating all of the above logistic regressions, but replacing the EHR-based Gagne comorbidity score with the individual condition indicators that underlie the score in all 3 models and adding in a survey-based measure of the number of life stressors experienced in the past year, adapted from the EPESE study,34 to the set of survey measures included in the EHR Plus Survey model.

    Results

    There were 4685 individuals who completed the survey (response rate 46.9%, eFigure 3 in the Supplement). Respondents and 5315 nonrespondents were comparable across most (3 of 36) variables, but survey respondents were older (eg, >80 years old, 881 [18.8%] vs 800 [15.1%]), had a higher percentage who were male (4391 [93.7%] vs 4794 [90.2%]), were composed of more White non-Hispanic patients (3366 [71.8%] vs 3259 [61.3%]), and resided in a rural location (1732 [37.0%] vs 1596 [30.1%]). In addition, respondents and nonrespondents differed somewhat across a few individual clinical conditions (eg, hypertension, 3425 [73.1%] vs 3366 [63.3%]) (Table 1), hospitalization rate (492 [10.5%] vs 718 [13.5%]), and mortality rate (89 [1.9%] vs 282 [5.3%]) at 180 days. Respondent survey-based characteristics organized by Cycle of Complexity domains19 are presented in Table 2.

    Estimation of 90-Day Hospital Admission Risk

    The unadjusted 90-day VA hospital admission rate among the survey respondents was 5.6%. Based on AIC scores (Table 3), the logistic regression that included both patient-reported and EHR-based covariates improved estimates of 90-day hospitalization (AIC, 1947.7) compared with regressions that only had a restricted (AIC, 1963.9) or full set (AIC, 1980.2) of EHR-based covariates, indicating a substantial (ie, AIC>10) improvement. The corresponding improvement in the C statistic from restricted-EHR to survey plus EHR models was 0.643 to 0.671. Estimation accuracy, relative to models using only EHR data, was improved by the additional inclusion of 8 SDH survey measures: marital or partner status, smoking status, resilience, powerful others health-related locus of control, medication security status, global health status, depression symptoms, and health literacy as indicated by patient-reported confidence in completing medical forms (Table 3).

    In post hoc analyses that incorporated a categorized measure of the number of recent life stressors and replaced the Gagne score with individual condition indicators (eTable 1 in the Supplement), the logistic regression that included both patient-reported and EHR-based covariates also improved estimates of 90-day hospitalization (AIC, 1940.2) compared with regressions that had only a restricted set (AIC, 1954.0) or a full set (AIC, 1994.1) of EHR-based covariates. The 8 measures that improved the estimates of 90-day hospitalization in the primary analyses were the same ones that improved the estimates of the 90-day post hoc model (Table 4).

    Estimating Risk of 180-Day Hospital Admission

    The unadjusted 180-day VA hospital admission rate was 10.0%. Based on AIC scores, augmenting EHR covariates with patient-reported survey measures improved estimates of 180-day outcomes (AIC, 2951.9) compared with regressions with the restricted set (AIC, 2961.3) and the full set (AIC, 2981.9) of EHR-based covariates (Table 5). For the restricted models, the corresponding C statistics ranged from 0.636 to 0.651. Three of the same factors that emerged in the primary 90-day model also improved estimates of 180-day hospitalization (marital or partner status, resilience, and powerful others health-related locus of control), along with 2 others (activities of daily living and chaotic lifestyle) (Table 4). In post hoc analyses (eTable 2 in the Supplement), patient-reported survey measures also improved estimates of 180-day outcomes (AIC, 2941.4) over the restricted EHR model (AIC, 2949.9) and the full EHR model (AIC, 2994.8). Four of the 5 survey measures (marital or partner status, resilience, activities of daily living, and chaotic lifestyle) that improved estimates of 180-day hospitalization in the primary analysis were the same measures that improved estimates of 180-day hospitalization in the post hoc analysis.

    Discussion

    In this study of a nationally representative sample of high-risk veterans, augmenting EHR data with patient-reported SDH measures consistently improved estimates of 90-day and 180-day hospital admission risk, based on substantial reduction in AIC, compared with models with EHR-based covariates. Our findings contribute to a growing body of literature examining the association between SDH factors and hospitalization, including studies highlighting the critical roles of social support and neighborhood socioeconomic status (SES).3,50-52

    This study has implications for the National Academy of Medicine (previously the Institute of Medicine) recommendations53,54 for the routine integration of certain SDH factors into the EHR.9,10,55 One of the measures that improved estimates of hospitalization in our models (smoking status) is in the list recommended by the National Academy of Medicine.55 Our findings suggest that adding factors such as physical function, marital or partner status, and medication insecurity to annual preventive health screenings could offer an opportunity to improve prediction algorithms. Routine collection of these risk factors would enable the VA to flag patients at heightened risk for hospitalization due to these factors and enable EHR-based alerts for clinicians about individuals who need supportive services. Incorporating these types of factors into the EHR could also assist with population management and health system decisions, for example by highlighting the need for partnerships with certain community agencies.8,56,57

    Additional implementation considerations are important when applying these findings. First, different screening instruments may be needed to address key SDH factors and disparities in specific populations, and validation of SDH measures in diverse populations is necessary. There are a growing number of brief screening instruments available, such as PRAPARE,58,59 THRIVE,8 and the Centers for Medicare & Medicaid Services Accountable Health Communities screening tool,60 several of which are being tested in clinics serving vulnerable patient populations. Second, although there are potential benefits to more standardized assessment of SDH factors in clinical settings, there is also a need to explore some of the potentially negative nuances.61 Screening for SDH factors carries potential for inappropriate treatment choices based on inadequate decision support around SDH information or risk of discrimination.62,63 In addition, clinicians in many health care settings may not have the comfort or capacity to address SDHs64; thus, integration of these efforts with community and government resources is critical.65 Finally, there are significant (cumulative) time and financial costs for collecting SDH measures, so only those associated with important clinical or economic outcomes should be adopted.

    Limitations

    There are several limitations to this analysis that must be acknowledged. First, this study focused on individuals at high risk for hospitalization; thus, the results may not generalize to the entire VA and, therefore, cannot address how to improve estimation of hospitalization risk for the entire VA population. Second, respondents differed from nonrespondents in terms of hospitalization rate (10.5% vs 13.5%) and mortality rate (1.9% vs 5.3%) at 180 days. Veterans needed to have a valid mailing address to participate in the study. Thus, individuals who are homeless or have unstable housing are underrepresented. Women, minorities, and veterans who are younger or reside in rural locations also appeared to be underrepresented, but these were the only 3 of 36 covariates that were severely imbalanced, suggesting that the selection bias on observables was modest overall. In addition, owing to data availability at the time of analyses, hospitalizations were limited to those that took place in the VA system and do not capture hospitalizations that were covered by Medicare, Medicaid, or private insurance. Future work on more common outcomes is needed to understand whether these same covariates (or others) are associated with outcomes that matter most to veterans and the VA health system.

    Conclusions

    In conclusion, we found that integration of certain patient-reported SDH measures improved estimates of 90-day and 180-day hospitalization risk among veterans with a high risk for hospitalization. Our findings contribute to ongoing discussions about SDH measures that might be of greatest value for screening efforts in the clinical setting. Incorporating select measures into clinical care and the EHR could help identify individuals at high risk for hospitalization who might benefit from targeted support.

    Back to top
    Article Information

    Accepted for Publication: August 11, 2020.

    Published: October 20, 2020. doi:10.1001/jamanetworkopen.2020.21457

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Zulman DM et al. JAMA Network Open.

    Corresponding Author: Matthew L. Maciejewski, PhD, Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Center for Health Services Research in Primary Care, Durham Veterans Affairs Health Care System, 411 W Chapel Hill St, Ste 600, Durham, NC 27705 (matthew.maciejewski@va.gov).

    Author Contributions: Drs Zulman and Smith had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Zulman, Maciejewski, Weidenbacher, Blalock, Zullig, Whitson, Hastings, Smith.

    Acquisition, analysis, or interpretation of data: Zulman, Maciejewski, Grubber, Blalock, Greene, Whitson, Smith.

    Drafting of the manuscript: Zulman, Maciejewski, Grubber, Zullig, Smith.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Zulman, Maciejewski, Grubber, Greene, Smith.

    Obtained funding: Zulman, Maciejewski.

    Administrative, technical, or material support: Maciejewski, Grubber, Weidenbacher, Blalock, Zullig, Whitson, Hastings.

    Supervision: Maciejewski, Zullig, Smith.

    Conflict of Interest Disclosures: Dr Zulman reported receiving research grants from the US Department of Veterans Affairs (VA) Health Services Research and Development (HSR&D), the VA Office of Rural Health, and the Gordon & Betty Moore Foundation outside of the submitted work. Dr Maciejewski reported receiving grants from the VA HSR&D, the National Institutes of Health, and the National Committee for Quality Assurance; and owning Amgen stock owing to his spouse’s employment during the conduct of the study. Dr Weidenbacher reported receiving grants from VA Operations during the conduct of the study. Dr Blalock reported receiving grants from the VA Office of Academic Affairs during the conduct of the study. Dr Zullig reported receiving grants from Proteus Digital Health and from the PhRMA Foundation; and receiving personal fees from Novartis outside the submitted work. Dr Hastings reported receiving grants from the VA during the conduct of the study and outside the submitted work. Dr Smith reported receiving grants from VA HSR&D, VA Office of Rural Health, VA Caregiver Support Program, and the National Institutes of Health during the conduct of the study. No other disclosures were reported.

    Funding/Support: This study was funded by award XVA 21-159 from the Veterans Health Administration Office of Primary Care (to Drs Maciejewski and Zulman). Dr Maciejewski was also supported by a Research Career Scientist award RCS 10-391 from the VA and by award CIN 13-410 from the Center of Innovation to Accelerate Discovery and Practice Transformation at the Durham VA Health Care System. Dr Blalock was supported by grant TPH 21-000 from the VA Office of Academic Affiliations.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the United States Government, Stanford University, or Duke University.

    Meeting Presentation: Preliminary results were presented at the national meeting for the Society of General Internal Medicine; May 10, 2019; Washington, DC; and at the national meeting for AcademyHealth; June 4, 2019; Washington, DC.

    Additional Contributions: Karin Nelson, MD, Puget Sound Health Care System, Ann-Marie Rosland, MD, VA Ann Arbor Healthcare System, and members of the VA Primary Care High Risk Care Investigator Network provided feedback during the design and execution of this project. Cindie Slightam, MPH, VA Palo Alto Health Care System, and Camila Chaudhary, MPH, VA Palo Alto Health Care System, provided project management and manuscript support, respectively. No one was compensated for the stated contribution.

    References
    1.
    Pantell  M, Rehkopf  D, Jutte  D, Syme  SL, Balmes  J, Adler  N.  Social isolation: a predictor of mortality comparable to traditional clinical risk factors.   Am J Public Health. 2013;103(11):2056-2062. doi:10.2105/AJPH.2013.301261 PubMedGoogle ScholarCrossref
    2.
    Wolff  JL, Nicholas  LH, Willink  A, Mulcahy  J, Davis  K, Kasper  JD.  Medicare spending and the adequacy of support with daily activities in community-living older adults with disability: an observational study.   Ann Intern Med. 2019;170(12):837-844. doi:10.7326/M18-2467 PubMedGoogle ScholarCrossref
    3.
    Hatef  E, Searle  KM, Predmore  Z,  et al.  The impact of social determinants of health on hospitalization in the Veterans Health Administration.   Am J Prev Med. 2019;56(6):811-818. doi:10.1016/j.amepre.2018.12.012 PubMedGoogle ScholarCrossref
    4.
    Bradley  EH, Taylor  LA.  The American Health Care Paradox: Why Spending More Is Getting Us Less. New York, NY: Public Affairs Press; 2013.
    5.
    Commonwealth Fund. Review of evidence for health related social needs interventions. Accessed September 12, 2020. https://www.commonwealthfund.org/sites/default/files/2019-07/ROI-EVIDENCE-REVIEW-FINAL-VERSION.pdf
    6.
    Mathematica Policy Research. Population segmentation and targeting of health care resources: findings from a literature review: working paper 58. Published December 7, 2017. Accessed September 12, 2020. https://www.mathematica.org/our-publications-and-findings/publications/population-segmentation-and-targeting-of-health-care-resources-findings-from-a-literature-review
    7.
    Behforouz  HL, Drain  PK, Rhatigan  JJ.  Rethinking the social history.   N Engl J Med. 2014;371(14):1277-1279. doi:10.1056/NEJMp1404846 PubMedGoogle ScholarCrossref
    8.
    Buitron de la Vega  P, Losi  S, Sprague Martinez  L,  et al.  Implementing an EHR-based screening and referral system to address social determinants of health in primary care.   Med Care. 2019;57(6)(suppl 2):S133-S139. doi:10.1097/MLR.0000000000001029 PubMedGoogle ScholarCrossref
    9.
    Institute of Medicine.  Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 2. Washington, DC: National Academies Press; 2014.
    10.
    Institute of Medicine.  Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 1. Washington, DC: National Academies Press; 2014.
    11.
    Joynt  KE, De Lew  N, Sheingold  SH, Conway  PH, Goodrich  K, Epstein  AM.  Should Medicare value-based purchasing take social risk into account?   N Engl J Med. 2017;376(6):510-513. doi:10.1056/NEJMp1616278 PubMedGoogle ScholarCrossref
    12.
    Buntin  MB, Ayanian  JZ.  Social risk factors and equity in medicare payment.   N Engl J Med. 2017;376(6):507-510. doi:10.1056/NEJMp1700081 PubMedGoogle ScholarCrossref
    13.
    Walker  RJ, Smalls  BL, Campbell  JA, Strom Williams  JL, Egede  LE.  Impact of social determinants of health on outcomes for type 2 diabetes: a systematic review.   Endocrine. 2014;47(1):29-48. doi:10.1007/s12020-014-0195-0 PubMedGoogle ScholarCrossref
    14.
    Berkowitz  SA, Seligman  HK, Meigs  JB, Basu  S.  Food insecurity, healthcare utilization, and high cost: a longitudinal cohort study.   Am J Manag Care. 2018;24(9):399-404.PubMedGoogle Scholar
    15.
    Gundersen  C, Ziliak  JP.  Food insecurity and health outcomes.   Health Aff (Millwood). 2015;34(11):1830-1839. doi:10.1377/hlthaff.2015.0645 PubMedGoogle ScholarCrossref
    16.
    US Department of Veterans Affairs. Office of Health Equity. Accessed June 7, 2018. https://www.va.gov/healthequity/
    17.
    Agha  Z, Lofgren  RP, VanRuiswyk  JV, Layde  PM.  Are patients at Veterans Affairs medical centers sicker? a comparative analysis of health status and medical resource use.   Arch Intern Med. 2000;160(21):3252-3257. doi:10.1001/archinte.160.21.3252 PubMedGoogle ScholarCrossref
    18.
    Selim  AJ, Berlowitz  DR, Fincke  G,  et al.  The health status of elderly veteran enrollees in the Veterans Health Administration.   J Am Geriatr Soc. 2004;52(8):1271-1276. doi:10.1111/j.1532-5415.2004.52355.x PubMedGoogle ScholarCrossref
    19.
    Zullig  LL, Whitson  HE, Hastings  SN,  et al.  A systematic review of conceptual frameworks of medical complexity and new model development.   J Gen Intern Med. 2016;31(3):329-337. doi:10.1007/s11606-015-3512-2 PubMedGoogle ScholarCrossref
    20.
    Global Adult Tobacco Survey Collaborative Group.  Tobacco Questions for Surveys: A Subset of Key Questions from the Global Adult Tobacco Survey (GATS). 2nd ed. Atlanta, GA: Centers for Disease Control and Prevention; 2011.
    21.
    Zajacova  A, Dowd  JB.  Reliability of self-rated health in US adults.   Am J Epidemiol. 2011;174(8):977-983. doi:10.1093/aje/kwr204 PubMedGoogle ScholarCrossref
    22.
    DeSalvo  KB, Bloser  N, Reynolds  K, He  J, Muntner  P.  Mortality prediction with a single general self-rated health question: a meta-analysis.   J Gen Intern Med. 2006;21(3):267-275. doi:10.1111/j.1525-1497.2005.00291.x PubMedGoogle ScholarCrossref
    23.
    Fillenbaum  GG, Smyer  MA.  The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire.   J Gerontol. 1981;36(4):428-434. doi:10.1093/geronj/36.4.428 PubMedGoogle ScholarCrossref
    24.
    Yu  L, Buysse  DJ, Germain  A,  et al.  Development of short forms from the PROMIS™ sleep disturbance and Sleep-Related Impairment item banks.   Behav Sleep Med. 2011;10(1):6-24. doi:10.1080/15402002.2012.636266 PubMedGoogle ScholarCrossref
    25.
    Chew  LD, Griffin  JM, Partin  MR,  et al.  Validation of screening questions for limited health literacy in a large VA outpatient population.   J Gen Intern Med. 2008;23(5):561-566. doi:10.1007/s11606-008-0520-5 PubMedGoogle ScholarCrossref
    26.
    Wolf  MS, Smith  SG, Pandit  AU,  et al.  Development and validation of the Consumer Health Activation Index.   Med Decis Making. 2018;38(3):334-343. doi:10.1177/0272989X17753392 PubMedGoogle ScholarCrossref
    27.
    Moser  A, Stuck  AE, Silliman  RA, Ganz  PA, Clough-Gorr  KM.  The eight-item modified Medical Outcomes Study Social Support Survey: psychometric evaluation showed excellent performance.   J Clin Epidemiol. 2012;65(10):1107-1116. doi:10.1016/j.jclinepi.2012.04.007 PubMedGoogle ScholarCrossref
    28.
    Hughes  ME, Waite  LJ, Hawkley  LC, Cacioppo  JT.  A short scale for measuring loneliness in large surveys: results from two population-based studies.   Res Aging. 2004;26(6):655-672. doi:10.1177/0164027504268574 PubMedGoogle ScholarCrossref
    29.
    Berkowitz  SA, Meigs  JB, DeWalt  D,  et al.  Material need insecurities, control of diabetes mellitus, and use of health care resources: results of the Measuring Economic Insecurity in Diabetes study.   JAMA Intern Med. 2015;175(2):257-265. doi:10.1001/jamainternmed.2014.6888 PubMedGoogle ScholarCrossref
    30.
    Kroenke  K, Spitzer  RL, Williams  JB.  The Patient Health Questionnaire-2: validity of a two-item depression screener.   Med Care. 2003;41(11):1284-1292. doi:10.1097/01.MLR.0000093487.78664.3C PubMedGoogle ScholarCrossref
    31.
    Smith  BW, Dalen  J, Wiggins  K, Tooley  E, Christopher  P, Bernard  J.  The brief resilience scale: assessing the ability to bounce back.   Int J Behav Med. 2008;15(3):194-200. doi:10.1080/10705500802222972 PubMedGoogle ScholarCrossref
    32.
    Kelly  PA, Kallen  MA, Suárez-Almazor  ME.  A combined-method psychometric analysis recommended modification of the multidimensional health locus of control scales.   J Clin Epidemiol. 2007;60(5):440-447. doi:10.1016/j.jclinepi.2006.08.005 PubMedGoogle ScholarCrossref
    33.
    Matheny  AP  Jr, Wachs  TD, Ludwig  JL, Phillips  K.  Bringing order out of chaos: psychometric characteristics of the confusion, hubbub, and order scale.   J Appl Developmental Psychology. 1995;16(3):429-444. doi:10.1016/0193-3973(95)90028-4 Google ScholarCrossref
    34.
    Cornoni-Huntley  J, Ostfeld  AM, Taylor  JO,  et al.  Established populations for epidemiologic studies of the elderly: study design and methodology.   Aging (Milano). 1993;5(1):27-37.PubMedGoogle Scholar
    35.
    Duckworth  AL, Quinn  PD.  Development and validation of the short grit scale (grit-s).   J Pers Assess. 2009;91(2):166-174. doi:10.1080/00223890802634290 PubMedGoogle ScholarCrossref
    36.
    Fihn S, Box T. Care Assessment Need (CAN) score and the Patient Care Assessment System (PCAS): tools for care management. Published June 27, 2013. Accessed September 14, 2020. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/713-notes.pdf
    37.
    Wang  L, Porter  B, Maynard  C,  et al.  Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration.   Med Care. 2013;51(4):368-373. doi:10.1097/MLR.0b013e31827da95a PubMedGoogle ScholarCrossref
    38.
    Dillman  DA.  Mail and Telephone Surveys: The Total Design Method. New York, NY: John Wiley; 1978.
    39.
    Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.   J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004 PubMedGoogle ScholarCrossref
    40.
    Kinosian  B, Wieland  D, Gu  X, Stallard  E, Phibbs  CS, Intrator  O.  Validation of the JEN frailty index in the National Long-Term Care Survey community population: identifying functionally impaired older adults from claims data.   BMC Health Serv Res. 2018;18(1):908. doi:10.1186/s12913-018-3689-2 PubMedGoogle ScholarCrossref
    41.
    Bradley  KA, DeBenedetti  AF, Volk  RJ, Williams  EC, Frank  D, Kivlahan  DR.  AUDIT-C as a brief screen for alcohol misuse in primary care.   Alcohol Clin Exp Res. 2007;31(7):1208-1217. doi:10.1111/j.1530-0277.2007.00403.x PubMedGoogle ScholarCrossref
    42.
    R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Published 2019. Accessed September 12, 2020. https://www.R-project.org/
    43.
    Yoshida K, Bohn J. tableone: Create 'Table 1' to describe baseline characteristics with or without propensity score weights. R package version 0.9.3. Published 2018. Accessed September 12, 2020. https://CRAN.R-project.org/package=tableone
    44.
    Austin  PC.  Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples.   Stat Med. 2009;28(25):3083-3107. doi:10.1002/sim.3697 PubMedGoogle ScholarCrossref
    45.
    Wasserstein  RL, Lazar  NA.  The ASA statement on P values: context, process, and purpose.   Am Statistician. 2016;70:129-133. doi:10.1080/00031305.2016.1154108 Google ScholarCrossref
    46.
    Sanchez-Pinto  LN, Venable  LR, Fahrenbach  J, Churpek  MM.  Comparison of variable selection methods for clinical predictive modeling.   Int J Med Inform. 2018;116:10-17. doi:10.1016/j.ijmedinf.2018.05.006 PubMedGoogle ScholarCrossref
    47.
    Heinze  G, Wallisch  C, Dunkler  D.  Variable selection—a review and recommendations for the practicing statistician.   Biom J. 2018;60(3):431-449. doi:10.1002/bimj.201700067 PubMedGoogle ScholarCrossref
    48.
    Friedman  J, Hastie  T, Tibshirani  R. The Elements of Statistical Learning. Vol 1, No 10. Springer; 2001.
    49.
    SAS. SAS macros and data step programs useful in survival analysis and logistic regression. Accessed March 17, 2020. http://biostat.mc.vanderbilt.edu/wiki/pub/Main/SasMacros/survrisk.txt
    50.
    Xu  H, Covinsky  KE, Stallard  E, Thomas  J  III, Sands  LP.  Insufficient help for activity of daily living disabilities and risk of all-cause hospitalization.   J Am Geriatr Soc. 2012;60(5):927-933. doi:10.1111/j.1532-5415.2012.03926.x PubMedGoogle ScholarCrossref
    51.
    Wolff  JL, Nicholas  LH, Willink  A, Mulcahy  J, Davis  K, Kasper  JD.  Medicare spending and the adequacy of support with daily activities in community-living older adults with disability: an observational study.   Ann Intern Med. 2019;170(12):837-844. doi:10.7326/M18-2467 PubMedGoogle ScholarCrossref
    52.
    Castrucci B, Auerbach J. Health Affairs blog. Meeting individual social needs falls short of addressing social determinants of health. Posted January 16, 2019. Accessed September 12, 2020. https://www.healthaffairs.org/do/10.1377/hblog20190115.234942/full/
    53.
    Selby  JV, Beal  AC, Frank  L.  The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda.   JAMA. 2012;307(15):1583-1584. doi:10.1001/jama.2012.500 PubMedGoogle ScholarCrossref
    54.
    Glasgow  R, Emmons  KM.  The public health need for patient-reported measures and health behaviors in electronic health records: a policy statement of the Society of Behavioral Medicine.   Transl Behav Med. 2011;1(1):108-109. doi:10.1007/s13142-011-0017-3 PubMedGoogle ScholarCrossref
    55.
    Adler  NE, Stead  WW.  Patients in context—EHR capture of social and behavioral determinants of health.   N Engl J Med. 2015;372(8):698-701. doi:10.1056/NEJMp1413945 PubMedGoogle ScholarCrossref
    56.
    Gold  R, Cottrell  E, Bunce  A,  et al.  Developing electronic health record (EHR) strategies related to health center patients’ social determinants of health.   J Am Board Fam Med. 2017;30(4):428-447. doi:10.3122/jabfm.2017.04.170046 PubMedGoogle ScholarCrossref
    57.
    Trinacty  CM, LaWall  E, Ashton  M, Taira  D, Seto  TB, Sentell  T.  Adding social determinants in the electronic health record in clinical care in Hawai’i: supporting community-clinical linkages in patient care.   Hawaii J Med Public Health. 2019;78(6)(suppl 1):46-51.PubMedGoogle Scholar
    58.
    Proser  M. Collecting social determinants of health data using PRAPARE to reduce disparities, improve outcomes, and transform care. Published 2016. Accessed September 12, 2020. http://www.nachc.org/wp-content/uploads/2016/07/SDH-Academy-Presentation-10-19-16.pdf
    59.
    National Association of Community Health Centers. PRAPARE Implementation and Action Toolkit. Published 2019. Accessed September 12, 2020. https://www.nachc.org/research-and-data/prapare/toolkit/
    60.
    Center for Medicare and Medicaid Services. The Accountable Health Communities Health-Related Social Needs Screening Tool. Accessed September 16, 2020. https://innovation.cms.gov/files/worksheets/ahcm-screeningtool.pdf
    61.
    Fuchs  VR.  Social determinants of health: caveats and nuances.   JAMA. 2017;317(1):25-26. doi:10.1001/jama.2016.17335 PubMedGoogle ScholarCrossref
    62.
    Garg  A, Boynton-Jarrett  R, Dworkin  PH.  Avoiding the unintended consequences of screening for social determinants of health.   JAMA. 2016;316(8):813-814. doi:10.1001/jama.2016.9282 PubMedGoogle ScholarCrossref
    63.
    Healthcare IT Today. Downsides of incorporating behavioral and social data into an EHR. Published 2015. Accessed June 3, 2020. http://www.emrandhipaa.com/emr-and-hipaa/2015/06/19/downsides-of-incorporating-behavioral-and-social-data-into-an-ehr/
    64.
    Schickedanz  A, Hamity  C, Rogers  A, Sharp  AL, Jackson  A.  Clinician experiences and attitudes regarding screening for social determinants of health in a large integrated health system.   Med Care. 2019;57(6)(suppl 2):S197-S201. doi:10.1097/MLR.0000000000001051 PubMedGoogle ScholarCrossref
    65.
    Davidson  KW, McGinn  T.  Screening for social determinants of health: the known and unknown.   JAMA. 2019;322(11):1037-1038. doi:10.1001/jama.2019.10915 PubMedGoogle ScholarCrossref
    ×