Primary Care Physicians’ Satisfaction With Interoperable Health Information Technology

Key Points Question What were primary care physician perspectives on access to health information from outside organizations after years of policy support for interoperability (the ability of health information technology systems to exchange information and to use that information without special effort)? Findings In this survey study of 2088 physicians, 70% indicated being at least somewhat satisfied with access to outside information. However, only 23% indicated that it was very easy to use outside information, and very few (8%) indicated that it was very easy to use information from different electronic health record systems. Meaning The findings of this study suggest that 6 years after the 21st Century Cures Act, the policy goal of ubiquitous, high-value interoperable health information appears to remain a work in progress with heterogeneous barriers—including both too much information and missing information—limiting the value of interoperability.


Introduction
Starting in 2016 with the 21st Century Cures Act and concurrent changes to the Merit-Based Incentive Payment System, public policy on health information technology focused more directly on interoperability, which is the ability of health information technology (IT) systems to exchange information between systems and use that information without special effort. 1The core regulations from the Cures Act intended to improve interoperability are now in place, with central applicability dates in 2021 and after.These include efforts to raise the bar for investment in interoperability (with some provisions applicable April 2021), 2 create stronger financial disincentives for impeding interoperability (applicable as of April 2021), 2,3 improve data standards to enhance the usability of exchanged information (with some provision applicable as of January 1, 2023), 2 improve electronic health record (EHR) capabilities to allow outside applications to access EHR data (with some provisions applicable as of January 1, 2023), 2 and create a universal floor for nationwide health information exchange (launched December 2023). 4These efforts, along with ongoing federal support for networks that help federally qualified health centers use health IT 5 and efforts to enable bidirectional exchange of public health information 6 present a broad attempt to increase investment in interoperability and support the effective use of health information to care for patients.
Available data indicate that these recent policy levers are needed.According to a nationally representative survey of outpatient physicians, even as recently as 2021, only 16% indicated that they engaged in 4 domains of interoperable exchange: electronically sending, receiving, searching for, and integrating information into their EHR, which is far below similar measures among hospitals. 7,8More granular information on physicians' experience accessing and using information from outside organizations, which effective interoperability should improve, could provide insight into areas where interoperability currently supports the delivery of care and where challenges persist; identify prominent barriers to achieving value from interoperable information and the fitness of ongoing policymaking to address those barriers; and indicate targeted objectives for future policymaking.
0][11] More than one-third of all health care visits occur in primary care settings and more than 80% of people in the US have health care relationships in primary care. 12The ability to easily access and use information about patients from diverse sources is critical to PCPs, and PCPs have described interoperability as the most important improvement they wish to see in EHRs over the long term. 13 therefore sought to generate novel, robust measures of PCP's experiences with interoperability that captured several dimensions and can serve as markers of progress on interoperability.We used an annual survey of family physicians to describe their (1) experience accessing and using information from outside organizations and variation by physician practice characteristics, (2) satisfaction with outside information by information type, and (3) variation in satisfaction by whether information came from the same or different EHR.These survey data are notable in that they accrue a 100% response rate from a nationally representative sample of family medicine physicians by being included as part of an annual certification requirement, avoiding response bias present in other surveys of physicians. 14,15

Data Source
The American Board of Family Medicine (ABFM) administers a Continuous Certification Questionnaire (CCQ) among participants who provide direct patient care.The CCQ is a required part of the ABFM certification process for more than 100 000 family physicians and has a 100% response rate.Family physicians take this web-based survey on a rolling basis every year based on their initial year of certification, offering a nationally representative sample.The CCQ is designed to help the ABFM understand changes in family physician practice relevant to training, certification, and competency to fulfill its business requirements and capacity to inform policies that affect practice.These data are secondarily valuable for research and are often used to study and inform understanding about practice.Questions were developed in collaboration between the ABFM, the Office of the National Coordinator for Health Information Technology, and researchers at the University of California, San Francisco.We developed and revised the questionnaire items and response options based on cognitive interviews with family physicians and conducted quality checks when programming the questions on the CCQ following recommendations by the American Association for Public Opinion Research (AAPOR) reporting guideline.A core set of questions was answered by all ABFM diplomates in the 2022 certification cohort (from December 12, 2021, to October 12, 2022), but most questions relevant to this study were randomly given to rotating panels to reduce overall question burden.This study followed relevant portions of the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline and the AAPOR reporting guideline.This study was approved by the institutional review board of the American Academy of Family Physicians, with a waiver of informed consent for this research because of the use of secondary data.

Measures
In total, 18 items on the 2022 CCQ assessed specific dimensions related to family physicians' experience accessing and using information from outside organizations, a crucial marker for how interoperable technology is shaping the delivery of health care as experienced by clinicians.The questionnaire is included in the eAppendix in Supplement 1.
The CCQ included 10 items about respondents' satisfaction with electronically accessing different types of clinical information from organizations outside their health system.For each item, respondents could indicate that they did not have or use it, or were not at all, somewhat, or very satisfied with accessing each type of information.
The CCQ included 4 items to capture respondents' experience using information accessed from outside organizations.These items focused on the ease of using outside information to effectively care for patients, the ease of finding specific information, and the ease of using information from outside organizations that used EHR products from the same or different developer.
Four items on the CCQ assessed the prevalence of specific barriers related to using information from outside organizations accessed via EHRs.The barriers included not having a patient record available, missing key information within the record, difficulty finding important information due to a large amount of low-value information, and information was not integrated within the EHR.
Questions related to physicians' experience and specific barriers to using information from outside organizations were not focused solely on information exchanged via electronic methods but rather were intended to capture a broader assessment of the ease of using information, which still often relies on telephone, fax, and other means.

Statistical Analysis
We first assessed the distribution of responses by each information type individually and then derived a composite scale based on the 10 satisfaction-related items by assigning a value of 0 to each item that respondents did not have, did not use, or were not at all satisfied with because both responses represented no value to the clinician; a value of 1 to items respondents were somewhat satisfied with, and a value of 2 to items respondents were very satisfied with.The resulting scale ranged from 0 to 20.After calculating scale reliability (Cronbach α=0.96), we used this scale to calculate the empirical cumulative density function of respondents' satisfaction accessing information from outside organizations overall.The resulting percentages can be interpreted as the percentage of respondents with some minimum level of satisfaction (for instance, at least somewhat satisfied with all items) or an equivalent score.with associated 95% CIs.

JAMA Network Open | Health Informatics
We next tabulated respondents' experience using outside information, including the ease of using information from outside organizations to provide beneficial care for patients, the ease of finding specific information, and the ease of using information from outside organizations.We stratified respondents' answers to questions about ease of using information from the same and different developers by the respondent's developer.We then similarly tabulated their experience of the 4 barriers described earlier.We tested for differences in response patterns to ordinal survey items using χ 2 tests; the data were not paired.Statistical significance was set at P < .01.Statistical analysis was conducted using Stata SE, version 15 (StataCorp LLC).

Results
A total of 2088 family physicians (1053 women [50%]; 1035 men [50%]; age was reported categorically as either Ն50 years or <50 years) received and responded to the interoperability questions (100% response rate).Of these respondents, 27% practiced in independently owned medical practices, 35% practiced in hospital or health system-owned practices, and the remainder practiced in a variety of settings (eTable in Supplement 1).A total of 42% of respondents practiced in principal practice sites with between 1 and 5 clinicians.A total of 24% of respondents indicated that more than half of the patient population was part of a vulnerable group (ie, defined in survey question as uninsured, receiving Medicaid, homeless, low income, non-English speaking, racial and ethnic minority, or otherwise traditionally underserved group).Because respondents were required to complete the CCQ, there were no missing responses to individual items.
Satisfaction with electronic access to information varied notably by data type (Figure 1).A total of 34% of respondents indicated that they were very satisfied with their ability to access laboratory information from external organizations-this was the highest reported rate.The lowest reported rate of satisfaction was for information on preventive care: 21% of respondents indicated they were very satisfied with their ability to access that information.
Five percent of respondents reported either not having or not using access to outside immunization information, and an additional 23% were not at all satisfied with their access to it.Similarly, 5% of respondents did not have access to preventive care information and 24% of respondents were not satisfied with their access to information on preventive care.The next highest rate of dissatisfaction was reported for access to electronic notifications regarding patient visits to emergency departments, with 6% of respondents indicating that they either did not have or did not use this information, and another 19% saying that they were not at all satisfied with their access to that information.
The composite scale of physician satisfaction with electronic access to outside information across information types indicated substantial variation in PCP satisfaction as evident by the slope of the empirical cumulative distribution function (Figure 2).A mean of 70% of respondents reported a composite score equivalent to at least somewhat satisfied with all information types or better (with 13% of respondents indicating exactly somewhat satisfied on all items).Twenty-five percent of respondents reported a composite score equivalent to half somewhat satisfied and half very satisfied or better, 11% of respondents indicated that they were very satisfied with access to outside information for all information types, and 11% of respondents indicated that they were not at all satisfied with their access to half or more of the information types.
Family physicians who treated a large proportion of vulnerable patients (OR, 0.75; 95% CI, 0.58-0.99)(Table ) or did not have staff and linkages to community programs to address patients' social needs (OR, 0.52; 95% CI, 0.40-0.66)were less likely to report being very satisfied with their access to outside information, as were family physicians working at practices with more than 20 clinicians (OR, 0.68; 95% CI, 0.53-0.87).Family physicians who used Epic or Allscripts EHR products were substantially more likely than those using eClinicalWorks, the reference category, to be satisfied with access to external information (Epic: OR, 1.77; 95% CI, 1.25-2.51and Allscripts: OR, 1.73; 95% CI, 1.06-2.81).
When asked about the ease of using information when it was available, 23% of family physicians reported that it was very easy to use outside information to effectively care for their patients, 65% indicated it was somewhat easy to use, and 9% indicated it was not at all easy to use (Figure 3).
Reponses were somewhat less positive when physicians were asked about the ease of finding specific information from external organizations: 20% of family physicians indicated it was very easy to find specific information, 59% indicated it was somewhat easy, and 15% indicated it was not at all easy to find specific information.Reported ease of using information overall was significantly different than ease of finding specific information (χ 2 test P < .001).Thirty-eight percent of respondents indicated that information from outside organizations that used the same EHR was very easy to use.In contrast, 8% of respondents indicated that information from outside organizations that used a different EHR was very easy to use.Reported ease of using information from the same EHR was significantly different from ease of using information from a different EHR (χ 2 test: P < .001).When asked how often they encountered specific barriers, a substantial proportion of physicians indicated some level of missing information or difficulty finding information.Forty-four percent of physicians indicated a great extent of difficulty finding important information due to a large amount of low-value information (vs experiencing this to some extent or not at all), 40% indicated that information was not integrated into their EHR to a great extent, 32% indicated that the entire patient record was not available to a great extent, and 23% indicated that key information was missing within the patient's record to a great extent (Figure 3).In addition, 6% to 7% of respondents indicated not at all encountering these barriers.
Rates of reported ease of using information from the same and EHR developers varied substantially by EHR developer (Figure 4).Sixty percent of physicians using Epic indicated that information from outside organizations that used the same developer was very easy to use, as did 28% of respondents using eClinicalWorks and 34% of respondents using athenahealth.No more Ease of...

Not applicable Not at all
To some extent To a great extent Response to 2022 American Board of Family Medicine Continuous Certification Questionnaire by 2088 physicians.A, Ease of access to information in the electronic health record (EHR).B, Difficulties using the EHR.
than 11% of respondents indicated that information from outside organizations using different EHR developers was very easy to use.

Discussion
Primary care physicians have a critical need for information about their patients from all health care professionals, settings, and over time to fulfill their role in care integration, sense-making, and coordination.Nationally representative data on family physicians' experience accessing and using information from outside organizations highlighted substantial but incomplete access to external information and moderate ease of using that information along with substantial variation in physicians' experiences.Taken together, these data suggest that policy supporting interoperability has facilitated substantial but uneven progress in sharing information between organizations, and therefore numerous barriers must be addressed to ensure high-value access to interoperable information by clinicians working in a wide range of settings and with diverse technologies.
The composite findings related to satisfaction with access to information show substantial variation, with 11% of physicians reporting low satisfaction (ie, not at all satisfied with at least half of information types), just over one-quarter reporting high satisfaction (ie, very satisfied with at least half of information types), and 11% reporting that they were very satisfied with all information types.This variation highlights a challenge for initiatives aiming to improve information access for physicians because physicians at different points of satisfaction may have different interoperability needs or benefit from different initiatives to improve interoperability.For example, physicians who are highly dissatisfied-who we found to be more likely to serve vulnerable populations and to lack Ease of using ouside information from a different EHR developer

B
Response to 2022 American Board of Family Medicine Continuous Certification Questionnaire by 2088 physicians.Ease of using access to the same (A) or different (B) developers.Common Agreement (TEFCA), and efforts to make directories of health care professionals' electronic addresses widely available so that other professionals more often know where to direct information. 16More satisfied physicians may not need initiatives aimed at making it easier to participate in their first exchange network, since they are already participants.

JAMA Network Open | Health Informatics
Variation in satisfaction with access to different information types also suggests the need to address specific interoperability use cases that support the role of PCPs.As one example, the statistic that 28% of PCPs either do not have access to or are not at all satisfied with their ability to access immunization information suggests that part of the ongoing Centers for Disease Control and Prevention Data Modernization Initiative may need to focus on both PCPs' ability to use immunization information and the public health system's capacity to ensure that immunization information is available to PCPs. 6As a second salient example, one-quarter of PCPs indicated either not having or not being satisfied with notifications related to emergency department visits, which represents somewhat greater dissatisfaction than other information types. 17This is despite the fact that the Medicare Condition of Participation requires hospitals to provide PCPs with these event notifications.A model for evolving this requirement could be the approach to information reconciliation during care transitions within the Medicare Promoting Interoperability Program, in which clinician action to reconcile information into the EHR is incentivized, rather than only the sending and receiving of information.
One clear challenge is that patient information is often missing: 32% of respondents indicated frequently missing the patient's entire record.While some amount of missing information may be due to care processes that do not support routine communication between health care professionals, improved health IT can play an important role in addressing missing information.
Recent and ongoing policies, including full implementation of the 21st Century Cures Act's information blocking provisions and launch of the TEFCA, which is meant to facilitate data sharing between disparate health information exchange networks, have the potential to increase access to information. 4,18en information was present, family physicians indicated that it was frequently not easy to use.Policies aimed at supporting or requiring the adoption and use of data standards are one approach to enable greater usability of exchanged data.Starting in 2023, health IT certified by the Office of the National Coordinator for Health Information Technology, which is used by the vast majority of physicians, must support the exchange of data classes as defined in the US Core Data for Interoperability. 19The use of standards aims to ensure that information is widely available in a structure that the receiving EHR understands so that useful information can be extracted and displayed for user review and follow-up.Increased adoption of standards may enable industry-led solutions aimed at increasing the usability of information, such as Google's CareStudio and Epic's HappyTogether, which both aim to make information easier to find for clinicians but are only available to a select set of physicians.[22] In addition, to some degree it should not be surprising that PCPs report far greater ease of use of information from outside organizations that use the same, rather than a different, EHR developer.
A previous investigation found objectively greater interoperability between sites using the same EHR developer, 23 and the Epic Care Everywhere product is well known to offer useful connectivity between its customers. 24,25Although PCPs using Epic were the most likely to report information from outside organizations that used the same developer was easy to use, only 5% of Epic users

Figure 2 .
Figure 2. Cumulative Distribution of Family Physician Satisfaction With Access to External Information 100

Figure 3 .
Figure 3. Family Physician Experience Using Outside Information information to effectively care for your patients Finding specific information Using clinical information from clinicians outside your organization that use the same EHR vendor (developer) Using clinical information from clinicians outside your organization that use a different EHR vendor (developer) To what extent do the following occur?information due to a large amount of low-value information Information is not integrated within my EHR Entire record is not available Key information within record is missing/not available Difficulty finding necessary information within the record for other reasons Very Somewhat Not at all Not applicable Don't know

Figure 4 .
Figure 4. Ease of Using Outside Information From Electronic Health Record (EHR) Developers Primary Care Physicians' Satisfaction With Interoperable Health Information Technology JAMA Network Open.2024;7(3):e243793.doi:10.1001/jamanetworkopen.2024.3793(Reprinted) March 26, 2024 3/12 Downloaded from jamanetwork.comby guest on 03/31/2024 Based on examining the cumulative density function described, we used a multivariable logistic regression model with a binary outcome indicating whether respondents were very satisfied with at least 5 information types, representing approximately the top quarter of physicians.This model examined the association between the physician's practice characteristics, EHR developer, and satisfaction with access to external information.Model coefficients are reported as odds ratios (ORs)

Table .
Association Between Physician Satisfaction With Access to Information From Outside Organizations and Physician, Practice, and Developer Characteristic a a Total of 2031 observations.JAMA Network Open | Health InformaticsPrimary Care Physicians' Satisfaction With Interoperable Health Information Technology JAMA Network Open.2024;7(3):e243793.doi:10.1001/jamanetworkopen.2024.3793(Reprinted) March 26, 2024 6/12 Downloaded from jamanetwork.comby guest on 03/31/2024 Downloaded from jamanetwork.combyguest on 03/31/2024 the resources necessary to address social needs (Table)-likely receive very limited or incomplete information from outside organizations and might benefit from focused efforts to make it easier to join an initial exchange network.In contrast, physicians who are somewhat to very satisfied overall may benefit from initiatives that increase the scale of exchange, such as work to link disparate networks to fill in remaining information gaps, such as the Trusted Exchange Framework and