Effect of Clinical Decision Support at Community Health Centers on the Risk of Cardiovascular Disease

Key Points Question Does a clinical decision support system (CDSS) proven to reduce cardiovascular risk in integrated care settings also reduce cardiovascular risk in community health centers? Findings In this cluster randomized clinical trial of 18 578 eligible patients, although CDSS adoption rates were low, CDSS use was associated with significantly improved reversible risk of cardiovascular disease among patients with the highest levels of baseline risk. Meaning The use of a CDSS in community health centers has the potential to improve reversible risk of cardiovascular disease among socioeconomically vulnerable high-risk patients; strategies to increase CDSS adoption in this setting are needed.


210
CHC patient consent: We will interview a sample of patients who receive the CV Wizard CVD risk summary, to 211 assess their experience with it. No PHI will be collected during patient interviews. Participation will be 212 completely voluntary -providers will let patients with whom they used the CV Wizard tool that a researcher 213 would be interested in talking with them about their perceptions of the acceptability / utility of CV Wizard and 214 related discussion with the provider. Because a signed consent form would be the only document linking the 215 patient's name with the interview data, we will request IRB approval to provide a study fact sheet and obtain 216 recorded verbal consent from interviewees prior to data collection.

222
 Recruit 60 CHCs to participate in the study.

225
 Conduct extensive validity testing.

226
 Refine how the Health Partners system performs with OCHIN EHR data.

227
 Develop training materials.

228
 Engage patient stakeholders to adapt patient-facing aspect of CV WIZARD to meet needs of CHC 229 patients.

257
 Assess sustainability and differences in rate and speed of uptake across arms.

259
Year 5: Months 49 -60 260  Analyze qualitative and quantitative data on CV WIZARD tool use and outcomes 261  Create CVD WIZARD implementation guide for use by any CHCs

263
Quantitative data points: Below are details about the specific outcomes and measures for each. Please refer to 264 Table 6 further below for additional information.   Qualitative: We will take a 3-tiered approach to qualitative data collection. Below is a detailed 316 description of the process and potential data points.

317
All clinics: Collect i) baseline data on clinic characteristics ii) all 'trouble-tickets' relevant to CV Wizard 318 submitted to OCHIN via its member help request system, and all user feedback submitted via CV 319 Wizard's feedback mechanism; and iii) all exchanges from the implementation support webinars; iv) 320 periodically debrief with the practice facilitator and others involved with implementation support.

321
Selected clinics: Conduct semi-structured phone interviews with one care team member involved with 322 implementing / using CV Wizard, at 20 clinics (10 per Arm), about 10 months post-implementation. Sites will be purposively sampled for diversity in CV Wizard use rates. Follow-up interviews will be conducted 324 as needed, to explore perceptions, acceptance, and use of CV Wizard and the implementation support 325 provided. Prior to data collection, we will submit a study modification with an example of structured 326 interview questions for CHC staff.

327
Case study clinics: We will recruit two clinics per Arm for in-depth ethnographic case study, which uses 328 observation / personal interaction to explore the dynamics underlying implementation outcomes. We 329 will follow Arm 1 clinics for 2.5 years, and Arm 2 clinics for 1.5 years. Methods will include naturalistic 330 observation (of workflows, patient encounters, clinic meetings), key informant and in-depth interviews, 331 and collecting relevant artifacts (process maps, communications, etc.) Prior to data collection, we will 332 submit a study modification with an example of in-depth interview questions for CHC staff.

a. Analysis Plan 335
In this clinic-randomized pragmatic trial, 60 CHCs will implement CV Wizard via a staggered process. We 336 will compare patients' CVD outcomes in the 30 Arm 1 vs. 30 Arm 2 CHCs, in months 13-30 (Aim 1); 337 assess whether the revised implementation support materials expedite CDS adoption (Aim 2); and use 338 mixed methods to identify multi-level barriers / facilitators to adoption of the CDS tool (Aim 3).

339
Quantitative data: Study data will be audited to detect outliers and check variable distributions to 340 ensure they meet analysis assumptions. We will look for between-arm differences in the study CHCs' 341 baseline characteristics for all elements in Table 5, via chi-square, t-test, and nonparametric Wilcoxon 342 rank-sum tests, to verify that randomization yielded similar groups; analyses will be adjusted for any 343 differences. Analysis datasets will be hierarchically structured as encounters within patients within 344 CHCs. To be included in Aim 1 analysis a patient must have (a) an index visit in the 6 months after the 345 Arm 1 implementation date, at an Arm 1 or Arm 2 CHC, and (b) ≥2 CHC visits in the 12-month post-index 346 visit period. We will describe those who had inadequate follow-up data. For each study subject in Aim 1 347 analyses, CVD risk score / risk factor management status will be calculated at index visit and each 348 encounter during follow-up. Since clinics' ability to implement CDS is influenced by diverse contextual 349 factors, our analyses will consider the impact of factors expected to affect acceptance and uptake of CV 350 Wizard, specified based on our intervention logic model (Table 3) and past research from our team's 351 implementation scientists and others on uptake of EHR tools in CHCs.

352
CV Wizard can accept data from any EHR; the study CHCs use Epic, the same EHR as at HealthPartners. Working 353 with HealthPartners programmers who previously disseminated CV Wizard to other large care systems, OCHIN 354 programmers will (1) build tables for data extraction (e.g., medication, laboratory, diagnostic / problem list

366
Statistical power: To conservatively calculate power for our main outcome (CVD risk score) we estimated 60 367 clinics with >35 high-CVD risk patients, randomized to two arms; then, using means and standard deviations for 368 this measure from previous studies, we varied effect size associated with the group x time interaction and 369 intraclass correlation. We examined a 1.5%, 2%, 3% greater absolute reduction in risk score over time in 370 intervention vs. control CHCs; Table 7. Aggressive management can reduce 10-year CVD risk by an absolute 4-5% 371 in high-risk patients in a short time. We do not expect so large an effect, and have power to detect smaller 372 effects.

373
Qualitative data: Using a realist approach our process evaluation will explore 'what works, for whom and 374 in what circumstances' related to uptake / impact of the CDS system. It will be guided by our logic model 375 and the TAM's conceptualization of the relationship between user perception and CDS acceptance / use.

376
We will identify mechanisms of change (barriers / facilitators; impact of implementation strategy) at the 377 clinician and health care system levels, and assess how contextual factors impact outcomes.

378
We will triangulate process data from all sources for a deep understanding of CV Wizard's acceptance / 379 use in the CHCs. We will look at areas of consistency / inconsistency in these data (where different data 380 sources show the same thing, or do not). A grounded theory approach coupled with an immersion-381 crystallization process will be used to identify themes and patterns in the qualitative data. We will 382 emphasize factors influencing use of CV Wizard, and its impact on care decisions. Data collection and 383 analyses will be parallel and iterative, letting us identify salient constructs and knowledge gaps while 384 implementation is ongoing. The study team will meet regularly to discuss and integrate qualitative and 385 quantitative process evaluation and outcome data to help us identify factors that affect success. The 386 integration of mixed methods data will provide a more complete, nuanced understanding of the impact 387 and use of the CV Wizard system than either method alone, and permit examination of the reliability 388 and validity of the data sources.

b. Sharing of Results with Subjects 390 391
When / if the CV Wizard decision support tool is used, to ensure patient confidentiality, the name of the patient 392 is printed on the CDS sheet given to the patient and their care provider at the clinic visit.

394
This study does not carry the risk of incidental findings.

396
This study does not involve laboratory results.

398
c. Data and Specimen Banking 399 400 Not applicable. Please note that all data storage will be at OCHIN.

402
7. Privacy, Confidentiality, and Data Security 403 404 Quantitative data management, sharing and protection: Yearly datasets will be created with data from CV 405 Wizard's web server, imported into SAS, and securely maintained at OCHIN. Data will be extracted for all target 406 patients seen in study CHCs, and stored (limited data set) in a Web-based secure repository. Data from 407 subsequent encounters is stored and linked to the same patient. OCHIN, CHR, and HealthPartners all have expertise in high-quality data management, project operations, and multisite research collaborations. Data 409 standards in place exceed Level 1 HIPAA requirements, and meet most Level 2 requirements. Data structures 410 will remove personal identifiers. Analyses will be done in SAS® v9 or higher. Only aggregate EHR data will be 411 shared outside of OCHIN.

413
Qualitative data management, sharing and protection: Qualitative data will be securely saved at CHR, in a 414 database on a secure network; only de-identified data will be shared. Data will be catalogued within 1 week of 415 collection, and field notes within 24 hours. Interviews will be recorded and professionally transcribed by an 416 outside vendor (with who KPCHR has a Business Agreement with). Data will be entered into QSR NVivo; data 417 files will be password protected; data sharing agreements / protections will be set as necessary; transfers will 418 use a secure website.

420
Quantitative data: Data from the CHCs will come from OCHIN's EHR. OCHIN uses safeguards similar to those 421 used at CHR to ensure confidentiality when handling data. All patient-related study data will be de-identified 422 and unique patient identification codes will be used, and all data sources will be linked through a secure 423 relational database at OCHIN. Data sent to CV Wizard, and then returned to OCHIN for analysis, will be de-424 identified and protected as described below. Aggregate data will be shared with CHR through a secure data 425 transfer web site, as necessary. The CV Wizard CDS system is a web service that uses input EHR data to identify 426 eligible patients, compute CV risks, prioritize CV risk reduction, and provide treatment recommendations based 427 on a complex set of evidence-based algorithms. A web-based display of the results is then provided to the 428 primary care providers.

429
Confidentiality measures: All CHR investigators and project staff, all HealthPartners data managers, and all 430 OCHIN staff who handle data, sign annual confidentiality pledges and receive annual IRB and HIPAA training and

435
Protocol Secure (HTTPS) computer network. There is a double firewall in the Web service so that once the data 436 flow through the initial Web service firewall, the data cross another firewall into a new secure pathway that 437 once again employs SOAP, SSL, and HTTPS to process the data and provide recommendations. This includes 438 sending the data through a batch server for more efficient processing, but all within the double-firewall Web 439 service. Limited clinical data for all adults are initially extracted from the EHR to determine CDS eligibility.

440
We estimate that 15-25% may meet the eligibility criteria at which point more clinical information is extracted 441 and full processing through the clinical algorithms occurs and is displayed back to the providers. When / if the 442 CV Wizard decision support tool is used, to ensure patient confidentiality, the name of the patient is printed on 443 the CDS sheet given to the patient and their care provider at the clinic visit. This is needed to avoid mixing up 444 printed pages and giving protected health information (PHI) to the wrong patient. It is also desirable to have 445 each patient's name on this sheet of paper to assure the patient that the information on the paper is related to 446 their own health state and not someone else's. The technical aspects of this system are also relevant; the 447 printout is controlled from the CDS Web site, so the Web site must have the patient's name and other PHI. With 448 the analytic databases, measures will be taken to protect PCPs and patients from the risk of breach of 449 confidentiality: A unique study ID code unrelated to the EHR record number or other study subject-specific 450 information will be assigned to each patient and provider study subject and used to link data from various 451 sources needed for analysis. A crosswalk table linking this code number to a provider PCP or patient name or 452 medical record number will be destroyed within 12 months of completion of the linked databases needed for 453 study analyses. To minimize the risk that a PCP will act wrongly on the basis of information provided through CDS developed for this study, communication to providers will have a written explanation that the CDS is a 455 suggestion, not a mandate, and that the action should only be taken if judged to be clinically appropriate by the 456 treating provider on the basis of the patient's current clinical status and preferences.

Provisions to Monitor the Data to Ensure the Safety of Subjects 458
This is required when research involves more than Minimal Risk to subjects. Describe:

459
 Who will monitor the study data for safety.

460
 Who will verify data accuracy and conduct quality assessments.

461
 How objectivity in the monitoring process will be ensured.

462
 What data and/or events will be reported to the monitor or monitoring board and how 463 frequently.

464
 The procedures and methods that the monitor or board will use to evaluate the data.

471
This study involves data from OCHIN, and through OCHIN from 60 safety net CHCs. The web service for CV 472 WIZARD is hosted by Health Partners. All EHR data from OCHIN clinics are stored at OCHIN. Each clinic has 473 business use agreements with OCHIN to handle and manage PHI from their clinical data. EHR data will be linked 474 using OCHIN's unique patient identification codes, and data sources will be linked into a secure relational 475 database at OCHIN. When data linkages have been completed, the data will be fully de-identified. Data analysis 476 will be conducted at OCHIN. De-identified, aggregate data will be shared with study team members from CHR 477 and HealthPartners.

479
If necessary to share PHI, we will use a secure data transfer website with access limited to appropriate members 480 of the research team. IRB and HIPAA approval will be obtained for all study steps. CHR and OCHIN have expertise 481 in high quality data management and project operations and multisite research collaborations. Data structures 482 are typically designed to separate personal identifiers from other critical data, further enhancing protections.

483
CHR, HealthPartners, and OCHIN standards meet or exceed requirements for patient data safety established in 484 the federal HIPAA guidelines. Data structures will derive from the data confidentiality, security, and privacy 485 standards that CHR /OCHIN/HealthPartners have in place to meet or exceed all current HIPAA requirements.

486
These standards exceed Level 1 requirements, and meet the majority of Level 2 requirements, as specified by 487 DHHS in its Automated Information Systems Security Handbook. All analyses will be carried out using SAS® 488 version 9 or later. Quality control will begin with real-time, inter-field checks in the data at OCHIN. Additional 489 back-end checks (for missing data and logical inconsistencies) will be conducted to ensure the highest standards 490 of data reliability. We will examine the distribution and measurement properties of variables before making final 491 decisions about analyses.

493
In addition, we will compose a data safety and monitoring board (DSMB) of three members with expertise in 494 clinical trials methodology and the clinical domains addressed in the proposed research. Members will include 495 one clinician with expertise in CV disease and treatment, one expert in information technology, and one 496 statistical expert. A representative of the funding agency will be invited to participate via conference call at all 497 formal DSMB meetings. The PI will participate in the DSMB meetings in a limited way, as recommended in NIH 498 policy. The DSMB will provide input and guidance on the study evaluation and intervention protocols, including 499 quality assurance and safety issues related to the protocols and intervention strategy, as well as data-handling 500 activities. The DSMB will provide periodic input via email, conference calls, and annual meetings. The DSMB will meet within 3 months of the beginning of the study and twice annually through the study. A special focus of 502 interest will be the safety of patients exposed to the study intervention. The intervention provides point-of-care 503 CDS related to management of elevated CV risk and suggests evidence-based treatment options based on 504 national guidelines, and further vetted by clinical leaders at OCHIN and HealthPartners. CDS recommendations 505 provided as part of the intervention are designed to support clinicians' decision-making, not to override clinical 506 judgment. Adverse-events information will be collected by the Institute Project Manager and recorded on 507 standard forms based on those used in other trials (HL102144). Consistent with NIH, CHR, and HealthPartners 508 IRB policy, all adverse events will be promptly reported in writing to the NIH, the DSMB, the CHR IRB, and the 509 HealthPartners IRB.

511
9. Risks and Benefits 512 a. Risks to Subjects 513 514 One risk to clinic patients is: breach of privacy and subsequent loss of confidentiality, but this is 515 extremely unlikely. All CHR, HealthPartners, and OCHIN staff are highly trained and appropriately 516 certified in data security and confidentiality, and the current protection measures represent the cutting 517 edge of electronic protection. All patient data used in analyses will be de-identified by OCHIN staff, who 518 work with patient data from their member clinics on a regular basis and are appropriately trained in 519 data security. Thus, we feel any risk due to loss of confidentiality is remote.

520
The other potential risk is: the possibility that the intervention may provide advice on the basis of 521 national evidence-based CVD guidelines, which may be inappropriate for a given individual patient and, 522 if applied without further checking the clinical status of the patient, could lead to erroneous therapy, or 523 adverse events. However, the recommendations are evidence-based and operationalize current national 524 and regional standards of care and, therefore, the risk of untoward consequences of such clinical actions 525 is considered minimal. Moreover, this potential risk is routinely present in every clinical encounter in the 526 healthcare system. All treatment recommendations will be evidence-based, but acting on them will 527 ultimately be up to the provider's judgement; thus, we do not anticipate any clinical harm to patients 528 based on patients / their providers seeing the CV Wizard recommendations. We have described below 529 the methods used to minimize this risk. In addition, our qualitative data collection will include a focus on 530 potential harms to patients -such as anxiety -that may be incurred by their exposure to the CV Wizard 531 CVD risk assessment.

533
b. Potential Benefits to Subjects 534 535 Overall benefits: This study is expected to yield several key benefits. It will determine whether and how 536 clinical decision support (CDS) tools that address multiple aspects of guideline-based CVD prevention 537 and care, have provider-and patient-facing elements (enabling patient engagement), and involve some 538 workflow changes, can and will be adopted by, and will successfully increase rates of guideline-539 concordant care in CHCs. The use of sophisticated CDS in populations with persistent disparities is a long 540 overdue, critical step towards addressing CVD risk / outcome disparities in socioeconomically vulnerable 541 (i.e., CHC) populations in the US. This study will determine how persistent guideline-to-practice gaps 542 impacting high-risk / prevalence patients can be addressed using targeted, innovative, multi-level, team-543 based decision support tools. Thus, key benefits include: 1) Collecting, documenting and presenting data 544 on prioritized CVD risk to care teams and patients may result in patients receiving care that ultimately 545 improves their health. This potential benefit will incur no additional clinic visits or costs to patients. 2)

546
The intervention will increase care teams' knowledge about CVD risks that may influence their patients' health. 3) This intervention may bring improvements in EHR functionality to CHCs that would otherwise 548 not receive them. 4) Because this study develops and tests different ways of presenting CVD risk data in 549 clinic EMRs, it will provide needed information on how prioritized CVD risk data can be used to improve 550 care and services in community health center populations nationwide.

551
Findings will support further improvements in the US healthcare system to mitigate health disparities, 552 and will inform future efforts to bring cutting-edge CDS to CHCs. If the intervention significantly 553 improves primary care with respect to identification or management of elevated CV risk factors in 554 adults, the risk of CVD events and/or mortality related to elevated CV risk may be reduced later in life 555 for large numbers of patients. If the intervention fails to improve identification or management of CV 556 risk factors, that knowledge will also be important because it will direct the attention of investigators to 557 other potentially more fruitful lines of investigation. Thus, regardless of specific findings, the results of 558 this trial will provide important new knowledge that may ultimately contribute to improved care for 559 adults with elevated CV risk.

560
Quantitative: Patients at the study CHCs will have no defined personal benefit from this project.

561
Qualitative: Care team members at the study CHCs will have no defined benefits from participating in 562 this project. However, the intervention is designed to optimize identification and management of adult 563 subjects with elevated CV risk. Some providers exposed to this potentially useful CDS may use it to 564 improve their clinical care during the study or after.

Costs to Participants 567 568
There are no new costs to participants as a result of this study; however they will continue to pay for co-569 pays, routine treatment costs, etc. as part of normal care at their CHC.

571
11. Compensation to Participants 572 Describe any compensation provided to participants, for example, for time inconvenience, 573 discomfort, travel, or in the event of research related injury.

575
If applicable, describe how you will inform participants of this prior to their enrollment in the 576 study, including if payment will be prorated if the subject withdraws early from the study.

578
Note: payment may not be withheld as an incentive for participants to complete the study.

580
We will recruit two clinics per arm for an in-depth case study. These four clinics will receive annual 581 impact fees to compensate for staff time spent on this evaluation. Clinic impact fees are estimated at 582 $1,000 per clinic per year and total $10,000 in Years 2-5.

583
Clinic staff who participate in qualitative phone interviews, and patients who participate in an in-person 584 qualitative interview, will each receive a $25 gift card.

Resources Available 587 588
No special resources or expertise are required to conduct this study.

Drugs or Devices 591 592
Not applicable.

594
14. Multi-Site Coordination 595 596 CHR will act as the coordinating center for this study. We will ensure that: 597  All sites have the most current version of the protocol, consent document, and HIPAA 598 authorization.

599
 All required approvals have been obtained at each site (including approval by the site's IRB of 600 record).

601
 All modifications have been communicated to sites, and approved (including approval by the 602 site's IRB of record) before the modification is implemented.

603
 All engaged participating sites will safeguard data as required by local information security 604 policies.

605
 All local site investigators conduct the study appropriately.

606
 All non-compliance with the study protocol or applicable requirements will be reported in 607 accordance with local policy.

608
 Communication of problems, interim results, and study closure.

610
15. Community-Based Participatory Research 611 612 This study has system-level support: OCHIN's leadership strongly supports activating CV Wizard in its member 613 CHCs, which aligns with OCHIN's ongoing efforts to improve care and outcomes in CHCs via EHR-based 614 strategies. In all study steps, the study team will engage OCHIN's operational leadership (including co-I Karen

615
Parr, a CHC clinician and informaticist). We will also engage OCHIN CHC clinicians via existing communication

619
Operations Group, and the Clinical Review Advisory Committee. We received enthusiastic support from these 620 groups for this proposal. In the proposed work, we will engage OCHIN leaders and clinicians by working directly 621 with these groups to obtain their input and direction at key junctures.

Protocol Title
CV Wizard: Does a Prioritized, Point-of-Care Clinical Decision Support Tool Improve Guideline-Based CVD Risk Factor Control in Safety Net Clinics?

Objectives
This project aims to reduce disparities in cardiovascular disease (CVD) risk factor control and in rates of heart attacks and strokes among the low-income, racially/ethnically diverse Americans who receive primary care at safety net community health centers (CHCs). To achieve this objective, we will implement a successful clinical decision support (CDS) system-CV WIZARD (currently used in CVD care at several large, integrated health care systems)-in at least 60 CHCs, with a staggered randomized implementation approach. We will modify CV WIZARD as needed and feasible to fit CHC preferences. We will determine if use of this CDS improves CVD care, reduces disparities in CVD care and outcomes, and increases patient engagement in CVD treatment choices, in CHCs.

Specific Aims
Aim 1. Conduct a clinic-randomized trial of the impact of an evidence-based point-of-care CDS system on (i) overall CVD risk scores, and (ii) control of individual CVD risk factors (blood pressure; HbA1c, lipid levels; smoking; body mass index), among high CVD risk adult CHC patients.

Aim 2.
Develop and hone need-based implementation support protocols to help Arm 1 CHCs implement the CV Wizard CDS system into their standard workflows; assess whether use of the protocols developed for Arm 1 CHCs accelerates implementation and adoption of the CDS system in the Arm 2 CHCs. Supplement Addendum: Compare CVD risk management in high-risk patients at in-person vs. virtual care (VC) encounters, in the periods pre-and post-COVID's onset.
Aim 3. Conduct a mixed methods process evaluation, guided by the Technology Acceptance Model (TAM), to identify and address patient, provider, and delivery system barriers to uptake/impact of this CDS in CHCs. Supplement Addendum: In a subset of study clinics, explore how CV Wizard's individualized, prioritized CVD risk summary is used in VC to guide care decisions and engage patients in shared decision-making.

Background
Substantial progress in reducing cardiovascular disease (CVD) morbidity and mortality would be achieved if evidence-based guidelines for CVD risk factor control were implemented consistently in primary care settings. Electronic health record (EHR)-based clinical decision support (CDS) systems that identify uncontrolled CVD risk factors and provide individualized care recommendations improved rates of guideline-concordant CVD care in large, integrated healthcare settings, but little is known about how effective such CDS may be in safety net community health centers (CHCs). CHCs' socioeconomically vulnerable patients have far worse CVD risk factor control and higher rates of major CVD events than the general population. Implementing CDS that leads to improved CVD risk factor control in CHCs could reduce national disparities in CVD outcomes, but CHCs rarely have the resources to develop sophisticated CDS, and very few currently have such systems for CVD care. The proposed study is designed to address this issue. Supplement Addendum: Many healthcare settings rapidly shifted to virtual care (VC; defined here as video or telephone clinical encounters) because of the COVID-19 pandemic. Little is known about whether or how the shift to VC is impacting CVD risk management in primary care. The potential detrimental impacts of this shift are particularly concerning in community health centers (CHCs), which serve vulnerable patients who face health disparities (e.g., higher rates of unmanaged CVD risk than in the general population). However, the impact of VC on disparities in CVD care quality and outcomes remains largely unexplored. In response, we adapted CV Wizard for use in the VC setting and will assess how the COVID-19-driven shift to VC impacts CVD risk management in CHC populations. NOTE: The bibliography has been uploaded as a separate document in the eIRB.

Study Design
CV WIZARD is a clinical decision support (CDS) tool that provides point-of-care cardiovascular disease (CVD) care recommendations to the primary care team and the patient. CV Wizard identifies a patient's uncontrolled CVD risk factors, prioritizes those factors based on potential CVD risk reduction for that patient, and generates specific guideline-based treatment recommendations for each uncontrolled risk factor.
We will recruit at least 60 safety net community health centers (CHCs) and randomize them to implement CV Wizard in study year 2 (Arm 1), or 18 months later (Arm 2). Prior to full implementation of the CV Wizard in Arm 1, two CHC organizations will be engaged to pilottest CV Wizard. The pilot sites will test the accuracy and usability of the tool in clinic workflows.
We will compare outcomes from Arm 1 (immediate implementation) and Arm 2 (delayed implementation), which will enable us to measure the intervention's impact on CVD risk factor control in CHCs. Arm 1 CHCs will receive implementation support to address any barriers to adoption/sustained use of the CDS tool that are identified through study activities. We will apply these learnings to improve adoption rates in Arm 2.
Through this design, we will (1) assess whether this CDS tool is effective in CHCs, (2) identify and address barriers to its effectiveness and adoption, and (3) fine-tune strategies to support implementation of CDS systems in CHCs. To meet these aims, we will conduct mixed methods analyses (described below). Our assessment includes interviews with CHC staff, providers, and patients to determine if use of CV WIZARD increases patient engagement in CVD treatment choices.

a. Number of Subjects
NOTE: This study does not include any KP subjects.

Study Aim 1
Clinics -We will randomize at least 60 CHCs from within the OCHIN collaborative to either immediate (Arm 1) or delayed (Arm 2) CV Wizard tool implementation. While CV Wizard will be used with patients in those clinics, they are not considered study subjects.

Study Aim 2 Clinic Staff
• We will conduct semi-structured phone interviews with staff from 20 clinics (10 per study arm). We will interview at least one staff member from each of these clinics and may conduct follow-up "clarifying" interviews as needed with up to 5 additional staff per clinic. Up to 60 clinic staff members may be interviewed for this data collection component. • We will ask selected clinics (may include main trial and pilot clinics) to send a survey to their providers. While we do not know how many providers will respond to the survey, it is possible we will receive up to 500 responses.

Study Aim 3
Case Study Clinics -We will recruit four clinics from those taking part in the study (two per study arm; may include main trial and pilot clinics) for additional qualitative data collection. For these case study clinics, we will conduct semi-structured interviews with: • Patients whose most recent encounter involved CV Wizard. We will conduct up to 60 patient interviews in total (up to 30 per study arm). • Staff (providers and non-providers) who use the CV Wizard tool with patients or are involved in implementation efforts. We will conduct up to 60 staff interviews in total (up to 30 per study arm).
Supplement Clinics -We will recruit 10 clinics from those taking part in the study (from both study arms; may include main trial and pilot clinics) to collect qualitative data on how CV Wizard's individualized, prioritized CVD risk summary is used in VC to guide care decisions and engage patients in shared decision-making.
• We will interview 40 providers for whom CV Wizard was recommended at ≥3 VC encounters in the period post-COVID's onset. • We will interview 30 patients who had a VC encounter at which CV Wizard was used in the post-COVID period from these same clinics.

b. Inclusion and Exclusion Criteria
CV Wizard -Clinical Decision Support Tool (Study Aim 1) NOTE: Individual patients are not being enrolled for this clinic-randomized study. The goal of the study is to examine the uptake and impact of implementing CV Wizard into regular care processes. In this clinic-randomized trial, the intervention is provided at the clinic level and targets clinic processes that are part of routine patient care. CV Wizard will automatically identify patients age 40-75 years with either (a) reversible risk of cardiovascular disease (CVD) >10%, (b) diabetes + ≥1 uncontrolled risk factor or high A1c, or (c) existing atherosclerotic cardiovascular disease (ASCVD) + ≥1 uncontrolled risk factor for CVD events. However, clinics can manually run the CV Wizard tool for any patient aged 18 and older. Below is a list of vulnerable population categories and whether they will be included/excluded with regard to use of the CV Wizard tool:

Interviews with Clinic Staff and Patients (Study Aims 2 & 3)
• Children: Excluded • Pregnant Women: Excluded (when identified as having a current pregnancy diagnosis); may be incidentally included, but not in a focused or targeted manner • Neonates: Excluded • Prisoners: Excluded (this study does not include any clinics that serve the prison population) • Decisionally Impaired Adults: May be incidentally included, but not in a focused or targeted manner-just as part of the overall qualified sample NOTE: Children younger than 18 years will be excluded, as different clinical guidelines apply to children with CVD risk. The study will not preferentially recruit any vulnerable populations. However, the criteria listed above may result in the inclusion of pregnant women and/or decisionally impaired adults. It is important to systematically address CV risk in these populations, as these patients may be at risk for elevated CV risk and often have been underrepresented in previous research.

Interviews with Clinic Staff/Patients and Provider Survey (Study Aims 2 & 3)
Below is a list of vulnerable population categories and whether they will be included/excluded with regard to clinic staff/patient interviews and provider surveys: • Children: Excluded • Pregnant Women: May be incidentally included, but not in a focused or targeted manner-just as part of the overall qualified sample • Neonates: Excluded • Prisoners: Excluded • Decisionally Impaired Adults: May be incidentally included, but not in a focused or targeted manner-just as part of the overall qualified sample.

Center for Health Research-Northwest -The Center for Health Research-Northwest (CHR-NW)
is an academic-model organization that conducts independent research in a wide variety of areas including health services, public health, behavioral health, obesity and weight loss, mental health, maternal health, cost-effectiveness, cancer screening, genetics and genomics, and many others. In addition to conducting studies with our own databases and with KP's EMRs, CHR-NW researchers participate in a variety of formal and informal research networks. We collaborate locally with scientists and physicians at Oregon Health & Science University, and nationally with other Kaiser Permanente health plans and research divisions. We also undertake research in partnership with the Portland, Oregon-based OCHIN network of community health centers, and through our participation in networks such as HCSRN, CHARN, CRN, PCORnet, and many others. These networks allow our investigators to share data across multiple health systems and engage in multi-site research projects that access millions of individual health records.

Study Procedures Performed
• Provide coordination across study sites • Manage overall study budget and subcontract awards • Obtain IRB approval for study; submit continuing reviews; submit study modifications as needed • Prepare and submit all required reports to funder (e.g., recruitment reports; yearly progress reports) • Oversee DSMB creation and coordinate meetings • Conduct qualitative data analysis • Support subcontract sites with study tasks/deliverables as needed IRB Review Requirements -CHR-NW will rely on the Kaiser Permanente Northwest IRB for all IRB review.

e. Recruitment Methods
Clinic Recruitment (Aim 1) OCHIN's Research Associate/Project Manager will work with KPCHR and the OCHIN research team to recruit CHCs. OCHIN's clinical leadership (including CEOs of its member CHC organizations) may also help with recruitment, as needed. Recruitment will be targeted to optimize diversity in clinic baseline characteristics.
OCHIN utilizes a variety of recruitment methods when they invite CHCs to participate in research studies. Recruitment activities for this study may include: • Verbal conversations, in-person or by phone, with CHCs that the OCHIN research team has standing relationships with • Sending an introductory recruitment email to allow clinics to opt-in to the study, with study-related attachments: o Video summarizing the CV Wizard tool use in a clinical setting o Slide deck with background information about the tool o One-page summary document with a high-level description of the tool and the study • Presentation of recruitment materials at standing OCHIN webinars and meetings where members are in attendance. Any interested CHCs will be encouraged to contact study staff to discuss potential participation.
Clinic Staff Recruitment (Aim 2) OCHIN and CHR's research teams, working by email and phone with clinic contacts identified during the recruitment process, will identity and recruit those staff members from clinics who have experience using CV Wizard, for phone interviews. The team will perform similar liaison and recruitment activities for any necessary follow-up interviews.

Case Study & Supplement Site Recruitment (Aim 3)
• Clinics -Once clinic-level recruitment from study aim 1 is complete, an OCHIN Research Associate/Project Manager will identify and engage clinics to be case study and/or supplement sites. A separate Memorandum of Understanding (MOU) outlining the participation requirements for this portion of the study will be executed for these sites. Each clinic will be asked to appoint a staff member to be the liaison between researchers and clinic staff. • Staff Interviews -The KPCHR qualitative research team, supported by OCHIN study staff, will work with the clinic liaison to identify and recruit clinic staff to observe and/or interview for qualitative data collection activities. Interviews may take place either inperson (when site visits occur), or by telephone. (1) they may give the patient a flyer during an in-person visit; (2) they may send the flyer via post mail to a patient (in cases where the visit was virtual or the patient was identified after the visit); (3) they may indicate the patient is a good fit by (a) responding to a BPA alert in the EHR (a prompt that will fire when an eligible encounter is detected) or (b) using a smartphrase in the after visit summary (AVS) that indicates CV Wizard was used during the encounter; and/or (4) they may message the patient (e.g., via email or MyChart account) to share contact information for the research team (e.g., electronically sending the flyer, using the email message template provided by the research team). o Recruitment by Research Staff: If CV Wizard is used during a patient encounter that is being observed as part of a site visit, research staff may directly invite the patient to participate in the interview. If CV Wizard is used during a patient encounter when the research team is not onsite (e.g., virtual visits; post-covid qualitative data collection), research staff will rely on clinic staff recommendations (e.g., direct referral, positive responses to the BPA alert) and/or other indicators of CV Wizard use (e.g., smartphrase entry) to identify potential patients to recruit for the interviews. Once identified, the OCHIN research team will use a variety of methods to follow up with patients (e.g., phone calls, email, text message, and/or post mail). o NOTES: ▪ The same recruitment flyer will be used for case study and supplement clinics. ▪ Only OCHIN research team members will conduct the initial outreach to patients. ▪ All recruitment materials will provide the patient with information on how to contact the CHR qualitative research team if the patient is interested in completing the interview by telephone. Patients calling the study phone number will be asked to leave a message, and research staff will return phone calls to complete recruitment and conduct the interview.

Provider Survey Recruitment (Aims 2 & 3)
The OCHIN team will email clinic contacts to introduce the provider survey. Clinic contacts will be asked to distribute the survey to the providers in their clinics via email, using the message template provided.

f. Consent Process
Clinic Consent (Aim 1) Once a clinic agrees to participate, OCHIN's Research Associate/Project Manager will follow up with a Memorandum of Understanding (MOU) that outlines the study timeline and participation expectations. This MOU is signed by OCHIN's leadership (either the CEO or CFO) and leadership at the participating CHC. A separate MOU will be executed for the two (2) pilot sites that will be testing the tool prior to the go-live date for Arm 1 clinics.

Clinic Staff/Provider Consent (Aim 2)
• Clinic staff recruited for interviews will be provided with a study fact sheet (typically via email) that outlines the purpose of the research, study procedures, and contact information for the researchers and clinic liaison(s). As the risk to CHC staff is minimal, we will ask for their verbal consent for all qualitative data collection activities. We will also record interviews with staff permission; in these cases, we will document verbal consent as part of the audio recording. • Clinic staff invited to complete the provider survey will be presented with text in the survey invitation that contains the necessary elements of consent. Because the risk to providers is minimal, we will infer consent if the survey is completed.

Case Study & Supplement Site Consent (Aim 3)
• Clinics -A separate MOU outlining the participation requirements for this portion of the study will be executed for case study and supplement sites. • Staff Interviews -Clinic staff recruited for in-person interviews (at case study sites) will be provided with a study fact sheet that outlines the purpose of the research, study procedures, and contact information for the researchers and clinic liaison(s). They will be informed that participation in the interview is voluntary. Clinic staff recruited for telephone interviews (at case study and supplement clinic sites) will be read introductory text containing the required elements of consent (see provider/MA interview guides). As the risk to CHC staff is minimal, we will ask for their verbal consent for all qualitative data collection activities. We will also record interviews with permission from staff. When permission is given to record interviews, verbal consent will be documented on the audio recording. While PHI will not be solicited, it is possible that staff may inadvertently include PHI in their response to a question. • Patient Interviews -Patients recruited for in-person interviews (at case study sites) will be given the study fact sheet and will be informed that participation in the interview is voluntary. Patients recruited for telephone interviews (at case study and supplement sites) will be read introductory text containing the required elements of consent (see patient interview guide). Because risk is minimal, and some patients will be interviewed by telephone, we will ask the patient to provide verbal consent for participation. We will also record interviews with patient permission. When permission is given to record interviews, verbal consent will be documented on the audio recording. Given that the interview will revolve around the patient's perceptions of the CV Wizard tool and use of the tool itself is capturing health information, it is possible that PHI will be included on the recording. • Observations -We will conduct observation of workflows, patient encounters, and clinic meetings relevant to CV Wizard during in-person case study site visits. Unlike the staff and patient interviews, these observations will not be audio recorded; instead, researchers will take field notes. When opportunities to observe patient encounters arise, the patient will be introduced to research staff and verbal consent will be obtained prior to the observation (note: the study fact sheet will not be presented during observation, as this has the potential to interfere with normal clinic workflow). While PHI may be observed by research staff, it will not be written down as part of the field notes.

Justification for Waiver of Documentation of Consent:
• Clinic Staff/Providers -The research involves minimal risk to clinic staff. All staff will be informed that participation in the interview is voluntary and may be discontinued at any time. In addition, the research does not include procedures for which written consent is normally required outside of the research context (e.g., quality improvement efforts). • Patients -The research involves minimal risk to patients. While all patients invited to participate in an interview are, by definition, at increased risk of CVD, the research does not involve more than minimal risk. Patients will be informed that participation in the interview is voluntary. At any time, patients may choose to not answer specific questions or discontinue the interview, and all such requests will be immediately respected.

Justification for Waiver of HIPAA Authorization:
Patients -Interviews will include discussion of the patient's health information. While interviewers will not intentionally solicit PHI, it is possible that a patient might share this information during the semi-structured interview. While we could not practicably carry out this portion of the research (i.e., obtaining patient perceptions of the CV Wizard tool) without the possibility of collecting PHI, the PHI that might be collected is not the focus of our data collection efforts. Given that we will not be analyzing PHI, we will request that any identifiers be removed during the transcription process. The transcriptionist will be instructed to delete any identifiable patient information in the transcript and replace with the type of information removed (e.g., [*Name*]) and to delete the recording once the transcription is complete. Only transcribed interviews will be shared with the research teams at OCHIN and/or HealthPartners.

Study Procedures
a. Study Aim 1 Clinic Identification, Recruitment & Assignment Clinics will be included in the recruitment pool for this study if they (1) provide primary care to adults with high CVD risk annually, and (2) have been using OCHIN's EHR for at least 18 months. Once all clinics have been recruited as outlined in Section 5.e., they will be assigned to either arm 1 or arm 2. Clinics were grouped based on the baseline number of encounters each service area had, percent of patients with hypertension, and percent of patients who use tobacco. Five randomly generated groupings were created and the grouping that created the most even distribution across both Arms was then selected to assign service areas and the clinics within each service area to an Arm.
Programming CV Wizard to Accept OCHIN Data Working with HealthPartners' programmers who previously disseminated CV Wizard to other large care systems, OCHIN's programmers will (1) build tables for data extraction (e.g., medication, laboratory, diagnostic/problem list codes, vitals, smoking); (2) develop the interconnect routines needed to transmit EHR data to the CV Wizard web service, save response information in EHR flow sheets, and display the CDS results in the EHR; (3) create the alert to prompt CDS use in targeted patients; (4) develop "smart tools" (e.g., dot phrases) to facilitate documenting CV Wizard use and results in EHR encounter notes; (5) program the dynamic order sets that are tailored to facilitate clinical actions recommended for each patient; (6) create CDS use rate feedback reports; and (7) develop a process to manage provider feedback on clinical aspects of CV Wizard.

Validity Testing
Since 2007, CV Wizard's algorithms have been continuously modified, and re-validated after each modification. OCHIN programmers will validate that CV Wizard works correctly with data sent from OCHIN's EHR as follows: (1) OCHIN staff who routinely validate all changes made to the EHR will perform standard validation of data extracts for accuracy and completeness. This will include creating 'test patients' with variation in CVD risk factors and location of relevant data in the EHR, to assess how well CV Wizard performs with these patients. The validation team will also try to 'break' the CDS by testing how it performs with incomplete data. This will occur in a 'copy' of the Epic production environment used for testing EHR tools before they go into production. (2) We will conduct preliminary pilot work with 2-3 CHC clinician advisors to assess how CV Wizard performs in real (not test) patient data. We will ask them to identify 20-30 of their high CVD risk patients, run those patients' data through CV Wizard's algorithms, and review CV Wizard's results for accuracy and clinical plausibility.
(3) Next, CV Wizard will be activated in pilot clinics served by clinician advisors, to further identify potential errors; these sites will be excluded from subsequent study randomization. (4) Further quality checks will occur throughout the study, via CV Wizard's feedback mechanism and built-in website monitoring routes designed to identify aberrant web service input or output. Any changes needed to the data sent to the CV Wizard website, as identified through these processes, will be reviewed by OCHIN / HealthPartners.

Patient Engagement
CV Wizard's patient-facing component is designed to support patient engagement in CVD risk management. To improve this aspect of the tool, we will engage OCHIN's Patient Engagement Panel (PEP). This panel is comprised of >20 patients from OCHIN member CHCs who meet regularly to contribute input on research conducted at OCHIN. In months 1-6, we will meet with the PEP monthly to engage them in a user-centered process on whether and how to redesign the CV Wizard 'Patient View'.

Data Collection
All outcome variables collected for analyses align with RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance), a widely used framework for evaluating implementation success. Below are details about the specific outcomes and measures for each. Please refer to Table 6 further below for additional information.
Primary Outcome Measures • Implementation: User perceptions -Perceived ease of use, usefulness, acceptability of CV Wizard; intent to use it. • Maintenance over time -All measures over 2.5 years of follow-up, Arm 1; 1.5 years, Arm 2.  Age; gender; race / ethnicity; primary language; poverty level; insurance status at visit or insurance type through study period; # visits to that site / provider in last year; whether visit was with patient's primary care provider; provider type. Others TBD based on input from clinician advisors. Other patient comorbid conditions Renal function: diagnosed with end stage renal disease or chronic kidney disease; Charlson Comorbidity Score (Modified): Indicator of serious comorbid conditions that may shorten life expectancy, modified to exclude CV components; depression 296.xx, 311.xx diagnosis. Other relevant comorbidities TBD. Visit type New patient or established patient Provider type Provider type (PCP or other); # in patient panel Other clinic factors See Table 5; data from baseline survey CVD care / process Measurement Recommended care that is ordered.
Whether care suggestions were acted on within 7 days of the encounter; e.g., if CV Wizard suggests starting a statin, we will ask whether a prescription was issued.

b. Study Aim 2 & 3
Develop Training Materials Research team members will work with OCHIN's training and QI teams, with CHC clinician input, to develop an initial set of resources for implementing CV Wizard in CHC workflows. These materials will be augmented over time, as described below, but at first will include information on how CV Wizard works, how it might be integrated into CHC workflows, and how care team members can use CV Wizard to present/discuss CVD risks with patients in a manner that minimizes patient discomfort, reduces potential adverse effects, and maximizes patient engagement.
Implementation Support CV Wizard's impact will be affected by the CHCs' ability to implement it and achieve ongoing use at targeted visits. We will provide iterative, adaptive, pragmatic approach to providing the study CHCs with implementation support, as follows: (1) When Arm 1 CHCs implement CV Wizard, they will receive the basic implementation support that OCHIN's QI team provides with any new EHR functions (e.g., written materials, a live monthly webinar describing CV Wizard and suggested workflows for its use). We will also provide monthly CV Wizard use-rate reports to the Arm 1 CHC providers and clinic managers. CDS use rates are based on proportion of targeted visits where the CDS is viewed and / or printed.
(2) For the next 18 months, we will review monthly use rates of CV Wizard in the Arm 1 CHCs. The OCHIN practice facilitator will contact operational leaders at the CHCs with low adoption rates to ask about barriers to uptake, and what might help address these barriers; this aligns with the QI support that OCHIN regularly provides its member CHCs.
We will work iteratively with OCHIN's Implementation and Training teams to develop and provide additional support as needed to address identified barriers to adoption and sustained use at the desired level (80% of targeted visits). The specifics of this ongoing support will be determined based on CHC need. However, based on our team's expertise/experience and the implementation science literature, we anticipate that some CHCs may need additional training on integrating CDS tools with clinic workflows; harnessing use rate data to guide subsequent efforts; the evidence-based guidelines underlying the CDS; and how to use CV Wizard to engage patients. Such trainings will be provided to clinic leaders, providers, managers, QI coordinators and administrators at the study CHCs via written materials and monthly webinars. If indicated, we will also provide academic detailing relating to the CDS, and customized implementation plans. These resources will become part of our revised implementation materials. We will also conduct monthly interactive webinars to answer questions, identify additional barriers to uptake/needed training, and enable peer-to-peer learning in the Arm 1 CHCs. If there are CHCs for whom these approaches do not enhance uptake, we will confer with CHC clinicians, OCHIN's QI team, and our team's implementation scientists to identify and implement alternative strategies. (3) Arm 2 clinics will receive the revised implementation materials when CV Wizard is activated in their EHR. We will conduct a similar process to attempt to further improve these materials. We will collect quantitative data on which staff members from each study CHC take part in each training activity, and qualitative data on perceptions of the training strategies.

Data Collection
The mixed methods process evaluation will explore "what works, for whom and in what circumstances" related to CDS uptake, guided by our logic model and the Technology Acceptance Model (TAM). Our 3-tiered approach to data collection follows recent guidance on process evaluation of complex interventions which advocates complementing collecting key variables from all sites with in-depth data from purposively selected samples. This balances collecting data detailed enough to enable transferability (degree to which results can be applied to other contexts) and pragmatism (what is possible).
(1) All Clinics -We will collect the following information as it pertains to all clinics participating in the study: (1) baseline clinic data (e.g., location, patient characteristics, clinic characteristics, clinic size, etc.); (2) all 'trouble-tickets' relevant to CV Wizard submitted to OCHIN via its member help request system; (3) all user feedback submitted via CV Wizard's feedback mechanism; (4) all exchanges from the implementation support webinars; and (5) feedback from the practice facilitator and others involved with implementation support.
We will also collect attendee evaluations of all webinars. This pragmatic, adaptive approach will let us explore how to help diverse CHCs implement CDS, fine-tune materials and strategies for doing so, identify key barriers to adoption of CDS, and inform developing a guide for CHCs implementing CV Wizard. (i) % patients with up-to-date BP documented in EHR by 6 months postindex visit, and (ii) % patients with up-to-date A1c data documented in EHR by 6 months post-index visit, by encounter type Note: These outcomes will be adjusted for # of encounters in follow-up period. Analyses of A1c updates will account for whether prior A1c measure indicated lack of control and need for more frequent monitoring.

SUPPLEMENT ADDENDUM
Note: We will consider using coding of a hypertension diagnosis at index and follow-up encounters as a proxy for updated BP. Effectiveness (all eligible patients) Needed care provided within 7 days of encounter, by encounter type; e.g., If CV Wizard suggested (or would have suggested, if used) starting a statin, was prescription issued: YN; If an A1c test, was the test ordered: YN; If a referral to a nutritionist, was the referral made: YN; If medications to help quit smoking, was prescription issued; etc. Effectiveness (subset of patients with an index encounter and ≥1 encounters ≥6 months post-index) Change in CVD risk over time, measured as: (i) Decrease in total reversible CVD risk score (ACC/AHA 10-year pooled ASCVD risk score 64-67 ) (ii) Change from BP at index visit to increased, decreased, or no change, by end of follow-up period. (iii) Among patients with uncontrolled A1c (>8%) at index encounter, % in control by end of follow-up period; among patients with controlled A1c (≤8%) at index, % who had a 'relapse' by end of follow-up. (2) Selected Clinics • We will conduct semi-structured phone interviews with care team members involved with implementing/using CV Wizard, about 10 months postimplementation. Follow-up interviews will be conducted as needed, to explore perceptions, acceptance, and use of CV Wizard and the implementation support provided. Sites will be purposively sampled for diversity in CV Wizard use rates. • We will administer a provider survey that asks about use of the tool and solicits feedback. The survey will be programmed in REDCap and sent to clinic contacts via email. The clinic contact will be asked to email the providers at their clinics directly to request survey completion. Providers are informed (via the consent language that accompanies the survey) that (1) all responses/results are anonymous, and (2) we will aggregate non-identifiable response data to provide feedback to their clinic's study champion. NOTE: This survey may be administered to some clinics more than once over the course of the study period.

Potential covariates Description
(3) Case Study/Supplement Clinics • For case study sites, we will perform an in-depth ethnographic case study, which may include observation/personal interaction to explore the dynamics underlying implementation outcomes. We will follow the Arm 1 case study clinics for 2.5 years, the Arm 2 case study clinics for 1.5 years. Methods may include naturalistic observation (of workflows, patient encounters, clinic meetings), key informant and in-depth interviews, and collecting relevant artifacts (process maps, communications, etc.). • For supplement sites, we will perform interviews with CHC staff and patients.
Interviews will occur ≥3 months after CV Wizard is activated at a given clinic, and we will follow these clinics for one year.

a. Analysis Plan
Overview In this service area pragmatic clinical trial, we will implement CV Wizard via a staggered process. We will compare patients' CVD outcomes in the Arm 1 vs. Arm 2 CHCs, in months 18-35 (Aim 1); assess whether the revised implementation support materials expedite CDS adoption (Aim 2); and use mixed methods to identify multi-level barriers/facilitators to adoption of the CDS tool (Aim 3).

Quantitative Data
To be included in Aim 1 analyses a patient must have at least one visit in the 6 months after the Arm 1 implementation date at an Arm 1 or Arm 2 CHC and at least one additional visit at a study clinic in the 12-month post-index visit period. The first visit is referred to as the index visit. We will describe those who had inadequate follow-up data (did not have at least one additional visit in the post-index period). For each study subject, CVD risk score / risk factor management status will be calculated at index visit and each encounter during follow-up. While the tool can be run for patients of any age, we will restrict our analyses to patients 40-75 years of age.
We will test for between-group differences in baseline characteristics such as age, sex, race and ethnicity, insurance type, visit count, and health status using chi-square, t-test, and nonparametric Wilcoxon rank-sum tests as appropriate. We will also test for association between baseline characteristics and our outcomes of interest. We will examine the number of patients seen in each service area and clinic and each Arm and compare the characteristics of Arm 1 patients for whom the tool was used to those in Arm 1 for whom the tool was not used and to Arm 2 patients. Because the two arms were selected based on clinic size, percent of patients with a diagnosis for hypertension and the percent of patients that use tobacco, we anticipate there will be differences between the two groups resulting in the need to adjust for these differences. We will also examine characteristics at the service area and clinic-level in Arm 1 for whom the tool was used compared to Arm 1 service areas / clinics for whom the tool was not used and Arm 2 service areas / clinics.
Unadjusted trends in the monthly use and view rates for each of the study CHCs will be explored with use rates defined as: the number of encounters where CV Wizard was printed / number of targeted encounters with high-CVD risk patients per month and view rates defined as the number of encounters where CV Wizard was viewed / number of targeted encounters with high-CV risk patients per month.
We will also examine overall rates of each of the impact outcomes (BMI in normal range, controlled blood pressure, controlled LDL, controlled HbA1c, and non-smoking status) pre/post tool use to assess any changes in patterns for patients in Arm 1 for whom the tool was used compared to those in Arm 1 for whom the tool was not used as well as to Arm 2 patients. The date the tool was first used for each patient will be assigned as the start of the post-period.
Extent of adoption of a CDS tool like CV Wizard directly impacts its population-level impact. To differentiate between the tool's population level impact versus impact when used, we will conduct intent to treat (ITT) analyses to compare all targeted patients at intervention versus control clinics. We will also conduct effect of treatment on the treated (ETOT; also called per protocol) analyses limited to intervention CHC patients for whom the CV Wizard tool was used (results viewed / printed out) at the index visit, compared to intervention clinic patients for whom it was not used, and separately, to control clinic patients. The ITT analyses test the hypotheses that targeted patients in intervention clinics will have significantly lower CVD risk scores and higher rates of CVD risk factor control, at visits in the year post-index visit period, compared to similar patients in control clinics. The ETOT analyses test the hypotheses that targeted patients in intervention clinics for whom the CV Wizard CDS tool results are viewed and / or printed will have significantly lower CVD risk scores and higher rates of CVD risk factor control, at visits in the year post-index visit period, compared to similar patients in control clinics, and compared to patients in intervention clinics for whom the tool alerted but was ignored at the index visit.
Analyses will be stratified by first-time and repeat patients to see if there is a difference in tool use as well as examine whether the time between visits effects tool use among repeat patients.

Intent to Treat Analysis
Differences in change in outcomes will assessed using multi-level mixed models. Intra-class correlation coefficients will be obtained at the patient, clinic, and CHC organization levels for each outcome showed that much organization-level variance could be attributed to the clinic level. Thus, clustering was accounted for by random effects for clinic and patient, and a fixed effect for organization. Analyses will adjust for fixed effects: distribution of eligible patients by age, race / ethnicity, gender, rural / urban status, and federal poverty level (FPL) at index visit; number of ambulatory visits during the follow-up period; and time from index to last visit. These variables were selected a priori and will be confirmed by descriptive analyses, to determine whether there are significant differences between baseline intervention and control organization and patient characteristics. After checking distributions, negative binomial mixedmodels or poisson mixed-model will be used for reversible risk analyses, and linear-mixed models for all other outcomes.
Patients diagnosed with diabetes during the follow-up period will be excluded from reversible risk analyses because diabetes might be diagnosed more often with CDSS use. Presence of diabetes raises reversible risk estimations even if CVD risk factor control was addressed; this exclusion will remove the possibility of confounding based on likelihood on new diabetes diagnosis.

Effect of Treatment on the Treated Analysis
Per-protocol analyses require adjusting for loss to follow-up and off-protocol therapies or treatments, but if group differences vary too greatly, model misspecification can yield biased estimates. Propensity score methods are an alternative. We will match intervention organization patients in three categories of tool use (never used during follow-up, used once, used more than once) to control organization patients. Counts of tool use will include use at index visit or subsequent study period visits, excluding the last visit. Matching variables will include age, FPL, outcome of interest at baseline, race / ethnicity, sex, count of ambulatory care visits post-index visit, time from index visit to last study period visit, and clinic rural / urban status. Propensity scores will be estimated using nearest-neighbor matching with replacement.
As analyses in other settings indicated CV Wizard's potential for greater impact among patients with higher baseline risk, analyses of change in reversible risk will be stratified by baseline risk <10% or ≥10%. The assumption being that patients with lower baseline risk are less likely to improve. Additionally, we will run analyses by quartile of baseline reversible risk among patients for whom the tool was ever used during follow-up, matched to controls as described above. Analyses of change in reversible risk will include all patients meeting study inclusion / exclusion criteria; for other outcomes, analyses were restricted to patients with uncontrolled baseline risk. Linear mixed models will be used for all outcomes and checked for normality of residuals.

Supplement Addendum
(1) Descriptive Analyses (Aim 2a) • We will conduct a descriptive analysis on the following process outcomes: (1) Encounter patterns; (2) Rates of missed or cancelled appointments at in-person vs. VC (telephone, video) encounters; (3) Appointment wait times by encounter type, and (4) CV Wizard use (in encounters where there are CV Wizard CDS suggestions by encounter type. • We will use standard statistical tests (e.g., chi-square, t-tests) to describe measures in the periods pre-vs. post-COVID's onset.
• We will assess associations between encounter type and: (1) Action taken on needed care plan components; (2) Up-to-date documentation of BP and A1c by 6 months postindex visit (accounting for whether BP data were taken in person or reported by the patient); and (3) Change in CVD risk factors by end of follow-up period. To do so, we will conduct adjusted regression analyses using longitudinal electronic health record data. • To assess action taken on needed care, data for each factor will be extracted at each visit in the study period. We will use adjusted logistic regression models, with a dichotomous outcome of action taken or not taken. • To assess impact on individual CVD risk factors, data for each factor will be extracted at index visit and all visits in the follow-up period (coded as controlled vs. not). • Generalized linear mixed models (GLMM) with time nested within patients will be used to determine if risk factor control differs across encounter types, to estimate the effect of encounter type on change over time in CVD risk score and BP and A1c control. All encounters for each eligible patient will be included. CVD risk is influenced by factors amenable to clinical intervention (e.g., BP) and others that are irreversible (age); absent intervention, overall risk should increase with age. These analyses will be adjusted for or stratified by potential confounders at the patient, provider, and encounter level. This will include accounting for whether post-index visit encounters were in-person or VC, and the reason for the encounter, as in-person encounters in the post-COVID period may be for different reasons than VC encounters. • To assess whether CV Wizard moderated the relationship between encounter type and CVD outcomes, we will also conduct these analyses using only encounters where CV Wizard alerted the user that there are CDS suggestions. We will compare outcomes between encounters where CV Wizard was used (viewed or printed out) vs. not used, by including its use as a variable in the models described above. Data on race / ethnicity and gender will be obtained for all study patients; analyses will adjust or stratify for them as indicated.

Statistical Power
To conservatively calculate power for our main outcome (CVD risk score) we estimated 60 clinics with >35 high-CVD risk patients, randomized to two arms; then, using means and standard deviations for this measure from previous studies, we varied effect size associated with the group x time interaction and intraclass correlation. We examined a 1.5%, 2%, 3% greater absolute reduction in risk score over time in intervention vs. control CHCs. Aggressive management can reduce 10-year CVD risk by an absolute 4-5% in high-risk patients in a short time. We do not expect so large an effect and have power to detect smaller effects. Supplement Addendum (Aim 2b). This is a 2x2 repeated measures design with two groups of subjects measured at two time points. The primary goal is to compare change across time in group 1 (inperson index encounters) to change across time in group 2 (VC index encounters). A conservative estimate (based on preliminary results from the parent study) of 6000 patients in group 1 and 6000 in group 2 achieve 100% power to detect a difference in mean changes in CVD risk score of 5.0, with a standard deviation of 9.1 at the first time point, a standard deviation of 9.1 at the second time point, and a correlation between measurement pairs of 0.200. The significance level (alpha) is 0.050 using a two-sided, two-sample t-test.
Qualitative Data Using a realist approach our process evaluation will explore "what works, for whom and in what circumstances" related to uptake/impact of the CDS system. It will be guided by our logic model and the TAM's conceptualization of the relationship between user perception and CDS acceptance/use. We will identify mechanisms of change (barriers/facilitators; impact of implementation strategy) at the clinician and health care system levels and assess how contextual factors impact outcomes. Supplement Addendum: The parent study involves assessing how CV Wizard is used at in-person visits. We will account for the shift to VC by adding qualitative data on its use in VC. As in the parent study, process evaluation will be informed by the Technology Acceptance Model to understand (1) how CVD care takes place in VC and (2) the role of CDS in VC.
We will triangulate process data from all sources for a deep understanding of CV Wizard's acceptance/use in the CHCs. We will look at areas of consistency/inconsistency in these data (where different data sources show the same thing, or do not). A grounded theory approach coupled with an immersion-crystallization process will be used to identify themes and patterns in the qualitative data. We will emphasize factors influencing use of CV Wizard and its impact on care decisions. Data collection and analyses will be parallel and iterative, letting us identify salient constructs and knowledge gaps while implementation is ongoing. The study team will meet regularly to discuss and integrate qualitative and quantitative process evaluation and outcome data to help identify factors that affect success. The integration of mixed methods data will provide a more complete, nuanced understanding of the impact and use of the CV Wizard system than either method alone, and permit examination of the reliability and validity of the data sources. Supplement Addendum: Supplement data will be added to an existing NVIVO database to allow integration of findings. It will also be flagged to facilitate sub-analyses specific to this supplement's goals. To incorporate data from this study while allowing new themes to be considered, we will use an immersioncrystallization process which entails multiple iterations of data immersion, reflection, and code development and application, to identify new data themes or patterns related to the use of CV Wizard, and its impact on CVD risk management, in the VC context.

b. Sharing of Results with Subjects
If the CV Wizard decision support tool is used during a patient encounter, the provider has the option to print the provider-facing report. If the provider wishes to discuss the results with the patient, he/she may print the patient-facing report and give a copy to the patient or post to clinic's patient portal (if available). To ensure patient confidentiality (i.e., the correct patient receives the printout), the patient's name is printed on the top of the report.

c. Data and Specimen Banking
Not applicable.

Privacy, Confidentiality, and Data Security
Quantitative Data Management, Sharing and Protection OCHIN uses safeguards similar to those used at CHR to ensure confidentiality when handling data. All patient-related study data will be de-identified and unique patient identification codes will be used, and all data sources will be linked through a secure relational database at OCHIN. Data sent to CV Wizard, and then returned to OCHIN for analysis, will be de-identified and protected as described in the Security Measures section below. Only aggregate data will be shared with CHR through a secure data transfer web site, as necessary. The CV Wizard CDS system is a web service that uses input EHR data to identify eligible patients, compute CV risks, prioritize CV risk reduction, and provide treatment recommendations based on a complex set of evidence-based algorithms. A webbased display of the results is then provided to the primary care providers.
Qualitative Data Management, Sharing and Protection Qualitative data will be securely stored at both CHR and OCHIN. All data work will be done on password-protected workstations, in a secure environment (HIPAA-compliant). At CHR, only study team members have access to the saved data. At OCHIN, access to study data is limited to the research department.
Data will be catalogued within 1 week of collection and will be entered in QSR NVivo. Data sharing agreements/protections will be set as necessary, and electronic data transfers will use a secure website or encrypted email. Hard copies of fieldnotes will be stored in a locked office at KPCHR that only research staff can access.
Audio recordings of staff/patient interviews will be stored on a secure file server that is accessible via password protected computers. Audio recordings will be transcribed by a KP/CHR-approved vendor. Audio files and transcripts will be transferred between CHR and the transcriptionist using a secure file transfer website. All audio files will be destroyed at the end of the study.
When research staff travel to/from the case study site clinics, they will follow the CHR procedure for physically moving data. The data being transferred may include encrypted audio recordings and notebooks containing fieldnotes (e.g., observation date/time, provider/staff name, patient encounter activity, etc.). Because clinic visits may take place at various locations and times throughout the day, there may be some instances (e.g., early morning or late afternoon/evening) that necessitate study staff transporting study materials to their home. In these cases, staff will: (1) Place all data in a secured pouch with the appropriate labeling outlined in the CHR procedure for physically moving data.
(2) Transport data in the trunk of their vehicle and commute directly home.
(3) Bring the data into their home and place in a secure location.

Storage of Recruitment Data
As outlined in Section 5.e (Case Study & Supplement Site Recruitment), we may identify patients for qualitative interviews using a BPA alert in the EHR (where providers flag patients) or by pulling data on smartphrase use in the after-visit summary (AVS) that indicates CV Wizard was used during an encounter. In both cases, patient data will need to be accessed and stored for recruitment and tracking purposes. We will store the following data in OCHIN's instance of REDCap: patient name, encounter date, referring provider, clinic name, mailing address, phone number, email address, and notes entered by the study team related to recruitment efforts (e.g., outreach type, date of outreach, etc.). NOTE: Only OCHIN staff will have access to this information.
Additional Information about Sensitive Data We plan to pull data related to mental health diagnoses (depression, anxiety, and severe mental illness) to determine whether it explains variance in CV Wizard tool use. This data will be pulled from the EHR (i.e., this information will not be collected directly from patients that are participating in the qualitative interviews). We will follow the same safeguards for this sensitive data as we do for all data collected in this study (see sections on data management, sharing, and protection; confidentiality; and security). This data will be collected (as applicable) for all clinics participating in the study. Given that individual patients are not enrolled for this aim of the study, there is no consent form or PRA to which this information should be added. NOTE: Each OCHIN member clinic has business use agreements with OCHIN to handle and manage PHI from their clinical data, and each CHC agrees that their electronic health record data may be used in research as part of their membership agreement.

Confidentiality Measures
All CHR investigators and project staff, all HealthPartners data managers, and all OCHIN staff who work on research projects and handle human subjects data/have human subjects contact, sign confidentiality pledges and receive IRB and HIPAA training/certification.

Security Measures
Multiple measures are in place to ensure the security of PHI. Data transfer to and from the EHR (at OCHIN), the web service (at Health Partners), and the web display (accessed at OCHIN member CHCs) uses a Simple Object Access Protocol (SOAP) with Secure Sockets Layer (SSL) encryption over a Hypertext Transfer Protocol Secure (HTTPS) computer network. There is a double firewall in the Web service so that once the data flow through the initial Web service firewall, the data cross another firewall into a new secure pathway that once again employs SOAP, SSL, and HTTPS to process the data and provide recommendations. This includes sending the data through a batch server for more efficient processing, but all within the double-firewall Web service. Limited clinical data for all adults are initially extracted from the EHR to determine CDS eligibility.
We estimate that 15-25% of study clinic patients may meet the eligibility criteria at which point more clinical information is extracted and full processing through the CV Wizard clinical algorithms occurs and is displayed back to the providers. If/when the CV Wizard tool is used, to ensure patient confidentiality, the name of the patient is printed on the CDS sheet given to the patient and their care provider at the clinic visit. This is needed to avoid mixing up printed pages and giving protected health information (PHI) to the wrong patient. It is also desirable to have each patient's name on this sheet of paper to assure the patient that the information on the paper is related to their own health state and not someone else's. The technical aspects of this system are also relevant; the printout is controlled from the CDS Web site, so the Web site must have the patient's name and other PHI.
With the analytic databases, measures will be taken to protect PCPs and patients from the risk of breach of confidentiality: A unique study ID code unrelated to the EHR record number or other study subject-specific information will be assigned to each patient and provider study subject and used to link data from various sources needed for analysis. A crosswalk table linking this code number to a provider PCP or patient name or medical record number will be destroyed within 12 months of completion of the linked databases needed for study analyses. To minimize the risk that a PCP will act wrongly on the basis of information provided through CDS developed for this study, communication to providers will have a written explanation that the CDS is a suggestion, not a mandate, and that the action should only be taken if judged to be clinically appropriate by the treating provider on the basis of the patient's current clinical status and preferences.

Provisions to Monitor the Data to Ensure the Safety of Subjects
This study involves data from OCHIN, and through OCHIN, patient data from enrolled CHCs. The web service for CV Wizard© is hosted by Health Partners. All EHR data from OCHIN clinics are stored at OCHIN. Each clinic has business use agreements with OCHIN to handle and manage PHI from their clinical data. EHR data will be linked using OCHIN's unique patient identification codes, and data sources will be linked into a secure relational database at OCHIN. When data linkages have been completed, the data will be fully de-identified. Data analysis will be conducted at OCHIN. Deidentified, aggregate data will be shared with study team members from CHR and HealthPartners.
Yearly datasets will be created with data from CV Wizard's web server, imported into SAS, and securely maintained at OCHIN. Data will be extracted for all target patients seen in OCHIN study CHCs and stored (limited data set) in a Web-based secure repository. Data from subsequent encounters is stored and linked to the same patient. If necessary, to share PHI, we will use a secure data transfer website with access limited to appropriate members of the research team. IRB and HIPAA approval will be obtained for all study steps. CHR and OCHIN have expertise in high quality data management and project operations and multisite research collaborations. Data structures are typically designed to separate personal identifiers from other critical data, further enhancing protections. CHR, HealthPartners, and OCHIN standards meet or exceed requirements for patient data safety established in the federal HIPAA guidelines. Data structures will derive from the data confidentiality, security, and privacy standards that CHR/OCHIN/HealthPartners have in place to meet or exceed all current HIPAA requirements. These standards exceed Level 1 requirements, and meet the majority of Level 2 requirements, as specified by DHHS in its Automated Information Systems Security Handbook. All analyses will be carried out using SAS® version 9 or later. Quality control will begin with real-time, inter-field checks in the data at OCHIN. Additional back-end checks (for missing data and logical inconsistencies) will be conducted to ensure the highest standards of data reliability. We will examine the distribution and measurement properties of variables before making final decisions about analyses.
In addition, we will compose a data safety and monitoring board (DSMB) of three members with expertise in clinical trials methodology and the clinical domains addressed in the proposed research. Members will include one clinician with expertise in CV disease and treatment, one expert in information technology, and one statistical expert. The PI will participate in the DSMB meetings in a limited way, as recommended in NIH policy. The DSMB will provide input and guidance on the study evaluation and intervention protocols, including quality assurance and safety issues related to the protocols and intervention strategy, as well as data-handling activities. The DSMB will provide periodic input via email, conference calls, and annual meetings. The DSMB will convene for one meeting in project year 2 to review the study protocol and adopt a formal charter. They will then convene twice each year during project years 3-4 (the active intervention phase) and one final time in project year 5 to review study results. A special focus of interest will be the safety of patients exposed to the study intervention. The intervention provides point-of-care CDS related to management of elevated CV risk and suggests evidence-based treatment options based on national guidelines, and further vetted by clinical leaders at OCHIN and HealthPartners. CDS recommendations provided as part of the intervention are designed to support clinicians' decisionmaking, not to override clinical judgment. Information related to adverse events will be collected by OCHIN and/or HealthPartners and sent in writing to KPCHR for reporting to the IRB.

Risks and Benefits a. Risks to Subjects
Breach of Privacy/Loss of Confidentiality One risk to clinic staff and patients is breach of privacy and subsequent loss of confidentiality, but this is extremely unlikely. All CHR, HealthPartners, and OCHIN staff are highly trained and appropriately certified in data security and confidentiality, and the current protection measures represent the cutting edge of electronic protection. All patient data used in analyses will be de-identified by OCHIN staff, who work with patient data from their member clinics on a regular basis and are appropriately trained in data security. Thus, we feel any risk due to loss of confidentiality is remote.

CV Wizard Recommendations
Another potential risk is the possibility that the CV Wizard CDS tool may provide advice (according to national evidence-based CVD guidelines) that is inappropriate for a given individual patient and, if applied without further checking the clinical status of the patient, could lead to erroneous therapy or adverse events. However, the recommendations generated by CV Wizard are evidence-based and operationalize current national and regional standards of care and, therefore, the risk of untoward consequences of such clinical actions is considered minimal. Moreover, this potential risk is routinely present in every clinical encounter in the healthcare system. All treatment recommendations will be evidence-based, but acting on them will ultimately be up to the provider's judgement; thus, we do not anticipate any clinical harm to patients based on patients/their providers seeing the CV Wizard recommendations. We have described below the methods used to minimize this risk. In addition, our qualitative data collection will include a focus on potential harms to patients -such as anxiety -that may be incurred by their exposure to the CV Wizard CVD risk assessment.

b. Potential Benefits to Subjects
Overall Benefits This study is expected to yield several key benefits. It will determine whether and how clinical decision support (CDS) tools that address multiple aspects of guideline-based CVD prevention and care, have provider-and patient-facing elements (enabling patient engagement), and involve some workflow changes, can and will be adopted by, and will successfully increase rates of guideline-concordant care in CHCs. The use of sophisticated CDS in populations with persistent disparities is a long overdue, critical step towards addressing CVD risk/outcome disparities in socioeconomically vulnerable (i.e., CHC) populations in the US. This study will determine how persistent guideline-to-practice gaps impacting high-risk/prevalence patients can be addressed using targeted, innovative, multi-level, team-based decision support tools. Thus, key benefits include: (1) Collecting, documenting and presenting data on prioritized CVD risk to care teams and patients may result in patients receiving care that ultimately improves their health. This potential benefit will incur no additional clinic visits or costs to patients. (2) The intervention will increase care teams' knowledge about CVD risks that may influence their patients' health.
(3) This intervention may bring improvements in EHR functionality to CHCs that would otherwise not receive them. (4) Because this study develops and tests different ways of presenting CVD risk data in clinic EMRs, it will provide needed information on how prioritized CVD risk data can be used to improve care and services in community health center populations nationwide.
Findings will support further improvements in the US healthcare system to mitigate health disparities and will inform future efforts to bring cutting-edge CDS to CHCs. If the intervention significantly improves primary care with respect to identification or management of elevated CV risk factors in adults, the risk of CVD events and/or mortality related to elevated CV risk may be reduced later in life for large numbers of patients. If the intervention fails to improve identification or management of CV risk factors, that knowledge will also be important because it will direct the attention of investigators to other potentially more fruitful lines of investigation. Thus, regardless of specific findings, the results of this trial will provide important new knowledge that may ultimately contribute to improved care for adults with elevated CV risk.

Patients and Clinic Staff
Patients at the study CHCs will have no defined personal benefit from this project. However, the CVD reduction recommendations may help assist in better patient-provider communication around CVD risks and lead to improved treatment outcomes for high-risk patients.
Clinic staff at the study CHCs will have no defined benefits from participating in this project. However, the intervention is designed to optimize identification and management of adult subjects with elevated CV risk. Some providers exposed to this potentially useful CDS may use it to improve their clinical care during the study or after.

Costs to Participants
Patients will not incur any costs if CV Wizard is used during their office visit. Clinic staff and patients that participate in the qualitative interviews will not incur any costs.

Compensation to Participants
Case study site clinics will receive annual impact fees to compensate for staff time spent on qualitative data collection efforts. Clinics will receive $1000 for each year they participate as a case study site. Supplement study clinics that participate in qualitative activities will receive an impact fee of $1610 for their year of participation.
Patients who participate in Aim 2 and 3 qualitative interviews (either by phone or in-person) will each receive a $25 gift card. Staff who participate in Aim 2 and 3 qualitative interviews (either by phone or in-person) may receive a $25 gift card if their workplace allows them to