Implementation of a Secure Firearm Storage Program in Pediatric Primary Care A Cluster Randomized Trial

Importance
Increased secure firearm storage can reduce youth firearm injury and mortality, a leading cause of death for children and adolescents in the US. Despite the availability of evidence-based secure firearm storage programs and recommendations from the American Academy of Pediatrics, few pediatric clinicians report routinely implementing these programs.


Objective
To compare the effectiveness of an electronic health record (EHR) documentation template (nudge) and the nudge plus facilitation (ie, clinic support to implement the program; nudge+) at promoting delivery of a brief evidence-based secure firearm storage program (SAFE Firearm) that includes counseling about secure firearm storage and free cable locks during all pediatric well visits.


Design, Setting, and Participants
The Adolescent and Child Suicide Prevention in Routine Clinical Encounters (ASPIRE) unblinded parallel cluster randomized effectiveness-implementation trial was conducted from March 14, 2022, to March 20, 2023, to test the hypothesis that, relative to nudge, nudge+ would result in delivery of the firearm storage program to an additional 10% or more of the eligible population, and that this difference would be statistically significant. Thirty pediatric primary care clinics in 2 US health care systems (in Michigan and Colorado) were included, excluding clinics that were not the primary site for participating health care professionals and a subset selected at random due to resource limitations. All pediatric well visits at participating clinics for youth ages 5 to 17 years were analyzed.


Interventions
Clinics were randomly assigned in a 1:1 ratio to receive either the nudge or nudge+.


Main Outcomes and Measures
Patient-level outcomes were modeled to estimate the primary outcome, reach, which is a visit-level binary indicator of whether the parent received both components of the firearm storage program (counseling and lock), as documented by the clinician in the EHR. Secondary outcomes explored individual program component delivery.


Results
A total of 47 307 well-child visits (median [IQR] age, 11.3 [8.1-14.4] years; 24 210 [51.2%] male and 23 091 [48.8%] female) among 46 597 children and 368 clinicians were eligible to receive the firearm storage program during the trial and were included in analyses. Using the intention-to-treat principle, a higher percentage of well-child visits received the firearm storage program in the nudge+ condition (49%; 95% CI, 37-61) compared to nudge (22%; 95% CI, 13-31).


Conclusions and Relevance
In this study, the EHR strategy combined with facilitation (nudge+) was more effective at increasing delivery of an evidence-based secure firearm storage program compared to nudge alone.


Trial Registration
ClinicalTrials.gov Identifier: NCT04844021.


F
irearms are a leading cause of death for young people. 1 Firearm suicides make up approximately half of youth suicides, contributed to a cumulative loss of 1.3 million years of life from 2013 to 2020, 2 and comprise more than onethird of all youth firearm deaths. 3,4In 2021, approximately 4.6 million children in the US lived in households with a loaded and unlocked firearm. 5Deploying secure firearm storage programs with parents to reduce children's unauthorized access to firearms is a promising yet underused strategy that can reduce firearm mortality and suicide, given that most youth firearm suicides involve a family member's firearm. 6A modest increase in secure firearm storage could prevent up to 32% of youth firearm suicide and unintentional injury deaths. 7vidence-based secure firearm storage programs include brief firearm safety counseling between a pediatric primary care clinician and parent, plus locking device distribution. 8An American Academy of Pediatrics policy statement recommends the role that pediatric clinicians can play in firearm safety-related anticipatory guidance. 9,10However, few pediatric clinicians report routinely counseling parents on secure firearm storage (28%) or offering locks (2%) during well-child visits, 11 representing a research-to-practice gap.The 2024 Surgeon General advisory explicitly calls for implementation research focused on closing this gap in firearm injury and mortality. 12he primary aim of the Adolescent and Child Suicide Prevention in Routine Clinical Encounters (ASPIRE) trial was to test 2 strategies to implement a brief evidence-based secure firearm storage program, SAFE Firearm, [13][14][15] in pediatric primary care.This trial builds on preimplementation work [13][14][15] and explicitly targets barriers relevant to the use of secure firearm storage programs. 16We conducted a parallel cluster randomized effectiveness-implementation trial in which 30 pediatric primary care clinics within 2 health care systems were randomly assigned to receive either a strategy using the electronic health record (EHR; nudge), which included integration of a new EHR documentation template into the standard well-child visit workflow, or the EHR strategy combined with facilitation (ie, clinic-level support to embed the program; nudge+).We examined the hypothesis that the EHR strategy coupled with facilitation would perform significantly better than the EHR strategy alone with a 10% superiority margin.

Study Design, Population, and Setting
The ASPIRE trial was a 1-year unblinded parallel cluster randomized effectiveness-implementation trial conducted within 30 clinics that are part of 2 large and geographically, demographically, and culturally diverse health care systems that are members of the Mental Health Research Network, a consortium of 14 embedded research centers dedicated to improving patient mental health through research, practice, and policy.The health care systems provide comprehensive medical and behavioral health care to urban, suburban, and rural communities in Michigan and Colorado, respectively, and the demographic characteristics of patient populations served across the 2 systems are diverse in age, ethnicity, and race.The trial protocol (Supplement 1) was published previously 17 and was approved by the independent data safety and monitoring board that oversaw the study and the University of Pennsylvania single institutional review board.Informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization for the firearm storage program delivery (considered standard of care) and EHR data extraction were waived by the institutional review board because the research was considered no more than minimal risk and would not adversely impact the rights and welfare of participants; moreover, consent for every health care professional and parent was not considered practicable.
Participants included parents or legal guardians (hereafter, parents) at participating pediatric clinics who attended an in-person well-child visit with their child aged 5 to 17 years from March 14, 2022, until March 20, 2023, and pediatric physicians and nonphysician clinicians (eg, advanced practice nurses and physician assistants) delivering well-child visit care through pediatrics or family medicine within the 2 systems (hereafter, clinicians).The Consolidated Standards of Reporting Trials (CONSORT) reporting guideline was followed in reporting this study (Figure 1).

Randomization
Clinics were randomly assigned to either nudge or nudge+ using covariate-constrained randomization 18 to achieve balance with respect to 3 practice-level covariates.These included health care system, clinic size, and whether a clinician partner (ie, someone involved in study planning and advising) worked at the clinic.

Program
Prior to this trial, we adapted an evidence-based secure firearm storage program (Safety Check 15 ) using a systematic approach informed by constituent input. 13,14The adapted firearm storage program is a universal approach that includes brief counseling (<1 minute) between the pediatric clinician and parent(s) about secure firearm storage and offering free cable locks (Table 1 19 ).The firearm storage program does not include firearm ownership screening or documentation in the EHR 20 to increase acceptance among firearm-owning families. 21The focus

Key Points
Question Will an electronic health record (EHR) strategy coupled with facilitation (nudge+) perform better than an EHR strategy alone (nudge) at increasing clinician-reported delivery of an evidence-based secure firearm storage program (SAFE Firearm) during pediatric well visits?Findings In this cluster randomized trial among 47 307 well-child visits at 30 clinics in Michigan and Colorado, the chance of receiving the firearm storage program was significantly higher in clinics in the nudge+ group than in the nudge group.

Meaning
The findings suggest that an EHR strategy combined with facilitation may be more effective at increasing delivery of an evidence-based firearm storage program in pediatric primary care than an EHR strategy alone.on maximizing secure storage rather than curtailing firearm ownership is consistent with a harm-reduction approach. 9,22arm reduction embraces autonomy and pragmatism, supporting incremental steps toward more secure storage with the ultimate goal of reducing firearm accessibility to unauthorized people. 23Program materials are available on request. 24

Study Conditions
Study conditions were (1) nudge, the integration of a new EHR documentation template into the standard well-child visit workflow, and (2) nudge+, the nudge protocol combined with facilitation.For the nudge condition, we worked with health care system leadership and information technology to develop, pilot test, and deploy a new EHR documentation template within the standard well-child visit workflow to serve as a reminder and allow for tracking of the firearm storage program delivery.Specifically, we used a SmartList approach in each health care system's instance of the Epic EHR (Epic Systems).SmartLists are predefined lists of choices that are helpful for documenting values within progress notes that clinicians use repeatedly, thus improving efficiency.The SmartList was incorporated into the standard template used by all clinicians for a well-child visit for children aged 5 to 17 years.We designed the nudge strategy based on insights from behavioral economics [25][26][27] (eg, requiring clinicians to make an active choice regarding the firearm storage program delivery).During a routine well-child visit, the clinician keystrokes through the required prompts and selects responses from a drop-down list (eg, respond yes or no separately for each: "secure firearm storage discussed" and "cable lock offered") (eFigure 1 in Supplement 2).While we asked clinicians to complete it at each visit, clinicians sometimes left it blank (ie, skipped) which was treated as nondelivery.
For the nudge+ condition, the nudge strategy was combined with facilitation provided by health care system employees-who were members of the research team and not affiliated with the pediatric primary care clinics-to provide clinic-level implementation support.][30][31][32] Facilitators, trained through the US Department of Veterans Affairs Quality Enhancement Research Initiative facilitation program, spent on average 8.7 hours per clinic over the trial year (approximately 6% via in-person interactions, with the rest delivered via asynchronous and synchronous communication, including email and chat) delivering these supports.See Table 1 for a detailed description of conditions.The clinic in the nudge plus facilitation (nudge+, the EHR strategy plus facilitation support) condition that closed during the trial had their data included in analyses for the period that the clinic was open during the trial.f Visit data were extracted from the EHR from all participating clinics using the following criteria: (1) the well-child visit was completed at a participating study clinic from March 14, 2022, to March 13, 2023 (Michigan), or March 21, 2022, to March 20, 2023 (Colorado); (2) the visit was conducted in person; (3) the department assigned to the visit was pediatrics (both systems) or primary care (Colorado only); (4) the health care professional for the visit was a clinician within the health care system's medical group (Michigan only); (5) the health care professional for the visit had a specialty of pediatrics or had a specialty of primary care with 5% or more of their patient panel being pediatric patients (Colorado only); (6) the health care professional for the visit was a medical doctor, doctor of osteopathy, physician assistant, nurse practitioner, registered nurse, or resource clinician (ie, a clinician filling in at a clinic temporarily); (7) the patient was not on the health care system's research exclusion list; and (8) the patient's age was between 5.0 and 17.9 years at the time of the visit.

Statistical Analysis
We powered our study based on the availability of 32 clinics to randomize (18 in the Colorado system and 14 in the Michigan system).We determined 32 clinics would provide more than 90% power to detect a difference of 10% or more in the rate of program delivery in nudge+, assuming a type I error of 0.05, intracluster correlation of 0.03 within clinic, and 30% reach in nudge.In the months preceding randomization and trial launch, we evaluated all clinics for eligibility and identified that 12 were eligible in Michigan and 27 in Colorado.Due to resource constraints, we were unable to add more trial clinics at the Colorado site to reach 32 total clinics across the 2 systems.Thus, we only randomized and included 30 clinics in the trial (12 in Michigan and 18 in Colorado, with a random subset of Colorado clinics selected for exclusion).
We first calculated standardized mean differences (SMDs) to assess whether the clinic-level randomization sufficiently balanced clinic-and patient-level variables across study conditions.The primary analysis involved fitting generalized estimating equations with a binomial distribution and logit link 33,34 to estimate reach for nudge and nudge+ along with the risk difference between conditions.In all models, we specified an exchangeable working correlation structure at the clinic level and used the delta method to generate confidence intervals for the risk difference.We fit adjusted generalized estimating equation models that included terms for the variables used to constrain the cliniclevel randomization (health care system, clinic size, and clinician partner at clinic), variables prespecified based on results from our team's preimplementation pilot (patient sex), 35 and variables with an absolute SMD value that exceeded 0.2.We used Kauermann and Carroll 36 small cluster correction in all generalized estimating equation analyses.All analyses were conducted in Stata version 18 (Stata-Corp), using the xtgeebcv 37 procedure and the margins command (K.A.L. and K.B.).
We used a multiple hypothesis testing framework to evaluate the effectiveness of nudge+ compared to nudge, 38 considering 3 scenarios: (1) nudge+ is more effective than nudge by more than 10 percentage points (PP); (2) nudge+ is as effective as nudge, with a difference of no more than 10 percentage points in either direction; and (3) nudge+ is less effective than nudge by more than 10 percentage points.0][41][42] We treated each scenario as a separate hypothesis to be tested at a 5% significance level for the primary outcome, reach.Since these scenarios are mutually exclusive, we did not need to adjust for testing multiple hypotheses. 38If the 95% CI for the risk difference for reach did not contain any values within a region (nudge+ superior, equivalence, or nudge+ inferior), that region could be rejected.For the secondary outcomes (counseling and lock), we applied a Bonferroni correction to account for multiplicity, constructing 97.5% CIs to test the 3 scenarios at a 2.5% significance level.Two-sided P values less than .05were considered statistically significant for the primary outcome.Two-sided P values less than .025were considered statistically significant for the secondary outcomes.See Supplement 1 for a detailed statistical analysis plan.Variables with absolute value SMD Ն0.2 were included in the adjusted model(s).b Defined as the number of well-child visits at the clinic in the calendar year prior to the trial (2021).c Clinician partners were clinicians or administrative leaders who advised on study design and delivery of the program across clinics.d For each clinic, we calculated the percentage of well-child visits in the calendar year preceding the trial (2021) with patients with addresses in rural areas-ie, a micropolitan or noncore area according to the National Center for Health Statistics Urban-Rural Classification Scheme for Counties 2013. 43e Reporting race and ethnicity in this study was mandated by the US National Institutes of Health.Additionally, we used race and ethnicity to characterize the representativeness of the study population, generalizability of the results, and provide information about potential disparities and inequities.f Including Asian, American Indian or Alaska Native, Hawaiian/Other Pacific Islander, other, and more than one race.These groups were consolidated owing to small numbers.g Spanish was endorsed as the primary language by 1.2% of the sample (n = 554).All other primary languages (n = 53) were endorsed by less than 1% of the sample.

Study Population
across 30 pediatric primary care clinics were included in the final analytic sample (24 989 in nudge and 22 318 in nudge+).Table 2 43 presents clinic-and patient-level summaries by condition and the SMD of each variable.Hispanic ethnicity (SMD, 0.22) and Black race (SMD, −0.31) were imbalanced across arms.We included Hispanic ethnicity as a covariate in adjusted models but not race because models that included both health system and race did not converge.

Effectiveness of Strategies: Reach
Based on our adjusted model (Table 3), a higher percentage of well-child visits received both program components of the firearm storage program (ie, reach) as documented by the clinician in the EHR in the nudge+ condition (49%; 95% CI, 37-61) compared to nudge (22%; 95% CI, 13-31).Figure 2 presents 2-sided 95% CIs for reach and 97.5% CIs for each of its 2 components (counseling and lock) by condition.The lower limit of the confidence interval for the risk difference in reach between the groups exceeded the equivalence margin of 0.1.Based on this result, we can conclude that nudge+ had superior reach compared to nudge with statistical significance (P < .05) at the prespecified margin of 0.1.Results based on unadjusted models and adjusted models with alternative specifications were similar to those based on the adjusted models (eResults in Supplement 2).

Secondary Outcomes: Reach by Program Component
Nudge+ demonstrated superiority to nudge on clinicianreported offer of a cable lock alone, but not on clinicianreported counseling alone, at a margin of 0.1.The estimate of the risk difference for the lock component was 0.27 (97.5% CI, 0.12-0.43),and for counseling it was 0.21 (97.5% CI, 0.06-0.36).The lower limit of the 97.5% CI for lock exceeded the 0.1 equivalence margin, indicating superiority of nudge+.The lower limit of the 97.5% CI for counseling did not exceed the 0.1 equivalence margin; however, it exceeded zero, suggesting that nudge+ resulted in higher delivery of counseling.Results based on the unadjusted models were similar to those based on the adjusted models (eResults in Supplement 2).

Discussion
In this cluster randomized trial, we implemented an evidence-based program for secure firearm storage in 30 pediatric primary care clinics across 2 geographically, demographically, and culturally diverse health systems, and found that an EHR strategy with facilitation was more effective at increasing the firearm storage program reach (49%) compared to the EHR strategy alone (22%).Based on a clinically significant margin of 10 percentage points, this finding provides support for both clinical and statistical significance.Secondary analyses indicated that facilitation significantly increased the likelihood that a clinician would offer a lock, but not counseling.This work represents one of the first studies conducted across multiple health systems and leveraging rigorous methods from implementation science to investigate how to routinize secure firearm storage programs within pediatric primary care well visits as a universal population health approach.5][46][47][48] Previous studies that used a combination of educational and EHR-based strategies corroborate our findings that multiple discrete strategies, rather than education or EHR-based strategies alone, may more fully support clinician behavior change to close research-to-practice gaps. 46,49e found that when implemented alone, the EHR strategy, which was designed with insights from behavioral economics, [25][26][27] resulted in modest but meaningful change in clinician behavior vs rates of baseline delivery of counseling and lock offers observed in previous work.For example, previous work found that only 2% of clinicians report both delivering counseling about secure storage and offering a cable lock most or all the time. 11Facilitation, 50 which consisted of internal health system personnel providing support to clinicians and clinics around deployment of the new program, emerged as an important approach for boosting behavior change, consistent with prior literature. 51While facilitation included a number of discrete activities, the most commonly delivered activities included providing feedback, providing updates, and ongoing action and implementation planning.Providing feedback includes sharing clinical performance data related to reach with clinicians and administrators to support implementation, while providing updates includes sharing general information about the project or health system that is relevant to implementation.Ongoing action and implementation planning refers to using and updating the implementation plan, developed at the beginning of the trial, to guide the implementation effort over the year.Our findings support the hypothesis that in the case of this secure firearm storage program, facilitation, which provides targeted support to implementers, is an effective strategy.Notably, facilitation offered within this trial was not resource or time intensive, although it required staff who were trained in deploying this strategy, but nonetheless resulted in a robust effect.Health systems considering how best to implement the firearm storage program in their contexts should consider which strategies are most feasible for them to deploy this evidence-based program while balancing resources needed, available staffing, and scalability.This trial built on preimplementation work that likely contributed to the success of this effort.This formative work included interviews with all affected constituent groups, 16,20 systematic adaptation 13 of the program centering voices of end users, 14 development of a blueprint for implementation via implementation mapping, 52,53 and a pilot trial 35 to problem-solve barriers prior to fully powered trial execution.Further, this work was done within the context of a broader sociopolitical context that was affected by increasing rates of firearm injury and death in young people.However, even after observing robust practice change with respect to the firearm storage program delivery in this trial, there is still room for improvement.Greater implementation across wellchild visits is needed for this approach to fully realize its potential.Our qualitative work 54 with clinicians and leaders in these clinics, which is ongoing, will allow us to better understand barriers to implementation including patient-, clinician-, clinic-, system-, and societal-level barriers to further improve our approach.Generally, more work is needed to ensure that secure firearm storage information reaches all families, even if it cannot always occur within the context of well-child visits due to competing priorities and time constraints or in large health care systems, as in this trial.Potential complementary approaches could include messages through patient portals following visits and implementation of the firearm storage program in other contexts where children and parents frequent (eg, community health centers or federally qualified health centers, adult primary care, afterschool programs, and faith-based organizations).

Limitations
Although this study has methodological strengths, including a demographically diverse participant sample, rigorous randomized design, and constituent-partnered approach, it has limitations.These include: (1) no control condition (as required by the health care systems we were working with); (2) the limitations of EHR data and strategies (ie, our primary outcome extracted from the EHR does not allow us to understand granular delivery of the program [ie, depth, quality, and dosage]; we were not able to extract parentlevel characteristics; and not all clinicians actively document in the EHR during visits, thus potentially reducing the effectiveness of the EHR strategy); (3) patients, clinicians, and clinics could have differed at baseline in important but unmeasured ways; (4) heterogeneity in health care systems regarding involvement of trainee clinicians which we were unable to measure 49 ; and (5) only providing cable locks, given recent work suggesting that they may not be the preferred locking mechanism for firearms that are kept for protection purposes. 55,56We provided cable locks given they are the most scalable approach for distribution in health care systems.After-visit summaries and parent handouts at clinics linked to the study website, which described alternative locking mechanisms and resources for those who did not prefer cable locks; we view the locks as an entry point for the discussion between the clinician and parent.

Conclusions
Our EHR and facilitation strategies led to robust practice change related to the firearm storage program delivery during pediatric primary care visits across 2 health care systems, with the EHR strategy plus facilitation being more effective at increasing program reach compared to the EHR strategy alone.Future work should explore the impact of program delivery on parental secure firearm storage, which is the primary mechanism through which injury and mortality risk is reduced for young people.

Nudge+ inferior
The 95% CI generated by the generalized estimating equation model for the risk difference of the primary outcome (reach) between the nudge condition and the nudge+ condition was (0.14-0.41).The 97.5% CIs for the risk difference between conditions for our secondary outcomes were as follows: counseling, 0.06-0.36and lock, 0.12-0.43.Risk differences were computed using a generalized estimating equation model that adjusted for health care system, clinic size, clinician partner at clinic, patient sex, and an indicator of Hispanic ethnicity.Because the CIs for reach and lock are both contained within the nudge+ superior region alone, we can conclude that nudge+ was significantly better than nudge at a 10% margin for both reach and lock.Because the CI for counseling partially overlaps with the equivalence region, nudge+ cannot be considered superior for counseling.

Figure
Figure 1.Adolescent and Child Suicide Prevention in Routine Clinical Encounters (ASPIRE) Trial Clinic Screening, Randomization, and Follow-Up

Figure 2 .
Figure 2. Adjusted Model Results for the Risk Difference in Reach and Its 2 Components 0.5 1. Adolescent and Child Suicide Prevention in Routine Clinical Encounters (ASPIRE) Trial Clinic Screening, Randomization, and Follow-Up e

Table 1 .
Firearm Storage Program and Implementation Strategy Components Delivered in Each Study Condition aClinician partners within the pediatric department at each health care system who advised on study design and delivery of the program across clinics.aInformationtechnology specialists and the research team developed (in collaboration with clinician partners at the health care systems) and deployed a SmartList to add to the well-child visit template; it included prompts related to both program components (counseling and locks).SmartLists are predefined lists of choices that users can select, which are helpful for documenting values that a clinician is required to use repeatedly.The prompt was refined based on behavioral economic theory and pilot tested prior to trial launch.OutcomesAll data used in our analyses, including reach and patientreported demographic characteristics (eg, age, race, ethnicity, and sex), were extracted from the EHR.Reporting race and ethnicity in this study was mandated by the US National Institutes of Health.Additionally, we used race and ethnicity to characterize the representativeness of the study population, generalizability of the results, and provide information about potential disparities and inequities.Response options for patient-reported demographic characteristics were set by the health care systems.The primary outcome, reach, was a visit-level binary indicator of whether the parent received both components of the firearm storage program (counseling and lock), as documented by the clinician in the EHR.Secondary outcomes explored whether the parent received each individual program component (ie, separate binary indicators for counseling and lock).

Table 3 .
Primary Outcomes From Adjusted Model a Marginalized over the empirical distribution of model covariates.The marginal probability of each outcome can take a value between 0 and 1, with values close to 0 indicating low delivery and values close to 1 indicating high delivery.The risk difference in the rightmost column can take values −1 to 1 with positive values indicating higher delivery under nudge+ than nudge.c Reach is a composite outcome consisting of counseling and lock and thus is driven by the component with lower delivery rate.