Effect of a Mobile App on Prehospital Medication Errors During Simulated Pediatric Resuscitation

Key Points Question Does the use of an evidence-based, custom-designed, mobile app result in decreased rates of pediatric medication errors compared with conventional preparation methods in prehospital emergency care? Findings In this multicenter, simulation-based, randomized clinical trial including 150 advanced paramedics in 14 emergency medical services centers and 600 drug preparations, the proportion of medication errors committed during sequential preparation of 4 intravenous emergency drugs in prehospital settings was significantly decreased with the use of the app in absolute terms by 66.5%. Meaning Dedicated mobile apps have the potential to change practices in prehospital emergency medicine and to improve quality of care in pediatric populations by decreasing the rate of medication errors.


57
Children are a vulnerable population with specific medical needs compared to adults. The fast, 58 accurate and safe preparation and administration of intravenous (IV) drugs is both complex and time-59 consuming in pediatric critical situations, such as cardiopulmonary resuscitation (CPR) 1-4 . Most drugs 60 given intravenously to children are provided in vials originally prepared for the adult population. This 61 leads to the need for a specific individual, weight-based drug dose calculation and preparation for each 62 child that varies widely across age groups 3, 5-9 . This error-prone process and the lower dosing error 63 tolerance of children 10 place them at a high risk for life-threatening medication errors 3, 5, 6, 11 . Despite 64 well-equipped and staffed environments with numerous available safeguards, direct IV medication 65 errors have been reported in up to 41% of cases during simulated in-hospital pediatric resuscitations, 66 65% of which were incorrect medication dosage, thus making it the most common error 12 . The rate of 67 errors is also important in the high-risk prehospital setting, which is reported as occurring in more than 68 30% of all pediatric drugs administered, with an error rate for epinephrine dosage alone of more than 69 60% 13 . In this particular context, initial care has to be delivered quickly by emergency medical

76
In resuscitation, time is a critical success criterion. During the first 15 min of pediatric CPR, survival 77 and favorable neurological outcome decrease linearly by 2.1% and 1.2% per min, respectively 17 , and 78 rely in part on drug preparation time both in-18 or out-of-hospital settings 19 . Among non-shockable 79 pediatric out-of-hospital cardiac arrests, each minute delay to epinephrine delivery is associated with a 80 9% decrease in survival odds 19,20 . Regrettably, most patients in the prehospital setting receive 81 epinephrine more than 10 min after EMS arrival 19,20 . Therefore, the chain of survival critically relies 82 on early out-of-hospital CPR by EMS 21 and onsite administration of emergency drugs without delay 19, 83 20, 22 before a rapid transfer to pediatric emergency departments (PED) and advanced care. Despite 84 efforts to solve this problematic, out-of-hospital preparation and delivery of pediatric emergency drugs 85 remain a worldwide health challenge. The evaluation of new methods to reduce pediatric medication 86 errors is of paramount importance, but research in this area is scarce.

88
In a previous multicenter, randomized crossover trial, we showed that medication errors, time to drug 89 preparation, and time to drug delivery for continuous infusions during simulation-based pediatric in-

95
The primary aim of this multicenter study protocol is to compare the impact of the app with 96 conventional calculation methods for the preparation of direct IV drugs during standardized, 97 simulation-based, pediatric out-of-hospital cardiac arrest scenarios.

99
We hypothesized that the use of the app might extend and scale-up our previous multicenter in-100 hospital observations by similarly reducing the occurrence of medication errors and time to drug 101 preparation and delivery when used in out-of-hospital settings by paramedics, independent of EMS 102 skills.

104
We will conduct a prospective, multicenter, randomized controlled trial with two parallel groups in 105 several EMS located in different regions of Switzerland, a pluralistic country with 4 official languages 106 without uniformly standardized or benchmarked EMS clinical guidelines, protocols or operating 107 procedures. Participants allocated to the conventional preparation method group will be allowed to use 108 a calculator, but not any other drug preparation support enabling weight-based drug dose calculation, 109 such as an online calculator or a mobile device app. The final correct volume of drugs to be drawn will 110 not be released to the paramedics. To calculate the volume of drug to inject, the desired drug to be 111 delivered in mg is first selected from a calculation of the original weight-based prescription in mg/kg.

112
The next step is to convert the mg into mL of drug to be drawn. For the purpose of the study, we will 113 not select drugs that can be directly drawn from the vial without calculation. Participants allocated to 114 the app group will not be allowed to use any other drug preparation support.

133
Registered paramedics working in Swiss EMS are eligible for inclusion in the study. Inclusion criteria 134 are having followed a standardized 5-min introductory course on the use of the mobile device app and 135 willing to grant written informed consent. They will be excluded if they had previously used a 136 numerical device aimed at helping with drug preparation. All participants will be assumed to have an 137 equivalent competence with direct IV drug preparation and dose calculation as this is part of their  The app was developed at Geneva University Hospitals (Geneva, Switzerland) following a user-141 centered and evidence-based approach with emergency department caregivers, software developers 142 and ergonomists. On the basis of pediatric resuscitation observations and focus groups, the team 143 worked closely together to identify the key functionalities and processes to be implemented 29 . The app 144 lists all the available resuscitation drugs for either direct IV injection or continuous infusion with 145 doses automatically adapted to the weight or age of the patient based on information entered when 146 starting the app. With one touch, any of the listed drugs can be selected and shown with a detailed 147 preparation according to a standardized and simplified pathway. In the case of a direct IV injection, 148 this pathway is composed of two steps: 1) drug selection and 2) conversion of the prescribed dose in 149 mg/kg into a volume in mL. If necessary, an additional step is provided for the dilution of the initial 150 drug concentration with compatible fluids (sodium chloride 0.9%, etc.). For each drug, the exact 151 amount to prepare is clearly displayed and thus avoids the need for calculations. This is based on the 152 app's ability to automatically calculate the optimal weight-based final volume to inject and describe 153 the preparation sequence required to achieve it, independent of the user's competency in this domain.

154
When using the app, the user can interact with it at any time. Multiple drugs can be prepared and run 155 in parallel, including continuous infusions. All actions by the user are sequentially saved locally on the 156 device in historic files to preserve information that can be retrieved at any time for debriefing or 157 medicolegal purposes. Historic files can also be erased or safely exported and saved in electronic 158 health records.

160
On the day of participation after random allocation, each participating paramedic will: 1) complete a 161 survey collecting data regarding their demographics, care training, and simulation and computer 162 experience; 2) receive a standardized 5-min training session on how to use PedAMINES; as well as 3) 163 a presentation of the simulation manikin characteristics. The paramedics will then be asked to perform 164 a 20-min highly realistic pediatric CPR scenario on a high-fidelity WiFi manikin (Laerdal SimBaby,

165
Laerdal Medical, Stavanger, Norway). The procedure will be standardized across all sites to follow the 166 same chronological progression and range of difficulty in order to ensure that each participant is 167 exposed to exactly the same case, with similar challenges in decision-making and treatment 168 preparation provided on the same manikin. The uniform delivery of the scenario throughout the entire 169 study will minimize confounders. Study team members will only adapt to the progression speed of 170 participants through the scenario by maintaining a stressful resuscitation atmosphere. The scenario 171 will be conducted in an out-of-hospital simulated child bedroom environment to increase realism.

172
High levels of realism are known to immerse participants in the simulated experience and prevent 173 confounding variables that might potentially affect the way individuals perform 30 . The room will be 174 exclusively devoted to the simulation to prevent unexpected interruptions or external stimuli. Portable 7 monitoring alarms will be activated to increase realism and stress. The scenario will be filmed with 3 176 action video cameras (GoPro, Hero 5 Black edition; San Matteo, CA, USA) worn by the participating 177 paramedic and placed within the room.

179
The untimed portion of the simulation will involve a resuscitation team comprised of the same 2 study 180 team members throughout the whole study period. One member (LB) will play the role of a second 181 paramedic leading the CPR and assisting the study participant by performing chest compressions and

198
The untimed portion of the simulation will start by turning on the 3 video cameras and a fitness watch 199 on the participant's wrist, with both paramedics waiting outside the room. Both will be invited to enter 200 the child's bedroom by the patient's father. When entering the room, a clinical statement to recognize 201 the life-threatening condition of the patient, including his exact weight and age, will be given by the 202 father as follows: "Here is Junior, a 12-kg, 18-month-old boy who suddenly collapsed 15 minutes ago.

203
Oral pills belonging to his grandmother were found in his mouth and on the floor of his room. He is 204 unconscious, pale and not breathing". Looking at the empty medicine boxes, the second paramedic 205 says "that the pills are an oral tricyclic antidepressant, as well as antidiabetic medication". At this 206 moment, the second paramedic says "OK, I'll take the lead of the resuscitation" and asks the 207 participant to take a central pulse. Due to the invariable absence of a pulse, the participant is asked to 208 assist the leader in doing a 2-min full-course massage and ventilation (30:1 ratio) maneuver, with the 209 massage carried out by the participant to increase his/her stress level. During this time, the leader 210 places a supraglottic airway device in the manikin's throat and the defibrillator patches on the trunk.

211
The physician then enters the room and an asystole rhythm is recognized and verbalized. Both the 8 physician and leader rotate the person performing the massage-ventilation maneuvers (new 15:2 ratio), 213 ask the participant to place a vascular access on the manikin's right hand (not intra-osseous to preserve 214 the manikin integrity; first IV attempt successful) and then to prepare the drugs.

216
On the basis of the American Heart Association pediatric cardiac arrest algorithm for asystole 33

228
The physician says "the patient has a severe hypoglycemia" and prompts the participant to prepare and 229 inject a direct IV bolus of 4 mL/kg dextrose 10%. Return of a state of consciousness ensues with 230 normal vital signs, but a wide QRS on electrocardiogram monitoring. The physician says "this child 231 needs a direct IV bolus of 1 mmol/kg sodium bicarbonate (of 4.2% = 0.5 mmol/mL concentration)".

232
As soon as this last medication is administered, the physician asks for transport to advanced hospital 233 care and the scenario ends. The GoPro cameras and the watch are turned off 1 min later.

235
During the timed scenario, the resuscitation team will maintain a stressful resuscitation atmosphere by 236 frequently reporting vital signs aloud and asking the participant to promptly provide the drugs, the 237 monitoring alarms will be turned on, and the father will repeatedly verbalize his dismay. The 238 measured deviation between the amount of drug delivered and the actual prescribed dose will be 239 measured by the amount of drug in the syringe and video recorded. All usual EMS resuscitation 240 equipment will be at the disposal of the paramedic. In both allocation groups, the decision to use or 241 not use any equipment will remain personal as in real life. Neither pilot testing nor repetitions will be 242 permitted. There will be no interventions or educational adjuncts prior to or after the study period. To 243 ensure that participants hear and understand the prescription orders correctly and to avoid 244 comprehension bias, they will have to confirm the orders verbally and written transcriptions will be 245 checked and video-recorded. Immediately after the scenario, participants will be asked to recall and 246 describe precisely how they had prepared the drugs and to complete a questionnaire about the scenario.

248
The primary outcome will be the proportion of medication dosage containing errors that occur during 249 the sequence from drug preparation to drug injection. We define an emergency medication dose 250 administration error as a deviation from the correct weight dose of more than 10% 7, 34 . These errors 251 will be measured both as the percentage deviation from the amount of delivered drug compared with 252 the correct weight dose as prescribed by the physician and the absolute deviations from that dose.

253
Miscalculation of the final drug amount, the inability to calculate drug dosage without calculation and 254 guidance help from the second paramedic, and deviation of more than 10% of the final administered 255 concentration of sodium bicarbonate from the prescribed 4.2% will also be considered as medication 256 errors.

258
The secondary outcome will be the elapsed time in seconds between the oral prescription by the

280
Acceptability and usability testing of the app will be assessed using a 52-item questionnaire based on 281 the unified theory of acceptance and use of technology (UTAUT) model 43 . The UTAUT is a standardized instrument for measuring the likelihood of success of new technology introductions and 283 helps to understand the drivers of its acceptance.

285
Research using simulation as a valid and reliable investigative methodology to study factors affecting 286 human and systems performance in health care has been reviewed 30 . In this study, all actions (i.e., 287 outcomes) performed by the paramedics during the scenario will be automatically recorded and stored 288 by the responsive simulator detectors (Laerdal SimBaby) and the 3 GoPro video cameras. The set-up 289 of the 3 cameras will be standardized to record at a resolution of 1080p at 90 frames per sec, a wide 290 field of view, and a 16:9 aspect ratio. Similarly, the position of the cameras will be standardized. The 291 first camera will be mounted on a head strap placed on the paramedic's head with a 45° downward 292 inclination to allow to capture footage of the front scene. The second camera will be placed on a tripod 293 in front of the paramedic and the manikin, slightly above head height, with a 90° downward 294 inclination to film the place where the drugs will be prepared. The third camera will be placed on a 295 tripod 1 m away from the paramedic on his/her left (if right-handed) or right (if left-handed) at the 296 navel level to film the scene from the side. The recorded videos will be safely stored in triplicate on 297 secured hard disk drives, kept in a locked room, and centralized at the Children's Hospital in Geneva.

298
As all scenarios will be fully video-recorded, medication errors and any other errors will be recorded 299 and later analyzed.

301
All actions performed with the app will be automatically saved locally in log files for further analysis.

302
The validity and reliability of the app has been assessed in prior studies 23, 32 . STAI and VAS 303 questionnaires will be used to measure perceived stress. HR will be measured as a surrogate of 304 physiologic sympathetic response to stress 44 . A single continuous measurement at 1-second intervals 305 will be recorded during the scenario with optical photoplethysmography using a Polar A360 wrist-306 worn HR monitor (Polar Electro Oy, Kempele, Finland). Data locally stored on the wristwatch itself 307 during the scenarios will be synchronized with the dedicated Polar FlowSync web service for latter 308 offline analyses. Several time-points of cardiovascular reactivity will be measured: 1) the minimal HR 309 measured within the 5 min before the scenario starts (HR baseline ) while participants are not performing 310 mental or physical exercise, 2) peak HR (HR peak ) for each drug, defined as the maximal HR reached 311 during the sequence from drug prescription by the physician to drug delivery, 3) increased percentage 312 of HR change for each drug by substracting HR baseline from corresponding HR peaks [((HR peak -313 HR baseline )/HR baseline )*100]. An additional HR recovery will also be measured as the minimal HR measured 314 during the 5 minutes immediately following scenario completion (i.e., at the stop of the timed period 315 of the scenario represented by patient's arousal). The investigators will double-check on-site that the 316 questionnaires are fully and accurately completed. Data collection will be carried out using the advantage to observe a unique 60-min period per paramedic. Therefore, neither follow-up nor 319 retention plans will be necessary. The intervention protocol is highly standardized and paramedic 320 deviation from the protocol in terms of drug preparation is a parameter that is of interest in our study 321 (i.e., in terms of medication errors or delays in drug preparation).

323
The expected proportion of errors made by EMS without PedAMINES is 60% 6 . The sample size was 324 calculated to provide the trial with 90% power at a two-sided alpha level of 5% in detecting an 325 absolute difference of at least 30% in proportions of medication errors between intervention groups.

326
The required sample size is 56 paramedics per study arm. To prevent a potential loss of power due to 327 misspecification of assumptions, 60 paramedics will be recruited per randomized group (total sample 328 size: 120 paramedics). To achieve adequate participant enrolment to reach the target sample size, 329 shift-working paramedics will be randomly recruited weeks before the start of the study by a blinded 330 non-investigator. They will be informed of the upcoming simulation study but not of its purpose and 331 outcomes. 332 2.9 Group allocation 333 Paramedics will be randomized using a stratified, single, constant 1:1 allocation ratio determined with 334 web-based software 45 . One randomization list per EMS center will be produced (randomization 335 stratified on center) and random block sizes will be used to generate the randomization lists. On the 336 day of participation, each participant and an investigator will sign informed consent, and selection 337 criteria will be checked prior to participation in the study.

339
Blinding to the direct IV drugs and doses intended for use will be maintained during recruitment to 340 minimize preparation bias. Allocation concealment will be ensured with the allocation software and 341 will not be released until the paramedics start the scenario. Study team members will be revealed to 342 the participants just before the scenario starts. Although the intervention could not be masked, all 343 investigators will remain unaware of the outcomes until all data are unlocked for analysis at the end of 344 the trial. All scenarios will be video-recorded for later analysis. Post-scenario video review will be 345 done without blinding by two reviewers, but undertaken independently with each blinded to the 346 other's reviews. In the case of disagreement, a third independent evaluator will help reach a consensus.

348
Information about study subjects will be kept confidential. All data will be entered into the REDCap 349 data management system where all data on study subjects are assigned an individual identifying code 350 that does not contain identifying information.

352
Planned statistical analyses will be published with the protocol. Due to the study population and 353 interventions, we did not anticipate any missing data. The full analysis of the case will be the primary 354 analysis. In the case of missing data, a sensitivity analysis will be performed by replacing the missing 355 data with a multiple imputation procedure.

357
The overall risk of medication errors (primary outcome) related to the four drugs will be compared 358 between study arms by using a logistic regression model with mixed effects. A random intercept will 359 be introduced in the model with a random effect to account for the repetition of measures within 360 paramedics. We plan to adjust for the center (fixed effect) since the randomization will be stratified on 361 centers. Additionally, an adjustment for the type of medication is planned. In case of low number of 362 paramedics or lack of medication errors when using the app in some centers, the center and the type of 363 drug will be introduced as random effects instead of fixed effects. All random effects will be crossed.

364
The intervention (app versus conventional methods) will be set as a fixed effect. With this model, the 365 regression coefficients of fixed effects will model the risk of medication errors for a typical paramedic, 366 a typical center, and a typical drug (that is when all random effects are set to 0). Therefore, the 367 estimates of the odds ratio (OR) for the app's effect can be different from the OR calculated with raw 368 data. As recommended by the CONSORT 2010 guidelines, 46 an additive measure of the effect size 369 will be assessed. For this purpose, a parametric bootstrap approach will be used: 1) the risk of 370 medication in both arms predicted by the model will obtained from the regression coefficients (fixed 371 effects) by using the inverse of the logit transformation and the risk difference will be calculated; 2) 372 100,000 sets of regression coefficients will be randomly generated from a multivariable normal 373 distribution with the estimates of the fixed effects as mean and the covariance matrix of the estimates 374 as variance; 3) the risk difference will be obtained for each generated set by using the inverse of the 375 logit transformation as in step 1; 4) the percentile 95% confidence interval (CI) will be obtained from 376 the 100,000 risk differences calculated in step 3.

378
A similar approach is planned to assess the effect size of the app on the components of the medication 379 error (dose deviation > 10%, help required). In case of an absence of assistance from the paramedic 380 investigator, and therefore an absence of convergence of the model, the risk difference will be 381 assessed using Miettinen-Nurminen's approach 47 to account for the stratified randomization. With this 382 approach, the repetition of measures within paramedics (outcome assessed on four drugs) will not be 383 accounted. The third component (drug concentration deviation > 10%) will also be investigated by 384 using a logistic regression model with mixed effects. Since this component will be assessed only for 385 the fourth drug (sodium bicarbonate), a single random effect (center) will be introduced in the model.

13
drug. Only the P-value for testing the app's effect on the medication errors over the four drugs 388 obtained with the logistic regression model will be reported. The analyses of the components of the 389 primary outcome and the analyses per drugs aim to better characterise the app's effect. The 95% CIs 390 of the effect sizes will not be adjusted for multiple analyses.

392
Secondary outcomes, the time to drug preparation and the time to drug delivery, will be analyzed 393 using linear regression models with mixed effects. We plan to adjust for center and the type of drugs 394 (fixed effects). Pre-specified sub-group analyses will be performed by introducing an interaction term 395 in the regression models with mixed effects.

397
Dose deviations will be investigated: for each drug, the frequency of under-and overdoses will be 398 assessed as well as the median (interquartile interval) relative dose deviations. The cumulative 399 distribution of the absolute value of the relative dose deviation will be graphically represented. The 400 distribution of the relative dose deviation will be compared between study arms by using Van

403
For primary and secondary outcomes, logistic regression analyses will be conducted if applicable to 404 test a difference in error rates between an urban EMS (defined as a primary EMS in an area populated 405 with 50,000 or more people in the immediate proximity of a tertiary care PED) 48 and a rural EMS 406 (EMS agency not included within an urban area) with the app and conventional methods. In a 407 generalized estimating equation (GEE) logistic regression model, an interaction between interventions 408 and urban/rural EMS will be tested to investigate a potential modification of the efficacy of the app in 409 an urban area compared with a rural area. Results will be also correlated to the EMS exposure (i.e., 410 total number of emergency calls per year per EMS divided by the number of paramedics working in 411 that EMS). Analyses of primary and secondary outcomes will be also conducted with both preparation 412 methods according to paramedics' experience, expressed as years since certification.

414
A first reviewer will review all videos. To assess the reproducibility of the video review procedure, a 415 second reviewer will independently duplicate the review in a random sample of 10% of all videos.

416
Interrater reliability scores on video reviewing will be calculated using Cohen's kappa coefficient for 417 the medication errors. As the other outcomes are continuous variables, the Bland-Altman method will 418 be used to plot the difference of values reported by both reviewers against the mean value for each 419 outcome. The limits of agreement will be assessed by the interval of ±1.96 standard deviations (SD) of 420 the measurement difference either side of the mean difference. The null hypothesis that there is no 421 difference on average between both reviewers will be tested using a t-test. The mean difference will be 422 reported with its 95% CI. Additionally, the intraclass correlation coefficients for volumes of drugs 423 drawn, time to drug preparation, and time to drug delivery will be assessed, assuming that raters are a 14 sample from a larger population of possible raters. The agreement will be investigated for the data of 425 each study period.

427
Finally, means and SDs will be determined for perceived stress and satisfaction scores of individuals 428 for each questionnaire item, as well as for the UTAUT questionnaire, and reported with descriptive 429 statistics. Pearson correlations will be computed between the HR measures obtained with the watch 430 and the scenario phases for each of the drugs and preparation methods used. In the case of missing 431 data, a complete case analysis will be conducted. No multiple imputations are planned. All statistical 432 tests will be two-sided with a type one error risk of 5%. All statistical analyses will be performed with  The time needed to complete the full scenario was set to only a 20-minute period for each participant 438 and will be booked several weeks in advance to ensure availability of the participants without 439 repercussions on their normal duty. All participants will receive full permission from their hierarchy to 440 participate to the study. Using a simulation-based CPR scenario makes our study highly feasible 441 without any risks to patients or participants.

444
The study will take place during a one-week period in each EMS and will occupy each paramedic for 445 a single 60-minute period. Due to our experience gathered from previous studies, we anticipate a very 446 low rate of drop-outs or loss of follow-up. To ensure the presence of participants on the day of 447 participation, shift-working paramedics will be randomly recruited 1 month before the start of the 448 study by a blinded non-investigator. They will be informed of the upcoming simulation study, but not 449 of its purpose and outcomes. Adherence of the collaborating EMS to our study will be reinforced by  On-site staff in each collaborating EMS will organize the participation of paramedics to the study. No 454 information about the ongoing study will be given to paramedics at this stage. The best study period 455 will be matched with study coordinators. The paramedics will be spread across one week in order to 456 link up the several scenarios. On scheduled weeks, the study coordinators will visit the collaborating 457 EMS, provide trial information to participants before they begin the scenario, and assessment of 458 eligibility will be verified. Participants will then sign the informed consent form. They will be asked to 459 complete an anonymized demographic questionnaire and confirm their willingness to participate. If 460 this is the case, they will pursue with a 5-minute educational session given by the investigators on the 461 Version 1.000 June 10, 2021 15 use of PedAMINES. Each participant will then be randomly allocated to the study arms. Immediately 462 after randomization, the paramedic will start the scenario. The total study period to enrol all 463 paramedics in the various EMS will take 12 months.

470
We already have the study participation approval of the different EMS. Each collaborating EMS will 471 organize a dedicated time slot for the study to be run and for paramedics to be available at the defined 472 period. In the case of the unavailability of a paramedic at the selected date, the collaborating EMS will 473 handle the procedure to replace the paramedic. The investigators and collaborating EMS will be in 474 close contact by telephone/email to guarantee the availability of the paramedics.