Study participants were cluster randomized by health care facility, including health care workers and women admitted for delivery. SDA indicates safe delivery app.
eFigure. The safe delivery app
Customize your JAMA Network experience by selecting one or more topics from the list below.
Lund S, Boas IM, Bedesa T, Fekede W, Nielsen HS, Sørensen BL. Association Between the Safe Delivery App and Quality of Care and Perinatal Survival in EthiopiaA Randomized Clinical Trial. JAMA Pediatr. 2016;170(8):765–771. doi:10.1001/jamapediatrics.2016.0687
Copyright 2016 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Health apps in low-income countries are emerging tools with the potential to improve quality of health care services, but few apps undergo rigorous scientific evaluation.
To determine the effects of the safe delivery app (SDA) on perinatal survival and on health care workers’ knowledge and skills in neonatal resuscitation.
Design, Setting, and Participants
In a cluster-randomized clinical trial in 5 rural districts of Ethiopia, 73 health care facilities were randomized to the mobile phone intervention or to standard care (control). From September 1, 2013, to February 1, 2015, 3601 women in active labor were included at admission and followed up until 7 days after delivery to record perinatal mortality. Knowledge and skills in neonatal resuscitation were assessed at baseline and at 6 and 12 months after the intervention among 176 health care workers at the included facilities. Analyses were performed based on the intention-to-treat principle.
Health care workers in intervention facilities received a smartphone with the SDA. The SDA is a training tool in emergency obstetric and neonatal care that uses visual guidance in animated videos with clinical instructions for management.
Main Outcomes and Measures
The primary outcome was perinatal death. Secondary outcomes included the knowledge and clinical management of neonatal resuscitation (skills) of health care workers before the intervention and after 6 and 12 months.
The analysis included 3601 women and 176 health care workers. Use of the SDA was associated with a nonsignificant lower perinatal mortality of 14 per 1000 births in intervention clusters compared with 23 per 1000 births in control clusters (odds ratio, 0.76; 95% CI, 0.32-1.81). The skill scores of intervention health care workers increased significantly compared with those of controls at 6 months (mean difference, 6.04; 95% CI, 4.26-7.82) and 12 months (mean difference, 8.79; 95% CI, 7.14-10.45) from baseline, corresponding to 80% and 107%, respectively, above the control level. Knowledge scores also significantly improved in the intervention compared with the control group at 6 months (mean difference, 1.67; 95% CI, 1.02-2.32) and at 12 months (mean difference, 1.54; 95% CI, 0.98-2.09), corresponding to 39% and 38%, respectively, above the control level.
Conclusions and Relevance
The SDA was an effective method to improve and sustain the health care workers’ knowledge and skills in neonatal resuscitation as long as 12 months after introduction. Perinatal mortality was nonsignificantly reduced after the intervention. The results are highly relevant in low-income countries, where quality of care is challenged by a lack of continuing education.
clinicaltrials.gov Identifier: NCT01945931
Twenty-five years after the introduction of the Millennium Development Goals, the world has seen a 53% decline in mortality among children younger than 5 years. However, neonatal mortality as a percentage of mortality among children younger than 5 years has increased from 37% in 1990 to 47% in 2015, and newborn health remains a priority in the post-2015 agenda.1,2 The most critical period for the survival of children is the perinatal period. More than 2.7 million stillbirths and 2.6 million early neonatal deaths occur each year worldwide.3,4 The 3 major causes of perinatal deaths are preterm birth complications, intrapartum asphyxia, and infections, contributing approximately one-third each.5,6 High perinatal mortality rates are associated with the lack of quality antenatal, obstetric, and early neonatal care.5
Health systems widely fail to provide safe delivery care because the quality of care is severely deficient or absent and the available skilled birth attendants often lack the requisite skills.7,8 The United Nations has identified key medical interventions, commonly called signal functions, of basic emergency obstetric neonatal care that must be provided by all skilled birth attendants. Neonatal resuscitation is one of them. Quality of care is challenged by an outreach gap because most deliveries and emergencies in health care facilities take place peripherally, where health care workers often are not properly trained to manage the complexity of emergencies. Training courses have proved effective to improve quality of care and health outcomes, but barriers prevent access by most rural health care workers.9,10 Solutions that use mobile phones to improve health system functions, termed mHealth, potentially overcome some of these challenges.11,12 However, most of the studies in this area are of poor methodologic quality, and few have evaluated the effects on client health outcomes.12,13 This cluster-randomized clinical trial sought to determine the effect of an mHealth training tool for birth attendants in low-income countries, the safe delivery app (SDA), on the knowledge and skills of health care workers in neonatal resuscitation and perinatal mortality.
Question What is the effect of the safe delivery app on perinatal mortality and the neonatal resuscitation skills and knowledge of health care workers in Ethiopia?
Findings In this cluster-randomized clinical trial that included 3601 pregnant women and 176 health care workers in Ethiopia, an insignificant lower perinatal mortality in intervention clusters was found compared with control clusters. Skills in neonatal resuscitation and knowledge scores improved significantly in intervention clusters.
Meaning The safe delivery app improved health care workers’ knowledge and skills in neonatal resuscitation for as long as 12 months after introduction.
The study took place in 5 districts (Nole Kaba, Haru, Homa, Genji, and Gimbie) in the West Wollega Zone, Oromiya Region, of Ethiopia. Seventy-three health care facilities with health care workers and newborns constituted clusters eligible for randomization owing to deliveries in the previous year and a midwife or health extension worker among the staff. A health extension worker is a frontline health care worker with training in managing basic health problems, including childbirth. Two referral hospitals were excluded because of incomparability with other clusters, leaving 70 health care facilities. From September 1, 2013, to February 1, 2015, 3601 women were included at admission and followed up until 7 days after delivery. Of these, 482 women withdrew or were lost to or unavailable for follow-up (Figure). Six women died of direct obstetric complications. The study population for secondary outcomes included the 176 health care workers at the 70 health care facilities, 46 of whom were excluded or lost to or unavailable for follow-up, primarily because of transfer, leaving 130 in the analysis (Figure). The Ethiopian Oromiya Regional Health Bureau provided ethical clearance for the study on May 7, 2013. The trial protocol is provided in Supplement 1. All women and health care workers were informed of the nature and purposes of the study and the possibility of dropping out at any time without change in the quality of care before providing consent to participate by signature or fingerprint.
The SDA study is a cluster-randomized clinical trial with the health care facility as the unit of randomization. We followed the Consolidated Standards of Reporting Trials guidelines for reporting cluster-randomized trials.14 The Ethiopian Ministry of Health agreed to let the facilities be included in the trial (cluster-level consent).
Health care facilities, stratified by district and level of care, were assigned by simple, random allocation using a computer-generated random number table to the SDA intervention or the control group (Figure). The study participants and the clinic staff were not masked because the intervention required overt participation. Facilities were randomized rather than individuals to avoid contamination among health care workers in the same facility, such as health care workers showing the application and animation videos to the control group.
The SDA is designed to provide training for rural health care workers in low-income countries on how to manage obstetric and neonatal emergencies. The SDA conveys knowledge and skills, such as how to ventilate a newborn in need of resuscitation, through visual guidance in animated videos and voiceovers in the local language (eFigure in Supplement 2). The SDA also contains a catalog with essential obstetric drugs and equipment. Smartphones with the SDA were implemented at intervention health care facilities. Intervention health care workers received a 1-day introduction explaining how to use the smartphone and SDA with joint video viewing and interactive exercises in small groups. At the nonintervention health care facilities, the health care workers provided standard care without the assistance of the SDA. To ensure equal possibilities to provide standard care, the availability of a minimum package of drugs and equipment was ensured. The SDA can be downloaded free of charge (https://itunes.apple.com/us/app/safe-delivery/id985603707?mt=8 or https://play.google.com/store/apps/details?id=dk.maternity.safedelivery).
Staff in participating facilities were explained the nature and purpose of the trial. Data collection was conducted in parallel in the intervention and control facilities using the same methods. The enrolled health care facility staff also functioned as research assistants by recording an inclusion questionnaire with demographic and covariate information and answers to a short, structured questionnaire on the background characteristics of women and their families. Each health care facility was allocated an individual study identification number. Pregnancy outcome was recorded at delivery. Supervisors oversaw each research assistant’s data collection to evaluate the accuracy of collected data on site and visited each facility at least once a week. All women included in the study received a follow-up contact (at home or by telephone) with a supervisor at 7 days post partum regarding information about perinatal mortality.
Data for secondary outcomes were collected at baseline and 6 and 12 months after the intervention. The clinical performance of and knowledge of management for neonatal resuscitation were assessed by using a key feature questionnaire and objective, structured assessment tools of technical skills, which consisted of simulated scenarios with scoring on skills performance on mannequins and with relevant equipment and drugs available. The sessions and scoring were performed by the same 2 independent and specially trained midwives (T.B. and W.F.). The key feature questionnaire was administered in a classroom; a scoring sheet was provided by the tool. Data were entered in Epi-Data (http://www.epidata.dk) and subsequently transferred and validated in SPSS (version 23; SPSS Inc).
The primary outcome was perinatal death, which was defined as a composite of a stillbirth or an early neonatal death. We used the World Health Organization definition of stillbirth as any delivery in the third trimester (birth weight ≥1000 g or gestational age ≥28 weeks) with no signs of life (breathing, crying, heartbeat, and movement).15 An early neonatal death was defined as any infant born alive who died on or before postnatal day 7. The perinatal mortality rate was perinatal death per 1000 total births. Secondary outcomes included the knowledge of and clinical performance by the health care workers in the management of neonatal resuscitation before the start of the intervention and after 6 and 12 months.
Power calculations were made on the primary outcome of perinatal death. The null hypothesis was that no difference in the probability of this outcome would be found between the intervention and control groups. Correlation between births within the same health care facility was taken into account. Based on limited available information, the proportion of births with the primary outcome was assumed to be 5%. Assuming a 40% reduction in the primary end point owing to the intervention (from 5% to 3%), an intra–health care facility correlation of 0.003, 30 deliveries per health care worker, and 2 health care workers per health care facility, we estimated that 70 health care facilities were needed to achieve a power of 80% at a significance level of 5%.16
Analyses were performed based on the intention-to-treat principle, and all available data were included in the analysis. We adjusted for the clustering of our data using generalized estimating equations in all logistic regression analyses. We specified an exchangeable working correlation to allow for the within-cluster correlation, and SEs were based on the robust covariance matrix. We used the traditional logit link, which resulted in odds ratios (ORs) as an effect measure with 95% CIs. Statistical significance was defined as P < .05. For our binary outcome measure, the presence or absence of perinatal death, we used logistic multilevel analysis to determine whether a difference in perinatal deaths existed between the intervention and control groups. In this model, we included all socioeconomic and obstetric variables. We found no interaction between the intervention and explanatory variables.
The secondary analysis focused on health care workers’ knowledge and skills. Differences in the objective structured assessment tools of technical skills and key feature questionnaire were determined between the intervention and control clusters. For secondary outcomes of skills and knowledge scores assessed at baseline and 6 and 12 months, we used a linear mixed-effect model to compare the intervention and control groups. The model accounts for the correlation between health care workers from the same facility by including health care facility as a random factor. Intervention status was included as a fixed effect. Where appropriate, organizational and individual factors strongly correlated with the outcome were also included as fixed effects in the model. Baseline data were used as covariates (linear effect), and the scores after 6 and 12 months were used as the dependent variables in 2 separate analyses. The estimated intervention effect was reported separately as the differences between the intervention and control groups for 6 and 12 months. For all models, the criterion for significance was set at P < .05, and all analysis was performed using SPSS (version 23).
The analysis included 3601 women and 176 health care workers. Table 1 and Table 2 show that the socioeconomic characteristics of the study population for primary and secondary outcomes were similar in intervention and control clusters. Most of the women participating were housewives younger than 25 years with no or limited education. Five hundred twenty-three of 1411 intervention women (37.1%) and 622 of 1601 control women (38.9%) were pregnant for the first time, whereas 70 (5.0%) and 105 (6.6%), respectively, were in their fifth pregnancy or more. Twenty-four pregnancies (0.9%) were multiple gestations, of which 5 infants were included among perinatal deaths. Most deliveries took place in health care centers. The participating health care workers included 47 health extension workers (74.6%) in intervention clusters and 41 (69.5%) in control clusters; the remaining workers were clinical nurses and midwives. The intervention clusters had less experience than the control clusters, with 47 (83.9%) conducting 5 or fewer deliveries during the previous month, and only 5 (8.9%) conducting more than 10 deliveries. In comparison, 38 (64.4%) and 13 (22.0%) workers, respectively, had this experience in the control clusters. Fifteen intervention health care workers (23.1%) and 26 control health care workers (40.0%) had tried using a smartphone before the study.
Overall, 3102 children were live born, 41 were stillborn, and 60 died within the first 7 days of life (Table 3). The overall perinatal mortality rate was 19 per 1000 total births. The rate was lower in the intervention clusters, with 14 per 1000 births compared with 23 per 1000 births in the control clusters (Table 3). The intervention was associated with a nonsignificant reduction in the primary outcome of perinatal mortality (OR, 0.76; 95% CI, 0.32-1.81) (Table 4). Secondary outcomes showed that skills scores in neonatal resuscitation at baseline were similar for health workers in the intervention and control clusters at 7.10 (SD, 5.26) and 7.19 (SD, 4.50) of 24 possible points (both 30%). Similarly, knowledge scores were 4.43 (SD, 1.80) in the intervention clusters and 4.34 (SD, 1.79) in the control clusters, of 12 possible points (both 36%). We found a significant effect of the intervention on health care workers’ knowledge and skills after 6 and 12 months from baseline. Skills improved; the mean difference between scores of intervention and control health care workers was 6.04 (95% CI, 4.26-7.82) at 6 months and 8.79 (95% CI, 7.14-10.45) at 12 months, corresponding to improvements of 80% and 107%, respectively, above the control level (Table 4). The increase from 6 to 12 months was also significant. Knowledge also improved; the mean difference between scores of intervention and control health care workers was 1.67 (95% CI, 1.02-2.32) at 6 months and 1.54 (95% CI, 0.98-2.09) at 12 months, corresponding to improvements of 39% and 38%, respectively, above the control level (Table 4). The knowledge score decreased from 6 to 12 months.
The SDA mHealth training tool led to a more than 2-fold significant increase in health care workers’ skills and knowledge scores for neonatal resuscitation and a nonsignificant 24% reduction in perinatal mortality. The increase in skills of neonatal resuscitation was sustained and even further increased 12 months after the introduction of the SDA. This study is important because it (1) addresses the need found in systematic reviews for trials of mobile phone interventions in low- and middle-income countries that have a health outcome as a primary outcome17-19; (2) addresses one of the largest challenges faced by health care workers in the periphery of the health care system, namely, inadequate access to training and reference materials to handle situations that are beyond their skills11,12; (3) suggests the efficacy of using the SDA to improve the quality of basic emergency obstetric neonatal care; and (4) shows a sustained and increased effect 12 months after introduction.
Frontline health care workers in low-income countries increasingly consider mHealth tools as a useful means to improve their services and felt empowered by the SDA.12 Several studies20,21 have shown that text message interventions can improve adherence to essential health care services. Zurovac et al22 reported that health care workers who received motivational messages about the management of malaria in children demonstrated improvement in correct management by 24% compared with workers who did not receive the messages. This effect was maintained even 6 months after the start of the study. Another study23 in Tanzania provided health care workers with electronic decision-support tools for integrated management of childhood illnesses and reported a significant improvement in the health care professional’s ability to adhere to these treatment regimens. Our findings support those of other studies24,25 that the use of mobile tools is perceived as an opportunity for self-improvement with effects on health care workers’ motivation, self-efficacy, and enthusiasm. In line with our experience, evidence also suggests that the use of mobile phones for delivery of health care services is feasible for health care workers, irrespective of their educational level or prior training.26-28
The baseline assessments in our study showed that management of neonatal resuscitation was very poor among intervention and control health care workers; on average, only 30% of important steps were performed. This finding underlines the urgent need to develop and assess innovative solutions to improve health care workers’ knowledge and skills in neonatal resuscitation. Conventional training of skilled birth attendants in basic obstetric and neonatal emergency care has proved effective to improve health care outcomes.9,10 For instance, an intervention study at several hospitals in Tanzania10 assessed basic neonatal resuscitation training and found a remarkable sustained 47% reduction in early neonatal mortality and a 24% reduction in stillbirths. From a health system perspective, the effectiveness of mHealth tools, such as the SDA, needs to be established as well as which training model is most effective in terms of cost, feasibility, scale, and receptiveness. The advantage of the SDA is that it is self-explanatory, is free of charge for download, and, once installed on the mobile device, does not need network coverage. The SDA also has the potential to easily adapt to local contexts and be developed into local language versions. In this light, we found a significant 2-fold increase in skills scores at 6 months that further increased at 12 months and a significant increase in knowledge at 6 months that was sustained but did not further increase at 12 months.
We found a statistically nonsignificant 24% reduction in perinatal mortality among children born to women in the intervention group compared with children born to women in the control group. Another trial from Zanzibar29 has assessed the association between a mobile phone intervention and perinatal mortality and similarly found a significant 50% reduction in perinatal mortality. Our results are in line with those of other studies of perinatal mortality in sub-Saharan Africa, although the perinatal mortality rate for our study (19 per 1000 births) was below the estimates for Ethiopia. Our study confirms results from others30 indicating that stillbirths may constitute as many as 70% of perinatal deaths and underlines that, with access to adequate quality care in the antenatal, intrapartum, and early neonatal periods, many perinatal deaths might be prevented.
Our findings are limited by the study location and population. Facilities, rather than individuals, were randomized to avoid a spillover effect from the intervention group to the control group (ie, where health care workers from the control group obtain the SDA or hear about its contents from others in the intervention group). To reach similar socioeconomic and health service compositions in the intervention and control groups, we performed the randomization of facilities based on districts. We subsequently performed statistical control of within-cluster correlation and confounders. Blinding of intervention and control clusters was impossible owing to the nature of the intervention, which increased the risk for a selection or information bias. The implications of this study are that mobile phone interventions, such as the SDA, should be considered to improve the ability of health care workers to provide quality of care during emergencies and to reduce perinatal mortality. An increasing number of low-income countries have a framework in place with national mHealth policies, but few mHealth tools are being scaled up and integrated into existing systems.19 The causalities of maternal deaths, stillbirths, and early neonatal deaths are interlinked. Therefore, monitoring, policies, and health system interventions should be integrated. The SDA was developed to contribute to the larger goal of every woman and newborn being attended by health care professionals capable of handling emergencies in a timely fashion. Evidence is growing to include reduction of perinatal mortality and appropriate use of technology in the post-2015 agenda to further reduce child mortality in low-income countries.
The SDA was an effective method to improve and sustain health care workers’ knowledge and skills in neonatal resuscitation as long as 12 months after introduction. Perinatal mortality was nonsignificantly reduced. The results are highly relevant, particularly in low-income countries where quality of care is challenged by lack of continuing education programs. More research is needed in effects on clinical outcome and large-scale implementation in resource-limited settings.
Corresponding Author: Stine Lund, MD, PhD, Department of Public Health, University of Copenhagen. Blegdamsvej 3, 2200 Copenhagen N, Denmark (firstname.lastname@example.org).
Accepted for Publication: March 14, 2016.
Published Online: June 20, 2016. doi:10.1001/jamapediatrics.2016.0687.
Author Contributions: Drs Lund and Sørensen had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Lund, Boas, Nielsen, Sørensen.
Acquisition, analysis, or interpretation of data: Lund, Boas, Bedesa, Fekede, Sørensen.
Drafting of the manuscript: Lund.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Lund.
Obtained funding: Lund.
Administrative, technical, or material support: Lund, Boas, Bedesa, Fekede, Sørensen.
Study supervision: All authors.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by funding from Merck for Mothers.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: