Introduction
Methods
Results
Discussion
Limitations
Future directions
Supplementary information

INTRODUCTION

The lack of emergency care systems has been associated with lower survival rates in adults and children in low- and middle-income countries (LMICs).^1,2 At the 72^nd World Health Assembly, delegates recognized the value of emergency and trauma care in reducing morbidity and mortality, and adopted a resolution that would support countries in the development of systems to deliver timely care to critically ill and injured people.³ In addition to needs assessments and standards for equipment and processes to support the development of emergency care systems, training for all cadres of health workers was one of the identified interventions in the resolution.

Numerous studies surrounding emergency medicine (EM) curriculum implementation have demonstrated decreased mortality in adult populations^4-6 without a significant increase in the use of resources or economic burden.^5-7 While nearly 95% of the one million traumatic injuries occurring in children worldwide occur in LMICs,⁸ to date, there remains a paucity of open access and comprehensive, pediatric-focused curricula for emergency and trauma care. Furthermore, of the pediatric curricula that have been implemented, most have only evaluated providers’ self-efficacy and knowledge acquisition.^9-12 A rare few have attempted to show changes in provider behavior or patient outcomes.^13,14

In response, the African Federation for Emergency Medicine (AFEM) assembled a working group with expertise in pediatric EM (PEM) from seven American and African academic institutions to develop a comprehensive PEM curriculum for three different tiers of healthcare professionals that would be made freely available. The curriculum development process was based upon a widely accepted model for medical education.¹⁵ Curriculum topics were based upon a needs assessment conducted in Tanzania two years prior, and learning objectives were determined by expert consensus review using a modified Delphi process.^16,17 As part of the development process, the implementation and evaluation of the curriculum for the first tier of providers, nurses as described here, was conducted as a pilot study at Muhimbili National Hospital (MHN), the national referral hospital for Tanzania (see Figure 1). Specific efforts were made to broaden the curriculum’s evaluation beyond self-efficacy and knowledge acquisition, to include changes in practice behavior.

METHODS

This study was a non-randomized, controlled pilot study to evaluate the effectiveness of a novel PEM curriculum in nurses (Tier 1 providers) by examining the association between participation in this curriculum and nurses’ self-efficacy, knowledge, and changes in behavior.

Setting and Study Population

MHN is the national referral hospital for Tanzania and is located in the capital city, Dar es Salaam. It houses an emergency department (ED) that treats approximately 45–50 pediatric patients (under 18 years old) per day.

Population Health Research Capsule

What do we already know about this issue?

Emergency care reduces morbidity and mortality, but countries in low-resource settings may lack dedicated training, especially in pediatric emergency medicine (PEM).

What was the research question?

Would Tanzanian nurses who participate in a novel PEM curriculum demonstrate improved self-efficacy, knowledge, and practice during pediatric emergencies?

What was the major finding of the study?

Trained nurses showed improved self-efficacy and knowledge, but failed to show increased performance of critical actions during live pediatric resuscitations.

How does this improve population health?

This PEM pilot has strengthened the curriculum and will be available to train nurses across Africa, to improve the care of critically ill and injured children.

Nurses were recruited by ED staff at MHN to participate in the training and were matched to control nurses who worked in the same setting based upon their level of experience. All ED Tier 1 nurses or prehospital providers for whom caring for pediatric patients on a daily basis was within their scope of practice were eligible to be enrolled. Any provider not proficient in the English language was excluded from the study. As a retention strategy, a certificate of completion was provided to all participants who attended at least 80% of the training sessions and completed all measurement tools.

The sample-size calculation was based off change in knowledge scores in previously published literature.^7,18 A minimum of 24 nurses (12 intervention and 12 control) was required to detect a 15% change in test scores.

Control Group

As mentioned, the control group of nurses was recruited from the same group of ED nurses as those in the intervention group, and were matched to participants in the intervention group based upon level of experience. The majority of the nurses in both groups (> 60%) hold a diploma in nursing (three-year program following secondary school or high school), while the remainder possess either a bachelor’s degree or a certificate in nursing. At baseline, all nurses in the ED receive annual training in pediatric emergencies via the American Heart Association’s Pediatric Advanced Life Support course, and a five-day, multidisciplinary course focusing on pediatric resuscitation and trauma management that was developed by local physicians. In addition, they participate in a monthly course entitled “Basic Emergency Nursing Training,” which includes some basic pediatric resuscitation training. Further exposure to pediatric-specific training for the nurses is more sporadic, with up to 25% of weekly continuing nursing education sessions being relevant to pediatrics.

Intervention

Training

Training sessions were conducted over two and a half days and included 30-minute lectures on nine topics, small-group skills stations and simulated scenarios, as well as scheduled intervals for pre-and post-training measurements and frequent breaks (see Appendix 1). The lectures were delivered by two instructors, one international instructor, and one local instructor, and the small-group sessions were facilitated by an additional three international instructors and two local instructors.

Outcomes/Measurements

The non-randomized, controlled study design was approved by institutional review boards at both the University of California San Francisco (UCSF) and MHN. Outcome measurements focused on three well-described dimensions to evaluate training programs.¹⁹ Primary outcomes were 1) participant self-efficacy (as measured by a survey that used five-point Likert scales to rate participants’ confidence with PEM concepts and skills); and 2) participant knowledge acquisition (as measured by written test scores on a 20-question multiple-choice test). Since no surveys for this target audience existed, items were adapted from previously published and validated surveys, with new content development informed by interviews conducted during a needs assessment and review by local experts (EM specialists in low-resource settings).^16,20-22 A similar process for developing questionnaires for educational research has been described.²³

A secondary outcome was change in participant behavior (as measured with a binary checklist of critical actions for pediatric sepsis, respiratory distress, and trauma). Content validity for all tools was obtained through expert review; and a duplicate survey and written test were used for pre-training and post-training evaluations, demonstrating reliability (see Appendices 3 and 4).²⁴ Inter-observer reliability for the checklist was not assessed. Each group completed all measurements both pre- and post-training, except for the changes in participant behavior, which was only able to be measured post-training (see “Limitations” section). These data were collected for seven weeks post-training.

Statistical Analyses

We used paired t tests to compare means between normally distributed groups and Mann-Whitney U test to compare data that were not normally distributed. Performance on the checklist of critical actions was compared using chi-squared or Fisher’s exact tests as appropriate. For all comparisons, a two-tailed p-value less than 0.05 was considered statistically significant.

RESULTS

A total of 15 nurses participated in the training; however, only 11/15 completed all of the pre- and post-training measurements. Fourteen nurses were recruited as control participants; 11/14 completed all measurements. For the survey measurement of self-efficacy, median ratings were similar between the intervention and control groups; however, overall participant post-training ratings were significantly greater than pre-training ratings in the intervention group. Control participants showed no significant difference between their pre-training and post-training ratings (see Tables 1 and 3).

Similarly, for the test of knowledge acquisition, no pre-training difference existed between the intervention and control groups; however, a significant increase in median scores was seen within the intervention group across the two time points, as well as when comparing the intervention group to the control group post-training (see Tables 2 and 3).

A total of 402 live cases of pediatric respiratory distress (121 intervention; 281 control) were observed and measured using the critical actions checklist post-training (Table 4). Only one critical action was observed at a higher proportion in the intervention group (+8.6% (confidence interval [CI], −0.8 – 18.1%): “States that the child is in respiratory distress” (which was intended to serve as a proxy for recognition of an emergency condition – see Appendix 2 for the complete tool). For pediatric trauma, 394 live cases were observed (115 intervention; 279 control) with no statistically significant differences in performance of critical actions. For pediatric sepsis, 396 live cases were observed (117 intervention; 279 control). In two related critical behaviors – “States whether the child is or is not anemic” (eg indicating that the nurse checked for anemia) and “Attempts to place IV or IO (if available)” – the intervention group performed these actions at higher rates with estimated differences of + 6.3% (CI, −0.9 – 13.7%) and 12.6% (CI, 2.1 – 23.0%), respectively.

DISCUSSION

Emergency care has been proven to save lives, and educational curricula have been shown to be one way to effectively and feasibly support the development and expansion of emergency care services in LMICs.^4-7 Few comprehensive, open-access, pediatric-specific emergency curricula exist despite the high burden of critically ill and injured children in these settings. This study describes the pilot implementation of such a curriculum developed by AFEM to fill this gap, and demonstrates its effectiveness in improving both PEM self-efficacy and knowledge. Once finalized, this curriculum will be made freely available via the Internet to be modified and used to train nurses and prehospital providers across the African continent.

Combined pediatric emergency and critical care fellowships are starting to be implemented on the continent.²⁵ At the same time that sub-specialty training for physicians in LMICs is being conducted at referral centers, it is important to recognize that many emergencies take place far from these large centers. To ensure that children receive the care they need, nurses and physicians at sites that are further afield also need training in the recognition and initial management of pediatric emergency conditions, but may not have the financial and logistical ability to commit to two or more years of full-time training.

Our curriculum addresses several of these issues. First, while this pilot training included all components and was held over two and a half days, it is designed in a modular fashion so that participants can view the lectures (available both in PowerPoint and PDF formats to accommodate local bandwidth restrictions) at their convenience. The hands-on, small-group skills sessions and simulated cases can be offered in a brief, one-day training with experienced facilitators. This multimodal format has been shown to be preferred by working emergency care nurses, interns, residents, and physicians in a similar setting.¹⁶ Secondly, the final AFEM curriculum is designed to target multiple different cadres of healthcare workers through its tiered development. The Tier 1 curriculum piloted in this study is directed toward nurses and pre-hospital providers, recognizing that in most LMICs, the majority of the healthcare workforce is made up of professionals who are not physicians.

This study demonstrated a significant improvement in self-efficacy and knowledge scores of participating nurses. As mentioned, multiple studies of educational curricula have shown similar benefits; however, fewer have demonstrated actual changes in behavior. We attempted to show a change in nursing behaviors with this curriculum. However, an improvement in critical actions during specific pediatric emergencies (respiratory distress, trauma, sepsis), which was expected, was not seen for most actions (Table 4).

There are several potential reasons for this. First, critical actions included stating the existence of certain conditions, such as respiratory distress, as a proxy for recognition of the emergency condition. However, observations were being conducted during actual resuscitations and not in a traditional testing environment, so the fact that a nurse did not verbalize critical action statements may not be a true reflection of his or her recognition of these conditions, but rather a reflection of the lack of utility of such statements during live resuscitations. This hypothesis is supported by the observation that rates of performance of critical actions that required statements were relatively low in both groups, ranging from 6–36% (except for “States that the patient is in septic shock”). However, critical actions that followed recognition of these conditions were performed at relatively higher rates, suggesting that nurses may have been acting upon these emergent conditions, even if they were not stating their recognition of them.

Additionally, this pilot study was conducted at only one site (due to a limitation in funding), so there may have been information transfer among the ED nursing staff, which could have led to an improvement in performance of members of the control group, which makes the lack of significance in the rest of the critical actions difficult to interpret. Since pre-training data was not available to help triangulate the results, it is unclear whether participants were unable to transfer the knowledge acquired into action, or knowledge gained spilled over into the control group. The lack of pre-training data also prohibits us from assessing whether the intervention and control groups performed comparably at baseline. However, a significant difference in baseline is unlikely as there was no difference in their pre-training confidence or knowledge scores. The disproportionately higher number of post-training observations in the control group could be due to an unintentional counting of all nurses who did not undergo the training as controls; however, this could not be confirmed due to lack of identifying information.

Other groups have shown that practice change following educational interventions is often difficult to achieve, and our study supports this notion.⁵ However, given the knowledge acquisition and improved confidence after the course, we believe that this does not suggest that such curricula are not effective, but, if the data are accurate, might not be sufficient, and that another component of training such as direct oversight or on-site mentorship is needed. The value of such presence has been stressed by other researchers.⁵ Future studies are needed to confirm or refute the lack of translation of knowledge into practice, and if confirmed to examine the effect of direct oversight on practice change.

LIMITATIONS

Our study has several important limitations, including small sample size, possible sample contamination, limited value of specific critical actions as described above, and a failure to confirm inter-observer reliability. In addition, the study design was not randomized due to resource limitations and departmental staffing needs; we were provided with a convenience sample of participant nurses that was matched to a group of control nurses. Our pre-training data collection period for observation of critical actions was eliminated due to delays in obtaining IRB approval, making a comparison across the two time periods impossible. In addition, there was selection bias in recruitment, as many of the nurses in the intervention arm had expressed a particular interest in pediatrics and therefore likely had more experience and motivation. The same questions were used pre- and post-training to assess for knowledge acquisition, which could suggest a positive effect of exposure, however, this is likely limited as the control group did not show a significant increase in scores. Due to funding constraints, this pilot was conducted at only one site, which limits the overall generalizability of the findings.

FUTURE DIRECTIONS

The results of this pilot study have revealed several important areas needing further investigation. As mentioned above, future studies will focus on determining whether true practice change is occurring from this curriculum, as well as the most important factors contributing to behavior change in nurses and physicians in such settings. Interest in re-examining how this curriculum could alter provider behavior and pediatric mortality, especially with larger sample sizes, has already been expressed by institutions in other countries in Africa.

While this study demonstrated an improvement in confidence and knowledge from this curriculum, the ultimate goal is to develop a course that is effective in creating behavior change that leads to a reduction in pediatric mortality. Once we better understand the findings of this research and can make appropriate adjustments to the course, the goal is to make this course freely available to nurses and prehospital providers to download all of the material and adapt it to their needs and their setting. In addition, three local trainers were trained through this pilot study and the hope is that they will later be able to train future trainers to ensure the local sustainability of the course.

The curriculum for Tier 2 providers (eg, clinical officers, intern physicians) is currently being created and will be piloted at a large medical center in West Africa. The results of this study and subsequent planned studies will be used to modify this curriculum as well. Once these two tiers have been modified and piloted, the curriculum for Tier 3 providers (eg, specialist physicians) will be created and similarly evaluated.

Supplementary Information

Footnotes

Section Editor: Andrew W. Phillips, MD, MEd

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Carol Chen, MD, Pediatric Emergency Medicine, UCSF PEM, Box 0632, 550 16th Street, Floor 5, San Francisco, CA 94143. Email: carol.chen@ucsf.edu. 1 / 2020; 21:134 – 140

Submission history: Revision received July 15, 2019; Submitted October 18, 2019; Accepted October 28, 2019

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Hsiao M, Morris SK, |Malhotra A, et al. Time-critical mortality conditions in low-income and middle-income countries. Lancet. 2013;381(9871):993-4.

2. Reynolds TA, Sawe H, Rubiano AM, Shin SD, Wallis L, Mock CN. Chapter 13: Strengthening health systems to provide emergency care. Disease Control Priorities: Improving Health and Reducing Poverty. 2017:247-266.

3. Emergency care systems for universal health coverage: ensuring timely care for the acutely ill and injured. 2019. Available at: https://www.physiospot.com/2019/05/30/new-who-emergency-and-trauma-care-resolution/. Accessed October 31, 2019.

4. Wang P, Li NP, Gu YF, et al. Comparison of severe trauma care effect before and after advanced trauma life support training. Chin J Traumatol. 2010;13(6):341-4.

5. Petroze RT, Byiringiro JC, Ntakiyiruta G, et al. Can focused trauma education initiatives reduce mortality or improve resource utilization in a low-resource setting?. World J Surg. 2015;39(4):926-33.

6. Arreola-Risa C, Mock C, Herrera-Escamilla AJ, et al. Cost-effectiveness and benefit of alternatives to improve training for prehospital trauma care in Mexico. Prehosp Disaster Med. 2004;19(4):318-25.

7. Crouse HL, Vaides H, Torres F, et al. Quality and effectiveness of a pediatric triage training program in a Guatemalan public hospital. Pediatr Emerg Care. 2016;32(8):525-8.

8. Mirkuzie AH, Sisay MM, Bedane MM. Standard basic emergency obstetric and neonatal care training in Addis Ababa; trainees reaction and knowledge acquisition. BMC Med Educ. 2014;14:201.

9. Kiragu AW, Dunlop SJ, Wachira BW, et al. Pediatric trauma care in low- and middle-income countries: a brief review of the current state and recommendations for management and a way forward. J Pediatr Intensive Care. 2017;6(1):52-9.

10. Chang MP, Walters CB, Tsai C, Sampson J, et al. Evaluation of a neonatal resuscitation curriculum in Liberia. Children (Basel). 2019;6(4):E56.

11. Fant CD, Schwartz KR, Patel H, et al. Developing and implementing a pediatric emergency care curriculum for providers at district level hospitals in Sub-Saharan Africa: a case study in Kenya. Front Public Health. 2017;5:322.

12. Wesson HK, Plant V, Helou M, et al. Piloting a pediatric trauma course in western Jamaica: lessons learned and future directions. J Pediatr Surg. 2017;52(7):1173-6.

13. Chomba E, Carlo WA, Goudar SS, et al. Effects of essential newborn care training on fresh stillbirths and early neonatal deaths by maternal education. Neonatology. 2017;111(1):61-7.

14. Carlo WA, Goudar SS, Jehan I, et al. Newborn-care training and perinatal mortality in developing countries. N Engl J Med. 2010;362(7):614-23.

15. Thomas PA, Kern DE, Hughes MT. Curriculum Development for Medical Education. 2016.

16. Vo H, George U, Straube S, Tenner A, Sawe H, Chen C. A needs assessment of pediatric emergency and critical care in Tanzanian providers: a model for curriculum development. Pediatric Academic Society Global Pediatric Research. 2018.

17. Carman A, Daftary R, George U, Sawe H, Chen C. Development of the objectives for a pediatric emergency medicine curriculum for the African Federation of Emergency Medicine. Western Regional Society for Academic Emergency Medicine. 2019.

18. Tchorz KM, Thomas N, Jesudassan S, et al. Teaching trauma care in India: an educational pilot study from Bangalore. J Surg Res. 2007;142(2):373-7.

19. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 1998.

20. Spaite DW, Karriker KJ, Seng M, et al. Increasing paramedics’ comfort and knowledge about children with special health care needs. Am J Emerg Med. 2000;18(7):747-52.

21. Dobson JV, Brancati DS, Nagel R. Pediatric resuscitation: evaluation of a clinical curriculum. Med Educ Online. 2003;8(1):4337.

22. Shilkofski N, Crichlow A, Rice J, et al. A standardized needs assessment tool to inform the curriculum development process for pediatric resuscitation simulation-based education in resource-limited settings. Front Pediatr. 2018;6:37.

23. Artino J, Anthony R, La Rochelle JS, et al. Developing questionnaires for educational research: AMEE guide no. 87. Med Teach. 2014;36(6):463-74.

24. Sullivan GM. A primer on the validity of assessment instruments. J Grad Med Educ. 2011;3(2):119-20.

25. Fellowship training program. PECC-Kenya pediatric emergency and critical care web site. Available at: http://www.pecc-kenya.org/. Accessed October 31, 2019.