Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Acts of terrorism, mass casualty incidents (MCI), and natural disasters continue to occur and overwhelm medical systems nationally and globally.^1-3 An MCI is any incident where emergency medical services (EMS) resources are overwhelmed by the number and severity of casualties. Terrorism, both domestic and global, has recently been increasing. Specifically, the number of active shooter events and bombings has risen rapidly in the United States, with more than a third of events since 2000 occurring in the last three years.^4-5 This increasing frequency has required healthcare providers to become more familiar with how to respond to these public health threats and emergencies.

Historically, the educational methods for these topics have included standard didactics and tabletop exercises (a low-fidelity type of simulation). However, passive educational delivery methods such as didactic lecture have been associated with the lowest average knowledge-retention rates compared to more active methods of learning.^6-7 Simulation has been effectively used to train providers in areas such as EMS and critical care and provides a way to improve the quality of education and training to respond to these public health emergencies.

Simulation encompasses any process or technology that recreates a contextual background allowing the learner to make decisions, experience success, make mistakes, receive feedback, and gain confidence in an environment that is void of patient risk.⁸ Research has suggested that simulation is superior to conventional educational delivery methods in emergency and critical care medicine.^9-12 However, in many areas around the world where healthcare providers must respond to acts of terror, there is a paucity or absence of simulation resources. Telesimulation is an innovative educational delivery method that can address this need.

Telesimulation is a process by which telecommunication and simulation resources are used to provide education, training, and/or assessment to learners at an off-site location.¹³ Because telesimulation is a new niche within simulation, the evidence demonstrating its effectiveness is scant. To our knowledge, no studies exist that evaluate the feasibility of telesimulation to deliver an EMS-based course to healthcare providers across continents. The objective of this study was to evaluate the feasibility and effectiveness of using telesimulation to deliver an EMS-based course on MCI training to healthcare providers on a different continent.

METHODS

Study Design and Setting

We performed a feasibility study to establish the process for successful delivery of educational content through a telesimulation course, that would yield data amenable for qualitative and quantitative analysis. The study was conducted over a five-month period with content creation, delivery, and broadcast from the University of California, Irvine Medical Simulation Center, a 65,000 square-foot state-of-the-art medical education center that provides telemedicine and simulation-based educational programs and continuing medical education courses for thousands of healthcare providers each year.¹⁴ Resources for education and training include a full-scale operating room, an inpatient ward room, emergency department resuscitation bay, obstetrical suite, and a critical care unit. The simulation center has a complement of full-time staff, including full-time simulation specialists.

Population Health Research Capsule

What do we already know about this issue?

The increasing frequency of global terrorism has required healthcare providers to become familiar with how to respond to these public health threats and emergencies.

What was the research question?

Can telesimulation be used to deliver a mass casualty incident training course to healthcare providers overseas?

What was the major finding of the study?

We demonstrated the successful implementation of an intercontinental mass casualty incident triage course using telesimulation and wearable/mobile technology.

How does this improve population health?

Leveraging technology to improve knowledge and skills at the provider level allows for the optimization of healthcare delivery at the population level.

Selection of Participants

Participation in the study was obtained from registrants in an EMS-based course in MCI triage that was designed for this study and offered in collaboration with King Abdullah Bin Abdulaziz University Hospital and Princess Nourah bint Abdulrahman University Simulation and Skills Development Center in Riyadh, Saudi Arabia.

The study was open to all healthcare providers with a focus on emergency medical technicians, paramedics, nurses and physicians. The sole exclusion criterion was the inability to understand and speak the English language as the course and content materials were in English. The study was approved by the university’s institutional review board, and subjects provided informed consent.

Interventions

The educational intervention in this study included an EMS-based course on MCI training on Simple Triage and Rapid Treatment (START) in the prehospital care setting. Mass casualty triage occurs when there is more than one casualty and the available resources require a provider to initiate care for one patient over another.¹⁵ The START system, developed by Hoag Hospital and the Newport Beach Fire Department (Newport Beach, California), helps prepare emergency personnel to quickly organize their resources to handle multi-casualty emergencies.¹⁶ It is designed to allow the provider to triage each patient in less than 60 seconds. This knowledge base and skill set is particularly critical for healthcare professionals responding to MCIs including active shooters and explosive devices. Currently, START remains the most commonly used mass casualty triage algorithm in the U.S.¹⁷

Methods and Measurements

The MCI course content was delivered using various telecommunication software resources. The course introduction and orientation was performed using join.me (https://www.join.me). Join.me is a web-based collaboration software application for screen sharing and online meetings. The course content was delivered using this software application as it allows real-time teleconferencing with simultaneous educational content broadcast. Educational content materials were delivered via PowerPoint (Version 12.0, Microsoft Corporation, Redmond, Washington). The course content was delivered by physicians who were board certified in both emergency medicine and EMS, and experienced in simulation course design, creation, and implementation. The course was delivered over 2.5 hours (which included a half-hour online check in and pre-course software testing period) (Figure 1).

After core content delivery of MCI education pertaining to START triage, the students underwent a live interactive training session to apply the knowledge gained. The course core content, live interactive training session, virtual simulation component, and question/answer sessions occurred over a two-hour time period (Figure 1). A MCI scenario was created and staged at the broadcasting institution simulation center using a combination of live standardized patients and high-fidelity simulation mannequins. We used the software platform EyeSight (Pristine Eyesight, Austin, Texas) for the live interactive MCI practical application training session. EyeSight allows real-time audio and video collaboration via smart glasses (Google Glass, Mountain View, California) and mobile devices (Figure 2A).

During this live interactive scenario, the course instructor played the role of a paramedic walking through a MCI scenario evaluating each patient and verbalizing information needed for participants to assign each patient the appropriate triage category. Course instructors at the receiving institution were available to answer any questions during the session (Figure 2B). After the live scenario was complete, a debriefing walkthrough of each patient ensued with the course instructors reviewing the appropriate assessment and triage categorization of each patient. The START triage method results in the assignment of patients into one of four categories: black (expectant), red (immediate), yellow (delayed), and green (minor).

The virtual simulation component of the course followed the live interactive MCI practical application training session. The virtual simulation consisted of a MCI of an active shooter in an office building. The virtual simulation was created, staged and recorded in an actual high-rise office building using standardized patients and moulage to create realistic-looking wounds. The simulation was recorded in the first-person perspective of an individual performing a continuous walkthrough of the MCI scene using Google Glass (Figure 2C). Each standardized patient underwent a pre-course training session regarding their roles and setting and also followed a pre-written script for his or her specific role. This pre-recorded MCI scenario served as the standardized resource that was used for individual participant assessment. Each student viewed the Google Glass recording of the high-rise active shooter incident and was responsible for categorizing each victim using START triage.

Outcomes

The primary outcome of this feasibility study was to demonstrate the successful implementation of an EMS-based educational course on MCI management to healthcare providers in a different country via telesimulation. Designing, implementing, and reporting upon the process to deliver educational content to healthcare providers overseas was the major goal of this project. We also sought to obtain data amenable for both qualitative and quantitative analysis. The qualitative component of this study consisted of a survey to evaluate the learners’ thoughts, feelings, and attitudes about taking an EMS-based course in MCI triage via telesimulation. We used the quantitative data obtained to demonstrate the process and feasibility of collecting data to evaluate the diagnostic accuracy of triage performed by the learners in a course delivered via telesimulation.

Analysis

We used descriptive statistics to express the percentage of participants with favorable responses to survey questions. Survey response categories consisted of ordinal data using a five-point Likert scale ranging from “strongly disagree” to “strongly agree.” Triage accuracy was summarized as percent correct.

RESULTS

A total of 32 participants enrolled in the course: 12 physicians (37%); four nurses (13%); and five EMT/paramedics (16%). As the course offerings were open to all participants who could be responsible for providing service in a MCI, participants with varying backgrounds also enrolled, including two pharmacists (6%), and eight in “other” category (25%), which included educators, administrators and technicians. One participant (other*) did not fill out his or her profession on the survey.

We evaluated feasibility by the successful utilization of resources for specific goals and in delivering the final product to the learners – an educational course on the EMS-based topic of MCI triage. Course participants’ thoughts, feelings, and attitudes toward learning EMS-based content on MCI triage were obtained by an immediate post-course survey that maintained subject anonymity. Learners reported that this experience added educational value beyond learning from standard lectures and that this method of virtual simulation is more effective than standard tabletop exercises to learn the MCI triage method (Figure 3). The participants also supported the notion that wearable technology can be an effective tool to transmit critical patient information in the prehospital care setting between providers. Furthermore, they reported that the telesimulation course enhanced their ability to provide care for patients involved in a MCI (Figure 3). Participants provided a favorable response to all questions on the survey regarding their thoughts, feelings, and attitudes toward learning EMS-based content via telesimulation.

For triage accuracy, we were able to collect data in real time of participants’ assessment of patients during the virtual simulation component of the course. These data allow the calculation of triage accuracy according to provider category and experience. As this feasibility study was not intended or designed as an observational-analytical study to evaluate the triage accuracy of participants, data are presented as simple descriptive statistics for illustrative purposes, without any inferences being stated or implied about the larger population from which the sample was drawn (Table 1).

DISCUSSION

In this feasibility study we demonstrated the successful implementation of an EMS-based educational course on MCI triage training to healthcare providers overseas via telesimulation. This process allowed the collection of data amenable for both qualitative and quantitative analysis. As telesimulation is a relatively new niche, there is a paucity of literature to support the evidence base behind its use.

Our survey results revealed an overall positive view toward learning EMS content of MCI triage with telesimulation. Participants reported that this teaching experience added educational value above and beyond their learning from standard lectures and that this method of virtual simulation was more effective than standard tabletop exercises to learn the MCI triage method. The participants also supported the notion that wearable technology can be used as an effective tool to transmit critical patient information in the prehospital care setting between providers and reported that the telesimulation course enhanced their ability to provide care for patients involved in a MCI. Research has provided evidence to suggest that simulation is superior to conventional educational delivery methods in emergency and critical care medicine education.^9-12 Our learner-favorability responses are consistent with those found in the telesimulation and simulation literature and support the research suggesting that simulation is superior to conventional educational-delivery methods.

The positive thoughts, feelings, and attitudes expressed by our learners toward telesimulation supports the findings in previous telesimulation studies. In a study using telesimulation to provide distance medical education, the authors used telesimulation to teach emergency scenarios with a remote expert providing instruction.¹⁸ A Likert-scale questionnaire revealed overwhelming satisfaction with the simulation-based distance training, and the authors concluded that simulation-based distance medical training proved to be a highly effective tool in improving emergency medical skills of junior physician trainees. They also reported that international simulation-based training may ultimately prove the most realistic platform for large-scale training of emergency medical personnel in less-developed countries and in rural/remote regions of the globe.

A military study designed to assess the efficacy and feasibility of training isolated emergency medical personnel at a naval hospital concluded that human patient simulation improves perceived preparedness and self-efficacy in U.S. Navy emergency medical personnel.¹⁹ They also reported that simulation and distance education allows isolated medical personnel the opportunity to practice skills unconstrained by time or distance.¹⁹ Telesimulation has also received favorable reviews in fields such as pediatric critical care and neonatal resuscitation.^20-22

For the quantitative data collection component of the study, we were able to capture real-time learner assessment data during the virtual simulation component of the course. Using Google Glass to record the triage scenario used to assess participant triage accuracy provided a standardized experience for each learner from the first-person perspective. This immersive method of education and training is in stark contrast to abstract learning experiences such as standard didactics or tabletop exercises, which lack the virtual reality, stressful depiction, moulaged victims, high-stakes responsibility, personal perceived danger, and time pressure of a simulated first-person immersive experience. Research has shown that learning during emotional stress is associated with enhanced declarative memory for emotionally arousing events.²³ The time learners had to evaluate each patient and make a decision with regard to their triage category – less than 60 seconds per patient – was consistent with that intended by the triage decision tool.

We were also able to demonstrate the successful implementation of wearable technology in the creation and delivery of an intercontinental MCI training course. Google Glass is a hands-free, wearable device that allows healthcare providers to evaluate and manage patients while simultaneously recording or transmitting data. The utility of wearable technology in healthcare has been evaluated in medical specialties including surgery,^24-26 cardiology,^27,28 ophthalmology,²⁹ and emergency medicine.^30,31 Our study contributes to the scientific knowledge pertaining to the utility of wearable technology in healthcare in the field of EMS and emergency medicine.

EMS is a relatively new subspecialty in emergency medicine, recognized by the American Board of Emergency Medicine (ABEM) in 2010, with the first certification exam in 2013.³² Literature regarding telesimulation in EMS is virtually nonexistent, and as a result there is limited research to support its evidence base.

Two previous reports using Google Glass have been described, both with substantial methodology limitations. In a feasibility study to determine the effect telemedicine has on the accuracy and timeliness to perform triage in an airplane-related MCI, the authors reported that there was no increase in triage accuracy when paramedics evaluated victims using Google Glass.³³ They also reported that telemedicine required more time than conventional triage. Although reported as a feasibility study, the authors measured and reported quantitative metrics similar to that reported in an observational-analytical study with a nonrandomized intervention and control group. The small sample size (total of four paramedics), lack of randomization, different number of patients triaged by the teams, and unintended technology challenges that precluded real-time transmission of audio and video data through Google Glass, are significant limitations. The second related study used Google Glass during a full-scale exercise to perform visually guided triage and to identify casualties and collect georeferenced notes, photos, and videos into the debriefing.³⁴ The authors reported that Google Glass is a promising technology both for telemedicine applications and augmented-reality disaster response support to increase operators’ performance, helping them to make better choices in the field.³⁴

The aforementioned studies are similar to this report, in that Google Glass was used as an evaluative tool to collect data for assessment and/or educational purposes. This study differs by virtue of its use of wearable/mobile technology to provide real-time training in a live, simulated MCI with integrated debriefing with participant interaction. The wearable technology was also used to create the evaluation resource that was used during the virtual simulation component of the course. To our knowledge, this is the first study of its kind to implement an intercontinental MCI triage course to healthcare providers via telesimulation with Google Glass.

We also considered or encountered barriers and challenges when implementing this telesimulation course. We define barriers as those minimum requirements that must be attained or resources that must be obtained to conduct a telesimulation course. The major barriers include limited availability of telecommunication equipment, simulation resources, and personnel experienced in designing and delivering simulation-based course content (Table 2). Challenges pertain to those problems that educators may encounter in the interim between securing the minimum resources to conduct a telesimulation course and successfully delivering on the educational course protocol. The challenges we encountered were primarily operational. Choosing a software program proved to be an initial challenge as the country we were broadcasting to had strict limitations on the type of software platforms that could be used. Other challenges included Internet connectivity and live broadcasting to an institution 10 hours ahead in time.

LIMITATIONS

The course delivered in this feasibility study was geared toward healthcare providers who would be responsible for managing patients in a MCI. Enrollment for the course was voluntary; thus, the sample may not be representative of all the professions that may be responsible for patient care during a MCI. However, we also believe that those responsible for managing patients during a MCI would be more enthusiastic to engage in this type of course.

The study was not designed as an observational-analytical study, nor was it powered to detect any specific significant difference between the groups. Having noted this, our results are consistent with triage performance demonstrated by healthcare providers undergoing START triage training using non-immersive educational methods.^35,36 We believe this feasibility study makes possible future interventional studies to test whether specific content can be taught effectively with a new educational delivery method. And finally, the course was intended primarily for healthcare providers. However, professionals in the educational setting also participated. The background of this group (“other” category) was heterogeneous and their summary performance measure may not be representative of any particular individual profession in that group.

This telesimulation obviated the need for expert instructors from the sending site to travel to the recipient site to deliver the content, saving cost. Nevertheless, substantial infrastructure investments were necessary on both ends to make the telesimulation possible.

CONCLUSION

This study demonstrates that implementation of an intercontinental MCI-triage training course, and the use of wearable/mobile technology to create and deliver the content though telesimulation, was feasible and well-accepted by learners. Further study is needed to validate telesimulation as a content delivery method to emergency providers.

Footnotes

Section Editor: Joshua B. Gaither, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: C. Eric McCoy, MD, MPH, University of California, Irvine, Department of Emergency Medicine, 333 City Boulevard West, Suite 640, Orange, CA. Email: cmccoy@uci.edu, ericmccoymd@gmail.com. 5 / 2019; 20:512 – 519

Submission history: Revision received August 29, 2018; Accepted March 27, 2019

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Schenk E, Wijetunge G, Mann NC, et al. Epidemiology of mass casualty incidents in the United States. Prehosp Emerg Care. 2014;18(3):408-16.

2. FBI releases study on active shooter incidents. Available at: https://www.fbi.gov/news/stories/2014/september/fbi-releases-study-on-active-shooter-incidents/fbi-releases-study-on-active-shooter-incidents. Accessed April 19, 2017.

3. Guha-Sapir D, Hoyois P, Below R. Annual Disaster Statistical Review 2015: The numbers and trends. Available at: http://www.cred.be/sites/default/files/ADSR_2015.pdf. Accessed April 19, 2017.

4. Blair J, Schweit K. A study of active shooter incidents in the United States between 2000 and 2013. 2014 Accessed January 1, 2017.

5. Global Terrorism Database. 2016. Available at: https://www.start.umd.edu/gtd/. Accessed January 3, 2017.

6. The Learning Pyramid. Available at: https://siteresources.worldbank.org/DEVMARKETPLACE/Resources/Handout_TheLearningPyramid.pdf. Accessed April 19th, 2017.

7. Masters K. Edgar Dale’s Pyramid of Learning in medical education: a literature review. Med Teach. 2013;35(11):e1584-93.

8. Gaba DM. The future vision of simulation in healthcare. Qual Saf Health Care. 2004;13(Suppl):i2-10.

9. McCoy CE, Menchine M, Anderson C, et al. Prospective randomized crossover study of simulation vs. didactics for teaching medical students the assessment and management of critically ill patients. J Emerg Med. 2011;40(4):448-55.

10. Sullivan N, Duval-Arnould J, Twilley M, et al. Simulation exercise to improve retention of cardiopulmonary resuscitation priorities for in-hospital cardiac arrest: A randomized controlled trial. Resuscitation. 2015;86:6-13.

11. Zakhary BM, Jam LM, Kaufman BS, et al. The utility of high-fidelity simulation for training critical care fellows in the management of extracorporeal membrane oxygenation emergencies: a randomized controlled trial. Crit Care Med. 2017;45(8):1367-73.

12. Steadman RH, Coates WC, Huang YM, et al. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006;34(1):151-7.

13. McCoy CE, Sayegh J, Alrabah R, et al. Telesimulation: an innovative tool for health professions education. AEM Education and Training. 2017;1:132-6.

14. Available at: http://www.medsim.uci.edu/index.asp. Accessed April 28, 2017.

15. Lerner E, Schwartz R, McGovern J. Prehospital triage for mass casualties. Emergency Medical Services Clinical Practice and Systems Oversight. 2015:288-91.

16. Simple Triage and Rapid Treatment – background of START. Available at: http://citmt.org/Start/default.htm. Accessed April 28, 2017.

17. START adults triage algorithm. Available at: https://chemm.nlm.nih.gov/startadult.htm#more. Accessed April 28, 2017.

18. von Lubitz DK, Carrasco B, Gabbrielli F, et al. Transatlantic medical education: preliminary data on distance-based high-fidelity human patient simulation training. Stud Health Technol Inform. 2003;94:379-85.

19. Treolar D, Hawayek J, Montgomery JR, et al. On-site and distance education of emergency medicine personnel with a human patient simulator. Mil Med. 2001;166(11):1003-6.

20. Everett T, Ng E, Power D, et al. The Managing Emergencies in Paediatric anaesthesia global rating scale is a reliable tool for simulation-based assessment in pediatric anaesthesia crisis management. Paediatr Anaesth. 2013;23(12):1117-23.

21. Everett B, Bould D, Elaine M, et al. Board 152– Program innovations abstract transcontinental telesimulation: the global proliferation of the Managing Emergencies in Paediatric Anaesthesia (MEPA) course. Available at: http://journals.lww.com/simulationinhealthcare/Abstract/2013/12000/Board_152___Program_Innovations_Abstract.62.aspx. Accessed May 8, 2017.

22. Manhas D, Anderson J. Educational perspectives: telesimulation in neonatal resuscitation. NeoReviews. 2014;15(12). Available at: http://neoreviews.aappublications.org/content/15/12/e514. Accessed May 8, 2017.

23. Cahill L, McGaugh JL. Mechanisms of emotional arousal and lasting declarative memory. Trends Neurosci. 1998;21(7):294-9.

24. Davis CR, Rosenfield LK. Looking at plastic surgery through Google Glass: Part 1. Systematic review of Google Glass evidence and the first plastic surgical procedures. Plast Reconstr Surg. 2015;135:918-28.

25. Iversen MD, Kiami S, Singh K, et al. Prospective, randomized controlled trial to evaluate the effect of smart classes on vestibular examination skills. BMJ Innov. 2016;2:99-105.

26. Liebert CA, Zayed MA, Aalami O, et al. Novel use of Google Glass for procedural wireless vital sign monitoring. Surg Innov. 2016;23:366-73.

27. Duong T, Wosik J, Christakopoulos GE, et al. Interpretation of coronary angiograms recorded using Google Glass: a comparative analysis. J Invasive Cardiol. 2015;27(10):443-6.

28. Jeroudi OM, Christakopoulos G, Christopoulos G, et al. Accuracy of remote electrocardiogram interpretation with the use of Google Glass technology. Am J Cardiol. 2015;115:374-7.

29. Trese MG, Khan NW, Branham K, et al. Expansion of severely constricted visual field using Google Glass. Ophthalmic Surg Lasers Imaging Retina. 2016;47(5):486-9.

30. Chai PR, Wu RY, Ranney ML, et al. Feasibility and acceptability of Google Glass for emergency department dermatology consultations. JAMA Dermatol. 2015;151(7):794-6.

31. Wu TS, Dameff CJ, Tully JL. Ultrasound-guided central venous access using Google Glass. J Emerg Med. 2014;47:668-75.

32. EMS – Overview. Available at: https://www.abem.org/public/subspecialty-certification/emergency-medical-services/ems-overview. Accessed May 11, 2017.

33. Cicero MX, Waish B, Soledad Y, et al. Do you see what I see? Insights from using Google Glass for disaster telemedicine triage. Prehosp Disaster Med. 2015;30(1):4-8.

34. Carenzo L, Barra FL, Ingrassia PL, et al. Disaster medicine through Google Glass. Eur J Emerg Med. 2015;22(3):222-5.

35. Risavi BL, Salen PN, Heller MB, et al. A two-hour intervention using START improves prehospital triage mass casualty incidents. Prehosp Emerg Care. 2001;5(2):197-9.

36. Sapp RF, Brice JH, Hinchey P. Triage performance of first-year medical students using a multiple-casualty scenario, paper exercise. Prehosp Disaster Med. 2010;25(3):239-45.