Author	Affiliation
Philipp Ehlers,	University Hospital Bonn, Department of Emergency Medicine, Bonn, Germany
Matthias Seidel, MD	University Hospital Bonn, Department of Emergency Medicine, Bonn, Germany
Sylvia Schacher, MD	Hospital Köln Kalk, Department of Emergency Medicine, Cologne, Germany
Martin Pin, MD	Hospital Florence Nightingale Düsseldorf, Department of Emergency Medicine, Düsseldorf, Germany
Rolf Fimmers, DSc	University Hospital Bonn, Institute for Medical Biometry, Informatics and Epidemiology, Bonn, Germany
Monika Kogej, MD	University Hospital Bonn, Department of Emergency Medicine, Bonn, Germany
Ingo Gräff, MD	University Hospital Bonn, Department of Emergency Medicine, Bonn, Germany

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Medical handover from prehospital care to the emergency department (ED) is defined as the transfer of responsibility of the care of one or more patients to another person or team.^1,2 Handovers, especially in the ED, are of enormous significance for the subsequent emergency treatment because that treatment requires precise timing, rapid decision-making, and specific expertise.^2,3 Furthermore, the handover is critical for the relaying of information, such as interventions that have occurred and details from the emergency scene. The transfer from prehospital care to the ED is always an interprofessional process involving at least two professional groups. This can lead to misunderstandings and dissatisfaction due to different expectations and approaches.^2,4-6

Importance

Studies by the Australian Commission on Safety and Quality in Health Care have shown that the quality of handover decreases based on an increasing rate of adverse events due to a lack of structure and communication, particularly in the presence of complex patient problems.⁷ Inadequate, incorrect, or misleading information puts patients at risk.⁸ Inadequate communication was one of the most frequent causes of malpractice claims reported to the Joint Commission between 1995–2006.⁹ A study published in 2016 showed that communication errors caused 1744 deaths and resulted in costs of 1.7 billion US dollars over a period of five years in American hospitals.^10,11 The transmission of information in a stressful, highly dynamic work environment such as the ED represents a high-risk source of treatment errors and avoidable adverse events and is therefore relevant to patient outcomes influencing mortality.^12-14

Aim

As early as 2007, the Joint Commission called for the process of handover to be standardized with the aim of increasing patient safety.¹⁵ In 2008, the World Health Organization (WHO) formulated the development of “standard operating procedures” in communication as one of the five priorities in the area of patient safety for industrialized countries.^9,16 To date, a large number of protocols for the standardization of oral handovers have been published.^17,18 In addition to checklists, computer-assisted handover programs and algorithms as well as specific mnemonics have been established to serve as reminders intended to provide guidance when following a process.¹⁹

The goal of this study was to examine the handover procedures in an ED, focused on content, scope and structure and the application of existing handover mnemonics. Our project was a prospective observational study of the handover process, focusing on the interface between the prehospital care and the ED.

METHODS

Emergency Medical Services System Organization

Emergency care in Germany is provided by rescue vehicles that are manned by one of two types of clinicians – paramedics or emergency physicians (EP). The responsible emergency call center decides when an EP is called to the scene according to pre-specified criteria. In many cases, following initial emergency medical care under the supervision of the EP, further transport of the patient is then carried out by the paramedics. In Germany, paramedic training consists of a three-year course with theoretical and practical content as well as a final examination. Physicians can acquire an additional qualification with focus on emergency medicine (EM). This includes 24 months of clinical specialist training, an additional six months of anesthesia, intensive care and EM expertise, an 80-hour theory course, 50 life-saving emergency medical services (EMS) missions, and a final examination. The EMS staff or the responsible control center, respectively, is in charge of the pre-registration of emergency patients at the ED. The above-mentioned training courses historically have had no specific focus on training with regard to the EMS handover process.

 

Population Health Research Capsule

What do we already know about this issue?

The handover process is relevant for adequate treatment of patients and therefore affects patient outcomes. Consequently, it plays a major role in patient safety.

What was the research question?

We sought to provide a current status of handover practice in EDs with regard to content and structure.

What was the major finding of the study?

To date no handover standard has been established and current practice reveals deficits in structure.

How does this improve population health?

This study raises awareness of the need to include handover in national health policy programs, thereby accelerating the process of standardization.

 

Design

We carried out a prospective, multicenter observational study. To minimize bias and to allow comprehensive assessment of EMS handovers, we used a checklist. The checklist was derived by including elements from the following established mnemonics, which are benchmarks in the handover literature: ISBAR (Identification, Situation, Background, Assessment, Recommendation), MIST (Mechanism, Injuries, Signs/Symptoms, Treatment) and BAUM (Situation [German: Bestand], Anamnesis, Examination [German: Untersuchung], Measures).^20-23 The checklist was developed by a selected expert committee of five EPs and paramedics with experience in both prehospital and clinical EM. The final checklist contains all relevant core contents for memory (SAMPLER algorithm – Symptoms, Allergies, Medication, Past medical history, Last oral intake, Events prior to incident, Risk factors) and assessment of the patient’s condition (ABCDE algorithm – Airway, Breathing, Circulation, Disability, Environment/Exposure) as well as vital signs. The ABCDE and SAMPLER algorithms are recommended by WHO for patient treatment according to priority but not specifically for handover process. Both algorithms are core elements of baseline paramedic training and advanced training courses such as International Trauma Life Support (ITLS) and Prehospital Trauma Life Support (PHTLS).^24,25 Detailed descriptions of all mnemonics and algorithms are provided in the electronic attachment.

To gather more nuanced data, handovers were assessed by profession (paramedic vs physician), and severity of the disease/treatment priority (resuscitation vs treatment room), as well as trauma vs non-trauma patients.

Setting

The study was carried out at three hospitals with different emergency care levels. The University Hospital Bonn (UKB) is a comprehensive care hospital with about 45,000 ED visits per year. The Florence Nightingale Hospital in Düsseldorf sees approximately 37,000 emergency patients per year and is a teaching hospital of the University Hospital Düsseldorf. The Protestant Hospital in Cologne Kalk is a non-tertiary teaching hospital of the University Hospital Cologne that cares for approximately 20,000 ED patients per year. In these three EDs, EMS are not specifically required or trained to use any particular handover structure. In this study, we examined only handovers by EMS paramedics or physicians.

Data Collection

During the test period from March 11, 2019–October 31,2019, under supervision of the Institute for Medical Biometry, Informatics and Epidemiology of the University Hospital Bonn, study personnel documented the handovers in EDs between 6:30 am – 9:30 pm. To minimize loss of information, details on prehospital care were obtained from the emergency services documentation record. To ensure standardized application of the checklist and to minimize errors in the documentation, only persons directly involved in the development of the checklist carried out the application. Due to limited personnel, not all handovers within the observation period were recorded.

Statistical Analysis

We used Microsoft Excel 2017 (Microsoft Corporation, Redmond, WA) to manage and tabulate the comprehensive data set. The duration of patient transfer was evaluated descriptively as a continuous variable by specifying the mean value and standard deviation. All other collected data were categorical and were represented by the specification of absolute and relative frequencies, the odds ratio (OR), and the specification of 95% confidence intervals (CI). A statistical comparison of subgroups was carried out using the Chi-square test, or for smaller group sizes, Fisher’s exact test to a significance level of 0.05 (5%). We evaluated all data using SPSS version 26 (SPSS Inc. Chicago, IL).

Ethics Statement

The study received approval (No. 002/19) from the chairman of the local ethics committee (K. Racké, MD, PhD, Professor, University Bonn). Data obtained from the clinical information system may be used in accordance with the code of medical ethics (article 15/1) (https://www.aekno.de/aerzte/berufsordnung#_15 http://www.aekno.de/page.asp?pageID=57#_15) of the General Medical Council. Furthermore, as stipulated by German data protection regulations, the physician may use existing patient data for analyses without explicitly asking for the consent of patient. All collected clinical data evaluated in this study were fully anonymized prior to analysis. Furthermore, the data collected do not contain any patient information. The study design is consistent with the Declaration of Helsinki.²⁶

RESULTS

Baseline Characteristics

During the observation period, a total of 721 handovers were examined in the three EDs. Of these handovers, 44.5% (n = 321) were carried out by EPs and 55.5% (n = 400) by paramedics. 79.1% (n = 570) of the transfers invovled non-trauma emergency patients, and 20.9% (n = 151) patients after trauma. Of the transfers, 30.5% (n = 220) took place in the resuscitation room, the remaining 69.5% (n = 501) in normal treatment rooms or in the triage room. The mean value of the transfer time was one minute 11 seconds. (standard time deviation STD ± 0:34 minutes). In 74.5% (n = 537), the ED personnel raised further questions for better understanding.

Identification, Mechanism and Medical Situation

The sex of the emergency patient was mentioned with a frequency of 95.6% (n = 689) at the time of delivery; the name of the patient was mentioned with a frequency of 83.8% (n = 604) and the age of the patient was mentioned in 47.0% (n = 339) of the cases. The suspected diagnosis was reported in 95.7% (n = 690) and the emergency event in 90.4% (n = 652). Comparatively less frequently, information regarding the place where the emergency occurred was reported in 66.4% (n = 479) and the time it occurred in 37.7% (n = 272) of cases (Table 1).

Leading Priority and Vital Signs

The frequency of the handovers in which the ABCDE algorithm recommended by WHO was completely applied (chronological mention of all elements) was 4.3 % (n = 31). The subgroup analysis shows that physician staff performed a complete ABCDE handover 7.2% of the time, compared to paramedics who used it 2.0% of the time (OR: 3.8, p < 0.05). Also, the complete ABCDE algorithm was applied to resuscitation room patients more frequently (OR: 7.2, p < 0.05), compared to transfers in the conventional treatment rooms or the triage room. The same trend was observed in the transfer of trauma patients compared to non-trauma emergency patients (OR: 18.7, p < 0.05). In 86.1% (n = 621) of the handovers, the ABCDE algorithm was not applied, while in 3.2% (n = 23), a handover with at least three points of the ABCDE algorithm took place (Table 2).

Looking at the prehospital vital signs and their communication during handover, the following pattern becomes apparent: In only 44.7% (n = 289) of cases was the blood pressure (BP) mentioned in the handover. In 30.6% (n = 199) of handovers the heart rate (HR) was verbalized, while the oxygen saturation (SpO₂) was only communicated in 25.6% (n = 165) of cases. The respiratory rate was only communicated in 12.8% of handovers. The testing of circulation, sensation and mobility (CSM) was communicated much more often, in 76.9% of cases. Other important elements, such as Glasgow Coma Scale (GCS), blood sugar (BS) and temperature are listed in Table 3.

The subgroup analysis of the different occupation groups shows that trained EPs more often refer to the transmitted vital parameters BP (OR: 1.9), HR (OR: 2.2), SpO₂ (OR: 2.7) and GCS (OR: 5.1) at the time of handover (Table 4). The subgroup analysis of transfers in resuscitation room patients shows that the above-mentioned vital signs were also more frequently reported compared to handovers in normal treatment rooms (Table 5). Differentiation between trauma patients and non-trauma emergency patients revealed that GCS was mentioned more frequently in trauma patients (p < 0.05).

Medical History and Risk Factors

Previous illnesses of the emergency patient were reported at the handover with a frequency of 49.7% (95% CI, 46.0–53.3 / n = 358) and the risk factors of the patient in 54.4% (95% CI, 50.7–58.0 / n = 392). The patient’s home medication was mentioned in 41.2% (95% CI, 37.6–44.8 / n = 297) of the cases. Information on existing allergies was significantly less often reported in 17.0% (95% CI, 14.3–19.8 / n = 123) and on the last meal in 3.9% (95% CI, 2.6–5.3 / n = 28) of cases.

In just 1.1% of the cases (n = 8) was the SAMPLER algorithm, recommended by WHO, fully applied (chronological mention of all letters or their contents). In 27.2% (n = 200) of the handovers, at least three contents of the SAMPLER algorithm were mentioned at the handover. The subgroup analysis shows that in comparison to the paramedics, physicians more frequently mentioned at least three SAMPLER components (p < 0.05). The same is true for resuscitation room handovers when compared to the treatment room patients (p < 0.05), and for the trauma vs the non-trauma emergency patients (p < 0.05). In 20.0% (n = 144) of the handovers, no information of the SAMPLER algorithm was transmitted.

3.5 Emergency Treatment

Analysis of prehospital therapeutic activities shows the following results: Intravenous (IV) access was mentioned in only 37.2% (n = 132) of the cases at handover and had the lowest ratio between performance and handover of all preclinically performed measures. The preclinically derived 12-lead electrocardiogram was discussed in 75.7% (n = 109) of the cases at handover. In 58.9% (n = 63) of cases, information on prehospital oxygen therapy was provided at the handover. Drug administration and airway management were the most frequently mentioned rescue measures at handover. Defibrillation as a life-saving measure was mentioned in 85.7% of the cases, if performed as a prehospital treatment (Table 6).

The subgroup analysis of resuscitation room patients shows that, in comparison to handover of patients in normal treatment rooms, all prehospital therapeutic measures were mentioned with the same frequency. The only significant difference was found in the establishing of an IV access (p < 0.05).

DISCUSSION

This is the first prospective study to examine the EMS handover process in German EDs in terms of content, scope, and structure in relation to existing handover mnemonics. The work is intended to present the current handover practice and demonstrates that the handover does not follow a clear protocol and that a pronounced inconsistency exists in information transfer. In addition, differences in the extent and completeness of the handovers are apparent depending on staff and the priority of treatment (resuscitation room vs treatment room) and the injury pattern (trauma vs non-trauma patients). The data collected from the three EDs refer to a large supply area of the rescue service in the German federal state of North Rhine-Westphalia (NRW). Since NRW is the federal state with the highest population in Germany (approximately 18 million) and the structure of emergency services does not differ significantly from that of the other regions of Germany, we believe that the data presented have a high scientific validity for Germany and may have important implications for other countries as well.

The data of the present study are supported by another European study conducted by Delupis et al in Italy. They found comparable results in their work: the absence of standardization of the handover process; a high variability in information transfer; and deficiencies in the transfer of responsibility of patient care.²⁷

It is notable that the presence of a higher disease severity with pathological vital signs appears to be a trigger for more verbalization at the handover. Conversely, in less critical patients, information regarding the leading medical problem, vital signs, and other information from the patient’s medical history may not be considered relevant for the handover. To date, numerous studies have shown that vital signs, especially respiratory rate, BP, and GCS, have a predictive value for the outcome of critical emergency patients.^28,29 In this context, vital signs play an important role in order to evaluate critical conditions of patients by using scores such as CRB 65 and qSOFA.^30,31 Here, a transfer of vital signs is categorically called for, independent of the severity of the illness and the qualification of the person transmitting the data. Information on the time component of the emergency event is essential regarding time-critical therapeutic measures including thrombolytics for stroke or time-sensitive sepsis bundles.³²

The main findings show that with regard to MIST, ISBAR and BAUM, no mnemonics were applied during handover, resulting in a lack of structure and information transfer. This is supported by the high demand for additional information from the receiving team. One explanation lies in the individual design of the handover process, resulting in incongruence between expected and actually transferred information. In our opinion, this is not due to a lack of handover mnemonics, but rather to the fact that to date, no handover practice exists that fully meets the high requirements of a transfer in the ED. According to Nasarwanji et al, not all information necessary for the transfer can be accommodated in a generally valid mnemonic.³³ Hence, the handover process needs a specifically adapted mnemonic, with elements from the ABCDE or SAMPLER algorithms. Since the handover is strongly influenced by human factors, consideration should be given to integrating crew resource management aspects into the handover process to improve patient safety.³⁴ Other handover practices to promote effective transfer of information include the following: no actions performed on patients during the handover; face-to-face communication; presence of all team members; a repeat back of essential handover content; and an opportunity for questions.

This thesis is supported by the work of Keebler et al, who with the help of a systematic literature review and a series of meta-analyses, examined many publications on handover standardization. Keebler et al took on the standardization of the handover in 2017, as called for by the Joint Commission in 2007, and found that all studies follow different standards, enabling only limited comparability.¹⁹ In their conclusion, the authors recommended that protocols should standardize the handover and provide users with orientation as to what information should be transmitted.

It becomes clear that despite the available mnemonics and the numerous studies on standardization of the handover, we still have a gap between the theoretical handover approach and its practical implementation. The target must be the creation of a shared mental model between emergency services and hospital staff. This would enable handovers in an interprofessional, team-based manner.^35,36 Therefore, future research should concentrate on combining elements of clinical effectiveness and implementation using hybrid study designs to enhance the practical application of specifically adapted mnemonics.³⁷ In concrete terms this means developing a mnemonic with the requirements described above, which then is validated using the Delphi method. Subsequently, the effectiveness of the mnemonic and its implementation (ie. its acceptance by paramedics) has to be examined by prospective studys.

Furthermore, national initiatives for the general implementation of handover approaches in the clinical setting are necessary for Germany and other countries, in line with the initiatives already taken in Australia, Great Britain, and the USA. The provision of appropriate financial and human resources for the implementation of this health policy objective is an indispensable prerequisite. In the near future, external audits must review the introduction and application of structured handover processes in relation to triage in the ED. It also seems necessary to include the topic of handover as training content in the curricula of the proven prehospital and hospital course concepts such as Advanced Life Support, ITLS, PHTLS, Advanced Trauma Care for Nurses, and Advanced Trauma Life Support. The handover should be incorporated into the training of paramedics, as well as into the further training programs for EPs and nurses.

LIMITATIONS

It is possible that this study includes repetitive handovers by the same EMS staff during the observation period. Thus, some of our results may have been limited by our sample population. However, given the large catchment area of the three EDs, this is unlikely to affect the overall significance and results of the study. Additionally, the selected period from March 2019–October 2019 did not allow any conclusions to be drawn for an entire year, as possible seasonal fluctuations were not considered. Furthermore, patient transfers during night shifts were not documented. It cannot be ruled out that the content and scope of the handover may vary regarding the time of day.

It is possible that while applying the checklist, information may not have been recorded or missed. However, we consider the percentage as negligible, since the person documenting never participated in direct patient care and was as an external observer. Finally, since it could not be avoided that several handovers took place at the same time, the external observers were not able to record the data of all handovers in the given observation period. Therefore, it must be assumed that in comparison to the results, both better structured as well as worse structured handovers were not recorded. Nevertheless, due to the high number of cases and the observation at three EDs, the present results create a representative picture of the current handover process.

CONCLUSION

The present study shows that despite many existing handover protocols, there is no widespread implementation or acceptance of these protocols. Not even the measures recommended by the World Health Organization to increase patient safety are reliably transmitted during handover. Future research should aim at establishing appropriate user-friendly handover protocols for the ED. Improving and standardizing the EMS-to-ED handover process has a high potential to improve patient safety and emergency care.

Footnotes

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Ingo Gräff, MD, University Hospital Bonn, Department of Emergency Medicine, Venusberg-Campus 1, 53127 Bonn, Germany. Email: ingo.graeff@ukbonn.de. 3 / 2021; 22:401 – 409

Submission history: Revision received April 23, 2020; Submitted September 6, 2020; Accepted September 13, 2020

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Merten H, van Galen LS, Wagner C. Safe handover. BMJ. 2017;359:j4328.

2. Evans SM, Murray A, Patrick I, et al. Clinical handover in the trauma setting: a qualitative study of paramedics and trauma team members. Qual Saf Health Care. 2010;19(6):e57.

3. Fernando K, Adshead N, Dev S, et al. Emergency department multiprofessional handover. Clin Teach. 2013;10(4):219-223.

4. Owen C, Hemmings L, Brown T. Lost in translation: maximizing handover effectiveness between paramedics and receiving staff in the emergency department. Emerg Med Australas. 2009;21(2):102-107.

5. Dawson S, King L, Grantham H. Review article: Improving the hospital clinical handover between paramedics and emergency department staff in the deteriorating patient. Emerg Med Australas. 2013;25(5):393-405.

6. Najafi Kalyani M, Fereidouni Z, Sarvestani RS, et al. Perspectives of patient handover among paramedics and emergency department members; a qualitative study. Emerg (Tehran). 2017;5(1):e76.

7. a structured evidence-based literature review effectiveness of improvement interventions in clinical handover. 2008. Available at: https://www.safetyandquality.gov.au/sites/default/files/migrated/Clinical-Handover-Literature-Review-for-release.pdf. Accessed September 14, 2020.

8. Sieber R. Strukturierte Patientenübergabe: Schritt für Schritt durch den komplexen Ablauf an der Schnittstelle Rettungsdienst-Notfallstation. 2009. Available at: https://www.notfallpflege.ch/files/_Demo/Dokumente/Artikel/Strukturierte_Patientenuebergabe.pdf. Accessed September 14, 2020.

9. Eggins S, Slade D. Communication in clinical handover: improving the safety and quality of the patient experience. J Public Health Res. 2015;4(3).

10. Rodziewicz TL, Hipskind JE. Medical Error Prevention. StatPearls [Internet]. 2019.

11. Bailey M. Communication failures linked to 1, 744 deaths in five years, US malpractice study finds. 2016. Available at: https://www.statnews.com/2016/02/01/communication-failures-malpractice-study/. Accessed September 14, 2020.

12. Zinn C. 14,000 preventable deaths in Australian hospitals. BMJ. 1995;310(6993):1487.

13. Joint Commission Center for Transforming Healthcare Releases Targeted Solutions Tool for Hand-Off Communications. 2012. Available at: https://www.jointcommission.org/assets/1/6/tst_hoc_persp_08_12.pdf. Accessed September 14, 2020.

14. Theobald CN, Choma NN, Ehrenfeld JM, et al. Effect of a handover tool on efficiency of care and mortality for interhospital transfers. J Hosp Med. 2017;12(1):23-28.

15. The Joint Commission’s Annual Report on Quality and Safety. 2007. Available at: https://www.jointcommission.org/-/media/tjc/documents/accred-and-cert/hap/annual-report/2007_annual_reportpdf.pdf?db=web&hash=2092D322E8296BCE340B5FF331BDAD82. Accessed September 14, 2020.

16. Global Priorities for Research in Patient Safety. 2008. Available at: https://www.who.int/patientsafety/research/priorities/global_priorities_patient_safety_research.pdf. Accessed September 14, 2020.

17. Shah Y, Alinier G, Pillay Y. Clinical handover between paramedics and emergency department staff: SBAR and IMIST-AMBO acronyms. International Paramedic Practice. 2016;6(2):37-44.

18. Porteous JM, Stewart-Wynne EG, Connolly M, et al. iSoBAR–a concept and handover checklist: the National Clinical Handover Initiative. Med J Aust. 2009;190(S11):S152-6.

19. Keebler JR, Lazzara EH, Patzer BS, et al. Meta-analyses of the effects of standardized handoff protocols on patient, provider, and organizational outcomes. Hum Factors. 2016;58(8):1187-1205.

20. Jensen SM, Lippert A, Østergaard D. Handover of patients: a topical review of ambulance crew to emergency department handover. Acta Anaesthesiol Scand. 2013;57(8):964-970.

21. Waßmer R, Zimmer M, Oberndörfer D, et al. Kann durch eine einfache Schulung das Kommunikations- und Patientenübergabemanagement in der Notfallmedizin verbessert werden?. Notfall Rettungsmed. 2011;14(1):37-44.

22. Iedema R, Ball C, Daly B, et al. Design and trial of a new ambulance-to-emergency department handover protocol: ‘IMIST-AMBO’. BMJ Qual Saf. 2012;21(8):627-633.

23. Schacher S, Glien P, Kogej M, et al. Strukturierte Übergabeprozesse in der Notaufnahme. Notfall Rettungsmed. 2019;22(1):3-8.

24. Häske D, Beckers SK, Hofmann M, et al. Quality of documentation as a surrogate marker for awareness and training effectiveness of phtls-courses. part of the prospective longitudinal mixed-methods EPPTC-Trial. PLoS ONE. 2017;12(1):e0170004.

25. Maegele M. Prehospital care for multiple trauma patients in Germany. Chin J Traumatol. 2015;18(3):125-134.

26. Heilmann J. Gesundheitsdatenschutz. 2012. Available at: https://www.bund-verlag.de/dam/jcr:c804f91a-40cc-49a4-ac37-35e970ea0c12/CuA%20Jahresregister%202012.pdf. Accessed September 14, 2020.

27. Di Dojmi Delupis F, Mancini N, Di Nota T, et al. Pre-hospital/emergency department handover in Italy. Intern Emerg Med. 2015;10(1):63-72.

28. McNett M, Amato S, Gianakis A, et al. The FOUR score and GCS as predictors of outcome after traumatic brain injury. Neurocrit Care. 2014;21(1):52-57.

29. Barthel P, Wensel R, Bauer A, et al. Respiratory rate predicts outcome after acute myocardial infarction: a prospective cohort study. Eur Heart J. 2013;34(22):1644-1650.

30. Park JE, Hwang SY, Jo IJ, et al. Accuracy of the qSOFA Score and RED sign in predicting critical care requirements in patients with suspected infection in the emergency department: a retrospective observational study. Medicina (Kaunas). 2020;56(1).

31. Kabundji DM, Musekiwa A, Mukansi M, et al. Determining need for hospitalisation: evaluation of the utility of the CRB-65 score in patients with community-acquired pneumonia presenting to an emergency department. S Afr Med J. 2014;104(11):769-772.

32. Levy MM, Evans LE, Rhodes A. The Surviving Sepsis Campaign Bundle: 2018 Update. Crit Care Med. 2018;46(6):997-1000.

33. Nasarwanji MF, Badir A, Gurses AP. Standardizing handoff communication: content analysis of 27 handoff mnemonics. J Nurs Care Qual. 2016;31(3):238-244.

34. Haerkens MHTM, Kox M, Lemson J, et al. Crew resource management in the intensive care unit: a prospective 3-year cohort study. Acta Anaesthesiol Scand. 2015;59(10):1319-1329.

35. Manser T, Foster S, Gisin S, et al. Assessing the quality of patient handoffs at care transitions. Qual Saf Health Care. 2010;19(6):e44.

36. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):1803-1812.

37. Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-226.