Background
Objectives
Curricular design
Impact/effectiveness
Limitations
Supplementary information

BACKGROUND

Medical error has received considerable attention since the Institute of Medicine estimated that, in the United States alone, as many as 98,000 patients die annually from preventable medical mistakes.¹ While the exact numbers are disputed, and remain difficult to measure, more recent studies estimate the number of deaths attributable to medical errors to be around 250,000 per year in the U.S.² Emergency departments (ED) are notoriously prone to such errors^3,4, and evidence suggests that one key to decrease the rate of mistakes in acute care settings is to improve teamwork.⁵

Diagnostic accuracy can be increased through interaction in the ED,^6,7 and improved coordination within teams in intensive care is associated with decreased patient mortality.⁸ Transfer of care situations, such as a handover from prehospital to hospital teams, are particularly susceptible to medical errors, due to communication failures and loss of information.⁹ These factors, fortunately, seem to be amenable to training.¹⁰ The World Health Organization specifically suggested improving interprofessional collaboration as an important way to reduce medical error.¹¹ The recent “Call to Action for Emergency Medicine” by Wilbur¹² highlights the importance of this collaboration, advocating for the implementation of interprofessional education and its evaluation in emergency medicine.

Education is interprofessional “when students from two or more professions learn about, from, and with each other to enable effective collaboration and improve health outcomes.”¹² One central goal of such education is the improvement of team skills¹³ and communication. Consequently, early educational interventions that improve communication within healthcare teams are likely to be beneficial to patients. Still, professionals who are meant to routinely collaborate with others in an interdisciplinary ED are trained and educated in separate “silos”¹² in many countries, rendering the development of shared mental models, a common language, or a clear conception of each other’s roles virtually impossible.

OBJECTIVES

We aimed to conceptualize, implement, and evaluate a training module for final-year medical students, as well as advanced trainees from emergency nursing and student paramedics. The objective was to sensitize participants to professional barriers for their communication across borders, especially in the ED, as it represents an important interface between prehospital and ED teams. Our goal was to establish a mutual understanding of each other’s roles, and of professional self-conceptions. We further aimed to enable participants to conduct basic emergency care for a critically ill patient as an independent interprofessional ad hoc team, with a special focus on communication and team interaction.

CURRICULAR DESIGN

Conception

Due to the above-described lack of interprofessional training, we developed a longitudinal, simulation-based training format for three professions based on Kern’s six-step approach¹⁴ (problem identification, needs assessment, formulation of objectives, developing formats, implementation, and evaluation). An interprofessional team consisting of medical, nursing, and paramedic educators planned the educational activity. The resulting training consists of several one-day training modules. We conducted two modules as a pilot. Each module has the same overall structure of an introduction followed by simulated scenarios. The scenarios differ between modules. We invited the same population of student paramedics and emergency nursing trainees, who are both organized in classes. For both groups, the two modules were made a part of their schedule. As a result, most of the student paramedics and emergency nursing trainees participated in both pilot modules and will participate in the following one as a longitudinal course.

Such a longitudinal integration was not possible for medical students, because the training modules were not compulsory and not planned as a longitudinal format, due to difficulties in acquiring a series of time slots in their busy academic schedule. As a result, different medical students participated in the first and second training module. However, this also provided a greater number of medical students the opportunity to attend a training module at least once and relates to real-world circumstances, where teams often form ad hoc without prior acquaintance.¹⁵

Our long-term objective is to implement this longitudinal format into the new curricula for emergency nursing and paramedics trainees, as well as to offer the format as a voluntary course for medical students. As we have extensive experience with the team-training format in the ED setting,^17,18 we decided to use simulation-enhanced interprofessional education as our educational strategy, due to its well-known positive effects on attitudes towards teamwork and communication.^19,20

Implementation

After providing oral and written informed consent at the beginning of every module, participants were randomly assigned into four groups, equally staffed with the three professions. Each team met for an introduction session in the morning. The purpose of this first interprofessional meeting was to get acquainted with each other, to discuss each member’s roles, professional self-concepts, common misconceptions, and communication barriers. Every module had a “Topic of the Day”, such as “handover,” “Manchester Triage System,” or “Crisis Resource Management,” which was introduced by an impulse presentation. Furthermore, the interprofessional team of instructors asked for expectations and personal goals for each day.

After introduction, every team rotated through different simulated emergency cases throughout the day. Cases were selected by the interprofessional team of instructors, reflected common emergencies, and contained challenges for interprofessional collaboration, such as team communication and interaction. Every case used high-fidelity simulators or simulated patients. Examples of cases used in the first training module are depicted in Table 1. Each case consists of a prehospital section (for paramedics and medical students), a handover (everyone), and an ED section (medical students and emergency nurses).

After every case, all students underwent extensive structured debriefing by an interprofessional team of instructors, which consisted of members of at least two professions, as well as one expert in communication. Debriefing structure followed the common three-step GAS-model^21, and is composed of the following parts:

Gather information from participants (“How do you feel after this case?”)

Analyze information with further questions (“What went well?;” “What happened during handover?;” “Do you see any chance for improvement?”) and directive feedback

Summarize debriefing with learning goals for the next simulation by the instructor team. Debriefing focuses especially on the “Topic of the Day.”

Adaption to Participant Feedback

We implemented some changes in our concept after the first training module, based on oral and written feedback from the participants and group discussions among the instructors:

– The number of cases was decreased; simulation/debriefing-time was increased to 90 minutes, allowing for more debriefing time.

– Focus of cases was changed, and more time for paramedic treatment was given: transportation by ambulance was added, so as to balance treatment time between disciplines.

– Due to confusion during debriefing in the first event, roles during debriefing were specified: the medical and communication debriefings were divided between different instructors, so as to establish a more focused observation during simulation. We also allocated a time slot to allow for peer observers to give individual (one-on-one) feedback to their colleagues.

More simulated patients were added, as feedback indicated that they were particularly challenging during scenarios. Furthermore, other challenges, such as pediatric emergencies, distractors, bystanders, and technical incidents, were added to the cases.

IMPACT/EFFECTIVENESS

Although the relationship between team performance and team culture has long been recognized in acute care,²² establishing a link between team characteristics and patient outcome is notoriously difficult.^5,23 One reason for this is that the effect of any educational intervention is likely diluted by the many other factors influencing the transition from individual learning, to behavior within teams, team performance, and finally to patient care, which ultimately determines patient outcome.^5,24 Thus, Cook and West recommend chains of carefully designed studies linking educational interventions to learning effects, learning to behavioral change in the workplace, and behavioral changes to changes in patient care, finally influencing patient outcomes.²⁴ Many studies, however, fall short of assessing educational outcomes beyond participant satisfaction. ^25,26 “Commitment to Change” (C2C) is one of the few tools that can be used to promote and assess behavioral changes induced by an educational intervention.^27-29 It has been extensively used in different areas, inside and outside healthcare, to stimulate and evaluate performance change.^27,29-35 C2C has been associated with behavioral change,^29,36,37 and is predictive of success in change initiatives.^34,38 In this C2C approach, participants are anonymously asked to state up to three changes they wish to implement as a result of a course, as well as the strength of their commitment to these changes. After a timespan that allows for implementation, participants are asked to report on their success, and reflect on factors that fostered or hindered implementation.

We translated the original English version of C2C³⁹ into German using the established TRAPD (translation, review, adjudication, pre-test, documentation) methodology.⁴⁰ The translated version is available as an appendix.

We collected C2C directly after training (t1), ensuring participant anonymity, while also enabling a follow-up survey after two months (t2). Specifically, we asked participants to generate a unique individual code by appending the first two letters of their mother’s given name, the last two digits of their father’s year of birth, and the first two letters of their place of birth (e.g. PE62BE for a mother named Petra, a father born in 1962, and Berlin as place of birth). For follow-up, we provided the participants with envelopes labeled with their code, containing a follow-up survey on their personal commitments to change. Medical students, who are more difficult to reach as they are not organized into classes, were invited via mail to participate in the follow-up survey. An incentive of 20€ was granted to every medical student participating in follow-up.

We analyzed commitments, together with basic demographic data, in a mixed method approach, both quantitatively and qualitatively. Basic demographic characteristics of our participants and attendance at follow-up are shown in Table 2.

Textual data, such as commitments, or responses regarding factors that fostered or hindered implementation of the intended changes, were inductively categorized by three researchers (DE, FS, and JG) according to Mayring.^41,42 All three researchers (two physicians and one senior medical student) discussed each commitment until full consensus was reached regarding which category was the most appropriate. Emerging categories were defined and adapted, regrouping statements until all commitments were assigned to as little categories as we deemed appropriate. After categorization, the results were presented to an independent psychologist, who was responsible for consistency check and content validation. The process of inductive categorization is often used with qualitative data. The indicators used to assess the quality of qualitative research are generally different from the quantitative methods commonly used in biomedicine, although the quality principles applied to both are similar.⁴³

In total, 64 of 90 participants (71.1%) made at least one commitment to change after the training modules (18 trainees of emergency nursing, 22 student paramedics, 15 medical students, and 9 not assignable). That led to a total of 123 commitments made by our participants (see Table 3), which were divided into four broad areas. Commitments were evenly distributed over three emerging categories, namely communication, behavior, and knowledge, as well as a slightly less prominent fourth category, attitude. Roughly one third of behavior- and attitude-related commitments refer to interprofessional topics (e.g., “Acknowledge other professions’ work”), and these were equally distributed among professions. Table 3 presents all categories and examples of commitments to change.

At the two months follow-up (t2), 32 participants (50%) provided written comments on their original commitments. At follow-up, 57 of the 62 (91.9%) commitments were reported to be at least partly realized, and only five (8.1%) commitments (still) lacked realization. The best rate of commitment realization was (self-) reported by trainees of emergency nursing, with 13 fully implemented commitments out of 31 (41.9%). We did not observe any significant correlations between the strength of commitments and the probability of their realization (r=0.222; p=0.1), suggesting that realization is more strongly influenced by external factors in the workplace than by participant motivation. This hypothesis is further supported by the qualitative analysis of factors that hinder implementation, namely “not enough practice,” “not enough time,” “unsupportive colleagues,” and “excessive demand.” Likewise, the most frequently mentioned factors fostering change were “practice,” “colleagues,” and “teachers.”

Validity Argument

In this description of an educational intervention, we report on self-reported commitments to change and self-reported implementation rates. One apparent question that results from the nature of these data is, whether or not C2C is a valid measure of educational outcome for our training. In the following section, we will thus discuss the validity argument for the conclusions drawn from this study, guided by Messick’s five sources of validity evidence as adapted to medical education by Cook et al⁴⁴ and Beckman.⁴⁵

The C2C survey basically consisted of one item: “I commit to complete the following in the next 2 months:” with the option to make up to three statements of anticipated changes, together with a strength of one’s commitment. For all its brevity, this approach has been taken successfully for many years in different contexts.^27,29-35 Purkis et al. were able to demonstrate that self-reported intentions of changing behavior were followed by actual behavior changes in physicians following a continuing medical education (CME) intervention.²⁹

Content evidence: At present, the use of C2C has rarely been reported in an interprofessional setting. However, behavioral and attitudinal changes were emerging categories in our study and insufficient time was frequently cited as a barrier for realization, consistent with Evan’s findings.³⁵ Because there was no possibility to directly measure and observe changes of our participants in their workplace, we chose the well-established method of C2C, which has been developed and validated for this very content.

As for the response process, we report that C2C was part of the evaluation at the end of every module. Statements made by participants were consistent, reflecting a good understanding of the question. All participants had protected time to complete the survey, with an instructor available for questions. We observed a slightly increased motivation to take part in the survey after the second module as some participants already knew the tool and had received their own statements of the first module as a reminder during follow-up. Furthermore, we investigated the possibility of a non-response bias as a possible consequence of the response process. (See below.)

Many participants committed to similar changes, which we were able to cluster into different categories as shown above. Since data from C2C do not allow for elaborate quantitative analyses, we regard this as the best possible internal structure evidence. Due to this relatively new approach in an interprofessional educational setting we, however, fail to provide relationship evidence.

However, regardless of the content of the commitments made or the ability to realize the anticipated changes, the first consequence of the C2C survey was that participants had to reflect on what they had just learned, helping them to identify areas of personal improvement. As a second consequence, at least some participants will try to actually put their committed changes into realization in their workplace (consequences evidence).

Nonresponse Bias Analysis

Due to the dropout of 50% between first assessment and follow-up, we conducted a nonresponse bias evaluation and tested for differences between the responder and non-responder group using exploratory statistics. Nonresponse bias is a bias resulting from one group of participants being systematically more likely to answer a survey than another;⁴⁶ e.g., participants who were successful in implementing their intended changes could be more willing to report on those successes than participants who could not realize these changes. There were no significant differences between groups in age (p = 0.340; independent samples t-test) and gender (p = 0.294; Fisher’s exact test). However, trainees of emergency nursing (n=14; 77.8% response rate on follow-up) and student paramedics (n=13; 59.1%) were significantly more likely to respond than medical students (n=1; 6.7%; p < 0.01; Pearson’s chi-squared test).

LIMITATIONS

Relying on self-reports only, our data are inherently limited. Also, although the C2C-approach employed in this study has been extensively used in continuous medical education, further research is warranted to strengthen the link between teaching events, C2C, and objective changes in the workplace. Furthermore, C2C is meant as a tool to enhance change in the workplace, and as such, from a theoretical perspective, its use as a measurement instrument is limited.

Another limitation is the 50% response rate in the follow-up survey, which may introduce nonresponse bias. There is only little reported use of the C2C approach in an interprofessional setting.³⁵ Compared to studies surveying students, that report response rates of 46%-31%,^47-50 a 50% response rate in our sample seems satisfactory. However, conclusions about medical students remain limited, despite an incentive, due to the high dropout rate. This effect could be related to poor availability, as the training modules are not part of medical students’ mandatory curriculum, and students hardly participated more than once.

Lessons Learned and Future Directions

Planning an interprofessional simulation training requires considerable time, coordination, and resources. It is very rewarding to see that the effort has an effect beyond participant satisfaction. C2C is an easy-to-use tool to help students reflect on what lessons to take home – and into their work place. While lack of time is a frequently cited obstacle hindering change,^35,51 a lack of practical training, as well as unsupportive colleagues and teachers in the workplace, seem to be neglected as a factor preventing students from change. As practice is important, interprofessional simulation trainings and internships should be implemented as longitudinal programs in the respective curricula of all involved health professions.

Supplementary Information

Footnotes

Section Editor: Andrew W. Phillips, MD, MEd

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Dorothea Eisenman, MD, Lernzentrum der Charité Universitätszmedizin Berlin, Department of Anesthesiology and Operative Intensive Care Medicine, Charitéplatz 1, 10117 Berlin, Germany. Email: dorothea.eisenmann@charite.de 1 / 2018; 19:185 – 192

Submission history: Revision received June 15, 2017; Submitted October 23, 2017; Accepted November 3, 2017

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Kohn LT, Corrigan J, Donaldson MS. To Err Is Human: Building a Safer Health System. 2000.

2. Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016:i2139.

3. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008;121(5 Suppl):S2-23.

4. Fordyce J, Blank FSj, Pekow P, et al. Errors in a busy emergency department. Ann Emerg Med. 2003;42(3):324-33.

5. Hautz WE, Kämmer JE, Exadaktylos A, et al. How thinking about groups is different from groupthink. Med Educ. 2017;51(2):229.

6. Hautz WE, Kammer JE, Schauber SK, et al. Diagnostic performance by medical students working individually or in teams. JAMA. 2015;313(3):303-4.

7. Kämmer JE, Hautz WE, Herzog SM, et al. The potential of collective intelligence in emergency medicine: pooling medical students’ independent decisions improves diagnostic performance. Med Decis Making. 2017.

8. Knaus WA, Draper EA, Wagner DP, et al. An evaluation of outcome from intensive care in major medical centers. Ann Intern Med. 1986;104(3):410-8.

9. Horwitz LI, Moin T, Krumholz HM, et al. Consequences of inadequate sign-out for patient care. Arch Intern Med. 2008;168(16):1755.

10. Stojan J, Mullan P, Fitzgerald J, et al. Handover education improves skill and confidence. Clin Teach. 2016;13(6):422-6.

11. Framework for Action on Interprofessional Education & Collaborative Practice. 2010. Available at: http://www.who.int/hrh/resources/framework_action/en/.

12. Wilbur L. Interprofessional education and collaboration: a call to action for emergency medicine. Acad Emerg Med. 2014;21(7):833-4.

13. Thistlethwaite J. Interprofessional education: a review of context, learning and the research agenda. Med Educ. 2012;46(1):58-70.

14. Kern DE. Curriculum Development for Medical Education: A Six-Step Approach. 2009.

15. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature: Teamwork and patient safety. Acta Anaesthesiol Scand. 2009;53(2):143-51.

16. Machner M, Möckel M, Liehr B, et al. Entwicklung eines Curriculums für Notfallpflege an der Charité: Qualifikation gleich Qualität. Notf Rettungsmedizin. 2017;20(6):522-36.

17. Stroben F, Schröder T, Dannenberg KA, et al. A simulated night shift in the emergency room increases students’ self-efficacy independent of role taking over during simulation. BMC Med Educ. 2016;16(1):177.

18. Freytag J, Stroben F, Hautz WE, et al. Improving patient safety through better teamwork: how effective are different methods of simulation debriefing? Protocol for a pragmatic, prospective and randomised study. BMJ Open. 2017;7(6):e015977.

19. Palaganas JC, Epps C, Raemer DB. A history of simulation-enhanced interprofessional education. J Interprof Care. 2014;28(2):110-5.

20. Wong AH-W, Gang M, Szyld D, et al. Making an “attitude adjustment”: using a simulation-enhanced interprofessional education strategy to improve attitudes toward teamwork and communication. Simul Healthc J Soc Simul Healthc. 2016;11(2):117-25.

21. Sawyer T, Eppich W, Brett-Fleegler M, et al. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simul Healthc J Soc Simul Healthc. 2016;11(3):209-17.

22. Krimsky WS, Mroz IB, McIlwaine JK, et al. A model for increasing patient safety in the intensive care unit: increasing the implementation rates of proven safety measures. Qual Saf Health Care. 2009;18(1):74-80.

23. Kaba A, Wishart I, Fraser K, et al. Are we at risk of groupthink in our approach to teamwork interventions in health care?. Med Educ. 2016;50(4):400-8.

24. Cook DA, West CP. Perspective: Reconsidering the focus on “outcomes research” in medical education: a cautionary note. Acad Med. 2013;88(2):162-7.

25. Reeves S, Perrier L, Goldman J, et al. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database of Systematic Reviews. 2013.

26. Thistlethwaite JE. Collaboration, cooperation, communication, contact and competencies. GMS J Med Educ. 2016;33(2).

27. Mazmanian PE, Daffron SR, Johnson RE, et al. Information about barriers to planned change: a randomized controlled trial involving continuing medical education lectures and commitment to change. Acad Med. 1998;73(8):882-6.

28. Purkis IE. Commitment for changes: an instrument for evaluating CME courses. J Med Educ. 1982;57(1):61-3.

29. Curry L, Purkis IE. Validity of self-reports of behavior changes by participants after a CME course. J Med Educ. 1986;61(7):579-84.

30. Overton GK, MacVicar R. Requesting a commitment to change: Conditions that produce behavioral or attitudinal commitment. J Contin Educ Health Prof. 2008;28(2):60-6.

31. Shershneva MB, Wang M, Lindeman GC, et al. Commitment to practice change: An evaluator’s perspective. Eval Health Prof. 2010;33(3):256-75.

32. Karg A, Boendermaker PM, Brand PLP, et al. Integrating continuing medical education and faculty development into a single course: Effects on participants’ behaviour. Med Teach. 2013;35(11):e1594-e1597.

33. Adams SG, Pitts J, Wynne J, et al. Effect of a primary care continuing education program on clinical practice of chronic obstructive pulmonary disease: translating theory into practice. Mayo Clin Proc. 2012;87(9):862-70.

34. Wakefield J, Herbert CP, Maclure M, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof. 2003;23(2):81-92.

35. Evans JA, Mazmanian PE, Dow AW, et al. Commitment to change and assessment of confidence: tools to inform the design and evaluation of interprofessional education. J Contin Educ Health Prof. 2014;34(3):155-63.

36. Abrell-Vogel C, Rowold J. Leaders’ commitment to change and their effectiveness in change – a multilevel investigation. J Organ Change Manag. 2014;27(6):900-21.

37. Herscovitch L, Meyer JP. Commitment to organizational change: extension of a three-component model. J Appl Psychol. 2002;87(3):474-87.

38. Armenakis AA, Harris SG, Mossholder KW. Creating readiness for organizational change. Hum Relat. 1993;46(6):681-703.

39. Lowe M, Hebert D, Rappolt S. ABCs of CTCs: an introduction to commitments to change. OT Now. 2009;11(2):20-3.

40. Briter D. Information for the Second National Co-ordinators Meeting – ESS8 translation guidelines. Available at: http://www.europeansocialsurvey.org/docs/round8/methods/ESS8_translation_guidelines.pdf. Accessed September 6, 2017.

41. Mayring P. Qualitative Inhaltsanalyse: Grundlagen und Techniken. 12., überarbeitete Auflage. 2015.

42. Guetterman TC. Descriptions of Sampling Practices Within Five Approaches to Qualitative Research in Education and the Health Sciences. Forum Qual Sozialforschung Forum Qual Soc Res. Available at: http://www.qualitative-research.net/index.php/fqs/article/view/2290. Accessed June 15, 2017.

43. Frambach JM, van der Vleuten CPM, Durning SJ. AM last page. Quality criteria in qualitative and quantitative research. Acad Med. 2013;88(4):552.

44. Cook DA, Kuper A, Hatala R, Ginsburg S. When Assessment Data Are Words: Validity evidence for qualitative educational assessments. Acad Med. 2016;91(10):1359-69.

45. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166.e7-16.

46. Johnson TP. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805.

47. Henderson-King D, Kaleta A. Learning about social diversity: the undergraduate experience and intergroup tolerance. J High Educ. 2000;71(2):142.

48. Kahn JH, Nauta MM. Social-cognitive predictors of first-year college persistence: the importance of proximal assessment. Res High Educ. 2001;42(6):633-52.

49. Umbach PD, Porter SR. How do academic departments impact student satisfaction? Understanding the contextual effects of departments. Res High Educ. 2002;43(2):209-34.

50. Pike GR. The influence of fraternity or sorority membership on students’ college experiences and cognitive development. Res High Educ. 2000;41(1):117-39.

51. Dolcourt JL. Commitment to change: A strategy for promoting educational effectiveness. J Contin Educ Health Prof. 2000;20(3):156-63.