| Author | Affiliation |
|---|---|
| Steven J. Warrington, MD | Akron General Medical Center, Department of Emergency Medicine, Akron, Ohio |
| Michael S. Beeson, MD, MBA | Akron General Medical Center, Department of Emergency Medicine, Akron, Ohio |
| Frank L. Fire, EMT-P | Akron General Medical Center, Simulation Learning Center, Akron, Ohio |
Introduction Methods Results Discussion Conclusion
ABSTRACT
Introduction: Emergency medicine residents use simulation training for many reasons, such as gaining experience with critically ill patients and becoming familiar with disease processes. Residents frequently criticize simulation training using current high-fidelity mannequins due to the poor quality of physical exam findings present, such as auscultatory findings, as it may lead them down an alternate diagnostic or therapeutic pathway. Recently wireless remote programmed stethoscopes (simulation stethoscopes) have been developed that allow wireless transmission of any sound to a stethoscope receiver, which improves the fidelity of a physical examination and the simulation case.
Methods: Following institutional review committee approval, 14 PGY1-3 emergency medicine residents were assessed during 2 simulation-based cases using pre-defined scoring anchors on multiple actions, such as communication skills and treatment decisions (Appendix 1). Each case involved a patient presenting with dyspnea requiring management based off physical examination findings. One case was a patient with exacerbation of heart failure, while the other was a patient with a tension pneumothorax. Each resident was randomized into a case associated with the simulation stethoscope. Following the cases residents were asked to fill out an evaluation questionnaire.
Results: Residents perceived the most realistic physical exam findings on those associated with the case using the simulation stethoscope (13/14, 93%). Residents also preferred the simulation stethoscope as an adjunct to the case (13/14, 93%), and they rated the simulation stethoscope case to have significantly more realistic auscultatory findings (4.4/5 vs. 3.0/5 difference of means 1.4, p=0.0007). Average scores of residents were significantly better in the simulation stethoscope-associated case (2.5/3 vs. 2.3/3 difference of means 0.2, p=0.04). There was no considerable difference in the total time taken per case.
Conclusion: A simulation stethoscope may be a useful adjunct to current emergency medicine simulation-based training. Residents both preferred the use of the simulation stethoscope and perceived physical exam findings to be more realistic, leading to improved fidelity. Potential sources of bias include the small population, narrow scoring range, and the lack of blinding. Further research, focusing on use for resident assessment and clinical significance with a larger population and blinding of graders, is needed.
INTRODUCTION
A variety of medical simulation tools, such as task trainers, computer-based systems, virtual reality and haptic systems, as well as simulated patients and environments, exist as adjuncts to simulation-based medical education.1–4 High-fidelity mannequins frequently serve as adjuncts in case-based simulation as they have the capability to simulate vital signs and physical exam findings while providing a patient to interact with. Anecdotally the quality of the physical examination findings present on these mannequins frequently criticized. The auscultatory findings are criticized because of their importance in the role of diagnosis and treatment-related decisions. Criticisms are often directed at mechanical background noise present, as well as the difficulty of interpreting findings due to the radiation of sounds (such as hearing diffuse crackles instead of basilar crackles on a respiratory examination).
Recently a simulation stethoscope has been developed to serve as an adjunct to simulation-based learning.1 The simulation stethoscope consists of a receiver and a transmitter. The receiver appears similar to a stethoscope with the addition of a small black box on the tubing, while the transmitter is handheld black box with 4 buttons and a switch. An SD card used for storage of sounds and is capable of holding 12 unique sounds per card. This system allows for an individual to hear different findings based on the location auscultated over the course of a case without picking up background noise.
The objective of the study was to assess the utility of a simulation stethoscope as an adjunct in emergency medicine resident’s simulation-based training using high-fidelity mannequins. Specifically, we wanted to determine its utility perceived fidelity of physical examination findings, resident performance, and resident preference.
METHODS
Study design
This was a prospective, randomized, non-blinded, crossover observational study comparing 2 case-based scenarios with and without the use of a simulation stethoscope (Ventriloscope®, Lecat’s Ventriloscope, Canton, OH). The study was approved by the local institutional review committee. The study subjects were PGY 1-3 emergency medicine residents who volunteered to participate in 2 simulation-based scenarios between June and July of 2011. Case-based scenarios were run using high-fidelity mannequins (MetiMan, CAE Healthcare, formerly METI of Sarasota, Fl.). Subjects were assigned a unique ID number that was kept confidential and used for data analysis only. Performance in the cases was not used for longitudinal assessment of subjects, and they were notified of this prior to participation. Randomization to a case associated with the simulation stethscope occurred by an even/odd rotating fashion based off of the order in which subjects signed up. Randomization of case order (simulation stethoscope-associated case first or second) was through an even/odd rotating fashion based off time slot assigned.
Statistics
Data were collected from the scoring sheets based off scoring anchors (Appendix 1) and from the evaluation questionnaire (Appendix 2) and entered into an Excel spreadsheet. Subjects were given 1 point for “Needs Improvement,” 2 points for “Meets Expectations,” and 3 points for “Above Expectations.” If a point on the scoring anchor was not applicable it was left out when calculating the average score (i.e., if a patient did not deteriorate into pulseless electrical activity, then the recognition/management of that rhythm was not applicable).
We used data from scoring anchors to compare overall performance score and time taken per case. We averaged scores for each case and calculated a difference of means for the overall average score (average of subject’s combined scores) between the cases in which a simulation stethoscope was used and those in which it was not. We used a 2-tailed paired student’s T-test to compare subject’s average scores. Total time taken per case was also compared with a difference of means as well as a 2-tailed paired student’s T-test.
We used data from the evaluation questionnaire to compare subject’s perceived realism of a case and associated physical exam findings, subject’s preference for the use of a simulation stethoscope, as well as subject’s confidence in diagnosis and treatment. Using the 5-point Likert scale, we used a 2-tailed paired student’s T-test to evaluate the difference in how realistic subjects perceived the auscultatory findings. The percent of subjects who ranked the simulation stethoscope-associated case more realistic was calculated, as was the perecent of subjects who preferred the use of one. We used a 2-tailed paired student’s T-test to compare results from the 5-point Likert scale assessing the subject’s confidence in diagnosis and treatment of heart failure exacerbations and tension pneumothoraces.
Intervention
Subjects were randomized into two groups, with each group completing two cases. The first group used a simulation stethoscope for case #1 and the high-fidelity mannequin’s natural auscultatory findings for case #2, while the second group experienced the reverse. We used pre-determined scoring anchors (Appendix 1) for the evaluation of their performance specific to each simulation-based case. The scoring anchors involved assessment on critical actions, time taken, and communication skills. Following the completion of both cases residents were given an evaluation questionnaire (Appendix 2) that was partially based off a previously validated questionnaire for physical exam.5 A debriefing session followed the questionnaire regarding critical actions and instructional points involved in both cases. Prompting was given during each case at pre-determined intervals. The first case was based on diagnosing a patient in decompensated heart failure from history, a pulmonary exam with bilateral basilar crackles, and corresponding vital signs. The second case was based on diagnosing a patient with a tension pneumothorax in extremis from history, a pulmonary exam with decreased breath sounds unilaterally, and corresponding vital signs. Each case required diagnosis from history and physical examination as subjects were unable to recieve test results (such as radiologic studies or blood work) until treatment was initiated.
RESULTS
Subjects average scores were significantly better on the case associated with the use of the simulation stethoscope, 2.5/3 compared to 2.3/3 (difference of means 0.2, p=0.04). Both groups, however, scored above “meets expectations.” Taking into consideration the narrow score range (1–3), as well as a small sample, it was not possible to determine if this translates into clinical significance. The simulation stethoscope-associated case was found to have significantly more realistic auscultatory findings, 4.4/5 as compared to 3.0/5 without (difference of means 1.4, p=0.0007). Total case times did not differ considerably between the cases with or without the use of the simulation stethoscope, 28:49 with the simulation stethoscope and 30:02 without (difference of means 1:13, p=0.8). Subjects noted that the physical exam findings were most realistic in cases associated with the simulation stethoscope in 13/14 (93%) and that their preference was for the use of the simulation stethoscope as an adjunct to simulation in 13/14 (93%). There was no difference of either group’s confidence in their diagnostic or treatment skills in heart failure exacerbations (p=0.24 and p=0.55 respectively) or tension pneumothoraces (p=1 and p=1 respectively).
DISCUSSION
The techniques and adjuncts of simulation-based medical education are broad and range from recorded sounds to standardized simulated patient encounters.1–4 Simulation training has been studied in many fields, including anesthesia, surgery, obstetrics and gynecology, internal medicine, and emergency medicine, and is used to teach a variety of skills, including physical diagnosis, communication, and procedures.2,3,6,7 Anecdotally there are numerous complaints with auscultatory findings present in current simulation-based training. In standardized patients, unless the patient has the finding present already, it has been impossible to have the actual examination correlate with the expected examination. Current high-fidelity mannequins are critiqued secondary to mechanical background noise when auscultating, and for the non-specific locations of auscultatory findings
LIMITATIONS
Limitations in this study included the small population size, the lack of blinding, and the narrow score range. Specifically, the use of the narrow score range (subjects either received a 1, 2, or 3 for a score on each datapoint) makes it difficult to determine when clinical significance would be present. In addition, the authors were responsible for scoring each case and we attempted to prevent bias by using predetermined and well-defined scoring anchors for each case. While a reference anchor was used throughout each scenario to ensure accuracy, there still exists a potential bias. The other primary potential bias is due to the inability to have either party blinded as the authors were responsible for controlling the simulation stethoscope, which has a black box on it differentiating it from a regular stethoscope.
CONCLUSION
We believe that a simulation stethoscope represents a useful adjunct in emergency medicine case-based simulation on high-fidelity mannequins. The simulation stethoscope was an easy device to learn and use, and did not significantly alter the amount of time required for each case. Subjects preferred use of the simulation stethoscope, and associated it with more realistic findings. Scores were significantly better with the use of the simulation stethoscope; however, the impact on clinical significance is yet to be determined.
Footnotes
Supervising Section Editor: Laleh Gharahbaghian, MD
Submission history: Submitted April 27, 2012; Revisions received, June 20, 2012; Accepted August 13, 2012
Full text available through open access at http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.2012.8.12519
Address for Correspondence: Steven Jay Warrington, MD, Akron General Medical Center, Department of Emergency Medicine, 400
Wabash Avenue, Akron, OH 44307. Email: swarring@neomed.edu
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
REFERENCES
1. Castilano A, Haller N, Goliath C. The Ventriloscope®: ‘am I hearing things?’ Med Teach.2009;31:e97–101. et al. [PubMed]
2. Abrahamson S, Denson J, Wolf R. Effectiveness of a simulator in training anesthesiology residents.Acad Med. 1969;44:515–519. [PubMed]
3. Issenberg S, Petrusa E, McGaghie W. Effectiveness of a computer-based system to teach bedside cardiology. Acad Med. 1999;74:S93–95. et al. [PubMed]
4. Maran N, Glavin R. Low- to high-fidelity simulation – a continuum of medical education? Med Educ. 2003;37:22–28. [PubMed]
5. Vivekananda-Schmidt P, Lewis M, Hassell A. Validation of MSAT: an instrument to measure medical students’ self-asssessed confidence in musculoskeletal examination skills. Med Educ. 2007;41:402–410.et al. [PubMed]
6. Wayne D, Butter J, Siddall V. Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial. Teach Learn Med. 2010;17:202–208. et al. [PubMed]
7. Bond W, Deitrick L, Arnold D. Using simulation to instruct emergency medicine residents in cognitive forcing strategies. Acad Med. 2004;29:438–446. et al. [PubMed]