Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Interruption of physicians during task performance has been well documented.¹ These distractions to patient care occur more frequently in the emergency department (ED) than in outpatient settings.² Both an increase in time-to-task completion and failure to return to task are correlated with interruptions.³ Previous studies have observed a difference in the ability to manage a simulated patient when comparing between participants’ level of experience.^4,5 Task-switching is identified as a patient care competency in the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System Milestones project.⁶ Simulation has been proposed as a method of assessing these milestones.⁷ We assumed that level of training affects emergency medicine (EM) resident physicians’ ability to execute required patient care tasks and hypothesized that this effect could be measured in the simulation environment when the care of a septic shock patient was interrupted by a second patient with an ST-elevation myocardial infarction (STEMI).

METHODS

We performed a multicenter, prospective, observational cohort study on a convenience sample of residents in their first three years of training at three ACGME-accredited EM residency programs. At each site all eligible residents were enrolled. Participating sites included Loma Linda University Medical Center, UMass Memorial Medical Center and The University of New Mexico Hospital. Sites were selected during a Medical Education Research Certificate Program session at a Council of EM Residency Directors meeting by virtue of having a pre-existing simulation program and faculty members interested in measuring resident task-switching abilities. Institutional review board approval was obtained at each of the participating sites. Data collection occurred in the spring at all sites. Prior to data collection, residents at each of the study sites regularly participated in simulation as part of their residency curriculum, and all sites had previously covered sepsis and STEMI in didactic educational sessions. The three investigators collaborated to develop an immersive simulated patient encounter with the primary objective being the application of task-switching in order to address a time-sensitive distraction while caring for a critically ill patient. Prior to being used with study subjects each investigator piloted the simulated patient encounter and the data collection sheet by observing American Board of EM–eligible attending physicians managing the simulation. Subsequently, issues with the simulated case were resolved by discussion and consensus among the principal investigators. We assessed for baseline differences in electrocardiogram (ECG) interpretation skills by having each subject provide written interpretations to a series of ECGs in the weeks prior to the simulation testing. The STEMI ECG used in the simulation testing was incorporated among the ECGs for the written test, and each subject’s interpretation of the STEMI ECG was recorded for later analysis. This allowed subjects to serve as their own controls with respect to ECG interpretation abilities.

For the simulations, all sites used a high-fidelity mannequin in an environment that closely resembled an ED patient care area by having typical equipment, personnel, and systems found in the ED present for the simulation activity. The three investigators administered the simulation using a script with predetermined verbal responses and physiologic changes to interventions. Each study subject was presented with a 61-year-old diabetic male complaining of cough, fever, and shortness of breath with initial vital signs consistent with sepsis. Chest radiograph and physical exam findings were consistent with right lobar pneumonia. In each case the septic patient became hypotensive immediately after the chest radiograph was interpreted. At this point in the simulation each study subject was given a STEMI ECG of a separate chest pain patient presenting to triage and asked by the ECG technician for an interpretation and next action. Using a standardized data collection sheet with dichotomized responses, data were collected on the subjects’ treatment of the septic shock patient and on recognition and treatment of the STEMI patient. Acceptable interventions for the septic shock patient were defined as administration of appropriate empiric antibiotics, intravenous (IV) fluids, and pressors per early goal-directed therapy standards. Acceptable interventions for the STEMI patient were defined as performing any of the following actions after being given the STEMI ECG: activation of the cardiac catheterization lab, verbalizing their intent to personally evaluate the STEMI patient immediately, or verbalizing a request for another physician to evaluate the STEMI patient immediately.

We used the Cochran-Mantel-Haenszel contingency table analysis to analyze whether the number of years of training predicted the ability of a resident to properly manage sepsis while being presented with the STEMI ECG. We used logistic regression to analyze if the number of years of training predicts the ability of a resident to properly interpret a STEMI ECG while managing a septic shock patient.

RESULTS

Ninety-one subjects participated: 30 post-graduate year (PGY)1s, 32 PGY2s, and 29 PGY3s. Eighty-seven subjects properly managed the patient with septic shock: 90.0% PGY1s, 100% PGY2s, and 96.6% PGY3s (p=0.22). Four subjects did not properly manage the patient with septic shock. One PGY1 did not appropriately order IV fluids and three PGY1s and one PGY3 did not appropriately order antibiotics.

Of the 87 who successfully managed the septic shock, 80 correctly identified STEMI on the simulated STEMI patient (86.7% PGY1s, 96.9% PGY2s, and 93.1% PGY3s; p=0.35). Of the 80 who successfully managed the septic shock patient and correctly identified STEMI on the simulated STEMI patient, 79 provided appropriate interventions for the STEMI patient (73.3% PGY1s, 93.8% PGY2s, 93.8% PGY3s; p=0.07). Both of the subjects who failed to provide appropriate interventions for the simulated STEMI patient correctly identified the STEMI in the simulation session but failed to identify the STEMI in the written test given prior to the simulation session. The Table and Figure provide resident testing characteristics by number of years of training.

DISCUSSION

When we used simulation to measure EM resident physicians’ ability to task-switch between management of septic shock, and responding to a STEMI ECG, we observed no statistically significant difference between groups with different years of EM residency training.

We considered two possible interpretations of this data: (1) EM residents acquire the task-switching skills needed to concurrently execute sepsis and STEMI-related patient care tasks during their first year of EM residency; (2) our simulation scenario lacked the discriminatory power needed to measure task-switching ability differences between different years of EM residency training.

The first proposed interpretation contradicts prior research showing that resident year of training and EM residents’ scores on a multi-tasking assessment tool can explain variability in resident work efficiency.⁸ It also contradicts the authors’ anecdotal experiences after more than 20 collective years of supervising EM residents. Expert performance is mediated in part by highly structured and richly interconnected domain-specific knowledge.⁹ Dynamic decisions are real-time decisions that are interdependent and highly constrained by the decision-making environment.¹⁰

The second proposed interpretation of our data seems more in accord with the authors’ experiences and the aforementioned literature. The EM Milestone Project lists a series of multi-tasking (task-switching) milestones (subcompetency number 8) that residents are expected to achieve during training. Our data failed to show a difference between junior and senior EM residents in regards to ability to task-switch between different patients–a level 2 milestone.⁶ It seems most plausible that our scenario was unable to differentiate resident physicians’ abilities to manage patients amidst distraction because the simulation scenario did not require behaviors described in the level 3–5 milestones.

LIMITATIONS

Our study has several limitations: (1) Inter-rater reliability was not assessed; (2) the study was performed in the simulation environment so the conclusions may not be applicable to the ED; (3) the number of residents who did not successfully perform the critical actions required for the patient in septic shock or the patient with a STEMI was small. Assuming that 60% of PGY1s would manage both patients correctly, our study had an 80% power to detect a 29% difference between PGY1 and PGY2-3s ability to manage both patients correctly. Because 73% of PGY1s managed both cases correctly, more than 100% of PGY2-3s would have to manage the case correctly in order for our study to measure a difference, the impossibility of this highlights the main limitation of our study–that the cases were not difficult enough to differentiate learners’ abilities. It is unknown how many distractions per hour would be needed to see a difference in residents at different levels of training if such a difference exists. Interestingly, while not obtained in the data collection tool, each researcher observed that with the less experienced subjects there appeared to be a trend towards a longer time period between verbalization of the ECG as a STEMI and performing appropriate interventions. It is possible that a time-to-action metric may have captured these differences; this would correlate with multi-tasking milestone level 3. This may be an additional area for future fruitful research as studies in other domains have demonstrated decreased decision-making performance by individuals performing dynamic tasks when under time constraints compared to static task performance under time constraints.^11,12

CONCLUSION

When management of septic shock was interrupted with a STEMI ECG in a simulation environment we observed no significant difference in completion of sepsis therapy or treatment of STEMI when compared between years of training. In our study 73% of PGY1s effectively task-switched between patients, while 94% of PGY2s and PGY3s demonstrated this ability. This observation supports the expectation that most EM residents achieve level 2 of the task-switching milestone by the end of their PGY1 year and nearly all should achieve level 2 by the end of their PGY2 year. This study suggests that future attempts to use simulation to measure differences in EM resident abilities, when comparing years of training, must incorporate skills at or above the level 3 milestone descriptors.

Footnotes

Section Editor: John Ashurst, DO

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Dustin D. Smith, MD, Loma Linda University Health, 11234 Anderson Street Suite A108, Loma Linda, CA 92354. 3 / 2016; 17:149 – 152

Submission history: Revision received July 22, 2015; Submitted November 30, 2015; Accepted December 2, 2015

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

REFERENCES

1. Chisholm CD, Collison EK, Nelson DR, et al. Emergency department workplace interruptions: are emergency physicians “interrupt-driven” and “multitasking”?. Acad Emerg Med. 2000;7(11):1239-43.

2. Chisholm CD, Dornfeld AM, Nelson DR, et al. Work interrupted: a comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38(2):146-51.

3. Westbrook JI, Coiera E, Dunsmuir WT, et al. The impact of interruptions on clinical task completion. Qual Saf Health Care. 2010;19(4):284-9.

4. Waldrop WB, Murray DJ, Boulet JR, et al. Management of anesthesia equipment failure: a simulation-based resident skill assessment. Anesth Analg. 2009;109(2):426-33.

5. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. J Trauma. 2002;52(6):1078-85.

6. The emergency medicine milestone project: a joint Initiative of the ACGME and ABEM. Available at: https://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed Jul 18, 2015.

7. Use of simulation to measure milestones. Available at: http://www.cordem.org/files/DOCUMENTLIBRARY/2013%20AA/2013%20Day%20One/Simulation%20Milestones%20Hart.pdf. Accessed Nov 19, 2015.

8. Ledrick DS, Fisher S, Thompson J, et al. An assessment of emergency medicine residents’ ability to perform in a multitasking environment. Acad Med. 2009;84(9):1289-94.

9. Patel VL, Glasser R, Arocha JF. Cognition and expertise: acquisition of medical competence. Clin Invest Med. 2000;23(4):256-60.

10. Gonzalez C. Task workload and cognitive abilities in dynamic decision making. Hum Factors. 2005;47(1):92-101.

11. Kerstholt JH. Decision making in a dynamic situation: The effect of false alarms and time pressure. J Behav Decision Making. 1995;8:181-200.

12. Payne JW, Bettman JR, Johnson EJ. The adaptive decision maker. 1993.