Author	Affiliation
Leonardo Aliaga, MD	University of California – Davis, Department of Emergency Medicine, Sacramento, California
Samuel Owen Clarke, MD, MAS	University of California – Davis, Department of Emergency Medicine, Sacramento, California

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Emergency physicians (EPs) must be able to identify life-threatening and time-sensitive findings on head computed tomography (CT) that require immediate action, often before a radiologist’s report is available.^1,2 Learning to accurately interpret head CTs requires detailed instruction and repeated exposure to both normal studies and diverse pathologic findings, elements that are difficult to achieve in a time-restricted setting such as emergency medicine (EM) residency didactic conference. Perhaps unsurprisingly, head CT interpretation concordance between EPs and radiologists has been shown to be poor across a variety of practice settings.^1-7

Prior studies addressed this knowledge gap using one-time didactic lectures or instructional videos.^8-12 However, demonstrating CT findings on single images poorly represents the cognitive work of identifying these findings in clinical practice.¹³ Even if a lecturer “scrolls” through a CT, learners are unable to actively engage with the images. While we know that passive learning methods lead to poorer retention,¹⁴ active learning curricula for radiographic interpretation have remained elusive. Despite a body of evidence supporting the benefits of active learning,^15-22 time and resource barriers exist to implementing these methods into residency didactic curricula.^18-24

Pacsbin (Orion Medical Technologies, Baltimore, MD) is a web-based radiology picture archiving and communication system (PACS) that provides learners with a familiar platform to scroll through CTs, simulating the way they engage with imaging studies in clinical practice and providing a potential vehicle for active learning (www.pacsbin.com).^25-29 While practice cases on various PACS platforms have supplemented existing curricula,^27-29 no prior study has used this technology to directly compare active and passive learning strategies.

To evaluate this approach, we created an active learning-based curriculum using Pacsbin for teaching head CT interpretation to EM residents. Our objective was to compare the effectiveness of this active learning approach to a passive learning strategy within our didactic conference while maintaining resource neutrality in terms of time and access to instruction. We hypothesized that EM residents who learned head CT interpretation using our active learning curriculum would demonstrate greater diagnostic accuracy on a head CT interpretation test.

METHODS

Study Population and Design

This study was conducted at the University of California, Davis EM residency program and approved by our institutional review board. We used a convenience sample of first-, second-, and third-year residents at our institution. As this was a pilot study, we did not perform an a priori power calculation. After consent, residents took a pre-test of head CT interpretation and were subsequently randomized to an active learning cohort (ALC) or passive learning cohort (PLC). Three educational sessions (on intracranial hemorrhage, acute ischemic stroke, and increased intracranial pressure) were delivered monthly over a three-month period via Zoom conference (Zoom Video Communications, Inc., San Jose, CA). The ALC convened in a virtual breakout room where residents accessed head CT teaching cases on Pascbin using their individual computers (cases in Supplement). Pacsbin simulates a radiology PACS, allowing learners to scroll through CTs (including axial/coronal/sagittal views), annotate images, adjust brightness and contrast, and access built-in links to instructional diagrams. Residents scrolled through head CTs guided by teaching points built into each case. After finishing the cases the ALC had a 10-minute, live question-and-answer (Q&A) session led by one of the investigators.

The PLC watched pre-recorded instructional videos in a live, synchronous fashion via video conference which was immediately followed by a 10-minute, live Q&A session. These videos scrolled through the same cases and explained the same teaching points the ALC received through Pacsbin. We controlled the length of these educational sessions using virtual breakout rooms with a pre-set time limit of 60 minutes. Residents took a post-test one month after the last session using the same questions on the pre-test. We tested the data for normality and analyzed pre- and post-test scores using paired and unpaired t-tests.

Head Computed Tomography Test and Active Learning Cases

We created a head CT interpretation test on Pacsbin and pilot tested it with three EM education and simulation faculty to collect content and response process validity evidence for the instrument. All faculty agreed the test cases represented critical knowledge and skills needed in EM and noted there was an appropriate range of difficulty. Faculty agreed the image quality was essentially identical to what we would encounter on our institution’s radiology PACS. Feedback from faculty was used to revise the test. We reviewed and modified answer choices across all the questions to reduce potential construct-irrelevant variance from learners inferring correct/incorrect choices based on where they appeared.

The test included cases with obvious pathologic findings as positive controls (e.g., classic “star” pattern of acute subarachnoid hemorrhage filling the basal cisterns, large acute subdural hematoma) and normal studies as negative controls. We built active learning modules on Pacsbin. Each module consisted of six to eight cases illustrating critical findings relevant to EM practice and normal comparisons. The modules guided the learner to incorporate predefined heuristics for identifying critical findings. The videos watched by the PLC presented the same heuristics and cases (test, modules, and videos in Supplement). The primary author completed four years of neurosurgery residency before switching to EM and used his expertise in head CT interpretation to develop the learning modules and heuristics.

RESULTS

Forty-two residents took the pre-test. Twelve residents in the ALC and eight residents in the PLC completed all three educational sessions and took the post-test. Test score distributions passed the Shapiro-Wilk normality test. Mean pre-test scores and 95% confidence intervals (CI) were as follows: for the ALC 43.8% (CI: 38.0–49.5), and for the PLC 41.7% (CI: 36.5–46.8) (P = 0.62). Mean post-test scores and 95% CI were as follows: for the ALC 59.0% (CI: 53.3–64.8), and for the PLC 45.3% (CI: 38.2–52.5) (P = 0.009) (Figure). The score increase for the ALC was statistically significant using a paired t-test (P <0.001); however, it was not for the PLC (P = 0.29). The effect size was large when comparing the ALC and PLC post-test scores (Cohen’s d = 1.34).

DISCUSSION

While EPs do not need the same level of mastery in head CT interpretation as radiologists, they must be able to identify critical and time-sensitive findings, often before a radiologist’s report is available.^1,2 This is particularly true in practice settings that do not have attending radiologists in house at all times.^10,30,31 Nonetheless, the skill of head CT interpretation exists in a border region of knowledge domains between clinical specialties. As educators, this forces us to consider the complex issue that teaching one topic to sufficient depth can come at the expense of time for other topics in residency education. In this study, we designed an evidence-based and learner-centered solution for teaching head CT interpretation and found this was achievable within the time and resource constraints of our residency’s didactic conference curriculum.

Active learning is rooted in constructivist learning theory and posits that learners build knowledge frameworks through active engagement with learning material.³² Despite extensive evidence supporting active learning approaches, passive learning remains the dominant modality in most educational settings.^14,19,21,22 Avoidance of active learning may be related to the perception that it requires extra time or resources; however, our intervention fit into an existing residency didactic conference schedule without requiring extra time either during or outside the session.

Our novel curriculum created active engagement by making learners scroll through head CT images themselves, setting the conditions for active learning and accurately reproducing the cognitive work used to identify these findings in clinical practice. This intervention embedded the didactic content into a Pascbin and incorporated all the skills needed to correctly interpret a non-contrast head CT (e.g., manipulating window presets, brightness and contrast, and identifying pathologic findings in relation to key anatomic structures). This forced learners to interact with the didactic content in a way that is lost with lectures or videos. This approach to teaching head CT interpretation has not been previously described in the literature and represents an important step forward from the historical reliance on passive learning strategies to address this key content area.

We designed this study to specifically isolate the influence of passive vs active engagement with the learning material. Both the ALC and PLC were exposed to identical cases, embedded prompts, and questions. The two groups received identical amounts of time to review the material using Zoom breakout rooms to control length of exposure, and both received the same amount of time to ask clarifying questions. The crucial difference between the two groups was how they engaged with the learning material. The PLC watched videos where an instructor scrolled through cases whereas the ALC had to scroll through cases themselves. Given that all other learning conditions were controlled for, we hypothesize that making residents in the ALC search for and identify key findings on their own may have facilitated deeper knowledge encoding and greater improvement in diagnostic accuracy.

It is notable that the PLC did not significantly improve despite receiving the same content, teaching points, and heuristics. The videos watched by the PLC were made to be engaging, clear, and easy to follow. It is possible the videos’ cognitive fluency produced an illusion of learning and robbed the viewers of effortful learning, leading to poorer retention.^16,33-35 In contrast, the ALC had to scroll through images and search for findings, which likely contributed to some degree of effortful learning. We realize that the three 60-minute Pacsbin sessions given to the ALC were insufficient to ensure complete understanding of all this content, despite showing improved performance compared to the PLC. However, this work serves as a proof of concept and a potential springboard for spaced repetition. After residents complete the initial modules, single cases can be delivered synchronously or asynchronously and completed in a shorter time frame. We suspect these subsequent cases might serve as booster inoculations, strengthening knowledge encoding and potentially improving scores on future tests.

LIMITATIONS

This pilot study is not without limitations. We collected content and response process validity evidence for our head CT interpretation test; however, this evidence relied on expert (i.e., attending-level) opinion and might have been strengthened by incorporating junior learners. The intervention was conducted without an a priori power calculation and used a convenience sample of residents at a single EM residency program, limiting its generalizability. We used pre-recorded instructional videos as our passive learning control, which differ from traditional lectures and limit our results’ generalizability. However, the videos allowed us to standardize the control intervention while providing some resemblance to lectures by being shown in live, synchronous fashion followed by a Q&A session. Our study also suffered from attrition, with 22 residents missing one or more educational sessions due to schedule conflicts. We nonetheless found a large effect size despite a relatively small sample, highlighting the potential impact of our intervention.

CONCLUSION

Our active learning head CT curriculum using Pascbin led to greater diagnostic accuracy when compared to a typical passive learning strategy. We achieved this superior outcome while maintaining resource neutrality in terms of time and access to instruction. We believe this study adds to the landscape of active learning literature by demonstrating an effective way to strengthen radiology curricula in EM residency programs.

Footnotes

Section Editor: Jeffrey Love, MD and Danielle Hart, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Leonardo Aliaga, MD, Stanford University, Department of Emergency Medicine, 900 Welch Road, Suite 350, Palo Alto, CA 94304. Email: aliagal@stanford.edu. 1 / 2022; 23:47 – 51

Submission history: Revision received June 14, 2021; Submitted October 13, 2021; Accepted October 12, 2021

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Alfaro D, Levitt MA, English DK, et al. Accuracy of interpretation of cranial computed tomography scans in an emergency medicine residency program. Ann Emerg Med. 1995;25(2):169-74.

2. Evans LR, Fitzgerald MC, Mitra B, et al. Emergency department interpretation of CT of the brain: a systematic review. Postgrad Med J. 2017;93(1102):454-9.

3. Arendts G, Manovel A, Chai A. Cranial CT interpretation by senior emergency department staff. Australas Radiol. 2003;47(4):368-74.

4. Mucci B, Brett C, Huntley LS, et al. Cranial computed tomography in trauma: the accuracy of interpretation by staff in the emergency department. Emerg Med J. 2005;22(8):538-40.

5. Khoo NC, Duffy M. “Out of hours” non-contrast head CT scan interpretation by senior emergency department medical staff. EMA – Emerg Med Australas. 2007;19(2):122-8.

6. Boyle A, Staniciu D, Lewis S, et al. Can middle grade and consultant emergency physicians accurately interpret computed tomography scans performed for head trauma? Cross-sectional study. Emerg Med J. 2009;26(8):583-5.

7. Gallagher FA, Tay KY, Vowler SL, et al. Comparing the accuracy of initial head CT reporting by radiologists, radiology trainees, neuroradiographers and emergency doctors. Br J Radiol. 2011;84(1007):1040-5.

8. Levitt MA, Dawkins R, Williams V, et al. Abbreviated educational session improves cranial computed tomography scan interpretations by emergency physicians. Ann Emerg Med. 1997;30(5):616-21.

9. Perron AD, Huff JS, Ullrich CG, et al. A multicenter study to improve emergency medicine residents’ recognition of intracranial emergencies on computed tomography. Ann Emerg Med. 1998;32(5):554-62.

10. Jamal K, Mandel L, Jamal L, et al. “Out of hours” adult CT head interpretation by senior emergency department staff following an intensive teaching session: A prospective blinded pilot study of 405 patients. Emerg Med J. 2014;31(6):467-70.

11. Minkowitz S, Leeman K, Giambrone AE, et al. Emergency radiology “boot camp”: educating emergency medicine residents using e-learning radiology modules. AEM Educ Train. 2017;1(1):43-7.

12. Pourmand A, Woodward C, Shokoohi H, et al. Impact of asynchronous training on radiology learning curve among emergency medicine residents and clerkship students. Perm J. 2018;22:1-5.

13. Gaddam DS, Awan OA. Back to the future: shortcomings of an archaic model for radiology lectures. RadioGraphics. 2020;40(4):1196-7.

14. Winter RO, Picciano A, Birnberg B, et al. Resident knowledge acquisition during a block conference series. Fam Med. 2007;39(7):498-503.

15. Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410-5.

16. Deslauriers L, McCarty LS, Miller K, et al. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc Natl Acad Sci U S A. 2019;116(39):19251-7.

17. Ozuah PO, Curtis J, Stein REK. Impact of problem-based learning on residents’ self-directed learning. Arch Pediatr Adolesc Med. 2001;155(6):669-72.

18. Thomas KG, Thomas MR, York EB, et al. Teaching evidence-based medicine to internal medicine residents: the efficacy of conferences versus small-group discussion. Teach Learn Med. 2005;17(2):130-5.

19. Graffam B. Active learning in medical education: strategies for beginning implementation. Med Teach. 2007;29(1):38-42.

20. Shellenberger S, Seale JP, Harris DL, et al. Applying team-based learning in primary care residency programs to increase patient alcohol screenings and brief interventions. Acad Med. 2009;84(3):340-6.

21. Batalden MK, Warm EJ, Logio LS. Beyond a curricular design of convenience: replacing the noon conference with an academic half day in three internal medicine residency programs. Acad Med. 2013;88(5):644-51.

22. Wolff M, Wagner MJ, Poznanski S, et al. Not another boring lecture: engaging learners with active learning techniques. J Emerg Med. 2015;48(1):85-93.

23. Sawatsky AP, Zickmund SL, Berlacher K, et al. Understanding the challenges to facilitating active learning in the resident conferences: A qualitative study of internal medicine faculty and resident perspectives. Med Educ Online. 2015;20(1):27289.

24. Kim AM, Speed CJ, Macaulay JO. Barriers and strategies: implementing active learning in biomedical science lectures. Biochem Mol Biol Educ. 2019;47(1):29-40.

25. Hostetter J, Khanna N, Mandell JC. Integration of a zero-footprint cloud-based picture archiving and communication system with customizable forms for radiology research and education. Acad Radiol. 2018;25(6):811-8.

26. Berge M, Dalane S, Savioli S, et al. Interactive radiology case conference utilizing flipped classroom. American College of Radiology. 2018.

27. Morin CE, Hostetter JM, Jeudy J, et al. Spaced radiology: encouraging durable memory using spaced testing in pediatric radiology. Pediatr Radiol. 2019;49(8):990-9.

28. Sajedi P, Salamon N, Hostetter J, et al. Reshaping radiology precall preparation: integrating a cloud-based PACS viewer Into a flipped classroom model. Curr Probl Diagn Radiol. 2019;48(5):441-7.

29. Gomez E, Azadi J, Magid D. Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic. Acad Radiol. 2020;27(9):1285-90.

30. Saketkhoo DD, Bhargavan M, Sunshine JH, et al. Emergency department image interpretation services at private community hospitals. Radiology. 2004;231(1):190-7.

31. DeFlorio R, Coughlin B, Coughlin R, et al. Process modification and emergency department radiology service. Emerg Radiol. 2008;15(6):405-12.

32. Bransford JD, Brown AL, Cocking RR. How People Learn: Brain, Mind, Experience, and School. 2000.

33. Oppenheimer DM. The secret life of fluency. Trends Cogn Sci. 2008;12(6):237-41.

34. Carpenter SK, Wilford MM, Kornell N, et al. Appearances can be deceiving: Instructor fluency increases perceptions of learning without increasing actual learning. Psychon Bull Rev. 2013;20(6):1350-6.

35. Gooding HC, Mann K, Armstrong E. Twelve tips for applying the science of learning to health professions education. Med Teach. 2017;39(1):26-31.