Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Medical students, residents, and practicing physicians increasingly use podcasts for their education, with some preferring podcasts to textbooks and journal articles.^1-6 Podcasts are an easy-to-use and engaging medium that provide learners with broad exposure to educational content and targeted learning opportunities.^7,8 While educational podcasts are used in many medical specialties,^9-13 podcasting is particularly popular in emergency medicine (EM). To date, EM has the largest number of active podcasts, podcast episodes, and hours of content.¹⁴

While some hail the rise of podcasts as the “Netflix” of medical education,¹⁵ others sound notes of caution. In addition to concerns over quality,^16,17 some argue that online educational resources lack the comprehensiveness of textbooks and other commonly used curricula designed to impart the breadth of core-content knowledge dictated by the American Board of Emergency Medicine (ABEM) Model of the Clinical Practice of Emergency Medicine (EM Model).^18,19 Thus, learners who rely primarily on these resources may have knowledge gaps due to over- or under-representation of topics.²⁰

This begs the question, however, of whether the amalgam of EM podcasts should be comprehensive. If podcasts are poised to “fundamentally reimagine medical education” in the next decade,²¹ then we must wrestle with their role and purpose. Although some podcasts have been designed specifically to cover EM core content,^22-24 should we ask podcasts to cover the breadth of an EM education? And if so, can they? It is unclear the extent to which even the core content-focused podcasts cover the breadth of the specialty. Further, within EM core-content podcasts, are they disproportionately covering topics that are probably best taught in a visual (eg, dermatology) or kinesthetic (eg, procedures) format? Or should we ask instead of podcasts that they bring multi-specialty expertise to controversial topics,²⁵ model clinical reasoning and diagnosis,²⁶ analyze recent clinical literature,²⁷ focus on mindset and mental performance,²⁸ tackle gender equity in medicine,²⁹ or carry forward conversations about racism in medicine?^30-32

Given trainees’ current dependence on these resources, and the ways in which many learners listen by absorbing whatever topic is pushed to them in the queue,⁷ our responses to these debates should be informed by a comprehensive understanding of what podcasts are currently covering. This knowledge will begin to help educators make thoughtful recommendations to their learners, help podcast creators see the domains they may have been over- or under-emphasizing, and help scholars make educated arguments about the role and purpose of podcasts in EM education. The only existing studies looking at the comprehensiveness of online educational resources have focused on blogs,^19,20 leaving a gap in our understanding of the content podcasts provide. Thus, we sought to quantify the extent to which podcasts represent the core content of our specialty.

METHODS

Study Design

We performed a retrospective review of all available episodes from 54 EM educational podcasts posted online in 2019. We followed the approach used in a previous study evaluating blog content, and we mapped the content of all included podcast episodes to the 2016 ABEM EM Model content topics.^18,19 Our local institutional review board deemed the study exempt.

Podcast Selection

As there was no updated listing of all EM podcasts, we used a three-step process to identify all EM podcasts that released episodes in 2019. First, we included all accessible podcasts documented in a recent study that relied on a web search to identify podcasts in every medical specialty (including EM).¹⁴ This yielded 32 EM podcasts for inclusion. Next, we searched the Social Media Index (SMI) on March 31, 2020, to find podcasts that we had not yet included. The SMI is a website that lists EM and critical care blogs and podcasts, ranking them based on their impact.^33,34 The SMI yielded 39 additional podcasts. Finally, we emailed a group of seven EM podcast creators for their content expertise. We showed them our existing list and asked them to recommend any podcast we might have missed. They recommended 14 additional podcasts. From a total of 85 podcasts, we then excluded duplicates (those on both the web search study and the SMI), podcasts not focused on EM, and podcasts without any episodes during 2019.

Population Health Research Capsule

What do we already know about this issue?

Emergency medicine (EM) trainees utilize podcasts extensively for learning.

What was the research question?

Does the representation of core content topics in EM podcasts differ significantly from the American Board of Emergency Medicine (ABEM) qualifying exam?

What was the major finding of the study?

Podcasts had an imbalanced yet broad coverage of ABEM EM Model core content subtopics.

How does this improve population health?

Listeners, educators, and podcast creators should be mindful of these gaps and consider which parts of the core curriculum of our specialty are best suited to audio learning.

The 54 remaining podcasts yielded 1,270 unique podcast episodes. We excluded individual episodes if they were a video/vodcast, mailbag-response episode, summarized another podcast, largely advertisements, or focused on personal stories. A total of 77 episodes were excluded, leaving 1,193 for our analysis (Figure 1). The list of included podcasts is available as eAppendix A.

Measurement methods

We used the 2016 ABEM EM Model clinical categories and subtopics to define EM core content.¹⁸ The 2016 model was used because the updated 2019 ABEM EM Model was not publicly available at the start of the study. There are a total of 20 broad categories in the EM Model, which are each divided into subtopics. We defined each category within the EM Model as a core content topic.

One author (GP) evaluated each podcast episode for EM Model, core content coverage by reviewing the podcast episode title and show notes. If show notes were unavailable, the author listened to the podcast. If a podcast covered any EM Model core content topic in an educational manner, we documented that podcast as having covered its corresponding EM Model core content category. We gave podcasts credit for as many EM Model core content topics as were mentioned. In other words, a single podcast could have covered multiple EM Model core content topics.

If a topic was merely mentioned incidentally but not expanded upon in an educational manner, the podcast episode was not credited with covering that topic. For example, if a podcast was discussing a cardiovascular diagnosis and then mentioned that the differential diagnosis included a gastrointestinal topic but did not describe it in any detail, then the podcast would only receive credit for the cardiovascular topic. However, if the gastrointestinal topic was described in further detail, then the podcast episode would be credited with covering that topic as well.

Throughout the coding process a second author (JR) reviewed and discussed any unclear podcast topic coverage until consensus was achieved. A third author (SK) then abstracted data from a random sample of 5% of podcasts and coded each podcast in a similar manner to the primary abstractor. Their inter-rater agreement was excellent (pooled k = 0.84). The inter-rater agreement broken down by each EM Model topic is available in eAppendix B.

Outcomes

For the purpose of comparison given our primary focus on resident education, and in keeping with previous blog-based studies,^19,20 we used the ABEM Qualifying Exam (QE) content specifications as the standard weighting of core content.³⁵ The primary outcome was a description of how ABEM EM Model content topics were represented by EM podcasts from January 1–December 31, 2019 compared to the topic representation of the ABEM QE. Additional outcomes included the number of unique EM podcast episodes that covered each EM Model category.

Data Analysis

We calculated proportional representation of ABEM categories covered by podcasts with associated 95% confidence intervals (CI). We also calculated differences between categorical representation in podcasts and the ABEM QE.

RESULTS

The 54 unique EM podcasts yielded 1,193 included podcast episodes. They covered 2,965 EM Model topics, with a mean of 2.49 (SD 1.77) subtopics per podcast. Of all topics, those most commonly covered were as follows: “other” (29.1%, 95% CI 27.75–30.53); procedures (15.8%, 95% CI 14.54–16.96); and signs and symptoms (12.9%, 95% CI 11.76–14.01). When compared to their relative weight on the ABEM QE, the same topics were the only ones notably over-represented (Table 1). Musculoskeletal (0.7%, 95% CI 0.43–1.05), hematology (0.8%, 95% CI 0.52–1.17), and environmental (0.9%, 95% CI 0.54–1.21) each accounted for less than 1% of all topics covered. The most under-represented topics relative to their weight on the ABEM QE were abdominal/GI (−5.1%), trauma (−4.0%), respiratory (−3.7%), and cardiovascular (−3.5%).

The percentage of all podcast episodes that covered each ABEM topic are shown in Table 2. Almost three-quarters (72.4%, 95% CI 70.8–74.0) of podcast episodes covered other core competencies of the practice of EM, which was 70.4% more than the topic’s relative weight on the ABEM QE. Signs and symptoms and procedures were also covered on more than 32% and 39% of podcast episodes, respectively. While many topics were over-represented relative to the ABEM QE, relatively few were under-represented as a proportion of all podcast episodes, including abdominal/GI (−2.4%); head, eyes, ears, nose, and throat exam (−1.3%); musculoskeletal (−1.2%); and hematology (−0.9%).

DISCUSSION

We found notable differences in how ABEM EM Model content topics were represented by EM podcasts compared to the topic representation of the ABEM Qualifying Exam. Overall, our data suggests imbalanced yet broad coverage of the core content of EM. The finding that, as a percentage of all podcast episodes, very few topics were proportionally under-represented relative to their ABEM QE weight speaks to the thorough coverage of core content that podcasts provided. This is in line with a previous study of blogs that also found five topics under-represented compared with the QE model.²⁰

When considering all topics covered, we found a range of proportional representation relative to the QE from +27% to −5%. The imbalance we found with podcasts is not strikingly dissimilar to previous studies of blog posts.^19,20 The imbalance in podcasts, however, appears different from the imbalance found in the blog studies. While these studies also reported high proportions of signs and symptoms, procedures, and other competencies,¹⁹ we found a strikingly higher level of discussion of other core competencies of the practice of EM on podcasts. The subtopics for other core competencies of the practice of EM are shown in Figure 2.

The finding that other competencies accounted for 29% of all topics covered and was covered on 72% of all podcasts reveals a lot about the role podcasts may be filling in the EM educational milieu. Whereas a leading EM textbook has now relegated chapters on multiculturalism, bioethics, medicolegal issues, wellness, stress, and the impaired physician to its “Bonus Online Content,”³⁶ it appears podcasts are elevating, or at least making more frequent, these discussions. There are several factors likely contributing to this discrepancy. First, it may be a function of time. As the first study highlighting imbalanced coverage of core content studied the landscape in 2013–2014,¹⁹ and the other between 2015–2017,²⁰ it is possible that podcast creators have reacted to these studies and/or current events in creating content that addresses current pressing issues around subjects such as professionalism or systems-based practice.

Similarly, while some topics like “other” may be over-represented because they are covered by recently published, high-impact, primary literature articles, the topics that were under-represented might not have been featured in recent primary literature publications. For example, the EM Model subtopic “status epilepticus” made up 8.5% of the coverage for the nervous system category (data not shown). This is largely due to multiple podcast episodes in 2019 that discussed the ConSEPT and EcLiPSE trials following their publication in Lancet in May 2019.^37,38 We did not, however, find similarly impactful trials published about “inter-departmental and medical staff relations” or “clinical decision support,” two of the most covered topics in the category “other.”

To address the questions raised in the introduction, our data suggests that learners who rely primarily on podcasts may have knowledge gaps due to over- or under-representation of topics. Given that most podcast listeners (including EM residents, attendings, and other healthcare professionals) indicate that they listen to podcasts to learn EM core content,^2,3,5,39 it is possible that some learners may be unaware of the gaps in EM core content coverage. If they do not continue to use a broad range of resources for their education, they could miss out on the full breadth of knowledge necessary to pass the ABEM QE and be prepared for every patient who enters the ED. ⁴⁰ Therefore, listeners, educators, and podcast creators should be aware of these gaps. Several podcasts have, however, been created to specifically cover the breadth of EM core content. Most notably, there is a podcast that covers every chapter of Rosen’s Emergency Medicine textbook.^22,41

The question of whether the amalgam of EM podcasts should be comprehensive is a thornier one. While our data does not definitively answer that question, it does suggest that podcast creators are overwhelmingly bringing what they think is important into their audio conversations. It is likely that resident learners also listen to podcasts, and podcast producers create podcasts for reasons other than learning EM core content. Podcast users have various reasons for listening including to be entertained, to connect with the EM community, to be inspired, and to engage in targeted personalized learning.^5,7 These important reasons for using podcasts do not rely on them to broadly cover all of EM core content. Rather, the benefit of podcasts in the EM educational space may have more to do with the depth that the format enables.

Podcasts offer long-form “deep dives” infused with immersive storytelling that can provide more gravitas than their written counterparts.⁴² The personal nature of audio allows the creator to stimulate the imagination of the listener using richly textured narratives and self-reflexive gestures, thereby influencing both motivation and cognition.^43,44 In contrast to blogs and textbooks, the heightened intonations and subtle adjustments in volume, inflexion, and phrasing allowed by the spoken word can communicate emotions, create a sense of intimacy, and captivate audiences for extended periods.^8,42,45 The format allows for conversations about workplace-based violence, compliance, litigation, reimbursement issues, burn-out, and diversity in ways that just aren’t possible with other educational media (blogs, textbooks, journal articles, etc).

Therefore, future work might focus less on ensuring that every core content subtopic gets covered by a podcast and more on understanding the content that is included in podcasts. It would be fruitful to explore why topics are chosen, how they stimulate emotions and imagination, how to make podcast learning more effective,⁴³ how podcasts impact professionalism and systems-based practice and, ultimately, which parts of the core curriculum of our specialty are best suited to audio learning.

LIMITATIONS

This study has many limitations. First, we evaluated only podcasts released in 2019, and it is possible that including podcast episodes released in prior years would reduce the gaps in coverage of EM core content. Further, impactful studies during the year of release could also have skewed the data. Second, podcasts were evaluated based mostly on the episode title and show notes. We were not able to listen to every podcast episode due to logistical time constraints. It is possible that EM Model core content could have been covered in actual audio podcast recordings without being outlined in the show notes, which could have led us to underestimate the extent of EM core content coverage by podcasts and perhaps points to the need for a standardized template for show notes.

Third, determining what constitutes a meaningful educational discussion of a subtopic was difficult. We attempted to remedy this by discussing all questionable educational points and achieving group consensus, including a large number of podcast episodes, and having a second abstractor review a random percentage of podcasts with excellent inter-rater reliability. Finally, our study was designed to simply evaluate how podcasts covered ABEM EM Model core content. We did not evaluate the depth or quality of coverage, nor did we explore podcast creators’ qualifications or their use of primary literature references. Future studies should continue to explore the issues affecting the quality of EM educational podcasts.^33,46

CONCLUSION

Podcasts had an imbalanced yet broad coverage of ABEM EM Model core content subtopics in 2019. We found imbalances in content representation compared to the weight given in the ABEM Qualifying Exam, with a strikingly higher level of discussion of other core competencies of the practice of emergency medicine on podcasts. Learners, educators, and scholars should be mindful of these gaps and focus future work on exploring how podcasts should best be used in EM education.

Footnotes

Section Editor: Jules Jung, MD, MEd

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Jeff Riddell, MD, Keck School of Medicine of USC, Department of Emergency Medicine, 1200 N State St, Los Angeles, CA 90033. Email: jriddell@usc.edu 1 / 2023; 24:15 – 22

Submission history: Revision received June 14, 2022; Submitted November 17, 2022; Accepted November 15, 2022

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Mallin M, Schlein S, Doctor S, et al. A survey of the current utilization of asynchronous education among emergency medicine residents in the United States. Acad Med. 2014;89(4):598-601.

2. Riddell J, Swaminathan A, Lee M, et al. A survey of emergency medicine residents’ use of educational podcasts. West J Emerg Med. 2017;18(2):229-34.

3. Purdy E, Thoma B, Bednarczyk J, et al. The use of free online educational resources by Canadian emergency medicine residents and program directors. CJEM. 2015;17(2):101-6.

4. Lien K, Chin A, Helman A, et al. A randomized comparative trial of the knowledge retention and usage conditions in undergraduate medical students using podcasts and blog posts. Cureus. 2018;10(1):e2065.

5. Thoma B, Goerzen S, Horeczko T, et al. An international, interprofessional investigation of the self-reported podcast listening habits of emergency clinicians: A METRIQ Study. CJEM. 2020;22(1):112-7.

6. Chorley A, Azzam K, Chan TM. Redesigning continuing professional development: harnessing design thinking to go from needs assessment to mandate. Perspect Med Educ. 2020.

7. Riddell J, Robins L, Brown A, et al. Independent and interwoven: a qualitative exploration of residents’ experiences with educational podcasts. Acad Med. 2020;95(1):89-96.

8. Riddell JC, Robins L, Sherbino J, et al. Residents’ perceptions of effective features of educational podcasts. West J Emerg Med. 2020;22(1):26-32.

9. Ekambaram K, Lamprecht H, Lalloo V, et al. An electronic survey of preferred podcast format and content requirements among trainee emergency medicine specialists in four Southern African universities. Afr J Emerg Med. 2021;11(1):3-9.

10. Cho D, Cosimini M, Espinoza J. Podcasting in medical education: a review of the literature. Korean J Med Educ. 2017;29(4):229-39.

11. Berk J, Trivedi SP, Watto M, et al. Medical education podcasts: Where we are and questions unanswered. J Gen Intern Med. 2020;35(7):2176-8.

12. Patrick MD, Stukus DR, Nuss KE. Using podcasts to deliver pediatric educational content: development and reach of PediaCast CME. Digit Health. 2019;5:2055207619834842.

13. Bernstein E, Bhardwaj N, Pfoh ER, et al. A nationwide survey of educational resource utilization and perception among internal medicine residents. J Gen Intern Med. 2021.

14. Little A, Hampton Z, Gronowski T, et al. Podcasting in medicine: a review of the current content by specialty. Cureus. 2020;12(1):e6726.

15. Tarchichi TR, Szymusiak J. Continuing medical education in an on-demand world: Is podcasting the Netflix of medical education?. Hosp Pediatr. 2019;9(10):818-9.

16. Ostermayer D. Podcasts are great, but…. ACEP Now. 2013. Available at: http://www.acepnow.com/article/podcasts-great/. Accessed Jan 11, 2022.

17. Zanussi L, Paget M, Tworek J, et al. Podcasting in medical education: Can we turn this toy into an effective learning tool?. Adv Health Sci Educ Theory Pract. 2012;17(4):597-600.

18. Counselman FL, Babu K, Edens MA, et al. The 2016 Model of the Clinical Practice of Emergency Medicine. J Emerg Med. 2017;52(6):846-9.

19. Stuntz R, Clontz R. An evaluation of emergency medicine core content covered by free open access medical education resources. Ann Emerg Med. 2016;67(5):649-53.

20. Grock A, Chan W, Aluisio AR, et al. Holes in the FOAM: an analysis of curricular comprehensiveness in online educational resources. AEM Educ Train. 2020;5(3):e10556.

21. Rodman A, Trivedi S. Podcasting: a roadmap to the future of medical education. Semin Nephrol. 2020;40(3):279-83.

22. Thomas A. CRACKCast. 2016. Available at: https://canadiem.org/crackcast/. Accessed June 14, 2022.

23. Herbert M. EM:RAP. Available at: https://www.emrap.org/. Accessed June 14, 2022.

24. Core EM. Podcast. Available at: https://coreem.net/. Accessed June 14, 2022.

25. Helman G. Emergency Medicine Cases. Available at: https://emergencymedicinecases.com/. Accessed June 14, 2022.

26. The Clinical Problem Solvers. Available at: https://clinicalproblemsolving.com/. Accessed June 14, 2022.

27. Swaminathan A, Rezaie S. REBEL CAST. Available at: https://rebelem.com/. Accessed June 14, 2022.

28. Dworkis D. The Emergency Mind. Available at: https://www.emergencymind.com/. Accessed June 14, 2022.

29. Vu K, Woods J. Little Big Med. Available at: http://www.littlebigmed.com/. Accessed June 14, 2022.

30. Mason J, Briggs M. Special Report: Implicit Bias. Available at: https://www.emrap.org/episode/emrap2020june2/specialreport. Accessed June 14, 2022.

31. Frayha N, Allen JS. Race and Medicine – An Introduction. 2022. Available at: https://www.iheart.com/podcast/256-the-osa-insider-30950489/episode/episode-61-race-and-medicine–65458774/. Accessed June 14, 2022.

32. Lindo E. The Praxis. Available at: https://clime.washington.edu/teaching-resources/podcasts/the-praxis/. Accessed June 14, 2022.

33. Thoma B, Chan TM, Kapur P, et al. The social media index as an indicator of quality for emergency medicine blogs: a METRIQ Study. Ann Emerg Med. 2018;72(6):696-702.

34. Thoma B, Sanders JL, Lin M, et al. The social media index: measuring the impact of emergency medicine and critical care websites. West J Emerg Med. 2015;16(2):242-9.

35. Exam Content. Available at: https://www.abem.org/public/become-certified/qualifying-exam/exam-content. Accessed April 15, 2020.

36. Walls R, Hockberger R, Gausche-Hill M, et al. Rosen’s Emergency Medicine: Concepts and Clinical Practice: 2-Volume Set. 2022.

37. Dalziel SR, Borland ML, Furyk J, et al. Levetiracetam versus phenytoin for second-line treatment of convulsive status epilepticus in children (ConSEPT): an open-label, multicentre, randomised controlled trial. Lancet. 2019;393(10186):2135-45.

38. Lyttle MD, Rainford NEA, Gamble C, et al. Levetiracetam versus phenytoin for second-line treatment of paediatric convulsive status epilepticus (EcLiPSE): a multicentre, open-label, randomised trial. Lancet. 2019;393(10186):2125-34.

39. Sherbino J, Upadhye S, Worster A. Self-reported priorities and resources of academic emergency physicians for the maintenance of clinical competence: a pilot study. CJEM. 2009;11(3):230-4.

40. Reiter DA, Lakoff DJ, Trueger NS, et al. Individual interactive instruction: an innovative enhancement to resident education. Ann Emerg Med. 2013;61(1):110-3.

41. Walls R, Hockberger R, Gausche-Hill M. Rosen’s Emergency Medicine – Concepts and Clinical Practice E-Book. 2017.

42. Dowling DO, Miller KJ. Immersive audio storytelling: podcasting and serial documentary in the digital publishing industry. J Radio Audio Media. 2019;26(1):167-84.

43. Weinstock M, Pallaci M, Aluisio AR, et al. Effect of interpolated questions on podcast knowledge acquisition and retention: a double-blind, multicenter, randomized controlled trial. Ann Emerg Med. 2020;76(3):353-61.

44. Rodero E. Stimulating the imagination in a radio story: the role of presentation structure and the degree of involvement of the listener. J Radio Audio Media. 2012;19(1):45-60.

45. Lee MJW, Chan A. Exploring the potential of podcasting to deliver mobile ubiquitous learning in higher education. J Comput High Educ. 2006;18(1):94-115.

46. Thoma B, Chan TM, Paterson QS, et al. Emergency medicine and critical care blogs and podcasts: establishing an international consensus on quality. Ann Emerg Med. 2015;66(4):396-402e4.