Author	Affiliation
Michael Kaduce, MPS	Falck Health Institute, Orange, California
Antonio Fernandez, PhD	ESO, Austin, Texas
Scott Bourn, PhD	ESO, Austin, Texas
Dustin Calhoun, MD	University of Cincinnati, Cincinnati, Ohio
Jefferson Williams, MD	Wake County Emergency Medical Services, Wake County, North Carolina
Mallory DeLuca, BS, NRP	Wake County Emergency Medical Services, Wake County, North Carolina
Heidi Abraham, MD	Austin-Travis County Emergency Medical Services, Austin, Texas
Kevin Uhl,	Cincinnati Fire Department, Cincinnati, Ohio
Brian Bregenzer, LP	Austin-Travis County Emergency Medical Services, Austin, Texas
Baxter Larmon, PhD	University of California Los Angeles, David Geffen School of Medicine, Los Angeles, California
Remle P. Crowe, PhD	ESO, Austin, Texas
Alison Treichel, MPH	ESO, Austin, Texas
J. Brent Myers, MD	ESO, Austin, Texas

Background
Methods
Results
Discussion
Limitations
Conclusion

BACKGROUND

Emergency medical services (EMS) providers provide the majority of prehospital medical care in the United States and serve as a crucial component of the nation’s healthcare delivery system. For EMS providers to maintain their license, every state has unique continuing education (CE) requirements, typically in the fashion of required hours, for licensure renewal. The National Registry of Emergency Medical Technicians also maintains an hour and topic requirement for recertification. This hour model has been used to ensure the continual competency of EMS providers,
¹
as it has been historically accepted that competency can be maintained through CE hour requirements despite a lack of empirical evidence.
²

In continuing medical education (CME), a broader concept of continued professional development (CPD), is emerging. Continued professional development includes education focused on problem identification and solution development, allowing the healthcare professional to tailor the learning process to their individual needs.
³
The process of CPD recognizes a one-size-fits-all approach, which is not specific enough for each learner. The Institute of Medicine recommends a CPD system that includes patient-centered care, interprofessional teamwork, quality improvement application, and clinical outcome data utilization for individual, team, and institutional assessment.
<sup>2

,

4</sup>
Feedback has also been demonstrated to improve system performance and patient outcomes.
⁵
Very public voices, including the EMS Agenda 2050, have included calls for EMS systems and providers to receive feedback, including patient outcomes, in real time, as a means for continuous quality improvement, thus moving toward a CPD approach to competency.
⁶
The National Association of EMS Physicians has also called for continual monitoring of airway performance data and its use in the continued credentialing process and quality management activities with large-scale bidirectional information shared between EMS and receiving facilities in their position paper on airway management and training.
⁷

Despite this desired transition to CPD, providing patient-specific outcomes to EMS providers has long been a challenge. Bidirectional data sharing between EMS and hospitals has raised concerns about patient privacy and technological challenges
<sup>8

,

9</sup>
Fortunately, this trend is improving, based on the provision of outcome elements as a part of the National Emergency Medical Services Information System dataset and the clarification that such data-sharing is consistent with the Health Insurance Portability and Accountability Act guidelines.
<sup>10

–

12</sup>
When surveyed, EMS providers reported a desire for patient-specific outcomes. They even reported using informal networks or going around the system to obtain patient outcome information, assisting them to develop clinical skills.
⁷
This lack of insight, specifically centered around patient outcomes and EMS provider diagnosis accuracy, has been reported to impact provider mental health.
¹³

The EMS provider is interested in including patient outcome data as a means of professional development similar to their colleagues in medicine.
<sup>14

,

15</sup>
Physicians’ continued medical education has included electronic health records (EHR) to assess the quality of care and has been used to suggest areas for improvement through the use of CME.
¹⁶
Similarly, medical databases have found ways to help physicians receive CME with routine clinical questions and problem-solving in their daily practice.
¹⁷
Graduate medical education also envisions a system in which patient health records and outcome data can be incorporated into the curriculum. This framework includes mentors or instructors using the outcome data to assess, supervise, and teach, creating a mature, professional community where everyone receives and provides feedback.
¹⁸
The Accreditation Council for Graduate Medical Education similarly requires programs to connect resident-physician education to patient outcomes.
¹⁹

Using patient outcome data as feedback continues to be called for and, on a small scale, has been demonstrated to be an effective part of CPD programs for EMS providers. In fact, the EHR serves as a valuable resource for CPD in providing patient-specific outcomes.
²⁰
When evaluating CE in pediatric emergencies, brief and frequent CE programs are recommended as a means to provide repetition with immediate feedback and error correction.
²¹
While feedback following a call is outside the proposed theoretical framework for clinical judgment in EMS, it has been noted as important in the development of EMS providers and allows for improvement in performance.
²²
Post-resuscitation provider feedback for patients who have suffered cardiac arrest and heart attacks has led to improvements in time and treatment.
<sup>23

,

24</sup>
Sammuel and colleagues conducted a scoping review of the effects of CPD on healthcare professionals’ performance and patient outcomes and were also able to demonstrate changes in providers’ behavior and patient outcomes.
²⁵

Despite calls from leading national EMS organizations and other healthcare professions to incorporate patient outcomes into CE, little is known about EMS providers’ perceptions of automated patient outcome data nor its use as microlearning to obtain continuing education. Our primary objective in this study was to evaluate EMS providers’ perceptions of the utility of automated hospital outcome data for professional development. Secondarily, we evaluated their participation in outcome review as a means of microlearning to obtain CE.

METHODS

Study Design and Population

In this mixed-methods study we ued quantitative and qualitative methods to understand EMS provider perceptions regarding the use of an automated system enabling them to obtain Commission on Accreditation for Pre-Hospital Continuing Education (CAPCE)-approved CE by reviewing patient outcomes and completing associated learning modules. The CAPCE is an accrediting body charged with the review and accreditation of EMS CE.
²⁶
The study was conducted from October–December 2023 in three high-volume urban EMS systems. The EMS providers were certified at the emergency medical technician through the paramedic level and were continuously provided 100% of their required CE hours through their employer.

At the beginning of the study, EMS providers in each system received a prompt inviting them to anonymously participate in this pilot study when they logged into their EHR (ESO, Austin TX). Each participating agency uses a system that allows EMS providers to automatically receive outcome data from the hospital, specific to patients they encountered in the prehospital setting. All CE activities were completed in a single online learning platform.

For those who agreed to participate in the first phase of the pilot study, participants were asked to voluntarily complete a pre-survey. They were also informed that they would be eligible to receive CE hours, approved by CAPCE, upon completion of outcome reviews, associated videos, and outcome review assessments. A prompt to review outcomes as part of this study was received by participants when they logged into the EHR and navigated to the outcome review section. The CE phase was open for at least two weeks. This was followed by another invitation in the EHR to complete the post-survey that mirrored the pre-survey. Participants were required to specifically opt into each survey and the CE phase of the study.

CAPCE-Approved Continuing Education

During the CE phase, participants were provided a link from the outcome review page that directed them to login to the learning platform. Two prerequisite videos provided background information and described the types of outcome information available: International Classification of Diseases diagnosis codes and Centers for Medicare and Medicaid Services disposition (eg, discharge to home, discharge to skill nursing, transfer, death, etc). Upon completion of these prerequisites, participants were able to review their patient-specific outcomes and answer a series of questions regarding each review.

Data Analysis

We used quantitative analyses to examine perceptions between pre- and post-surveys using chi-square tests or Fisher exact tests, as appropriate. Likert-scale response options (agree vs disagree) and demographics (Level: paramedic vs other; Role: patient care provider vs other; years of experience: 0–4 years, 5–10 years, 11–20 years, ≥21 years; previous frequency of outcomes review: very frequently/frequently vs occasionally/ rarely/never) were collapsed, as needed and where appropriate, due to cell size. Participation in the CAPCE CE phase of the study was also quantified. Data is reported as percentages and frequencies. P-values of statistical tests are also included. We performed quantitative analysis using STATA MP version 18.0 (StataCorp LLC; College Station, TX).

Qualitative analyses identified themes among open-ended responses. We used conventional content analysis as described by Hsieh and Shannon (2005). An inductive approach was used to extract meaning and themes from responses. All codes were generated from the content directly without a priori themes. Once codes were identified, we used a deductive approach to determine the frequency and distribution of themes as well as any relationships that existed between themes and respondent characteristics. Qualitative analysis was assisted using the QDA Miner Lite 3.0 (Provalis Research; Montreal, Quebec) software package.

This study was deemed exempt by the University of California Los Angeles Institutional Review Board.

RESULTS

During the study period, 843 EMS providers from the participating agencies logged into the EHR and were therefore eligible to participate. A total of 217 (25.7%) EMS providers anonymously responded to the pre-survey while on duty, and 152 (18.0%) anonymously responded to the post-survey while on duty. Demographics were similar among those that responded to the pre- and post-survey (Table 1).

Overall, responses were similar when comparing those who responded to the pre-survey and those who responded to the post-survey (Table 2). Notably, 89% in the pre-survey vs 95% in the post-survey indicated agreement that outcomes review enhanced job satisfaction (P = 0.05). When evaluating only those with a paramedic certification, we saw a statistically significant increase (P = 0.03) in the percentage of those indicating they review outcomes to improve clinical knowledge when comparing pre-survey respondents to post-survey respondents (94% vs 99%, respectively). We also noted that pre- and post-survey differences with respect to improvements in job satisfaction remained significant when only evaluating paramedics. No statistically significant difference was noted among pre-and post-survey respondents of those with other certification levels (Table S1). When examining years of EMS experience or role in the EMS system, responses between pre- and post-survey respondents were similar regardless of how many years the individual had worked in EMS (Table S2).

There were two significant differences when comparing those who responded to the pre-survey and those who responded to the post-survey when stratified by historical frequency of outcomes review. Among those who indicated that they review patient outcomes frequently or very frequently, we saw a significantly higher percentage of post-survey respondents indicating that they reviewed patient outcomes to obtain closure (90% vs 98%, P = 0.01, respectively) and that reviewing patient outcomes improves job satisfaction (92% vs 99%, P = 0.02, respectively) (Table S3).

CAPCE-Approved Continuing Education

A total of 211 individuals from the three participating EMS systems opted in to the CAPCE CE phase of this pilot study. Among those, 63% (133/211) successfully logged into the ESO learning platform. Of those who successfully logged in, 74% (98/133) completed their required profile. The prerequisites were completed by 70% (64/98) of those with a completed profile, and 88% (56/64) of those completed at least one CE activity. Among those who completed any CE, 88% (35/64) completed more than one CE activity and less than 10% (4/64) completed all available CE activities (Figure 1). A total of 287 outcomes were reviewed, and responses following the outcome review activity are listed in Table 3.

Qualitative Analysis

There were 72 pre-survey participants who answered the free-text question that asked, “Please describe any other reasons you review outcomes.” Free-text responses were uploaded into QDA Data Miner for analysis. All coding was performed by a single researcher (SB) and reviewed by AF and MK. Most initial codes related to reasons respondents reviewed the outcomes or benefits gained by doing so. The most common code identified was improve understanding of patients I’ve seen, which was identified 14 times (13% of all codes). The second most identified code was affirming (that my diagnosis and treatment were correct), identified eight times (7% of all codes). We excluded from further analysis 10 respondents whose only text content was coded as “do not review outcomes” or “wish list” (a code for entries that described features they wanted to see in the future).

We grouped the codes into themes organized by common attributes. These initial codes were organized by beneficiary of patient outcome review CE: the respondent; the patient encounter; and the EMS operation (Figure S1). Notably, unknown, a group created for codes that didn’t seem to fit any of the identified themes, comprised 17% of all codes. As a result, we performed a second thematic analysis in an attempt to integrate “unknown” codes and identify higher level meanings behind the codes. The resulting themes represented broader personal, operational, and system-related benefits to reviewing patient outcomes. These revised themes and their associated codes were reviewed by AF and MK with minor revisions recommended and integrated.

The final themes and code groupings can be found in Figure S2. We believe these codes and themes accurately reflect survey responses, and we used them for analysis of the post-survey, free-text question using the same method. Table 4 provides an overview of the themes, dominant codes, and key quotes for pre- and post-survey groups. In addition, we compared thematic frequency reports between the entire survey group and various demographic groups by respondent level, experience, and frequency of outcome review. These results can be found in Table S1–S4.

There were shifts in the perceived value of patient outcome review between the pre- and post-survey periods and across individuals with different backgrounds and experience (Figure 2). Those respondents who indicated they did not provide direct patient care placed the most value on operations and systemwide quality. For the remainder of respondents, pre-survey results found that the predominant value identified was related to acquiring personal knowledge, with enhancing personal confidence and competence, and improving personal clinical practice following—nearly 10% lower. During the post-survey, acquiring personal knowledge fell to the fourth position, with enhancing personal confidence and competence the predominant theme, followed by improving personal clinical practice and operations. In the post-survey population, acquiring personal knowledge remained valued by EMTs and individuals with 0–4 years of experience but was not valued at all in those with ≥21 years of experience.

DISCUSSION

Enhance Personal Confidence/Competence and Improve Patient Care

When surveyed anonymously, EMS providers favored viewing their patient outcomes. Qualitative analysis of pre-survey results revealed that the majority of certified personnel felt the primary benefit would be increases in personal knowledge; this attitude was particularly prevalent for EMTs and individuals who had limited experience. In post-survey results, providers found that reviewing patient outcomes was more relevant to gaining confidence and competence rather than knowledge acquisition. McGuire and colleagues evaluated feedback requests received from EMS providers and found the most common request was for the final diagnosis and outcome/disposition.
²⁸
This enhancement is commonly reflected in EMS providers who assess and care for their patients but are limited to only the beginning of the patient’s experience, not the final outcome. Providing them with the outcome of their patient allows them to either increase confidence in their assessment and diagnosis or participate in an opportunity for continued learning. This finding represents the workforce’s desire for an outcome-centric continued competence model over traditional forms of CE.

Providers also perceived improved patient care by viewing their hospital outcome data, which previous work has demonstrated is beneficial to the learning process and can improve patient outcomes. The qualitative review of post-survey results revealed an increase in the emphasis on enhancing personal confidence and competence and improving personal clinical practice, suggesting that providers see the potential for this education to improve their clinical practice rather than just providing knowledge. Post-resuscitation feedback has also been demonstrated to improve the quality of Advanced Life Support, specifically in survival until hospital discharge and favorable neurological outcomes and is why real-life, post-resuscitation feedback is recommended.
²³
Time to treatment in heart attack victims has also been shown to decrease following the implementation of data feedback to EMS providers, even in a system that was already achieving internationally established goals.
²⁴
Sammuel and colleagues conducted a scoping review of the effects of CPD on health professionals’ performance and patient outcomes and demonstrated changes in providers’ behavior and patient outcomes.
²⁵
Survey results published by Pollard and Black found that after receiving patient outcome data, most EMS providers reflected on the call and did further reading.
²⁹

EMS Provider Job Satisfaction

Study participants felt that learning their patient-specific hospital outcomes improved job satisfaction. Similar results have demonstrated that when asked, EMS providers anticipate patient outcome feedback benefiting their well-being and work engagement.
¹⁴
Providing patient outcomes helps bring EMS providers further into the healthcare continuum, and doing so reduces the causes of burnout for them.
³⁰
This is an important finding as agencies face a workforce shortage and the US sees fewer and fewer EMTs and paramedics.
³¹

Microlearning

The CAPCE-approved CE was presented to EMS providers in the form of microlearning. Microlearning consists of small doses of content in the form of lesson modules or short-term activities. This method allows learners to control all aspects of their learning, including the time in which they review, the pace, and the method by which they complete the activity. Providers reported being more likely to complete the review if they were provided with a means to use it to complete certification renewal; thus, if an agency provides patient outcomes to providers, it is best to include it as a means for certification renewal.

This patient outcome-specific education describes a new method of CE for EMS providers and aligns with the goals of CPD. Additionally, patient outcome-led education has been demonstrated to improve provider competency and improve patient outcomes. More study is needed on a wider scale to determine whether this type of education delivery ensures more competency than an hour-based model. This type of learning may also be appealing from an operational perspective, providing flexibility with respect to scheduling educational offerings and reducing the time that EMS providers are required to spend in a classroom.

LIMITATIONS

The sample size is specific to these three agencies (convenience sample) and those who chose to complete the survey; as such it may not be generalizable to the entire EMS workforce. The window for completing the survey was open for two weeks; thus, there may have been some providers who did not work in that period and could not complete the survey. Additionally, the request to complete the survey occurred during operational hours; so, participation may have been limited due to other, more urgent tasks. By its nature, qualitative analysis is heavily dependent on the background and skills of the researcher and may be influenced by personal biases.

A low proportion of providers who began the CE process actually gained CE credits, indicating the steps or process may be too difficult or cumbersome. It is also possible that participants did not need the CE credits, as each of the participating agencies provides 100% of the required hours. Additional research is needed to determine how to best integrate this learning method with existing work patterns and CE programs.

CONCLUSION

Emergency medical services providers supported the personal and professional development and patient care improvement value of reviewing patient outcomes, including microlearning activities. Participation in the required activities to obtain continuing education was low. However, subjects who did participate demonstrated a shift in perceived value from mere acquisition of knowledge to development of improved personal and systemwide clinical practice. Further study is warranted to evaluate the generalizability of these findings and the best user experience to facilitate the completion of CE.

Footnotes

Section Editor: Joshua Gaither, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Michael Kaduce, MPS, Falck Health Institute, 1517 W. Braden Ct., Orange, CA 92868. Email: Michael.Kaduce@falck.com
11 / 2024; 25:949 – 957

Submission history: Revision received May 9, 2024; Submitted August 9, 2024; Accepted August 12, 2024

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. Antonio Fernan, Scott Bourn, Remle P. Crowe, Alison Treichel, and J. Brent Myers are employees of ESO. There are no other conflicts of interest or sources of funding to declare.

REFERENCES

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.