Introduction
Methods
Results
Discussion
Limitations
Conclusion
Supplementary information

INTRODUCTION

In recent years, educators have been increasingly challenged to apply evidence-based practice to their teaching. Despite increased production and dissemination of education scholarship, there is still a great need to improve the quality of medical education research and associate educational practices with patient care outcomes.^1-9 Medical educators have reported multiple challenges to their scholarly pursuits, including lack of time, expertise in research methodology, funding, mentorship, collaborators, research support, and reward for their efforts.^10-12 Limited data suggest that lack of time may be the greatest barrier.¹¹

A recent workforce study of emergency medicine (EM) educators suggested that while education faculty make up a substantial proportion of a department’s core faculty, departments often lack the full complement of education leadership positions.¹³ Additionally, education faculty must frequently divide their non-clinical time among multiple academic roles.¹³ It was also noted in this study that many departments lack personnel with education research expertise.¹³ Potential interventions have been proposed to address these needs, including building communities of practice to enhance collaboration, increasing opportunities for funding, and devising strategies to gain protected time.¹⁰ Despite these preliminary studies, how EM educators perceive these barriers and what interventions would be most beneficial in helping to overcome them is still not well understood.

The Council of Emergency Medicine Residency Directors (CORD) Education Scholarship Task Force and CORD Academy for Scholarship in Education in Emergency Medicine recommended that the EM education research community conduct a formal needs assessment to analyze the specific needs of EM educators in order to design and implement interventions to support educators and the field of education research. The objective of this study was to evaluate the perspectives of both core-faculty educators and successful education researchers with regard to the supporting factors and motivators to performing education research, as well as the barriers and their perceived impact, and proposed solutions to assist them in their scholarly endeavors.

METHODS

Study Setting and Participants

Core EM education faculty (defined as those individuals whose main academic role is dedicated to the educational mission of the department, including undergraduate medical education, graduate medical education, and faculty development), were identified through email inquiry of individual program leadership (program director and/or program coordinator), program websites, and personal knowledge. We identified successful EM education researchers in one of two ways: (1) by authorship on a manuscript included in Academic Emergency Medicine’s “Critical Appraisal of Emergency Medicine Education Research: The Best Publications of [years 2008–2014]”; or (2) designation of “Scholar” from the Association of American Medical Colleges Medical Education Research Certificate at CORD program.^14-21 When an individual belonged to both cohorts, s/he was enrolled in the education researcher arm of the study only. Data collection occurred between October 2015 and December 2015.

This study was deemed exempt by the institutional review board of the Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center.

Population Health Research Capsule

What do we already know about this issue?

Educators face multiple challenges in achieving their education scholarship goals. There is a need to illuminate effective ways to support them and to improve the quality of education research.

What was the research question?

What are educators’ perceptions regarding barriers to performing scholarship and potential strategies for success?

What was the major finding of the study?

Common barriers were lack of time and competing demands. Suggested interventions were training, mentorship, and collaboration.

How does this improve population health?

This information may inform interventions to support educators in their scholarly pursuits and improve the overall quality of education research in emergency medicine.

Study Design

This was a cross-sectional, mixed-methods needs assessment study, employing a standardized, survey instrument (with validity evidence previously collected) that allowed for free responses suitable for qualitative analysis. Subjects were invited to participate by email and provided with a link to an Internet-based survey, administered through SurveyMonkey®.²² Two follow-up email invitations were sent at weekly intervals to non-responders. Informed consent was implied by those participants who chose to click on the survey link. To maximize response rate and include all possible relevant data, completion of all survey questions was not required.

Instrument Development

The authors developed two surveys, one for each stakeholder group, after literature review and input from members of the CORD Education Scholarship Taskforce to maximize content validity. Instrument development followed established guidelines for survey research.²³ The surveys consisted of multiple-choice, 10-point Likert scale, and free-response items. To optimize response process validity, items were read aloud among members of the study group and piloted with a small group of reference subjects. Based on results of piloting, we then revised survey items for clarity and brevity. Final versions of the survey instruments are available in Appendix.

Statistical Analysis

We calculated and reported descriptive statistics for multiple-choice and rating-scale items. Two researchers experienced in qualitative methods, JJ and LY, independently analyzed data from free-response items using a thematic approach. They examined data line by line to identify recurring concepts and then assigned codes, which were further refined into themes using the constant comparative method.²⁴ After independent review, the two researchers met to establish a final coding scheme that was applied to all data. Inter-rater agreement was 93.9% and 89.4% for data from core educators and education researchers, respectively. Discrepancies were resolved by in-depth discussion and negotiated consensus.

RESULTS

General Results

A convenience sample of 204 core educators and 42 education researchers, from 118/164 (72%) EM training programs in the U.S. and Canada completed the surveys. Of the core educators responding, 159/197 (80.7%) reported performing research, of whom 111 (69.8%) performed research in medical education. Education researchers were highly productive: 19/42 (45.2%) reported more than 20 peer-reviewed education scholarship publications on their curricula vitae. In contrast, 68/197 (34.5%) of core educators had not published any education scholarship in the last five years. Characteristics of participants and scholarly productivity are shown in Table 1.

Motivators, Rewards, Career Satisfaction

Our qualitative analysis revealed a number of motivating factors for performing education research in both cohorts. The most prominent of these factors were the desire to disseminate knowledge, support evidence-based practices, meet academic promotion requirements, and personal interest. Results of qualitative analysis for education researchers and core educators are shown in Table 2 and Table 3, respectively. When asked to specifically rate various motivators, education researchers identified personal intellectual stimulation and to become a better teacher as most influential with mean ratings of 8.52 and 7.21 respectively on a 10-point scale (Figure 1). Core educators also rated these factors highest with mean ratings of 7.57 and 6.91 respectively (Figure 1).

The most common rewards education researchers reported include a sense of accomplishment by contributing to the body of knowledge of the field (39/42; 92.9%) and intellectual satisfaction from solving a problem (39/42; 92.9%) (Figure 2). Core educators also reported rewards of satisfaction of contributing to the body of knowledge of the field (123/147; 83.7%) and intellectual satisfaction of solving a problem (114/147; 77.6%) (Figure 2). Education researchers were satisfied with their achievements in education research and their overall careers, with mean ratings of 7.02 and 8.22 respectively, and felt that performing research contributed positively to their career (mean rating 7.14). Core educators were also satisfied with their careers with a mean rating of 7.62, but less satisfied with their achievements in education research with a mean rating of 4.54. Teaching was the most prominent contributor to career satisfaction for core educators (Table 3).

Barriers and Challenges

Lack of time was the greatest barrier for core educators, with mean rating of 8.61 on a 10-point scale (Figure 3). Core educators reported spending the majority of their time on clinical duties, with mean hours per week of 21.95 ± 10.90, followed by administrative duties 17.53 ± 10.38, teaching 7.58 ± 5.62, research 3.6 ±4.30, and other scholarly work 3.91 ± 3.51. Ideally, core educators would prefer to spend less time on clinical and administrative duties and more time on teaching, research, and other scholarly work. Desired mean hours/week include 18.42 ± 8.45 on clinical duties, 11 ± 7.38 on administrative duties, 9.61 ± 5.91 on teaching, 6.92 ± 5.16 on research, and 4.81 ± 3.41 on other scholarly work. The most prominent challenges for core educators and education researchers were lack of time and competing demands (Tables 2 and 3).

Core educators also cited lack of methodologic expertise as a major barrier. Approximately half of responding core educators (91/183; 49.7%) reported having a mentor. Major themes regarding the positive impact a mentor had on their ability to perform education scholarship for core educators included motivation and training. It should be noted, however, that a contrasting major theme identified was that the core educator’s mentor did not impact this area at all (Table 3). The major theme regarding reasons educators did not have a mentor was lack of an identifiable candidate.

Strategies for Success

Education researchers and core educators felt that protected time, a collaborative community/research network, and mentorship would help them achieve their research goals (Tables 2 and 3). Core educators indicated they would like to acquire more skills in research study design (112/183; 61.2%), qualitative analysis (88/183; 48.1%), scientific writing (91/183; 49.7%), and quantitative analysis (77/183; (42.1%). The preferred formats for learning skills in medical education research were an online longitudinal course or a longitudinal faculty development course offered at their home institution with 65/181 (35.9%) and 61/181 (33.7%) selecting these options. Less-preferred formats included a daylong session at a professional society national meeting (25/181; 13.8%) or an advanced degree (21/181; 11.6%). Major themes regarding advice from both stakeholder groups to those wishing to become more involved in research included obtaining formal training, finding collaborators, and securing mentorship (Tables 2 and 3).

DISCUSSION

Critics of medical education research often cite a lack of researcher training and expertise as a key barrier to successful scholarship. While this was identified as a major barrier by our core educator stakeholder group, it is important to note that even formally trained, successful education researchers experienced challenges in this field similar to those who were untrained. In this study, both successful education researchers and core educators identified barriers consistent with prior literature.^10-12 These include barriers that are intrinsic to the researcher such as time constraints and lack of formal research training; extrinsic factors such as lack of funding, lack of research resources, collaborators, mentorship, and leadership support; and barriers inherent to this type of scholarship such as challenges with learners as a study population and perceived futility. The fact that we saw a great deal of overlap between these two stakeholder groups suggests that these barriers may exist regardless of researcher experience or career stage.

The greatest barrier in this study seemed to relate to time constraints. This is not surprising given prior literature emphasizing the complexity and importance of this barrier as well as a prior EM workforce study demonstrating that educators often play multiple critical academic roles and have less time available for scholarly pursuits compared to other job requirements.^11,13 This previously identified mismatch between actual and ideal distribution of workload may not only negatively impact an educator’s ability to perform scholarship but may also adversely affect career satisfaction and burnout.^13,25

Mentorship was also identified as barrier for core educators: less than half of participants in this group reported having a mentor owing to lack of availability. This may indicate that currently few experts are available to meet the needs of education scholars and/or that those with expertise exist outside the field of EM.¹³ Interestingly, successful education researchers were less likely to cite lack of mentorship as a barrier, but did recognize its importance to those looking to pursue education research. This may be because these individuals had available mentorship in their formative years, which may have contributed to their success. Those educators who did have mentorship identified multiple ways their mentorship positively contributed to their scholarly pursuits. It will be important to continue training interested educators in this area to build a cadre of medical education research experts who can meet the training and mentorship needs of future generations and for would-be scholars to look outside of their institution and/or specialty to find this expertise.

We also found a great deal of overlap between motivators to performing education scholarship between core educators and successful education researchers, which may reflect core values such as an emphasis on life-long learning, creation of community of inquiry, desire to achieve success and contribute positively to the field, and to satisfy job requirements and achieve promotion. In this study educators and successful education researchers reported receiving more intrinsic than extrinsic rewards for performing education research. This is in line with prior literature identifying lack of reward as a barrier to performing education scholarship.¹⁰

Performing research positively contributed to career satisfaction for education researchers, which is not surprising if this is something that they chose to spend their time on despite identifying multiple barriers. Interestingly, despite being satisfied overall with their careers, core educators were less satisfied with their achievements in research. Since research may impact other factors that educators have identified as positively contributing to their satisfaction in this study, such as variety, sense of accomplishment and the potential to enhance teaching, it would be interesting to explore whether educator career satisfaction could be further enhanced by improving their satisfaction with their research achievements.

Advice from those with experience and success in the field is well aligned with the needs identified by participants in both stakeholder groups, further supporting that these are the areas where resources and support should be targeted. Suggested strategies span multiple levels including addressing needs both intrinsic and extrinsic to the would-be scholar, and barriers specific to the field of education research. The aspect of training and acquiring expertise is an expressed need and also recommended advice from those who have been successful. Core educators specifically seek more methodologic training and would prefer a longitudinal online course or one locally available at their institution. These formats are likely preferred because of accessibility since it has already been demonstrated in this study and others that time is an important issue and workload demands for educators are high.^10-11,13 To meet the expressed needs identified in this study and follow advice of those with experience and success in the field, future interventions should target an increase in training opportunities, access to expertise, creation of a cadre of trained medical education research experts to serve as mentors, increased funding opportunities and better research infrastructure, and emphasis on the value of this work to garner leadership support and assist in the development of mechanisms to ensure adequate protected time for educators to be successful in their scholarly endeavors.

LIMITATIONS

This was a convenience sample and completion of all items on the survey was not required as we desired to include all relevant data. It is possible that we may have failed to capture important information. However, this is a fairly large study and given the broad distribution of programs represented, we expect the perspectives expressed by participants to be representative of the group as a whole. Additionally, as this was a survey study, the results must be considered within the context of limitations inherent to this type of design. Despite these limitations, we still believe this study sheds further light on the barriers educators face in performing education research and illuminates motivators and potential strategies for improvement.

CONCLUSION

Our study identified multiple barriers, motivators/discouragers, as well as strategies for success in performing education scholarship, which were common to both core educators and successful education researchers. The most commonly cited barriers were lack of time and competing demands. Core educators were interested in attaining new skills in education research through faculty development. Key motivators to perform education research for both education researchers and core educators were the desire to disseminate knowledge, support evidence-based practices, and achieve promotion. Suggested strategies for success included formal training, mentorship, and collaboration. This information may inform interventions to support educators in their scholarly pursuits and improve the overall quality of education research in EM.

Supplementary Information

Footnotes

Section Editor: Andrew W. Phillips, MD, MEd

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Jaime Jordan, MD, Harbor-UCLA Medical Center, Department of Emergency Medicine, 1000 West Carson Street Box 21, Torrance, CA 90502, Email: jaimejordanmd@gmail.com. 5 / 2018; 19:619 – 629

Submission history: Revision received October 13, 2017; Submitted January 12, 2018; Accepted January 11, 2018

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Sullivan GM, Simpson D, Cook DA, et al. Redefining quality in medical education research: a consumer’s view. J Grad Med Educ. 2014;6(3):424-9.

2. McGaghie WC, Issenberg SB, Petrusa ER, et al. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44(1):50-63.

3. Cook DA, Beckman TJ, Bordage G. Quality of reporting of experimental studies in medical education: a systematic review. Med Educ. 2007;41(8):737-45.

4. Cook DA, Levinson AJ, Garside S. Method and reporting quality in health professions education research: a systematic review. Med Educ. 2011;45(3):227-38.

5. Reed DA, Cook DA, Beckman TJ, et al. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-9.

6. Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med. 2004;79(10):955-60.

7. Lurie SJ. Raising the passing grade for studies of medical education. JAMA. 2003;290(9):1210-12.

8. Reed DA, Beckman TJ, Wright SM. An assessment of the methodologic quality of medical education research studies published in The American Journal of Surgery. Am J Surg. 2009;198(3):442-4.

9. Cook DA, Bowen JL, Gerrity MS, et al. Proposed standards for medical education submissions to the Journal of General Internal Medicine. J Gen Intern Med. 2008;23(7):908-13.

10. Yarris LM, Juve AM, Artino AR, et al. Expertise, Time, Money, Mentoring, and Reward: Systematic Barriers that Limit Education Researcher Productivity – Proceedings from the AAMC GEA Workshop. J Grad Med Educ. 2014;6(3):430-6.

11. Zibrowski EM, Weston WW, Goldszmidt MA. ‘I don’t have time’: issues of fragmentation, prioritisation and motivation for education scholarship among medical faculty. Med Educ. 2008;42(9):872-8.

12. Goldszmidt MA, Zibrowski EM, Weston WW. Education scholarship: it’s not just a question of ‘degree.’. Med Teach. 2008;30(1):34-9.

13. Jordan J, Coates WC, Clarke S, et al. Exploring scholarship and the emergency medicine educator: a workforce study. West J Emerg Med. 2017;18(1):163-8.

14. Farrell SE, Coates WC, Kuhn GJ, et al. Highlights in Emergency Medicine Medical Education Research: 2008. Acad Emerg Med. 2009;16(12):1318-24.

15. Kuhn GJ, Shayne P, Coates WC, et al. Critical appraisal of emergency medicine educational research: the best publications of 2009. Acad Emerg Med. 2010;17(S2):S16-25.

16. Shayne P, Coates WC, Farrell SE, et al. Critical appraisal of emergency medicine educational research: the best publications of 2010. Acad Emerg Med. 2011;18(10):1081-9.

17. Fisher J, Lin M, Coates WC, et al. Critical appraisal of emergency medicine educational research: the best publications of 2011. Acad Emerg Med. 2013;20(2):200-8.

18. Lin M, Fisher J, Coates WC, et al. Critical appraisal of emergency medicine education research: the best publications of 2012. Acad Emerg Med. 2014;21(3):322-33.

19. Farrell SE, Kuhn GJ, Coates WC, et al. Critical appraisal of emergency medicine education research: the best publications of 2013. Acad Emerg Med. 2014;21(11):1274-83.

20. Yarris LM, Juve AM, Coates WC, et al. Critical appraisal of emergency medicine education research: the best publications of 2014. Acad Emerg Med. 2015;22(11):1327-36.

21. Available at: https://www.cordem.org/i4a/pages/index.cfm?pageid=3344. Accessed September 12, 2017.

22. Available at: www.surveymonkey.com. Accessed September 12, 2017.

23. Rickards G, Magee C, Artino AR. You can’t fix by analysis what you’ve spoiled by design: developing survey instruments and collecting validity evidence. J Grad Med Educ. 2012;4(4):407-10.

24. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42:1758-1772.

25. Rothenberger DA. Physician burnout and well-being: a systematic review and framework for action. Dis Colon Rectum. 2017;60(6):567-76.