Author	Affiliation
Peter R. Chai, MD, MMS	Brigham and Women’s Hospital, Department of Emergency Medicine, Boston, Massachusetts; Harvard Medical School, Department of Emergency Medicine, Boston, Massachusetts; The Fenway Institute, Boston, Massachusetts; Massachusetts Institute of Technology, The Koch Institute for Integrated Cancer Research, Cambridge, Massachusetts
Stephanie Carreiro, MD	University of Massachusetts Medical Center, Department of Emergency Medicine, Division of Medical Toxicology, Worcester, Massachusetts
Brittany P. Chapman, BS	University of Massachusetts Medical Center, Department of Emergency Medicine, Division of Medical Toxicology, Worcester, Massachusetts
Edward W. Boyer, MD, PhD	Brigham and Women’s Hospital, Department of Emergency Medicine, Boston, Massachusetts; Harvard Medical School, Department of Emergency Medicine, Boston, Massachusetts; The Fenway Institute, Boston, Massachusetts
Kelli N. O’Laughlin, MD, MPH	University of Washington, Department of Emergency Medicine, Seattle, Washington; University of Washington, Department of Global Health, Seattle, Washington

Introduction
Methods
Results
Discussion
Limitations
Conclusion
Supplementary information

INTRODUCTION

Research grants awarded from federal sources such as the National Institutes of Health (NIH) remain a leading measure of success in academic medicine. Clinician scientists pursing a research career, including emergency physicians (EP), often pursue a career development award (ie, K grants) as a bridge to independent funding status (ie, R-series awards).¹ Career development awards are considered prestigious to the investigator as well as to his or her sponsoring department, but they require a significant investment of time from the trainee and mentorship team.² Successful submission of a federal career development grant may require up to two years of preparation, which includes finding a committed mentorship team, crafting a research and training plan, initiating pilot data collection, generating a sufficient number of publications to demonstrate commitment to academic practice, and undergoing a rigorous review process. During this time, emergency medicine (EM) faculty are faced with significant barriers including clinical commitments, administrative obligations, and time required for mentorship meetings and grant writing.³ They also face the challenges associated with the transition into junior faculty.³ Compared to other specialties, EPs submit the fewest mentored career awards (K grants), have the third lowest success rate (60% funded), and submit the fewest grants per faculty size.^2,4

EPs are well positioned to make meaningful contributions to research given their breadth of clinical expertise across a wide spectrum of disease, unique window into the communities where they practice, and natural collaboration with multiple clinical disciplines. Despite this, a 2006 Institute of Medicine report demonstrated that few NIH training grants were awarded in emergency departments (ED).⁵ Recognizing the immediate importance of training future clinician-researchers in EM led to the creation of the Office of Emergency Care Research (OECR) in 2012 to coordinate research in this rapidly growing space. ^6,7

From 2011 to 2014, emergency care research made up only 0.7% of NIH spending on new grants.⁸ In response, the Society for Academic Emergency Medicine (SAEM) and the American College of Emergency Physicians (ACEP) recommended four key strategies to increase the pipeline of federally funded emergency care researchers: 1) promote research as a viable career trajectory; 2) identify the availability of senior mentors; 3) understand the process of applying for NIH funding as a financial investment; and, 4) build a supportive culture that fosters research.⁹ Currently, several EM-based, NIH-funded training programs (eg, T32, K12) provide structured mentorship and funded protected time that allow junior academicians to gather preliminary data in support of subsequent investigations.¹⁰ These programs, in addition to increased support for a research career path, have resulted in 1.7% of funded NIH grants attributed to EM faculty from 2008 to 2017.¹¹ Although improved, these statistics indicate that EM researchers still receive relatively few NIH grants in comparison to other specialties.

The NIH Research Portfolio Online Reporting Tool (NIH RePORTER) is a publicly available database and a tool that can be used to better understand NIH funding related to EM faculty in the United States.¹² Outside of official NIH reporting, however, limited data exists to assess variables that are important to achieving NIH funding, such as the clinical specialty of mentors, the availability of protected time, and access to department-funded research infrastructure. Knowledge regarding the prevalence of these important variables among NIH-funded EM investigators may be useful for individuals seeking a research career to help in selecting academic positions, and for EDs working to enhance research among their faculty.

 

Population Health Research Capsule

What do we already know about this issue?

Emergency physicians (EP) face challenges in obtaining funding, finding mentors, and managing the balance between research and clinical work.

What was the research question?

To define the demographics of federally funded EPs and their barriers to obtaining funding.

What was the major finding of the study?

Most 2010–2017 awardees were White males, although women got more mentored career grants. EPs still experience difficulty finding adequate mentors.

How does this improve population health?

Increased support for the EP researcher, especially for women and minorities, remains important in growing the body of federally funded EPs.

 

The goal of our study was to use both NIH RePORTER data and individual surveys to describe the following: 1) the distribution and characteristics of NIH grants awarded to EPs; 2) the principal investigator (PI) characteristics and resources accessed for these successful applications; and, 3) perceived facilitators and barriers to the NIH grant funding process from the PI’s perspective.

METHODS

Study Design and Selection of Participants

This investigation was composed of two parts. Part one was a retrospective database search using the NIH RePORTER to collect data regarding federally funded grants awarded to EM PIs. We included individuals who were funded by the Agency for Healthcare Research and Quality and the Centers for Disease Control and Prevention, as these federal grants are included in NIH RePORTER. Part two was a survey-based investigation that queried the NIH-funded EM PIs identified in part one to obtain additional demographics as well as information on their career paths, research environment, and perceived barriers to applying for funding. The study protocols for parts one and two were deemed exempt by the human subjects institutional review board by the Partners Human Research Committee.

Part I: NIH RePORTER Data Abstraction

A list of relevant NIH-funded research projects was curated from the NIH RePORTER search function using the following search criteria: 1) funding received between fiscal years 2010–2017 (10/01/10 – 09/30/17); and 2) department type listed as EM. One author (BPC) manually reviewed this list and removed projects funded in this period without a start date in this timeframe. We extracted variables from all remaining records including the PI’s gender and academic rank, grant mechanism (eg, K- or R-series), PI contact information, start year of the grant, total years of funding awarded to the grant, and NIH funding institute. We also recorded the geographic location of the PI’s primary institution.

Part II: EM PI Survey

Using data extracted from part one, we stratified the identified EM PIs into two cohorts: 1) “mentored PIs” with career development awards (eg, K08, K12, K23); and 2) “independent PIs” with independent research grants (eg, R01, R34, R21). We included individuals supported by a cooperative agreement (U-mechanism) or mid-career mentoring award (K24) with the independently funded cohort. Individuals who were listed as having both a K grant and a subsequent or parallel R grant were included only in the independent PI group. We electronically distributed surveys to all of the PIs identified, based on these two cohorts.

We designed two separate surveys to answer key questions about the demographics of NIH-funded investigators, and the relationship of their research area to EM. Surveys were created through an iterative process among the study team. The study team identified themes surrounding funding, research topic, and mentorship and then drafted several questions surrounding these concepts. Next, the study team selected questions that were clear and piloted these on non-study team EP investigators to ensure clarity of the questions. These final surveys were then administered to mentored PIs and independent PIs. The mentored PI survey (Appendix 1) included questions on demographic data (gender, ethnicity, race, and academic rank) and on research focus and environment (including mentor’s academic department, administrative support, pre- and post-award grant administrative support, and average monthly hours worked in the ED during the award period).

The independent PI survey (Appendix 2) was designed to collect demographic data (gender, age, ethnicity, race, academic rank), information about research career (prior K award funding), research focus, and research environment. Both surveys included open-ended questions asking about barriers that EM PIs faced in obtaining career development awards.

Data Management

We used the Research Electronic Data Capture (REDCap) tool to capture and manage all study data including surveys.¹³ Reminder emails were sent on days 10, 20, and 30 to individuals who had not started or who had partially completed the survey; the survey link expired on day 31. All study communications were sent via the REDCap database in survey mode.

Data Analysis

For quantitative data, we determined basic descriptive statistics for sociodemographic variables, grant characteristics, and institution characteristics. We calculated percentages for categorical variables, and calculated medians with interquartile ranges for continuous variables. We constructed heat maps using key regions of the United States defined by the US Census Bureau to explore the geographic distribution of identified grants. We analyzed all quantitative data using STATA version 15.1 (StataCorp LLC, College Station, TX).

For qualitative data, we used a conventional content analysis approach to understand participants’ experiences.¹⁴ In conventional content analysis, coded categories are taken from the text allowing us to derive information from responses without preconceived categories. Two analysts (PRC, SC) reviewed all of the responses and inductively derived codes based on content similarity within the text. We revised groupings using an iterative process of content review and returning to the data. Analysts debated discrepancies until consensus was achieved.

RESULTS

Part I: NIH RePORTER Results

Over the seven-year study period, we identified 219 grant awards from NIH RePORTER records that met inclusion criteria (Table 1), which were awarded to 156 unique individual investigators. A majority of grants (N = 162, 74%) were awarded to males. Grants were fairly evenly distributed by academic rank, with 39% of PIs listed as professor, 26% as associate professor level, and 31% as assistant professors. The most common awarding NIH institutions were the National Heart Lung and Blood Institute (26%), the National Institute of Neurological Disorders and Stroke (13%), and the Agency for Healthcare Research and Quality (12%). An average of 31 NIH awards were obtained annually from fiscal year 2010–2017, with a steady upward trend from 2013–2017. The most commonly identified mechanisms were R01 (29%), K23 (6%), K12 (5%), and R03 (5%).

We identified a greater clustering of grants awarded to PIs located in the Midwest and New England (Figure 1) than within other regions of the country. Additionally, mentored career development awards and independent investigator awards appeared to cluster together in the same regions.

Part II: Survey results

We sent electronic surveys to 51 unique mentored PIs identified in Part I with a 69% response rate (N = 35), and to 105 independent PIs identified from Part I with a 58% response rate (N = 61).

Demographics, Investigator and Research Characteristics

The majority of mentored PIs were male (N = 20, 59%), White (N = 28, 82%), and at the rank of assistant professor (N = 24, 69%) (Table 2). Overall, the proportion of grants awarded to women over the study period was stable. Mentored PIs reported their primary mentors were in EM (N = 11, 32%), cardiology (N = 5, 15%), internal medicine (N = 4, 12%), and infectious disease (N = 2, 6%). Most research conducted under the mentored career development mechanism was focused in EM (N = 24, 71%). Only three (9%) participants did not have grants administration support within their department. Participants reported that they worked an average of 48 clinical hours per month during the time of their K award.

Of the independent PI respondents, the majority identified as male (N=46, 77%) and White (N = 48, 80%). Most participants (N = 38, 63%) did not have a mentored career development award prior to obtaining independent funding. Participants who received a K award reported working an average of 41 clinical hours per month during their K award period. Independent investigators commonly identified having a mentor within EM (N = 22, 40%) or internal medicine (N = 12, 22%).

Barriers to obtaining mentored career development awards

When asked about barriers to obtaining mentored career development awards, 88% (N = 30) of K survey respondents and 28% (N = 19) of R-funded survey respondents provided answers. Commonly identified barriers were similar between both groups: 1) finding appropriate mentorship; 2) having appropriate time to prepare and submit a K award; and, 3) lacking a robust administrative infrastructure to support NIH awards (Table 3). With respect to mentorship, participants specifically reported significant barriers in finding adequate mentors in EM. Twenty-four participants reported that they sought mentorship outside of EM in disease-specific areas; many cited the structure of disease-based NIH awarding institutes as a driving factor. Participants also reported that a lack of mentorship at times prevented them from learning about the value of career development awards and detracted from their perceived ability to pursue this line of funding

Participants also reported difficulty in finding time away from clinical and administrative commitments to prepare and submit a K application. The competing priorities of clinical work, teaching, and completing administrative tasks interfered with the time needed to meet mentors, generate sufficient publications, and prepare the grant application. Additionally, participants reported variability in the degree of administrative support their departments provided in applying for NIH awards.

DISCUSSION

This investigation demonstrates that despite recent efforts to foster NIH-funded research in EM, there remain critical barriers to successful funding, particularly for early-career investigators. However, for EM to achieve its maximum potential impact in research, we should focus on providing investigators with mentorship, protected time, and grants of administrative support to enhance success.

Almost half (41%) of K-funded respondents identified as female compared to 23% of independently funded investigators. The proportion of female EM investigators funded under a K mechanism is higher than reported in other specialties such as surgery and anesthesiology, yet the total number of grants overall awarded to women has not changed.¹⁵ This may demonstrate success of recent initiatives to promote research among female junior faculty. While the higher proportion of female-mentored PIs compared to independent PIs may reflect a trend toward gender equity, an alternative explanation may be that female investigators are not successfully transitioning to independent funding status. This is concerning because overall NIH data and investigations within other specialties demonstrate increasing parity among men and women who transition from mentored to independent awards.^16,17,18 Among EM investigators, the transition rate from mentored to independent investigator is approximately 40%, yet the proportion of women who successfully complete this transition is unknown.¹⁹ Our data suggests that continued efforts to support women, especially during the end of the mentored award period, is needed within EM to improve gender equity among independent researchers

While initiatives such as junior faculty development programs and female-specific mentoring may increase the number of women pursuing careers in research, a focus on continued mentorship and support for female faculty as they transition from mentored research to independent funding status may help increase the number of independently-funded female investigators in EM. The difficult transition from mentored grants to independent investigator status should not be ignored; support from academic departments at the early career phase should occur in synergy with support at the transition to independent funding.²⁵

In parallel with promoting careers in NIH-funded research for women, we should also focus on increasing racial diversity. Our survey respondents of both training and independently funded NIH awards overwhelmingly identified as White. Initiatives that promote research careers among women, as well as among racial and ethnic minorities, should continue to be high priorities for EM.²⁰

We found that mentored career-oriented grants and independent investigator awards tended to cluster by geographic region, which is consistent with prior literature.²¹ One explanation for this phenomenon is that independently funded investigators attract other EPs who initiate research careers. Departments with a strong research division also offer a large professional network and access to resources that benefit junior investigators. Finally, EDs with senior investigators may indicate a commitment by the department to a career in research, thereby providing strong grants support, mentorship, and even seed funding to junior investigators. Departments with focused mentoring programs result in increased NIH funding success, increased number of publications, and higher levels of perceived success.²² Academic EDs that may not have NIH-funded scientists on faculty, specifically in rural areas, may have important and fundable priorities to study. Targeted interventions that extend resources to traditionally less research heavy institutions (eg, seed funding for preliminary data, and a network of available mentors willing to work with new investigators at remote sites) can create equitable opportunities for research careers across the country.

Our qualitative survey data suggests that lack of mentoring, time, and grants infrastructure hamper successful NIH award applications among EM investigators. Less than half of mentored PIs identified that their primary mentor was in EM. Participants reported that they sought mentorship outside of EM due to a lack of NIH-funded EM faculty and lack of mentorship within a disease state. Continued promotion of the NIH career track and increasing numbers of independently funded EM faculty will hopefully increase the availability of those who can serve as mentors for junior faculty. Comprehensive mentorship programs as part of faculty development programs or department-initiated research mentoring can help create an environment that promotes and supports federally funded research.^3,23

Participants also commented on the difficulty of preparing a K award in the context of transitioning to junior faculty status and completing clinical and administrative duties within EM. Balancing the unpredictable clinical schedule of an EP with time needed to meet and establish mentoring plans, publish manuscripts, generate preliminary data, and prepare grant submission materials was a barrier reported by participants in this study. Finding protected time to write and conduct preliminary research is especially difficult for junior faculty, yet the value of dedicated time early in one’s career cannot be underestimated.²⁴ The barriers we identified are similar to those addressed by a joint SAEM/ACEP research committee.⁹ While many institutions have junior faculty development programs geared toward supporting early-career physicians, EDs must continue to consider the importance of dedicating time and funding to their junior faculty to boost and support research careers during this vulnerable stage. Peer mentoring groups should also be considered as a supplement, although they cannot replace senior mentorship.³

LIMITATIONS

Our primary data source from Part One (NIH RePORTER) is limited and would not capture grants or EM PIs listed under a different department, or those listed under institutional career development awards (eg, K12, KL2 mechanisms). We do not know how many potential EM investigators would have been missed through our query using NIH RePORTER. The constraints of our search mechanism and the nature of our study population led to a small sample size, limiting our analysis. The low response rate on our survey component creates potential for missing and/or biased data. Finally, some survey participants reported they had limited time to answer the survey questions, and as a result, did not provide answers to all questions.

CONCLUSION

Our review of funded EM grants through the NIH RePORTER system and individual surveys of mentored and independent investigators demonstrated several positive trends, including increasing gender diversity in early-career mentored grants, yet there continue to be important areas for improvement such as access to mentorship, grants infrastructure, and dedicated time early in a career to develop important research opportunities. Continued support for research as a career path and mentoring of early-career physicians is important in growing the cadre of EM researchers. Future work, such as longitudinal studies to evaluate individual characteristics associated with success and interventions geared toward increasing mentorship and support, are needed.

Supplementary Information

 

Footnotes

Section Editor: Gary Johnson, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Peter R. Chai, MD, Brigham and Women’s Hospital, Department of Emergency Medicine, 75 Francis St, Boston, MA 02115. Email: pchai@bwh.harvard.edu 3 / 2020; 21:304 – 312

Submission history: Revision received September 20, 2019; Submitted November 19, 2019; Accepted December 8, 2019

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. The following authors were NIH funded: PC (K23DA044874), SC (K23DA045242), KO (K23MH108440), and EB (R01DA047236).

REFERENCES

1. Koroshetz WJ, Brown J. NIH: developing and funding research in emergency care and training the next generation of emergency care researchers. Health Aff (Milwood). 2013;32(12):2186-92.

2. Brown J. National Institutes of Health support for individual mentored career development grants in emergency medicine. Acad Emerg Med. 2014;21(11):1269-73.

3. Chai PR, Carreiro S, Carey JL, et al. Faculty member writing groups support productivity. Clin Teach. 2019;16(6):565-9.

4. Bessman SC, Agada NO, Ding R, et al. Comparing National Institutes of Health funding of emergency medicine to four medical specialties. Acad Emerg Med. 2011;18(9):1001-4.

5. Committee on the Future of Emergency Care in the US Health System. Hospital-Based Emergency Care: at the Breaking Point. 2006.

6. Mitka M. NIH signals intent to boost funding of emergency care research and training. JAMA. 2012;308(12):1193-4.

7. McKenna M. NIH launches emergency care research office. Ann Emerg Med. 2013;61(3):A19-22.

8. Brown J. National Institutes of Health support for clinical emergency care research, 2011 to 2014. Ann Emerg Med. 2016;68(2):164-71.

9. Ranney ML, Limkakeng AT, Carr B, et al. Improving the emergency care research investigator pipeline: SAEM/ACEP recommendations. Acad Emerg Med. 2015;22(7):849-51.

10. Newgard CD, Morris CD, Smith L, et al. The first National Institutes of Health institutional training program in emergency care research: productivity and outcomes. Ann Emerg Med. 2018;72(6):679-90.

11. Jang DH, Levy PD, Shofer FS, et al. A comparative analysis of National Institutes of Health research support for emergency medicine-2008 to 2017. Am J Emerg Med. 2019;37(10):1850-4.

12. NIH Research Portfolio Online Reporting Tools (RePORT). Available at: https://report.nih.gov. Accessed January 5, 2019.

13. Harris PA. Research Electronic Data Capture (REDCap) – planning, collecting and managing data for clinical and translational research. BMC Bioinformatics. 2012;13(Suppl 12):A15.

14. Guest GS, MacQueen K, Namey E. Applied Thematic Analysis. 2011.

15. Mayes LM, Wong CA, Zimmer S, et al. Gender differences in career development awards in United States’ anesthesiology and surgery departments, 2006–2016. BMC Anesthesiol. 2018;18(1):95.

16. Pohlhaus JR, Jiang H, Wagner RM, et al. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med. 2011;86(6):759-67.

17. Kalyani RR, Yeh H-C, Clark JM, et al. Sex differences among career development awardees in the attainment of independent research funding in a department of medicine. J Womens Health. 2015;24(11):933-9.

18. Warner ET, Carapinha R, Weber GM, et al. Gender Differences in Receipt of National Institutes of Health R01 Grants Among Junior Faculty at an Academic Medical Center: The Role of Connectivity, Rank, and Research Productivity. J Womens Health. 2017;26(10):1086-93.

19. Nishijima DK, Yadav K, May L, et al. Description and productivity of emergency medicine researchers receiving K23 or K08 mentored research career development awards. Acad Emerg Med. 2013;20(6):611-7.

20. Choo EK, Kass D, Westergaard M, et al. The development of best practice recommendations to support the hiring, recruitment, and advancement of women physicians in emergency medicine. Acad Emerg Med. 2016;23(11):1203-9.

21. Mumma BE, Chang AM, Kea B, et al. Career development awards in emergency medicine: resources and challenges. Acad Emerg Med. 2017;24(7):855-63.

22. Welch J, Sawtelle S, Cheng D, et al. Faculty mentoring practices in academic emergency medicine. Acad Emerg Med. 2017;24(3):362-70.

23. Spence JP, Buddenbaum JL, Bice PJ, et al. Independent investigator incubator (I): a comprehensive mentorship program to jumpstart productive research careers for junior faculty. BMC medical education. 2018;18(1):186.

24. Freel SA, Smith PC, Burns EN, et al. Multidisciplinary mentoring programs to enhance junior faculty research grant success. Acad Med. 2017;92(10):1410-5.

25. Good M, McElroy SJ, Berger JN, et al. Limited achievement of NIH research independence by pediatric K award recipients. Pediatr Res. 2018;84(4):479-80.