Author	Affiliation
Michelle D. Lall, MD, MHS	Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia
Theodore J. Gaeta, DO, MPH	New York-Presbyterian Brooklyn Methodist Hospital, Department of Emergency Medicine, New York, New York
Arlene S. Chung, MD, MACM	Maimonides Medical Center, Department of Emergency Medicine, Brooklyn, New York
Erin Dehon, PhD	University of Mississippi Medical Center, Department of Emergency Medicine, Jackson, Mississippi
William Malcolm, MD	Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia
Adam Ross, MD	University of Louisville School of Medicine, Department of Emergency Medicine, Louisville, Kentucky
David P. Way, MEd	The Ohio State University College of Medicine, Department of Emergency Medicine, Columbus, Ohio
Lori Weichenthal, MD	University of San Francisco-Fresno, Department of Emergency Medicine, Fresno, California
Nadine T. Himelfarb, MD	Alpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode Island

Introduction
Methods
Results
Limitations
Conclusion
Supplementary material

INTRODUCTION

The word “burnout” was originally used by Herbert Freudenberger in 1974 to describe a state of emotional fatigue that was becoming more prevalent during the free clinic movement, attributed to the mismatch of resources to the needs of patients.¹ Thought to be a reaction to chronic emotional and interpersonal workplace stressors, it is a three-dimensional syndrome consisting of emotional exhaustion, depersonalization or cynicism, and a sense of reduced personal accomplishment.² Exhaustion from increased workloads and extended work hours combined with the stress of cognitive decision-making in the setting of emotionally-charged situations contribute to physician burnout.³ These causes of physician burnout have, in recent years, been only exacerbated by increased clerical workload from electronic health records and reduced sense of work efficacy.⁴

The first national study of burnout in United States (U.S.) physicians was conducted in 2011 across all specialty disciplines. From a sample of over 7,000 physicians, approximately 46% reported at least one symptom of burnout and only 49% reported satisfaction with their work-life balance. Variability was noted across medical specialties, with the highest rates of burnout noted among physicians at the front lines of access to care, such as primary care and emergency medicine (EM). This study further compared U.S. physicians to working adults in non-medical matched cohorts and concluded that physicians comparatively had both more symptoms of burnout and more job dissatisfaction than their non-physician peers.⁵

In 2014, rates of physician burnout and job dissatisfaction were compared with the results from 2011 and both were discovered to be on the rise, with 55% of physicians having reported one symptom of burnout and only 41% reporting satisfaction with work-life balance.⁶ The U.S. adult working population had not seen the same increased rates of burnout and dissatisfaction in the same amount of time, thus further increasing this disparity between physicians and non-physician working adults. This pattern of burnout has not only been identified in attending physicians, but also in resident physicians and medical students.^7,8

The prevalence of burnout in the physician population is significant when taken in consideration with the effects that it has upon physicians as individuals, the patient population that physicians serve, and the institution of medicine itself. Physicians who suffer from burnout have higher rates of substance abuse, personal relationship problems, anxiety, and depression.^9-11 These same physicians are more likely to self-report performing suboptimal patient care practices, such as admitting or discharging patients early, not offering options or answering questions, ordering more tests, not treating patients’ pain, not communicating important handoffs, and not discussing plans with staff.¹² Burnout has additionally been identified as a risk factor for higher rates of medical errors, patient safety errors, and mortality ratios among hospitalized patients.^3,13,14 At the institutional level, physician burnout has been linked with reduction in clinical care hours, which threatens to intensify the projected shortage of physicians in the year 2025.^15,16

As burnout is studied more, other variables such as depression, anxiety, and stress have been identified beyond emotional exhaustion, adding to the complexity of this syndrome. There is significant overlap in symptoms between burnout and depression and anxiety. Physicians reporting burnout are at greater risk for depression and anxiety. While there is an association, suffering from burnout does not equate to a clinical diagnosis of depression.¹⁷ It is important to note that depression and anxiety remain mental disorders well-defined in the Diagnostic and Statistical Manual of Mental Disorders, fifth edition, (DSM-V) published by the American Psychiatric Association (APA) whereas burnout remains a work-related, non-DSM defined syndrome. Given that burnout is defined as a condition resulting from severe stress relative to one’s own emotional and cognitive reserves, stress has been determined to be a considerable variable in burnout assessment. Stress arising from uncertainty, risk of poor outcomes, and high-stakes environments in medical practice often leads to anxiety, which in one study of emergency physicians (EP) was the greatest predictor of career burnout.¹⁸

Summary

Physician burnout is an increasingly prevalent crisis in our healthcare system, has become a focus of multiple medical organizations, and has been highlighted in both the popular media and the medical literature. The highest rates of burnout among physicians are among those specializing in primary care and EM. In the EM literature, research has shown that faculty have a poor ability to accurately identify burnout in trainees and that additional education is needed on methodology of trainee assessment.¹⁹ Accurate measurement is key to conducting needs assessments, developing appropriate interventions to problems, and ongoing monitoring.²⁰

Numerous assessment tools are available to the EP. The goal of the Assessment Tools Workgroup, a sub-committee of the Council of Emergency Medicine Residency Directors (CORD) Resilience Committee, was to research and summarize the various assessment tools available for burnout and related factors and compile them as a collated resource. This is the first resource available in this series and will focus on assessment tools to measure burnout and other negative states. For assessment tools related to well-being, resilience, and positive states, please refer to “Assessment of Physician Well-being Part Two: Beyond Burnout.”

METHODS

The instruments included in this article are the result of a scoping review of English-language publications with abstracts indexed in PubMed, Web of Science, and MedEd Portal within the past 10 years. Searches were based on the main Medical Library Subject Heading (MeSH) terms “burnout,” “anxiety,” and “depression.”

In addition to the search on the main term, subheadings included the following: each in quotes measurement, assessment, evaluation, diagnosis, education, etiology, trends, derivation, validation, tool, instrument, scale, measure, survey, or questionnaire and resident, residency, intern, internship, medical student, clerk, attending, physician, and clinician. A complete listing of search terms can be found in Appendix 1. This search was augmented by reviewing article reference lists and performing further citation searches. We did not include instruments cited only in abstracts or as reports of meetings.

Abstractors performed a comprehensive review of the identified assessment tools. Details of all scales and where they can be found are presented in Table and Appendix 2. The tools identified as most relevant, accessible, and practical in evaluating EP well-being were included for further review. The tools were selected by multiple abstractors. Abstractors worked in groups of two or three focused on one subject (e.g., burnout or depression). Discrepancies between abstractors were reviewed by either the first, second, or senior author. Consensus between at least two reviewers was required for an instrument to be included here.

Our primary inclusion criteria was use of the tool in a physician population in the medical literature. Exclusion criteria included tools that either were not used in a physician population or were not cited in the medical literature relating to physicians more than 2–3 times. Excluded tools that did not meet these two criteria are referenced in Appendix 2. The figure illustrates the search algorithm and tool-selection process. The articles reviewed were organized by the subcategory of the tool (e.g., burnout tools), then by individual tool, and finally, by the populations for which the tool had been used.

A summary of the scale’s purpose, structure, and evidence of its psychometric properties were derived from the original source references. Due to the varied psychometric properties of each tool, abstractors relied on the reported validity and reliability from the source manuscripts. Where available, published cutoff scores are provided for guidance, although their validity or utility in other clinical or research contexts should not be assumed. Where psychometric properties were not explicitly described in the primary sources, potential users may need to check for any subsequent information pertaining to reliability and validity.

The order of presentation is based on the following two subsections: Burnout Tools; and Depression and Anxiety Tools. The following comments and discussions should be read in conjunction with the details reported in Table and Appendix 2, as well as with the recommendations provided at the end of the review.

RESULTS

Burnout Tools

In the mid-1970s a group of researchers led by Christina Maslach began to seriously consider the complex and often difficult relationship that people in helping professions have with their work and the subsequent impact on their health and social networks. These researchers conceptualized burnout as a psychological syndrome in response to chronic interpersonal stressors on the job defined in three key dimensions: overwhelming exhaustion; feelings of cynicism and detachment from the job; and sense of ineffectiveness and lack of accomplishment.²¹

Maslach Burnout Inventory

The Maslach Burnout Inventory (MBI) is a self-assessment tool to measure experienced burnout in individuals. Across a wide range of demographics and occupations, the tool has demonstrated reliability, convergent validity, and discriminant validity.^2,22 The original and most widely used version of the MBI is known as the MBI-Human Services Survey (MBI-HSS). The MBI-HSS scores participants on three distinct but inter-related subscales: emotional exhaustion; depersonalization; and diminished personal accomplishment. While the authors of the MBI consider burnout as existing on a continuum rather than as a dichotomous state, they provide population norms for some groups as a benchmark for comparing scores. The MBI suggests burnout for professionals scoring in the high range on emotional exhaustion, in the high range for depersonalization, and in the low range for personal accomplishment. Official score reports also contain information on reducing burnout and resources for seeking help.²

Studies on physician burnout have almost exclusively used the MBI-HSS as a measurement tool due to the large body of literature supporting its reliability.^2,22 This includes many of the often-cited, population-based studies,^5,33,34 as well as studies in residents^23-25 and EPs.^25,26 Consequently, the MBI-HSS has in many ways become the preferred assessment tool for a wide variety of uses, such as evaluating the effectiveness of wellness programs, faculty and resident surveillance, and demographic trends.

While the MBI-HSS has a number of strengths, there are also some limitations to its use. The high cost of administration may limit access. Users of the MBI must also consider that burnout has clear discriminant validity.^2,22 In other words, it is truly a distinct phenomenon from other established constructs, such as depression and job dissatisfaction, and should not be used as a comprehensive catchall for determining individual or population mental health. It is important to note that the MBI was not normed on physicians-in-training. Attending physicians were sampled for the normative data. Finally, the MBI does not consider non-professional confounders of burnout, such as child care demands, the schedule and support of partners, life events, or financial concerns.²⁷

Well-being is a complicated and multidimensional construct, and the simple absence of burnout as determined by the MBI does not necessarily equate overall well-being. Nevertheless, the MBI-HSS remains one of the most recognized, widely used, well-validated, and reputable tools in the toolbox for assessing occupational burnout in physicians and residents.

Single-Item Emotional Exhaustion and Depersonalization Scale

Because the MBI is a 22-item instrument, its use may be constrained by the time required for completion. To address this limitation, Shanafelt and colleagues created the single-item emotional exhaustion (EE) and depersonalization (DP) scale.²⁸ These authors used the following two questions from the MBI: “I feel burned out from my work,” and “I have become more callous towards people since I took this job.” Multiple studies have significantly correlated these two questions with the EE and DP subscales of the MBI, respectively.

The single-item EE and DP scale has been well-validated in physicians, medical students, and residents with very large sample sizes.²⁹ The scale is brief and free to use. However, some authors have raised concerns regarding the reliability of single-item surveys.³⁰ In general, however, models that used the single-item EE and DP scale did show consistency with those who used the full 22-item MBI.²⁹ This scale may be most useful for larger surveys in which only a few questions can be dedicated to burnout.

Copenhagen Burnout Inventory

The creators of the Copenhagen Burnout Inventory (CBI) felt the MBI could only apply to those who do “people work,” and thus could not be extrapolated to those who do not explicitly work with clients. They reported that the personal burnout section of the CBI could be applied to anyone, regardless of whether they worked with people or were employed at all.³¹ During their pilot in Danes, respondents reacted negatively toward phraseology of the MBI questions, some of which did not translate well into Danish, and the CBI creators thus sought to create a different tool altogether. The CBI was originally used in the human services sector with multiple professions, only one of which was hospital physicians. There have since been studies completed using the CBI in anesthesiologists and critical care physicians, as well as EPs. These have yielded consistent, valid, and reliable results measuring burnout.^32-34

The personal burnout section asks how tired or exhausted the individual is on both physical and emotional levels, as well as how often an individual feels weak and susceptible to illness. The work-related burnout category asks to what degree an individual’s physical and psychological fatigue and exhaustion are related to his or her work. The authors stressed that they were not looking at causality, but merely how much the respondent attributed his or her stress/burnout to work. Comparison of work-related burnout and personal burnout then allows causal assessment of an individual’s burnout, be it due to work or other non-work factors such as family or health issues. The client-related burnout section determines if the respondent’s burnout is due to people-oriented work focus.³¹

The CBI, unlike the MBI, is open access and free to use. Additionally, it performs very well in assessing burnout, and also has the added benefit of looking at burnout in different aspects of an individual’s life. It has been translated into multiple languages and has been used in the physician population.³⁵ Like other tools for assessing burnout, it should not be used to measure depression or an individual’s overall well-being as these are different and complex phenomena. Overall though, the CBI is a helpful tool for evaluating burnout and one that could easily be substituted for the MBI with the added benefit of being free to use.

Utrecht Work Engagement Scale

The Utrecht Work Engagement Scale (UWES) is derived from positive psychology – the study of human strength and optimal functioning. In contrast to the MBI, the UWES measures positive feelings such as vigor, dedication, and absorption in one’s job.³⁶ The UWES attempts to measure burnout by measuring the opposite of burnout, with the underlying assumption that engaged workers have a positive, fulfilling, work-related state of mind. Multiple studies have demonstrated that work engagement is significantly inversely related to burnout.³⁷ It should be noted that the same group of researchers who developed the MBI also developed the UWES.

The UWES has been studied in very large populations and has been validated in non-U.S. physicians.^38-40 It is also available in multiple languages. However, the normative values are based on a general, Dutch, working population. The UWES is free, easy to use, and can be repeated to monitor progress. This scale could easily be combined with the MBI or the single-item EE and DP scale to measure both physician engagement as well as burnout in order to improve the work environment.

Jefferson Scale of Physician Empathy

Empathy is the ability to share and understand the feelings of another. Many neuroscientists believe that empathy is hardwired in human beings and essential to our survival.⁴¹ Patients want genuine empathy from doctors and doctors want to provide it, but there is a tension in medicine between being able to maintain a healthy detachment from patients while still being able to connect with them.⁴² One of the three key components of burnout is depersonalization, which is not only an inability to feel empathy for others but a loss of connection with oneself. Thus, the ability to measure and monitor empathy in healthcare professionals is important to assessing the degree of depersonalization, one of the major components of burnout.

The Jefferson Scale of Physician Empathy is a validated measure of empathy in healthcare professionals. If decreased empathy, in the form of depersonalization, is a hallmark of burnout, then the routine monitoring of the empathy of healthcare professionals could help to identify loss of compassion that could contribute to burnout.

Depression and Anxiety Tools

Compared to peers in other fields, medical students and residents experience significantly higher levels of depression. A large study of medical students and residents found that over half screened positive for depression and 8–9% screened positive for suicidal ideation within the prior 12 months.²³ Another systematic review of 54 studies involving resident physicians (n = 17,560) found that between 20.9% and 43.2% screened positive for depression. Although numerous studies have examined depressive symptoms among medical residents in various specialties, few studies have focused on EM specifically.⁴³

Anxiety can be defined as an acute emotional response to stressful conditions (state anxiety), or as a personality characteristic (trait anxiety) that can also be thought of as a predisposition to respond to external threats in fixed ways. Beck’s model of psychopathology suggests that anxiety and depression are separate but related constructs and can be measured independently. In Beck’s definition, anxiety refers to negative feelings that are specific to certain situations, whereas depression involves more absolute and pervasive negative feelings.^44,45 As opposed to the assessments for burnout, the tools for measuring depression and anxiety are validated diagnostic clinical tools widely used for the purpose of identifying DSM-V illnesses as defined by the APA. They are presented with a focus on prevalence and trends within physician populations.

Beck Depression Inventory II

The Beck Depression Inventory (BDI-II) is one of the most widely used, self-report measures of depression. The purpose of the BDI-II is to assess the existence and severity of symptoms of depression. Both the total BDI-II score and the single suicidal-ideation items have demonstrated accuracy in predicting suicide attempts and death by suicide.^46-50 The BDI-II has been used in several studies examining depressive symptoms among medical residents in the U.S. and other countries.^51-53

Data from seven studies of resident physicians found the overall prevalence of depression to be 26.6% when using the BDI-II with a cutoff score of ≥ 10. Prevalence of depression was significantly lower among U.S. resident physicians (10.7%) compared to non-U.S. resident physicians (44.6%). The prevalence of depression was higher in more recent studies, and no association was found between prevalence and specialty or post-graduate year (PGY) training level.^54-60 Chronic sleep deprivation was associated with depression.⁶¹

Center for Epidemiologic Studies Depression Scale

The Center for Epidemiologic Studies Depression Scale (CES-D) is commonly used in clinical settings to screen for depression and in research studies with clinical and nonclinical samples.^62,63 Data from seven different studies of resident physicians found the overall prevalence of depression to be 25.6% when using the CES-D and a cutoff score of ≥ 16. Data from two other studies of resident physicians used a higher cutoff of ≥ 19 and found the prevalence to be 33.4%.^64-70 The CES-D is the only measure used in studies examining depressive symptoms of EM residents. A single-site study of 51 EM residents found the prevalence of depression to be 12.1% when using the CES-D and a cutoff of ≥ 15.^71,72 Depression was not associated with gender, rotation type, PGY level, or number of hours worked.⁷³

Primary Care Evaluations of Mental Disorders: Patient Health Questionnaire and Generalized Anxiety Disorder Instrument

The full Primary Care Evaluation of Mental Disorders (PRIME-MD) and subsequent Patient Health Questionnaire (PHQ) and Generalized Anxiety Disorder Instrument (GAD) were developed as tools for primary care providers to screen for a range of psychiatric disorders, including depression and anxiety. A subsequent shorter version, the Public Health Questionnaire-9 (PHQ-9) is a self-report version of the PRIME-MD depression screen.^74,75 The GAD is the instrument designed to screen for generalized anxiety disorder.⁷⁶ All versions of the PHQ and GAD are considerably shorter and faster to administer than the original PRIME-MD.

The PHQ-9 was used in several studies examining depressive symptoms among residents. Data from four studies of resident physicians found the overall prevalence of depression to be 20.9% when using the PHQ-9. When using a slightly modified version of the PHQ-2 the prevalence of depression among resident physicians was 43.2 %.^77-80 Internal medicine residents who screened positive for depression were more likely to experience burnout⁸¹ and to report making a medical error.⁸²

The PHQ instruments are free and easy to use. With very little training and preparation, clinicians of all types can use these instruments to screen for common psychiatric disorders with relative accuracy. Reliability varies by form, and inter-rater reliability was established on the original PHQ by comparing the use of the instrument by a clinician with assessment of the patient by a mental health professional.⁸³ The PHQ instruments do measure depression and anxiety as disorders rather than responses to stress, making it a less favorable instrument for assessing across a physician population.

Beck Anxiety Inventory

The Beck Anxiety Inventory (BAI) is a self-report questionnaire that measures severity of anxiety in adults and adolescents. The instrument was specifically designed to “minimize confounding of symptoms of depression.”⁴⁵ The BAI is relatively brief and easy to administer in a short period of time and is most effective as a measurement of somatic symptoms of anxiety.⁸⁴ The instrument does not assess other symptoms of anxiety such as worry or other cognitive aspects and thus may underestimate the presence of anxiety.⁸⁵ The reliability and validity evidence for this instrument has been widely studied; however, the BAI has not been widely studied in medical professional populations.^86,87

State-Trait Anxiety Inventory

The State-Trait Anxiety Inventory (STAI) was derived from the Minnesota Multiphasic Personality Inventory (MMPI). The instrument is designed to measure the presence and severity of current symptoms of anxiety and a generalized propensity to be anxious. It is a self-report questionnaire containing two subscales, one for assessing state anxiety (S-Anxiety), questions about how one feels “right now,” and one for assessing trait anxiety (T-Anxiety), questions about how one generally feels.^45,93 The STAI is one of the most widely researched and used measures of general anxiety. The instrument measures both S-anxiety, which is more likely to be prevalent in the emergency department, but it can also measure T-anxiety, illuminating patterns of response to anxiety that may be unhealthy. Because of the overlap of the T-anxiety scale with depression and depressive symptoms, this instrument is limited, having a difficult time achieving respectable levels of discriminant validity. In other words, the T-anxiety scale correlates more with other depression instruments than it does with other measures of anxiety.^94-97

Second Victim Experience Support Tool

The Second Victim Experience Support Tool (SVEST) consists of seven subscales that measure psychological distress, physical distress, four types of support, and professional self-efficacy. SVEST also has two outcome measures related to the second victim’s job: intention to leave and absenteeism. The phrase “second victim” refers to healthcare providers who experience an adverse event during the care of their patient, who may be considered the “first victim.”⁸⁸ Medical errors or inadvertent injuries to the patient during care may cause the caregiver to suffer feelings of anxiety, stress, shame, or guilt as a result of adverse clinical event.^89-91 The only major study of the instrument’s psychometric properties was conducted at a pediatric hospital with a very small physician sample.⁹²

LIMITATIONS

There are an overwhelming number of assessment tools available in the literature that can be used to measure the different components of physician well-being. While our literature search was methodical and broad, we acknowledge that we may have missed some key assessment tools. At times, a single author determined the inclusion eligibility of the tools identified in our literature search strategy. However, consensus between at least two reviewers was required for an instrument to be included in this paper.

Assessment tools must be suitable for and validated in the population of interest. A majority of the tools that we found have been used in a physician population but have never been validated in this population. Many of the tools have been designed for and validated in special populations, and their applicability, reliability, and validity in a physician population is not clearly demonstrated in the medical literature. In the absence of independent validation, however, the results of these tools should be interpreted with caution.

Physician well-being is multifactorial, and it is difficult to divide these components purely by topic or sub-category as they have a complex interplay with one another. We have reviewed the tools based on the well-being topics that were most commonly found in the medical literature and that were of highest potential value. There are very few tools that were either designed for use in a physician population or have been validated in physicians. We have highlighted the tools from each topic that are most relevant for use in assessing an EP population.

CONCLUSION

Given the wide range of associated factors and the psychosocial impact of burnout, it seems unlikely that any one tool will be recognized as comprehensive for evaluating physician well-being. It is hoped that the present review will provide guidance on choosing between currently available instruments, whether assessing mentees or testing interventions in the research setting.

Supplementary Material

 

Footnotes

Section Editor: Jeffrey Druck, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Michelle D. Lall, MD, MHS, Emory University School of Medicine, Department of Emergency Medicine, 49 Jesse Hill Jr. Drive SE, Atlanta, GA 30303. Email: michelle.d.lall@emory.edu. 3 / 2019; 20:278 – 290

Submission history: Revision received July 10, 2018; Submitted January 24, 2019; Accepted January 21, 2019

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Freudenberger H. Staff burnout. J Soc Issues. 1974;30(1):159-65.

2. Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory. 1996.

3. Wallace JE, Lemaire JB, Ghali WA. Physician wellness: a missing quality indicator. Lancet. 2009;374(9702):1714-21.

4. Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burnout and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91(7):836-48.

5. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172(18):1377-85.

6. Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin Proc. 2015;90(12):1600-13.

7. Prins JT, Gazendam-Donofrio SM, Tubben BJ, et al. Burnout in medical residents: a review. Med Educ. 2007;41(8):788-800.

8. Dyrbye LN, Massie FS, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173-80.

9. Myers MF. The well-being of physician relationships. West J Med. 2001;174(1):30-3.

10. Oreskovich MR, Shanafelt T, Dyrbye LN, et al. The prevalence of substance use disorders in American physicians. Am J Addict. 2015;24(1):30-8.

11. Williams D, Tricomi G, Gupta J, et al. Efficacy of burnout interventions in the medical education pipeline. Acad Psychiatry. 2015;39(1):47-54.

12. Lu DW, Dresden S, McCloskey C, et al. Impact of burnout on self-reported patient care among emergency physicians. West J Emerg Med. 2015;16(7):996-1001.

13. Shanafelt TD, Balch CM, Bechamps G, et al. Burnout and medical errors among American surgeons. Ann Surg. 2010;251(6):995-1000.

14. Wallace JE, Lemaire J. Physician well being and quality of patient care: an exploratory study of the missing link. Psychol Health Med. 2009;14(5):545-52.

15. Shanafelt TD, Mungo M, Schmitgen J, et al. Longitudinal study evaluating the association between physician burnout and changes in professional work effort. Mayo Clin Proc. 2016;91(4):422-31.

16. The physician workforce: projections and research into current issues affecting the supply and demand. 2008. Available at: https://bhw.hrsa.gov/sites/default/files/bhw/nchwa/projections/physiciansupplyissues.pdf. Accessed January 22, 2019.

17. Bianchi R, Boffy C, Hingray C, et al. Comparative symptomatology of burnout and depression. J Health Psychol. 2013;18(6):782-7.

18. Kuhn G, Goldberg R, Compton S. Tolerance for uncertainty, burnout, and satisfaction with the career of emergency medicine. Ann Emerg Med. 2009;54(1):106-13.

19. Lu DW, Lank PM, Branzetti JB. Emergency medicine faculty are poor at predicting burnout in individual trainees: a prospective study. AEM Educ Train. 2017;1:75-8.

20. Dyrbye LN, Trockel M, Frank E, et al. Development of a research agenda to identify evidence-based strategies to improve physician wellness and reduce burnout. Ann Intern Med. 2017;166(10):743-5.

21. Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annu Rev Psychol. 2001;52:397-422.

22. Maslach C, Jackson SE. The measurement of experienced burnout. J Occup Behav. 1981;2:99-113.

23. Dyrbye LN, West CP, Satele D, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014;89(3):443-51.

24. West CP, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306(9):952-60.

25. Takayesu K, Ramoska EA, Clark TR, et al. Factors associated with burnout during emergency medicine residency. Acad Emerg Med. 2014;21(9):1031-5.

26. Goldberg R, Boss RW, Chan L, et al. Burnout and its correlates in emergency physicians: four years’ experience with a wellness booth. Acad Emerg Med. 1996;3(12):1156-64.

27. Eckleberry-Hunt J, Kirkpatrick H, Barbera T. The problems with burnout research. Acad Med. 2018;93(3):367-70.

28. West CP, Dyrbye LN, Sloan JA, et al. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. J Gen Intern Med. 2009;24(12):1318-21.

29. West CP, Dyrbye LN, Satele DV, et al. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment. J Gen Intern Med. 2012;27(11):1445-52.

30. Hays RD, Reise S, Calderón JL. How much Is lost in using single items?. J Gen Intern Med. 2012;27(11):1402-3.

31. Kristensen TS, Borritz M, Villadsen E, et al. The Copenhagen Burnout Inventory: a new tool for the assessment of burnout. Work Stress. 2005;19(3):192-207.

32. Molinero Ruiz E, Basart Gómez-Quintero H, Moncada Lluis S. Validation of the Copenhagen Burnout Inventory to assess professional burnout in Spain. Rev Esp Salud Publica. 2013;87(2):165-79.

33. Ilić IM, Arandjelović MŽ, Jovanović JM, et al. Relationships of work-related psychosocial risks, stress, individual factors and burnout – Questionnaire survey among emergency physicians and nurses. Med Pr. 2017;68(2):167-78.

34. Kassam A, Horton J, Shoimer I, et al. Predictors of well-being in resident physicians: a descriptive and psychometric study. J Grad Med Educ. 2015;7(1):70-4.

35. Doppia MA, Estryn-Béhar M, Fry C, et al. Burnout in French doctors: a comparative study among anaesthesiologists and other specialists in French hospitals (SESMAT study). Ann Fr Anesth Reanim. 2011;30(11):782-94.

36. Schaufeli WB, Salanova M, Gonzalez-Romá V, et al. The measurement of engagement and burnout: a confirmative analytic approach. J Happiness Stud. 2002;3:71-92.

37. Schaufeli WB, Bakker AB. Utrecht Work Engagement Scale, Preliminary Manual, version 1.1. 2004.

38. Schaufeli WB, Bakker AB, Salanova M. The measurement of work engagement with a short questionnaire: A cross-national study. Educ Psychol Meas. 2006;66(4):701-16.

39. Nerstad CG, Richardsen AM, Martinussen M. Factorial validity of the Utrecht Work Engagement Scale (UWES) across occupational groups in Norway. Scand J Psychol. 2010;51(4):326-33.

40. Seppala P, Mauno S, Feldt T, et al. The construct validity of the Utrecht Work Engagement Scale: multisample and longitudinal evidence. J Happiness Stud. 2009;10:459.

41. Corradini A, Antonietti A. Mirror neurons and their function in cognitively understood empathy. Conscious Cogn. 2013;22(3):1152-61.

42. Halpern J. What is clinical empathy?. J Gen Intern Med. 2003;18(8):670-4.

43. Mata DA, Ramos MA, Bansal N, et al. Prevalence of depression and depressive symptoms among resident physicians: a systematic review and meta-analysis. JAMA. 2015;314(22):2373-83.

44. Beck AT, Epstein N, Brown G, et al. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893-7.

45. McDowell I. Measuring health: A guide to rating scales and questionnaires. Chapter . 2006:273-327.

46. Steer RA, Ball R, Ranieri WF, et al. Further evidence for the construct validity of the Beck Depression Inventory-II with psychiatric outpatients. Psychol Rep. 1997;80:443-6.

47. The construct validity of the Utrecht Work Engagement Scale: multisample and longitudinal evidence. J Happiness Stud Meas Eval Couns Dev. 1997;30:128-36.

48. Dozois DJA, Dobson KS, Ahnberg JL. A psychometric evaluation of the Beck Depression Inventory-II. Psychol Assess. 1998;10(2):83-9.

49. Brown GK, Beck AT, Steer RA, et al. Factors for suicide in psychiatric outpatients: a 20-year prospective study. J Consult Clin Psychol. 2000;68(3):371-7.

50. Green KL, Brown GK, Jager-Hyman S, et al. The predictive validity of the Beck Depression Inventory Suicide Item. J Clin Psych. 2015;6(12):1683-6.

51. Kirsling RA, Kochar MS, Chan CH. An evaluation of mood states among first-year residents. Psychol Rep. 1989;65(2):355-66.

52. Demir F, Ay P, Erbaş M, et al. The prevalence of depression and its associated factors among resident doctors working in a training hospital in Istanbul [in Turkish]. Turk Psikiyatri Derg. 2007;18(1):31-7.

53. Waldman SV, Diez JCL, Arazi HC, et al. Burnout, perceived stress, and depression among cardiology residents in Argentina. Acad Psychiatry. 2009;33(4):296-301.

54. Costa EF, Santana YS, Santos AT, et al. Depressive symptoms among medical intern students in a Brazilian public university [in Portuguese]. Rev Assoc Médica Bras. 2012;58(1):53-9.

55. Al-Maddah EM, Al-Dabal BK, Khalil MS. Prevalence of sleep deprivation and relation with depressive symptoms among medical residents in King Fahd University Hospital, Saudi Arabia. Sultan Qaboos Univ Med J. 2015;15(1):e78-84.

56. Velásquez-Pérez L, Colin-Piana R, González-González M. Coping with medical residency: depression burnout [in Spanish]. Gac Med Mex. 2013;149(2):183-95.

57. Cubero DIG, Fumis RRL, de Sá TH, et al. Burnout in medical oncology fellows: a prospective multicenter cohort study in Brazilian institutions. J Cancer Educ. 2016;31(3):582-7.

58. Godenick MT, Musham C, Palesch Y, et al. Physical and psychological health of family practice residents. Fam Med. 1995;27(10):646-51.

59. Hainer BL, Palesch Y. Symptoms of depression in residents: a South Carolina Family Practice Research Consortium study. Acad Med. 1998;73(12):1305-10.

60. Jiménez-López JL, Arenas-Osuna J, Angeles-Garay U. Depression, anxiety and suicide risk symptoms among medical residents over an academic year [in Spanish]. Rev Med Inst Mex Seguro Soc. 2015;53(1):20-8.

61. Rosen IM, Gimotty PA, Shea JA, et al. Evolution of sleep quantity, sleep deprivation, mood disturbances, empathy, and burnout among interns. Acad Med. 2006;81(1):82-5.

62. Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. App Psychol Meas. 1977;1:385-401.

63. Eaton WW, Muntaner C, Smith C, et al. Center for Epidemiologic Studies Depression Scale: Review and revision (CESD and CESD-R). The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. 2004:27-51.

64. Hsu K, Marshall V. Prevalence of depression and distress in a large sample of Canadian residents, interns, and fellows. Am J Psychiatry. 1987;144(12):1561-6.

65. Becker JL, Milad MP, Klock SC. Burnout, depression, and career satisfaction: cross-sectional study of obstetrics and gynecology residents. Am J Obstet Gynecol. 2006;195(5):1444-9.

66. Yi MS, Luckhaupt SE, Mrus JM, et al. Religion, spirituality, and depressive symptoms in primary care house officers. Ambul Pediatr. 2006;6(2):84-90.

67. Sakata Y, Wada K, Tsutsumi A, et al. Effort-reward imbalance and depression in Japanese medical residents. J Occup Health. 2008;50(6):498-504.

68. Goebert D, Thompson D, Takeshita J, et al. Depressive symptoms in medical students and residents: a multischool study. Acad Med. 2009;84(2):236-41.

69. Govardhan LM, Pinelli V, Schnatz PF. Burnout, depression and job satisfaction in obstetrics and gynecology residents. Conn Med. 2012;76(7):389-95.

70. Reuben DB. Depressive symptoms in medical house officers: effects of level of training and work rotation. Arch Intern Med. 1985;145(2):286-8.

71. Revicki DA, Gallery ME, Whitley TW, et al. Impact of work environment characteristics on work-related stress and depression in emergency medicine residents: a longitudinal study. J Community Appl Soc Psychol. 1993;3(4):273-84.

72. Katz ED, Sharp L, Ferguson E. Depression among emergency medicine residents over an academic year. Acad Emerg Med. 2006;13(3):284-7.

73. Ito M, Seo E, Ogawa R, et al. Can we predict future depression in residents before the start of clinical training?. Med Educ. 2015;49(2):215-23.

74. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Gen Intern Med. 2001;16(9):606-13.

75. Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41(11):1284-92.

76. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-97.

77. Sen S, Kranzler HR, Krystal JH, et al. A prospective cohort study investigating factors associated with depression during medical internship. Arch Gen Psychiatry. 2010;67(6):557-65.

78. Earle L, Kelly L. Coping strategies, depression, and anxiety among Ontario family medicine residents. Can Fam Physician. 2005;51:242-3.

79. Al-Ghafri G, Al-Sinawi H, Al-Muniri A, et al. Prevalence of depressive symptoms as elicited by Patient Health Questionnaire (PHQ-9) among medical trainees in Oman. Asian J Psychiatr. 2014;8:59-62.

80. Stoesser K, Cobb NM. Self-treatment and informal treatment for depression among resident physicians. Fam Med. 2014;46(10):797-801.

81. Campbell J, Prochazka AV, Yamashita T, et al. Predictors of persistent burnout in internal medicine residents: a prospective cohort study. Acad Med. 2010;85(10):1630-4.

82. West CP, Tan AD, Habermann TM, et al. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;302(12):1294-300.

83. Spitzer RL, Kroenke K, Williams JB, et al. Validation and utility of a self-report version of PRIME-MD: the PHQ Primary Care Study. JAMA. 1999;282(18):1737-44.

84. Fydrich T, Dowdall D, Chambless DL. Reliability and validity of the Beck Anxiety Inventory. J Anx Disord. 1993;6(1):55-61.

85. Waller Niels G. Review of the Beck Anxiety Inventory. Buros Mental Measurement Yearbook 13. 1993.

86. Creamer M, Foran J, Bell R. The Beck Anxiety Inventory in a nonclinical sample. Behav Res Ther. 1995;33(4):477-85.

87. Osman A, Barrios FX, Aukes D, et al. The Beck Anxiety Inventory: psychometric properties in a community population. J Psychopath Behav Assess. 1993;15(4):287-97.

88. Wu AW. Medical error: the second victim. The doctor who makes the mistake needs help too. BMJ. 2000;320(7237):726-7.

89. Devencenzi T, O’Keefe J. To err is human: supporting the patient care provider in the aftermath of an unanticipated clinical event. Int J Emerg Ment Health. 2006;8(2):131-5.

90. Wolf ZR. Stress management in response to practice errors: Critical events in professional practice. PA PSRS Patient Saf Advis. 2005;2(4):1-4.

91. Aasland OG, Forde R. Impact of feeling responsible for adverse events on doctors’ personal and professional lives: the importance of being open to criticism from colleagues. Qual Saf Health Care. 2005;14(1):13-7.

92. Burlison JD, Scott SD, Browne EK, et al. The Second Victim Experience and Support Tool: validation of an organizational resource for assessing second victim effects and the quality of support resources. J Patient Saf. 2017;13(2):93-102.

93. Spielberger CD. State-Trait Anxiety Inventory. 1983.

94. Chlan L, Savik K, Weinert C. Development of a shortened state anxiety scale from the Spielberger State-Trait Anxiety Inventory (STAI) for patients receiving mechanical ventilatory support. J Nurs Meas. 2003;11(3):283-93.

95. Kaipper MB, Chachamovich E, Hidalgo MP, et al. Evaluation of the structure of Brazilian State-Trait Anxiety Inventory using a Rasch psychometric approach. J Psychosom Res. 2010;68(3):223-33.

96. Tluczek A, Henriques JB, Brown RL. Support for the reliability and validity of a six-item state anxiety scale derived from the State-Trait Anxiety Inventory. J Nurs Meas. 2009;17(1):19-28.

97. Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med. 2006;81(4):354-73.