| Author | Affiliation |
|---|---|
| Fiona E. Gallahue, MD | The University of Washington, Division of Emergency Medicine, Seattle, Washington |
| Amy E. Betz, MD | The University of Washington, Division of Emergency Medicine, Seattle, Washington |
| Jeffrey Druck, MD | The University of Colorado, Department of Emergency Medicine, Aurora, Colorado |
| Jonathan S. Jones, MD | The University of Mississippi Medical Center, Department of Emergency Medicine, Jackson, Mississippi |
| Boyd Burns, DO | The University of Oklahoma School of Community Medicine, Tulsa, Oklahoma |
| Gene Hern, MD, MS | Alameda Hospital System-Highland Hospital, Department of Emergency Medicine, Oakland, California |
Methods
Results
Discussion
Limitations
Conclusion
ABSTRACT
This study aimed to assess current education and practices of emergency medicine (EM) residents as perceived by EM program directors to determine if there are deficits in resident discharge handoff training. This survey study was guided by the Kern model for medical curriculum development. A six-member Council of EM Residency Directors (CORD) Transitions of Care task force of EM physicians performed these steps and constructed a survey. The survey was distributed to program residency directors via the CORD listserve and/or direct contact. There were 119 responses to the survey, which were collected using an online survey tool. Over 71% of the 167 American College of Graduate Medical Education (ACGME) accredited EM residency programs were represented. Of those responding, 42.9% of programs reported formal training regarding discharges during initial orientation and 5.9% reported structured curriculum outside of orientation. A majority (73.9%) of programs reported that EM residents were not routinely evaluated on their discharge proficiency. Despite the ACGME requirements requiring formal handoff curriculum and evaluation, many programs do not provide formal curriculum on the discharge transition of care or evaluate EM residents on their discharge proficiency.
METHODS
This survey study was guided by the six-step Kern model for medical curriculum development.13 A similar study by the CORD Transitions of Care taskforce membership was performed on general handoff training, which provided a foundation for this study.14 The first two stages–problem identification and creation of a targeted needs assessment–were the goals of this study. The latter four stages of the Kern model include determining the goals and objectives of the curriculum, developing educational strategies for teaching the curriculum, implementing the curriculum, and receiving feedback on and evaluating the curriculum. The application of this model provides the opportunity to eventually develop a discharge handoff curriculum for EM residents. Establishing validity evidence was an important consideration throughout the process; validity evidence comes in the form of content, response process, internal structure, relationship to other variables, and consequences.15 We conducted a review of discharge literature to survey common practices and sources of error and to discover EM discharge education techniques. One identified problem was the lack of formal emergency discharge education available or required during residency training. The survey went through a thorough development process using the expertise of those involved in its development to contribute to the validity of its content.15 This process was an iterative approach by the authors who are on the Council of Residency Directors in EM (CORD) Transitions of Care Committee (FG, JD, JJ, BB, HGH) or have served as a director of Quality Assurance and Improvement (AB). The initial survey was developed by two authors (HGH, JD) based on focus group comments and suggestions from membership during a CORD Transitions of Care Committee meeting (approximately 20 faculty in attendance). We sent the draft survey to the authorship team via SurveyMonkey® (an online survey development cloud-based service) to complete and suggest further content changes. This was repeated twice more until the team felt that the final questionnaire best addressed the areas where knowledge content gaps were identified by the focus group. The survey focused on elucidating current practices of discharge training and clinical practices within the clinical environments, including current education offered, perceptions of best educational practices, methods of resident evaluation and perceived competence of residents. Multiple-choice questions were the primary vehicle for the response process.
The institutional review board at Alameda County Medical Center (Highland Hospital, Oakland, CA) granted exempt approval for this study. Members of CORD were invited to complete the survey electronically. The CORD e-mail listserve is exclusive to educators in EM residency programs and includes associate, assistant and primary residency program directors from the 167 ACGME-accredited EM residency programs. Program leaders were recruited via the CORD listserve from March to April 2014. The survey was opened for six weeks in which 87 identified programs responded. Duplicate responses were reviewed and clarified with program directors directly in April 2015 (22 programs). Two programs requested their duplicate responses be deleted and completed new surveys to accurately represent their current practice. All other programs selected their most accurate survey responses. Direct emails were then sent to program directors with links to the survey in April 2015; 32 additional programs completed the survey and the survey was closed on April 23, 2015. Only programs that identified their program name were included in the study to ensure that all surveyed programs represented ACGME-accredited programs and to avoid potential duplicative responses.
RESULTS
A total of 119 programs were surveyed, making the overall response rate 71.2% of the 167 currently accredited ACGME EM residency training programs. A majority of programs indicated that residents are given informal education regarding discharge training by senior residents and faculty (87.4%); just under half provide formal curriculum at orientation (42.9%) and/or outside of orientation (5.9%). A small percentage of programs offer no training (6.7%) (Table 1). Over half of programs felt that optimal discharge training would be formal curriculum offered at orientation (76, 63.9%) and/or outside of orientation (63, 52.9%).
| Answer options | Percent response (count):Q1 | Percent response (count):Q2 |
|---|---|---|
| No training | 6.7% (8) | 1.7% (2) |
| Specific formal training regarding discharges during initial orientation to the program | 42.9% (51) | 63.9% (76) |
| Structured workshops/classes to teach proper discharge processes during residency (not in orientation) | 5.9% (7) | 52.9% (63) |
| Instruction by attending/senior resident within the clinical environment | 87.4% (104) | 69.7% (83) |
| Distributed educational packets/guides | 9.2% (11) | 18.5% (22) |
| Formal evaluation of residents on competency in performing effective discharges | 14.3% (17) | 58% (69) |
| Informal evaluation of residents on competency in performing effective discharges | 39.5% (47) | 24.4% (29) |
| Other | N/A (3)* | N/A (1)** |
*3 responses: 1) Discuss during Morbidity & Mortality conference. 2) Formal evaluation of effective discharges is covered loosely in chart reviews, call backs are performed intern year. 3) Grand rounds presentation of ED Discharges. **1 response: 1) I don’t know.
Most residency programs reported using a structured discharge system in the ED (100, 84%%), while a small minority report using none (13, 10.9%) or being unsure if this is provided in their main ED (4, 7.6%). For those with a structured process of discharge most reported that this is being performed most of the time (35, 35%) or always (56, 56%). A majority of programs reported providing structured written modifiable written discharge instructions (108, 90.8%) and bidirectional conversations with patients (91, 76.5%) (Table 2). A majority of training programs reported that their current process is safe and effective or extremely safe and effective (78, 66%) but a significant number reported their process to be somewhat safe and effective (41, 34%). Most programs do not formally evaluate their residents for discharge proficiency (88, 73.9%). Of those that reported resident evaluation of discharge proficiency is routinely performed (more than one response was acceptable), 25 programs (21%) report the evaluation is completed as part of required direct observations or other activities, nine (7.6%) reported it is completed through written feedback/evaluation of performance on ED rotations, and two programs (1.7%) reported a formal assessment of discharge proficiency is completed on junior residents as part of a checklist or similarly structured evaluation.
| Answer options | Percent response (count) |
|---|---|
| Provision of any prewritten non-modifiable instructions | 30.3% (36) |
| Provision of any structured written instructions that allow for modification | 90.8% (108) |
| Physician routinely has bidirectional conversation with patient regarding diagnosis, education, prescriptions, follow up and reasons to return to the ED | 76.5% (91) |
| Teach back method (or similar) routinely employed to assess patient understanding of their diagnosis, education, prescriptions, follow up and reasons to return to the ED | 10.9% (13) |
| Final discharge routinely completed by nursing | 68.9% (82) |
| Final discharge routinely completed by physician | 10.9% (13) |
| Final discharge routinely completed by either nursing or physician | 22.7% (27) |
| Other | N/A (2*) |
ED, emergency department *2 responses: 1) Provide follow up physician or clinic. 2) Nursing employs teach back with patients.
All programs reported a variety of tools to assist in the discharge process with only 34, or 43%, of the respondents being satisfied or extremely satisfied with these tools (Table 3). Over two-thirds of programs reported that key elements of discharge conversations, such as diagnosis, education, prescriptions, follow up, return precautions or assessment of understanding, are documented in the physician note (80, 67.2%). This information is not routinely documented in 31 programs (26.1%), and eight respondents were not sure if this is included in their documentation (6.7%).
| Answer options | Percent response (count) |
|---|---|
| Automated reminders within the computer interface | 42.9% (51) |
| Written template or other written aids (badge card checklist) | 24.4% (29) |
| Mnemonics | 1.7% (2) |
| Teach back or similar method | 3.4% (4) |
| None | 38.7% (46) |
| Other | 2* |
*2 responses: 1) Nursing feedback when discharge performed improperly. 2) Pre-populated recommendations from nursing triage such as smoking cessation for smokers, blood pressure re-check for patients with high blood pressure at triage; all patients without a primary care provider are provided a printout of the free and low cost medical, dental and mental health resources in the community.
Over three-quarters (90, 75.6%) of program leadership reported that junior level residents (equal to or less than eight months in the ED) are “somewhat competent” in their discharge competency. Almost a quarter felt that their junior residents were “competent” (28, 23.5%). One program (0.8%) reported their junior residents were extremely competent (1, 0.8%). For senior level residents, described as residents with over eight months experience, their program leadership identified them as competent (83, 68.7%) or extremely competent (26, 21.8%). A minority of programs reported their senior level residents as “somewhat competent” in their discharge skills (10, 8.4%). None of the respondents reported their junior or senior level residents to be incompetent in their discharge abilities.
DISCUSSION
These results provide insight into the discharge educational practices and clinical training experience surrounding discharge of EM residents as reported by their program leadership. Standardized formal training and evaluation is not the current norm at most programs. Most formal training provided to EM residents is at their orientation, and few programs offer formal educational opportunities beyond the first few weeks of training. Since a majority of program leaders indicated that ideal educational practices would include formal training at orientation and/or workshops or classes outside of orientation, programs may value structured education of discharge competencies but may be constrained by other limitations such as faculty time, didactic scheduling or curriculum availability. The same gap was seen between current training and ideal training for evaluation processes. While most programs do not perform formal evaluations on their residents’ discharge competency, a majority of programs identify that this would be an ideal practice. This implies that EM program leadership values formal evaluation of their residents’ discharge competency but they may be constrained by limitations such as a recognized evaluation tool and/or faculty time. These data also suggest that while program leadership values discharge competency training and evaluations, this education may not be valued as a high priority since most programs perceive their senior level residents to be competent. Although it is difficult to fully endorse competency without a standardized evaluation process, there is support that informal evaluation may be valid in identifying residents’ clinical competencies.16
LIMITATIONS
The fact that this study relies on perceptions of program leadership is a major limitation as there is no gold standard to formally measure discharge competency even for programs providing formal evaluation of their residents’ discharge competency. While most program leadership feel that their current process of discharge is “safe and effective,” this perception may be limited. Given that each respondent based their program results on his or her perceptions of the discharge education and performance of residents within the clinical environment, construct underrepresentation and irrelevant variance represent threats to the validity of clinical performance ratings in this study.17 Program leadership could have responded based on too few or incomplete observations of residents’ clinical behavior or responded with low-reliability ratings. Survey, rater and recall bias could have affected these results.
Program leadership relies on general gestalt that their residents are competent in their discharge proficiency since most are not evaluating this competency. Program leadership may not identify any limitations in senior resident discharge competency because there may be larger, more systematic failures of discharge communication with patients and/or caregivers. In this scenario, resident performance of discharge may be at an acceptable level at the departmental level but departmental expectations of patient discharge competency may not be meeting the patient needs to create safe and effective discharges. Standardizing the process of handoffs between providers in the hospital environment has demonstrated improvement in the quality of communication and increased patient safety in the clinical arena.9 Current discharge literature suggests that there may be similar communication improvements to be made in EM around discharge.3-5 That over one-third of programs report their current discharge process is only “somewhat safe and effective” suggest that there may be quality gaps in the current institutionally acceptable patient discharge processes.
The conflation of the concept of “safety” and “effectiveness” may be another limitation of this survey. Programs were asked about safe and effective discharges in a single question. These two concepts may be inappropriately linked together and it may be that safety may exist without being effective and vice versa. This may represent a construct error in the survey design and affected results of this survey.
Lastly, a major limitation of this study was the gap of approximately one year between the surveys of the first cohort of surveyed programs (87) and the second (32). Although it is unlikely that most programs changed their educational practices dramatically within that time period, it is possible.
Further work should focus on the more structured assessment of resident discharge competency through direct observation and evaluation of resident performance to corroborate program leaders’ assessment of resident competence. These evaluation tools should then be validated and studied in a clinical setting with specific process measures and patient outcomes. Following Kern’s six-step model for curriculum development, the next step would be to create specific goals and objectives of the discharge curriculum and develop educational interventions aligned with these goals. Curricular tool suggestions that have been made to structure education and evaluation for provider-to-provider handoffs might be used directly or modified to educate residents in the discharge transitions of care.18
CONCLUSION
The results of this targeted needs assessment indicate a lack of structured training and assessment of resident discharge competency despite current guidelines for formalized training in all handoffs. Although most programs reported senior residents are competent in discharge proficiency, the residents’ training is primarily informal which may lead to significant variability in resident experience and performance. Further research should be aimed at assessing proficiency of resident discharge performance through objective observation with validated evaluation tools. Structured training and assessment recommendations should follow from this research with increased attention to implementing a standard curricular model or toolbox, objective, valid evaluation methods, and identification and management of high-risk discharges.
Footnotes
Section Editor: Sally Santen, MD, PhD
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Address for Correspondence: Fiona E. Gallahue, MD, University of Washington, Harborview Medical Center, 325 Ninth Ave., Division of Emergency Medicine, Box 359702, Seattle, WA 98104. Email: fionagallahue@gmail.com. 11 / 2015; 16:879 – 884
Submission history: Revision received April 30, 2015; Submitted September 4, 2015; Accepted September 16, 2015
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
REFERENCES
1. 2011 National Hospital Ambulatory Medical Care Survey. Available at: http://www.cdc.gov/nchs/. Accessed Apr 23, 2015.
2. Improving the Emergency Department Discharge Process: Environmental Scan Report. Available at: http://www.ahrq.gov/professionals/systems/hospital/edenvironmentalscan/index.html. Accessed Apr 23, 2015.
3. Samuels-Kalow ME, Stack AM, Porter SC. Effective discharge communication in the emergency department. Ann Emerg Med. 2012;60(2):152-9.
4. Vashi A, Rhodes KV. “Sign right here and you’re good to go”: a content analysis of audiotaped emergency department discharge instructions. Ann Emerg Med. 2011;57(4):315-22.
5. Kuan WS, Mahadevan M. Emergency unscheduled returns: can we do better?. Singapore Med J. 2009;50(11):1068-71.
6. Kripalani S, Jackson AT, Schnipper JL, et al. Promoting effective transitions of care at hospital discharge: a review of key issues for hospitalists. J Hosp Med. 2007;2(5):314-23.
7. Chang VY, Arora VM, Lev-Ari S, et al. Interns overestimate the effectiveness of their hand-off communication. Pediatrics. 2010;125(3):491-6.
8. Powell CK, Kripalani S. Brief report: Resident recognition of low literacy as a risk factor in hospital readmission. J Gen Intern Med. 2005;20(11):1042-4.
9. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):1803-12.
10. Cheung DS, Kelly JJ, Beach C, et al. Improving handoffs in the emergency department. Ann Emerg Med. 2010;55(2):171-80.
11. Critical Access Hospital and Hospital National Patient Safety Goals. 2006. Available at: http://www.splashcap.com/JCAHO_2006-NPSG-3D. Accessed Apr 23, 2015.
12. Clinical Learning Environment Review. 2014. Available at: https://www.acgme.org/acgmeweb/Portals/0/PDFs/CLER/CLER_Brochure.pdf. Accessed Apr 23, 2015.
13. Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six Step Approach. 2009.
14. Kessler C, Shakeel F, Hern HG, et al. A survey of handoff practices in emergency medicine. Am J Med Qual. 2014;29(5):408-14.
15. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37(9):830-7.
16. Schwind CJ, Williams RG, Boehler ML, et al. Do individual attendings’ post-rotation performance ratings detect residents’ clinical performance deficiencies?. Acad Med. 2004;79(5):453-7.
17. Downing SM, Haladyna TM. Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ. 2004;38(3):327-33.
18. Wohlauer MV, Arora VM, Horwitz LI, et al. The patient handoff: a comprehensive curricular blueprint for resident education to improve continuity of care. Acad Med. 2012;87(4):411-8.