Author	Affiliation
Sreeja Natesan, MD	Duke University, Department of Emergency Medicine, Durham, North Carolina
Jaime Jordan, MD	David Geffen School of Medicine at UCLA, Department of Emergency Medicine, Los Angeles, California
Alexander Sheng, MD	Boston Medical Center, Department of Emergency Medicine, Boston, Massachusetts
Guy Carmelli, MD	University of Massachusetts, Department of Emergency Medicine, Worcester, Massachusetts
Brian Barbas, MD	Loyola University Chicago, Stritch School of Medicine, Loyola University Medical Center, Department of Emergency Medicine, Maywood, Illinois
Andrew King, MD	The Ohio State University Wexner Medical Center, Department of Emergency Medicine, Columbus, Ohio
Kataryza Gore, MD	Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois
Molly Estes, MD	Loma Linda University, Department of Emergency Medicine, Loma Linda, California
Michael Gottlieb, MD	Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois

Background
Critical appraisal
Giving feedback
Feedback culture
Limitations
Conclusion

BACKGROUND

Feedback is an important tool within medical education for the improvement of clinical skills and professional development.¹ However, the emergency department (ED) presents a uniquely complex environment for feedback due to the rapid pace and workflow for patient care, relative lack of privacy, and need for constant task-switching.¹ Incorporating feedback into this environment can negatively impact an emergency medicine (EM) resident’s training, with consistent reports of dissatisfaction regarding the quality of feedback received from faculty.² The Accreditation Council for Graduate Medical Education (ACGME) Milestones stipulate that important domains for high-quality feedback should include timeliness, specificity, balance, recipient feedback/reflection, and an action plan.^1,3-30

Despite the importance of feedback, evidence to inform best practices in the ED is limited, and there is a need for evidence-based guidelines to optimize feedback within the ED setting.^2,31,32 Based on the best available evidence through a critical review of the literature, we offer expert guidelines on feedback from members of the Council of Residency Directors in Emergency Medicine (CORD) Best Practices Subcommittee. This paper provides readers with recommendations on the use of feedback, with a focus on giving and receiving feedback, and suggestions for fostering a positive culture of feedback.

CRITICAL APPRAISAL

This is the tenth article in a series of evidence-based best practice reviews from the CORD Best Practices Subcommittee.^33-41 Created for medical educators, these best-practice reviews cover a wide breadth of topics from clinical teaching, didactics, and journal club to guidance for increasing diversity, equity, and inclusion (DEI) initiatives for faculty and resident recruitment. We conducted a literature search in conjunction with a medical librarian using MEDLINE with a combination of medical subject heading (MeSH) terms and keywords focused on feedback searching for articles published from inception to March 15, 2021 (Appendix). We also reviewed the bibliographies of all included articles. Two authors (SN, MG) independently screened and included articles that addressed delivering feedback, receiving feedback, or feedback culture. We included articles based on discussion and negotiated consensus. Articles were excluded if they were not related to the three domains of feedback. The search yielded 2,402 articles, of which 207 were deemed to be directly relevant to this review. The level and grade of evidence were provided for each best-practice statement implementing the Oxford Centre for Evidence-Based Medicine criteria (Tables 1 and 2).⁴² When supporting data was not available, we made recommendations based upon our combined experience and consensus opinion drawn from expertise in research and scholarship regarding feedback and medical education. Prior to submission, our manuscript was peer reviewed by the CORD Best Practices Subcommittee and posted to the CORD website for two weeks for review by the entire CORD medical education community. We reviewed the comments and feedback prior to incorporating them into the final manuscript.

GIVING FEEDBACK

Components and Characteristics of High-quality Feedback

Delivering feedback is a complex process with many influencing factors. Prior literature has demonstrated that feedback practices by faculty vary.^31,44 Educational experts, learners, and regulatory bodies agree on several key components and characteristics of high-quality feedback. They recommend that feedback should be clear, specific, timely, and actionable.^{1,3-22,24-30,45}

Clarity in feedback is essential; lack of learner understanding leads to an inability to incorporate feedback into an action plan for improved performance.⁴⁵ For feedback to be effective and valued by learners, it should be specific and based on directly observed behaviors and encounters. ^{1,9,13,14,24,46-55} In today’s era of competency-based medical education, it is important that feedback be targeted toward learner goals and a shared mental model of competencies and expectations.^{4,6,11,22,29,47,56-59} Feedback should be given using descriptive, non-judgmental language.^{14,15,22,48,60} It is important to target feedback toward actions and behaviors rather than judgment of the individual.^17,29,61,62 Doing so has the benefit of mitigating the shame response in learners, which can worsen performance and feedback efficacy.^10,57,60

Experts and learners advocate for feedback to be timely, which increases the likelihood that the feedback will be used for improvement.^{3-7,20,26,28,30,52,54,63} While finding time to provide feedback during clinical work can be a challenge,^46,64-67 real-time feedback has been shown to improve performance. ^68,69 Additionally, there is literature to support that real-time, workplace-based assessments provide more specific and effective feedback than end-of-rotation evaluations.⁷⁰ The optimal volume and frequency of feedback are unknown. Multiple observations are likely required to achieve reliable assessments.⁷¹ Regular feedback is important to improve performance,^72,73 and learners appreciate receiving frequent feedback.^6,21 Some experts recommend that more feedback is necessary for the current generation of learners.⁶¹ It is important to note, however, that many learners may value quality over quantity in feedback.⁸

Constructive feedback is important and can lead to motivational learning and enhanced future performance.⁷⁴ While some learners may value constructive feedback over reinforcing or encouraging feedback, both have been shown to be valuable.^{10,18,29,30,75-78} It may not be necessary or helpful to include both constructive and reinforcing feedback during the same conversation.⁷⁹ Giving constructive feedback may be particularly challenging due to fear of retaliation (especially in systems where learner feedback is tied to summative evaluations, linked to author, pay or promotion).^80,81 However, limited literature suggests that the fear of retaliation may be unwarranted.^82,83 Other potential concerns surrounding constructive feedback include damaging rapport with learners or triggering an emotional response from trainees; however, these can be reduced by ensuring the feedback is clear, focused on actions or behaviors (as opposed to the individual), and supported by specific examples.^2,84

As a step toward improved performance, incorporating co-creation of goals^58,85-87 and the formation of learning or action plans into feedback can increase the benefit to trainees.^{1,6,12,17,21-23,60,78,87,88} Additionally, encouraging collaborative discussion and learner reflection during the feedback conversation may be beneficial.^{7,12,48,60,74,89} Faculty should be attentive and dedicated to providing feedback, as faculty effort and engagement have been shown to improve feedback.^7,90

The setting in which feedback is given is also important.^5,26 Feedback should be given in a non-threatening and supportive environment.^{15,17,25,29,48,74,91} It may be prudent to use different types of settings for different feedback activities.⁴ For example, constructive feedback may best be given in a one-on-one setting after a clinical encounter, whereas positive feedback on physical exams, procedural skills, or clinical decision-making may be more effective if given during or immediately after the patient encounter.^4,92 The optimal method used to provide feedback is unknown, and strengths and weaknesses of various forms of feedback have been highlighted.^19,90 Verbal feedback may be more helpful for engaging in collaborative discussion, but written feedback is more easily recognized and can serve as a reference for future reflection.^19,90 It is important to note that inconsistencies exist between verbal and written feedback. An example is when learners receive positive laudatory feedback verbally, only to find disparaging or negative critiques in the written review. This inconsistency can lead to distrust and frustration among the learner and should be avoided.⁴⁷

Sources of Feedback and Personnel Involved

The source of feedback and the individuals involved can also impact feedback quality.^88,93 It is important that feedback be from a credible source. ⁹⁴ Learners consider feedback more valuable and credible when given by those they consider experts in that specific domain.^8,21,94,95 However, the reliability of assessment may vary with assessor groups for different skills assessed; so, it can be valuable to deliberately align assessment and feedback areas with rater domains of expertise when possible.⁷¹ The relationship between the individuals involved in the feedback discussion is also important. Having a good relationship based on mutual respect and trust can enhance the quality and accuracy of feedback.^{3,12,57,62,77,87,90,96-100}

Training individuals on how to give feedback can also improve the quality and specificity of feedback delivered.^{32,48,56,91,101-107} Training can lead to improved comfort with providing feedback and increase the likelihood of the learner incorporating reflection and goal-setting into feedback discussions.^108,109 This is important, as lack of training in those providing feedback has been highlighted as a barrier to giving meaningful feedback.²⁰

Feedback may come from multiple sources, and prior literature has demonstrated that both learners and supervisors value multisource feedback (MSF) as described in Table 1.^{49,52,63,110-119} Limited literature supports that MSF may be more helpful for identifying strengths and weaknesses compared to standard assessment methods and may be more likely to result in behavior change.^52,63 Multisource feedback may also be effective in distinguishing between high, intermediate, and low performance in learners.¹¹³ Additionally, data on the correlations of assessments between assessor groups is mixed, and different assessor groups may provide distinct feedback.^{63,71,110,120-128} Variations in assessments between assessor groups could suggest that assessments may be different but not necessarily less valuable, lending support to the importance of having multiple perspectives in feedback systems to provide learners with more comprehensive data about their skills.^63,120,122 Moreover, learners may value feedback from various groups differently⁵¹; so, care should be taken to align assessor qualifications with the assessments they will be performing.

Barriers to MSF do exist and include lack of training in those providing feedback, time and resources required to gather MSF, and the ability of learners to incorporate this type of feedback.^{20,112,124,129} Multisource feedback can be gathered synchronously or asynchronously,¹²¹ but regardless of route, it should be timely and ideally incorporate multiple settings.^20,63 The incorporation of learner self-assessment into feedback can also have a positive impact.^{17,22,23,128,130,131}

Techniques and Tools for Providing Feedback

Currently, there is no consensus regarding the best methods for feedback and no formal endorsement by educational bodies of a single strategy.¹⁸ When providing feedback, it is important to use a variety of techniques and tools tailored to the individual learner and situation. We summarize several feedback techniques including direct observation, real-time feedback, self-assessment, multiple sources, and other specialized techniques in Table 3.

Each of these techniques has strengths and weaknesses. Direct observation has been shown to be highly valued and can increase clinical knowledge, skills, and attitudes; however, there is limited data to suggest a behavioral change.¹³² Strengths of direct observation include the emphasis on timely, learner-centered feedback.¹⁰⁷ Challenges to direct observation include resources required, competing time demands of faculty and learners, perceived loss of credibility with patients by learners, and the Hawthorne effect.^46,132-135 These barriers may be overcome by creating a structured, longitudinal direct observation and feedback program. ¹³⁶ Real-time feedback is highly learner-centered, has been shown to improve the quantity of feedback given, and is generally well liked by users.¹³⁷ However, it doesn’t necessarily improve feedback quality; studies have shown that less than 20% of the feedback given in real time is specific or corrective, often only focusing on positive and encouraging aspects of care.^27,138

While learner self-assessment may not correlate well with external assessments,^122,127,128 it can contribute positively to feedback discussions by encouraging reflection and establishing a shared understanding and mental model for feedback.^{17,22,23,74,85,128,130,131,139} Combining self-assessment with feedback can positively impact improvement behaviors.^130,131 Importantly, while evaluative models for feedback have been shown to improve the number of feedback evaluations, they may not improve the quality of corrective feedback. ^140,141 Multisource feedback tools are generally well liked and have good efficacy for competencies such as inter-professional communication and professionalism; however, they may be limited in their ability to identify struggling learners.¹⁴² Overall these techniques are quick and efficient and can work for a wide variety of learners to provide formative feedback.¹⁴³

Tools for Giving Feedback

Much like the variety of techniques for giving feedback, many tools have been developed to assist in providing feedback. Feedback tools have been demonstrated to increase the number of feedback encounters and improve learner satisfaction with feedback.^{7,32,59,155-157} However, it is important to note that feedback tools are not a replacement for verbal feedback or preceptor experience.^7,19 We provide a summary of physical and electronic feedback tools including feedback cards, mini-cards, field note tool, MSF tools, web-based platforms, apps, crowdsourcing, and video recording in Table 4.

Like the techniques described above, each tool has its own strengths and weaknesses. Feedback encounter cards have repeatedly been shown to increase the perceived number of feedback encounters and, typically, improve learner satisfaction of quality, amount, and timeliness of feedback.^{2,32,155,156,158} However, some studies have reported that feedback may not be specific enough.^156,159 This challenge can be mitigated by pairing encounter cards with a curriculum for educators and learners regarding giving and receiving feedback.^2,32 Mini-cards and the Mini-Clinical Examination Exercise can identify the struggling learner and provide formative assessments to support their growth.^{88,148,160-162} Both tools can be integrated into routine clinical work while providing reliable assessments if at least 6–8 such encounters are used.¹⁶² A limitation noted for these card-based observation tools is that they may be perceived as a one-way evaluation and less likely to result in a learner-driven action plan. ^148,161 As MSF has become more incorporated into feedback approaches, several tools have been developed and studied as listed in Table 3.^{112,121,122,124,129,142}

With the increased availability of smartphones and portable devices, an array of new electronic-based feedback tools have been created and implemented with the hope of making the administration of feedback more convenient, accessible, and timely for educators and learners.¹⁸ Studies have shown that these web-based tools can be beneficial for improving faculty engagement in and frequency of their feedback.^11,163,164 The timely nature of this feedback also leads to increased satisfaction from learners. ¹⁶⁴ However, these platforms can be limited by faculty comfort with, and knowledge of, technology.¹⁶⁵ Additionally, specific and corrective feedback may be challenging.^27,138

It is important to consider data consolidation and distribution with these tools to ensure that feedback is distributed in a timely manner.^137,166 To improve the accessibility of online feedback tools, several platforms have used quick response (QR) codes.^163,167 The use of QR codes to access online feedback forms was found to be user-friendly and resulted in faster completion than paper and online web-based tools not associated with a QR code.¹⁶⁷ Various apps have been created, which have led to an increase in the quality of feedback.^{18,70,168-170} Additional strengths include accessibility, low cost, and ability to trend resident progression.¹⁸ However, much like web-based platforms, app-based platforms can be limited by faculty and resident engagement.^18,170 When instituting any app-based evaluation tool, it is important to pair it with training on the app and changes to feedback culture, such as regular encouragement, incentivization, physician champions, or regular reminders.^18,168,170

Using online social media platforms (eg, Twitter messaging) is another tool to increase the volume and timeliness of feedback; however, effectiveness may be limited.^18,171 Video-assisted feedback can be a valuable tool for feedback similar to direct observation.^133,172 However, much like other forms of direct observation, video recording may not represent true, real-world encounters as learners may act differently due to the Hawthorne effect. Additionally, video recording can cause anxiety in trainees.¹³³

Inviting Co-workers to Evaluate Physicians Tool (INCEPT); Mini Peer Assessment Tool (Mini-PAT); Team Assessment of Behavior (TAB); Emergency Medicine Humanism Scale (EM-HS); Communication Assessment Tool (CAT); Observed Structured Teaching Exercises (OSTE), or an Objective Structured Clinical Examination (OSCE) are other useful evaluation tools.

 

Best Practice Recommendations

 

Feedback should be clear, specific, timely, and actionable. (Level 1a, Grade B)

 

Feedback should be based on observed behaviors. (Level 3b, Grade B)

 

Both corrective and reinforcing feedback should be provided to learners, although not necessarily at the same time. (Level 4, Grade C)

 

Feedback tools are recommended to increase learner satisfaction and volume of feedback; however, the use of tools must be combined with faculty development and a culture of feedback to improve the quality of feedback. (Level 3b, Grade C)

 

Feedback should incorporate learner self-assessment. (Level 3b, Grade C)

 

Receiving Feedback

Traditional approaches place learners in the role of passively receiving feedback,^{79,82,176,177} which have been criticized for being too centered on the actions of the instructor. More modern models shift to include the learner as an active participant in soliciting and responding to feedback.^4,13,153

Soliciting Feedback

A crucial initial step to engaging in effective feedback is the act of soliciting feedback that opens the individual to the critiquing process.^55,178 The ability to engage in feedback-seeking behaviors is dependent on four factors: the purpose and quality of the feedback; the learner’s emotional response to feedback; the learner-evaluator relationship; and the workplace culture.^{4,13,50,176,179,180} While the environment is outside our control, appropriately prepping learners to take contextual factors into account and shifting the focus to environmentally appropriate feedback models may be particularly helpful.^181,182 One common example is the implementation of end-of-shift feedback evaluations. While these have not been identified by faculty as providing a higher quality of feedback, their systematic and reliable delivery results in higher resident satisfaction with the feedback.³²

Accepting Feedback

Despite the best intentions of the feedback giver, feedback receptivity is never assured. Literature demonstrates that faculty and learners even disagree on their perceptions of how much feedback is being given.^1,16,55 Nevertheless, learner perception significantly impacts feedback acceptance and integration.^130,180,183 Different experts have categorized such factors in different ways.^1,50,57,184 One of the more usable classifications includes categorization of personal (ie, resilience, humility), relational (ie. the strength of supervisory relationship, power differentials), and contextual (ie, culture) factors.⁵⁷

Personal Factors

Much of feedback receptivity depends on the learner’s frame of reference. Possessing a growth mindset and employing routine self-reflection is key.^{62,89,96,100,182,183,185-187} Learners often approach feedback situations as a performance, probing the situation to see what is expected of them and then acting in a way to better shape their reputation and evaluations.^{1,13,16,184,188} Those who have blind spots regarding their weaknesses may be resistant to feedback that challenges their existing self-perception.^130,185-191 Failure to internalize feedback happens when a mismatch in external and internally generated assessment occurs. For instance, EM residents consistently assign themselves higher milestone competency ratings than their evaluating attendings.¹⁸⁹

When feedback is perceived as an attack on personal identity, feedback internalization is effectively hindered. Thus, learners should perceive feedback as opportunities for improvement, rather than statements on character.^1,134,192 Evidence suggests that learners educated on feedback have shown comfort in giving and receiving feedback.¹⁰⁵ Melding self-generated learning goals with faculty-provided observations closes the feedback loop and produces more improved, usable, and well-received feedback aimed at mastering current skills and setting goals for future accomplishments.^12,22,130 To bridge the gap between reception of the feedback to internalizing it, multiple experts have outlined various practical tips for learners to use feedback for performance improvement.^{190,191,193-200} We distilled the consistent themes among our recommendations below.

Relational Factors

Feedback receptivity is significantly impacted by relational factors such as the strength of the supervisory relationship and power differentials. Regardless of the experience level of the assessor, learners consistently recognize feedback as valid when coming from someone they trust and respect, find credible,^1,182,192 and have sought out rather than been assigned,^181,197 such as from role models.^198,199 Mutual respect, establishing shared priorities, and the strength of the educational alliance (defined as the learner’s belief of shared goals, activities, and bonds)²⁰⁰ facilitated better feedback receptivity.⁵⁷ Interpersonal skills also affect the relationship and receptivity. Power dynamics and fear of the effect of corrective feedback are barriers to feedback integration.⁵⁷ Learners value feedback when given in a caring, nonjudgmental manner^31,62,99 from educators who are friendly and approachable.²⁰¹

Contextual Factors

Environmental and cultural considerations affect the receptivity of feedback. The tension between assessment and feedback, specifically the fear of consequences, can lead to learner development of a fixed mindset, limiting growth opportunities.^57,96 For professionalism issues, feedback should be given one on one.^1,20 In busy learning environments, learner-centered approaches grounded in self-directed learning theories (eg, Learner-Centered Approach to Raise Efficiency [L-CARE)] in Clinical Teaching) have been proposed to facilitate more efficient learning.²⁰² Ultimately, various studies demonstrated benefit and/or learner preference for standardized,^139,203 structured,^150,203 multisource,¹⁵⁰ and longitudinal^{1,105,204,205} feedback processes.

FEEDBACK CULTURE

Feedback culture is defined as written or verbal comments regarding medical knowledge, performance, technique, or patient care within the pedagogical approaches that are routine within a profession.^206,207 The learning culture and type of clinical environment influences learners’ feedback behaviors such as recognizing, seeking, and implementing feedback, namely whether this process is encouraged or not.^99,100 The ED is particularly challenging due to the nature of the work environment, including time constraints, frequent interruptions, and patient acuity, among other factors.^29,208,209

 

Best Practice Recommendations

 

Encourage learners to take an active role in the feedback process. (Level 2b, Grade B)

 

Take the work environment into account when creating appropriate feedback systems that are contextually appropriate as a way to improve learner perception of feedback. (Level 2a, Grade B)

 

Provide opportunities for learners to build longitudinal trusting relationships in order to promote a strong educational alliance and a growth mindset and to facilitate feedback reception. (Level 4, Grade C)

 

Address the tension between assessment and feedback as fear of consequences can predispose a learner to have a fixed mindset, thus limiting learner growth. (Level 4, Grade C)

 

Develop and maintain standardized, structured, multisource, and longitudinal feedback processes. (Level 3a, Grade B)

 

Implementation

Institutions should provide and encourage educational opportunities to all individuals involved in feedback interactions including learners and educators. This will allow a culture of growth emphasizing a bidirectional feedback approach^1,62,100 with a shift from performance-oriented assessments to learner-oriented feedback.⁵⁶ One method is to emphasize the concept of lifelong learning and normalize the need to identify strengths and weaknesses as a way to grow. Training on giving feedback upward and receiving feedback as an educator can help provide the framework for effective bidirectional feedback.^{1,16,99,181,194,200} Learners need an environment where vulnerability is acceptable and assessment focuses on a set of shared goals.^14,47 Other strategies include establishing expectations for both educators and learners, promoting specific tasks for all involved, and providing professional development sessions.^57,210 For establishing longitudinal relationships, providing protected faculty time for observational assessments and using standardized feedback tools are beneficial.^97,136,207 Furthermore, institutions should encourage a culture of growth. Learners develop a fixed mindset when they perceive performance is linked to assessment, rather than a growth mindset when the relationship is not tied to assessments.⁹⁶

An interdisciplinary, multimodal approach to feedback through MSF can provide additional insight regarding communication, professionalism, and team dynamics and broaden the scope of the feedback received by the learner.^{18,20,121-123,203,211} Using non-physician medical education specialists to observe learners in the clinical setting may be a useful way to provide tangible feedback on communication, task-switching, professionalism, accountability, and team management skills.⁵⁰

Barriers to Successful Implementation

Successful implementation of an optimal feedback culture requires a firm understanding of the potential barriers. Grade inflation, discomfort in providing negative feedback, concern with preserving healthy working relationships,^2,32,84 time constraints, and personal deficiencies in feedback delivery each present unique challenges.^67,84 Administrative support and the encouragement of the importance of feedback are also important.⁶⁷ While feedback tools may pose a barrier, choosing a user-friendly tool that is of appropriate length and provides sufficient detail with required narrative comments is key.^8,9,24

Although limited literature suggests this may be unwarranted,⁸² educators often avoid corrective feedback due to fear of retaliation (especially in systems where learner evaluations are linked to pay or promotion).^80,81 Transparency and focus on the importance of corrective feedback as a learning tool ¹² can prevent reluctance to provide negative feedback.^12,80 Finally, a culture of “niceness” can make the learning environment overtly positive, which can hinder the delivery of honest feedback and the creation of a culture of constructive feedback.^12,99,100 Being “nice” can be construed as focusing on the positive with a priority on minimizing any negative feelings in the other person, while being “kind” can be construed as focusing on what is best for the learner overall — even if it means creating negative feelings.

Special Considerations

Implicit bias, which is the unconscious attitudes we have toward people or associated stereotypes, impacts both feedback provided to learners and the perception or receptibility of feedback from faculty.⁴⁷ To minimize this potential bias, assessments should be performed by multiple assessors in multiple different settings.^47,63,210 Furthermore, institutions should implement training to identify areas where biases exist, while working to alleviate these biases with full transparency.⁴⁷ Gender bias may lead to different distributions of the frequency and type of feedback. One study found female preceptors completed more feedback forms and provided more corrective feedback to male learners, whereas male preceptors used more communal language and less agentic language with female learners.²¹² Additionally, female learners had more discordant feedback, especially regarding the balance of autonomy and feedback receptivity, than their male counterparts.²¹³ Finally, in a study by Stroud, female faculty were found to be perceived as less credible when delivering feedback.⁹⁵

Like racial, cultural, and gender bias, generational gaps can also affect meaningful feedback. Different generations have different patterns of learning. For example, the millennial generation is more engaged in technology and collaborative learning, while preferring clear objectives and timely feedback.^28,61 Additionally, feedback should be provided to all learners, not just low performers. High performers may exhibit the “halo effect,” which can result in them receiving less constructive feedback.¹ Learner shame responses can be triggered by repeated humiliation experienced in receiving feedback from facilitators. Providing feedback that is focused on behaviors, providing support that normalizes errors in the learning process, and guiding learners through reflection can decrease these learner responses.⁶⁰

 

Best Practice Recommendations

 

Maximize the impact of feedback by minimizing implicit bias through providing feedback from multiple different assessors in multiple different settings. (Level 4, Grade C)

 

Encourage a culture of growth and transparency, focusing on corrective feedback as a learning tool. (Level 4, Grade C)

 

Establish expectations for both educators and learners, promote specific tasks for all involved, implement processes to encourage bi-directional feedback, and provide development sessions for professional growth. (Level 4, Grade C)

 

Shift emphasis from performance-oriented assessment to learner-oriented feedback. (Level 2b, Grade B)

 

LIMITATIONS

Although we performed a comprehensive search guided by a medical librarian in conjunction with a bibliographic review and expert consultation to augment content when needed, we used a single search engine (MEDLINE), and it is possible that we may have missed some pertinent articles. In instances where evidence in the form of high-quality data was limited or lacking, we relied upon expert opinion and group consensus for the best practice recommendations. The literature specific to feedback for the field of EM and within graduate medical education is limited. To supplement, we included relevant articles from other medical specialties and health-related professions as we believe that EM, as a specialty, can learn from other colleagues across many disciplines. Finally, in areas where evidence was not available, we used the consensus from the expertise of our authorship group. While our author group possesses experience in research and scholarship in both feedback and medical education, there is a potential for bias to be introduced during this process. Therefore, we also sought peer review from the CORD Best Practices Subcommittee and posted it online for open review feedback by the CORD community.

CONCLUSION

Feedback is integral to professional development. This paper provides readers with guidance on the use of feedback in medical education, with a focus on instructor strategies for giving feedback, learner strategies for receiving feedback, and suggestions for fostering a culture of feedback.

Footnotes

Section Editor: Mark I. Langdorf, MD, MHPE

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Sreeja Natesan, MD, Duke University, Department of Emergency Medicine, PO Box 3096, 1320 Erwin Road, Durham NC 27710. Email: sreeja.natesan@duke.edu. 5 / 2023; 24:479 – 494

Submission history: Revision received February 25, 2022; Submitted August 18, 2022; Accepted September 3, 2022

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Buckley C, Natesan S, Breslin A, et al. Finessing feedback: recommendations for effective feedback in the emergency department. Ann Emerg Med. 2020;75(3):445-51.

2. Yarris LM, Linden JA, Gene Hern H, et al. Attending and resident satisfaction with feedback in the emergency department. Acad Emerg Med. 2009;16:S76-81.

3. Dudek NL, Dojeiji S, Day K, et al. Feedback to supervisors: Is anonymity really so important?. Acad Med. 2016;91(9):1305-12.

4. Gonzalo JD, Heist BS, Duffy BL, et al. Content and timing of feedback and reflection: a multi-center qualitative study of experienced bedside teachers. BMC Med Educ. 2014;14:212.

5. Iskander M. Offering effective feedback to trainees. Med Teach. 2015;37(1):92-3.

6. Kaul P, Gong J, Guiton G. Effective feedback strategies for teaching in pediatric and adolescent gynecology. J Pediatr Adolesc Gynecol. 2014;27(4):188-93.

7. Ritchie KC, Sjaus A, Munro A, et al. An interpretive phenomenological analysis of formative feedback in anesthesia training: the residents’ perspective. BMC Med Educ. 2020;20(1):493.

8. Bleasel J, Burgess A, Weeks R, et al. Feedback using an ePortfolio for medicine long cases: quality not quantity. BMC Med Educ. 2016;16(1):278.

9. Branfield Day L, Miles A, Ginsburg S, et al. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95(11):1712-7.

10. Carr S. The Foundation Programme assessment tools: an opportunity to enhance feedback to trainees?. Postgrad Med J. 2006;82(971):576-9.

11. Connolly KA, Azouz SM, Smith AA. Feedback in plastic and reconstructive surgery education: past, present, and future. J Craniofac Surg. 2015;26(8):2261-3.

12. Davila-Cervantes A, Foulds JL, Gomaa NA, et al. Experiences of faculty members giving corrective feedback to medical trainees in a clinical setting. J Contin Educ Health Prof. 2021;41(1):24-30.

13. Delva D, Sargeant J, Miller S, et al. Encouraging residents to seek feedback. Med Teach. 2013;35(12):e1625-31.

14. Ende J. Feedback in clinical medical education. JAMA. 1983;250(6):777-81.

15. Hewson MG, Little ML. Giving feedback in medical education. J Gen Intern Med. 1998;13(2):111-6.

16. Natesan S, Stehman C, Shaw R, et al. Curated collections for educators: five key papers about receiving feedback in medical education. Cureus. 2019;11(9):e5728.

17. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007;29(9):855-71.

18. Perkins SQ, Dabaja A, Atiemo H. Best approaches to evaluation and feedback in post-graduate medical education. Curr Urol Rep. 2020;21(10):36.

19. Tomiak A, Braund H, Egan R, et al. Exploring how the new entrustable professional activity assessment tools affect the quality of feedback given to medical oncology residents. J Cancer Educ. 2020;35(1):165-77.

20. Campbell S, Goltz HH, Njue S, et al. Exploring the reality of using patient experience data to provide resident feedback: a qualitative study of attending physician perspectives. Perm J. 2016;20(3):15-154.

21. Ivers NM, Grimshaw JM, Jamtvedt G, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534-41.

22. Moroz A, Horlick M, Mandalaywala N, et al. Faculty feedback that begins with resident self-assessment: motivation is the key to success. Med Educ. 2018;52(3):314-23.

23. Moroz A, King A, Kim B, et al. Constructing a shared mental model for feedback conversations: faculty workshop using video vignettes developed by residents. MedEdPORTAL. 2019;15:10821.

24. Moss HA, Derman PB, Clement RC. Medical student perspective: working toward specific and actionable clinical clerkship feedback. Med Teach. 2012;34(8):665-7.

25. Orlander JD, Fincke BG. Soliciting feedback: on becoming an effective clinical teacher. J Gen Intern Med. 1994;9(6):334-5.

26. Vickery AW, Lake FR. Teaching on the run tips 10: giving feedback. Med J Aust. 2005;183(5):267-8.

27. Shaughness G, Georgoff PE, Sandhu G, et al. Assessment of clinical feedback given to medical students via an electronic feedback system. J Surg Res. 2017;218:174-9.

28. Schwartz AC, McDonald WM, Vahabzadeh AB, et al. Keeping up with changing times in education: fostering lifelong learning of millennial learners. Focus (Am Psychiatr Publ). 2018;16(1):74-9.

29. Richardson BK. Feedback. Acad Emerg Med. 2004;11(12):e1-5.

30. Dickinson KJ, Bass BL, Pei KY. What embodies an effective surgical educator? A grounded theory analysis of resident opinion. Surgery. 2020;168(4):730-6.

31. Bing-You R, Hayes V, Varaklis K, et al. Feedback for learners in medical education: What is known? A scoping review. Acad Med. 2017;92(9):1346-54.

32. Yarris LM, Fu R, LaMantia J, et al. Effect of an educational intervention on faculty and resident satisfaction with real-time feedback in the emergency department. Acad Emerg Med. 2011;18(5):504-12.

33. Chathampally Y, Cooper B, Wood DB, et al. Evolving from morbidity and mortality to a case-based error reduction conference: Evidence-based Best Practices from the Council of Emergency Medicine Residency Directors. West J Emerg Med. 2020;21(6):231-41.

34. Wood DB, Jordan J, Cooney R, et al. Conference didactic planning and structure: an Evidence-based Guide to Best Practices from the Council of Emergency Medicine Residency Directors. West J Emerg Med. 2020;21(4):999-1007.

35. Parsons M, Bailitz J, Chung AS, et al. Evidence-based interventions that promote resident wellness from the Council of Emergency Residency Directors. West J Emerg Med. 2020;21(2):412-22.

36. Parsons M, Caldwell M, Alvarez A, et al. Physician pipeline and pathway programs: an evidence-based guide to best practices for diversity, equity, and inclusion from the Council of Residency Directors in Emergency Medicine. West J Emerg Med. 2022;23(4):514-24.

37. Davenport D, Alvarez A, Natesan S, et al. Faculty recruitment, retention, and representation in leadership: An Evidence-Based Guide to Best Practices for Diversity, Equity, and Inclusion from the Council of Residency Directors in Emergency Medicine. West J Emerg Med. 2022;23(1):62-71.

38. Gallegos M, Landry A, Alvarez A, et al. Holistic Review, Mitigating Bias, and Other Strategies in Residency Recruitment for Diversity, Equity, and Inclusion: An Evidence-based Guide to Best Practices from the Council of Residency Directors in Emergency Medicine. West J Emerg Med. 2022;23(3):345-52.

39. Natesan S, Bailitz J, King A, et al. Clinical teaching: An Evidence-based Guide to Best Practices from the Council of Emergency Medicine Residency Directors. West J Emerg Med. 2020;21(4):985-98.

40. Estes M, Gopal P, Siegelman JN, et al. Individualized Interactive Instruction: a Guide to Best Practices from the Council of Emergency Medicine Residency Directors. West J Emerg Med. 2019;20(2):363-8.

41. Gottlieb M, King A, Byyny R, et al. Journal club in residency education: an Evidence-based Guide to Best Practices from the Council of Emergency Medicine Residency Directors. West J Emerg Med. 2018;19(4):746-55.

42. Phillips R, Ball C, Sackett D. Oxford Centre for Evidence-Based Medicine: Levels of Evidence. 2021. Available at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence. Accessed October 1, 2021.

43. Kogan JR, Conforti LN, Bernabeo EC, et al. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012;46(2):201-15.

44. Roze des Ordons A, Cheng A, Gaudet J, et al. Adapting feedback to individual residents: an examination of preceptor challenges and approaches. J Grad Med Educ. 2018;10(2):168-75.

45. Desveaux L, Ivers NM, Devotta K, et al. Unpacking the intention to action gap: a qualitative study understanding how physicians engage with audit and feedback. Implement Sci. 2021;16(1):19.

46. Cheung WJ, Patey AM, Frank JR, et al. Barriers and enablers to direct observation of trainees’ clinical performance: a qualitative study using the theoretical domains framework. Acad Med. 2019;94(1):101-14.

47. Dolan BM, Arnold J, Green MM. Establishing trust when assessing learners: barriers and opportunities. Acad Med. 2019;94(12):1851-3.

48. Kritek PA. Strategies for effective feedback. Ann Am Thorac Soc. 2015;12(4):557-60.

49. Moreau KA, Eady K, Jabbour M. Exploring residents’ reactions to and use of parent feedback in a pediatric emergency department: a grounded theory study. Med Teach. 2019;41(2):207-14.

50. Waterbrook AL, Spear Ellinwood KC, Pritchard TG, et al. Shadowing emergency medicine residents by medical education specialists to provide feedback on non-medical knowledge-based ACGME sub-competencies. Adv Med Educ Pract. 2018;9:307-15.

51. Van Hell EA, Kuks JBM, Raat ANJ, et al. Instructiveness of feedback during clerkships: influence of supervisor, observation and student initiative. Med Teach. 2009;31(1):45-50.

52. Snydman L, Chandler D, Rencic J, et al. Peer observation and feedback of resident teaching. Clin Teach. 2013;10(1):9-14.

53. Smith J, Jacobs E, Li Z, et al. Successful implementation of a direct observation program in an ambulatory block rotation. J Grad Med Educ. 2017;9(1):113-7.

54. Brauch RA, Goliath C, Patterson L, et al. A qualitative study of improving preceptor feedback delivery on professionalism to postgraduate year 1 residents through education, observation, and reflection. Ochsner J. 2013;13(3):322-6.

55. Trawicki MC, Zuegge KL, Volz LM, et al. An intervention to improve medical student perception of observation and feedback during an anesthesiology clerkship. Ochsner J. 2018;18(2):159-63.

56. Balmer DF, Tenney-Soeiro R, Mejia E, et al. Positive change in feedback perceptions and behavior: a 10-year follow-up study. Pediatrics. 2018;141(1).

57. Denny B, Brown J, Kirby C, et al. “I’m never going to change unless someone tells me I need to”: fostering feedback dialogue between general practice supervisors and registrars. Aust J Prim Health. 2019;25(4):374-9.

58. Mehta F, Brown J, Shaw NJ. Do trainees value feedback in case- based discussion assessments?. Med Teach. 2013;35(5):e1166-72.

59. Peccoralo L, Karani R, Coplit L, et al. Pocket card and dedicated feedback session to improve feedback to ward residents: a randomized trial. J Hosp Med. 2012;7(1):35-40.

60. Bynum WE, Goodie JL. Shame, guilt, and the medical learner: ignored connections and why we should care. Med Educ. 2014;48(11):1045-54.

61. Moreno-Walton L, Brunett P, Akhtar S, et al. Teaching across the generation gap: a consensus from the Council of Emergency Medicine Residency Directors 2009 academic assembly. Acad Emerg Med. 2009;16:S19-24.

62. Ramani S, Könings KD, Ginsburg S, et al. Relationships as the backbone of feedback: exploring preceptor and resident perceptions of their behaviors during feedback conversations. Acad Med. 2020;95(7):1073-81.

63. Byrd A, Iheagwara K, McMahon P, et al. Using multisource feedback to assess resident communication skills: adding a new dimension to milestone data. Ochsner J. 2020;20(3):255-60.

64. Chaou C-H, Chang Y-C, Yu S-R, et al. Clinical learning in the context of uncertainty: a multi-center survey of emergency department residents’ and attending physicians’ perceptions of clinical feedback. BMC Med Educ. 2019;19(1):174.

65. Reddy ST, Zegarek MH, Fromme HB, et al. Barriers and facilitators to effective feedback: A qualitative analysis of data from multispecialty resident focus groups. J Grad Med Educ. 2015;7(2):214-9.

66. Chaou C-H, Monrouxe LV, Chang L-C, et al. Challenges of feedback provision in the workplace: A qualitative study of emergency medicine residents and teachers. Med Teach. 2017;39(11):1145-53.

67. Zehra T, Tariq M, Ali SK, et al. Challenges of providing timely feedback to residents: Faculty perspectives. JPMA The Journal of the Pakistan Medical Association. 2015;65(10):1069-74.

68. Kim JG, Morris CG, Heidrich FE. A tool to assess family medicine residents’ patient encounters using secure messaging. J Grad Med Educ. 2015;7(4):649-53.

69. Kim D, Spellberg B. Does real-time feedback to residents with or without attendings improve medical documentation?. Hosp Pract (Minneap). 2014;42(3):123-30.

70. Ahle SL, Eskender M, Schuller M, et al. The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments. Ann Surg. 2022;275(3):617-20.

71. Moonen-van Loon JMW, Overeem K, Govaerts MJB, et al. The reliability of multisource feedback in competency-based assessment programs: the effects of multiple occasions and assessor groups. Acad Med. 2015;90(8):1093-9.

72. Piquette D, Moulton C-A, LeBlanc VR. Model of interactive clinical supervision in acute care environments. Balancing patient care and teaching. Ann Am Thorac Soc. 2015;12(4):498-504.

73. Tanaka P, Bereknyei Merrell S, Walker K, et al. Implementation of a needs-based, online feedback tool for anesthesia residents with subsequent mapping of the feedback to the ACGME Milestones. Anesth Analg. 2017;124(2):627-35.

74. Sarkany D, Deitte L. Providing feedback: practical skills and strategies. Acad Radiol. 2017;24(6):740-6.

75. Morgan S. Supervising the highly performing general practice registrar. Clin Teach. 2014;11(1):53-7.

76. Carr BM, O’Neil A, Lohse C, et al. Bridging the gap to effective feedback in residency training: perceptions of trainees and teachers. BMC Med Educ. 2018;18(1):225.

77. Findlay N. General practitioner registrars’ experiences of multisource feedback: a qualitative study. Educ Prim Care. 2012;23(5):323-9.

78. Gauthier S, Cavalcanti R, Goguen J, et al. Deliberate practice as a framework for evaluating feedback in residency training. Med Teach. 2015;37(6):551-7.

79. Molloy E, Ajjawi R, Bearman M, et al. Challenging feedback myths: values, learner involvement and promoting effects beyond the immediate task. Med Educ. 2020;54(1):33-9.

80. McQueen SA, Petrisor B, Bhandari M, et al. Examining the barriers to meaningful assessment and feedback in medical training. Am J Surg. 2016;211(2):464-75.

81. Cousar M, Huang J, Sebro R, et al. Too scared to teach? The unintended impact of 360-degree feedback on resident education. Curr Probl Diagn Radiol. 2020;49(4):239-42.

82. Baker K, Haydar B, Mankad S. A feedback and evaluation system that provokes minimal retaliation by trainees. Anesthesiology. 2017;126(2):327-37.

83. Baker K. Clinical teaching improves with resident evaluation and feedback. Anesthesiology. 2010;113(3):693-703.

84. Sabey A, Harris M. “It’s the conversation they’ll learn from”: improving assessments for GP specialist trainees in hospital posts. Educ Prim Care. 2012;23(4):263-9.

85. Farrell L, Bourgeois-Law G, Ajjawi R, et al. An autoethnographic exploration of the use of goal oriented feedback to enhance brief clinical teaching encounters. Adv Health Sci Educ Theory Pract. 2017;22(1):91-104.

86. Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015;90(12):1698-706.

87. Armson H, Lockyer JM, Zetkulic M, et al. Identifying coaching skills to improve feedback use in postgraduate medical education. J Pak Med Assoc. 2019;53(5):477-93.

88. Pelgrim EAM, Kramer AWM, Mokkink HGA, et al. The process of feedback in workplace-based assessment: organisation, delivery, continuity. Med Educ. 2012;46(6):604-12.

89. Pelgrim EAM, Kramer AWM, Mokkink HGA, et al. Reflection as a component of formative assessment appears to be instrumental in promoting the use of feedback; an observational study. Med Teach. 2013;35(9):772-8.

90. Bowen JL, Boscardin CK, Chiovaro J, et al. A view from the sender side of feedback: anticipated receptivity to clinical feedback when changing prior physicians’ clinical decisions-a mixed methods study. Adv Health Sci Educ Theory Pract. 2020;25(2):263-82.

91. Brown LE, Rangachari D, Melia M. Beyond the sandwich: from feedback to clinical coaching for residents as teachers. MedEdPORTAL. 2017;13:10627.

92. McKenzie S, Burgess A, Mellis C. Interns reflect: the effect of formative assessment with feedback during pre-internship. Adv Med Educ Pract. 2017;8:51-6.

93. Pelgrim EAM, Kramer AWM, Mokkink HGA, et al. Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study. BMC Med Educ. 2012;12:97.

94. Valentine N, Schuwirth L. Identifying the narrative used by educators in articulating judgement of performance. Perspect Med Educ. 2019;8(2):83-9.

95. Stroud L, Sibbald M, Richardson D, et al. Feedback credibility in a formative postgraduate objective structured clinical examination: effects of examiner type. J Grad Med Educ. 2018;10(2):185-91.

96. Huffman BM, Hafferty FW, Bhagra A, et al. Resident impression management within feedback conversations: a qualitative study. Med Educ. 2021;55(2):266-74.

97. Voyer S, Cuncic C, Butler DL, et al. Investigating conditions for meaningful feedback in the context of an evidence-based feedback programme. Med Educ. 2016;50(9):943-54.

98. Weinstein DF. Feedback in clinical education: untying the Gordian knot. Acad Med. 2015;90(5):559-61.

99. Ramani S, Post SE, Könings K, et al. “it’s just not the culture”: a qualitative study exploring residents’ perceptions of the impact of institutional culture on feedback. Teach Learn Med. 2017;29(2):153-61.

100. Ramani S, Könings K, Mann KV, et al. Uncovering the unknown: a grounded theory study exploring the impact of self-awareness on the culture of feedback in residency education. Med Teach. 2017;39(10):1065-73.

101. Berbano EP, Browning R, Pangaro L, et al. The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006;21(5):430-4.

102. Holmboe ES, Fiebach NH, Galaty LA, et al. Effectiveness of a focused educational intervention on resident evaluations from faculty. J Gen Intern Med. 2001;16(7):427-34.

103. Junod Perron N, Nendaz M, Louis-Simonet M, et al. Effectiveness of a training program in supervisors’ ability to provide feedback on residents’ communication skills. Adv Health Sci Educ Theory Pract. 2013;18(5):901-15.

104. Kistler EA, Chiappa V, Chang Y, et al. Evaluating the SPIKES model for improving peer-to-peer feedback among internal medicine residents: a randomized controlled trial. J Gen Intern Med. 2021;36(11):3410-6.

105. Kruidering-Hall M, O’Sullivan PS, Chou CL. Teaching feedback to first-year medical students: long-term skill retention and accuracy of student self-assessment. J Gen Intern Med. 2009;24(6):721-6.

106. Zelenski AB, Tischendorf JS, Kessler M, et al. Beyond “read more”: an intervention to improve faculty written feedback to learners. J Grad Med Educ. 2019;11(4):468-71.

107. Schlair S, Dyche L, Milan F. Longitudinal faculty development program to promote effective observation and feedback skills in direct clinical observation. MedEdPORTAL. 2017;13:10648.

108. Ricotta DN, Hale AJ, Freed JA, et al. Peer observation to develop resident teaching. Clin Teach. 2020;17(5):521-5.

109. Rassbach CE, Blankenburg R. A novel pediatric residency coaching program: outcomes after one year. Acad Med. 2018;93(3):430-4.

110. Dupras DM, Edson RS. A survey of resident opinions on peer evaluation in a large internal medicine residency program. J Grad Med Educ. 2011;3(2):138-43.

111. Yuan D, Bridges M, D’Amico FJ, et al. The effect of medical student feedback about resident teaching on resident teaching identity: a randomized controlled trial. Fam Med. 2014;46(1):49-54.

112. Whitehouse A, Hassell A, Bullock A, et al. 360 degree assessment (multisource feedback) of UK trainee doctors: field testing of team assessment of behaviours (TAB). Med Teach. 2007;29(2–3):171-6.

113. Warm EJ, Schauer D, Revis B, et al. Multisource feedback in the ambulatory setting. J Grad Med Educ. 2010;2(2):269-77.

114. Riveros R, Kimatian S, Castro P, et al. Multisource feedback in professionalism for anesthesia residents. J Clin Anesth. 2016;34:32-40.

115. Richard-Lepouriel H, Bajwa N, de Grasset J, et al. Medical students as feedback assessors in a faculty development program: Implications for the future. Med Teach. 2020;42(5):536-42.

116. van der Leeuw RM, Slootweg IA. Twelve tips for making the best use of feedback. Med Teach. 2013;35(5):348-51.

117. van der Leeuw RM, Schipper MP, Heineman MJ, et al. Residents’ narrative feedback on teaching performance of clinical teachers: analysis of the content and phrasing of suggestions for improvement. Postgrad Med J. 2016;92(1085):145-51.

118. Tully K, Keller J, Blatt B, et al. Observing and giving feedback to novice PGY-1s. South Med J. 2016;109(5):320-5.

119. Gottlieb M, Jordan J, Siegelman JN, et al. Direct observation tools in emergency medicine: a systematic review of the literature. AEM Education and Training. 2021;5(3):e10519.

120. Archer JC, McAvoy P. Factors that might undermine the validity of patient and multi-source feedback. Med Educ. 2011;45(9):886-93.

121. Garra G, Thode H. Synchronous collection of multisource feedback evaluations does not increase inter-rater reliability. Acad Emerg Med. 2011;18:S65-70.

122. Garra G, Wackett A, Thode H. Feasibility and reliability of a multisource feedback tool for emergency medicine residents. J Grad Med Educ. 2011;3(3):356-60.

123. Hayward MF, Curran V, Curtis B, et al. Reliability of the interprofessional collaborator assessment rubric (ICAR) in multi source feedback (MSF) with post-graduate medical residents. BMC Med Educ. 2014;14:1049.

124. van der Meulen MW, Boerebach BCM, Smirnova A, et al. Validation of the INCEPT: a multisource feedback tool for capturing different perspectives on physicians’ professional performance. J Contin Educ Health Prof. 2017;37(1):9-18.

125. Sherbino J, Bandiera G. Improving communication skills: feedback from faculty and residents. Acad Emerg Med. 2006;13(4):467-70.

126. Tariq M, Govaerts M, Afzal A, et al. Ratings of performance in multisource feedback: comparing performance theories of residents and nurses. BMC Med Educ. 2020;20(1):355.

127. Keister DM, Hansen SE, Dostal J. Teaching resident self-assessment through triangulation of faculty and patient feedback. Teach Learn Med. 2017;29(1):25-30.

128. Calhoun AW, Rider EA, Peterson E, et al. Multi-rater feedback with gap analysis: an innovative means to assess communication skill and self-insight. Patient Educ Couns. 2010;80(3):321-6.

129. Mahoney D, Bogetz A, Hirsch A, et al. The challenges of multisource feedback: feasibility and acceptability of gathering patient feedback for pediatric residents. Acad Pediatr. 2019;19(5):555-60.

130. Bounds R, Bush C, Aghera A, et al. Emergency medicine residents’ self-assessments play a critical role when receiving feedback. Acad Emerg Med. 2013;20(10):1055-61.

131. Stalmeijer RE, Dolmans DHJM, Wolfhagen IHAP, et al. Combined student ratings and self-assessment provide useful feedback for clinical teachers. Adv Health Sci Educ Theory Pract. 2010;15(3):315-28.

132. Craig S. Direct observation of clinical practice in emergency medicine education. Acad Emerg Med. 2011;18(1):60-7.

133. Delbridge EJ, Wilson T, McGregor JD, et al. Interdisciplinary video review: assessing milestones and providing feedback. Int J Psychiatry Med. 2019;54(4–5):266-74.

134. Mann K, van der Vleuten C, Eva K, et al. Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med. 2011;86(9):1120-7.

135. Rizan C, Elsey C, Lemon T, et al. Feedback in action within bedside teaching encounters: a video ethnographic study. Med Educ. 2014;48(9):902-20.

136. Young JQ, Sugarman R, Schwartz J, et al. Faculty and resident engagement with a workplace-based assessment tool: use of implementation science to explore enablers and barriers. Acad Med. 2020;95(12):1937-44.

137. Havel LK, Powell SD, Cabaniss DL, et al. Smartphones, smart feedback: using mobile devices to collect in-the-moment feedback. Acad Psychiatry. 2017;41(1):76-80.

138. Georgoff PE, Shaughness G, Leininger L, et al. Evaluating the performance of the Minute Feedback System: A web-based feedback tool for medical students. Am J Surg. 2018;215(2):293-7.

139. McCutcheon S, Duchemin A-M. Formalizing feedback: introducing a structured approach in an outpatient resident clinic. Acad Psychiatry. 2020;44(4):399-402.

140. Bandiera G, Lendrum D. Daily encounter cards facilitate competency-based feedback while leniency bias persists. CJEM. 2008;10(1):44-50.

141. Martin L, Sibbald M, Brandt Vegas D, et al. The impact of entrustment assessments on feedback and learning: trainee perspectives. Med Educ. 2020;54(4):328-36.

142. Burford B, Illing J, Kergon C, et al. User perceptions of multi-source feedback tools for junior doctors. Med Educ. 2010;44(2):165-76.

143. Pascoe JM, Nixon J, Lang VJ. Maximizing teaching on the wards: review and application of the One-Minute Preceptor and SNAPPS models. J Hosp Med. 2015;10(2):125-30.

144. Smith MA, Cherazard R, Fornari A, et al. A unique approach to faculty development using an Observed Structured Teaching Encounter (OSTE). Med Educ Online. 2018;23(1):1527627.

145. Topps D, Evans RJ, Thistlethwaite JE, et al. The One Minute Mentor: a pilot study assessing medical students’ and residents’ professional behaviours through recordings of clinical preceptors’ immediate feedback. Education for health (Abingdon, England). 2009;22(1):189.

146. Fluit C, Bolhuis S, Grol R, et al. Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles. Med Teach. 2012;34(11):893-901.

147. Sheng AY. Trials and tribulations in implementation of the emergency medicine milestones from the frontlines. West J Emerg Med. 2019;20(4):647-50.

148. Holmboe ES, Yepes M, Williams F, et al. Feedback and the Mini Clinical Evaluation Exercise. J Gen Intern Med. 2004;19(5 Pt 2):558-61.

149. Yama BA, Hodgins M, Boydell K, et al. A qualitative exploration: questioning multisource feedback in residency education. BMC Med Educ. 2018;18(1):170.

150. Ferguson J, Wakeling J, Bowie P. Factors influencing the effectiveness of multisource feedback in improving the professional practice of medical doctors: a systematic review. BMC Med Educ. 2014;14(1):76.

151. Hassell A, Bullock A, Whitehouse A, et al. Effect of rating scales on scores given to junior doctors in multi-source feedback. Postgrad Med J. 2012;88(1035):10-4.

152. Lockyer J, Armson H, Könings KD, et al. In-the-moment feedback and coaching: improving R2C2 for a new context. J Grad Med Educ. 2020;12(1):27-35.

153. Sargeant J, Lockyer JM, Mann K, et al. The R2C2 model in residency education: how does it foster coaching and promote feedback use?. Acad Med. 2018;93(7):1055-63.

154. French JC, Colbert CY, Pien LC, et al. Targeted feedback in the milestones era: utilization of the Ask-Tell-Ask feedback model to promote reflection and self-assessment. J Surg Educ. 2015;72(6):e274-9.

155. Kogan JR, Shea JA. Implementing feedback cards in core clerkships. Med Educ. 2008;42(11):1071-9.

156. Prystowsky JB, DaRosa DA. A learning prescription permits feedback on feedback. Am J Surg. 2003;185(3):264-7.

157. Marcotte L, Egan R, Soleas E, et al. Assessing the quality of feedback to general internal medicine residents in a competency-based environment. Can Med Educ J. 2019;10(4):e32-e47.

158. Clay AS, Que L, Petrusa ER, et al. Debriefing in the intensive care unit: a feedback tool to facilitate bedside teaching. Crit Care Med. 2007;35(3):738-54.

159. Richards ML, Paukert JL, Downing SM, et al. Reliability and usefulness of clinical encounter cards for a third-year surgical clerkship. J Surg Res. 2007;140(1):139-48.

160. Donato AA, Park YS, George DL, et al. Validity and feasibility of the minicard direct observation tool in 1 training program. J Grad Med Educ. 2015;7(2):225-9.

161. Berz JPB, Cheng T, Quintiliani LM. Milestones-based direct observation tools in internal medicine resident continuity clinic. BMC Med Educ. 2017;17(1):240.

162. Singh T, Kundra S, Gupta P. Direct observation and focused feedback for clinical skills training. Indian Pediatr. 2014;51(9):713-7.

163. Harrison R, Tsyrulnik A, Wood DB, et al. An innovative feedback tool leading to improved faculty feedback and positive reception by residents. West J Emerg Med. 2019;21(1):47-51.

164. Hartranft TH, Yandle K, Graham T, et al. Evaluating surgical residents quickly and easily against the milestones using electronic formative feedback. J Surg Educ. 2017;74(2):237-42.

165. Chen F, Arora H, Zvara DA, et al. Anesthesia myTIPreport: a web-based tool for real-time evaluation of Accreditation Council for Graduate Medical Education’s milestone competencies and clinical feedback to residents. A A Pract. 2019;12(11):412-5.

166. Chan TM, Sebok-Syer SS, Cheung WJ, et al. Workplace-based assessment data in emergency medicine: a scoping review of the literature. AEM Educ and Train. 2021;5(3):e10544.

167. Snyder MJ, Nguyen DR, Womack JJ, et al. Testing quick response (QR) codes as an innovation to improve feedback among geographically separated clerkship sites. Fam Med. 2018;50(3):188-94.

168. Page CP, Reid A, Coe CL, et al. Learnings from the pilot implementation of mobile medical milestones application. J Grad Med Educ. 2016;8(4):569-75.

169. Page C, Reid A, Coe CL, et al. Piloting the Mobile Medical Milestones Application (M3App©): A multi-institution evaluation. Fam Med. 2017;49(1):35-41.

170. Gray TG, Hood G, Farrell T. The results of a survey highlighting issues with feedback on medical training in the United Kingdom and how a smartphone app could provide a solution. BMC Res Notes. 2015;8:653.

171. Desai B. A novel use of Twitter to provide feedback and evaluations. Clin Teach. 2014;11(2):141-5.

172. Bölter R, Freund T, Ledig T, et al. Video-assisted feedback in general practice internships using German general practitioners’ guidelines. GMS Z Med Ausbild. 2012;29(5):Doc68.

173. Paukert JL, Richards ML, Olney C. An encounter card system for increasing feedback to students. Am J Surg. 2002;183(3):300-4.

174. Mathew AE, Kumar Y, Angeline RP, et al. Workplace-based assessment of family medicine competencies using “field note tool”: a pilot study. J Family Med Prim Care. 2018;7(6):1458-63.

175. Connolly A, Goepfert A, Blanchard A, et al. myTIPreport and training for independent practice: a tool for real-time workplace feedback for milestones and procedural skills. J Grad Med Educ. 2018;10(1):70-7.

176. Fu R-H, Cho Y-H, Quattri F, et al. “I did not check if the teacher gave feedback”: a qualitative analysis of Taiwanese postgraduate year 1 trainees’ talk around e-portfolio feedback-seeking behaviours. BMJ Open. 2019;9(1):e024425.

177. Harre R. Positioning Theory. The International Encyclopedia of Language and Social Interaction. 2015:1-9.

178. Tuck KK, Murchison C, Flores C, et al. Survey of residents’ attitudes and awareness toward teaching and student feedback. J Grad Med Educ. 2014;6(4):698-703.

179. Gaunt A, Patel A, Fallis S, et al. Surgical trainee feedback-seeking behavior in the context of workplace-based assessment in clinical settings. Acad Med. 2017;92(6):827-34.

180. Watling CJ, Lingard L. Toward meaningful evaluation of medical trainees: the influence of participants’ perceptions of the process. Adv Health Sci Educ Theory Pract. 2012;17(2):183-94.

181. Bowen L, Marshall M, Murdoch-Eaton D. Medical student perceptions of feedback and feedback behaviors within the context of the “educational alliance.”. Acad Med. 2017;92(9):1303-12.

182. Lefroy J, Watling C, Teunissen PW, et al. Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education. Perspect Med Educ. 2015;4(6):284-99.

183. Nussbaum AD, Dweck CS. Defensiveness versus remediation: self-theories and modes of self-esteem maintenance. Pers Soc Psychol Bull. 2008;34(5):599-612.

184. Goffman E. The Presentation of Self in Everyday Life. 1959.

185. Veloski J, Boex JR, Grasberger MJ, et al. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006;28(2):117-28.

186. Chinn CA, Brewer WF. The role of anomalous data in knowledge acquisition: a theoretical framework and implications for science instruction. Rev Educ Res. 1993;63(1):1-49.

187. Butler DL, Winne PH. Feedback and self-regulated learning: a theoretical synthesis. Rev Educ Res. 1995;65(3):245-81.

188. Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: a conceptual model. Acad Med. 2010;85(7):1212-20.

189. Goldflam K, Bod J, Della-Giustina D, et al. Emergency medicine residents consistently rate themselves higher than attending assessments on ACGME milestones. West J Emerg Med. 2015;16(6):931-5.

190. van der Leeuw RM, Teunissen PW, van der Vleuten CPM. Broadening the scope of feedback to promote its relevance to workplace learning. Acad Med. 2018;93(4):556-9.

191. Davies K, Guckian J. How to ask for and act on feedback: practical tips for medical students. MedEdPublish. 2018;7:63.

192. Stone D. Thanks for the Feedback: The Science and Art of Receiving Feedback Well. 2014.

193. van der Leeuw RM, Overeem K, Arah OA, et al. Frequency and determinants of residents’ narrative feedback on the teaching performance of faculty: narratives in numbers. Acad Med. 2013;88(9):1324-31.

194. Algiraigri AH. Ten tips for receiving feedback effectively in clinical practice. Med Educ Online. 2014;19:25141.

195. Kowalski K. Giving and receiving feedback: part II. J Contin Educ Nurs. 2017;48(10):445-6.

196. Kowalski K. Giving and receiving feedback: part I. J Contin Educ Nurs. 2017;48(9):395-6.

197. Bharamgoudar R, Sonsale A. Twelve tips for medical students to make the best use of ward-based learning. Med Teach. 2017;39(11):1119-22.

198. Wright SM, Kern DE, Kolodner K, et al. Attributes of excellent attending-physician role models. N Engl J Med. 1998;339(27):1986-93.

199. Beaulieu AM, Kim BS, Topor DR, et al. Seeing is believing: an exploration of what residents value when they receive feedback. Acad Psychiatry. 2019;43(5):507-11.

200. Telio S, Ajjawi R, Regehr G. The “educational alliance” as a framework for reconceptualizing feedback in medical education. Acad Med. 2015;90(5):609-14.

201. Teunissen PW, Stapel DA, van der Vleuten C, et al. Who wants feedback? An investigation of the variables influencing residents’ feedback-seeking behavior in relation to night shifts. Acad Med. 2009;84(7):910-7.

202. Lacasse M, Lee S, Ghavam-Rassoul A, et al. Integrating teaching into the busy resident schedule: a learner-centered approach to raise efficiency (L-CARE) in clinical teaching. Med Teach. 2009;31(11):e507-13.

203. Gupta A, Villegas CV, Watkins AC, et al. General surgery residents’ perception of feedback: We can do better. J Surg Educ. 2020;77(3):527-33.

204. Bates J, Konkin J, Suddards C, et al. Student perceptions of assessment and feedback in longitudinal integrated clerkships. Med Educ. 2013;47(4):362-74.

205. Chou CL, Masters DE, Chang A, et al. Effects of longitudinal small-group learning on delivery and receipt of communication skills feedback. Med Educ. 2013;47(11):1073-9.

206. Albano S, Quadri SA, Farooqui M, et al. Resident perspective on feedback and barriers for use as an educational tool. Cureus. 2019;11(5):e4633.

207. Watling C, Driessen E, van der Vleuten CPM, et al. Music lessons: revealing medicine’s learning culture through a comparison with that of music. Med Educ. 2013;47(8):842-50.

208. Chinai SA, Guth T, Lovell E, et al. Taking advantage of the teachable moment: a review of learner-centered clinical teaching models. West J Emerg Med. 2018;19(1):28-34.

209. Ruhotina M, Burrell D. A Melting Pot of Medical Education: Challenges, solutions, and opportunities for improving trainee feedback and education in the ED. R I Med J (2013). 2018;101(8):37-40.

210. Driessen E, Scheele F. What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research. Med Teach. 2013;35(7):569-74.

211. Feller K, Berendonk C. Identity matters – perceptions of inter-professional feedback in the context of workplace-based assessment in diabetology training: a qualitative study. BMC Med Educ. 2020;20(1):33.

212. Loeppky C, Babenko O, Ross S. Examining gender bias in the feedback shared with family medicine residents. Educ Prim Care. 2017;28(6):319-24.

213. Mueller AS, Jenkins TM, Osborne M, et al. Gender differences in attending physicians’ feedback to residents: a qualitative analysis. J Grad Med Educ. 2017;9(5):577-85.