Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

The value of systematic error analysis has long been recognized in healthcare. Pioneered by Earnest Codman at the turn of the 20^th century,¹ and famously reinforced 100 years later in the Institute of Medicine’s landmark To Err is Human,² the importance of routine case reporting and detailed case review is now widely accepted as foundational in the practice of medicine. Integral in training, the Accreditation Council for Graduate Medical Education now mandates structured case review and discussion, or morbidity and mortality (M&M) conferences, as a part of their system-based practice and practice-based learning and improvement domains.³

Despite the express interest in this activity supporting system-based practice, M&Ms have instead traditionally been focused on individual cognitive errors and have often further reinforced a “blame and shame” culture in medicine, undermining the effectiveness of these conferences.^4,5,6 For trainees in particular, it has been noted that the impact of M&M conferences that focus on individual cognitive errors is to increase fear of blame resulting in decreased participant engagement, lower likelihood of reporting safety events, and an overall decrease in the effectiveness of the conferences.⁷ Despite this, there are no best practices to guide residency programs in the design of M&M conferences to mitigate the fear of blame, and to promote a non-punitive case discussion. To our knowledge, no work to date has looked to determine the association of different structural elements of M&M conferences and residents’ perceptions of the conference and the overall impact on the culture of safety.

Recent papers have described the characteristics of emergency medicine (EM) M&Ms nationally, and found significant variation in core structural elements.^8,9,10 Given the fear of “blame and shame,” we hypothesized that anonymous case discussion would cultivate a more effective, and less punitive, M&M conference. Secondarily, we were interested in determining whether this core structural element was associated with the overall culture of safety at an institution. We conducted a national survey, evaluating residents’ perceptions of M&M and the culture of safety at their institutions.

METHODS

Study Setting and Measurement

Study setting and measurement is discussed in detail in a previous paper.⁹ This is a convenience sample derived from all United States EM residency programs. We invited all programs to participate in a survey of all residents: 33 programs were both willing to participate and able to identify a local champion to serve as a co-investigator to help ensure a high response rate. The survey was conducted in May 2015, using a tool that was previously piloted with residency program directors (PD).⁹ This included questions used in a previous survey of EM PDs⁹ as well as questions from the validated Agency for Health Care Research and Quality (AHRQ) Safety Culture survey.¹¹ These questions are designed to assess aspects of a strong safety culture, including a non-punitive environment, comfort submitting and discussing errors, and an environment in which mistakes lead to positive change.

Population Health Research Capsule

What do we already know about this issue?

Structured case review is mandated by the Accreditation Council for Graduate Medical Education, but there are no best practices in the design of these conferences.

What was the research question?

Does anonymous case discussion cultivate a more effective, and less punitive, morbidity and mortality (M&M) conference?

What was the major finding of the study?

Anonymous M&Ms are associated with a perception of a less-punitive M&M and with better ratings in several conference-specific outcomes.

Analysis

We analyzed data using descriptive statistics and bivariate analyses. Likert scores were summarized using mean and 95% confidence intervals (CI). We calculated composite culture of safety scores by using the average of the four AHRQ safety domains surveyed. Anonymous and non-anonymous comparison was made using paired t-test. Data analysis was performed with STATA MP 13.1 (StataCorp, College Station, TX). We performed qualitative analysis of the free-text comments using conceptual content analysis.

To begin, a set of thematic codes was developed by three of the emergency physician investigators (ELA, JDS, KW) through an iterative reading of all reports. Subsequently, one author (ELA) used content-analysis techniques to code all transcripts in NVivo qualitative data analysis software version 10, 2014 (QSR International Pty Ltd. Doncaster, Australia). Multiple themes could be applied to a single response as appropriate. The final coded text and example quotations were reviewed with two other investigators (KW, JDS) iteratively until there was agreement on the coding structure. Descriptive statistics were generated to summarize the data and quantify the frequency of themes.

RESULTS

There were 1,248 residents at 33 programs surveyed during the study period. Of the 1002 who replied (80.3% response rate), 231 respondents reported anonymous case presentations, and 744 reported non-anonymous case presentations. There were no differences in the structural elements of the residency training programs between anonymous and non-anonymous respondents (Table 1).

When asked about features of M&M specifically, residents reporting anonymous case presentation reported that M&M was less punitive (Table 2; difference in percent agreeing = 8.21; 95% CI, 11.66–4.77; p<0.05). Regarding case submission, residents reporting anonymous case presentation trended toward being more comfortable submitting cases in which they were not involved as compared to residents reporting non-anonymous case presentations (difference in percent agreeing = 6.64; 95% CI, 7.24–14.01; p = 0.08). There was little difference between the groups in their degree of comfort when reporting cases they were involved in (difference in percent agreeing = 0.38; 95% CI, 6.44–7.21; p = 0.91).

Greater than 85% of all residents surveyed agreed that M&M was of value to their education, with no significant difference between the two groups (difference in percent agreeing =1.49; CI, 6.49–3.49; p = 0.55). Those reporting anonymous case presentation did report that case discussion was more focused on system errors (difference in percent agreeing = 6.73; CI, 0.85–12.62; p = 0.03); however, the two groups reported no significant difference in the perception that the discussions were more focused on cognitive errors (difference in percent agreeing = 0.004; CI, 7.22–7.31; p = 0.99]) The majority of residents surveyed agreed that mistakes led to positive change (65.9% of residents reporting anonymous case presentations; 68.6% of residents reporting non-anonymous case presentations).

Table 3 shows the positive composite score for the culture of safety survey stratified by the two different types of case presentation. There remained a significant difference between the two groups related to the perception of the punitive nature of the conference, with residents at programs with anonymous M&Ms significantly less likely to report that the M&M felt punitive.

When asked “Have you had a negative experience with having a case of yours discussed at M&M?,” there was no statistical difference between respondents reporting an anonymous case presentation described negative experiences and respondents reporting non-anonymous case presentations described negative experiences (2.4% vs 0.8%; p = 0.188). Similarly, there was no statistically significant difference between positive experiences reported by respondents reporting anonymous case presentations and respondents reporting non-anonymous case presentations (6.9% vs 8.7%; p = 4.16).

Narrative Comments

When we systematically analyzed and coded these statements, we found that comments related specifically to anonymity were very common and both positive and negative. The majority of comments from anonymous programs were related to non-punitive responses to error, with residents noting that “I feel bad about a decision that I made that I should have done differently; however, people are not punitive, they try to keep the discussion academic”; and “the environment is generally very constructive and not punitive, which makes it much easier to accept criticisms of the care I provided.” Another theme identified associated with anonymous case discussions was related to the impact of the providers not being present for the case discussion, noting that as a negative experience: the “Attending who drove most of decision making of case was not present.” While these two things, anonymity and the absence of the team involved, are not clearly related it does stand to reason that if the team will not be identified it becomes easier for them not to attend.

Additional themes identified were related to anonymity decreasing the punitive nature of these conferences and increasing the focus on systems and ability to learn from cases: “M&M for us is completely anonymous and focused on systems errors and ways to avoid a similar error in the future. It does not feel punitive or finger-pointing. It was very interesting to hear the discussion of a case I was involved in, and allowed me to better process a poor patient outcome and give me ideas on how to prevent a similar error in the future.” However, another resident noted that the same anonymous framework can lead to a loss of the context related to the original medical decision-making: “I don’t like that our institution does not allow the person involved in the case to OWN the case. Instead you listen to people talking about what they would have done etc. but the person involved in the case is not allowed to stand up and explain their motivations because it has to be anonymous.”

Other residents provided examples of hybrid approaches to anonymity and described the ability of non-anonymous conferences to provide closure; however, “at our institution the attendings and residents involved in the case are free to identify themselves and their experience but there is no pressure. When my case was presented, I thought it was helpful to discuss my experience, thought process during the case, and to ask if others in the room would have approached it differently. This gave me a sense of closure and afterwards I felt more resolution regarding the care of that patient.” Another resident describing a hybrid conference noted the limitations of the non-anonymous structure, impeding honest discussion, perhaps related to self-censorship: “at our institutions the presenter never identifies the resident or attending on the case but the resident and attending frequently self-identify and start discussing the case. That then makes it very difficult for others to comment honestly and changes the tone.”

A common theme in the comments from non-anonymous programs was related to residents perceptions of punitive responses to error, such as “I felt as though I was blamed by one of our senior attendings for this in front of the M&M attendees, even though I had ultimately no power in the decision made by the spine surgery team.” Alternatively, another theme identified was related to the non-anonymous conference’s ability to cultivate professionalism and accountability. One resident asserted that, “it is helpful to watch more seasoned providers accept responsibility”; and another resident noted, “It teaches patient safety, personal accountability, and management of difficult cases.”

Another theme that arose exclusively in the non-anonymous group was related to the absence of change resulting from case discussion. One resident noted that “I have as of yet been informed of any system change to address the issue” and noting concern that “when a system process is changed as a result of questions and answers, it usually is not effectively communicated to the group (Attendings and Residents) and is often not widely adopted.” Despite these concerns, residents reporting non-anonymous case presentations did point out the conferences’ ability to provide emotional support to the clinicians involved in the cases discussed, commenting that “ This gave me a sense of closure and afterwards I felt more resolution regarding the care of that patient” and “allowed me to better process a poor patient outcome.”

DISCUSSION

In this national survey of residents’ perception of M&M conferences and their institutional cultures of safety, we found that residents reporting anonymous M&Ms were less likely to report that the M&M felt punitive and more likely to report that case discussions were focused on system issues. We found no other association between the AHRQ culture of safety scores and anonymity in M&M. As we think about the core elements of a strong safety culture that could be cultivated through M&M it becomes important that these conferences are designed to encourage robust case reporting, cultivate a nonpunitive environment for discussion, and provide clear follow-up for issues discussed.¹² Our study suggests that residents at institutions with anonymous M&Ms feel the case discussions are less punitive and that they focus more on systems errors.

In keeping with our hypothesis, we believe that this impact stems from a relationship between the fear of individual blame for case outcomes and being explicitly named in case discussion. It should be noted, however, that it was still a small minority of residents, from either conference structure, who felt these conferences are punitive. Instead, only 16% of residents at anonymous programs and 31% of residents at non-anonymous programs felt that these discussions were punitive. Although other indicators, such as the educational value of the conference, showed no change between the two structures, these also had the clear majority of respondents from both programs (86% anonymous, 88% anonymous) agreeing that the conferences were of value.

Our study showed that, despite the impact of anonymous M&M on some indicators of safety culture, there was no impact on several others. This likely reflects the fact that M&M conferences are only one small determinant of an institution’s culture of safety and this structure alone is not enough to modify the overall culture. This was reinforced by the qualitative analysis, which demonstrated that there were both residents who felt that anonymity cultivated a safety culture, and those who felt it hindered it. This further demonstrates the complexity of safety culture and reinforces that any single input, such as M&M conferences, is only one factor in determining the overall culture.

The analysis of free-text comments provided deeper insight into the nuances surrounding anonymous case presentations, painting pictures of both residents for whom anonymity provided a non-punitive environment that enabled the discussion of systems issues, as well as those for whom anonymity was frustrating and obstructed the ability of the providers to accurately relay the details of the case. The same was true for those reporting non-anonymous case discussions, with some residents recalling situations in which this structure led to them feeling personally attacked or abandoned; however, others described the important impact that this structure had on cultivating personal accountability and professionalism.

LIMITATIONS

There were several limitations to our study. As with all survey-based research, our study was prone to response bias. Although we had a robust response rate, with a >80% response rate across the study population and > 70% at each institution, only 33 programs out of the 151 programs in the country elected to participate. We suspect that those that did elect to participate were more likely to have stronger safety cultures and that, therefore, our results were biased toward a smaller effect size of anonymity. The survey questions themselves, although taken from a previously piloted survey for PDs and validated AHRQ questions, did not undergo formal psychometric testing as this set of questions. The qualitative analysis of the free-text comments has limitations typical for qualitative analysis, that our findings are hypothesis generating and not generalizable.

CONCLUSION

In this national survey of EM residents, we found that anonymous M&Ms are associated with a non-punitive perception of the conference. Future study should focus on the impact, within a single program, of anonymous case discussion, as well as other structural elements of M&M conference.

Footnotes

Section Editor: Chris Merritt, MD, MHPE

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Emily L. Aaronson, MD, MPH, Lawrence Center for Quality and Safety, Massachusetts General Hospital, Department of Emergency Medicine, 55 Fruit St., Bulfinch Building, Suite 280, Boston, MA 02115. Email: Emily.aaronson@mgh.harvard.edu. 1 / 2020; 21:127 – 133

Submission history: Revision received July 15, 2019; Submitted October 17, 2019; Accepted October 11, 2019

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Donabedian A. The end results of health care: Ernest Codman’s contribution to quality assessment and beyond. Milbank Q. 1989;67(2):233-56.

2. Gruen R. To Err Is Human. 1999.

3. ACGME Common Program Requirements 2002–2019. Available at: https://www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements. Accessed May 10, 2019.

4. Harbison SP, Regehr G. Faculty and resident opinions regarding the role of morbidity and mortality conference. Am J Surg. 1999;177(2):136-9.

5. Orlander JD, Fincke BG. Morbidity and mortality conference: a survey of academic internal medicine departments. J Gen Intern Med. 2003;18(8):656-8.

6. Gordon L. Can Cedars-Sinai’s “M+M Matrix” save surgical education?. Bull Am Coll Surg. 2004;89(6):16-20.

7. Wu AW, Folkman S, McPhee SJ, et al. Do house officers learn from their mistakes?. Qual Saf Heal Care. 2003;12(3):221-8.

8. Wittels K, Aaronson E, Dwyer R, et al. Emergency medicine morbidity and mortality conference and culture of safety: the resident perspective. AEM E&T. 2017;1(3):191-9.

9. Aaronson EL, Wittels KA, Nadel ES, et al. Morbidity and mortality conference in emergency medicine residencies and the culture of safety. West J Emerg Med. 2015;16(6):810-7.

10. Seigel TA, McGillicuddy DC, Barkin AZ, et al. Morbidity and mortality conference in emergency medicine. J Emerg Med. 2010;38(4):507-11.

11. Sorra J, Gray L, Streagle S, et al. AHRQ Hospital Survey on Patient Safety Culture: User’s Guide. 2016. Available at: http://www.ahrq.gov/professionals/quality-patientsafety/patientsafetyculture/hospital/index.html. Accessed January 3, 2019.

12. McVeigh TP, Waters PS, Murphy R, et al. Increasing reporting of adverse events to improve the educational value of the morbidity and mortality conference. J Am Coll Surg. 2013;216(1):50-6.

Share on Facebook

Tweet

Share via email