| Author | Affiliation |
|---|---|
| Jami DelliFraine, MHA, PhD | University of Texas Health Science Center, Houston, TX |
| James Langabeer, II, PhD | University of Texas Health Science Center, Houston, TX |
| Brent King, MD | University of Texas Health Science Center, Houston, TX |
ABSTRACT
Introduction:
To assess academic emergency medicine (EM) chairs’ perceptions of quality improvement (QI) training programs.
Methods:
A voluntary anonymous 20 item survey was distributed to a sample of academic chairs of EM through the Association of Academic Chairs of Emergency Medicine. Data was collected to assess the percentage of academic emergency physicians who had received QI training, the type of training they received, their perception of the impact of this training on behavior, practice and outcomes, and any perceived barriers to implementing QI programs in the emergency department.
Results:
The response rate to the survey was 69% (N = 59). 59.3% of respondents report that their hospital has a formal QI program for physicians. Chairs received training in a variety of QI programs. The type of QI program used by respondents was perceived as having no impact on goals achieved by QI (χ2 = 12.382; p = 0.260), but there was a statistically significant (χ2 = 14.383; p = 0.006) relationship between whether or not goals were achieved and academic EM chairs’ perceptions about return on investment for QI training. Only 22% of chairs responded that they have already made changes as a result of the QI training. 78.8% of EM chairs responded that quality programs could have a significant positive impact on their practice and the healthcare industry. Chairs perceived that QI programs had the most potential value in the areas of understanding and reducing medical errors and improving patient flow and throughput. Other areas of potential value of QI include improving specific clinical indicators and standardizing physician care.
Conclusion:
Academic EM chairs perceived that QI programs were an effective way to drive needed improvements. The results suggest that there is a high level of interest in QI but a low level of adoption of training and implementation.
INTRODUCTION
Academic emergency medicine (EM) is a practice prone to medical errors due to the volume and complexity of activities that take place.1 Factors such as crowding, resource constraints, cognitive workload, diagnostic uncertainty, high activity levels, interruptions and distractions, shift changes, poor feedback and time pressures can often lead to medical errors.1 Emergency physicians (EP) have an especially critical role, since emergency departments (ED) are increasingly the gateway to acute care and hospitalizations. Between the years 1993 – 2003, there was a 26% increase in the number of annual visits to EDs, but a 12% decrease in the total number of departments.2 An increase in patient demand coupled with a decrease in EDs suggests potential for further crowding, as well as corresponding reductions in physician-patient interaction times. With these increased time pressures and workloads, we propose that it is critical for EPs to focus on continual learning to improve processes of care in EDs and ensure high quality efficient care.
In addition to adapting their behavior to reflect current clinical best practices, other larger organizational issues often affect their ability to practice medicine. EPs often encounter structural and process barriers that impact patient care. For example, structural issues in an ED that can affect quality of care include the design of the department, how it is equipped, availability of ancillary services, staffing mix, and departmental policies and procedures.3 Process issues can also impact the quality of care delivered by EPs. For example, bottlenecks in the ED can create delays in sending and receiving information pertinent to patient care, which can result in either delayed care or misdiagnosis.4 Physician training in quality improvement (QI) programs has the potential to prevent medical errors by training EPs to identify and correct process and systemic weaknesses. However, little is known about the impact of QI training programs on EP behavior and practice.
QI tools, such as Plan, Do, Check, Act (PDCA), total quality management (TQM), Six Sigma and LEAN, are management techniques that focus on improving processes. They are built on a foundation of constant learning by participants, adaptation to better practices and behaviors, and therefore reinforce change. As a result, most QI tools result in structural or process modifications within the organization. These tools are commonly used to identify, analyze, reduce, and monitor medical errors in healthcare organizations today.5, 6 Several tools, such as Six Sigma and TQM, are data driven and focus on the measurement of defects and errors, while others, such as Lean and PDCA, focus on removing waste and unnecessary steps from processes.7–10 Both TQM and Lean focus on cultural changes within the organization to improve quality.6,11
However, getting physicians to change their behavior is often difficult. Educational efforts such as continuing medical education (CME) are largely ineffective at changing physician behavior and performance.12 Additionally, CME rarely trains physicians how to solve issues involving other departments where there is an impact in the quality of care provided to patients in the ED. Furthermore, physicians in academic leadership roles have often achieved their position on the basis of their clinical expertise or scientific accomplishments rather than specific management training. Physicians may lack familiarity with basic QI techniques, but are often charged with maintaining and improving the quality of care, which involves changing the behavior of other physicians. This is a difficult task if physician-managers do not have the management tools to change structures and processes that impact the effective delivery of high-quality medical care. In response, many hospitals and academic medical centers have implemented QI training programs and are familiarizing physicians with these tools in hopes to give physician leaders the skills necessary to change the system and improve quality of care. 13, 14
Currently, there are no studies that examine the QI tools and techniques available to academic EPs. It is unclear how many EPs are being offered QI training, who is being offered the training (department chairs or practitioners) and which types of QI tools are being implemented. The purpose of this research is to assess academic emergency medicine (AEM) chairs perceptions of value, both realized and potential, of QI training programs.
METHODS
Study Design
The purpose of this project is to assess AEM chairs’ perceptions of value, both realized and potential, of QI training programs. No studies published to date have assessed academic EP perceptions of the value of QI training programs. To achieve this purpose, we developed a 20 item survey instrument to assess the percentage of AEM physicians who had received QI training, the type of training they received, their perception of the impact of this training on behavior, practice and outcomes, as well as any perceived barriers to implementing QI programs in the ED. This is a mixed-methods cross-sectional survey design using close-ended items with numerical responses and open-ended items on the same survey. 15 This mixed method survey design was chosen for triangulation purposes, to add breadth and scope to the project. 16
Study Setting and Population
The target population for this study is the chairs of academic EDs in the United States (U.S.). After obtaining Institutional Review Board approval, we gathered data from a sample of academic chairs of EM from the Association of Academic Chairs of Emergency Medicine (AACEM). This organization, comprised of the chairs of autonomous academic departments of EM in the U.S., currently has 85 members who are chairs of academic departments at Liaison Committee on Medical Education approved medical schools. We received a 69% (N = 59) response rate to the survey.
Study Protocol
Surveys were distributed to AACEM members at the annual meeting, as well as converted into an electronic form. To poll the chairs who did not attend the AACEM meeting, E-mails were sent to the AACEM list serve, asking members to participate in the survey chairs. Written informed consent of human subjects was waived by the authors’ Institutional Review Board.
Measurement
Survey items were created to address the specific aims of the study. Table 1 contains a list of specific aims and the corresponding survey questions created to address them. Two questions addressing demographic information were also included to control for length of practice in EM and size of ED.
Data Analysis
We used descriptive, correlation and qualitative analyses to determine the association between perceived impact of QI training, perceived change in practice and behavior, and type of training received. We used qualitative analyses to examine open-ended questions and analyzed themes of behavioral and practice changes analyzed to determine specific changes related to receiving QI training.
RESULTS
Overall, 59.3% of respondents report that their hospital has a formal QI program for physicians. A small percentage (3.4%) of respondents report that their hospital considers adopting a program in the next six months. The remaining respondents (37.3%) report that their hospital has no QI training program for physicians. Of the hospitals that have QI programs, the surveys indicate that they were implemented 20.6 months or a little less than two years ago. A majority (64.8%) of AEM Chairs who responded to our survey have not participated in a program for QI. Of the 19 respondents who participated in QI training, the average number of days spent in training was three, which did not include one outlier who reported over 30 days spent in training. On average, respondents reported it had been 18.4 weeks since returning from QI training, and that they had completed an average of 3.78 projects in that time period.
AEM chairs identified a variety of QI programs at their respective hospitals, but 22.5% chairs could not identify which type of QI program their hospital used. Figure 1 summarizes the types of QI programs identified by academic EPs.
Descriptive analyses show that the majority of respondents (63.2%) report that QI training was conducted by internal trainers. However, chi-square tests show that there is no statistically significant relationship between type of QI program implemented and internal or external trainers (χ2=4.638; p=0.914). Additionally, the type of QI program used by respondents had no impact on goals achieved by QI (χ2=12.382; p=0.260), whether or not changes in quality were measured (χ2=3.656; p=0.600), or perceptions about the return on investment for QI training (χ2=10.827; p=0.371). There was also no statistically significant relationships between who conducted the training and whether goals were achieved (χ2=4.886; p=0.299), changes were measured (χ2=2.825; p=0.244), or perceptions about return on investment for QI training (χ2=4.554; p=0.336). However, there was a statistically significant (χ2=14.383; p=0.006) relationship between if goals were achieved and AEM chairs perceptions about return on investment for QI training. If projects were successful, then the AEM chairs were significantly more likely to have positive perceptions about the return on investment of QI training. These findings did not vary significantly by years of practice or size of ED. When asked about the specific impact of QI training that was most noticeable on outcomes, AEM chairs reported (in the order of frequency) increased: 1 efficiency, 2 patient care quality and 3 morale and teamwork. However, 16% of respondents reported no noticeable outcomes. Increased efficiency included shorter lengths of stay in the ED, fewer patients leaving before treatment, decreased supply spending, and general increased physician productivity. Improved quality of care included fewer errors related to patient hand-offs, shorter wait times for critical procedures, and better nursing care. Improved morale and teamwork, both within the ED and between the ED and other departments, were also reported.
Figure 2 summarizes respondents’ plans to make changes to their behavior based on QI training.
When asked about specific changes initiated in their own practice or department, AEM chairs listed the following changes: decreased time from patient arrival to initial physician evaluation; decreased time from patient arrival to the completion of an electrocardiogram for patients with chest pain; improvement in compliance with Joint Commission core measures for pneumonia; increased compliance with airway management protocols; increasing the proper use of certain bedside tests (e.g. allowing a stool guaiac test to sit 2–3 minutes before reading); improving and standardizing end-of-shift patient hand-offs; improved compliance with Center for Medicare & Medicaid Services Physician Quality Reporting Initiative measures for stroke, myocardial infarction (MI) and pneumonia; implementation of rapid response systems for stroke and MI; creating special order sheets for pneumonia; improvements in teamwork in a “fast track” area; creation and implementation of streamlined admission and discharge processes; improvements in overall patient flow; improvements in team interactions; movement of patients to inpatient hallways to reduce ED crowding; development of rapid triage, bedside registration, and medication reconciliation processes; implementation of administrative “patient safety walk rounds”; development of a “Lean and Green” committee to improve efficiency and decrease environmental impact, creation of a process to improve the completeness of physician documentation, reduction of time from patient arrival to definitive care for patients with acute MI (“door to balloon time”); and increased follow-up for patients who elope from the department and “left without being seen” patients.
Correlation analyses show no statistically significant relationship between changes in behavior or practice and type of QI training received (χ2=21.867; p=0.643). However, there was a statistically significant (χ 2=21.659; p=0.017) relationship between behavioral changes and type of trainers. AEM chairs who received training from internal trainers were significantly more likely to have made changes and work to maintain those changes. Analysis of variance showed there were no statistically significant differences between changes in behavior based on number of weeks since returning from class (F-ratio=0.39; p=0.846), years of practice experience (F-ratio=1.78; p=0.197) or size of ED (F-ratio= 0.29; p=0.907).
When AEM chairs were asked if quality programs could have a significant positive or negative impact on their practice and the healthcare industry, 78.8% responded that the impact would be positive. Only 7.7% perceived that QI programs would have a negative impact, and the remaining 13.5% were not sure. AEM chairs also overwhelmingly (82%) responded that the medical industry has not done enough to improve quality of care, while only 6% disagreed, and 12% were unsure. Table 2summarizes the rankings in the areas of greatest potential value of QI programs.
Correlation analyses showed no statistically significant differences in perceptions of QI program impact between physicians who had participated in training and those who had not (χ2=2.920; p=0.232). However, data did show a strong statistically significant (χ2=30.781; p=0.000) positive relationship between the belief that QI training programs could have a positive impact on the healthcare industry and the belief that the medical industry has done enough to improve quality of care. There was no statistical significance among overall perceptions of QI training, years of practice and size of ED.
DISCUSSION
In this study measuring the perceptions of use and value of QI training programs in AEM, we collected survey data from 69% of the 85 academic chairs in AACEM. This is the first study of their perceptions of QI training and value. The chairs reported that QI programs are relatively new to their EDs. This finding is somewhat surprising given that some QI programs such as TQM and PDCA have been used in hospitals for over 20 years and that a majority of U.S. hospitals have adopted some form of quality management program.17 Although most of their institutions offer training in one or more QI programs, a majority of chairs (64.8%) have not participated in any of these programs. This finding is consistent with recent research that found only about one-third of physicians have participated in system redesign efforts to improve the performance of the system in which they practice.18 Of the chairs who participated in a QI program, most received a brief exposure provided by internal trainers. Nonetheless, many reported that they were able to successfully improve some aspect of their clinical practice using the principles they learned.
The type of QI training (Six Sigma, Lean, TQM, PDCA) received and whether training was administered by external or internal trainers did not seem to affect the chairs’ perceptions of effectiveness. As might be expected, however, those who perceived that the goals of the training had been met were more likely to believe that their training program had been effective. Although this finding is not necessarily surprising, it is surprising that 41.2% of respondents who participated in training were not measuring changes in outcomes, given that Six Sigma, Lean, and TQM propose establishing goals and measuring performance in quantifiable terms. 19–20 This finding suggests that some QI programs are not being implemented correctly, and the lack of measurement of quantifiable outcomes and progress towards goals may affect perceptions of effectiveness of QI programs. Furthermore, the chairs reported that training received from internal trainers was more likely to impact their behavior and desire to continue maintain the changes that they had made. We speculate that the reason for this finding is because internal trainers are likely to serve as QI experts and role models and are more accessible than external trainers who leave after the training is complete.
We found that most chairs perceived that QI is important and has the potential for improving patient care in U.S. EDs. Further analysis of the perceptions of potential benefits by the chairs suggests that QI could help to understand and reduce medical errors and improve patient flow and cycle times, both critical measures of ED performance. When asked about specific changes made to their own practice in order to improve quality, chairs listed numerous implementations to improve quality. These changes are consistent with research that shows that QI programs are currently being used in EDs to improve throughput and reduce medical errors.21–25 Outcomes targeted for improvement include ED workload or volume, patient wait time, ambulance diversion, patient walkout rate, length of stay, patient satisfaction, triage of patients, turn-around time of other services such as laboratory and radiology, available beds, registration, discharge processes and staffing patterns. 21, 23–25
LIMITATIONS
There are several limitations to this study. First, the population of AEM chairs is relatively small. This limits our statistical analyses to chi-square and correlation analyses. Although we received a 69% response rate to the survey, results may have been biased. Chairs interested in QI efforts or who have received training may have been more likely to respond to the survey thereby creating a positive bias in favor of QI in the response. Additionally, the responses to the survey are opinions only and are not quantitative measurements of the programs or types of changes.
CONCLUSION
The number of EDs continues to decline while patient numbers keep increasing. The result is crowding, less patient interaction time for diagnosis and treatment, and the need for more efficient processes, practices and behaviors to improve core quality measures. Academic EDs will play a pivotal role in improving processes of care since they are generally larger facilities and treat more complex and diverse cases. This results in an increased need for EPs who can learn rapidly, and proactively improve processes of care. The implications of this research for physicians and administrators in EM are that QI efforts are perceived to have value. AEM chairs believe that QI programs were an effective way to drive needed improvements. Overall, the results suggest that there is a high level of interest in QI, but a general low level of adoption of training and implementation. Unfortunately, actual testing and demonstration of value has been minimal to date and there are very few EDs that have monitored and recorded quantifiable gains – thereby slowing the adoption of these methods. However, initiation and use of QI methods is especially important for chairs, who serve as leaders, mentors and role models to AEM physicians in the march towards healthcare quality.
Footnotes
Supervising Section Editor: Jeffrey Druck, MD
Submission history: Submitted October 7, 2009; Revision Received March 16, 2010; Accepted April 2, 2010
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Address for Correspondence: Jami L. DelliFraine, MHA, PhD, Assistant Professor, Division of Management, Policy, and Community Health, University of Texas School of Public Health, 1200 Herman Pressler, RAS E-925, Houston, TX 77030
Email: Jami.L.Dellifraine@uth.tmc.edu
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
REFERENCES
1. Croskerry P, Sinclair D. Emergency medicine: a practice prone to error? J Can Assoc Emerg Physicians. 2001;3(4):271–276.
2. McCaig LF, Burt CW. Advance data from vital and health statistics; no 358. Hyattsville, Maryland: National Center for Health Statistics; 2005. National Hospital Ambulatory Medical Care Survey: 2003 emergency department summary.
3. Graff L, Stevens C, Spaite D, et al. Measuring and improving quality in emergency medicine. Acad Emerg Med. 2002;9(11):1091–107. [PubMed]
4. Asaro PV, Lewis LM, Boxerman SB. The impact of input and output factors on emergency department throughput. Acad Emerg Med. 2007;14(3):235–42. [PubMed]
5. Schroeder RG, Linderman K, Zhang D. Evolution of quality: first fifty issues of production and operations management. Prod Oper Manag. 2005;14(4):468–82.
6. Kelly D. Applying quality management in healthcare: a systems approach. 2nd ed. Chicago, IL: Health Administration Press; 2007.
7. Chassin MR. Is health care ready for Six Sigma quality? Milbank Q. 1998;76:565–91.[PMC free article] [PubMed]
8. Dean JW, Bowen DE. Management theory and total quality: improving research and practice through theory development. Acad Manag Rev. 1994;19(2):392–418.
9. Alukal G. All about Lean. Qual Prog. 2006;39(2):74–76.
10. Deming WE. Out of the Crisis. Vol. 2000. Cambridge, MA: The MIT Press;
11. Alukal G. Lean Kaizen in the 21st century. Qual Prog. 2007;40(8):69–71.
12. Davis DA, Thomson MA, Oxman AD, et al. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700–5. [PubMed]
13. Hemmer PR, Karon BS, Hernandez JS, et al. Leadership and management training for residents and fellows: a curriculum for future medical directors. Arch Pathol Lab Med. 2007;31(4):610–5.[PubMed]
14. Wurster AB, Pearson K, Sonnad SS, et al. The patient safety leadership academy at the University of Pennsylvania: the first cohort’s learning experience. Qual Manag Health Care. 2007;16(2):166–173. [PubMed]
15. Tashakkori A, Teddlie C. Mixed methodology: combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications; 1998.
16. Green JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy An. 1989;11:255–74.
17. Barsness ZI, Shortell SM, Gillies RR, et al. The quality march. Hosp Health Networks. 1993 Dec;:52–5.
18. Audet AJ, Doty MM, Shamasdin J, et al. Measure, learn, and improve: Physicians involvement in quality improvement. Health Aff. 2005;24(3):843–53.
19. Harry M, Schroeder R. Six Sigma. New York: Doubleday Publishers; 2000.
20. Printezis A, Gopalakrishnan M. Current pulse: Can a production system reduce medical errors in health care? Qual Manag Health Care. 2007;16(3):226. [PubMed]
21. Ben-Tovim DI, Bassham JE, Bennett DM, et al. Redesigning care at the Flinders Medical Centre: clinical process redesign using “lean thinking” Med J Aus. 2008;188(6):S27–S31.
22. Johnson T, Currie G, Keill P, et al. New York-Presbyterian Hospital: translating innovation into practice. J Qual Patient Safety. 2005;31(10):554–60.
23. Christianson JB, Warrick LH, Howard R, et al. Deploying Six Sigma in a health care system as a work in progress. J Qual Patient Safety. 2005;31(11):603–13.
24. Dickson EW, Singh S, Cheung DS, et al. Application of Lean manufacturing techniques in the emergency department. J Emerg Med. 2008;11(108):1–6.
25. Kelly A, Bryant M, Cox L, et al. Improving emergency department efficiency by patient streaming to outcomes-based teams. Aus Health Rev. 2007;31(1):16–21.
