Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Patient satisfaction is an increasingly important metric that is being measured in emergency departments (ED) across the country. Patient satisfaction scores have a wide effect on outcomes for patients, providers, and healthcare organizations. For patients, previous studies have associated high levels of patient satisfaction with improved outcomes.¹ In terms of outcomes for providers, high satisfaction scores have been associated with a lower rate of patient complaints and possibly a lower rate of malpractice claims.^2,3 In addition, patient satisfaction scores are commonly tied to physician compensation. A recent study reported that 59% of physicians reported that their compensation was tied to their satisfaction scores in what is essentially a pay-for-performance model. Conversely 20% of providers reported that their employment had been threatened as a result of patient satisfaction data.⁴ For healthcare organizations, patient satisfaction scores are playing an increasing role in determining overall reimbursement. In 2012 the Center for Medicare and Medicaid services (CMS) began development of a Hospital Consumer Assessment of Healthcare Providers and Systems (HCAPS) patient satisfaction survey. While the exact implementation of this program is unclear, it stands to tie a significant portion of reimbursement from Medicaid to the results of the patient satisfaction survey.⁵

Given the increasing importance placed on patient satisfaction in EDs nationwide, extensive efforts have been made to identify factors that contribute to patient satisfaction; and interventions have been developed to improve overall satisfaction. Staff interpersonal skills, perceived wait times, effective and timely analgesia, the use of ED structure information cards and follow-up telephone calls have all been shown to influence patient satisfaction.^6-10 Various strategies have been employed to improve overall patient satisfaction from adjusting provider staffing to standardizing communication with patients, and even playing background music in the ED.^11-13

To date there is minimal evidence to suggest that providers are able to accurately predict patient satisfaction. As an increasing number of programs are designed to improve patient satisfaction, it is crucial that providers are able to accurately estimate patient satisfaction. Our study evaluated the provider’s ability to predict patient satisfaction.

METHODS

Study Design and Setting

This cross-sectional was performed at the University of Alabama at Birmingham (UAB) Hospital, an urban academic teaching hospital. The ED has 50 beds, all of which are private, and sees approximately 72,000 patients per year. The institutional review board at UAB approved this study.

Selection of Participants

Based on research staff availability, a convenience sample of 100 eligible patients was selected at the time of discharge from the ED. Patients were enrolled over a two-month period (July–August 2013), seven days a week between 6am and 11pm. Research staff used the Cerner FirstNet Triage and Tracking system to identify patients being dispositioned. Eligibility criteria included English-speaking ambulatory patients over 18 years old and deemed healthy enough to participate in the study. Exclusion criteria included intoxicated patients, prison inmates, patients with a primary psychiatric diagnosis, patients 18 years of age and under, and patients who entered the department as a trauma alert.

Method of Measurement

At the time of disposition, research assistants approached eligible patients in their private treatment rooms. Prior to the patient exiting the department, study staff administered a face-to-face interview that consisted of 10 questions regarding their satisfaction with the visit and their pain management. Each of the questions were Likert items that allowed the patient to rate their satisfaction with physician care, nursing care, pain level and their understanding of the discharge instructions on a scale of 1 to 5.

Our survey was designed to resemble the format of our Press Ganey ED patient satisfaction surveys. Responses were measured on a scale of 1 to 5.¹⁴ Our survey asked the patient to verbally rate their satisfaction with their nursing care, satisfaction with their physician, their pain level at the time of discharge and their understanding of discharge instructions. (See Addendum 1.) After interviewing the patient, research assistants interviewed the treating resident physician and nurse. Physicians and nursing staff were blinded to patient responses.

Data Analysis

We used descriptive statistics to characterize the patient sample. We used kappa statistics to evaluate the agreement between patient and physician responses and patient and nurse responses on like questions from their respective interviews. To account for the ordered data, we used weighted kappa. To provide a clear analysis, we simplified the five-point scale used in the interview into a three-point scale. On the three-point scale, we categorized scores of 1–2 as “satisfied,” 3 as “neither” satisfied nor dissatisfied and 4–5 as “dissatisfied.”

We used Excel to manage data (Microsoft, Inc., Redmond, Washington) and performed statistical analyses with Stata v. 13.0 (Stata, Inc., College Station, Texas).

RESULTS

Demographics (Table 1)

The mean age of patients completing the survey was 49.9 years. Thirty-one patients were rated with an emergency severity index of 2, 61 were rated 3, and 8 were rated 4. Sixty-six patients spent less than four hours in the ED, and a total of 45 patients were admitted.

Patient Satisfaction (Tables 2 and3)

Overall patients had a high degree of satisfaction with 99% of patients reporting that they were satisfied with the nursing care and 96% reporting a similar degree of satisfaction with physician care. Despite the generally high level of satisfaction, there was poor agreement between the patient’s responses and the provider’s estimation of these responses. There was poor agreement between nurse and patient responses (weighted κ=0.11, standard error: 0.043);); similarly, there was poor agreement between physician and patient responses (weighted κ=0.23, standard error: 0.078). Providers tended to underestimate the patient’s degree of satisfaction. Providers underestimated patient satisfaction with nursing in 12% of cases and with physicians in 18% of cases. There were no cases where nurses overestimated patient satisfaction and only one case where physicians overestimated patient satisfaction.

Pain Levels (Table 4)

Patients reported a wide variety of pain levels at discharge with ~62% reporting no pain, 20% reporting moderate pain, and 18% reporting that they were experiencing the a high level of pain. Both physician and patient responses (weighted κ=0.43, standard error: 0.082) and nurse and patient responses (weighted κ=0.39, standard error: 0.081) had poor agreement when estimating a pain level. Both nurses and physicians underestimated the patient’s pain in 20% of cases. In 8% of the nurse’s cases and 6% of the physician’s cases, the patient reported severe pain while the providers predicted that the patient was in no pain.

Discharge Instructions (Table 5)

Patients reported that they fully understood their discharge instructions in 87% of the cases. Nurses underestimated patient comprehension in 6% of cases and physicians underestimated this response in 17% of cases. Nurses overestimated comprehension in 12% of cases, compared to 10% with physicians. There was poor agreement between physician and patient responses (weighted κ=0.19, standard error: 0.082) as well as nurse and patient responses. (weighted κ=0.13, standard error: 0.078)

We found that there was poor agreement between patient responses and provider estimates of the patient responses across all aspects of our survey. Overall providers tended to underestimate the level of patient satisfaction. In addition, providers underestimated patient’s level of pain at discharge, and tended to underestimate a patient’s comprehension of their discharge instructions.

DISCUSSION

In our study we found that providers are not able to reliably predict patient responses to questions similar to those commonly found on patient satisfaction surveys. While previous studies have attempted to identify factors that may contribute to patient satisfaction, few studies to date have evaluated provider’s ability to predict patient satisfaction.

Boudreaux et al. examined whether providers were able to accurately estimate patient satisfaction. Providers were asked to predict the patient’s response to a 22-question survey that focused on overall satisfaction and included questions regarding satisfaction with nursing and physician care and understanding of discharge instructions. Responses were obtained from 478 patients and 59 providers. The authors found that providers consistently underestimated the patients’ reported satisfaction.¹⁵

Our study addressed a major limitation that was found in the paper by Boudreaux et al. Rather than asking providers to estimate patient satisfaction with a particular visit, they were asked to estimate the overall survey results for all the patients that had been seen in the department during the study period. Asking providers to provide such a general estimate of satisfaction provides little information in terms of their ability to predict patient responses in particular situations. Our study asked providers to predict a response for an individual patient interaction. By focusing on a particular patient interaction, our data more closely evaluated the provider’s ability to assess various variables and predict patient satisfaction.

In our study we found that providers have difficulty predicting patient satisfaction. Our providers tended to underestimate the level of satisfaction that patients had with their care. Due to a high overall rate of satisfaction, our providers did not encounter a large number of unsatisfied patients. Providers did correctly identify all patients that were either dissatisfied or neither satisfied nor dissatisfied their care. Unfortunately, based on the extremely low incidence of dissatisfaction and relatively small sample size, our study does not provide compelling evidence that providers can accurately predict dissatisfied patients.

We found that providers underestimated patient pain at the time of discharge. Cases where a provider underestimates the patient’s pain accounted for 20% of our visits. These visits may represent instances where the patients received inadequate analgesia. Previous studies have suggested that adequate pain control can improve patient satisfaction.¹⁶ Our data suggests that providers are not able to reliably predict a patient degree of pain at discharge. Given the association between patient satisfaction and pain level, ED providers should focus on performing a more accurate assessment of a patient’s pain and providing appropriate analgesia prior to discharge.

The majority of our patients reported that they understood their discharge instructions. Despite the reportedly high level of comprehension, providers underestimated patient comprehension in 10–12% of cases. Boudreaux et al. found that patient comprehension of discharge instructions was a significant predictor of overall patient satisfaction.¹⁷ Providers should continue to focus on identifying patients who have poor comprehension of discharge instructions in order to improve overall satisfaction.

Our study illustrates the difficulty providers have when they are asked to predict patient responses to questions regarding patient satisfaction. Nationwide as more attention is placed on improving overall patient satisfaction, various initiatives have been developed in an effort to enhance the patient’s ED visit. Programs such as hourly rounding on ED patients, developing a system for follow-up communication after discharge from the ED have been credited with improving patient satisfaction scores. Typically these initiatives are applied to broad range of patients, such as all patients discharged home, rather than targeting specific patients who are at risk of having a low level of patient satisfaction.¹⁸

The inability of providers to accurately predict patient responses may lead to poor resource utilization when implementing programs to improve patient satisfaction. We found that providers had difficulty distinguishing between satisfied and unsatisfied patients. Identifying unsatisfied patients could allow departments to focus resources on improving particular at-risk interactions rather than applying broad initiatives to all patients in the ED. Our study demonstrated that providers have difficulty predicting patient responses to a wide variety of satisfaction metrics. As efforts to improve patient satisfaction continue to grow, departments should also focus on enhancing a provider’s ability to accurately assess patient satisfaction.

LIMITATIONS

We surveyed patients at the end of their ED visit, while previously most patients have received surveys at home several days to weeks after their ED visit. This delay between discharge and responding to the survey may influence the patient’s responses; therefore, our results may not be generalizable.

We relied on patient assessment of comprehension of their discharge instructions. We chose this subjective estimation to approximate the questions found on common patient satisfaction surveys. It is possible that patients overestimated their actual comprehension, as previous studies have reported a much lower rate of comprehension than we found, with Engel et al. reporting significant knowledge deficits in terms of return instructions and home care instructions in up to 80% of patients discharged from the ED.¹⁹ While our data accurately reflects patient self-assessment, it may not reflect actual comprehension of discharge instructions.

We used a five-point pain scale to assess patient’s satisfaction with their pain control, as is common in other satisfaction surveys. Commonly in the ED pain is scored on a 0–10 point scale. There may be some inconsistency in patient responses when they are asked to respond using a five-point scale after using a 10-point scale during their ED visit. While patient responses using a five-point scale may not be directly transferrable to the standard 10-point system, our data accurately reflect the scoring used on common patient satisfaction studies.

A disproportionately large subset of included patients were admitted, potentially skewing results.

CONCLUSION

Physicians and nurses are not able to accurately predict patient responses to standard patient satisfaction surveys. As increasing emphasis is being placed on patient satisfaction nationwide, efforts should be made to improve a provider’s ability to predict a patient’s level of satisfaction with his or her care.

Footnotes

Section Editor: Kenneth S. Whitlow, DO

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: David Page, MD, University of Alabama at Birmingham, Department of Emergency Medicine, 619 19th Street South, Birmingham, AL 35249. Email: dpage@uabmc.edu. 12 / 2015; 16:1088 – 1093

Submission history: Revision received July 20, 2015; Accepted September 3, 2015

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

REFERENCES

1. Kipnis A, Rhodes KV, Burchill CN, et al. The relationship between patients’ perceptions of team effectiveness and their care experience in the emergency department. J Emerg Med. 2013;45(5):731-738.

2. Hickson GB, Federspiel CF, Pichert JW, et al. Patient complaints and malpractice risk. JAMA. 2002;287(22):2951-2957.

3. Moore PJ, Adler NE, Robertson PA. Medical malpractice: the effect of doctor-patient relations on medical patient perceptions and malpractice intentions. West J Med. 2000;173(4):244-250.

4. Zgierska A, Rabago D, Miller MM. Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer Adherence. 2014;8:437-446.

5. Zusman EE. HCAHPS replaces Press Ganey survey as quality measure for patient hospital experience. Neurosurgery. 2012;71(2):N21-24.

6. Bursch B, Beezy J, Shaw R. Emergency department satisfaction: what matters most?. Ann Emerg Med. 1993;22(3):586-591.

7. Hedges JR, Trout A, Magnusson AR. Satisfied Patients Exiting the Emergency Department (SPEED) Study. Acad Emerg Med. 2002;9(1):15-21.

8. Guss DA, Gray S, Castillo EM. The impact of patient telephone call after discharge on likelihood to recommend in an academic emergency department. J Emerg Med. 2014;46(4):560-566.

9. Fosnocht DE, Swanson ER, Bossart P. Patient expectations for pain medication delivery. Am J Emerg Med. 2001;19(5):399-402.

10. Krishel S, Baraff LJ. Effect of emergency department information on patient satisfaction. Ann Emerg Med. 1993;22(3):568-572.

11. Sweeney LA, Warren O, Gardner L, et al. A simulation-based training program improves emergency department staff communication. Am J Med Qual. 2014;29(2):115-123.

12. Parlar Kilic S, Karadag G, Oyucu S, et al. Effect of music on pain, anxiety, and patient satisfaction in patients who present to the emergency department in Turkey. Jpn J Nurs Sci. 2015;12(1):44-53.

13. McDonough KS, Pemberton M. Evaluation and development of an ED management model: an effort to optimize patient-centered care. J Emerg Nurs. 2013;39(5):485-490.

14. Schwartz TM, Tai M, Babu KM, et al. Lack of association between press ganey emergency department patient satisfaction scores and emergency department administration of analgesic medications. Ann Emerg Med. 2014;64(5):469-481.

15. Boudreaux ED, Ary R, Mandry C. Emergency department personnel accuracy at estimating patient satisfaction. J Emerg Med. 2000;19(2):107-112.

16. Bhakta HC, Marco CA. Pain management: association with patient satisfaction among emergency department patients. J Emerg Med. 2014;46(4):456-464.

17. Boudreaux ED, Ary RD, Mandry CV, et al. Determinants of patient satisfaction in a large, municipal ED: the role of demographic variables, visit characteristics, and patient perceptions. Am J Emerg Med. 2000;18(4):394-400.

18. Menchine M, Oberfoell S, Schriger D, et al. Improving telephone follow-up for patients discharged from the emergency department: results of a randomized controlled trial. Acad Emerg Med. 2013;20(5):456-462.

19. Engel KG, Buckley BA, Forth VE, et al. Patient understanding of emergency department discharge instructions: where are knowledge deficits greatest?. Acad Emerg Med. 2012;19(9):E1035-1044.