Author	Affiliation
Lalena M. Yarris, MD, MCR	Oregon Health & Science University, Department of Emergency Medicine, Portland, Oregon Oregon Health & Science University, Center for Policy and Research in Emergency Medicine, Portland, Oregon
Brooke Frakes, BS	Oregon Health & Science University, Department of Emergency Medicine, Portland, Oregon Oregon Health & Science University, Center for Policy and Research in Emergency Medicine, Portland, Oregon
Nathan Magaret, MD	Providence St Vincent Hospital, Department of Emergency Medicine, Portland, Oregon
Annette L. Adams, MPH	Oregon Health & Science University, Department of Emergency Medicine, Portland, Oregon Oregon Health & Science University, Center for Policy and Research in Emergency Medicine, Portland, Oregon
Heather Brooks, BS	Calypte Biomedical Corporation, Portland, Oregon
Robert L. Norton, MD	Oregon Health & Science University, Department of Emergency Medicine, Portland, Oregon Oregon Health & Science University, Center for Policy and Research in Emergency Medicine, Portland, Oregon

INTRODUCTION

Patient satisfaction is one measure of emergency department (ED) quality of healthcare.^1,2 Previous studies have demonstrated that satisfied patients are more likely to comply with discharge instructions, return for future care, and refer friends and family to the same ED.^2–5 Factors that have been shown to be associated with increased patient satisfaction include perceived quality of communication between provider and patient, provider efforts to enhance patients’ understand of their care, perceived and expected waiting times, and higher acuity of illness or triage level.^5–14

While it is assumed that providers make some assessment of patient satisfaction in the course of their clinical care, little is known about how accurate these assessments are. However, provider perceptions of patient satisfaction may impact patient care. For example, providers who perceive that their patients are satisfied with care are more likely to report a positive work environment thus setting the stage for a positive physician-patient interaction. Likewise, providers who sense that patients are dissatisfied may feel defensive and experience burnout, which may negatively influence patients’ overall satisfaction with care.¹⁵ The goal of the study is to explore how well ED providers can estimate patient satisfaction. We aimed to compare patient-reported to provider-estimated overall satisfaction and hypothesized that providers could accurately estimate overall patient satisfaction.

METHODS

Study Design

This study was a cross-sectional survey of patient satisfaction with ED care. We recruited a convenience sample of patients during their visit to our ED and also their healthcare providers, both resident and supervising staff physician, to complete a paper-based, self-administered survey. Participation in this survey was voluntary. This study received approval from our institutional review board.

Study Setting and Population

Our ED is a level 1 trauma center with a volume of approximately 42,000 ED visits per year. The study population consisted of consecutive patients who were evaluated in the ED during the hours that trained research assistants were available (7 am to 11 pm, 7 days per week) from October 10, 2005, to December 10, 2005. During the study period, ED providers were not regularly receiving feedback regarding patient satisfaction. Eligible subjects included patients aged 18 years and older, those between the ages of 16 and 18 with parental consent, and the accompanying parent or guardian of patients who were younger than 16 years of age. Patients were excluded if they were medically unstable, a victim of a sexual assault, unable to comprehend or complete the survey instrument due to the presenting illness acuity, had difficulty understanding English, had impaired mental status (including drug or alcohol intoxication), or at the attending physician’s discretion. The exclusion of non-English-speaking subjects was based on limited financial resources to create translated versions of the survey and to have interpreter time available to conduct the survey and the concern that translated versions may not be comparable with the English version.

Patients who met inclusion criteria were selected for enrollment using a coin flip. Due to the number of other studies being conducted in the ED during this time period, it was not possible to enroll all eligible subjects, and this method of randomization was chosen to minimize selection bias by the research volunteers. Patients who agreed to participate were approached just prior to their discharge or admission for enrollment and verbal consent. Patient satisfaction surveys were self-administered and collected by research volunteers. Patients were informed that survey data was confidential, that it would be used for research purposes only and would not be reviewed by their providers, and they were instructed to place the surveys in a sealed envelope prior to collection by the research volunteers.

For each completed patient survey, the healthcare providers for that patient, both resident and supervising staff physician or nurse practitioner, were approached by the research volunteer at the time of discharge and asked to participate in the provider portion of the survey. There was an option on the survey for providers to opt out of completing the survey if they felt they did not spend enough time with the patient to answer questions regarding how satisfied they thought the patient was with their care. Demographic data, such as age, gender, ethnicity, and insurance status, were abstracted from the medical record by the research volunteer at the time of enrollment. ED length of stay (LOS), respondent disposition, ED divert status, and ED census were abstracted from the ED tracking system as well at the time the patient was enrolled in the study.

Survey Content and Administration

The patient surveys employed four- and five-point Likert-type scales for responses and were designed to assess overall satisfaction as well as factors that likely influence patient satisfaction. These other factors include perceived waiting times, provider communication, treatment of pain, and perceived competence of healthcare providers. Specific wording, format, and content for items were adapted from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS). The HCAHPS is a standardized survey instrument and data collection methodology for measuring patients’ perspectives on hospital care.16 As there were no validated versions of this survey for the ED setting at this time, we adapted the hospital version of the survey by including only pertinent items and making minor wording edits, as needed, to reflect the change in setting. The healthcare provider surveys were abbreviated but addressed the same topics as the patient survey from the perspective of estimating patient perceptions of satisfaction. The patient survey was piloted on approximately 10 patients and revised as needed. Selected questions from the patient survey were adapted for the provider survey, asking providers to predict how their patients would respond.

Data Analysis

Patient characteristics were summarized using descriptive statistics. Proportions were calculated for each level of all satisfaction responses for both patients and providers. To facilitate the assessment of accuracy of a provider’s estimate of his or her patient’s satisfaction, we dichotomized the satisfaction responses in 2 ways. First, all responses were dichotomized as “satisfied” versus “not satisfied.” For overall satisfaction, the 5-point responses were “poor,” “fair,” “good,” “very good,” and “excellent.” The satisfied subjects were those who reported good, very good, or excellent. The other 2 responses were included in the not satisfied group. For perceived ED LOS, “about right,” “shorter than expected,” and “much shorter than expected” were categorized as satisfied; and “much too long” and “longer than expected” were categorized as not satisfied. For quality of pain control and clarity of provider communication, “usually” and “always” were categorized as satisfied; and “never” and “sometimes” were categorized as not satisfied.

In the second method of categorization, all responses were dichotomized as “very satisfied” versus “not very satisfied.” For overall satisfaction, the excellent group was categorized as very satisfied, and all other responses were included in the not very satisfied group. For perceived ED LOS, shorter than expected and much shorter than expected were categorized as very satisfied; and others were categorized as not very satisfied, partly to ensure adequate sample size in the very satisfied group. For quality of pain control and clarity of provider communication, always responses were categorized as very satisfied; and others were categorized as not very satisfied.

To assess the accuracy of a provider’s estimate of his or her patient’s satisfaction, providers’ perceptions were considered tests, and patients’ satisfaction scores were considered the gold standard. Measures of diagnostic accuracy, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), were calculated as well as the proportion of agreement between provider’s perception and patient’s satisfaction. Difference in diagnostic accuracy between attendings and residents was compared using a 2-sample test for independent proportions or Fisher’s exact test for each measure.

Associations between patient characteristics and agreement in overall satisfaction between provider and patient were estimated using logistic regression. All comparisons were also assessed for the presence of confounding by patient age and sex. All analyses were conducted using Stata/IC 11.1 (StataCorp LP, College Station, Texas).

RESULTS

Of 457 eligible patients, 242 subjects (53%) completed the survey. Demographic information for respondents and ED characteristics at the time of the study are listed in Table 1. Of the 215 eligible subjects who did not participate, reasons for nonparticipation include: declined participation (85 patients); missed, usually due to early discharge (80 patients); initially eligible, but later excluded due to discharge time after 11 pm (40); and too ill to participate at the time the patient was approached (10). Also, 227 providers (94%) completed a corresponding survey; 52% of the provider surveys were completed by faculty; 48% were completed by residents. Patients reported an overall satisfaction of 96.6%, as compared with a provider-estimated 94.2%. In contrast, about 49.6% of patients reported being very satisfied, while only 13.5% of providers estimated so.

Proportions for each category of all satisfaction responses are reported in Table 2 for both patients and providers. In general, patients tend to be more likely to report being very satisfied than providers predict. For example, a substantially higher proportion of patients than providers responded that they were very satisfied overall and always satisfied with pain control and clarity of communication.

Results for accuracy of provider-estimated satisfaction are reported in Table 3. For each item of satisfaction, only observations that included both patient and provider data were able to be included in the analysis. For overall satisfaction, there was a high agreement of 93.1% between providers and patients, indicating that the providers correctly estimated patients’ satisfaction 93.1% of the time. The provider’s perception also had high sensitivity (95.2; 95% confidence interval [CI] 91.4, 97.7) and PPV (97.5; 95% CI 94.4, 99.2) to estimate patients’ overall satisfaction. Only 7 patients were categorized as not satisfied overall, and providers correctly identified 2 of them (specificity = 28.6; 95% CI 3.67, 71.0). For clarity of communication, the PPV of provider’s perception was similar to that for overall satisfaction. For perceived ED LOS, the sensitivity was 68.2 (95% CI 60.1, 75.6), and PPV was 78.9 (95% CI 70.8, 85.6), and results were similar for quality of pain control, which was needed for a fraction of patients. In addition, estimates of specificity and NPV were relatively low for the satisfaction responses, which may be partly due to the small sample size available to estimate these 2 measures.

Table 4 demonstrates the accuracy of providers’ estimations of the most satisfied patients, and the results were different from those for satisfaction. Compared to results for satisfaction, the agreement between providers and patients for the very satisfied category was generally lower. In all categories, providers’ perception underestimated the satisfaction as reported by patients. The provider’s perception also had low sensitivity. For example, the sensitivity was only 19.1% (95% CI 12.3, 27.7) and 19.1% (95% CI 9.2, 33.3) for patients’ overall satisfaction and perceived ED LOS, respectively. In contrast, specificity was high, eg, 91.5% (95% CI 84.5, 96.0) for overall satisfaction and 93.5% (95% CI 88.7, 96.7). When accuracy of provider estimated satisfaction was compared between attendings and residents, results were similar, and no significant difference was found between training levels. Based on results from logistic regression, the agreement between patient-reported and provider-estimated patient overall satisfaction was not associated with age, gender, time-to-provider, LOS, disposition of the patient, or ED divert status.

 

DISCUSSION

We began this study with the following question: How accurately can ED providers estimate patient satisfaction? We found that a large percentage of patients reported overall satisfaction with their care and that their providers accurately estimated that they would be satisfied. In our review of the literature, we found only 1 other study that directly evaluated ED personnel accuracy in estimating patient satisfaction.¹⁵ Boudreaux et al found that ED personnel estimated significantly lower satisfaction scores than patients reported and overestimated the average patient LOS. There are important differences in methodology between our study and theirs. The Boudreaux study used telephone surveys administered 7 to 10 days after the ED visit to query patients about their level of satisfaction. Our study surveyed patients immediately after patient care before being discharged from the ED. For the Boudreaux study, the ED personnel were given a blank copy of the patient satisfaction survey 2 to 3 weeks after all patient surveys were completed and were asked to estimate the average score of the group of patients for each question. In contrast, our study assessed providers’ estimates of the satisfaction scores from the specific patients for whom they provided ED care at the time of discharge from the ED.

Although providers’ ability to gauge overall satisfaction is important, our results suggest that providers may be less accurate at estimating satisfaction with regard to several factors that have been previously shown to be associated with overall patient satisfaction. We found lower sensitivity PPVs from providers’ estimations of patients’ perceived LOS and satisfaction with quality of pain control. This trend has important implications for those involved in provider education and quality improvement efforts and warrants further investigation since providers’ ability to assess how well they are providing these important aspects of quality care may determine the success of interventions to improve overall patient satisfaction. However, we found moderately high sensitivity and high PPVs from providers’ estimations of the clarity of communication. Since clarity of communication has been demonstrated to be an important determinant of patient satisfaction, this finding is encouraging, as it suggests that providers may have some insight into how effectively they communicate. Still, further studies to address how accurately providers can estimate the quality of their communication in samples with a larger variation of communication ratings would be helpful to determine if these findings, with a relatively satisfied patient population, can be generalized to other settings.

In considering the categorization of satisfied versus not satisfied patients, we report a NPV of 16.7. However, this value is based on a small sample size of dissatisfied patients, and further investigation with a population with a greater variation of satisfaction scores is necessary to draw conclusions from this finding. On the other hand, when considering the very satisfied versus not very satisfied categories, we found that in each item providers underestimated patient satisfaction. It appears that providers are able to accurately determine the patients who are satisfied overall, but are reluctant to respond that patients are very satisfied. That providers’ accuracy drops when estimating the perceptions of the most satisfied group of patients is an interesting finding. It is not clear whether this is a reflection of physicians’ hesitation to give their care the highest grade or that they really do not recognize when they have provided the highest quality of care. As providing this level of care is ultimately the goal of EDs, this finding may warrant further investigation as well.

Our study also considered whether experience level affected provider ability to estimate overall satisfaction and again found similar results between providers and residents. This suggests that experience alone does not improve the ability to assess patient satisfaction but does not exclude that experience may play a role that our study was not able to detect.

LIMITATIONS

The main limitation of this study is the homogeneity of responses. Since the prevalence of being dissatisfied was low, specificity and NPV could not be reliably estimated, and the results of this study may not be generalizable to populations with a different distribution of patient satisfaction scores. In addition, this study was conducted in a single ED and may not be applicable to other populations and settings. Although our research assistants attempted to enroll all selected eligible patients, they were unable to approach some patients, either because of their acute medical condition or because they were unwilling to participate. Also, because our protocol specified that surveys should be provided after patient care was completed but before discharge, some patients were missed due to expeditious discharge immediately after ED care was complete. We cannot determine if the responses of nonparticipants would have differed from those who completed surveys. In addition, this study measured patient satisfaction at the time of the ED visit. To our knowledge, there have been no published studies assessing this method of patient satisfaction assessment compared to mailed surveys, which are more commonly performed.

CONCLUSION

Providers in this study were able to estimate overall patient satisfaction and clarity of communication with a high degree of accuracy as compared with patient reports. PPV was lower for perceived ED LOS and quality of pain control. Future studies should address populations with a greater variation of patient satisfaction scores and seek to further elucidate interventions that providers can successfully implement to improve their patients’ experiences when they estimate dissatisfaction with ED care.

ACKNOWLEDGMENTS

The authors would like to acknowledge the Oregon Health & Science University (OHSU) Clinical Research Investigative Studies Program volunteers for their assistance in conducting this study. We would also like to thank the OHSU Department of Emergency Medicine for supporting this study with a Bridge grant.

This publication was made possible with support from the Oregon Clinical and Translational Research Institute, Grant Number UL1 RR024140 from the National Center for Research Resources, a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research.

Footnotes

Supervising Section Editor: Jeremy J. Hess, MD

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding, sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

Reprints available through open access at http://escholarship.org/uc/uciem_westjem

Address for correspondence

Lalena M. Yarris, MD, MCR

Oregon Health & Science University, 3181 SW Sam Jackson Park Rd, Mail code CDW-EM, Portland, OR, 97202

E-mail: yarrisl@ohsu.edu

REFERENCES

1. Yarnold PR, Michelson EA, Thompson DA, et al. Predicting patient satisfaction: a study of two emergency departments. J Behav Med. 1998;21:545–563. [PubMed]

2. Hansagi H, Carlsson B, Brismar B. The urgency of care need and patient satisfaction at a hospital emergency department. Health Care Manage Rev. 1992;17:71–75. [PubMed]

3. Hostutler JJ, Taft SH, Snyder C. Patient needs in the emergency department. Nurses’ and patients’ perceptions. J Nurs Adm. 1999;29:43–50. [PubMed]

4. Rydman RJ, Roberts RR, Albrecht GL, et al. Patient satisfaction with an emergency department asthma observation unit. Acad Emerg Med. 1999;6:178–183. [PubMed]

5. Björvell H, Stieg J. Patients’ perceptions of the health care received in an emergency department. Ann Emerg Med. 1991;20:734–738. [PubMed]

6. Hall MF. Keys to patient satisfaction in the emergency department: Results of a multiple facility study. Hosp Health Serv Admin. 1996;41:515–532.

7. Krishel S, Baraff LJ. Effect of emergency department information on patient satisfaction. Ann Emerg Med. 1993;22:568–572. [PubMed]

8. Thompson DA, Yarnold PR, Williams DR, et al. Effects of actual waiting time, perceived waiting time, information delivery, and expressive quality on patient satisfaction in the emergency department. Ann Emerg Med. 1996;28:657–665. [PubMed]

9. Rhee K, Perceptions Bird J. and satisfaction with emergency department care. J Emerg Med.1996;14:679–683. [PubMed]

10. Thomson DA, Yarnold PR. Relating patient satisfaction to waiting time perceptions and expectations: the disconfirmation paradigm. Acad Emerg Med. 1995;2:1057–1062. [PubMed]

11. Hedges JR, Trout A, Magnusson AR. Satisfied Patients Exiting the Emergency Department (SPEED) study. Acad Emerg Med. 2002;9:15–21. [PubMed]

12. McMillan JR, Younger MS, DeWine LC. Satisfaction with hospital emergency department as a function of patient triage. Health Care Manage Rev. 1986;11:21–27. [PubMed]

13. Waggoner DM, Jackson EB, Kern DE. Physician influence on patient compliance: a clinical trial.Ann Emerg Med. 1981;10:348–352. [PubMed]

14. Taylor C, Benger JR. Patient satisfaction in emergency medicine. Emerg Med J. 2004;21:528–532.[PMC free article] [PubMed]

15. Boudreaux ED, Ary R, Mandry C. Emergency department personnel accuracy at estimating patient satisfaction. J Emerg Med. 2000;19:107–112. [PubMed]

16. Centers for Medicare & Medicaid Services, Baltimore, MD. Hospital care quality information from the consumer perspective. HCAHPS Web site. Available at: http://www.hcahpsonline.org. Accessed September 13, 2010.