Author	Affiliation
Moon O. Lee, MD, MPH	Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
Jonathan Altamirano, MS	Stanford University School of Medicine, Office of Faculty Development and Diversity, Stanford, California
Luis C. Garcia, MS	Stanford University School of Medicine, Office of Faculty Development and Diversity, Stanford, California
Michael A. Gisondi, MD	Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
N. Ewen Wang, MD	Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
Suzanne Lippert, MD, MS	Kaiser-Permanente East Bay, Department of Emergency Medicine, Oakland, California
Yvonne Maldonado, MD	Stanford University School of Medicine, Office of Faculty Development and Diversity, Stanford, California
Laleh Gharahbaghian, MD	Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
Ryan Ribeira, MD, MPH	Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
Magali Fassiotto, PhD	Stanford University School of Medicine, Office of Faculty Development and Diversity, Stanford, California

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

In 2008, the Institute for Healthcare Improvement developed the Triple Aim framework to optimize health system performance by focusing on the following: improving the patient experience of care; improving the health of populations; and reducing the cost of healthcare.^1,2 Patient experience is often measured by patient satisfaction. Patient satisfaction is positively associated with improved physician-patient communication, medication compliance, provider job satisfaction, reductions in malpractice claims, and hospital profitability.^3-8 Hospitals have used financial incentives to link physicians’ professional and financial success to their patient satisfaction scores. Some surveys demonstrated that up to 43% of physicians have some portion of their financial compensation linked to patient satisfaction measures.⁹

Press Ganey Associates Inc. (South Bend, IN) first developed patient satisfaction surveys in 1985, and have become the industry standard for measuring patient experience in the outpatient setting.^10-15 Hospitals typically distribute Press Ganey (PG) standardized surveys to a random sample of patients to solicit feedback regarding providers, staff, and clinical environments. PG uses a five-point Likert scale for patient responses. A score of 5, the most favorable, is known as the “topbox” score.^13,16 Topbox scoring is the standard for customer satisfaction and consumer research.¹⁷

Despite widespread adoption of patient satisfaction measurement systems and associated incentives, concern was raised about the validity of these tools since current literature does not consistently demonstrate key predictors of higher or lower scores.¹⁸ Only a few studies have examined PG surveys specific to the emergency department (ED); some studies have found that ED PG scores are positively associated with employee satisfaction and retention, and negatively associated with ED crowding and wait times.^6,19,20 There is evidence that acuity of a patient’s illness and the patient care setting affect PG scores. Critical, emergent patients were more likely to give higher scores than non-urgent patients.²¹ Bendensky et al showed the same physicians had higher “courtesy of the doctor” scores from the urgent care setting than in the ED.¹¹

Gender also influences the perceptions, behavior, and communication of patients and their providers.^22,23 Patients have different expectations from male and female physicians.²⁴ The ED setting is unique in that patients have unscheduled visits and cannot choose their healthcare provider in the ED. The influence of patient or physician factors specific to ED PG scores has been limited to a few studies.^21-23,25 We hypothesized that patient factors (age, gender, race, and/or ED area where patient was seen) and physician factors (age, gender, race, years at institution) influence topbox scores for two ED PG survey questions: “Likelihood to recommend ED,” and “Courtesy of the doctor.”

 

Population Health Research Capsule

What do we already know about this issue?

Press Ganey scores are often used to benchmark physicians. The relationship between patient and physician factors with the highest (topbox) score is unclear.

What was the research question?

Are patient and physician factors associated with topbox scores on Press Ganey surveys?

What was the major finding of the study?

Patient factors were associated with topbox scores, but physician factors were not associated with topbox scores.

How does this improve population health?

Physicians and administrators will be informed about the contribution of non-service factors associated with Press Ganey topbox scores.

 

METHODS

Study Design and Setting

This was an observational, population-based study at an urban, academic, tertiary care hospital. The hospital is a designated Level I adult and Level I pediatric trauma center and a comprehensive stroke center. The annual ED volume is approximately 78,000 visits a year. The ED has a separate pediatric ED and adult ED. The adult ED is divided into different care areas based on age and patient acuity: emergent; urgent; vertical (urgent but able to sit in a recliner rather than a gurney); and fast track (non-urgent). The emergent area is for adult patients 18 years and older who require acute resuscitations, require trauma assessments, or are otherwise clinically high-risk patients. The urgent and vertical areas were designed for patients who do not require emergent intervention or assessment. The fast-track area was designed for patients over six months old who are triaged as non-urgent with an estimated discharge within 90 minutes. Approximately, eight ED attendings worked only during the overnight shift. The overnight physicians only worked in the emergent and urgent areas since these two areas were the only open areas on the adult overnight shifts. All other general emergency physicians worked in the different areas of the adult ED.

Study Population

We collected PG survey data from January 2015–December 2017 for adult patients (age ≥18 years) who were evaluated, treated, and discharged from the ED. All patients enrolled in the online patient portal received a PG survey after their ED visits. For patients without the online portal access, five unique patients per physician per month were randomly selected to receive a paper survey. If patients had multiple visits with several physicians within 21 days, only one visit was randomly chosen for evaluation. Patients did not receive a PG survey from the ED if they had received a PG survey from the hospital within one week of the ED visit. This study was approved by the Stanford University School of Medicine Institutional Review Board.

Measurements

Patient Factors

Self-reported patient demographic information obtained from PG surveys included age, gender, race, and ethnicity. Patients age 18–29 were grouped into age less than 30 years due to the small sample size. Patient age greater than 30 years was divided into 10-year intervals. Race and ethnicity were categorized as White, Asian, Black/African-American, Hispanic, Native American or Alaskan Native, and Native Hawaiian or other Pacific Islander. Surveys that reported race as “other” or “more than two racial backgrounds” were excluded from data analysis given low sample size in each category. We also excluded from the analysis surveys that reported race as “unknown”. The ED area where patients were seen and treated was provided for each PG survey.

Physician Factors

Physician demographic data included age, gender, race, ethnicity, and years at the current institution. Age, race, and ethnicity data were categorized into the same groups as the patients.

Outcome

Two PG questions that are often used to inform hospital-related incentives for physicians were chosen for the outcomes. The two primary outcomes were topbox scores for “Likelihood of your recommending our emergency department to others,” and “Courtesy of the doctor.”

Statistical Analysis

We used chi-squared tests of independence (χ² tests) to assess the associations between patient and physician factors and impact of ED area on “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” PG scores. Two generalized estimating equation (GEE) models were performed, one using topbox “Likelihood of your recommending our ED to others” as the outcome variable and the other using “Courtesy of the doctor.” Models controlled for patient and physician factors, and the ED area where the patient was seen. We used GEE models to cluster surveys by physician, using an exchangeable correlation structure to account for possible correlations within survey responses for the same physician. Models were performed using surveys with complete patient and physician demographic information. A P value ≤0.05 was considered statistically significant for all tests, and 95% confidence intervals were reported. We performed analysis in SAS 9.4 (SAS Institute, Cary, NC).

RESULTS

The response rate for ED PG surveys was 10%. Of the returned 5,325 surveys, 3,524 surveys answered both “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” questions. From the 3,524 surveys with both outcomes questions answered, 3,038 surveys had complete patient demographic information including age, gender, race, and ethnicity. See Figure 1 for study design. Out of the 3,038 surveys, 2,400 were paper surveys, and 638 were online surveys. Most of the online responses 389 (61%) were in 2017. For each year of the study (2015–2017) the mean topbox scores “Likelihood of your recommending our ED to others” were 69%, 70%, and 66%. For each year, the mean topbox scores for “Courtesy of the doctor” were similar: 73%, 74%, and 72%.

Patient Characteristics

Patients who responded to the PG survey did not mirror the demographics of the patients discharged from the ED. Women patients were 58% of the PG study population but only 53% of the ED discharge population. Patients over the age of 60 were 51% of PG study population, while patients over 60 years made up only 25% of the ED discharge population. White patients were 63% of the PG study population, but only 32% of the ED discharge population. Asian patients were 16% of the PG population and 14% in the ED discharge population. Hispanic patients were only 15% of the PG population, in contrast to 36% in the ED discharge population. Most patients were assigned to the urgent area (43%), and the next largest group was assigned to the vertical area (23%). Patient demographics are shown in Table 1.

Physician Characteristics

Most of the PG surveys were completed for male physicians (64%). Physicians were younger than patients, with 76% of ED visits with physicians younger than 50 years old. Physician race was similar to that of the patient population, and most visits were with White physicians (75%). The mean number of years that a physician worked in the Stanford ED was eight years, standard deviation 9.1. Physician demographics are shown in Table 1.

Chi-squared Tests Results

The proportion of topbox scores for “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” by patient and physician gender, race, and ED area are summarized in Table 2. Female patients gave significantly fewer topbox scores than male patients for “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” (P = 0.0023 and P = 0.027, respectively). Asian patients gave significantly fewer topbox scores than White patients for “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” (P = 0.0018 and P <0.0001, respectively). Patients seen in urgent and vertical areas gave significantly lower topbox scores for “Likelihood of your recommending our ED to others” (P <0.0001) and “Courtesy of the doctor” (P =0.0008) than compared to fast track. Physician gender and physician race were not significantly associated with topbox scores for either question.

Chi-squared tests showed that gender concordance may influence “Likelihood of your recommending our ED to others” and “Courtesy of the doctor” (Table 3). After stratifying data by physician gender, female patients were shown to give significantly fewer topbox scores for “Likelihood of your recommending our ED to others” if the physician was also female (P = 0.01). Male patients did not show significant difference for topbox scores with physician gender.

Generalized Estimating Equation Modeling Results for “Likelihood of Your Recommending Our ED to Others”

After controlling for patient and physician factors, we observed that patient age, patient gender and race, and ED area where they were seen were significantly associated with odds of a topbox score for “Likelihood of your recommending our ED to others” (Table 4). Each 10-year increase in patient age was associated with an increase in the odds of a topbox score (odds ratio [OR] = 1.07; 95% CI, 1.03 – 1.12, P = 0.001). Female patients had decreased odds of giving a topbox score when compared to male patients (OR = 0.81; 95% CI, 0.7 – 0.93, P = 0.003). Asian patients had lower odds of giving a topbox score when compared to White patients (OR = 0.71; 95% CI, 0.6 – 0.83, P <0.0001). Patients seen in the urgent area had lower odds of giving a topbox score when compared to patients seen in fast track (OR = 0.71; 95% CI, 0.54 – 0.93, P = 0.01), as did patients seen in the vertical area (OR = 0.71; 95% CI, 0.53 – 0.95, P = 0.02).

Generalized Estimating Equation Modeling Results for “Courtesy of the Doctor “

After controlling for patient and physician factors, we observed that patient age, patient race, and ED zone were significantly associated with odds of receiving a topbox score (Table 4). Each 10-year increase in patient age was associated with increased odds of a topbox score (OR = 1.1; 95% CI, 1.06 – 1.14, P <0.0001). Asian (OR = 0.70; 95% CI, 0.58 – 0.84, P = 0.0001), Black (OR = 0.66; 95% CI, 0.45 – 0.96, P = 0.03), and Hispanic (OR = 0.68; 95% CI, 0.55 – 0.83, P = 0.0001) patients all had lower odds of giving a topbox score when compared to White patients. Patients seen in the urgent area had a significantly lower odds of giving a topbox score when compared to patients seen in fast track, (OR = 0.69; 95% CI, 0.50 – 0.95, P = 0.02).

DISCUSSION

Our study found that patient factors were associated with topbox scores for PG questions while physician factors did not influence topbox scoring. As patients’ ages increased by decade, they were more likely to give topbox scores for “Likelihood of your recommending our ED to others” and “Courtesy of the doctor.” Asian patients and patients seen in the urgent and vertical zones of the ED were less likely to give topbox scores for “Likelihood to recommend emergency room” and “Courtesy of the doctor.”

Our study has multiple strengths that led to new results, which have not been previously published on PG surveys in the ED. First, our study detected a difference in race and topbox scores due to a diverse patient population. Boudreaux et al shows that ED patient demographics (age, gender, race) were unrelated to patient satisfaction scores but categorized patient race/ethnicity as “Black” or “other.”²⁵ Due to our distinct patient population, we were able to demonstrate for the first time that Asian patients in the ED are less likely to give a topbox score compared to White patients. Second, our large study adds new information about patient satisfaction in the ED using topbox scoring. Topbox scoring is a more accurate measure for customer satisfaction in consumer research and is associated with predicting growth.^17,26

A meta-analysis examined multiple PG studies in all specialties and found female physicians were slightly favored when the physician had less experience, when it was the first visit, and the survey was administered right after a visit.²⁷ A subsequent study in 2017 by Chen et al found physician gender, ethnicity, and race were not associated with topbox scores, but the scores were associated with specialty; obstetrics and surgery had higher scores compared to medicine, but they did not examine emergency physicians.²⁸ Milano et al examined PG surveys in the ED and in a small study of 398 surveys showed that the median score for “Courtesy of the doctor” of male emergency physicians and female emergency physicians did not significantly differ.²³ Our study examined PG surveys over a three-year period with a large number of completed surveys, (n = 3,038) with topbox scores as our outcome.

A third strength of our study is that it is one of the few studies to have demonstrated significant association between the area of the ED where patients are seen and PG topbox scores. A prior study by Bendensky et al demonstrated that the mean score for “Courtesy of the doctor” was higher in the urgent care setting compared to the ED setting with the same physicians working in both locations.¹¹ Boudreaux et al found “emergent” patients were more satisfied than “urgent” and “routine” patients with the ED visits. This study was based on the initial ED Emergency Services Index, which was determined at triage, and “routine” patients were seen in a rapid care area with a mean ED length of stay of 136 minutes.²¹ In contrast, our study demonstrated that patients seen in fast track were more likely to give topbox scores. In our fast track, patients were typically seen and discharged within 90 minutes of arrival to the ED. The second area of the ED associated with topbox scores was the “emergent” zone in which the most critical patients are seen, which is consistent with prior studies. Patients were least likely to give topbox scoring in the “vertical” zone where patients are classified as urgent, but able to sit in a recliner rather than a gurney.

Fourth, our study is the first to examine PG topbox scores in the ED and consider patient factors, physician factors, and the ED area where patients are seen. Prior studies of PG surveys in the ED focused only on physician gender and did not take into account patient gender or the area in the ED where the patient was seen.²³ By accounting for all of these factors, we found that age of the patient, Asian patient race, and ED area were associated with topbox scores.

LIMITATIONS

Our study has several limitations to consider. Our study is limited by the use of self-reported survey data that we cannot link with patient outcomes. Our response rate was 10%, which may have led to sampling bias. Patients who returned the survey may be different than those who did not respond. We did not have the response rate for each area of the ED, which may have led to sampling bias. Additionally, our study was conducted at one academic institution with a diverse patient population and may not be generalizable to other geographic areas of the country.

CONCLUSION

Many hospitals use Press Ganey surveys as a measure of quality of care and provide financial incentives to physicians based on their scores. Our study demonstrates that patient race, patient age, and location where patients are seen in the ED are associated with PG topbox scores. We encourage hospitals that use PG topbox scores as financial incentives to understand the contribution of non-service factors to these scores.

Footnotes

Section Editor: David Lee, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Moon Lee, MD, MPH, Stanford University School of Medicine, Department of Emergency Medicine, 900 Welch Road, MC 5119, Suite 350, Palo Alto, CA 94304. Email: moonlee@stanford.edu. 11 / 2020; 21:117 – 124

Submission history: Revision received March 10, 2020; Submitted August 4, 2020; Accepted August 6, 2020

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood). 2008;27(3):759-69.

2. Whittington JW, Nolan K, Lewis N, et al. Pursuing the triple aim: the first 7 years. Milbank Q. 2015;93(2):263-300.

3. Ha JF, Longnecker N. Doctor-patient communication: a review. Ochsner J. 2010;10(1):38-43.

4. Miaoulis G, Gutman J, Snow MM. Closing the gap: the patient-physician disconnect. Health Mark Q. 2009;26(1):56-68.

5. Hickson GB, Clayton EW, Entman SS, et al. Obstetricians’ prior malpractice experience and patients’ satisfaction with care. JAMA. 1994;272(20):1583-1587.

6. Press I. Patient satisfaction: defining, measuring, and improving the experience of care. J Healthc Qual. 2003;25(3):47.

7. Welch SJ. Twenty years of patient satisfaction research applied to the emergency department: a qualitative review. Am J Med Qual. 2010;25(1):64-72.

8. Kirsner RS, amd Federman DG. Patient satisfaction: quality of care from the patients’ perspective. Arch Dermatol. 1997;133(11):1427-31.

9. Hawkins M. Review of physician and advanced practitioner recruiting incentives. 2018. Available at: https://www.merritthawkins.com/uploadedFiles/Merritt_Hawkins_2018_incentive_review.pdf. Accessed March 28 2019.

10. Siegrist RB. Patient satisfaction: history, myths, and misperceptions. AMA J Ethics. 2013;15(11):982-7.

11. Bendesky BS, Hunter K, Kirchhoff MA, et al. Same physician, different location, different patient satisfaction scores. Ann Emerg Med. 2016;68(5):531-5.

12. Bernard AW, Martin DR, Moseley MG, et al. The impact of medical student participation in emergency medicine patient care on departmental Press Ganey scores. West J Emerg Med. 2015;16(6):830.

13. Sayah A, Lai-Becker M, Kingsley-Rocker L, et al. Emergency department expansion versus patient flow improvement: impact on patient experience of care. J Emerg Med. 2016;50(2):339-48.

14. Shuaib W, Hilmi J, Caballero J, et al. Impact of a scribe program on patient throughput, physician productivity, and patient satisfaction in a community-based emergency department. Health Informatics J. 2019;25(1):216-24.

15. Vorhies JS, Weaver MJ, Bishop JA. Admission through the emergency department is an independent risk factor for lower satisfaction with physician performance among orthopaedic surgery patients: a multicenter Study. J Am Acad Orthop Surg. 2016;24(10):735-42.

16. Messner ER. Quality of care and patient satisfaction: the improvement efforts of one emergency department. Adv Emerg Nurs J. 2005;27(2):132-41.

17. Lang SC, Weygandt PL, Darling T, et al. Measuring the correlation between emergency medicine resident and attending physician patient satisfaction scores using Press Ganey. AEM Educ Train. 2017;1(3):179-84.

18. Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care. 1999;11(4):319-28.

19. Kaldenberg DO, Regrut BA. Do satisfied patients depend on satisfied employees? Or, do satisfied employees depend on satisfied patients?. QRC Advis. 1999;15(7):9.

20. Pines JM, Iyer S, Disbot M, et al. The effect of emergency department crowding on patient satisfaction for admitted patients. Acad Emerg Med. 2008;15(9):825-31.

21. Boudreaux ED, Friedman J, Chansky ME, et al. Emergency department patient satisfaction: examining the role of acuity. Acad Emerg Med. 2004;11(2):162-8.

22. Derose KP, Hays RD, McCaffrey DF, et al. Does physician gender affect satisfaction of men and women visiting the emergency department?. J Gen Intern Med. 2001;16(4):218-26.

23. Milano A, Dalawari P, McGregor AJ, et al. Emergency department evaluation of patient satisfaction. Does physician gender impact Press Ganey scores? A multicenter study. Am J Emerg Med. 2018;36(9):1708-9.

24. Linzer M, Harwood E. Gendered expectations: do they contribute to high burnout among female physicians?. J Gen Intern Med. 2018;33(6):963-5.

25. Boudreaux ED, Ary RD, Mandry CV, et al. Determinants of patient satisfaction in a large, municipal ED: the role of demographic variables, visit characteristics, and patient perceptions. Am J Emerg Med. 2000;18(4):394-400.

26. Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81(12):46-55.

27. Hall JA, Blanch-Hartigan D, Roter DL. Patients’ satisfaction with male versus female physicians: a meta-analysis. Med Care. 2011:611-7.

28. Chen JG, Zou B, Shuster J. Relationship between patient satisfaction and physician characteristics. J Patient Exp. 2017;4(4):177-84.