Author	Affiliation
Jessica R. Schumacher, PhD	University of Florida, Department of Emergency Medicine, Gainesville, Florida
Barbara J. Lutz, PhD, RN	University of North Carolina-Wilmington, College of Health and Human Services, School of Nursing, Wilmington, North Carolina
Allyson G. Hall, PhD	University of Alabama at Birmingham, Department of Health Services Administration, Birmingham, Alabama
Jesse M. Pines, MD, MBA, MSCE	The George Washington University School of Medicine, Department of Emergency Medicine and Health Policy & Management, Washington, DC
Andrea L. Jones, PhD, MSW	University of North Carolina-Wilmington, College of Health and Human Services, School of Social Work, Wilmington, North Carolina
Phyllis Hendry, MD	University of Florida, College of Medicine, Department of Emergency Medicine, Jacksonville, Florida
Colleen Kalynych, MSH, EdD	University of Florida, College of Medicine, Department of Emergency Medicine, Jacksonville, Florida
Donna L. Carden, MD	University of Florida, Department of Emergency Medicine, Gainesville, Florida

Introduction
Methods
Results
Discussion
Limitations
Conclusion
Supplementary information

INTRODUCTION

Patient engagement is central to many health policy initiatives. ^1-5 Engaged patients make more informed healthcare decisions, avert health crises and incur lower healthcare costs.^6,7 Interventions to increase patient engagement increase the use of preventive care, reduce hospital-based care and improve outcomes.^6,8-12 The Patient Activation Measure SF® (PAM) is a way to quantify patient engagement, which is defined as patients’ knowledge, skills and confidence in managing their health and healthcare and the interventions that promote healthy behaviors. ¹³ Coaching interventions increase PAM scores, reduce hospital use, and improve medication and chronic disease self-management but have not been tested in the ED.^8,9,12

Although ED use is increasing in older adults, those with limited health literacy represent a particularly high-risk group who are often under-engaged in managing their health and frequently turn to the ED for care. ^14-18 Strategies aimed at engaging these patients at the critical ED juncture may help them stay engaged, better manage their health and avert future health crises. We tested the impact of a coaching intervention on patient engagement and follow-up doctor visits in chronically ill, older ED patients with limited health literacy. Because efforts to help patients manage their health are more effective if they align with patients’ perspectives,^8,19 we also explored reasons for ED use in this high-risk population.

METHODS

Study Design

We conducted a randomized controlled trial (RCT) comparing an ED-to-home intervention (“intervention”) to usual post-ED care (“usual care”) on patient engagement and follow-up doctor visits and in-depth interviews exploring patients’ healthcare-seeking decisions. The study was conducted from July 2013 to August 2014.

Study Setting

The intervention was tested in two communities. Site 1 ED (90,000 visits/year) is a tertiary referral center serving a community of 250,000 and a White (62%) and African-American (28%) population with various payers (40% public, 36% private). Site 2 ED (89,000 visits/year) is a tertiary referral center serving a metropolitan area of one million and African-American (59%), White (33%), publicly insured (44%) and uninsured (24%) patients.

Study Population

Older, chronically ill patients with limited health literacy insured by Medicare scheduled for ED discharge were eligible for study inclusion (Figure).

Study Protocol

Recruitment

The university institutional review board approved the study at both sites. Study procedures are outlined (Figure). Random assignment using a random number generator was provided to research associates (RAs) who determined patient eligibility by screening the ED electronic health record (EHR). RAs were blinded to assignment until baseline survey completion.

 

Population Health Research Capsule

What do we already know about this issue?

Coaching interventions increase patient engagement, improve medication and disease self-management and reduce hospital use but have not been tested in the ED.

What was the research question?

Can an ED-initiated coaching intervention increase patient engagement in older ED patients with limited health literacy?

What was the major finding of the study?

The ED-initiated coaching intervention significantly reduced declines in patient engagement observed after usual post-ED care.

How does this improve population health?

ED-initiated coaching interventions hold promise for helping high-risk and hard-to reach patients better manage their health and healthcare.

 

Health Literacy Screening

Following screening and informed consent, patients completed the Rapid Estimate of Adult Literacy in Medicine (REALM).^20,21 REALM is valid in diverse racial/ethnic groups ²¹ and older adults. ²² Categories include adequate (≥61 words correct; grade level ≥ 9) and limited health literacy (<61 words correct; grade level 0–8).

Intervention

The ED-to-home intervention was modeled on the Care Transitions Intervention^SM (CTI), an evidence-based program to increase patient engagement and reduce 30-day readmissions and healthcare costs in hospitalized patients. ²³ Trained coaches from community area agencies on aging administered the intervention. Coaches helped patients 1) schedule follow-up doctor visits; 2) recognize disease worsening; 3) reconcile medications; and 4) communicate with providers.^10,23,24 Coaches visited patients’ homes within three days of ED discharge, called three times over the ensuing month, and engaged patients by helping them set achievable goals.

Usual Care

Usual post-ED care included written and verbal discharge instructions and advice to follow up with a provider.

In-depth Interviews

Based on site, assignment and date of ED visit, a purposive sample (n=11; 6 “intervention,” 5 “usual care”) was invited to interview, and 9 agreed. Questions emphasized reasons for ED visit and access to post-ED care (Supplementary Appendix S1).

Data Sources

Baseline Survey

Participants completed a baseline ED survey to record PAM score, sociodemographic (age, gender, race), socioeconomic (education, employment, payer status) and health-related factors (self-rated health, number of chronic conditions).

Follow-Up Telephone Survey

Participants were called by the University Survey Center within 31–60 days of the ED visit to determine follow-up PAM score and doctor visits using Medicare Current Beneficiary Access-to-Care Survey items. ²⁵ The survey was administered using best practices (e.g., 10 call attempts, rotating call attempts, refusal conversion). ²⁶

Patient Activation Measure

We used the 13-item PAM^27,28 to assess engagement including patients’ knowledge, skills, and confidence in managing their health and healthcare. Degrees of agreement with statements, such as “When all is said and done, I am the person who is responsible for managing my health condition” and “Taking an active role in my own health care is the most important factor in determining my health and ability to function,” are scored on a 0–100 point scale. The lowest scores suggest a person does not understand their role in healthcare, while the highest levels indicate greater activation and proactive, healthy behaviors. The PAM is previously published and valid in older, chronically ill patients with limited health literacy.^29,30

In-depth Interviews

Interviews (60–90 minutes) were conducted in patients’ homes, audio-recorded and transcribed verbatim.

Outcome Measures

Outcomes included between-group differences in PAM scores and self-reported doctor visits. In-depth interviews identified factors influencing healthcare-seeking decisions.

Power Calculation

We conducted a pre-pilot power analysis based on an increase in primary care visits from 30–80% within 10 days of hospital discharge at one study site using the identical coaching intervention in hospitalized patients. Thirty-five participants in both groups were needed to detect similar differences in post-ED visit follow-up with power (1-β) of 80% and alpha of 0.05.

Quantitative Analysis

We conducted between-group comparisons in sociodemographic, socioeconomic, health status and doctor visits using chi-square and analysis of variance for categorical and continuous measures, respectively. We used an intention-to-treat approach for all analyses.

We assessed between-group differences in PAM scores between the baseline and follow-up time points in two ways. In our primary analysis, we assessed mean PAM score differences over time between the “intervention” and “usual care” groups using unadjusted linear regression. We then assessed between-group PAM score differences over time using a difference-in-differences (DID) approach that accounted for differential between-group loss to follow-up using inverse probability weighting. Inverse probability weighting adjusts for bias due to missing data by giving more weight to patients who resemble those lost to follow-up. ³¹ The DID approach ensures that background trends in outcomes unrelated to the program are not responsible for treatment effects by comparing outcomes in the treatment group to a group experiencing the same background trends but not exposed to the program. To account for the fact that patient measurements within a site were more likely to be similar than measurements between sites, all models were estimated with standard errors clustered by site. We conducted analyses using Stata v.13, with significance at p<0.05.

Qualitative Analysis

Interview transcripts were read independently by three members of the research team including qualitative methods experts and a health service researcher. Using thematic and constant comparative analysis, we coded data using procedures described by Charmaz. ^32-35 Team members wrote memos to document and record study findings and track methodological, theoretical, and substantive decisions made during the analysis to ensure rigor of data analysis and interpretation. We used open coding to identify concepts important to participants, and provisional themes were presented to the entire team for feedback and verification. Codes were reviewed, discussed and arranged into wider thematic structures to make meaning of participant narratives. ³⁶

RESULTS

Of 170 patients consenting to health literacy screening, 71 had limited health literacy and were eligible for inclusion (36 “intervention,” 35 “usual care”). All agreed to participate and 69 patients completed baseline ED surveys. Only baseline PAM scores were significantly different between “intervention” and “usual care” groups, respectively (64.0 ±16.9, 60.1±15.1, p=0.03) (Table 1).

Forty of 65 patients able to respond, completed the follow-up telephone survey (61%, 23 “intervention,” 17 “usual “care” [Table 1; Figure]). Patients with lower baseline PAM scores (odds ratio=0.92, 95% CI=0.86–0.98) were less likely to complete the follow-up telephone survey.

Seventy-three percent of “intervention” patients (76% at Site 1; 70% at Site 2) completed coaching. Median time from ED-to-home visit was 2.5 days (range 1–12 days). The home visit lasted approximately 60 minutes, and each of the coaching phone calls lasted about 15 minutes.

Quantitative Findings

Coaching Impact on Patient Engagement

PAM scores fell in both groups after the ED visit but fell significantly more in “usual care” than “intervention” participants (−4.64 and −2.77, respectively; unadjusted linear regression, β=1.87, p=0.043) (Table 2). This finding remained statistically significant after inverse probability weighting to account for loss to follow-up (DID=1.96, t=23.42, p=0.027).

Follow-Up Doctor Visits

Most patients (61%) did not report a doctor visit within two weeks of the ED visit (Table 1). “Intervention” participants were more likely to report a follow-up within four weeks of ED visit (74% vs. 65%, respectively, p=0.53).

Qualitative Findings

Nine interviews (5 “intervention,” 4 “usual care”) were conducted: 5 from Site 1 (4 female, 3 “intervention”) and 4 from Site 2 (3 female; 2 “intervention”). Participant ages ranged from 62–86 years, and all were African American with more than one chronic condition.

Patient Engagement and Decision to Seek ED Care

When participants decided to visit the ED, they were highly engaged and motivated to address their health concern. Decisions to visit the ED were well-thought out and driven by individual characteristics, including the nature and severity of symptoms, personal advice and prior healthcare system experiences. Representative quotes are described below.

Individual Characteristics

Unremitting pain and history of similar symptoms factored heavily into patients’ decisions to use the ED. Pain and at least one other precipitating factor (e.g., history of similar symptoms, advice of trusted sources, including providers, friends, family) led to uncertainty, fear, and a decision to seek emergency care for all (9/9) participants.

 

[2–4]: “I had a pain at the end of my spine and that hip bone that joined together …I had to go. I can’t stand pain no way. And that’s why I ended up going to emergency.”

[1–2]: “I was in a lot of pain for one thing. And I had, beforehand, had a blood clot. So I didn’t know if another one had come back or not, so I thought maybe I needed to go to the emergency room.”

 

Another participant sought ED care because of history of hyperglycemia.

 

[1–5]: “I can tell when my sugar goes up because I get really dizzy. The emergency room was the best place to go because if I went to a primary doctor, they were going to send me to an emergency room anyway.”

 

Participants often considered the advice of family, friends, or healthcare providers.

 

[2–2]: “I couldn’t straighten up, stand up, or sit down without having cramps. A friend of mine called the ambulance for me, and I was then rushed to the [ED].”

 

Healthcare System Experiences

Decisions to seek ED care were influenced by patients’ prior healthcare system experiences. Perceptions of availability of care in the provider’s office versus ED care were important. For most (6/9), obtaining a same-day appointment or speaking with a provider was not possible even if the patient believed the need was urgent. Patients’ perceptions that the ED provides comprehensive care led (3/9) participants to seek emergency treatment.

 

A participant with chest tightness and blood pressure of 235/96 [1–4]: “I went to [my doctor], but he was filled up. So I called him and asked him could I come back? And he said no. It would probably 4:00 or 5:00 if I got seen then. I said, “Well, I’m going to have to go to the emergency room because I’m sick. I feel bad.”

[2–4] [My] doctor would [not be able to see me at] “that particular time of day,” so [I] “just went on to emergency.”

 

Symptoms when providers’ offices were closed influenced decisions to seek ED care. The following participant developed what she thought were minor symptoms on a weekend and decided to wait for symptoms to subside. Her son convinced her to seek immediate care.

 

[2–1]: “And so I felt like something was sitting on my chest. So I got up and I got ready, and I went on to church. And [the pain] was still there and then I said, “Ah, it’ll go.” So I didn’t go [to the ED]. My son came and he said, ‘Either you let me take you [to the fire station around the corner], or I’ll call 911.’ That I didn’t want, so I said, ‘Well, okay, come and take me around there.’ The [fire fighter] said, ‘Well, we can’t let you go with it…You’re having chest pain.’ I said, ‘Yes, sir, but it feels like gas.’ He said, ‘Well, that’s what heart attacks are like. They are mostly like gas, but then you’re having a heart attack.’ So, I agreed for them to take me to the emergency room because I really wanted to know what it was.”

 

For participants attending large community-based clinics (health department or Veterans Affairs), timely contact with a provider usually was not possible. Appointments were viewed as a strategy to “maintain” health through check-ups and refilling prescriptions rather than addressing a healthcare concern. For these participants, ED care was considered a reasonable option.

 

[2–2]: “If I had called [my primary care clinic] to tell them I had cramps they would have given me an appointment two or three months down the line… As a matter of fact, after I went [to the ED in June] and called for a [follow-up] appointment, the earliest appointment they could give me was in August.”

 

In contrast to perceived lack of availability of timely outpatient care, participants perceived ED care as immediate and comprehensive because of staffing and availability of ancillary and specialty referral services.

 

[1–3]: “In the ER they get right on it…They won’t let me sit back and die. I know they are coming randomly and checking everything and they know what my levels are. I’m not dying because they would be in there.”

[1–4]: “I figured they had more equipment to do testing over [in the ED] than my primary care doctor did. They can do everything at once.”

 

Participants’ relationships with their doctor also heavily influenced healthcare-seeking decisions. If the provider was familiar with them and helped them navigate the system, participants were more likely to contact their provider with health concerns and questions about where to seek care. One participant described a partnership with her provider:

 

[1–1]: “[She] explained things and she insisted that you do things that you know you needed to do.” This patient reported their provider was attentive to the participant’s concerns, and arranged appointments for follow-up tests and regular screenings.

 

Intervention Impact on Patient Engagement

Three of five “intervention” patients indicated the coaching intervention helped them stay involved in their healthcare by increasing their understanding of chronic disease symptoms, appropriate use of medications and follow-up care. They appreciated having someone to call if they had questions. Patients reported receiving help with other needed services.

 

[2–2]: “Knowing you could call someone, and you don’t have to go through these channels with them. It makes you feel comfortable; not as stressful – there is this person that you know you can call and say ‘this is what is going on’…and they could do something to help you.”

 

Coaches also helped participants access other community services including transportation [1–2]:“because transportation is my biggest problems.”

DISCUSSION

Patient engagement has been called the blockbuster drug of the century. When patients are more engaged, they have better health outcomes and lower healthcare costs. ^1-5 Unfortunately, interventions aimed at engaging ED patients to help them better manage their health and healthcare have been largely ignored. This study is novel because it focused on chronically ill, older ED patients with limited health literacy, a high-risk population that is often under-engaged in their health and rely on the ED during a health crisis. ¹⁴ To our knowledge, this is the first study to assess the impact of an ED-initiated coaching intervention on patient engagement in a vulnerable ED population. The study documented three observations regarding patient engagement. First, chronically ill patients with limited health literacy are most engaged at ED presentation as assessed by PAM scores. Second, engagement falls in the weeks after the ED visit in all patients. Third, the ED-to-home “intervention” significantly reduced the post-ED fall in patient engagement relative to “usual care.”

Baseline PAM scores were higher in this study than reported in a nationally representative sample of Medicare beneficiaries (64.6 versus 63.4 previously reported) ²⁹ but fell to a mean of 61.0 in the weeks following the ED visit. Engagement is highest at ED presentation – a finding that is consistent with prior work that demonstrates patients use the ED when they feel their condition is emergent and are too worried about their condition to seek care in other settings.^{16, 37} Our in-depth interviews are consistent with these findings and provide insight into the critical individual and health-system factors that contribute to patients’ decisions to use the ED, which include the onset and severity of symptoms, advice of trusted sources, inability to gain timely access to a provider, and perception that the ED provides immediate, comprehensive care.

We were unable to identify studies where interventions designed to increase patient engagement led to decreased PAM scores. In this study, the observed decline in PAM scores may have occurred because the baseline survey was conducted in the ED. It is likely patients become engaged to take actions about their health when their symptoms suggest something is wrong that requires emergent attention. Although highly engaged in their health during the health crisis, patients’ engagement may decline as the crisis resolves without further health system contact. ³⁸ Importantly, the ED-to-home intervention blunted the decline in post-ED PAM scores by approximately two points. The fact that similar longer-term PAM score changes are associated with improvements in physical activity, medication adherence, self-management knowledge, and functional health suggest the clinical significance of these findings. ^38-40

Coaches helped participants advocate for themselves to schedule timely follow-up doctor visits but did not make appointments for patients. Most patients did not see a provider within two weeks of ED visit, and between-group differences were not observed. It is possible that even if ED patients attempted follow-up, they were unable to obtain an appointment because of barriers previously described ⁴¹ and noted by our participants. Indeed, a recent systematic review noted that interventions designed to improve post-ED follow-up have variable effectiveness depending on the capacity and willingness of the local primary care network to accommodate ED patients. ⁴² It is also possible that although this intervention had a significant impact on patient engagement, it did not motivate individuals to attend follow-up doctor visits if they were not already inclined to do so.

LIMITATIONS

Among this study’s limitations was the small pilot-sample size. Despite the small sample, statistically significant between-group differences in patient engagement were detected and the mixed-methods design allowed us to confirm quantitative findings through in-depth interviews. Second, the pre-pilot increase in follow-up doctor visits observed in hospitalized patients was not detected. Our study may have been underpowered to detect smaller but potentially clinically meaningful increases in post-ED outpatient follow-up visits. In addition, patients may have attempted but were unsuccessful in scheduling post-ED outpatient follow-up visits because of busy clinic schedules. This intervention was conducted in two EDs and the results may not generalize to other settings. In addition, PAM scores fluctuate over time and this study captures only two time points. ⁷ However, the RCT design and DID approach suggests the observed between-group differences were a true “intervention” effect rather than chance occurrence. Finally, the PAM is proprietary, is not available for use without a license and may not be practical as a clinical tool in the time-sensitive, ED setting.

CONCLUSION

Patient engagement in chronically ill older patients with limited health literacy was highest at the time of an ED encounter and fell in the following weeks. Qualitative findings confirm and extend these findings, demonstrating that patients’ decisions to visit the ED are the result of engaged decision-making. Our data suggest that an ED initiated coaching intervention reduced the degree of disengagement in healthcare after the ED visit. According to the National Quality Forum, “improved management of transitions of care into and out of the ED has the potential to improve person-centered care, quality, and cost efficiency.” ⁴³ Policy makers and health system managers should consider ED-initiated interventions like this to improve ED-to-home transitions and engagement in high-risk and hard-to-reach ED populations.

Supplementary Information

 

Footnotes

Section Editor: Mark I. Langdorf, MD, MHPE

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Donna L. Carden, MD, University of Florida, College of Medicine, Department of Emergency Medicine, P.O. Box 100186; 1329 SW 16^th Street Gainesville, FL 32610. Email: dcarden@ufl.edu. 6 / 2017; 18:743 – 751

Submission history: Revision received September 16, 2016; Submitted January 31, 2017; Accepted February 13, 2017

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. This study (ClinicalTrials.gov number, 01973296) was funded by a Health Policy Grant from the Emergency Medicine Foundation and supported by the UF Clinical and Translational Science Institute, the NIH National Center for Advancing Translational Sciences (UL1 TR000064), and the Patient-Centered Outcomes Research Institute (IHS-1306-01451). The opinions in this paper are solely the responsibility of the authors and do not necessarily represent the views of PCORI, its Board of Governors or Methodology Committee.

REFERENCES

1. Koh HK, Brach C, Harris LM, et al. A proposed ‘health literate care model’ would constitute a systems approach to improving patients’ engagement in care. Health Aff (Milwood). 2013;32(2):357-67.

2. Dentzer S. Rx for the ‘blockbuster drug’ of patient engagement. Health Aff (Milwood). 2013;32(2):202.

3. James J. Health Policy Brief: Patient Engagement. Health Aff (Milwood). 2013.

4. Reducing Non-urgent Use of Emergency Departments and Improving Appropriate Care in Appropriate Settings. 2014.

5. Koh HK, Berwick DM, Clancy CM, et al. New federal policy initiatives to boost health literacy can help the nation move beyond the cycle of costly ‘crisis care’. Health Aff (Milwood). 2012;31(2):434-43.

6. Greene J, Hibbard JH. Why does patient activation matter? An examination of the relationships between patient activation and health-related outcomes. J Gen Intern Med. 2012;27(5):520-6.

7. Greene J, Hibbard JH, Sacks R, et al. When patient activation levels change, health outcomes and costs change, too. Health Aff (Milwood). 2015;34(3):431-7.

8. Hibbard JH, Greene J, Tusler M. Improving the outcomes of disease management by tailoring care to the patient’s level of activation. Am J Manag Care. 2009;15(6):353-60.

9. Shively MJ, Gardetto NJ, Kodiath MF, et al. Effect of patient activation on self-management in patients with heart failure. J Cardiovasc Nurs. 2013;28(1):20-34.

10. Coleman EA, Parry C, Chalmers S, et al. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166(17):1822-8.

11. Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med. 2009;150(3):178-87.

12. Wallace AS, Seligman HK, Davis TC, et al. Literacy-appropriate educational materials and brief counseling improve diabetes self-management. Patient Educ Couns. 2009;75(3):328-33.

13. Hibbard JH, Stockard J, Mahoney ER, et al. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39(4Pt1):1005-26.

14. Pines JM, Mullins PM, Cooper JK, et al. National trends in emergency department use, care patterns, and quality of care of older adults in the United States. J Am Geriatr Soc. 2013;61(1):12-7.

15. Morganti K, Baufoff BS, Blanchard JC, et al. The Evolving Role of Emergency Departments in the United States. Rand Health Q. 2013;3(2):3.

16. Schumacher JR, Hall AG, Davis TC, et al. Potentially preventable use of emergency services: the role of low health literacy. Medical Care. 2013;51(8):654-8.

17. Griffey RT, Kennedy SK, D’Agostin , et al. Is low health literacy associated with increased emergency department utilization and recidivism?. Acad Emerg Med. 2014;21(10):1109-15.

18. Berkman ND, Sheridan SL, Donahue KE, et al. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97-107.

19. Cosgrove DM, Fisher M, Gabow P, et al. Ten strategies to lower costs, improve quality, and engage patients: the view from leading health system CEOs. Health Aff (Milwood). 2013;32(2):321-7.

20. Davis TC, Arnold C, Berkel HJ, et al. Knowledge and attitude on screening mammography among low-literate, low-income women. Cancer. 1996;78(9):1912-20.

21. Davis TC, Berkel HJ, Arnold CL, et al. Intervention to increase mammography utilization in a public hospital. J Gen Intern Med. 1998;13(4):230-3.

22. Raehl CL, Bond CA, Woods TJ, et al. Screening tests for intended medication adherence among the elderly. Ann Pharmacother. 2006;40(5):888-93.

23. Coleman EA, Smith JD, Frank JC, et al. Preparing patients and caregivers to participate in care delivered across settings: the Care Transitions Intervention. J Am Geriatr Soc. 2004;52(11):1817-25.

24. Coleman EA, Boult C. Improving the quality of transitional care for persons with complex care needs. J Am Geriatr Soc. 2003;51(4):556-7.

25. Medicare Current Beneficiary Survey.

26. Floyd J. Fowler Survey Research Methods. 2013.

27. Hibbard JH, Mahoney ER, Stockard J, et al. Development and testing of a short form of the patient activation measure. Health Serv Res. 2005;40(6 Pt 1):1918-30.

28. Mosen DM, Schmittdiel J, Hibbard J, et al. Is patient activation associated with outcomes of care for adults with chronic conditions?. The J Ambul Care Manage. 2007;30(1):21-9.

29. Hibbard JH, Cunningham PJ. How engaged are consumers in their health and health care, and why does it matter?. Res Brief. 2008(8):1-9.

30. Hibbard JH PE, Dixon A, Tusler M. Consumer Competencies and use of comparative quality information: it isn’t just about literacy. Med Care Res Rev. 2007;64(4):379-94.

31. Palta M. Quantitative Methods in Population Health: Extensions of Ordinary Regression. 2003.

32. Boyatzis R. Transforming qualitative information: Thematic analysis and code development. 1998.

33. Guest G, MacQueen KM, Namey EE. Applied thematic analysis. 2012.

34. Ryan GW, Bernard HR. Techniques to Identify Themes. Field Methods. 2003;15(1):85-109.

35. Charmaz K. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. 2006.

36. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101.

37. Ragin DF, Hwang U, Cydulka RK, et al. Reasons for using the emergency department: results of the EMPATH Study. Acad Emerg Med. 2005;12(12):1158-66.

38. Hibbard J, Greene J, Yunfeng Shi, et al. Taking the Long View: How Well Do Patient Activation Scores Predict Outcomes Four Years Later?. Med Care Res Rev. 2015;72(3):324-37.

39. Harvey L, Fowles JB, Xi M, et al. When activation changes, what else changes? the relationship between change in patient activation measure (PAM) and employees’ health status and health behaviors. Patient Educ Couns. 2012;88(2):338-43.

40. Fowles JB, Terry P, Xi M, et al. Measuring self-management of patients’ and employees’ health: further validation of the Patient Activation Measure (PAM) based on its relation to employee characteristics. Patient Educ Couns. 2009;77(1):116-22.

41. Pitts SR, Carrier ER, Rich EC, et al. Where Americans get acute care: increasingly, it’s not at their doctor’s office. Health Aff (Milwood). 2010;29(9):1620-9.

42. Katz EB, Carrier ER, Umscheid CA, et al. Comparative effectiveness of care coordination interventions in the emergency department: a systematic review. Ann Emerg Med. 2012;60(1):12-23 e11.

43. Emergency Department Quality of Transitions of Care. Available at: http://www.qualityforum.org/ProjectDescription.aspx?projectID=83442. Accessed January 25, 2017.