Author	Affiliation
Gelareh Z. Gabayan, MD, MSHS	University of California, Los Angeles, Department of Emergency Medicine, Los Angeles, California
Michael K. Gould, MD, MS	Kaiser Permanente Southern California, Department of Research and Evaluation, Pasadena, California
Robert E. Weiss, PhD	University of California, Los Angeles, Fielding School of Public Health, Department of Biostatistics, Los Angeles, California
Vicki Y. Chiu, MS	Kaiser Permanente Southern California, Department of Research and Evaluation, Pasadena, California
Catherine A. Sarkisian, MD, MSPH	University of California, Los Angeles, Department of Medicine, Los Angeles, California; Greater Los Angeles Veterans Affairs Healthcare System, Department of Medicine, Los Angeles, California

Introduction
Methods
Results
Discussion
Limitations
Conclusion
Supplementary information

INTRODUCTION

The emergency department (ED) is inherently a high-risk setting, and understanding outcomes following the ED visit is difficult. This is mainly due to the inability to track most patient visits after discharge. Knowledge of a risk score for negative outcomes following ED discharge and probabilities of those outcomes for adults discharged from the ED could help ED practitioners better manage patients as well as their discharge plan.

Risk scores have traditionally helped ED practitioners better understand the risks ED patients face when presenting with certain conditions and signs/symptoms.¹ The objective of this study was to conduct a retrospective cohort analysis and develop a risk score for adults experiencing an inpatient admission or death/intensive care unit (ICU) placement within seven days of ED discharge.

METHODS

Study Design

A multisite retrospective cohort study of ED visits was conducted following the Strengthening the Reporting of Observational studies in Epidemiology guidelines.² This study was approved by the institutional review boards of Kaiser Permanente Southern California and the University of California, Los Angeles.

Setting

We analyzed clinical data from Kaiser Permanente Southern California (KPSC), an integrated health system that provides comprehensive care to over 3.5 million members at 14 medical centers and 197 offices throughout Southern California. There were 13 health system EDs in operation during the study period. All members have very similar healthcare benefits, including coverage of emergency services both within and outside the health system. Members of the health plan are generally representative of the population of Southern California, which is a racially and socioeconomically diverse region.³ Approximately 7% of members enroll through Medicaid and 10% through Medicare.

Selection of Participants

Patients were members of KPSC with at least one ED visit and discharge from January 1, 2009, to December 31, 2010. A patient had to be a member of the health plan at the time of the ED visit; however, no minimum enrollment history was required. Analyses were restricted to adults age ≥ 65 years, as these patients have a greater number of poor outcomes after discharge.^4,5 All patients in the study were discharged from the ED to home or a non-acute care facility such as a nursing home or rehabilitation facility. If patients had multiple ED visits, then only the first visit was included in the analysis.

We exluded patients who left the ED without being seen by a health provider. Patients transferred to observation status from the ED were also excluded, as encounters in this setting could resemble an inpatient admission. Patients receiving hospice care were also excluded, as the goal of this type of care is to provide palliative services rather than prolong life. In addition, patients who were transferred to and from other hospitals were excluded. The small number (<0.1%) of visit records that had potentially erroneous day and time entries resulting in either negative or excessively long ED lengths of stay (LOS) (>48 hours), were also excluded from the analysis.

Data Sources

The analyses used the Kaiser Permanente Epic-based electronic health record (EHR) (KP HealthConnect) for all variables. The EHR contains records of all member visits to health plan EDs. This system contains past history, mode of arrival, vital signs, staff notes, orders, diagnoses, and test results. Standardized data fields from ED visits provide time-stamps for patient registration, triage, assignment to provider, and disposition order (discharge to home, a care facility, or an inpatient bed). KP HealthConnect was also used to identify the International Classification of Diseases (ICD) diagnoses associated with the ED visit.

 

Population Health Research Capsule

What do we already know about this issue?

We know that older white males are at greater risk for poor outcomes after emergency department (ED) discharge, a change in disposition from “admit” to “discharge”, cognitive impairment, systolic blood pressure (SBP) < 120 mmHg, and pulse > 90 beats/min.

What was the research question?

This study developed a risk score for adults experiencing an admission or death/intensive care unit placement within 7 days of ED discharge.

What was the major finding of the study?

Patients at risk for either outcome were: age > 80, body mass index<18.5, SBP < 120 mmHg, pulse >100 bpm, high comorbidities, ED length of stay > 4 hrs, and prior admission.

How does this improve population health?

This information helps ED providers and hospital administrators better manage ED patients.

 

Risk Factors

The clinical variables were dichotomized a priori by members of the project team (GZG, MKG, SFG, CAS) based on clinical judgment and prior literature.^6-8 Rather than incorporating all vital signs, two vital signs were chosen for parsimony of the risk score.⁹ For 96% of encounters, patients had at least a single vital sign recorded. For patients with visits with more than one measure for a given vital sign, the vital sign closest to discharge was chosen for the analysis. For extreme values of vital signs that were not compatible with life and most likely a coding error, the vital signs were coded as missing: systolic blood pressure (SBP) <50 or >300, heart rate (HR) < 25 or >225. In addition, as the team has shown ED LOS to be a possible risk factor for poor outcomes after discharge, ⁷ we included ED LOS in the model and defined it as the total time a patient spent in the ED from the time they checked in to ED triage to the time they were discharged from the ED.

Outcome Measures

The primary outcome was an inpatient admission within seven days of discharge from the ED and the secondary outcome was ICU admission or death over the same time period. The seven-day period was chosen based on frequency results that indicated the highest percentage of admissions occurring within seven days of discharge and also because of its clinical relevance, implications for health policy decisions, and use in previous studies.^6,10,11 Information regarding admissions to non-KPSC hospitals was obtained through Kaiser billing data. Deaths were identified using vital statistics data from the California Vital Statistics files linked to Kaiser billing data.

Analysis

We treated each outcome in the same manner. All patient ED visits were identified over the two years and randomly divided into a derivation sample (75% data) and validation sample (25% data). First, each patient characteristic was assessed for associations with the outcomes in the derivation sample using a Pearson’s chi square test. Then, we included statistically significant variables (p <0.1) in the full logistic regression model for each outcome. To evaluate the accuracy of the final logistic regression models, the study team inspected receiver operating characteristic curves and calculated a C-statistic.¹²

To arrive at the risk score, the study team standardized all coefficient estimates of the model variables by dividing by the smallest variable coefficient. Then, a numeric score (point) was applied to each variable based on the result. All points were rounded to the nearest whole number. Predicted probabilities were calculated for each risk score.¹³ To evaluate the calibration of the scoring system, the study team compared the predicted probability of a given score with the observed probability in both the derivation sample and the validation sample.

In the model for inpatient admission, the initial variables were age, gender, race/ethnicity, smoking, Emergency Severity Index, body mass index (BMI), vital signs of SBP and HR, CCI¹⁴ (see Appendix for measures), ED LOS, ED visit in week prior, and inpatient admission in week prior the ED visit. Then, the study team omitted the variables not statistically significantly associated with the outcome (p-value >0.1) and arrived at a final model of age, BMI, vital signs of SBP and HR, CCI, ED LOS, and inpatient admission in week prior. For death or ICU admission, the same methodology was used, except that the final model included gender.

RESULTS

Characteristics of Study Subjects

As illustrated in the figure, during years 2009–2010, there were 1,552,594 visits among 922,005 patients to KPSC ED facilities. Excluded from the analyses were the following: visits from non-KPSC members; visits with missing gender or birthdate; ages lower than 65 years; patients in hospice care; transfers out of or into the ED; death in the ED; direct admission to an inpatient or observation bed from the ED; and visits other than the first visit. The study cohort contained 104,025 patient visits, of which 4.5% experienced an inpatient admission within seven days of ED discharge and 0.5 % either died or had an ICU admission.

Characteristics of the patients are presented in Table 1 by outcome. The mean age of patients who visited the ED and were discharged was 75.3 years (standard deviation [SD] 7.6), while the mean age for patients with an admission and death/ICU placement respectively was 76.8 years (SD 7.9) and 78.0 years (SD 8.3). The cohort contained slightly more visits by females (57%, n=59,517), as well as White (54%, n=56,052) and Hispanic (22%, n=22,963) patients.

Main Results

Table 2 presents the risk scores for the two outcomes based on the logistic regression models that were composed. The minimum score a patient could receive was 0 for ages 65–79 years, BMI ≥ 18.5, SBP > 120 mmHg, HR < 100 bpm, no Charlson comorbidities, ED LOS < 5 hours, and no inpatient admission the week prior. The maximum score was 30. In the model predicting death/ICU placement, the minimum score a patient could receive was 0 and the maximum 21. While risk factors were similar for the two outcomes, the scoring was slightly different.

To illustrate the application of the risk score, assume a very thin (with an estimated BMI of 16), 70-year-old male is seen in the ED with a SBP of 110 millimeters mercury (mmHg), HR of 110 beats per minute (bpm), has a history of diabetes and hypertension, stays in the ED for 12 hours, and has not been admitted in the prior seven days. The patient’s risk factors would give him a score of 19 for inpatient admission and 16 for death/ICU placement.

To assess the validity of the risk scores, the predicted as well as the observed probabilities of seven-day admission and seven-day death/ICU placement were assessed (Appendix). Following the numerical predictions are plots and ROC curves for the two models. The C-statistic was 0.68 (for inpatient admission) and 0.76 (for death/ICU placement).

DISCUSSION

The study identified simple measures that can be used to calculate a risk score for developing a poor outcome after ED discharge. Patients with the greatest likelihood and highest score (score of 40) for developing an inpatient admission within seven days of discharge were age ≥ 80 years old (score of 1), BMI <18.5 (score of 3), SBP ≤120 mm Hg (score of 2), HR ≥ 100 bpm (score of 4), CCI score of 7 or greater (score of 8), ED LOS of 10–24 hours (score of 7), and an inpatient admission in the past seven days (score of 5). Patients at greatest risk for death or an ICU placement (score of 19) were male (score of 1), age ≥ 80 years old (score of 1), BMI <18.5 (score of 3), SBP ≤120 mmHg (Score of 2), HR ≥100 bpm (score of 3), CCI score of 7+ (Score of 5), and ED LOS of 10–24 hours (score of 4).

A low BMI (<18.5) led to greater risk for either outcome. Various studies have found that adults with higher BMIs, either overweight range (BMI ≥ 27.3) or obese (obese ≥ 30), often experience worse outcomes.^15-17 Yet, recent studies suggest that older adults with high BMIs have lower incidences of poor outcomes and that a low BMI could result in worse outcomes.^16,18 The current study results in older adults confirm these findings.

As can be clinically concerning, an SBP below or equal to 120 mmHg and a HR ≥ 100 bpm was associated with a poor outcome after discharge. While these vital signs are markers of hemodynamic instability, they should especially concern an emergency provider.

The study found a high CCI¹⁴ (>4) to be the greatest predictor (with highest number of points) for both outcomes. Since its publication, the CCI has undergone numerous modifications to conform with recent changes in ICD codes.^19,20 This study indicates that although a specific complaint (i.e., chest pain) requires attention, so too does the past medical history.

While ED LOS, defined as the total time a patient remains in the ED from registration to discharge, can be a marker of ED crowding, it may capture something unrelated to ED crowding about the patient’s complexity and risk for poor outcomes. There have been conflicting results regarding ED LOS and outcomes after discharge, which suggest that this is a complicated measure. A prior study that did not adjust for case-mix severity found a relationship between ED LOS and poor outcomes after discharge.¹¹ A study conducted by our project team that did adjust for case mix did not find an association with a poor outcome.⁷ This study found that a prolonged ED LOS past four hours contributes to the risk score (5–9 hours, 2 points) and (10–24 hours, 4 points).

Admission in the past seven days was also found to contribute to developing a poor outcome after discharge (inpatient admission, 5 points; death/ICU placement, 2 points). Older adults have a higher rate of utilization of medical services²¹ and prior studies have attempted to predict hospital utilization following an ED visit.^22-25 Yet there is insufficient evidence to understand whether patients with recent use of hospital services are at greater risk for a poor outcome after ED discharge. This study found that patients with an inpatient hospitalization within the seven days prior to the ED visit have a greater likelihood of a poor outcome after discharge.

Although this study identified patients who sustain poor outcomes after discharge, the study team did not determine whether the outcomes were preventable vs. inevitable. The study team suggests that the risk score components identified compose either an electronic or mental flag for each provider seeing the patient. The provider could then ensure that the patient receives an evaluation prior to discharge or arrange for close follow-up following discharge. In addition, the rate of discharge of patients in this cohort was higher than national averages, which have been found to be in the 40% range.²⁶ This could be attributed to the lack of generalizability of the KPSC system as indicated below. It may also affect the rate of admission rates after discharge.

LIMITATIONS

The study has some limitations. First, as indicated above, the results may not generalize to other settings. KPSC members have access to follow-up care that patients in other settings may lack. KPSC hospitals may also have different disposition courses for patients seen in the ED as a cause of their follow-up options. Second, the admission outcome did not include observation stays. Given the increasing use of observation services, however, future studies should consider incorporating observation stays into admission outcomes. A third limitation inherent to the type of study performed was the lack of available clinical information regarding the chief complaint as well as the extent of management/treatment performed. Also, the reason why patients were admitted or died following discharge from the ED is unknown. Finally, the data used for this analysis are for years 2009–2010; while this is an extended time frame, patients have not changed since then.

CONCLUSION

This study determined two risk scores for developing a poor outcome following ED discharge in an integrated health system. Patients at greatest risk for either inpatient admission or death/ICU placement within seven days of ED discharge have the following characteristics: age ≥ 80, BMI <18.5, SBP ≤ 120 mmHg, HR ≥100 bpm, high number of comorbidities, ED LOS greater than four hours, and prior inpatient admission in seven days prior to the ED visit.

Supplementary Information

 

Footnotes

Section Editor: David Lee, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Gelareh Z. Gabayan, MD, MSHS, 924 Westwood Blvd Ste 300, Los Angeles, CA 90024-1777. Email: Gelareh@gabayan.com. 9 / 2018; 19:842 – 848

Submission history: Revision received February 16, 2018; Submitted June 6, 2018; Accepted July 20, 2018

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa ankle rules. JAMA. 1994;271(11):827-32.

2. von Elm E, Altman DG, Egger M, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;370(9596):1453-7.

3. Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J. 2012;16(3):37-41.

4. Gabayan GZ, Sarkisian CA, Liang LJ, et al. Predictors of admission after emergency department discharge in older adults. J Am Geriatr Soc. 2015;63(1):39-45.

5. Gabayan GZ, Gould MK, Weiss RE, et al. Poor outcomes after emergency department discharge of the elderly: a case-control study. Ann Emerg Med. 2016;68(1):43-51.e2.

6. Gabayan GZ, Derose SF, Asch SM, et al. Patterns and predictors of short-term death after emergency department discharge. Ann Emerg Med. 2011;58(6):551-8.e2.

7. Gabayan GZ, Derose SF, Chiu VY, et al. Emergency department crowding and outcomes after emergency department discharge. Ann Emerg Med. 2015;66(5):483-92.e5.

8. Gabayan GZ, Asch SM, Hsia RY, et al. Factors associated with short-term bounce-back admissions following emergency department discharge. Ann Emerg Med. 2013;62(2):136-144.e1.

9. Seidel HM, Ball JW, Dains JE, et al. Mosby’s Guide to Physical Examination. 2011.

10. Sklar DP, Crandall CS, Loeliger E, et al. Unanticipated death after discharge home from the emergency department. Ann Emerg Med. 2007;49(6):735-45.

11. Guttmann A, Schull MJ, Vermeulen MJ, et al. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada. BMJ. 2011;342:d2983.

12. Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. 2013.

13. Wu C, Hannan EL, Walford G, et al. A risk score to predict in-hospital mortality for percutaneous coronary interventions. J Am Coll Cardiol. 2006;47(3):654-60.

14. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-83.

15. Aune D, Sen A, Prasad M, et al. BMI and all cause mortality: systematic review and non-linear dose-response meta-analysis of 230 cohort studies with 3.74 million deaths among 30.3 million participants. BMJ. 2016;353:i2156.

16. Berraho M, Nejjari C, Raherison C, et al. Body mass index, disability, and 13-year mortality in older French adults. J Aging Health. 2010;22(1):68-83.

17. Flegal KM, Kit BK, Orpana H, et al. Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA. 2013;309(1):71-82.

18. Janssen I, Katzmarzyk PT, Ross R. Body mass index is inversely related to mortality in older people after adjustment for waist circumference. J Am Geriatr Soc. 2005;53(12):2112-8.

19. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613-9.

20. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-9.

21. Aminzadeh F, Dalziel WB. Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med. 2002;39(3):238-47.

22. McCusker J, Bellavance F, Cardin S, et al. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med. 2000;36(5):438-45.

23. McCusker J, Bellavance F, Cardin S, et al. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc. 1999;47(10):1229-37.

24. McCusker J, Cardin S, Bellavance F, et al. Return to the emergency department among elders: patterns and predictors. Acad Emerg Med. 2000;7(3):249-59.

25. McCusker J, Healey E, Bellavance F, et al. Predictors of repeat emergency department visits by elders. Acad Emerg Med. 1997;4(6):581-8.

26. Gabayan GZ, Liang LJ, Doyle B, et al. Emergency department increased use of observation care for elderly Medicare patients. J Hosp Adm. 2018;7(3):9-16.