Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Background

Nearly 700,000 emergency department (ED) visits were due to acute heart failure (AHF) in 2009.^1-4 Most visits result in a hospital admission and account for the largest proportion of the projected $70 billion that will be spent on heart failure care by 2030.^4,5 There are few prognostic algorithms to guide in the decision to either admit or discharge a patient appropriately. Thus, clinicians may hospitalize some low-risk patients who have HF and may discharge home high-risk patients without being able to accurately assess prognosis.⁶ The lack of an accurate prognostic algorithm may be a contributing factor to the 80% admission rate for ED patients with AHF in the United States, which has remained unchanged over the last several years.⁷

A multicenter, Canadian cohort study reviewed the data of approximately 12,500 patients to derive and validate a risk model for predicting acute mortality in patients with HF who present to the ED. The randomly selected patients from 86 hospitals in the province of Ontario had visited an ED for HF and were discharged or hospitalized between April 1, 2004–March 31, 2007 (an average of 36 patients per hospital per year). Based on this data and assessing different variables such as age, transportation by emergency medical services (EMS), systolic blood pressure, heart rate, oxygen saturation (SpO₂), creatinine, potassium, troponin, active cancer, and metolazone use at home, the researchers calculated an Emergency Heart Failure Mortality Risk Grade (EHMRG). The EHMRG served to stratify seven-day mortality risk after initial presentation with AHF, regardless of whether the patient was discharged from the ED or hospitalized.⁸

A recent study by Lee et al examined the validation of the EHMRG score compared to physician-estimated risk of seven-day mortality.¹² Building on their conclusion that the EHMRG model proved to be a better indicator of seven-day mortality than physician estimates, we aimed to further validate this score by conducting a retrospective chart review to create a risk stratification of patients presenting to the ED with acute CHF exacerbation. There have been no studies performed in the US and independent of the original author to validate this score.

The purpose of our study was to validate and assess the performance of the EHMRG to risk stratify adult patients presenting to ED with acute CHF exacerbation based on seven-day mortality in our US population. Our secondary goal was to study the demographic patterns, disposition rates, and ED re-visit rate of CHF patients.

METHODS

Study Design and Setting

We conducted a retrospective chart review of patients presenting to the ED with acute CHF exacerbation between January 2014–January 2016. We extracted data from all patients admitted or discharged with an International Classification of Diseases, 9^th or 10^th Revision, (ICD 9 or 10) code for CHF as entered by an emergency physician. We obtained approval from our institutional review board before study initiation.

Selection of Participants

We collected data from eight EDs across the largest health system in New York via the electronic health record (EHR) system (Allscripts Healthcare Solutions, Chicago, IL). Of the eight EDs, four were located at tertiary care centers and four were at community centers spanning the boroughs of Manhattan and Queens and the counties of Nassau and Suffolk. The total number of annual visits was 442,059 in 2014 and 473,387 in 2015 (see Appendix B for the detailed distribution of volume per hospital site). Inclusion criteria included adult patients at least 18 years of age or older who presented to the ED for acute CHF categorized using ICD codes for CHF. We excluded patients who were younger than 18 and those who presented to the ED with HF related to end-stage renal disease. We also excluded patients who were pregnant or had renal failure. Patients receiving palliative care and patients who had a “do not resuscitate” directive on file were also excluded from the study.

Population Health Research Capsule

What do we already know about this issue?

The Emergency Heart Failure Mortality Risk Grade (EHMRG) has been validated in Canada to predict 7-day mortality but has not yet been validated in the United States (US).

What was the research question?

We assessed the performance of the EHMRG to predict 7-day mortality in patients presenting with congestive heart failure (CHF) exacerbation.

What was the major finding of the study?

The EHMRG adequately improved risk prediction of 7-day mortality for CHF-related ED visits in the US.

How does this improve population health?

Its use as a screening tool for the disposition decision of CHF-related ED visits encourages evaluation for social factors that may contribute to unsafe discharge.

Measurements

We collected both demographic and clinical data. Data abstraction was largely conducted electronically to reduce the error of documentation. Data not retrievable by electronic means, such as whether the patient was transported by EMS or used metolazone, was retrieved by three trained research associates who were blinded to the study hypothesis. The presence of active cancer was objective information obtained by chart extraction, and its clinical determination was made by chart review and confirmed with evidence of treatment. Troponin values were sorted as “yes” if values were above normal baseline levels and as “no” if values were not above normal levels (Table 1). Only the principal investigator and co-investigators had access to the data. The data was provided by a data analyst in a password-secured file and imported into REDCap (Research Electronic Data Capture, Vanderbilt University, Nashville, TN). Since EHMRG is a multivariable equation, an electronic, REDCap-based formula was developed to calculate the score and reduce the error of calculation. The data sheet was set to record the individual factors of EHMRG and then calculate the score. A pilot test was successfully conducted prior to the chart review.

Outcomes

Our primary outcome was seven-day mortality in these patients. The mortality data was collected by a proxy system. For patients who were discharged before seven days from the ED visit, we reviewed their EHR for subsequent clinic visit or revisit to any point of contact in our health system. The patients were flagged as deceased by the insurance provider, and dates of death were recorded to calculate mortality.

Analysis

We conducted descriptive analyses for all variables to assess their distribution. Categorical variables are presented as percentages and continuous variables as medians with interquartile ranges (IQR). All ED visits (including multiple ED visits for some patients) were used in multivariable analysis. We used two separate mixed-effects logistic regression models to allow for multiple ED visits per patient. We clustered by patient health record number in order to validate the assumption of correlated outcomes in the same patient. The first model used the individual metrics to derive the EHMRG score as individual-level predictors. The second model added the EHMRG score to the first model to predict seven-day mortality.

To determine appropriate sample size, we used events per variable ratio criteria. With 10 variables, an adequate number of events would be 100. We also estimated odds ratios and 95% confidence intervals (CI) for the EHMRG score quantiles from the second model. The apparent area under the receiver-operating characteristic (ROC) curve was estimated and plotted in a graph for the second model. We also estimated the C-statistic of each model and estimated the difference in C-statistic and P value for difference in C-statistic. The C-statistic is a unitless index denoting the probability that a randomly selected subject who experienced the outcome will have a higher predicted probability of having the outcome occur compared to a randomly selected subject who did not experience the event. It can also be interpreted as the rank correlation between predicted probabilities of the event occurring and the observed response. Finally, we estimated the continuous net reclassification index for events and nonevents separately. All analyses were conducted using R Statistical software (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Characteristics of Study Subjects

Over the study period we used ICD codes to identify 3,782 ED visits that met the inclusion criteria. Of these, 462 were excluded because they did not meet the inclusion criteria or had incomplete data, leaving a final sample size of 3320 ED visits among 2,495 unique patients. Disposition data was missing from 189 visits; 123 visits were excluded because of renal failure; 34 had missing data on EMS transport; six were missing cancer information; three were missing metolazone information; one was missing initial SpO₂; 22 were missing creatinine levels; 83 were missing potassium serum data; and one was missing heart rate data. We found no evidence that visits with missing data were different than visits with data (P = 0.51); thus, we analyzed only those with complete data. Of the 3,320 ED visits included in the analysis, 3,144 patients (94.7%) were admitted to the hospital and 113 (3.4%) were discharged from the ED. Nineteen patients (0.6%) left against medical advice; 22 (0.7%) were observed; 20 (0.6%) cases were transferred; and two (0.0%) passed away in the ED.

The median age among all ED visits was 78.9 years old with 1,607 (48.4%) females, and the median length of stay for those admitted to the hospital was seven days. The predominant racial category was White, constituting 55.4% of the ED visits. Table 1 presents the overall distribution of demographics and disposition of all ED visits. Table 2 presents median EHMRG scores by disposition type.

We fit two separate mixed-effects logistic regression models. The first logistic regression input all of the variables used to calculate EHMRG scores individually into the model. The second logistic regression model added the quantiles of the EHMRG scores. The C-statistic of the multivariable model using the individual variables was 0.748 (95% CI, 0.683 to 0.813) and the C-statistic of the model with EHMRG quantiles was 0.772 (95% CI, 0.729 to 0.815), suggesting that the model using EHMRG was superior for risk stratification (see Figures 1 and 2 for ROC curve). The C-statistic of the model with EHMRG was 0.024 (3.2% higher) than the C-statistic of the model with individual covariates, a significant improvement (P = 0.04). The net reclassification index for events was 0.25 and 0.13 for non-events. As shown in Table 3, patients who were in risk quantiles 5b had 6.16 times the odds of dying within seven days as compared to those in risk quantiles 1.

DISCUSSION

In this study of CHF-related ED visits in a large US health system, we found that the EHMRG score adequately improved risk prediction of seven-day mortality, validating previous studies in other countries. Specifically, the C-statistic was higher when using EHMR quantile and this model led to 25% and 13% of events and non-events to be classified into a better risk category, respectively. Given that both the C-statistic and the net reclassification improvement both have limitations, we used both to produce consistent and robust results.

Our median age was 79, which is comparable to the median age of 75 in the original EHMRG study and supports the fact that CHF is an advanced and chronic illness, mostly affecting the elderly. In our study we found that more than half of ED visits (53%) used EMS as a transport to the ED, which was much higher than the original study that recorded 38.5–43.4%.⁸ We also found that metolazone was used twice as often in the Canadian data set; however, we had a lower percentage of patients with active cancer who were suffering from CHF exacerbation in our data set. Our mortality rate overall was very similar to the Canadian study. Despite having the same seven-day mortality rate in our population as compared to the Canadian study, we identified that hospitalization mortality rates were higher among patients who were admitted to the hospital, as one may expect. We had a much higher percentage of non-normal troponin levels, 24% as compared to 10.5–14.6% in the original data set. The creatinine levels, systolic blood pressure, heart rate, and mean potassium levels were similar in our data set as compared to the Canadian data set. We found a similar mean SpO₂ level in our data set of 91.5% vs 92.9–93.9% in the original study.⁸

The Canadian study enrolled 12,500 patients from 86 hospitals over a three-year period, which amounts to an average of 36 patients per year per hospital. Patients were randomly selected. This was different from our selection of patients. However, the number of hospitals covered was greater and it adjusts for more variation and practice patterns.⁸ We had an admission rate of 94.7%, which was higher than the national average admission rate of 80–85%; we had 0 outpatient discharge seven-day mortality. Additionally, the ED revisit rate was 24.8%, which is the national US average for CHF patients.⁹

The US and Canadian healthcare systems are different and hence the risk involved in discharging a patient in the US is higher medically and legally, which would explain no mortality in the discharge group. However, in our system we still had a 24.8% ED revisit for CHF, which is similar to the national average. Therefore, it is unclear whether prior admission to the hospital prevents further revisits to the ED and hence readmissions for the same complaint. It is questionable whether the presence of a prior admission is helpful in decreasing the chance of a revisit to ED.¹⁰

The percentage of health costs paid by the government in Canada is 71.3% vs 49.1% in the US.¹¹ There was no difference in the mortality rate of our data set as compared to the Canadian study. This leads us to explore the role of hospitalization on short-term mortality. It has been well established that hospitalization is an independent predictor of long-term mortality in CHF patients.

LIMITATIONS

This was a retrospective study of a single (albeit large) health system. This hospital system is in the Northeast US, and the population is more diverse as compared to the rest of the country. All the patients were not unique in our study; however, we considered each CHF visit to be unique when a patient presented to the ED.

The predictive ability of the EHMRG score in our data set was less than the Canadian study. This may be due to the low power of the study (only 69 events). All patients in our data set who had a seven-day mortality were hospitalized, which leads us to question the impact of hospitalization on mortality. A lack of cohort of discharged patients with seven-day mortality is a significant limitation of our study, as it depicts that the US population and culture of medicine are quite different than those of Canada. Another limitation was the use of ICD codes to identify patients with CHF (as opposed to Framingham heart failure diagnostic criteria), as it was practically difficult to apply the criteria retrospectively due to documentation limitations.⁸ Finally, we did not account for patients who were diagnosed with CHF exacerbation after admission to the hospital.

CONCLUSION

In summary, risk prediction of seven-day mortality was superior when using a model that implemented the Emergency Heart Failure Mortality Risk Grade compared to a model that used the individual components of the EHMRG as covariates. EHMRG can be used as a good screening tool in the US while considering the disposition decision for these patients presenting with acute exacerbation of CHF. The objective evaluation may point physicians to evaluate for social factors that might contribute to unsafe discharge or poor outcomes. Potentially, some of the low-risk patients can be evaluated for social needs by a case manager or social worker, if available in the ED, and be safely discharged. This will potentially avoid some admissions and hence readmissions for the same issue.

Footnotes

Section Editor: Sam Torbati, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Nidhi Garg, MD, Southside Hospital, Department of Emergency Medicine, 301 E. Main Street, Bayshore, NY 11706. Email: ngarg1@northwell.edu. 5 / 2021; 22:672 – 677

Submission history: Revision received July 6, 2020; Accepted January 11, 2021

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Adams KF, Fonarow GC, Emerman CL, et al. Characteristics and outcomes of patients hospitalized for heart failure in the United States: Rationale, design, and preliminary observations from the first 100,000 cases in the Acute Decompensated Heart Failure National Registry (ADHERE). Am Heart J. 2005;149:209-16.

2. Cleland JG, Swedberg K, Follath F, et al. The EuroHeart Failure Survey programme: a survey on the quality of care among patients with heart failure in Europe. Part 1: patient characteristics and diagnosis. Eur Heart J. 2003;24:442-63.

3. Nieminen MS, Brutsaert D, Dickstein K, et al. EuroHeart Failure Survey II (EHFS II): a survey on hospitalized acute heart failure patients: Description of population. Eur Heart J. 2006;27:2725-36.

4. Schappert SM, Burt CW. Ambulatory care visits to physician offices, hospital outpatient departments, and emergency departments: United States, 2001-02. Vital Health Stat. 2006;159:1-66.

5. Hugli O, Braun JE, Kim S, et al. United States emergency department visits for acute decompensated heart failure, 1992 to 2001. Am J Cardiol. 2005;96:1537-42.

6. Stiell IG, Clement CM, Brison RJ, et al. A risk scoring system to identify emergency department patients with heart failure at high risk for serious adverse events. Academic Emergency Medicine. 2013;20(1):17-26.

7. Miró Ò, Peacock FW, McMurray JJ, et al. Acute Heart Failure Study Group of the ESC Acute Cardiovascular Care Association. Eur Heart J Acute Cardiovasc Care. 2016.

8. Lee DS, Stitt A, Austin PC, et al. Prediction of heart failure mortality in emergent care. a cohort study. Ann Intern Med. 2012;156:767-775.

9. Collins SP, Pang PS, Fonarow GS, et al. Is hospital admission for heart failure really necessary? The role of the ED and observation unit in preventing hospitalization and rehospitalization. J Am Coll Cardiol. 2013;61(2):121-26.

10. Guyatt GH, Devereaux PJ, Lexchin J, et al. A systematic review of studies comparing health outcomes in Canada and the United States. Open Medicine. ;1:1.

11. World development indicators, gross domestic product 2016, United States. 2017. Available at: http://databank.worldbank.org/data/download/GDP.pdf. Accessed June 29, 2020.

12. Lee DS, Lee JS, Schull MJ, et al. Prospective validation of the emergency heart failure mortality risk grade for acute heart failure. Circulation. 2019;139(9):1146-56.