Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Sepsis is a clinical syndrome of life-threatening organ dysfunction caused by a dysregulated host response to infection.¹ Over the past 30 years, sepsis has increasingly become an area of interest both in diagnosis and management because of its high mortality rate. Despite this increased focus, the mortality rate of sepsis is still high,² averaging 39% worldwide.³ The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) recommended the application of the Sequential Organ Failure Assessment (SOFA) to identify organ dysfunction or failure in sepsis patients.¹ When SOFA was compared with the original Systemic Inflammatory Response Syndrome (SIRS) criteria, SOFA outperformed SIRS in predicting hospital mortality. The consensus suggested quick sequential organ failure assessment (qSOFA) as a screening tool in patients who are likely to have sepsis; qSOFA was proven to offer predictive validity similar to SOFA.⁴

In 2016 the Surviving Sepsis campaign recommended the implementation of sepsis screening, which has been shown to improve outcomes and reduce the mortality rate.⁵ Many predictive scores, such as the National Early Warning Score (NEWS), were developed and implemented to detect deterioration in sepsis patients.⁴ These scores can be used as a general screening tool as well as an early warning tool in the emergency department (ED), guiding collaboration with other areas in the hospital and the patient care system. The Emergency Severity Index (ESI) triage tool is a five-level ED triage algorithm that provides clinically relevant stratification of patients from 1 (the most emergent priority) to 5 (the least urgent priority) based on acuity and resource needs.⁶. However, the ESI triage tool was not specifically designed for severity classification in sepsis patients. The Search Out Severity (SOS) score was the early sepsis score used in Thailand. It has been shown that the implementation of a combined SOS score for screening with a checklist for sepsis bundles could decrease the mortality rate in Thailand.²

This study compares the accuracy of qSOFA, NEWS, SOS, SIRS, and ESI triage for predicting 28-day mortality in adult patients with sepsis, with the goal of designing an appropriate screening tool for use in the ED triage area.

METHODS

Study Design and Setting

This was a retrospective cross-sectional study. We collected data in the ED of a tertiary-care university hospital, between January–December 2019. The study was approved by the Ethics Committee of our institution.

Study Population

The study included patients >18 years who presented to the ED with a diagnosis of sepsis or infection-related conditions (Appendix 1) and had been treated with the sepsis protocol in the ED. We enrolled patients by day and alternated the days of the ED visit to reach the calculated sample size. The exclusion criteria were patients who transferred from other hospitals or areas of the hospital and patients with incomplete 28-day follow-up data.

Data Measurement and Outcomes

Data collection included patient demographics, presenting symptoms, vital signs recorded at the triage area, provisional diagnosis, hemoculture status, site of infection, 28-day intubation status, 28-day vasopressor time, and 28-day mortality. The variables of qSOFA, NEWS, SOS, SIRS, and ESI triage were recorded using the information gathered from the triage area of the ED (Appendix 2). The primary outcome was 28-day mortality. The secondary outcomes were mechanical ventilator usage within 28 days and vasopressor usage within 28 days.

Suspected sepsis was defined by physicians in the ED using the sepsis protocol, including qSOFA in Sepsis-3 criteria¹ or physicians’ clinical judgment in the ED. Some physiologic parameters were not used because our goal was to compare predictive scores, which were used as a screening tool in the ED triage area. Thus, for example, the maximum score for SIRS was 3 because white blood cell count was disregarded, and the SOS score did not include urine output. Furthermore, a Barthel index of 20 was used to define totally dependent activities of daily living (ADL), and heart failure with reduced ejection function (HFrEF) was defined as a left ventricular ejection fraction of 40% on transthoracic echocardiography, which was documented in the medical records.

Population Health Research Capsule

What do we already know about this issue?

Many screening tools are available at the triage area of the emergency department.

What was the research question?

Which triage screening tool is the most accurate for predicting mortality in patients with sepsis?

What was the major finding of the study?

The National Early Warning Score outperforms other sepsis screening tools (area under the receiver operator characteristic curve of 0.77) in predicting mortality, need for ventilator and vasopressors for patients evaluated at ED triage.

How does this improve population health?

Using the most accurate screening tool at ED triage could enhance the healthcare of the population, including patients with sepsis.

Sample Size and Data Analysis

We calculated the sample size for this study by using the equation N = Z_α/2²p(1 − p)/d^2, with the standard normal variate (Z_α/2) at 5%, the probability of expected sensitivity (p) equals 0.9. A two-sided test concluded that the minimum sample size would be 139 samples. The mortality rate for sepsis is 39%, as reported in a previous study.³

Statistical Analysis

We compared the survival and the nonsurvival groups by using the chi-square or Fisher’s exact test for categorical variables and the t-test for continuous variables. The data was presented as a percentage for categorical data and as a mean with standard deviation or median with interquartile range, as appropriate, for numerical data. The area under the receiver operating characteristic curve (AUROC), with a 95% confidence interval (CI), was depicted to evaluate the discrimination performance of each score. Sensitivity and specificity were calculated for each score as well. A P-value less than 0.05 was considered significant. We used STATA version 16.1 for statistical analysis (StataCorp LLC, College Station, TX).

RESULTS

In total, 550 patients were included in the analysis. A protocol flow chart is shown in Figure 1. The overall 28-day mortality was 12.4%. The overall 28-day mechanical ventilator usage and 28-day vasopressor usage were 23.2% and 18.1%, respectively. The mean age was 69 years, and 46.7% of patients were male. The three most common comorbidities were diabetes mellitus (31.6%), solid-organ malignancy (25.8%), and totally dependent ADL (19.8%). The mortality was significantly higher in comorbidities such as the solid organ tumor group, the hematologic malignancy group, and HFrEF group. Vital signs such as higher heart rate (118 vs 106, P <0.001) and respiratory rate (27 vs 24), P <0.001) and lower systolic blood pressure (112 vs 106, P <0.001) and oxygen saturation (92 vs 96), P <0.014) were significant in mortality. The patient demographic data in the survival and nonsurvival groups, is summarized in Table 1.

The primary outcome, 28-day mortality, was best detected by NEWS (area under the receiver operating characteristic curve [AUROC] = 0.770; 95% CI: 0.705–0.835), followed by SOS (AUROC = 0.749; 95% CI: 0.685–0.815, qSOFA (AUROC = 0.7473; 95% CI: 0.688–0.806]), ESI triage (AUROC = 0.599 ;95% CI: 0.542–0.656), and SIRS (AUROC = 0.588; 95% CI: 0.522–0.654], as shown in Table 2 and Figure 2. The sensitivity and specificity for predicting the 28-day mortality rates of all predictive scores at different threshold are presented in Table 3.

For the secondary outcomes, 28-day mechanical ventilator usage and vasopressor usage, NEWS provided a high AUROC and outperformed as shown in Table 2 and Figures 3, 4.

DISCUSSION

Our study demonstrates that NEWS has the best predictive performance for the 28-day mortality of sepsis patients at the triage area of the ED. In the same way, Omar et al reported that NEWS outperformed both SIRS (AUROC 0.95 vs 0.89; P 0.001) and qSOFA (AUROC 0.95 vs 0.87; P 0.001) in predicting death in only the severe sepsis and septic shock groups in the ED.⁷ Anniek et al determined that NEWS performed substantially better than qSOFA and SIRS in predicting both 10-day mortality (AUROC = 0.837, 0.744, and 0.646, respectively) and 30-day mortality (AUROC = 0.779, 0.697, and 0.631, respectively).⁸ Furthermore, NEWS showed a high performance in predicting 28-day mechanical ventilator and 28-day vasopressor used. These results were in accordance with the previous study by Churpek et al,⁹ which showed that general early warning scores (EWS) are more accurate than qSOFA in predicting adverse outcomes of sepsis outside the intensive care unit setting.

In the triage area of the ED, qSOFA was easier to assess by less experienced medical professionals.¹ However, qSOFA has a limited ability to predict poor outcomes in sepsis patients.^10,11 Additionally, the metrics used in EWS are standard measures that can be readily and rapidly performed throughout the healthcare system as well as in the ED triage area. The NEWS also demonstrates a higher performance than the SOS score, which necessitates information not available in the triage area. Chompunot et al² conducted a study in the hospital referral system that did not focus on the triage area. Their study found that the ESI score, which is commonly used as the general screening tool at ED triage, was inferior to NEWS, SOS, and qSOFA in predicting sepsis-related 28-day mortality, 28-day mechanical ventilator, and 28-day vasopressor used. Moreover, determining ESI at triage requires evaluator experience, as well as differing cut-off values of the parameters with other tools from other patient care systems.

Because of its strong predictive accuracy and simplicity, our findings support the use of NEWS as a screening tool in ED triage.^5,12-15 An automatic calculation in the sepsis alert system likewise correctly uses NEWS.¹⁶ A NEWS cut-off prediction score of ≥ 4 (sensitivity 91.30%, specificity 28.69%) and ≥ 5 (sensitivity 86.96%, specificity 45.74%) predicted sepsis-related 28-day mortality, according to our findings. The score has the highest sensitivity (90%) and specificity (25%) for activating sepsis alarms.

LIMITATIONS

This was a retrospective, single-center study. Second, it should be noted that substantial numbers of patients had advanced-stage malignancies, including solid organ and hematologic malignancies, which had a higher mortality rate. Additionally, patients who did not resuscitate were not excluded from our study, which could have affected the outcome.

CONCLUSION

The National Early Warning Score outperforms qSOFA, SOS, SIRS, and ESI triage scores in predicting 28-day mortality, mechanical ventilator usage, and vasopressor usage of a patient with sepsis in the triage area of the ED.

Footnotes

Section Editor: Stephen Liang, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Thidathit Prachanukool, MD, Mahidol University, Department of Emergency Medicine, 270 Rama VI Road, Ratchathewee, Bangkok, Thailand 10400. Email: thidathit.pra@mahidol.ac.th 9 / 2022; 23:698 – 705

Submission history: Revision received March 14, 2022; Submitted June 15, 2022; Accepted June 20, 2022

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801-10.

2. Champunot R, Kamsawang N, Tuandoung P, et al. Saving 500 Lives Campaign: another way to improve the mortality rate of patients with severe sepsis and septic shock. Crit Care. 2012;16(Suppl 3):P105.

3. Beale R, Reinhart K, Brunkhorst FM, et al. Promoting Global Research Excellence in Severe Sepsis (PROGRESS): lessons from an international sepsis registry. Infection. 2009;37(3):222-32.

4. Keep JW, Messmer AS, Sladden R, et al. National early warning score at emergency department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study. Emerg Med J. 2016;33(1):37-41.

5. Mearelli F, Orso D, Fiotti N, et al. Sepsis outside intensive care unit: the other side of the coin. Infection. 2015;43(1):1-11.

6. Tanabe P, Gilboy N, Travers DA. Emergency Severity Index version 4: clarifying common questions. J Emerg Nurs. 2007;33(2):182-5.

7. Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the emergency department. Am J Emerg Med. 2019;37(8):1490-7.

8. Brink A, Alsma J, Verdonschot RJCG, et al. Predicting mortality in patients with suspected sepsis at the emergency department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score. PLoS One. 2019;14(1):e0211133.

9. Churpek MM, Snyder A, Han X, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017;195(7):906-11.

10. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison of the performance of SOFA, qSOFA and SIRS for predicting mortality and organ failure among sepsis patients admitted to the intensive care unit in a middle-income country. J Crit Care. 2018;44:156-60.

11. Raith EP, Udy AA, Bailey M, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290-300.

12. Williams JM, Greenslade JH, McKenzie JV, et al. Systemic Inflammatory Response Syndrome, Quick Sequential Organ Function Assessment, and Organ Dysfunction: Insights from a prospective database of ED patients with infection. Chest. 2017;151(3):586-96.

13. Gyang E, Shieh L, Forsey L, et al. A nurse-driven screening tool for the early identification of sepsis in an intermediate care unit setting. J Hosp Med. 2015;10(2):97-103.

14. Manaktala S, Claypool SR. Evaluating the impact of a computerized surveillance algorithm and decision support system on sepsis mortality. J Am Med Inform Assoc. 2017;24(1):88-95.

15. Thiel SW, Rosini JM, Shannon W, et al. Early prediction of septic shock in hospitalized patients. J Hosp Med. 2010;5(1):19-25.

16. Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: a systematic review. J Hosp Med. 2015;10(6):396-402.