Author	Affiliation
B. Jason Theiling, MD	Duke University School of Medicine, Department of Emergency Medicine, Durham, North Carolina
Kendrick V. Kennedy, MD	Duke University School of Medicine, Department of Emergency Medicine, Durham, North Carolina
Alexander T. Limkakeng, MD	Duke University School of Medicine, Department of Emergency Medicine, Durham, North Carolina
Pratik Manandhar, BA	Duke University School of Medicine, Department of Biostatistics and Bioinformatics, Durham, North Carolina
Alaatin Erkanli, PhD	Duke University School of Medicine, Department of Biostatistics and Bioinformatics, Durham, North Carolina
Stephen R. Pitts, MD	Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

In the current practice of emergency medicine, triage functions to quickly prioritize care and sort patients by anticipated resource needs. While the goal of accurate prioritization is direct improvement in the quality of care of individuals, the intention of predicting resource utilization is to streamline emergency department (ED) operations without causing harm. The second goal has become increasingly important as the number of ED visits continues to rise, hospitals function under reduced available capacity, and reliance on the ED as the safety net of hospital systems increases.¹ This dual function of ED triage was proposed by Wuerz, who pioneered a five-level Emergency Severity Index (ESI).² Several studies have evaluated ESI’s reliability and validity.^2-10 Other five-level triage schemes have been developed outside the United States: the Canadian Triage and Acuity Scale (CTAS),^11-12 and the Australasian Triage Scale (ATS).¹³ Both CTAS and ATS assign each triage level a target “time to be seen,” which in turn allows comparisons between sites on the basis of compliance with these targets. The ATS is unique in explicitly accepting a third role as a data source for describing case-mix to generate the adjusted estimates of ED visit characteristics that inform national policy.¹²

Comparing ED performance and ED visit characteristics is more problematic in the US. Despite geographic variation in important ED characteristics (eg, the proportion of safety-net visits,¹⁴ population-based ED visit rates,¹⁵ and hospital admission rates¹⁶) there is still no national mandated standard for categorizing the acuity and resource complexity of ED visits. Central to the ESI system is the idea of “immediacy,” a marker of how acutely ill a patient is believed to be and thus how “immediately” they may need to be seen. Unfortunately, because of the continued widely disparate triage procedures and non-response in surveys, adjustment of ED data on the basis of the “immediacy” item alone is potentially biased and may ignore the dimension of care complexity. Additionally, the acuity or “immediacy” of a patient, as denoted by ESI, often does not linearly correlate with resource utilization during that ED visit.

The purpose of this study was to evaluate the “immediacy” variable in existing National Hospital Ambulatory Medical Care Survey (NHAMCS) survey data, and to create a practical alternative method for grouping ED visits by both acuity and resource complexity in a manner analogous to the ESI. To minimize data loss we sought to derive an ESI from NHAMCS survey items with low frequencies of item non-response. We assessed the performance of this index with respect to several outcomes, including hospital admission, waiting time, and overall ED length of stay (LOS).

This study uses the combined 2010–2015 ED components of the NHAMCS.¹⁶ The NHAMCS is a probability sample of US hospital EDs and outpatient departments conducted annually since 1992. It is one of a family of healthcare surveys performed by the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (NCHS). The US Census Bureau is responsible for field operations and data collection. Although one of its data items is currently a five-level item called “immediacy with which patient should be seen,” with additional checkboxes for “no triage” and “unknown,” other measures of urgency have been abstracted from ED charts in the past. From 1992–1996 the survey captured a highly subjective two-level “Urgent/emergent vs. Non-urgent” item, which led to the widely cited and heavily criticized conclusion that “55% of ED visits are non-urgent.”¹⁷ In 1997 this item was replaced by a four-level variable to capture more degrees of immediacy, each succeeding level associated with a progressively longer target “time to be seen.” In 2005 “immediacy” was promoted to the current five-level item, each level again associated with target times.

 

Population Health Research Capsule

What do we already know about this issue?

Triaging prioritizes care and sorts patients by anticipated resource needs. Despite widespread use of the Emergency Severity Index (ESI), no universal standard exists.

What was the research question?

Derive an ESI tool from a national survey item and assess the performance of this index with respect to stratifying outcomes.

What was the major finding of the study?

This tool can be used to compare complexity of ED visits across hospitals and regions, and over time.

How does this improve population health?

ESI may not reflect resource needs in linear fashion. Our tool helps to compare data across regions and time periods.

 

METHODS

The NHAMCS is a four-stage probability sample, sampled in the following sequence: 1) 112 geographic primary sampling units of approximately county size; 2) probability sample of nonfederal, short-stay, general hospitals with EDs or outpatient departments or both, within the sampled primary sampling units, selected from a publicly available database of all US hospitals; 3) emergency service areas within 24-hour EDs and clinics within outpatient departments; and 4) a sample of about 100 visits within the selected EDs or outpatient departments during a randomly assigned four-week reporting period throughout the year. We limited our analysis to the ED component of NHAMCS and downloaded data from the NHAMCS website (ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Datasets/NHAMCS). Hospital staff were asked to complete a patient record form (PRF) for a sample of visits during a four-week reporting period, from which the data were abstracted and coded. The NHAMCS was approved by Duke University Insitutational Review Board. A report published elsewhere describes the plan and operation of the NHAMCS in greater detail.¹⁷ Unless otherwise noted, all estimates in this report are weighted to give national estimates. We considered estimates based on an unweighted count of less than 30 to be unreliable.

Creating The NHAMCS-ESI Index

We based the NHAMCS-ESI (ESI-N) on the published ESI,² but used only variables available in NHAMCS (Table 1). Since ESI and other tools are used in the initial triage process, they are dependent on data available immediately upon or shortly after presentation. Thus, ESI-N uses the presenting complaint rather than the final diagnosis as the main component. For NHAMCS-ED, this complaint is abstracted directly from the actual ED chart into up to three free-text entry fields on the PRF. The PRFs are then batched, and the hand-written text is converted to standard codes by the Constella Group, Inc. (Durham, NC). According to the reason for visit classification for ambulatory care (RVC), there is a very low rate (<1%) of nonresponse. Additionally, vital signs have been recorded since 2001 and can be used to modify triage class just as the ESI does. Vital signs are not obtained on every visit, especially among the pediatric population; thus, when vital signs were missing, we considered the ESI-N unchanged rather than missing. Unlike the ESI, the retrospective ESI-N tallies actual resources used, rather than predicted resources, and cannot account for any resources not listed in the PRF.

Describing The Acuity Of The Patients And Validating The Index

We described the acuity of the study population by generating basic descriptive statistics including the mean ESI-N level and corresponding confidence intervals (CI) for selected patient and hospital characteristics. To validate the ESI-N, we examined associations with typical indicators of acuity such as waiting time, LOS, disposition, and mode of arrival. Waiting time was defined as the number of minutes between the time of arrival and the time seen by a physician. We defined LOS as the number of minutes between the time of arrival and time of ED discharge. All of these outcomes measures of acuity (waiting time, LOS, disposition, and mode of arrival) were measured in their respective units or proportions for each level of the derived ESI-N and compared using 95% CIs. We conducted all analyses using Stata v10 (StataCorp, College Station, TX).¹⁹

RESULTS

The “immediacy” item in NHAMCS-ED (IMMED) was unknown, or triage was not performed, in 15.0% of visits. Among the remaining visits, the maximum value of IMMED was less than five in 145 (26.2%) of 553 emergency service areas in the 457 hospitals surveyed. This suggests the continued use of triage schemes with fewer than five levels. Unfortunately, having four or fewer triage levels creates a bias toward lower (more acute) immediacy levels when incorporated in aggregate estimates.

These results are given both in raw counts of 2010–2015 patient record forms, and in a nationally representative estimate of percent of all visits. Figure 1 compares this frequency distribution both to the distribution of the variable IMMED in the same data, and to the distribution of actual ESI levels assigned by triage nurses in a prospective validation study of two ED populations.²⁰

We assessed mean ESI levels for a variety of patient characteristics, as demonstrated in Table 2. For each of the four patient-level characteristics that we assessed (age, gender, race/ethnicity, and payer type), there were statistically significant differences (p<0.0001) between mean ESI-N levels. Of note, there was a monotonic increase in acuity with increasing age. Within their respective categories, visits by women and by non-Hispanic Whites had on average more acute ESI-N scores. Among payer types, Medicare visits had on average the most acute ESI-N, with the least acute categories being worker’s compensation, “no charge,” and Medicaid/State Children’s Hospital Insurance Program.

We assessed validity of the ESI-N against several outcomes, including hospital admission, intensive care unit (ICU) admission, ambulance arrival, and leaving without being seen and in each case found the expected relationships (Table 3). As can be seen, the ESI-N is able to differentiate all visits into five levels of care, with only a minority being classified in the most severe, “Immediate” category. Relative to their proportion of total visits, more severe acuity ESI-N levels account for a higher proportion of the patients admitted to the hospital and those admitted to a critical care unit.

Furthermore, ESI-N is able to differentiate visits (Table 4) by likelihood of being admitted to the hospital and ED LOS. The 95% CIs of percent admitted or LOS in minutes between categories largely do not overlap as the ESI-N increases in severity from Level 5 “Nonurgent” to Level 1 “Immediate.” ESI-N does not appear to differentiate well among categories for time waiting to be seen by a physician, however.

DISCUSSION

The NHAMCS-ED is the only nationally representative survey of ED visits; thus, it is a valuable resource for policy-making and has provided data for many published studies. However, most causal inferences of interest, such as the effect of demographic variables like race/ethnicity on hospital admission, length of ED visit, and other aspects of quality and cost, are potentially confounded by the main determinant of these outcomes: the seriousness or acuity of the patient’s initial problem. A related reason to measure acuity is the need for fair comparisons across regions or over time, i.e., for case-mix adjustment. We have derived a five-level index of acuity called the ESI-N, because a similar existing item in the survey, called “immediacy,” frequently offered difficulties in chart abstraction: either a triage score was not obtained, or, in some cases when the score was obtained, it was absent from the chart. In addition, even when a triage score was present, there still is no universally accepted standard of triage in the US. In many cases, EDs may use fewer than five triage levels; in others the identical levels in different systems may have different definitions or applications. We attempted to overcome these limitations by deriving ESI-N, a severity index based primarily on the patient’s RFV codes, modified by values in a small number of other data fields, including age, vital sign extremes, and resuscitative procedures. Finally, we separated the less acute visits by number of resources used, emulating the previously derived and validated ESI.²

The American College of Emergency Physicians (ACEP) and the Emergency Nurses Association (ENA) formed a task force in 2002 dedicated to the goal of creating and promoting a standard measure of presenting patient acuity.²¹ However, to this day, a number of EDs continue to use locally designed triage guidelines of varying complexity and evidence quality or do not perform formal triage.²² In fact, this continued lack of a standard resulted in the ACEP and ENA revising its position statement and advocating for “implementing a standardized emergency department (ED) triage scale and acuity categorization process” and endorsing ESI as that process.²³

While ESI remains a prominent triage tool, it does not always adequately reflect resource need in linear fashion. That is, ESI Level 1s do not always necessitate the greatest resources in the ED, with ESI 2s requiring less, and so on. In fact, one paper found that ESI Level 2 and 3 patients are actually very similar in their resource needs, but hospitalization varied dramatically.²⁴ In contrast, we found positive associations between ESI-N severity and hospital admission, ICU admission, and ambulance transport.² Unlike one prominent early study,²² we found that patients who left without being seen were much more likely to have a low severity by ESI-N, a change that could reflect increased use of the ED as a “safety net” rather than for emergencies. This is congruent to findings of some more recent studies.^25-26

Waiting time is often cited as an indicator of ED quality that depends both on hospital capacity as well as the average ED volume and acuity of visits. Recent studies suggest that ED crowding has increased waiting times even for serious problems.²⁷ But excluding triage category 1, the ESI-N index discriminates poorly, possibly in part because the nurse assigning a triage category also controls the patient’s waiting time.

LIMITATIONS

While the ESI-N derived here appears to be a consistent, valid measure of acuity and complexity, there are limits on its use. It is complex: Creating the index requires an algorithm executed by a computer program. It is derived from the 2010–2015 NHAMCS-ED survey files, which were formatted in a specific way; however, variable names and values, such as the RVF classification scheme, have changed and will continue to change over time. Use of the algorithm in survey years other than 2010–2015 may require its modification. Like the ESI itself, devising the ESI-N algorithm required subjective judgment. The index reflects two separate dimensions of ED visits: acuity and complexity. Distinguishing between these dimensions is impossible when evaluating aggregate data. In this paper we describe and validate the ESI-N. It has not been validated and tested for reliability.

CONCLUSION

Two technical tasks commonly required in health services research include accounting for confounding of a causal relationship, and adjusting for case-mix to minimize selection bias when comparing groups.²⁸ Our derivation of a five-level severity index for data abstracted from ED charts addresses these research needs. The ESI-N can be used to stratify results, or as an ordinal exposure or outcome variable in regression or propensity score models, increasing statistical power by reducing the need to include multiple covariates in a model. Caution should be exercised in its application. While it includes observations lost with the use of IMMED, it is less predictive of ED waiting time. It might be used together with IMMED to reduce residual confounding in some analyses. It is important to understand the origin of the index in complaint codes modified by age and a few other variables, and that it segregates less acute visits based on a list of specified services performed. Since this list is short, and since some of the data abstracted are relatively nonspecific (eg, five-level instead of 10-level pain scores), the ESI-N will misclassify some patients compared to a prospectively determined ESI. Future directions include validating ESI-N on another independent source of ED visits and testing its use in a prospective cohort of ED visits.

Footnotes

Section Editor: Stormy Monks, PhD, MPH

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: B. Jason Theiling, MD, Duke University Medical Center, Division of Emergency Medicine, 3401 Erwin Road, Durham, NC, 27710. Email: Jason.theiling@duke.edu. 9 / 2020; 21:1147 – 1155

Submission history: Revision received August 9, 2019; Submitted April 30, 2020; Accepted June 19, 2020

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Ieraci S, Digiusto E, Sonntag P, et al. Streaming by case complexity: evaluation of a model for emergency department Fast Track. Emerg Med Australas. 2008;20(3):241-9.

2. Wuerz RC, Milne LW, Eitel DR, et al. Reliability and validity of a new five-level triage instrument. Acad Emerg Med. 2000;7(3):236-42.

3. Wuerz RC, Travers D, Gilboy N, et al. Implementation and refinement of the Emergency Severity Index. Acad Emerg Med. 2001;8(2):170-6.

4. Eitel DR. The Emergency Severity Index triage algorithm version 2 is reliable and valid. Acad Emerg Med. 2003;10(10):1070-80.

5. Tanabe P, Gimbel R, Yarnold PR, et al. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs. 2004;30(1):22-9.

6. Tanabe P, Gimbel R, Yarnold PR, et al. Reliability and validity of scores on the Emergency Severity Index version 3. Acad Emerg Med. 2004;11(1):59-65.

7. Baumann MR. Evaluation of the Emergency Severity Index (version 3) triage algorithm in pediatric patients. Acad Emerg Med. 2008;12(3):219-224.

8. Worster A, Gilboy N, Fernandes CM, et al. Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial. Can J Emerg Med. 2004;6(4):240-5.

9. Worster A, Fernandes CM, Eva K, et al. Predictive validity comparison of two five-level triage acuity scales. Eur J Emerg Med. 2007;14(4):188-92.

10. Bullard MJ, Unger B, Spence J, et al. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. Can J Emerg Med. 2008;10(2):136-42.

11. Warren DW, Jarvis A, LeBlanc L, et al. Revisions to the Canadian Triage and Acuity Scale paediatric guidelines (PaedCTAS). Can J Emerg Med. 2008;10(3):224-43.

12. Policy on the Australasian Triage Scale. 2000. Available at: http://www.acem.org.au/media/policies_and_guidelines/P06_Aust_Triage_Scale_-_Nov_2000.pdf. Accessed October 23, 2018.

13. Burt CW, Arispe IE. Characteristics of emergency departments serving high volumes of safety-net patients: United States, 2000. Vital and Health Statistics. 2004:1-16.

14. Cunningham PJ. What accounts for differences in the use of hospital emergency departments across U.S. communities?. Health Affair. 2006;25(Suppl1):w324-36.

15. Pitts SR, Niska RW, Xu J, et al. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. Natl Health Stat Report. 2008;7:1-38.

16. Fernandes CM, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force. J Emerg Nurs. 2005;31(1):39-118.

17. McCaig LF, McLemore T. Series 1: Programs and Collection Procedures. Plan and Operation of the National Hospital Ambulatory Medical Survey. 1994:1-78.

18. Wuerz R. Emergency Severity Index triage category is associated with six-month survival. Acad Emerg Med. 2008;8(1):61-4.

19. NHAMCS-ED SAS input statements. Available at: ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHAMCS/sas. Accessed October 25, 2008.

20. McHugh M, Tanabe P, McClelland M, et al. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med. 2012;19(1):106-109.

21. Burt CW, McCaig LF, Valverde RH. Analysis of ambulance transports and diversions among US emergency departments. Ann Emerg Med. 2006;47(4):317-26.

22. ACEP Policy Statements: Triage Scale Standardization. 2010. Available at: http://www.acep.org/Content.aspx?id=29828&terms=triage%scale. Accessed June 1, 2011.

23. Hocker MB, Gerardo CJ, Theiling BJ, et al. NHAMCS Validation of Emergency Severity Index as an indicator of emergency department resource utilization. West J Emerg Med. 2018;19(5):855-62.

24. Baker DW. Patients who leave a public hospital emergency department without being seen by a physician. Causes and consequences. JAMA. 1991;266(8):1085-90.

25. Hall J, Jelinek GA. Characteristics and outcomes of patients who “did not wait” after attending Perth public hospital emergency departments, 2000–2003. Med J Australia. 2007;187(11–12):626-9.

26. Bragulat E, Ortega M, Miro O. Increasing number of patients who leave the ED without being seen. Emerg Med J. 2008;25(1):60.

27. Wilper AP, Woolhandler S, Lasser KE, et al. Waits to see an emergency department physician: U.S. trends and predictors, 1997–2004. Health Affair. 2008;27(Suppl 1):w84-95.

28. Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess Rep. 2003;7(27):iii-173.