Author	Affiliation
Timothy Horeczko, MD, MSCR	University of California Los Angeles, Department of Emergency Medicine, Torrance, California
Jeffrey P. Green, MD	University of California Davis, Department of Emergency Medicine, Sacramento, California
Edward A. Panacek, MD, MPH	University of California Davis, Department of Emergency Medicine, Sacramento, California

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Faced with burgeoning knowledge of the pathogenesis of sepsis and the need for early recognition, the American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference prefaced its landmark 1992 report with the expectation that “the broad definition proposed in this report will improve our ability to make early bedside detection of disease possible, and thus allow early therapeutic intervention.”¹ The term “systemic inflammatory response syndrome” (SIRS) was coined to encompass the “common pathogenic link now thought to be present in a number of disorders.”² In turn, the concept of SIRS was not limited to infectious disorders, but instead was used to describe a physiologic response to a variety of acute insults, such as pancreatitis, ischemia, trauma, hemorrhage, and immune-mediated organ injury.¹

Consensus guidelines recommend immediate diagnostic testing for adult patients with SIRS and a suspected infection.³ Based on these recommendations, large healthcare systems and international task forces have used SIRS as an inclusion criterion for adult sepsis screening protocols, an approach supported by the Joint Commission.^3–5 The process of screening requires venipuncture and diagnostic studies, conceivably leading to higher costs, prolonged ED length of stay, and increased exposure to potentially toxic medications and invasive procedures. Given recent emphasis on cost-effective, outcomes-based healthcare in an increasingly financially stressed climate,⁶ there is an exigent need to quantify objectively the national epidemiology of a common presentation: patients presenting with SIRS to the ED.

Previous epidemiological studies have focused on a numerator of sepsis or severe sepsis without studying the denominator of those who present with undifferentiated SIRS.^7–13 Other studies of SIRS have described its presentation in admitted patients only or reported results from a single site.^14–17 As a result, there is no comprehensive understanding of the undifferentiated presentation of patients with SIRS in the ED setting. Without this knowledge, the impact of using SIRS-based sepsis screening on the healthcare system cannot be ascertained. These limitations complicate the practical application of SIRS for the front-line provider and confound the implications of a SIRS-based sepsis screening for our healthcare system.

As clinicians and researchers work to refine the approach to the early identification of sepsis, it is important to understand the performance of the primary entry criterion – SIRS. We conducted a study of a large, nationally represented sample to clarify the epidemiology of SIRS in the ED and subsequent category of illness.

METHODS

Study design and setting

We studied ED visits made by adults, 18 years of age or greater, from 2007 to 2010 in the National Hospital Ambulatory Medical Care Survey (NHAMCS). NHAMCS is a national, representative, probability sample of visits to United States EDs.^18,19 The multi-staged NHAMCS sample design is composed of 3 stages for the ED component: (1) 112 geographic primary sampling units; (2) approximately 480 hospitals within primary sampling units; and (3) patient visits within emergency service areas. Sample hospitals are randomly assigned to 16 panels that rotate across 13 4-week reporting periods throughout the year. Hospital staff or Census Bureau field representatives complete a patient record form for each sampled visit according to information obtained from the medical record. The data collected include information on patient demographics, reasons for visit, vital signs, cause(s) of injury, diagnoses rendered, diagnostic tests ordered, procedures provided, medications prescribed, providers consulted, and disposition, including hospital discharge information if admitted. As part of the quality assurance procedure, a 10% quality control sample of patient record forms is independently keyed and coded. Error rates typically range between 0.3% and 0.9% for various survey items.²⁰ This study was approved by the institutional review board, as the data are deidentified and publicly available.

The study time frame was chosen because 2007 was the first year to include all vital signs at triage; 2010 is the most recent year for which data were available. NHAMCS records only whether a test was ordered, not its result.

Measurements

To satisfy the white blood cell count (WBC) criterion in SIRS, we developed a novel approach for our estimates. We used a Bayesian logical framework²¹,²² of prior probability distributions for WBC result to make minimum, moderate, and maximum estimates for SIRS.²³

For the minimum estimate, we required that the patient present with at least 2 of the following criteria: abnormal temperature (>38 °C or <36 °C), pulse (>90 beats/min), or respiratory rate (>20 respirations/min). The minimum estimate assumes that a WBC, if drawn, would have resulted as a “negative” test for SIRS. This corresponds to a strict prior probability for the WBC result. For the moderate estimate, even-numbered observations with a WBC ordered were assigned a “positive” test for SIRS (i.e. fulfilling the WBC criterion) and odd-numbered observations with a WBC ordered were assigned a “negative” test for SIRS (i.e. not fulfilling the WBC criterion). This corresponds to a uniform prior distribution. For the maximum estimate, we assumed that all WBCs ordered would fulfill the SIRS criterion. This corresponds to a lenient prior probability for the WBC result. The goal of this gradated approach was to offer Bayesian-style limit estimates akin to credible intervals; that is, the moderate estimate takes equipoise in terms of WBC count and is bound by strict (minimum) and lenient (maximum) “modified credible intervals” (mCI) that encompass the extreme possibilities for WBC results in the study sample.^24–26

Analysis

We sorted cases that qualified for SIRS into the following categories: infection, pancreatitis, ischemia, trauma, hemorrhage (atraumatic), toxin, anaphylaxis, and other. Previous work used a few key summary diagnoses for definition of SIRS or SIRS-related conditions.^27–30 We reviewed the entirety of the disease lexicon in the International Statistical Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and included every qualifying diagnosis in each category of illness (see online Appendices A-G). We did this to capture the SIRS-associated diagnosis with as much granularity as possible. NHAMCS allows up to 3 diagnoses; if a case had any qualifying diagnosis, it was included in that category. In the rare presence of more than one category (<0.5% of SIRS cases based on the moderate estimate), the first listed qualifying category was selected. Adults presenting to triage within 72 hours of a previous visit or those taking β–blocker or calcium-channel blocker medications were excluded from the analysis.

To accommodate the complex survey design of NHAMCS, we invoked the procedures PROC SURVEYMEANS for continuous data and PROC SURVEYFREQ for categorical data using SAS software, version 9.3 (SAS Institute Inc., Cary, NC, USA; 2011). We used the masked sample design variables CSTRATM and CPSUM as well as patient weights to generate population estimates. The sampling weights have been adjusted by the National Center for Health Statistics (NCHS) for survey non-response within time of year, geographic region, and urban/rural and ownership designations, yielding an unbiased national estimate of ED visit occurrences, percentages, and characteristics.²⁰ We report medians and inter-quartile ranges where appropriate. We tested differences in medians with the non-parametric Wilcoxon rank-sum procedure and differences in proportions with the Rao-Scott chi-square test, which accounts for the hierarchical survey design.³¹ Further, we complied with the minimum sample size and relative standard error requirements for reliable estimates, as recommended by the NCHS.^32,33 Reported statistics are for population-based estimates, rather than raw patient encounters, as recommended by the Centers for Disease Control and Prevention.²⁰

RESULTS

We surveyed 103,701 raw patient encounters corresponding to a population-based estimate of 372,844,465 visits over the 4r-year period (Table 1). The incidence of SIRS in adults 18 years of age and older presenting to the ED was at least 9.7% (95% CI: 9.2 to 10.2%), moderately 17.8% (95% CI: 17.2 to 18.4%), and at most 26% (95% CI: 25.1 to 26.8). Taking the minimum and maximum estimates as modified credible intervals, we report an overall moderate estimate of the incidence of adult SIRS presenting to the ED to be 17.8% (95% mCI: 9.7 to 26%). This yields a national moderate estimate of approximately 16.6 million (95% mCI: 9.0 to 24.2 million) visits per year made by adults presenting to the ED with SIRS criteria.

Table 1. Minimum, moderate, and maximum estimates of systemic inflammatory response syndrome (SIR) in adults presenting to United States emergency departments, 2007–2010; N=372,844,465 visits.

Using the moderate estimate, adults with and without SIRS had similar demographic characteristics, but were more likely to arrive by EMS (29.5% versus 17.1%, p<0.0001) and be categorized as emergent in triage (17.7% versus 9.9%, p<0.001) ^{(Table 2)}. Chronic conditions, such as diabetes, cerebrovascular disease, and congestive heart failure, were more common in SIRS patients (23.7% versus 14.8%, p<0.0001). Length of ED visit was longer in SIRS patients (210 minutes versus 153 minutes, p<0.0001).

Table 2. Characteristics of adults presenting to United States emergency departments with and without systemic inflammatory response syndrome (SIR) based on moderate estimate, 2007–2010; N=372,844,465 visits.

 

Patients with SIRS were more likely to be admitted (31.5% versus 12.5%, p<0.0001) and to be sent to a critical care unit or monitored bed (11.2% versus 3.7%, p<0.0001). Nonetheless, 68.6% of SIRS-positive patients were discharged home.

For those admitted, the median length of hospital stay for SIRS patients was one half-day longer than for non-SIRS patients (3.8 days versus 3.3 days, p<0.0001). Twenty-eight-day in-hospital mortality was higher for patients hospitalized with SIRS (4.6% versus 1.8%, p<0.0001).

Proportions of SIRS categories are reported based on the moderate estimate, as they were stable and consistent in all estimates (minimum, moderate, and maximum distributions). In patients presenting to the ED with SIRS, infection accounted for only 26% of subsequent diagnoses (Figure). Traumatic causes of SIRS accounted for 10% of presentations; other traditional categories of SIRS were rare (≤1%). The majority of diagnoses (56%) did not fall into any of the previously established categories for SIRS.

Figure. Adults with SIRS and subsequent category of illness based on moderate estimate presenting to United States emergency departments, 2007–2010; N=66,388,686 visits.

These SIRS-positive “other” diagnoses were further analyzed and found to populate the following ICD-9-CM domains: “Mental Disorders” (13.8%), “Diseases of the Respiratory System” (11.9%), “Diseases of the Digestive System” (9.4%), “Endocrine, Nutritional and Metabolic Diseases, and Immunity Disorders” (7.2%), “Diseases of the Sense Organs” (5.0%), “Symptoms, Signs, and Ill-defined Conditions” (3.6%), and “Diseases of the Genitourinary System” (3.5%). Neoplasm and disorders of the musculoskeletal, dermatologic, circulatory, and nervous systems together comprised the remaining 1.6% of SIRS cases.

DISCUSSION

We used a national representative survey of United States EDs to estimate the incidence of SIRS and subsequent category of illness, using a Bayesian approach for estimate limits. Previous studies focused on sepsis, relying on a handful of aggregate codes such as “bacteremia” (790.7) or “septicemia” (038).^7,28–30,34 Those studies did not use objective markers of systemic inflammation and relied on limited coding methods, an approach with potential bias. To enhance the accuracy of the estimates of SIRS-associated diagnoses, we used a detailed list of ICD-9-CM codes and vital signs measured at triage to infer an objective estimate of SIRS nationally. With this information, we can determine more fully the epidemiology of SIRS among adult ED patients nationally and the potential implications of a SIRS based severe sepsis screening program.

We found the presence of SIRS to be common in the emergency setting, with 16.6 million presentations per year, or approximately 17.8% of all adult ED visits. SIRS represented a heterogeneous group, with only about a quarter associated with infection. In addition, the majority of SIRS-positive patients were discharged home. This is consistent with previous authors’ findings of lack of specificity of SIRS and concerns regarding associated increased utilization of resources.^35,36 Shapiro et al²⁹ found in a single-center study that although a combination of clinical and laboratory parameters were predictive of short- and long-term mortality, SIRS itself offered no additional prognostic value.

In the current analysis, we found that patients with SIRS are more likely to be admitted, to be admitted to a higher level of care, and to have a slightly longer hospital length of stay. Additionally, patients hospitalized with SIRS had a higher 28-day mortality rate than those without SIRS. The significance of this finding is limited in that we were unable to adjust for illness severity. However, the finding that SIRS patients were more likely to be hospitalized and admitted to an intensive care unit setting demonstrates that SIRS may have some utility in the risk stratification of adult patients at ED triage.

The lack of specificity of SIRS for an infectious process limits its utility for infectious screening in the ED. Given the emphasis on SIRS in consensus guidelines for severe sepsis, clinicians may be compelled to pursue an infectious etiology in “SIRS-positive” patients, in what is clearly a heterogeneous population. Consensus recommendations that require screening millions of undifferentiated patients annually for severe sepsis may add unnecessarily to healthcare costs, length of ED stay, and exposure of additional patients to unnecessary antibiotics or invasive testing. Our findings suggest that a more accurate tool for sepsis screening is needed.

As the U.S. experiences a declining number of EDs and a concomitant rise in ED utilization,^37–40 triage and screening for occult disease become ever more important. For this reason evidence-based tools such as the Emergency Severity Index (ESI)⁴¹ and the Canadian Triage and Acuity Scale (CTAS)⁴² have been developed to prioritize patients. With both tools, the triage provider uses a combination of objective parameters and clinical judgment to classify the patient. SIRS, perhaps fuelled by published clinical guidelines, has been used increasingly as an up-front (i.e. at triage) pre-emptor to clinician judgment, with potential impacts on resource utilization.^43–45 As institutions adapt to the changing healthcare landscape, SIRS criteria may benefit from the success of validated screening tools, such as the ESI and CTAS with a modification that requires clinician input⁴⁶ prior to acting on a “SIRS alert,” and initializing a cascade of institutional processes.

The finding that 56% of adults with SIRS had miscellaneous other diagnoses emphasizes the lack of specificity for any particular disease condition. SIRS may have value as an early screening test (fairly sensitive) but not as a diagnostic test (poorly specific). In the proper clinical context, SIRS identifies a population with a somewhat higher risk of hospitalization, need for critical care, and short-term mortality. However, the lack of specificity for infection and the limited prognostic utility of SIRS imply that better early warning systems for sepsis are needed.

LIMITATIONS

This report has several important limitations. NHAMCS episodes represent ED visits, not necessarily unique patients. While it is possible that an individual may be represented more than once, the robust sampling procedures used by NHAMCS in addition to our excluding patients recently seen at the presenting hospital make this occurrence unlikely.

There is significant endogeneity inherent in the classification of patients at triage, their diagnosis, and their disposition. That is, the same parameters that qualify patients for SIRS will also affect their triage category, which in turn affects work-up and final diagnosis. In addition, disposition may be driven not only by the results of history, physical examination, and supplemental testing, but also by the patient’s initial presentation, including SIRS parameters. Nonetheless, triage or “first recorded” vital signs have been used successfully as entry criteria in previous SIRS and sepsis research.^29,30,47,48

The vital signs reported in NHAMCS are limited to those measured at triage. Accuracy of vital signs at triage may vary, and this one-time snapshot precludes trend analysis over the course of the ED stay. However, since international guidelines call for sepsis screening as early as possible in adults, many institutions have moved toward screening protocols at triage or as early as possible in the ED stay.^3,4,49,50 As initial vital signs have the most important role in screening programs for critical illness, an analysis of SIRS based on these variables in real-life conditions is relevant.

For this analysis, we assumed that patients who did not have a WBC ordered did not have an elevated WBC. This assumption could potentially slightly underestimate the true incidence of SIRS. However, we also assumed for the moderate estimate that 50% of patients with a WBC ordered had an abnormal result. This assumption likely overestimated the incidence of SIRS; we felt that on balance this approach was appropriate in the context of a screening test. Unfortunately, without WBC results on all ED visits in the NHAMCS database, we cannot determine the actual directly measured incidence of SIRS, but feel that our construct provides a moderate, reasonable estimate of incidence for the adult ED population. The minimum estimate, based only on vital signs, gives an objective baseline estimate against which the others (moderate, maximum) may be considered.

Finally, the use of ICD-9-CM codes may be problematic in reflecting the true clinical diagnosis.^51,52 Previous studies relied on a short list of (mostly sepsis-related) codes.^7,28–30,34 We sought to mitigate this limitation with a detailed categorization of the current ICD-9-CM. We also expanded on the previous epidemiologic studies of sepsis, which relied solely on coding data, by incorporating documented vital signs to improve on the estimation of the epidemiology of SIRS.

CONCLUSION

The presence of at least 2 SIRS criteria is common among adult ED patients. Infectious etiologies make up only a quarter of adult SIRS cases. SIRS may be sensitive for sepsis but it is very non-specific. SIRS may be more useful if modified by clinician judgment when used as a screening test in the rapid identification and assessment of patients with the potential for sepsis.

Footnotes

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

Supervising Section Editor: Christopher Kahn, MD, MPH

Address for Correspondence: Timothy Horeczko, MD, MSCR. Harbor-UCLA Medical Center, Department of Emergency Medicine, 1000 W. Carson St, Box 21, Torrance, CA 90509. Email: thoreczko@emedharbor.edu. 5 / 2014; 15:329 – 336

Submission history: Revision received April 26, 2013; Submitted July 24, 2013; Accepted September 30, 2013

Full text available through open access at http://escholarship.org/uc/uciem_westjem

REFERENCES

1. Bone RC, Balk RA, Cerra FB, et al. Definitions for Sepsis and Organ Failure and Guidelines for the Use of Innovative Therapies in Sepsis. Chest. 1992; 101:1644-1655.

2. Bone RC. Towards an Epidemiology and Natural History of SIRS (Systemic Inflammatory Response Syndrome). JAMA. 1992; 268:3452-3455

3. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med. 2010; 38:367-374.

4. Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente’s performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011; 37:483-93.

5. Joint Commission on Accreditation of Healthcare Organizations. Online Bulletin: Center launches project to reduce sepsis mortality. Joint Commision Online. June 27, 2012. http://www.jointcommission.org/ (Accessed March 20, 2013).

6. Mountford J, Davie C. Toward an outcomes-based health care system: a view from the United Kingdom. JAMA. 2010;304(21):2407-2408.

7. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303-1310.

8. Osborn TM, Tracy JK, Dunne JR, et al. Epidemiology of sepsis in patients with traumatic injury. Crit Care Med. 2004;32(11):2234-2240.

9. Angus DC, Wax RS. Epidemiology of sepsis: an update. Crit Care Med. 2001;29(7 Suppl):S109-116.

10. Brun-Buisson C, Meshaka P, Pinton P, et al. EPISEPSIS: a reappraisal of the epidemiology and outcome of severe sepsis in French intensive care units. Intensive Care Med. 2004; 30(4):580-588.

11. Engel C, Brunkhorst FM, Bone HG, et al. Epidemiology of sepsis in Germany: results from a national prospective multicenter study. Intensive Care Med. 2007;33(4):606-618.

12. Blanco J, Muriel-Bombin A, Sagredo V, et al. Incidence, organ dysfunction and mortality in severe sepsis: a Spanish multicentre study. Crit Care. 2008;12(6):R158.

13. Gray A, Ward K, Lees F, et al. The epidemiology of adults with severe sepsis and septic shock in Scottish emergency departments. Emerg Med J. 2013;30(5):397-401.

14. Sibbald WJ, Doig G, Inman KJ. Sepsis, SIRS and infection. Intensive Care Med. 1995;21(4):299-301.

15. Rangel-Frausto MS, Pittet D, Costigan M, et al. The natural history of the systemic inflammatory response syndrome (SIRS). A prospective study. JAMA. 1995;273(2):117-123.

16. Jones GR, Lowes JL. The systemic inflammatory response syndrome as a predictor of bacteraemia and outcome from sepsis. Q J Med. 1996;89:515-522.

17. Danai PA, Sinha S, Moss M Seasonal variation in the epidemiology of sepsis. Crit Care Med. 2007;35(2):410-415.

18. National Center for Health Statistics. NHAMCS Scope and Sample Design. Centers for Disease Control and Prevention. Atlanta, GA, 2010. http://www.cdc.gov/nchs/ahcd/ahcd_questionnaires.htm (Accessed March 20, 2013).

19. National Center for Health Statistics. NHAMCS Data Collection and Processing. Centers for Disease Control and Prevention. Atlanta, GA, 2010. http://www.cdc.gov/nchs/ahcd/ahcd_questionnaires.htm (Accessed March 20, 2013).

20. McCaig LF, Burt CW. Understanding and interpreting the National Hospital Ambulatory Medical Care Survey: key questions and answers. Ann Emerg Med. 2012;60(6):716-721 e1.

21. Genest C, Zidek JV. Combining Probability Distributions: A Crtique and an Annotated Bibliography. Stat Sci. 1986;1(1):114-148.

22. Poole D, Raftery AE. Inference for Deterministic Simulation Models: The Bayesian Melding Approach. J Am Statist Assoc. 2000;95(452):1244-1255.

23. Horeczko T, Green JP. The Emergency Department Presentation of the Pediatric Systemic Inflammatory Response Syndrome (SIRS). Pediatr Emerg Care. 2013;29(11):1153-1158.

24. Kadane JB. An application of robust Bayesian analysis to a medical experiment. J Stat Plan Inference. 1994;40:221-232.

25. Zhang T. A modification for Bayesian Credible Intervals. Commun Stat-Theor M. 2006;35:1703-1711.

26. Graham P, Moran J. Robust meta-analytic conclusions mandate the provision of prediction intervals in meta-analysis summaries. J Clin Epidemiol. 2012;65:503-510.

27. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303-1310.

28. Martin GS, Mannino DM, Eaton SE, et al. The Epidemiology of Sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546-1554.

29. Shapiro NI, Howell MD, Bates DW, et al. The association of sepsis syndrome and organ dysfunction with mortality in emergency department patients with suspected infection. Ann Emerg Med. 2006; 48:583-590.

30. Wang HE, Shapiro NI, Angus DC. National estimates of severe sepsis in United States emergency departments. Crit Care Med. 2007;35(8):1928-1936.

31. Rao JNK, Scott AJ. The analysis of categorical data from complex sample surveys: chi-squared tests for goodness-of-fit and independence in two-way tables. J Am Stat Assoc. 1981;76:221-230.

32. National Center for Health Statistics. NHAMCS Estimation Procedures. Centers for Disease Control and Prevention. Atlanta, GA, 2010. http://www.cdc.gov/nchs/ahcd/ahcd_questionnaires.htm (Accessed March 20, 2013).

33. National Center for Health Statistics. NHAMCS Reliability of Estimates. Centers for Disease Control and Prevention. Atlanta, GA, 2010. http://www.cdc.gov/nchs/ahcd/ahcd_questionnaires.htm (Accessed March 20, 2013).

34. Strehlow MC, Emond SD, Shapiro NI, et al. National study of emergency department visits for sepsis, 1992 to 2001. Ann Emerg Med. 2006; 48:326-331.

35. Dellinger RP, Bone RC. To SIRS with love. Crit Care Med. 1998;26:178-179.

36. Talan DA. Dear SIRS: it’s time to return to sepsis as we have known it. Ann Emerg Med. 2006;48:591-592.

37. Committee on the Future of Emergency Care in the U.S. Health System. The future of emergency care in the United States health system. Ann Emerg Med. 2006;48:115-120.

38. National Center for Health Statistics. Health, United States, 2011: With Special Feature on Socioeconomic Status and Health. National Center for Health Statistics. Hyattsville, MD, 2012. www.cdc.gov/nchs/data/hus/hus11.pdf (Accessed March 20, 2013).

39. Niska R, Bhuiya F, Xu J. National Hospital Ambulatory Medical Care Survey: 2007 emergency department summary. Natl Health Stat Report. 2010; 26:1-31.

40. Hsia RY, Kellermann AL, Shen YC. Factors associated with closures of emergency departments in the United States. JAMA. 2011;305(19):1978-1985.

41. Gilboy N, Tanabe P, Travers DA, et al. Emergency Severity Index, Version 4: Implementation Handbook. AHRQ Publication No. 12-0014. Agency for Healthcare Research and Quality, Rockville, MD. December, 2011. http://www.ahrq.gov/research/esi/ (Accessed March 20, 2013).

42. Bullard MJ, Unger B, Spence J, et al. Revisions to the Canadian Emergency Department Triage and Acuity. CJEM. 2008;10(2):136-142.

43. Bates DW, Yu DT, Black E, et al. Resource utilization among patients with sepsis syndrome. Infect Control Hosp Epidemiol. 2003;24(1):62-70.

44. Yu DT, Black E, Sands KE, et al. Severe sepsis: variation in resource and therapeutic modality use among academic centers. Crit Care. 2003;7(3):R24-34.

45. Hartman ME, Angus DC. Variation in sepsis care: a wake-up call. Crit Care. 2003;7(3):211-213.

46. Van der Vegt AE, Holman M, Ter Maaten JC. The value of the clinical impression in recognizing and treating sepsis patients in the emergency department. Eur J Emerg Med. 2012;19(6):373-378.

47. Shapiro NI, Wolfe RE, Moore RB, et al. Mortality in Emergency Department Sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med. 2003;31(3):670-675.

48. Sankoff JD, Goyal M, Gaieski DF, et al. Validation of the Mortality in Emergency Department Sepsis (MEDS) score in patients with the systemic inflammatory response syndrome (SIRS). Crit Care Med. 2008;36(2):421-426.

49. Nelson JL, Smith BL, Jared JD, et al. Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Ann Emerg Med. 2011;57(5):500-504.

50. Berger T, Green JP, Horeczko T, et al. The Shock Index and Early Recognition of Sepsis in the Emergency Department: A Pilot Study. West J Emerg Med. 2013;14(2):168-174.

51. Campbell SE, Campbell MK, Grimshaw JM, et al. A systematic review of discharge coding accuracy. J Public Health Med. 2001;23(3):205-211.

52. O’Malley KJ, Cook KF, Price MD, et al. Measuring diagnoses: ICD code accuracy. Health Serv Res. 2005;40(5 Pt 2):1620-1639.