Predictive Value of Initial Triage Vital Signs for Critically Ill Older Adults

Author Affiliation
Michael A. LaMantia, MD, MPH  Indiana University Center for Aging Research and Regenstrief Institute, Indianapolis, Indiana
Paul W. Stewart, PhD  University of North Carolina, Department of Biostatistics, Chapel Hill, North Carolina
Timothy F. Platts-Mills, MD  University of North Carolina, Department of Emergency Medicine, Chapel Hill, North Carolina
Kevin J. Biese, MD, MAT  University of North Carolina, Department of Emergency Medicine, Chapel Hill, North Carolina
Cory Forbach, MD  University of North Carolina, Department of Emergency Medicine, Chapel Hill, North Carolina
Ezequiel Zamora, MD  Watauga Medical Center, Boone, North Carolina
Brenda K. McCall, RN  University of North Carolina Hospitals, Chapel Hill, North Carolina
Frances S. Shofer, PhD  University of Pennsylvania, Department of Emergency Medicine, Philadelphia, Pennsylvania
Charles B. Cairns, MD  University of North Carolina, Department of Emergency Medicine, Chapel Hill, North Carolina
Jan Busby-Whitehead, MD  University of North Carolina, Division of Geriatrics and Center for Aging and Health, Chapel Hill, North Carolina
John S. Kizer, MD  University of North Carolina, Division of Geriatrics and Center for Aging and Health, Chapel Hill, North Carolina

Introduction
Methods
Results
Discussion
Limitations
Conclusion

Abstract

Introduction:

Triage of patients is critical to patient safety, yet no clear information exists as to the utility of initial vital signs in identifying critically ill older emergency department (ED) patients. The objective of this study is to evaluate a set of initial vital sign thresholds as predictors of severe illness and injury among older adults presenting to the ED.

Methods:

We reviewed all visits by patients aged 75 and older seen during 2007 at an academic ED serving a large community of older adults. Patients’ charts were abstracted for demographic and clinical information including vital signs, via automated electronic methods. We used bivariate analysis to investigate the relationship between vital sign abnormalities and severe illness or injury, defined as intensive care unit (ICU) admission or ED death. In addition, we calculated likelihood ratios for normal and abnormal vital signs in predicting severe illness or injury.

Results:

4,873 visits by patients aged 75 and above were made to the ED during 2007, and of these 3,848 had a complete set of triage vital signs. For these elderly patients, the sensitivity and specificity of an abnormal vital sign taken at triage for predicting death or admission to an ICU were 73% (66,81) and 50% (48,52) respectively (positive likelihood ratio 1.47 (1.30,1.60); negative likelihood ratio 0.54 (0.30,0.60).

Conclusion:

Emergency provider assessment and triage scores that rely primarily on initial vital signs are likely to miss a substantial portion of critically ill older adults.

INTRODUCTION

Accurate triage of older adults in the emergency department (ED) is important both because of the high frequency of life-threatening illness in this population and the attendant need for resource-intensive care provided to these patients.1,2 Existing evidence suggests that elderly patients are frequently under-triaged,35 that serious medical conditions may go unrecognized at the time of their triage,6 and that the Emergency Severity Index (ESI) triage instrument may not be adequately calibrated for use in older adults.4 Under-triage of patients may result in failure to implement critically important care and may lead to an increase in adverse outcomes; conversely, over-triage may result in inappropriate resource use and the diversion of care from other patients in more need.5 Triage, itself, has been recognized as critical to patient safety and experts in the field of emergency medicine have called for the development of a research agenda for its systematic study,7 as well as identified high yield research opportunities for future investigation in this area.8

The clinical assessment of patients in the ED is based in part upon the measurement and interpretation of initial vital signs. Emergency providers and nurses may use vital signs as markers of the presence of severe illness, upgrading triage designations and directing more immediate attention to those patients with abnormal vital signs.9 For older patients in particular, the ability of vital signs to indicate the severity of a patient’s illness has not been conclusively demonstrated, as some studies have suggested that vital signs may not accurately reflect the severity of dehydration or pneumoniain elderly patients,10,11 while other work has suggested that blood pressure changes on standing may reliably indicate dehydration in older adults.12 To our knowledge, however, no research performed to date has evaluated the relationship between initial triage vital signs and the presence of severe disease among elderly patients presenting to the ED. Therefore, we examined among elderly adults the relationship between initial vital signs taken at the time of triage and the presence of severe illness or injury, defined as death in the ED or admission from the ED to an intensive care unit (ICU).

METHODS

Study Design

We conducted a retrospective study of consecutive ED visits by patients aged 75 and older during a 1-year period in order to evaluate the association between initial triage vital sign abnormalities and severe illness and injury. The selection of this age cohort was made so as to parallel the oldest age cohort analyzed in the National Hospital Ambulatory Medical Care Survey.13 The institutional review board of the University of North Carolina approved this study with a waiver of informed consent.

Study Setting and Population

The site of this study was the University of North Carolina Hospitals ED (Chapel Hill, NC), a tertiary-care, Level 1 trauma center serving rural and urban populations with 61,200 ED visits in 2007.

Selection of Subjects

The electronic charts of all patients aged 75 and above presenting to the study ED from January 1, 2007 to December 31, 2007 were selected for inclusion in the study.

Study Protocol and Measurement

We obtained data from the ED’s computerized patient record system (The T-System® EV, version 2.5, 2000–2005; T-System Inc., Dallas, TX) and Webcis, the electronic medical record system developed and used by the University of North Carolina Hospitals. To avoid manual chart review and possible disagreement among chart reviewers, data were abstracted electronically14 from the hospital medical record system to the study database for the patients’ race, insurance status, and vital signs while data were electronically abstracted from the ED record system to the study database for each patient’s age, sex, disposition (admitted to the hospital floor, admitted to the ICU, transferred, discharged, or died), ESI score and chief complaint. We defined severe illness or injury as either admission to an ICU or death in the ED because they are extreme and concrete outcomes that are unlikely to be biased in a retrospective review. Vital signs are taken at the time of triage by a nurse or if the patient is placed directly into a bed by that patient’s nurse. In view of the lack of a general consensus as to what constitutes a normal set of vital signs, we defined normal vital signs as follows: systolic blood pressure from 100 to 200 mm Hg, heart rate from 60 to 100 beats/minute, respiratory rate from 8 to 20 breaths/minute, temperature from 36 to 38 degrees Celsius, and oxygen saturation of 90% or above. These normal ranges were selected to mirror the upper bounds for heart rate and respiratory rate used in the ESI triage algorithm (version 4).15 The upper range for blood pressure reflects the boundary at which the Emergency Medicine Cardiac Research Group (EMCREG) has recommended beginning outpatient blood pressure lowering therapy in asymptomatic ED patients.16 We added additional basement parameters to these vital sign measurements and included boundaries for oxygen saturation (irrespective of supplemental oxygen provision) and temperature based on our clinical practice.

Data Analysis

In the primary analysis for elderly patients, we evaluated the predictive value of initial triage vital signs in terms of the sensitivity and specificity of the presence of any abnormal vital sign for predicting severe illness. Because some patients contributed multiple visits, the estimates of sensitivity, specificity, and other performance parameters were obtained by fitting statistical models appropriate for repeated binary measures; specifically, we relied on generalized logistic-linear models with subject-specific random effects. We performed supportive auxiliary analyses to evaluate the robustness of the primary results and conclusions to choice of statistical methods and to evaluate the impact of including patients with incomplete sets of vital signs. To evaluate whether missing vital signs had a potential effect on our results, we performed analyses for 5 groups of data each defined by the number of vital signs that were missing: 1) those visits in which all 5 vital signs were recorded (3848 visits), 2) those visits with 0 or 1 vital sign missing (4702 visits), 3) those with 0 to 2 vital signs missing (4771 visits), 4) those visits with 0–3 missing (4779 visits), and 5) those visits with 0–4 missing (4780 visits). In sensitivity analyses, we explored the impact of subjects who had multiple visits, in particular to evaluate whether patients with more visits might be more likely to have an incomplete set of vital signs. Finally, secondary analyses were performed to characterize the predictive value of each individual vital sign in elderly patients. All statistical computations were performed using SAS System Software (version 9.2, SAS Institute, Cary, NC).

RESULTS

During calendar year 2007, there were 4,873 visits by 3,079 patients aged 75 and above to the study site ED. Of these visits, 228 resulted in admission to an ICU while 16 patients died in the ED. Table 1 lists the characteristics of these visits according to outcome of interest, including demographic information, ESI triage score, and vital signs taken at triage. A complete set of initial vital signs including blood pressure, pulse, respiratory rate, temperature, and oxygen saturation were available for 3,848 visits (79.0%), although almost all patients had at least some vital signs recorded. Pulse was recorded for 4,771 visits (97.9%), complete blood pressure readings were available for 4,703 visits (96.5%), respiratory rate for 4748 visits (97.4%), temperature for 4455 visits (91.4%), and oxygen saturation for 4215 visits (86.5%). The primary results were not sensitive to incomplete recording of vital signs (Table 2).

table1
Table 1.
Demographic and presenting characteristics of patients aged 75 and above.
table2
Table 2.
Vital signs for prediction of death or intensive care unit admission in patients aged 75 and above.

The presence of any vital sign abnormality was associated with severe illness or injury in elderly patients (odds ratio [OR] 2.48, confidence interval [CI] 1.50–3.30). However, the sensitivity (%) and specificity (%) of any abnormal vital sign as a predictor of severe illness or injury were only 73 with CI (66,81) and 50 with CI (48,52) respectively (Table 2.), based on visits with complete vital sign data. Practically identical sensitivities and specificities, 72 with CI (65,79) and 53 with CI (51,55) respectively, were obtained for any vital sign abnormalities when less than 4 of the vital signs were missing. The estimate of serial correlation for repeated visits was small (r=0.14 to 0.24 depending on model and completeness of vital signs).

For the secondary analyses of the predictive value of individual vital signs, Table 3 lists the odds ratios for a given vital sign abnormality and its association with severe illness. The following vital sign abnormalities were associated with an increased risk of ED death or ICU admission: systolic blood pressure <100 mm Hg, heart rate >100 beats/minute, respiratory rate <8 breaths/minute or >20 breaths/minute, and oxygen saturation <90%. Low temperatures were associated with slightly increased odds of suffering a serious outcome; however, temperature abnormalities in general were not strongly predictive of serious injury or illness.

table3
Table 3.
Odds ratios for the association of abnormal vital signs with severe illness or injury.

DISCUSSION

Our results demonstrate that a substantial portion of elderly patients presenting to the ED with severe illness or injury have normal initial vital signs. Secondarily, we find that not all initial vital signs are equally useful for predicting severe illness or injury. For example, hypotension, tachycardia, hypoxia, and high or low respiratory rate were associated with increased risk of ED death or ICU admission, whereas other initial vital sign abnormalities were not. In total, however, the presence or absence of one or more abnormal initial vital sign, as defined in our study, is not a reliable marker for severe illness or injury in the elderly patient. The positive and negative likelihood ratios associated with any initial vital sign abnormality are not of sufficient magnitude to accurately predict severe illness or death in the ED.

A previous study of elderly patients in the ED of one academic center has suggested that the ESI triage score, which incorporates vital signs into its algorithm, accurately assesses the risk of elderly patients for hospitalization, resource use, and mortality.17 However, the ESI performed better at predicting hospitalization in this original study17 (AUC score 0.77) than it did when we examined it in our own academic setting (AUC score 0.68).18 Neither algorithm, however, provides sufficient accuracy to reliably predict individual patient outcomes. Indeed, work from our center has suggested that the ESI triage score may not be the best predictor of admission to the hospital from the ED and that an algorithm for prediction of admission in elderly patients depends on vital signs only to a small degree.18 Further investigation has also suggested that the ESI scoring system does not adequately identify those patients 75 years of age and older who are in need of an immediate life-saving intervention.4

Although our study is the first, to our knowledge, to examine the ability of abnormal initial vital signs taken at ED triage to predict severe illness or injury in the elderly, it adds to the substantial literature that has called into question the reliability or usefulness of vital sign measurements in clinical practice, either due to problems in the reproducibility of readings or due to their failure to predict dire clinical outcomes. Previous studies have challenged the ability of respiratory rate to predict respiratory failure 19,20 and highlighted the substantial dependency of respiratory rate measurement on provider technique.1921 Moreover, in the ED, considerable interobserver variability exists in the measurement of vital signs, with measurement by sequential observers differing as little as 10 to 15% for heart rate and as much as 35% for respiratory rate.22 Finally, beyond all of these valid concerns, it has been noted that changes in anatomy and physiology associated with aging may affect the expected physiological response to injury or illness in older adults.23 It is not known, however, at this time how initial vital signs perform as predictors of severe illness in younger cohorts of patients.

In sum, given the challenges of appropriately triaging older adults in the ED, the problems inherent in vital sign measurement, and the central role that triage vital signs play in the initial assessment of patients by emergency providers, our work suggests that there may be some value to the development of alternative indicators of life-threatening disease in older patients upon which more accurate triage can be based. While functional status has been shown to be associated with prognosis in some cancer treatments,24 these types of assessments are not easily performed in the busy ED environment given current resources and are probably not sufficient for identifying patients with life-threatening conditions. When treating older adults with certain diseases, condition-specific assessment scales have been developed that predict severity of illness, of which the Pneumonia Severity Index (PSI) is one prominent example;25 however emergency physicians frequently do not follow its recommendations for a variety of reasons.26

Future tools for geriatric triage may include biomarkers,2730 but the challenges of identifying a single biomarker or a panel of biomarkers to accurately predict grave outcomes in older adults presenting to the ED with undiagnosed illness are challenging.31 Therefore, it seems, at least for the present, prudent to consider redesigning the triage of elderly patients so that physicians will have earlier contact with these patients in the course of their ED visit, as has been suggested previously.3234 Though the effect of such a redesign upon mortality or morbidity in this population of older adults has yet to be evaluated, its further study is warranted.

LIMITATIONS

Our work was limited in that it was collected from one study site, which may constrain the generalizability of its results. It does provide, however, an analysis of all patients aged 75 and above who were seen at our ED in a complete calendar year.

In this study, we have directly investigated only the relationship between initial vital signs and subsequent dire outcomes. Our results relate to the potential under-triage of older adults only in so much that vital signs are an important component of many triage schemes and under-triage is a concern in the care of older ED patients.4

Our study relied on retrospective data, rather than data collected prospectively from a cohort. We based our analyses, however, on the hard endpoints of death and admission to an ICU, 2 outcomes that we believe are likely to be free of bias and that have been used previously in the trauma literature among older adults.35 For these reasons, we submit that it is unlikely that a prospective approach would yield substantially different results. The presence of a “Do Not Resuscitate” order (DNR) was not included in our analysis, as a DNR order in our hospital does not preclude the admission of a patient to our hospital’s ICU and only influences medical “efforts at cardiopulmonary resuscitation” and “does not affect other medically indicated” acute care.36 However, it should be acknowledged that the care of patients receiving palliative services may influence the life-saving care that is received in the ED.

While ED death and ICU admission are relatively uncommon events in our study, these are proximal outcomes with very little ascertainment bias that are more likely to be related to initial vital signs and severity of illness than other considered outcomes, such as death during hospitalization or admission to a telemetry bed. Further, the low prevalence of the outcomes in the study should not affect the performance characteristics of the test in question (vital signs), as these test characteristics will remain independent of disease prevalence in this case where disease status has been strictly dichotomized and is not believed subject to misclassification bias.37 The relatively low number of events of interest, however, will be reflected in the precision of our estimates, as seen in our reported confidence intervals and p-values. Finally, to avoid the bias that is inherent in manual chart abstraction, we abstracted numerical values of vital signs contained in the electronic medical record by computer.

The available retrospective computerized records did not provide information about the several potential reasons for incomplete recording of initial vital signs. We acknowledge the potential confounding of our analysis due to the lack of a complete set of vital signs in nearly 20% of our sample. However, our sensitivity analyses indicate that the predictive accuracy of an initial set of vital signs in predicting dire outcomes was robustly invariant to the number of missing vital signs. It is also unlikely that 100% of patients presenting to an ED will either receive or need a full set of vital signs. We assume that failure to record initial vital signs may occur by chance due to logistical or administrative errors unrelated to the patient’s condition. This cause of missing data does not induce any selection biases when visits with missing vital signs are omitted from statistical analysis. We conjecture that initial vital signs will tend to be missing for visits in which the patient is obviously extremely ill or perhaps obviously well. Omission of such visits from statistical analysis, due to missing initial vital signs, may indeed induce selection bias by under-representation of those clearly in need of urgent medical care. We note, however, that for the elderly patient who is not breathing, the triage conclusion is already obvious and use of vital signs for predicting whether the patient may be seriously ill would be pointless. In sum, we conjecture that the patients who are in a position to benefit from vital sign triage will tend to have complete initial vital sign data. It would follow that the analyses using only the visits with complete initial vital sign data would be unbiased for making inferences about the visits of interest.

Our results are specific to our choices for what constitutes an abnormal initial vital sign. As there is little literature from which to base the assignation of “cut-offs” for individual vital signs, especially in older patients, we used the upper boundaries for heart rate and respiratory rates used with the ESI scoring (Version 4)15 and chose other boundaries for other initial vital signs based upon the medical literature for hypertension, temperature, and oxygen saturation in healthy adults and our clinical practice. It is possible that an alternate choice for values of abnormal initial vital signs would have yielded slightly different numerical results in our analysis; however, more lenient “cut-off” values are likely to increase the sensitivity at the expense of specificity. Our analyses also do not account for patients’ medications, supplemental oxygen, or the presence of pacemakers that may influence patients’ vital signs. Further, we do not stratify our results by the patient’s chief complaint or presenting illness, as this was not a pre-specified analysis agreed upon by our research group and the low number of outcomes of interest precludes this type of sub-group analysis.

Finally, our purpose in this work was to examine the predictive value of a reasonable set of initial vital signs criteria and not to develop the most useful set of vital sign parameters to assess clinical outcomes, since regression analysis has indicated that vital signs used even as continuous variables by themselves are poorly predictive of hospitalization in elderly adults.18 Further, although we recognize that clinicians may have multiple sets of vital signs as well as other clinical data available to them when they assess a patient, emergency providers and nurses frequently use initial triage vital signs to determine the initial resources provided to a patient9 early on in their ED course.

CONCLUSION

Our work demonstrates that initial vital sign abnormalities, while associated with increased risks of severe illness or injury, are neither sensitive nor specific for these outcomes in elderly patients presenting to the ED and do not produce positive or negative likelihood ratios of sufficient magnitude to shape clinical decision making. Thus, triage systems and clinical assessments that rely heavily upon initial vital sign abnormalities for the evaluation of disease severity are likely to be incorrect in many cases. As a result of the high rates of cognitive impairment3842 and incomplete information in older adults,4346 the potential for adverse outcome due to under-triage is probably greatest in this population. Further studies are needed to discover if modifications to the definitions of normal vital signs or the use of alternative indicators of disease severity can substantially strengthen the initial ED triage and clinical assessment of this vulnerable population.

Footnotes

Address for Correspondence: Michael A. LaMantia, MD, MPH. 410 West 10th Street, Suite 2000, Indianapolis, IN 46032. Email: malamant@iupui.edu.

Supervising Section Editor: Teresita M. Hogan, MD
Full text available through open access at http://escholarship.org/uc/uciem_westjem

Submission history: Revision received September 13, 2012; Submitted April 24, 2013; Accepted May 21, 2013

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

REFERENCES

1 Aminzadeh F, Dalziel WB Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med. 2002; 39:238-247

2 Singal BM, Hedges JR, Rousseau EW Geriatric patient emergency visits. Part I: Comparison of visits by geriatric and younger patients. Ann Emerg Med. 1992; 21:802-807

3 Phillips S, Rond PC, Kelly SM The failure of triage criteria to identify geriatric patients with trauma: results from the Florida Trauma Triage Study. J Trauma. 1996; 40:278-283

4 Platts-Mills TF, Travers D, Biese K Accuracy of the Emergency Severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med. 2010; 17:238-243

5 Scheetz LJ Trends in the accuracy of older person trauma triage from 2004 to 2008. Prehosp Emerg Care. 2011; 15:83-87

6 Rutschmann OT, Chevalley T, Zumwald C Pitfalls in the emergency department triage of frail elderly patients without specific complaints. Swiss Med Wkly. 2005; 135:145-150

7 Cooper RJ Emergency department triage: why we need a research agenda. Ann Emerg Med. 2004; 44:524-526

8 Carpenter CR, Gerson LW Geriatric emergency medicine. A Supplement to New Frontiers in Geriatrics Research: An Agenda for Surgical and Related Medical Specialties. 2008; :45-71

9 Cooper RJ, Schriger DL, Flaherty HL Effect of vital signs on triage decisions. Ann Emerg Med. 2002; 39:223-232

10 Gross CR, Lindquist RD, Woolley AC Clinical indicators of dehydration severity in elderly patients. J Emerg Med. 1992; 10:267-274

11 Lim WS, Macfarlane JT Defining prognostic factors in the elderly with community acquired pneumonia: a case controlled study of patients aged > or = 75 yrs. Eur Respir J. 2001; 17:200-205

12 Vivanti A, Harvey K, Ash S Clinical assessment of dehydration in older people admitted to hospital: what are the strongest indicators?. Arch Gerontol Geriatr. 2008; 47:340-355

13 Nawar EW, Niska RW, Xu J National Hospital Ambulatory Medical Care Survey: 2005 emergency department summary. Adv Data. 2007; :1-32

14 Biese KJ, Forbach CR, Platts-Mills TF Computer facilitated chart review for studying low frequency emergency department interventions. [abstract]. Acad Emerg Med. 2009; 16:1

15 Gilboy N TP, Travers DA, Rosenau AM Emergency Severity Index, Version 4: Implementation Handbook. . ;

16 Slovis CM, Reddi AS Increased blood pressure without evidence of acute end organ damage. Ann Emerg Med. 2008; 51:S7-9

17 Baumann MR, Strout TD Triage of geriatric patients in the emergency department: validity and survival with the Emergency Severity Index. Ann Emerg Med. 2007; 49:234-240

18 LaMantia MA, Platts-Mills TF, Biese K Predicting hospital admission and returns to the emergency department for elderly patients. Acad Emerg Med. 2010; 17:252-259

19 Hewson P, Humphries S, Roberton D Markers of serious illness in infants under 6 months old presenting to a children’s hospital. Arch Dis Child. 1990; 65:750

20 Hooker EA, O’Brien DJ, Danzl DF Respiratory rates in emergency department patients. J Emerg Med. 1989; 7:129-132

21 Simoes EA, Roark R, Berman S Respiratory rate: measurement of variability over time and accuracy at different counting periods. Arch Dis Child. 1991; 66:1199-1203

22 Edmonds ZV, Mower WR, Lovato LM The reliability of vital sign measurements. Ann Emerg Med. 2002; 39:233-237

23 Scheetz LJ Life-Threatening Injuries in Older Adults. AACN Advanced Critical Care. 2011; 22:128

24 Maione P, Perrone F, Gallo C Pretreatment quality of life and functional status assessment significantly predict survival of elderly patients with advanced non-small-cell lung cancer receiving chemotherapy: a prognostic analysis of the multicenter Italian lung cancer in the elderly study. J Clin Oncol. 2005; 23:6865-6872

25 Fine MJ, Auble TE, Yealy DM A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997; 336:243-250

26 Aujesky D, McCausland JB, Whittle Reasons why emergency department providers do not rely on the pneumonia severity index to determine the initial site of treatment for patients with pneumonia. Clin Infect Dis. 2009; 49:e100-108

27 Khan BA, Zawahiri M, Campbell NL Biomarkers for delirium-a review. J Am Geriatr Soc. 2011; 59:S256-261

28 Schuetz P, Christ-Crain M, Thomann R Effect of procalcitonin-based guidelines vs standard guidelines on antibiotic use in lower respiratory tract infections: the ProHOSP randomized controlled trial. JAMA. 2009; 302:1059-1066

29 Antman EM, Tanasijevic MJ, Thompson B Cardiac-specific troponin I levels to predict the risk of mortality in patients with acute coronary syndromes. N Engl J Med. 1996; 335:1342-1349

30 Hamm CW, Goldmann BU, Heeschen C Emergency room triage of patients with acute chest pain by means of rapid testing for cardiac troponin T or troponin I. N Engl J Med. 1997; 337:1648-1653

31 Buyse M, Sargent DJ, Grothey A Biomarkers and surrogate end points–the challenge of statistical validation. Nat Rev Clin Oncol. 2010; 7:309-317

32 Wiler JL, Gentle C, Halfpenny JM Optimizing emergency department front-end operations. Ann Emerg Med. 2010; 55:142-160

33 Russ S, Jones I, Aronsky D Placing physician orders at triage: the effect on length of stay. Ann Emerg Med. 2010; 56:27-33

34 Han JH, France DJ, Levin SR The effect of physician triage on emergency department length of stay. J Emerg Med. 2010; 39:227-233

35 Demetriades D, Sava J, Alo K Old age as a criterion for trauma team activation. J Trauma. 2001; 51:754-756

36 Do Not Resuscitate (DNR) & Medical Orders for Scope of Treatment (MOST) Forms. 2011;http://www.ncdhhs.gov/dhsr/EMS/dnrmost.html. Last accessed November 23, 2011.

37 Brenner H, Gefeller O Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997; 16:981-991

38 Hustey FM, Meldon SW The prevalence and documentation of impaired mental status in elderly emergency department patients. Ann Emerg Med. 2002; 39:248-253

39 Lewis LM, Miller DK, Morley JE Unrecognized delirium in ED geriatric patients. Am J Emerg Med. 1995; 13:142-145

40 Han JH, Morandi A, Ely EW Delirium in the nursing home patients seen in the emergency department. J Am Geriatr Soc. 2009; 57:889-894

41 Carpenter CR, Bassett ER, Fischer GM Four sensitive screening tools to detect cognitive dysfunction in geriatric emergency department patients: brief Alzheimer’s Screen, Short Blessed Test, Ottawa 3DY, and the caregiver-completed AD8. Acad Emerg Med. 2011; 18:374-384

42 Elie M, Rousseau F, Cole M Prevalence and detection of delirium in elderly emergency department patients. CMAJ. 2000; 163:977-981

43 Jones JS, Dwyer PR, White LJ Patient transfer from nursing home to emergency department: outcomes and policy implications. Acad Emerg Med. 1997; 4:908-915

44 Platts-Mills TF, Biese K, Lamantia M Nursing Home Revenue Source and Information Availability During the Emergency Department Evaluation of Nursing Home Residents. J Am Med Dir Assoc. 2011;

45 Terrell KM, Brizendine EJ, Bean WF An extended care facility– to–emergency department transfer form improves communication. Acad Emerg Med. 2005; 12:114-118

46 Hustey FM, Palmer RM An internet-based communication network for information transfer during patient transitions from skilled nursing facility to the emergency department. J Am Geriatr Soc. 2010; 58:1148-1152