Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Background

Approximately 80% of non-traumatic cases of subarachnoid hemorrhage are attributable to ruptured cerebral aneurysms, for which delays in definitive aneurysm treatment can increase the risk of disability or death. ^{1, 2} While the vast majority of aneurysmal subarachnoid hemorrhage (aSAH) cases are identified by cranial computed tomography (CT), the sensitivity of CT diminishes with time such that lumbar puncture is recommended as the definitive test to exclude a diagnosis of “CT-negative” SAH. While two large studies have reported that early cranial CT (i.e. performed within six hours of headache onset) may be up to 100% sensitive for SAH among patients presenting with a normal mental status, ^3-6 a prior study by our research group demonstrated imperfect sensitivity of this definition of early cranial CT in the non-academic emergency department (ED) setting. ⁷ Accordingly, we proposed that sequential application of a validated SAH clinical decision rule (absence of all the following: age ≥40, neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) in such a clinical scenario might further reduce the posterior probability of CT-negative SAH to an acceptable level of risk. ^8-10

Goals of this investigation

We sought to further examine the potential incremental gain in sensitivity when applying a previously validated SAH clinical decision rule to a cohort of patients diagnosed with aSAH after presenting to the ED with normal mental status and undergoing early cranial CT.

METHODS

Study Population

We screened electronic health records of patients treated within the Kaiser Permanente Northern California (KPNC) integrated healthcare delivery system between January 2007 and June 2013 for case inclusion if they had an ED or hospital encounter with an associated International Statistical Classification of Diseases and Related Health Problems, ninth edition (ICD-9) diagnosis code of SAH (430). Emergency care within KPNC is provided through 21 non-academic medical center-based EDs, serving a population of over 3.3 million Kaiser Foundation Health Plan (KFHP) members. This study was part of a larger project examining outcomes following misdiagnosis of aSAH. Patients were electronically excluded if they had an ICD-9 coded diagnosis of head or neck trauma within 24 hours of the index encounter, lacked continuous KFHP membership within the two weeks preceding diagnosis, were under 18 years of age or had a prior diagnosis of SAH between 2002 and 2006. Data on age, sex and race were electronically collected. We then manually reviewed charts for the following inclusion criteria: initial diagnosis at a KPNC ED, Hunt-Hess clinical grade of 1 or 2 at the time of ED presentation, non-contrast cranial CT imaging within six hours of headache onset, either evidence of SAH on non-contrast cranial CT or greater than five red blood cells per microliter on cerebrospinal fluid analysis, and angiographic evidence of cerebral aneurysm thought to be consistent with the clinical presentation and pattern of hemorrhage visualized on imaging, if applicable. The study was approved by the Kaiser Foundation Research Institute Institutional Review Board with a waiver of the requirement for informed consent.

Methods and Measurements

Two investigators (DGM and MVK) conducted a structured explicit chart review and abstraction of records using a standardized paper form as part of a larger study examining outcomes following misdiagnosis of aSAH. ¹¹ Abstractors confirmed the inclusion criteria and the final radiologist interpretation of the initial cranial CT, the location and size of the culprit aneurysm and documentation of the presence or absence of the following variables: neck pain or stiffness, loss of consciousness, physical exertion at the time of headache onset, need for external ventricular drainage and treatment of vasospasm during hospitalization. A best modified Rankin Scale (mRS) score at one year was assigned by reviewing neurosurgical, rehabilitation services and primary care clinical notes following hospital discharge, if applicable, using previously validated methodology. ¹² We considered a mRS score ≤2 a favorable neurologic outcome. Both abstractors reviewed 20% of the sample to establish the inter-rater reliability of the following variables with an estimated error margin less than 20%: early cranial CT (inclusion criteria), Hunt-Hess grade at ED presentation and the clinical decision rule. ¹³ All CT examinations were performed without contrast using multi-slice cine technology (16 slice or higher). Either general radiologists or neuroradiologists made the final interpretation of CT images.

Outcomes and Analysis

The primary outcome of interest was the combined sensitivity of early cranial CT and the SAH clinical decision rule (a negative result for the latter being defined as absence of all four clinical criteria). Secondary outcomes were the independent sensitivities of early cranial CT and the SAH clinical decision rule. Missing variables from the SAH clinical decision rule were imputed as being absent to provide the most conservative estimate of sensitivity. We calculated binomial confidence intervals (CI) using the Clopper-Pearson (exact) method. All statistical analyses were performed using STATA v 13.0 (College Station, TX).

RESULTS

Characteristics of study subjects

We identified 155 patients following application of exclusion and inclusion criteria (Figure). The median age was 55 years and 79% were female. Hunt-Hess grade was 2 in 95% of patients, though none of these had notation of a cranial nerve deficit upon initial ED evaluation. The most common aneurysm location was the anterior communicating artery (30%), followed by the posterior communicating artery (21%). Eighty percent of patients had a favorable neurologic outcome one year from initial hospitalization. Summary statistics of the study population are provided in Table 1.

Main results

Early cranial CT was reported as positive for SAH in 148 patients, yielding an estimated sensitivity of 95.5% (95% CI [90.9–98.2]). The SAH clinical decision rule was likewise positive in 148 patients with the same estimated sensitivity (95.5%, 95% CI [90.0–98.2]). Since the false negative cases for early cranial CT were mutually independent from the false negative cases by the SAH clinical decision rule, the combined estimated sensitivity for application of both early cranial CT and the SAH clinical decision rule was 100% (95% CI [97.6–100.0]). Seven patients (4.5%) underwent lumbar puncture, all of whom had negative early cranial CT interpretations. Pertinent details for false negative cases of early cranial CT and the SAH decision rule are provided in Tables 2 and 3, respectively.

The inter-rater agreement for electronic health record abstraction was 100% for early cranial CT, 87% for Hunt-Hess grade (1 versus 2) and 100% for a negative result on the overall SAH clinical decision rule (71% for neck pain or stiffness in isolation).

DISCUSSION

The goal of this study was to help clinicians further refine and understand current testing strategies for aSAH, specifically by highlighting the potential gain in sensitivity obtained with the “post-hoc” application of a SAH clinical decision rule following early cranial CT. While we recognize that this was not the original derivation or validation setting of the SAH clinical decision rule, we feel our study demonstrates that such an approach can potentially help inform shared decision-making between clinicians and patients when faced with uncertainty over the absolute sensitivity of early cranial CT, especially in the face of non-low pretest probability.

This testing strategy is similar in concept to performing a serum d-dimer assay to reliably exclude a lower extremity deep vein thrombosis in a patient with a high pre-test probability for disease but a negative lower extremity ultrasound examination; only the combined sensitivities of the two tests offer a low enough post-test probability to forgo further testing. ¹⁴ Additionally, although early cranial CT failed to detect aSAH in seven out of 155 cases (4.5%) in our retrospective cohort, this is only a point estimate and is specific to our practice setting. The findings of 100% sensitivity in the prospective Perry et al. ⁴ cohort and the retrospective Backes et al. ³ study rightfully prompt consideration of using early CT alone to rule out aSAH in those particular practice settings.

Important contextual differences between our study and prior reports involve both spectrum bias and radiologist staffing practices. Perry et al. ⁴ enrolled patients with acute headaches reaching maximal intensity within one hour. Backes et al. ³ retrospectively identified patients presenting to a SAH referral center with a 50% incidence of SAH among patients undergoing early cranial CT (as opposed to 13% in the Perry et al. study). Thus it is possible that our study cohort includes patients with less severe presentations of aSAH who may be less likely to manifest positive CT findings on early cranial CT. Likewise, radiology staffing at a tertiary neurosurgical referral center as in Backes et al. ³ is not representative of the vast majority of EDs. While the Perry et al. ⁴ study setting was similar to ours in that radiographic studies were interpreted by a mix of neuroradiologists and general radiologists who routinely interpreted cranial CTs, several of the medical centers participating in that study had active radiology residency training programs with over-reading of studies by faculty in the daytime, making it difficult to extrapolate that level of scrutiny to practicing radiologists in a non-academic hospital setting. Of note, one early cranial CT in that series was initially misinterpreted as being negative by a radiology trainee, and was only retrospectively re-interpreted as positive when a magnetic resonance angiogram performed several days later revealed an aneurysm. This example highlights the potential for introducing hindsight bias by using final written radiology reports as the gold standard for CT interpretations, an issue for all studies on this topic to date.

Finally, it is notable that all of the patients with false negative cranial CT studies met the age criteria of the SAH clinical decision rule. It is thus possible that a post-imaging rule could be further refined for improved specificity and ease of applicability.

LIMITATIONS

Given the retrospective nature of the study, appropriate characterization of early cranial CT is only as accurate as the available documentation. While the inter-rater agreement for abstraction of this dichotomous variable was 100%, we cannot exclude errors in reporting the actual time of headache onset such that some patients may have been both included and excluded inappropriately. However, such errors in reporting can occur in prospective observational studies as well, and thus is a more general limitation of using historical factors as part of any decision rule.

The completeness of our case identification was also limited by the accuracy of the diagnostic coding from hospital and ED encounters. However, we searched databases specific for diagnostic codes assigned during treatment within KPNC as well as those used to track services billed for outside of KPNC, thus making it unlikely that we failed to capture cases of aSAH that were transferred to a non-KPNC hospital. Regardless, it seems improbable that we failed to capture enough cases of aSAH to appreciably alter our results; for example, to raise the sensitivity point estimate for detection of aSAH by early cranial CT to 99.0%, we would require an additional 545 cases of aSAH, all with positive early cranial CT findings (693/700=0.99).

Given that we conservatively imputed missing variables for the SAH decision rule as being absent, it is also possible that the SAH clinical decision rule may have performed better than reported if missing variables were in fact present. However, recent prospective and retrospective validations of the same SAH clinical decision rule also revealed suboptimal sensitivities at 98.5% and 96.6%, respectively. ^{8, 15} Additionally, we cannot comment on the specificity of the SAH clinical decision rule when applied following early cranial CT, given the case-only cohort design of our study. It is possible that such an approach would not reduce overall testing rates from current practice, although individual clinicians may be differentially influenced given variable testing thresholds. ¹⁶

Finally, we did not include potentially missed cases of aSAH who may have undergone early cranial CT imaging with no confirmatory (LP) or subsequent testing, only to later present with evidence of aneurysm rupture. Inclusion of such cases in this cohort would bias towards a lower sensitivity of early cranial CT since it is impossible to be certain that evidence of SAH would have been otherwise detected (i.e., by lumbar puncture) at the initial evaluation.

CONCLUSION

In summary, we found the sensitivity of early cranial CT for aSAH to be 95.5% among patients presenting to non-academic EDs in an integrated healthcare system. Application of a SAH clinical decision rule in addition to early cranial CT improved sensitivity to 100% (95% CI [97.6–100.0]). Prospective decision rule refinement and validation of this approach is warranted.

Footnotes

Section Editor: William Whetstone, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Author contribution statement: DGM, MVK, DRV and DWB conceived the study and obtained research funding. DGM and NU refined participant and variable selection. DGM and MVK undertook chart abstraction and managed the data, including quality control. DGM drafted the manuscript, and all authors contributed substantially to its revision. DGM takes responsibility for the paper as a whole.

Address for Correspondence: Dustin Mark, MD, Kaiser Permanente, Department of Emergency Medicine, 275 W. MacArthur Blvd, Oakland, CA 94611. Email: Dustin.G.Mark@kp.org. 9 / 2015; 16:671 – 676

Submission history: Revision received March 6, 2015; Submitted May 26, 2015; Accepted July 9, 2015

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. Funded by a Kaiser Permanente Northern California Community Benefits Grant.

REFERENCES

1. Nibbelink DW, Torner JC, Henderson WG. Intracranial aneurysms and subarachnoid hemorrhage – report on a randomized treatment study. IV-A. regulated bed rest. Stroke. 1977;8:202-18.

2. Edlow JA, Caplan LR. Avoiding pitfalls in the diagnosis of subarachnoid hemorrhage. N Engl J Med. 2000;342:29-36.

3. Backes D, Rinkel GJ, Kemperman H, et al. Time-dependent test characteristics of head computed tomography in patients suspected of nontraumatic subarachnoid hemorrhage. Stroke. 2012;43:2115-9.

4. Perry JJ, Stiell IG, Sivilotti ML, et al. Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study. BMJ. 2011;343:d4277.

5. Claveau D, Dankoff J. Is lumbar puncture still needed in suspected subarachnoid hemorrhage after a negative head computed tomographic scan?. CJEM. 2013;15:1-3.

6. Vergouwen MD, Rinkel GJ. Clinical suspicion of subarachnoid hemorrhage and negative head computed tomographic scan performed within 6 hours of headache onset–no need for lumbar puncture. Ann Emerg Med. 2013;61:503-4.

7. Mark DG, Hung YY, Offerman SR, et al. Nontraumatic subarachnoid hemorrhage in the setting of negative cranial computed tomography results: external validation of a clinical and imaging prediction rule. Ann Emerg Med. 2013;62:1-10.e1.

8. Perry JJ, Stiell IG, Sivilotti ML, et al. Clinical decision rules to rule out subarachnoid hemorrhage for acute headache. JAMA. 2013;310:1248-55.

9. Perry JJ, Stiell IG, Sivilotti ML, et al. High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study. BMJ. 2010;341:c5204.

10. Newman-Toker DE, Edlow JA. High-stakes diagnostic decision rules for serious disorders: the Ottawa subarachnoid hemorrhage rule. JAMA. 2013;310:1237-9.

11. Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in emergency medicine research: Where are the methods?. Ann Emerg Med. 1996;27:305-8.

12. Raina KD, Callaway C, Rittenberger JC, et al. Neurological and functional status following cardiac arrest: method and tool utility. Resuscitation. 2008;79:249-56.

13. Gwet K. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters. 2012.

14. Bates SM, Jaeschke R, Stevens SM, et al. Diagnosis of DVT: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 2012;141:e351S-418S.

15. Kelly AM, Klim S, Edward S, et al. Sensitivity of proposed clinical decision rules for subarachnoid haemorrhage: An external validation study. Emerg Med Australas. 2014;26:556-60.

16. Perry JJ, Eagles D, Clement CM, et al. An international study of emergency physicians’ practice for acute headache management and the need for a clinical decision rule. CJEM. 2009;11:516-22.