Author	Affiliation
Daniel G. Miller, MD	University of Iowa, College of Medicine, Department of Emergency Medicine, Iowa City, Iowa
Priyanka Vakkalanka, ScM	University of Iowa, College of Medicine, Department of Emergency Medicine, Iowa City, Iowa; University of Iowa, College of Public Health, Department of Epidemiology, Iowa City, Iowa
Mark L. Moubarek, BA	University of Iowa, College of Medicine, Department of Emergency Medicine, Iowa City, Iowa
Sangil Lee, MD	University of Iowa, College of Medicine, Department of Emergency Medicine, Iowa City, Iowa
Nicholas M. Mohr, MD, MPH	University of Iowa, College of Medicine, Department of Emergency Medicine, Iowa City, Iowa; University of Iowa, College of Medicine, Iowa City, Iowa

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Headache is a common complaint in the emergency department (ED). ¹ Use of imaging has increased since computed tomography (CT) was introduced in 1972. ^2-4 In 2010, CTs were performed in 13.9% of U.S. ED visits, and 48% of these were of the head (HCT). ⁵ While this rise has been associated with a decline in rates of admission and transfer, ⁶ multiple sources have suggested that HCT use in the ED could be decreased through quality improvement efforts. ^7-10

The use of HCT by emergency physicians (EP) for evaluation of headache varies widely, and 97% of EPs surveyed felt that at least some of the imaging studies ordered in EDs were medically unnecessary.^9,15 The American College of Emergency Physiciansreleased its Choosing Wisely Campaign in 2013, which included avoiding HCTs in patients with minor head injury who are at low risk based on validated decision rules.^11,12 During the 2015 Academy of Emergency Medicine Consensus Conference on diagnostic imaging in the emergency department, participants suggested that allowing providers to influence metrics could produce better quality metrics; they also suggested that knowledge translation for the optimization of diagnostic imaging use should be a core area warranting further research.^13,17

The Centers for Medicare and Medicaid Services (CMS) proposed OP-15, “Use of Brain Computed Tomography in the Emergency Department for Atraumatic Headache,” to measure the proportion of HCTs performed on ED patients presenting with a primary complaint of headache that were supported by diagnosis codes; however, its methods were soon questioned.^11,16 In 2012 while OP-15 was still under consideration, we implemented a quality improvement (QI) effort intended to improve the documentation of appropriate diagnoses in support of HCT ordering. As part of this QI effort we addressed some of the criticisms of OP-15 by expanding the indications for HCT and getting input from practicing EPs. Reviewing this QI effort in 2014 we observed that the proportions of HCT use decreased after EPs had reviewed their individual practice data. The primary objective of our present study was to determine whether the observed decrease in HCT use was associated with changes in proportions of death or missed intracranial diagnosis. Secondarily, we sought to determine whether proportions of subsequent cranial imaging or reevaluation of headache differed when compared between those who did and those who did not undergo HCT in the ED.

METHODS

Study Setting

This study was a before-and-after study reviewing electronic medical records (EMR) of patients sampled during a QI effort that took place at a 60,000-visit Midwestern, university-based ED between April 2012 and August 2014. We collected follow-up data by EMR review performed between June–August 2016. This study was approved by the local institutional review board under waiver of informed consent.

Intervention

Quality Improvement Project

Our QI effort was structured to fulfill the practice improvement component of the American Board of Emergency Physicians’ Maintenance of Certification requirement. ¹⁴ This required collecting data on 10 visits per EP before and after an intervention. We performed two interventions in succession, so our QI effort yielded three epochs: pre-intervention (April–August 2012); post-education (December 2013–March 2014); and post-data review (April–August 2014) (Figure). At the end of each epoch, we sampled 10 visits for headache seen by each EP by searching the EMR for chief complaints of headache, and identifying the 10 most recent ED visits seen by each faculty EP.

 

Population Health Research Capsule

What do we already know about this issue?

A proportion of CTs performed to evaluate headache in ED patients show no acute intracranial pathology. Also, CT utilization rates vary significantly between emergency providers.

What was the research question?

Could we detect an increase in rates of death or missed intracranial diagnoses following a 9.6% decrease in head CT utilization?

What was the major finding of the study?

We observed no increase in rates of death or missed intracranial diagnoses in the 22.5 months following a reduction in head CT use.

How does this improve population health?

This study suggests that the use of collaborative, non-coercive means may enable emergency physicians to decrease head CT use in the ED without increasing death or missed diagnoses.

 

For our educational intervention we began by soliciting feedback from EPs on OP-15. Using this feedback we expanded the list of appropriate diagnoses supporting HCT (Table 1). We followed this with a series of emails and lectures explaining CMS OP-15. We also conducted group discussions during educational conferences and faculty meetings to educate EPs on selecting appropriate diagnoses to support HCT ordering and explaining the measurement process, highlighting common pitfalls. During group discussions we invited and answered questions. The explicit goal of education was to improve diagnosis documentation, rather than to decrease HCT ordering. This began in late 2012 and continued through 2013.

The data-review phase took place between January and March of 2014 when individual EPs reviewed their own HCT ordering practices based on data collected for the QI effort during the pre-intervention phase. These reviews occurred during individual faculty’s annual reviews with the department chair. In these meetings we presented each EP with his/her individual proportion of HCT ordering and proportion of appropriate diagnosis assignment. In cases where a HCT was ordered without the assignment of an appropriate supporting diagnosis (from Table 1) we reviewed the ED chart. In keeping with Schuur et al.’s findings, we found that in the majority of cases a more specific diagnosis than “headache” was clearly supported by information documented in the ED chart, but had not been assigned at the end of the ED visit.¹⁶ During each annual review we informed the EP of the specific cases where HCT was not supported by a diagnosis code and suggested an alternate, more-specific diagnosis or the addition of a secondary diagnosis that would have made this HCT appropriate according to CMS OP-15. This was followed by the post-data review phase when we sampled another 10 headache visits per EP. After our QI effort was completed, we were surprised to note that while there was no decrease in HCT use after the educational intervention, we observed a 9.6% reduction in HCT use after data review.

Data Collection and Measures

In 2016 we decided to use the dataset generated during the QI effort to investigate our study hypothesis: Was a decrease in HCT use followed by an increase in death or missed intracranial diagnosis? A pre-clinical medical student was trained as a reviewer by an emergency medicine attending who was a QI officer with experience in chart review. The trained reviewer then reviewed the EMR for all patients sampled during the QI effort. We reviewed each patient’s index ED note for the following: age at time of ED visit; gender; migraine history; known history of significant intracranial pathology; whether brain imaging was performed during the index ED visit; and findings from the brain imaging if performed. We reviewed all ED, neurology, neurosurgery, and primary care clinic notes as well as any HCT or brain magnetic resonance imaging (MRI) results occurring in the 22.5-month period following each index ED visit. This length of time was selected because it was the maximum window available from the last visit in the dataset at the time that data collection began.

Follow-up data included the following: whether a follow-up visit took place for a similar headache; diagnoses assigned at follow-up visits; date of follow-up visit; the service providing follow-up care; whether death was recorded in our EMR; whether brain imaging was performed in the follow-up period; and findings from brain imaging if performed. We distinguished between follow-up for any reason and those related to the ED visit as a marker for sample retention during the follow-up window. Data were entered into a standardized data collection spreadsheet. Prior to data collection we defined all terms in the spreadsheet in a data dictionary. No adjustment was made for trainee involvement or subsequent shift changes at the time of the index ED visit. A priori we defined potential missed diagnosis as the presence of any of the following conditions being found after the index ED visit: aneurysm involving the intracranial or cervical vessels; hydrocephalus; intracranial hypertension; stroke (ischemic or hemorrhagic); intracranial mass; subarachnoid hemorrhage; subdural hemorrhage; epidural hemorrhage; intraparenchymal hemorrhage; or dural sinus thrombosis.

To determine which subsequently-identified intracranial conditions should be counted as missed diagnoses we employed a board-certified EP (SL) to perform an independent review of all records where subsequent intracranial conditions were identified. This reviewer was blinded to the study hypothesis and had not been involved in or measured by the initial QI project. For each potential missed diagnosis, the independent reviewer reviewed the index ED visit note, follow-up visit notes and radiology reports before assigning a determination of whether the subsequently-diagnosed cranial condition could have potentially been diagnosed at the index visit. We labeled these as missed intracranial diagnoses.

Inclusion Criteria

Patients of all ages presenting to the ED complaining of headache who had been sampled for the initial QI effort were eligible for inclusion. Exclusion criteria included patients who arrived after inter-hospital transfer, patients admitted during their index visit, and those with a history of ventriculoperitoneal shunt. For patients with multiple ED visits, only the first visit was used as an index visit.

Key Outcome Measures

The primary outcomes of interest were the proportion of death or missed intracranial diagnoses by epoch. Our secondary outcomes were the proportions of patients that followed up for evaluation of similar headache or those who had subsequent cranial imaging (CT or MRI).

Data Analysis

We compared proportions of HCT performance across study epochs, and tested HCT ordering using chi-squared or Fisher’s exact tests, as appropriate. We identified descriptive statistics (proportions) for intracranial findings among those with HCT performed and those with neurological findings identified after the initial visit. We compared outcomes of death, missed diagnoses, identification of a follow-up visit for the same reason as the index ED visit, performance of cranial intervention and brain imaging by epoch and HCT performance. Differences were identified with the Kruskal-Wallis test and Wilcoxon rank-sum test. All tests were considered significant if p<0.05 using two-tailed tests, and analysis was completed using SAS 9.4 (SAS Institute, Inc., Cary, NC).

Quality Assurance

The primary reviewer (MM) extracted data from all charts, and a second reviewer (DM) independently reviewed 30 randomly selected charts for quality assurance. Simple agreement (“yes” vs. “no”) was greater than or equal to 90% for key reported measures.

RESULTS

We initially sampled 695 ED encounters for headache. After we excluded patients who were admitted to the hospital (19), transferred in after evaluation in another ED (19), or included in the study during a prior visit (75), we had a final sample of 582 separate, index ED encounters during the study period. Patient sex, age, migraine history, or pre-existing history of intracranial pathology did not vary across study epoch (Table 2). HCT performance at the index visit only varied by patient age and by patients with a known, pre-existing intracranial condition. Patients who received a CT during the index visit had a higher median age, and a greater proportion had no known, pre-existing intracranial condition. During our pre-education, post-education and post-data review epochs we observed CT ordering proportions of 33.3%, 36.7%, and 25.4%, respectively (p = 0.044) (Table 3).

Primary Outcomes

After a 9.6% reduction in the frequency of HCT use we did not observe a statistically significant difference in proportions of death (p = 0.337) or missed diagnoses (p = 0.312) between study epochs. Across all epochs, we observed a total of nine deaths and 10 missed intracranial diagnoses (Table 4). No deaths had a missed intracranial diagnosis.

Secondary Outcomes

Among patients who had a subsequent visit for evaluation of the same complaint as the index ED visit, 64% had not had a HCT during the index visit compared to 36% who did (p = 0.031). Among patients who had subsequent brain imaging after the index ED visit, 60% did not have a HCT during the index visit compared to 40% who did (p = 0.006) (Table 4).

DISCUSSION

During our QI effort we did not observe a decrease in HCT after a year of educational interventions, but we observed a 9.6% decrease after providers reviewed their own data. This accords with the Institute of Medicine suggestion that feeding providers’ data back to them may be an important part of effectively changing physician behavior. ¹⁹ It is worth noting that during our QI effort we never explicitly instructed providers to decrease HCT ordering. This was motivated by the assumption that our doctors were already trying to do the right thing and avoid unnecessary testing, but that doctors might be capable of being more diligent in diagnosis assignment. The decrease in HCT ordering that we observed came after providers reviewed their own data. So this decrease appears to have resulted from a change that providers took upon themselves after being given the opportunity to look at objective data of their practice patterns and to reflect on what this data told them about their own practice. Happily, this would seem to support our initial assumption that doctors are generally trying to do the right thing.

Previous studies have found that CT pulmonary angiography (CTPA) for evaluation of pulmonary embolism could be safely decreased, thereby decreasing resource utilization without causing harm to patients.^20,21 These studies used probabilistic decision models or looked at inpatient charges, limiting their generalizability to ED patients. The most compelling evidence supporting the safety and cost effectiveness of decreasing CTPA in ED patients had median hospital stays of 7.7 days and medical charges of $6,281.²² This was in contrast to the typical patient presenting to the ED with headache, where reduced testing may mean no testing. We found that a reduction in HCT use for the evaluation of ED patients with headache was not followed by increased death or missed diagnoses. However, the observations that those patients who returned for reevaluation of the same complaint and those who subsequently received brain imaging were more likely to have not had HCT during index visit calls the true impact of decreasing ED-based testing on overall resource utilization into question.

It may be the case that many patients simply feel that they need some sort of test to have had a thorough evaluation. This is supported by studies finding that ED patients who do not receive CT imaging for headache or for abdominal pain were more likely to return within 30 days.^23,24 A previous study has observed up to three-fold variability in the proportion of HCT use for the evaluation of atraumatic headache in the ED.¹⁵ In our study we observed a convergence between EPs’ HCT-ordering proportions when we compared the pre-intervention to the post-data review phases; however, because our study was not designed or powered to investigate this, our observation is only suggestive.

LIMITATIONS

This study has several limitations. As a retrospective chart review, we only had access to information contained in the EMR. Patients who did not follow up with us may have had death or missed diagnoses that we did not observe. In the pre-post study design, however, these factors are likely distributed across time periods, so we do not expect that this study type biased our findings. Approximately 86% of the sample had a subsequent visit within our EMR, suggesting that access to care was good and that the probability of patients seeking care outside our health system was low. Though we cannot exclude other causes of HCT reduction over time, there were no co-existing initiatives in place in the study institution to change HCT ordering practices.

Since we do not practice in a closed medical system, patients could have presented to other systems for care or could have died without presenting to our hospital. To address this issue, we limited the outcomes assessment to patients who received primary care within our university health system by excluding patients who were transferred in, improving the probability that we would capture events. Because of neurosurgical coverage in our predominantly rural state, nearly all patients in our region with significant intracranial pathology would be transferred to our institution for care; therefore, it is unlikely that such outcomes were not captured. This is supported by the observation that over 85% of patients in this study had another encounter in our health system within 22.5 months of the index visit.

The use of an outcome that did not account for the clinical conditions, comorbidities, or appropriateness of initial CT ordering limits the applicability of our findings. However, this type of metric was drafted as part of the proposed quality measure; so interpreting our CT ordering practices in this context parallels the outcomes that might be expected if this metric were more widely adopted. In this way, our study is pragmatic and reflects the limitations of case identification and administrative data use.

CMS OP-15 was found to be unreliable, in part because it relied upon administrative data.¹⁶ We addressed this issue by relying on chart review, the gold standard against which the aforementioned study compared OP-15. This resulted in a more reliable measure but at the cost of a highly labor-intensive technique.

CONCLUSION

We observed no increase in death or missed-diagnosis proportions following a 9.6% reduction in HCT to evaluate ED patients presenting with headache. Patients who subsequently had a repeat visit for the same complaint or underwent HCT after ED discharge were more likely to have not had imaging performed during their index visit. Our study adds to the compelling evidence that there is room to safely decrease CT imaging for ED patients. Determining the cost effectiveness of such reductions requires further research to measure what patients and their healthcare providers do after discharge from the ED when unnecessary testing is withheld.

Footnotes

Section Editor: Edward Sloan, MD, MPH

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Daniel G. Miller, MD, University of Iowa, College of Medicine, 200 Hawkins Dr., 1008 RCP; Iowa City, IA, 52242. Email: daniel-miller@uiowa.edu. 3 / 2018; 19:319 – 326

Submission history: Revision received May 18, 2017; Submitted December 15, 2017; Accepted December 15, 2017

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Friedman BW, Grosberg BM. Diagnosis and management of the primary headache disorders in the emergency department setting. Emergency Med Clin of N Am. 2009;27(1):71-7.

2. Talwalkar A, Ashman JJ. Annual percentage of emergency department visits with selected imaging tests ordered or provided – national hospital ambulatory medical care survey, United States, 2001–2010. MMWR Morb Mortal Wkly Rep. 2013;62:455.

3. Kocher KE, Meurer WJ, Fazel R, et al. National trends in use of computed tomography in the emergency department. Ann Emerg Med. 2011;58(5):452-62.e3.

4. Larson DB, Johnson LW, Schnell BM, et al. Rising use of CT in child visits to the emergency department in the United States, 1995–2008. Radiology. 2011;259(3):793-801.

5. National ambulatory medical care survey: 2010 summary tables. Available at: https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2010_namcs_web_tables.pdf. Accessed August 29, 2017.

6. Larson DB, Johnson LW, Schnell BM, et al. National trends in CT use in the emergency department: 1995–2007. Radiology. 2011;258(1):164-73.

7. Schroeder AR, Redberg RF. The harm in looking. JAMA Pediatr. 2013;167(8):693-5.

8. Mills AM, Raja AS, Marin JR. Optimizing diagnostic imaging in the emergency department. Acad Emerg Med. 2015;22(5):625-31.

9. Hess EP, Marin J, Mills A. Medically unnecessary advanced diagnostic imaging and shared decision-making in the emergency department: Opportunities for future research. Acad Emerg Med. 2015;22(4):475-7.

10. Gawande A. Overkill. An avalanche of unnecessary medical care is harming patients physically and financially. What can we do about it?. Available at: http://www.newyorker.com/magazine/2015/05/11/overkill-atul-gawande. Accessed August 29, 2017.

11. Imaging efficiency measures. Available at: http://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic2FPage%2FQnetTier2&cid=1228695266120.

12. Five things physicians and patients should question. 2013. Available at: http://www.choosingwisely.org/wp-content/uploads/2015/02/ACEP-Choosing-Wisely-List.pdf. Accessed August 29, 2017.

13. Marin JR, Mills AM. Developing a research agenda to optimize diagnostic imaging in the emergency department: An executive summary of the 2015 academic emergency medicine consensus conference. Acad Emerg Med. 2015;22(12):1363-71.

14. Patient care practice improvement (PI) activity. Available at: https://www.abem.org/public/abem-maintenance-of-certification-(moc)/moc-improvement-in-medical-practice/patient-care-practice-improvement-(pi)-activity/patient-care-practice-improvement-overview. Accessed August 29, 2017.

15. Prevedello LM, Raja AS, Zane RD, et al. Variation in use of head computed tomography by emergency physicians. Am J Med. 2012;125(4):356-64.

16. Schuur JD, Brown MD, Cheung DS, et al. Assessment of medicare’s imaging efficiency measure for emergency department patients with atraumatic headache. Ann Emerg Med. 2012;60(3):280-90.

17. Berdahl C, Schuur JD, Fisher NL, et al. Policy measures and reimbursement for emergency medical imaging in the era of payment reform: Proceedings from a panel discussion of the 2015 Academic Emergency Medicine Consensus Conference. Acad Emerg Med. 2015;22(12):1393-9.

18. Kanaan Y, Knoepp UD, Kelly AM. The influence of education on appropriateness rates for CT pulmonary angiography in emergency department patients. Acad Radiol. 2013;20(9):1107-14.

19. Ferguson TB. The Institute of Medicine committee report “best care at lower cost: The path to continuously learning health care.”. Circ-Cardiovasc Qual Outcomes. 2012;5(6):E93-4.

20. Fanikos J, Rao A, Seger AC, et al. Hospital costs of acute pulmonary embolism. Am J Med. 2013;126(2):127-32.

21. Troyer JL, Jones AE, Shapiro NI, et al. Cost-effectiveness of quantitative pretest probability intended to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Acad Emerg Med. 2015;22(5):525-35.

22. Kline JA, Jones AE, Shapiro NI, et al. Multicenter, randomized trial of quantitative pretest probability to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Circ Cardiovasc Imaging. 2014;7(1):66-73.

23. Patterson BW, Venkatesh AK, AlKhawam L, et al. Abdominal computed tomography utilization and 30-day revisitation in emergency department patients presenting with abdominal pain. Acad Emerg Med. 2015;22(7):803-10.

24. Patterson BW, Pang PS, AlKhawam L, et al. The association between use of brain CT for atraumatic headache and 30-day emergency department revisitation. AJR Am J Roentgenol. 2016;207(6):W117-W24.