Author	Affiliation
Young M. Lee, BSPH	Vanderbilt School of Medicine, Nashville, Tennessee
Timothy F. Platts-Mills, MD	University of North Carolina at Chapel Hill, Department of Emergency Medicine, Chapel Hill, North Carolina
Joel B. MacWilliams	University of Michigan, Transportation Research Institute, Ann Arbor, Michigan
Mark R. Sochor, MD	University of Virginia, Department of Emergency Medicine, Charlottesville, Virginia
Jeffrey S. Jones, MD	Spectrum Health Systems–Butterworth Campus, Department of Emergency Medicine, Grand Rapids, Michigan
Robert M. Domeier, MD	Saint Joseph Mercy Health System, Department of Emergency Medicine, Ann Arbor, Michigan
Lawrence W. Schneider, PhD	University of Michigan, Transportation Research Institute, Ann Arbor, Michigan
Samuel A. McLean, MD, MPH	University of North Carolina at Chapel Hill, Department of Anesthesiology, Chapel Hill, North Carolina

INTRODUCTION

More than 4 million people present to the United States’ emergency departments (ED) annually for evaluation after motor vehicle collision (MVC).¹ More than 90% of these patients do not experience any initial life-threatening injury and are discharged from the ED after MVC.² However, North American studies indicate that 10% to 20% of these individuals without life-threatening injury nonetheless develop chronic symptoms such as persistent pain, posttraumatic stress disorder, and depressive symptoms.^3–6These post-MVC outcomes cause substantial disability and are a significant public health problem in the United States and other industrialized countries.^4,7

Collision characteristics provide useful information to medical providers in judging the potential for severe injury.^8,9 Additionally, collision characteristics are essential information for studies that examine the epidemiology of long-term outcomes of MVC because they allow researchers to understand and adjust for variability in crash characteristics. Previously conducted research assessing the reliability of ED, emergency medical services, and police records compared to an in-depth crash investigation found that these data sources had fair to moderate agreement.¹⁰ However, no studies have assessed the accuracy of collision information collected prospectively from study participants in a research setting. The objective of this study was to assess the accuracy of collision information reported by ED patients enrolled as research participants by comparing their descriptions of the collisions to information recorded by police at the scene of the collisions.

METHODS

Study Design and Setting

This study was conducted as part of a prospective observational study of patients evaluated after MVC at 3 EDs in the state of Michigan (William Beaumont Hospital at Royal Oak; St Joseph Mercy Hospital at Ann Arbor; and Spectrum Health/Butterworth Hospital at Grand Rapids). William Beaumont Hospital Royal Oak is a 1,061-bed community teaching hospital and Level I trauma center with an annual census of 118,000 ED visits. St Joseph Mercy Hospital is a 529-bed community teaching hospital and Level II trauma center with an annual census of 83,635 ED visits. Spectrum Health/Butterworth Hospital is a 731-bed community teaching hospital and Level I trauma center with an annual census of more than 120,000 ED visits. Data were collected between October 2004 and March 2007. At each study site, all patients who presented to the ED for care at times when research assistants were working were screened, and all eligible patients were approached for inclusion. Owing to limited availability of research assistants for this study, the recruitment method resulted in a convenience sample. ED study sites participating in the study received approval from their local institutional review boards, and all study participants gave written informed consent.

Selection of Participants

Patients were eligible for the study if they were 18 years of age or older, alert and oriented, English speaking, and presented to the ED for evaluation of injuries incurred during an MVC that had occurred within the past 24 hours. Patients were excluded if they were judged to be clinically unstable, had potentially life-threatening or severe injuries, had long bone or spinal fractures, were pregnant, or were admitted to the hospital. Patients who were intoxicated were excluded only if they were not alert and oriented. Patients were eligible if they came to the ED via ambulance or if they came via private transportation.

Data Collection

Following patient consent, a structured clinical interview was used to collect data regarding MVC characteristics. Research assistants were trained in data collection and used a standard data collection template. During the data collection, meetings were held with research assistants on a monthly basis to answer questions and review eligibility criteria and interview questions. Demographic information, patient injury characteristics, and ED discharge diagnoses were extracted from the medical record by using another standard data collection template with explicit definitions of all variables. Research assistants were also trained in data extraction, and data extraction was completed for each patient within 1 week of the ED visit.

Collision characteristics reported by police were obtained via the Michigan State Police collision report database (UD-10 Traffic Crash Report), a publicly available online dataset. Traffic Crash Reports were linked by patient name, birthday, and date of collision to study participant ED interview results. Traffic Crash Report data were abstracted by trained research assistants who were blinded to results of study participant interviews. Traffic Crash Report data are completed by police officers at the scene of the MVC. All police officers are trained in the accurate collection of data using the UD-10 Traffic Crash Report and have demonstrated proficiency in completing this form.

Measures

Collision characteristics assessed during the ED interview included responses for number of lanes on the road, patient’s vehicle type, road conditions, speed limit, seat belt use, airbag deployment, seat position of the patient, extent of damage to the vehicle (0–4 numeric rating scale), method of patient transport to the ED, and time of collision. Patient-reported information about seat belt included options for use of a shoulder and lap belt or just a shoulder belt or just a lap belt. These results are reported separately, but for comparison with police reports, any seat belt use was regarded as using a seat belt. Patient injury information and discharge diagnoses were used to determine the patient’s highest abbreviated injury scale (AIS) score. In this study, the highest AIS score was defined as the highest AIS score for any of the 6 body regions. The AIS was coded by a research nurse with both trauma care and AIS scoring experience.

Traffic Crash Report data included number of road lanes at the location where the collision occurred, type of vehicle, road conditions, speed limit, seat belt use, airbag deployment, extent of damage to the vehicle (0–7 numeric rating scale), whether the patient was transported to the ED via ambulance, and time of collision. In the police report, no distinction was made between which part of the seat belt was used; the report simply stated whether the patient was or was not using a seat belt.

Data Analysis

Descriptive statistics are reported as mean and standard deviation for continuous variables and frequency and proportion for categorical variables. Accuracy of research participant responses was assessed by evaluating agreement between participant responses and police report data. Agreement was assessed with percentage agreement and κ coefficients for categorical response items. Strength of agreement for κ was assessed as follows: poor (<0), slight (0–0.2), fair (0.2–0.4), moderate (0.4–0.6), substantial (0.6–0.8), and almost perfect (0.8–1.0).11 A contingency table is used to provide additional information for seat belt use, for which there was a discrepancy between the κ statistic and the percentage agreement. Agreement was assessed with the Pearson product moment correlation for interval and continuous variables. Statistical analyses were performed by using SAS 9.2 software (SAS Institute, Cary, North Carolina). A sample size analysis was not performed, as this was part of an exploratory study.

RESULTS

Interviews were conducted with 156 patients during a 30-month period of data collection. Of these, there were 97 patients for whom police reports regarding the crash were obtained, and these patients formed the study sample. Demographic characteristics of study participants are shown in Table 1. Most study participants were diagnosed with musculoskeletal strain only; fewer than 10% had fractures or minor lacerations. All study participants had a maximum AIS score of 2 or less. Collision characteristics reported via study participant report and police report are shown in Table 2. These results are intended to provide information about the frequency of outcomes. Because they do not include information about agreement or disagreement for individual cases, the numbers in Table 2 are not intended as evidence of agreement. Most participants reported traveling on roads with 2 or more lanes in dry weather. Most reported being drivers and using seat belts. Of the 93 patients who reported using a seat belt, 1 reported using just a lap belt and 1 reported using just a shoulder belt. The remaining 91 patients reported using both a lap and a shoulder belt. About a third of participants reported airbag deployment and around half came to the ED via ambulance transport.

 

Percentage agreement between study participant and police ratings of MVC characteristics was generally excellent (Table 3). Agreement was greater than 95% for speed limit, ambulance transport, and position in vehicle, and greater than 90% for vehicle type, seat belt use, and airbags deployed. Only 2 characteristics had agreement below 90%: number of lanes (51%) and road condition (88%).

Reliability assessment using κ coefficients also indicated substantial agreement (0.61–0.80) or almost perfect agreement (0.81–1.0) for most items (Table 3). κ values indicated almost perfect agreement for airbag deployment, position in vehicle, and ambulance transport. Substantial agreement was present for vehicle type, road condition, and speed limit. κ values indicated only fair agreement for number of lanes and seat belt use. There were no cases in which both the patient and the police reported that the patient was not wearing a seat belt (Table 4).

Correlation was almost perfect (0.95) for time of collision but moderate (0.58) for the extent of damage to the vehicle. Most of the discrepancies between patient and police reports of the extent of vehicle damage were cases in which the patient described the accident as severe but the police did not. Specifically, among 64 cases for which the police reported the extent of vehicle damage as moderate or heavy (3 or more on the police scale), there were only 2 patients (3%) who reported the extent of damage as mild (2 or less on patient scale). In contrast, among 35 cases in which the police reported the extent of damage as slight or none (2 or less on the police scale), 16 patients (46%) reported the extent of damage to be moderate or severe (3 or more on patient scale). κ and correlation coefficients for patient versus police reports were similar between the subsets of patients brought to the ED by ambulance and those who came by private transportation.

DISCUSSION

Our results demonstrate that patient-reported characterizations of an MVC are accurate for most of the characteristics commonly used in MVC-related research studies and support the use of patient-reported MVC information in the conduct of research studies. The high percentage agreement and κ coefficients for most of the variables suggest that study participant report is an accurate source of MVC information for the following characteristics: vehicle type, road condition, speed limit on the road where the collision took place, airbag deployment, time of collision, and transport by ambulance. Percentage agreement and κ coefficients were less than substantial for the number of lanes. Correlation was also only moderate for extent of damage to the vehicle. Specifically, if we consider the police report as the reference standard, patients tended to overreport accident severity. While patients very rarely described a moderate or severely damaged vehicle as mildly damaged, almost half of patients called a mildly damaged vehicle moderate or severely damaged. It is perhaps not surprising that patients might overreport the severity of vehicle damage, and these results suggest that some caution should be used in interpreting this variable when obtained from patients.

Percentage agreement for airbag deployment (94%), seat belt use (92%), and seat position (99%) from our study is similar to that from previous studies comparing the agreement between the ED medical records and in-depth crash investigator reports.^10,12 Our results extend these findings to ED patients who are providing information as research participants.

Accuracy of data collected was assessed by using both percentage agreement and κ coefficients, because each method has its benefits and limitations. Percentage agreement may not be informative about agreement occurring by chance. In contrast, κ adjusts for the influence of chance, but underestimates agreement when the frequency of an event is either very common or very rare.¹³ The low κ for seat belt use (κ = 0.37), despite a high percentage agreement (92%), is in part a reflection of the limitation of the κ statistic for conditions in which the outcome is either very rare or very common. In this case, using either source of information, almost all patients reported using their seat belt. Given the high percentage agreement, we feel that a patient reporting use of a seat belt remains useful information despite the low κ coefficients for this variable.¹⁴ A previous study found that police overreported the use of seat belts compared to hospital records of patient seat belt use.¹⁵ Our data do not reproduce this finding.

LIMITATIONS

We studied only alert and oriented adult patients involved in MVC not resulting in significant injury but for whom a police crash report was available. It is possible that the accuracy of information obtained from other patient groups such as minors, heavily intoxicated patients, individuals experiencing very minor MVCs not resulting in police reports, or those experiencing life-threatening injuries would differ from the results reported here. Although our analyses compare patient and police reports without identification of a reference standard, in order to draw conclusions about the accuracy of patient reports it is necessary to assume that the police report provides a correct characterization of the MVC. We think this is a reasonable assumption because a previous study comparing ED, emergency medical services, and police reports found that police reports were the most accurate of the 3 sources of information in describing characteristics of MVCs when compared to a crash investigator’s report.¹⁰ Additionally, we would not expect that police reports would be exposed to the potential biases associated with litigation, insurance compensation, and post-MVC distress that may affect patient reporting. However, police reports may be inaccurate for some characteristics because the information may not be readily apparent when the police officer arrivers (eg, seat belt use), and this is a limitation of the study.

CONCLUSION

In our sample, most crash characteristics obtained from ED patients participating in a research study agreed closely with information obtained from police reports. These results extend findings from earlier studies comparing ED medical records with crash investigator reports and support the use of information obtained from patients during research interviews as accurate descriptors of MVCs.

Footnotes

Supervising Section Editor: Michael Menchine, MD, MPH

Reprints available through open access at http://escholarship.org/uc/uciem_westjem

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding, sources, and financial or management relationships that could be perceived as potential sources of bias. This study was supported by a National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01AR056328).

Address for correspondence

Young M. Lee, BSPH

Vanderbilt School of Medicine, 246 Louise Ave, Nashville, TN 37203

E-mail: young.m.lee@vanderbilt.edu

REFERENCES

1. McCaig LF, Burt CW. National Hospital Ambulatory Medical Care Survey: 2003 emergency department summary. Adv Data. 2005;358:1–38.

2. Richmond TS, Jacoby S. Emergency department trauma: The immediate aftermath of motor vehicle collisions. In: Duckworth MP, Iezzi T, O’Donohue WT, editors. Motor Vehicle Collisions: Medical, Psychosocial, and Legal Consequences. Vol. 2008. San Diego, CA: Elsevier; pp. 83–106. In. , eds. 1st ed.

3. Harder S, Veilleux M, Suissa S. The effect of socio-demographic and crash-related factors on the prognosis of whiplash. J Clin Epidemiol. 1998;51:377–384. [PubMed]

4. Freeman MD, Croft AC, Rossignol AM, et al. A review and methodologic critique of the literature refuting whiplash syndrome. Spine (Phila Pa 1976) 1999;24:86–96. [PubMed]

5. Hincapie CA, Cassidy JD, Cote P, et al. Whiplash injury is more than neck pain: a population-based study of pain localization after traffic injury. J Occup Environ Med. 2010;52:434–440. [PubMed]

6. Barnsley L, Lord SM, Wallis BJ, et al. The prevalence of chronic cervical zygapophysial joint pain after whiplash. 1995;20:20–25. Spine (Phila Pa 1976) discussion 26.

7. Spitzer WO, Skovron ML, Salmi LR, et al. Scientific monograph of the Quebec Task Force on Whiplash-Associated Disorders: redefining “whiplash” and its management. Spine (Phila Pa 1976)1995;20:1S–73S. [PubMed]

8. Jones IS, Champion HR. Trauma triage: vehicle damage as an estimate of injury severity. J Trauma.1989;29:646–653. [PubMed]

9. Knudson P, Frecceri CA, DeLateur SA. Improving the field triage of major trauma victims. J Trauma.1988;28:602–606. [PubMed]

10. Grant RJ, Gregor MA, Maio RF, et al. The accuracy of medical records and police reports in determining motor vehicle crash characteristics. Prehosp Emerg Care. 1998;2:23–28. [PubMed]

11. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics.1977;33:159–174. [PubMed]

12. Bilston LE, Brown J. Accuracy of medical and ambulance record restraint and crash data information for child occupants. Inj Prev. 2008;14:46–50. [PubMed]

13. Feinstein AR, Cicchetti DV. High agreement but low kappa: I, the problems of two paradoxes. J Clin Epidemiol. 1990;43:543–549. [PubMed]

14. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med.2005;37:360–363. [PubMed]

15. Kim K. Lie factor in traffic safety: comparison of police and hospital reporting of seat belt and alcohol use in Hawaii. Transportation Research Record. 1999;1665:141–146.