Author	Affiliation
Susan I. Woodruff, PhD	San Diego State University, School of Social Work, San Diego, California
Kimberly Eisenberg, MSW	San Diego State University Research Foundation, San Diego, California
Cameron T. McCabe, MA	San Diego State University Research Foundation, San Diego, California
John D. Clapp, PhD, MSW	San Diego State University, School of Social Work, San Diego, California
Melinda Hohman, PhD, MSW	San Diego State University, School of Social Work, San Diego, California

Introduction Methods Results Discussion Conclusion

INTRODUCTION

Alcohol and drug misuse in the United States are major public health problems that degrade the physical and psychological well-being of individuals, families, and communities.^1–2 In recent years, the need for improved alcohol and drug use behavioral risk reduction strategies has led to the rise in popularity of brief interventions (BI), time-limited, structured, goal-oriented interventions that typically last 30 minutes or less.³ In 2003, the Center for Substance Abuse Treatment (CSAT), within the Substance Abuse and Mental Health Services Administration (SAMHSA), began federal funding of Screening, Brief Intervention, and Referral to Treatment (SBIRT) demonstration programs. Unlike primary prevention that targets non or low-risk users, or treatment services for people already dependent, SBIRT provides early intervention services targeted at individuals who misuse alcohol or illicit drugs, but who may have not yet developed dependence.⁴

Although individual program frameworks vary, all SBIRT programs share 2 key components: screening and intervention. Individuals who screen positive for alcohol or drug problems are provided with an appropriate educational or therapeutic service. Most of those screening “positive” are categorized as relatively low risk and receive a BI, consisting of a time-limited motivational interview done in the ED that focuses on increasing patient awareness of the risks of substance abuse, feedback on normative use and safe limits, and eliciting motivation to change.³ Individuals at moderate to severe risk are provided brief intervention plus brief treatment (e.g., 6 face-to-face counseling sessions) or Referral to Specialty Treatment for more intensive support.³

Studies have suggested SBIRT’s effectiveness in emergency department (ED) patients.^5–11 The ED visit may present a “teachable moment” in which a patient may be more open to feedback and suggestions regarding their risky health-related behaviors. Despite the proliferation of BIs in EDs, a recent meta-analysis suggested that benefits from such services are not necessarily due to the BI itself, and that the benefits may be short-lived.¹² Studies have also identified substantial challenges with methodology and feasibility in such settings.¹³ Further research is needed to determine the true effectiveness of BI in acute medical care settings.

The purpose of this evaluation study was to examine substance use outcomes of Southern California’s large SBIRT service program, known as CASBIRT, which was conducted with a large convenience sample of ED/trauma patients in 12 acute care settings. Although it was expected that CASBIRT would exhibit levels of effectiveness similar to other SBIRT programs (particularly with regard to alcohol use), we believed it possible that some outcomes would be unique due to the socio-demographic characteristics of the residents of San Diego County, which includes a large Latino population. In addition, because data were collected in a border region with relatively high drug trafficking activity, it was possible that results would differ from other regions in the United States.

METHODS

Screening and Intervention Procedures

The California SBIRT program, CASBIRT, provided services from June, 2007 through July, 2010, and screened close to 120,000 patients in 12 San Diego County hospital EDs and trauma centers as part of routine care. A private area, usually the room in which the patient was waiting to receive care, was used to conduct the screening interviews. Screenings were conducted by trained Health Educators (HEs) at various times during the patient visits, and were frequently interrupted for medical care and resumed later in the visit. HEs attempted to screen all adult patients (18 years of age and older), regardless of the reason for their ED visit, with the exception of patients with severe illness/injury, acutely intoxicated patients, and patients who were not competent or capable to give consent. Patient participation was voluntary and permission (but not informed consent) to be screened was obtained prior to screening. HEs explicitly stated to the patient that the questions were asked of all patients for purposes of providing the medical team with comprehensive information about patient health status in order to improve overall quality of care. Typically, the screening process took about 10 minutes, although for higher risk patients, the process could take up to 30 minutes.

HEs screened patients using the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST), a 9-item instrument designed for the World Health Organization (WHO) in 1997 as a valid and brief method of screening for substance use in medical care settings.¹⁴ The severity, or risk level of the patient’s alcohol use and illicit drug use was derived from ASSIST items assessing past 3-month use of alcohol and eight individual illicit drugs (i.e., cocaine, cannabis, opioids, hallucinogens, amphetamine type stimulants, sedatives, inhalants, and an option for an “other” drug). We applied cut points, based on those of the developers but modified for our local ED population, were applied to raw severity scores to categorize patients into 1 of 4 risk categories for alcohol and each illicit drug.¹⁵ For alcohol use, patients were categorized as Low-risk (scores of 0–6), At-risk (scores of 7–19), High-risk (scores of 20–26), or Severe-risk (scores of 27 and over). For use of each of the eight illicit drugs, patients were categorized as Low-risk (scores of 0–1), At-risk (scores of 2–18), High-risk (scores of 19–26), or Severe-risk (scores of 27 and over). For each patient, the highest of the 8 ASSIST drug risk levels was used as an overallmeasure of drug use risk. Risk level cut points for alcohol were different than the cut points for illicit drugs because some alcohol use is considered within safe, while any use of illicit drugs is considered a problem.

Low-risk patients for both alcohol and drugs were congratulated for their status and encouraged to continue practicing healthy behaviors. Patients in the other risk categories were offered services that corresponded to the severity of their risk level: all patients received a minimum of a brief intervention (BI); high risk patients were offered an opportunity to participate in up to 6 sessions with a brief treatment counselor in person or over the phone; and severe risk patients were offered Referral to Specialty inpatient or outpatient Treatment. If a patient fell into different risk categories for alcohol and drug use, the service delivered was tailored to the individual’s highest level of risk. If a patient declined a more intensive level of intervention, he or she was offered a lower intensity level of service.

CASBIRT’s Brief Intervention

Core elements of the BI included focus on increasing patient awareness of the risks of their misuse, feedback on normative use and safe limits, and eliciting motivation to change. ³ The HE began by positively reinforcing healthy behavior, such as drinking within recommended limits or abstaining from illicit drug use. Depending on the severity of their substance misuse, HEs encouraged patients to reduce to the recommended drinking limits and to abstain from drug use while also detailing risks associated with heavy alcohol and/ or drug use (e.g., long and short term health risks, financial, social, and legal problems). These interventions utilized motivational interviewing, a communication method that determines a person’s willingness to change and attempts to negotiate a commitment to reduce substance use.^16–17 Brochures with educational information and guidelines for reducing risks were used to supplement and direct the dialogue between the HE and patient.

Health Educators (HEs) Training

CASBIRT utilized bi-cultural/bi-lingual (English/Spanish) HEs who were able to meet the linguistic and cultural needs of San Diego County’s large Latino population. Twenty-seven paraprofessional Health Educators (HEs) delivered SBIRT services after receiving 3 months of training, including two weeks of training in cognitive behavior therapy from a licensed psychotherapist, and motivational interviewing training provided by the author (MH).^16–17 The curriculum also included alcohol and drug education, intervention protocol adherence, videotaped role-playing sessions, and onsite shadowing with feedback.

About 85% of the HEs were female. The majority had interviewing experience and were students pursuing bachelor’s degrees in health and human service-related fields. Two HEs had master’s degrees, and 1 had previously worked as a dentist in Mexico. HEs were present in most ED/trauma centers 7 days a week, with coverage from 7am to 11pm.

Follow-up Procedures

As part of its program evaluation activities, CASBIRT’s goal was to recruit 10% of all those screened for a follow-up telephone survey. After screening and delivering an intervention, a subset of patients was targeted by the HE for the 6-month follow-up interview. Patients who fell within an elevated alcohol or drug risk level (At risk or above), and whose last 2 digits of their telephone number fell within a specified range were asked about their willingness to participate in a follow-up. The range of the last 2 digits of telephone numbers was used to introduce randomness to the selection of patients targeted for follow-up. If patients consented to participate in follow-up, they provided their own contact information as well as that for at least 1 friend or relative who could be contacted in an attempt to locate the patient. Cohort maintenance activities in the form of periodic telephone contact and a postcard were used to keep in contact with those identified for follow-up. Patients were also informed they would receive a $20 gift card by mail upon completion of the follow-up interview.

Bilingual Evaluation Assistants (EAs), separate from Health Educators, were trained in health surveying, cohort maintenance, and Telescript software (Telescript, Inc., Norwood, NJ) to track and conduct 6-month telephone follow-up interviews. EAs continued to call each participant until they completed the 6-month interview or until the participant fell outside of the follow-up window at 8-months post intake. Six-month follow-up interviews consisted of the same alcohol and drug use items asked at intake. Follow-up interviews typically took 15 to 20 minutes to complete.

Sampling

Figure 1 graphically presents the flow and number of patients receiving screening/intervention, recruited into follow-up, and participating in follow-up. HEs approached 150,979 patients presenting to ED and trauma departments. Approximately 22% of those approached (n=32,886) did not complete the screening assessment for various reasons. As shown in Table 1, most missed screening opportunities were due to patients having been previously screened by CASBIRT, being incapable (e.g., disoriented), or being ineligible by virtue of age and language barriers. Only 1% of patients refused to be screened. Approximately 20% of those patients screened (n=24,363) screened positive for alcohol or drugs (or both) with the ASSIST instrument. A total of 2,436 screened patients consented to be in the 6-month follow-up sample. Of those, 1,504 (69%) were lost to follow-up, leaving 672 patients who comprised the complete longitudinal sample. The 1,504 patients lost to follow-up were included in the analyses using an intent-to-treat (ITT) approach whereby their intake responses were carried over as follow-up values.

Design and Measures

This evaluation study utilized a single group pre-post test design. Prior to the analyses, approval for the study was obtained from San Diego State University Institutional Review Board.

Outcomes.

The Government Performance and Results Act (GPRA) assessment tool was used to assess past 30-day binge drinking and use of illicit drugs, as well as sociodemographic information.^18–19GPRA’s alcohol and drug use measures are modified items from the widely-used Addiction Severity Index. ²⁰ Six dependent variables were computed at baseline and at follow-up: (a) past 30 day prevalence of binge drinking, i.e., 5 or more drinks per sitting (yes/no), (b) number of days of binge drinking in the past 30 days, (c) past 30 day prevalence (yes/no) of illicit drug use, (d) number of days of illicit or nonprescribed drug use in the past 30 days, (e) past 30 day prevalence of marijuana use (yes/no), and (f) number of days of marijuana use in the past 30 days.

Independent Variables.

At intake, HEs asked the patient’s gender, age, and race/ethnicity. Age was treated as a categorical variable, with groups defined as 18–24, 25–34, 35–44, 45–54, and 55 years and older. Race/ethnicity was determined using GPRA options and recoded into the following: Latino/Hispanic, Black, White non-Latino, and Other. Patient’s ASSIST risk level for alcohol and drugs assessed at intake was categorized as low risk, at risk, high risk, or severe risk. Finally, a ‘type of user’ variable was computed with the 2 categories: (a) alcohol binger or drug misuser only, or (b) misuser of both alcohol and drugs.

Data Analysis

We used frequency distributions to describe the disposition of screening attempts and overall sample characteristics. We used Chi-square and t-test analyses to assess baseline differences in those followed and those lost to follow-up. Chi-square analysis was used to describe the sociodemographic characteristics of patients by type of user. To assess intake-to-follow-up change, we used a conservative ITT approach in which 6-month values for outcomes for those lost to follow-up were recoded with the last value carried forward (LVCF). ^6,21 This approach meant replacing missing follow-up values with intake responses to avoid potential non-response bias. We then used generalized linear mixed models (GLMM) to assess changes in past 30 day prevalence of use (i.e., logistic GLMM) and days of use (i.e., linear GLMM) among those At risk and above for misuse, adjusting for clustering by ED/trauma site. We also used GLMM to assess subgroup differential change by separately testing interactions between time and gender, age, race/ ethnicity, risk status, and type of user. In these interaction models, site and time main effects were included in the model. We conducted all analyses using SPSS Statistics release version 19 (IBM, Chicago, Illinois).

RESULTS

Adequacy of the Sample

Those lost to follow-up (n=1,504) and those successfully followed (n=672) were found to be similar with regard to sociodemographic characteristics and most baseline substance use measures, including: gender; age; race/ethnicity; status as an ED versus trauma patient; marijuana use risk level; prevalence of past 30 day alcohol binging, illicit drug use, and marijuana use; and the number of days in the past 30 days 1 used illicit drugs and marijuana. On the other hand, those lost to follow-up had a significantly higher alcohol risk level (p<0.01); a higher drug use risk level (p<0.05); and a higher number of days of binge drinking in the past 30 days (p<0.001) than those successfully followed.

Sample Characteristics

Fifty-nine percent of the ITT follow-up sample were men. The average age was 39 years of age (SD=13.9) with a range of 18 to 99 years. The follow-up sample was comprised of 42% non-Latino Whites, 38% Latinos, 14% Blacks, and 6% Other races/ethnicities. Roughly 40% of Latinos opted to have the intake interview in Spanish. Close to 90% of patients were screened in the ED while approximately 10% were screened in trauma units.

Of those screening positive for drug or alcohol abuse, 25% of the ITT sample misused alcohol exclusively, 54% misused illicit drugs only, and 21% misused both. As shown in Table 2, demographic characteristics differed significantly for those misuse groups. The alcohol-only group had a particularly high proportion of males, older patients, and Latinos. The drug use only group was almost equally comprised of males and females, was spread out fairly equally across age groups (with the exception of the 18–20 year olds), and was comprised of a large proportion of non- Latino Whites. Concurrent alcohol and drug misusers were predominately male, relatively young, and were typically non- Latino White or Latino. While trauma patients comprised a relatively small percent of patients overall, misusers of alcohol only and alcohol and drugs were more likely to be seen in trauma than were the drug only misusers.

Among those who were in the At risk or above categories for drug use, 48% had used marijuana in the last month, 14% methamphetamines, 7% cocaine, 5% heroin, 2% hallucinogens, 1.4% benzodiazepines, and 0.5% each morphine and oxycontin. About 2.5% reporting using another illegal drug in the past month; 3.5% reported injecting drugs in the past month. The mean number of different drugs used in the past month (excluding alcohol) was 0.7 (SD=0.78), with a range of 0–7 drugs. Among those using any illicit drug in the past month, the average number of drugs used was 1.2 (SD=0.62).

Changes in the Overall Sample

Figures 2 and ​and33 present changes in past 30 day prevalence and days of use, respectively, for alcohol binging, illicit drug use, and marijuana use for both the true longitudinal sample and the ITT sample. Differences in the magnitude of change in the two samples underscore the bias in estimates that can occur should data only from follow-up responders be used. ITT follow-up prevalence estimates were 1.4 higher (for binge drinking) to almost 2 times higher (for drug use and marijuana use) in the ITT sample than in the longitudinal sample (Figure 2). Regarding days of use, ITT sample estimates were over twice as high as those in the true longitudinal sample for all 3 substances.

All subsequent results will be based on ITT sample results. As shown in Figure 2 for the ITT sample, the past-month prevalence of binge drinking declined from intake to follow-up by a modest 4.3 percentage points; drug use prevalence reductions (10.4 percentage points) and marijuana use prevalence reductions (7.3 percentage points) were of greater magnitude than binge drinking prevalence changes. Days of binging for the ITT sample decreased by 0.39 days, illicit drug use decreased by almost 1 day, and marijuana use specifically decreased by 0.66 days (see Figure 3).

Changes in Those with Substance-specific Risk

Figure 4 presents changes in the past 30 day prevalence of binge drinking, drug use, and marijuana use for those in the categories At risk or above for the substance based on the ASSIST screener. GLMM analysis adjusting for site indicated statistically significant change in the past 30 day prevalence for all 3 substances, with binge drinking decreasing by 9.9 percentage points, and illicit drug use and marijuana use each decreasing by about 14 percentage points (Figure 4). As shown in Figure 5, reductions in days of use were also statistically significant: binge drinking decreased by 0.8 days, and both days of drug use and marijuana use decreased by 1.2 days.

Differential Change by Sociodemographic Characteristics and Risk at Intake

To examine whether change was similar by gender, age, race/ethnicity, risk category, or type of user (misuse of alcohol or drugs, versus misuse of both), interactions were tested for all 6 outcomes within GLMM, adjusting for site. For prevalence outcomes, a near consistent finding was seen in which males, those at high or severe risk for the substance, and misuse of one substance versus misuse of both showed greater past 30 day abstinence than did females, those at relatively lower risk, and those misusing both alcohol and drugs at intake (data not shown). Differential change in the past 30-day prevalence outcomes was not significant by age and race/ethnicity. Similarly, males and those at relatively higher risk at intake (high and severe risk) reported greater reductions in the number of days of use than did females and those at relatively lower risk at intake (data not shown). Age categories, racial/ethnic groups, and type of user did not vary greatly in days reduced, indicating reductions in days of use were fairly consistent among them.

DISCUSSION

This study examined patient alcohol and drug outcomes 6 months after participation in California SBIRT services which were routinely offered to patients at San Diego County EDs and trauma centers. The overall results were consistent with those of other studies which have demonstrated that screening and brief intervention programs are effective at reducing substance use.^4–5 Even after employing a conservative analysis approach that replaced missing follow-up data with intake values, there were statistically significant reductions in all 6 drug and alcohol use outcomes, although the clinical significance of these reductions is not known. Past-month abstinence from binge drinking, use of any illicit drugs, and use of marijuana specifically among those with risky levels of misuse increased by about 10 to 14 percentage points (reductions in the range of 17% to 25%). Days of use in the past 30 days decreased as well, with binging days decreasing by almost a full day, and drug use and marijuana use specifically each decreasing by 1.2 days.

The InSight project in Houston Texas was a well-implemented and thoroughly evaluated SBIRT project that overlaps conceptually and methodological to some degree with the current study.⁶ Therefore, we thought it useful to roughly compare our results to theirs. The reductions seen in the current study are considerably more modest that those reported by the InSight project, which reported an almost 50% reduction in the prevalence of heavy drinking, a 60% reduction in the prevalence of illicit drug use, and close to a 50% reduction in the number of days of heavy alcohol and drug use.⁶Geographical/demographic, methodological, and programmatic differences exist between the CASBIRT and InSight projects which make strict comparisons between the two projects’ results difficult. For example, the projects used different screening instruments and definitions of risk/severity status. Screenings were conducted by professional health care workers in the InSight project, whereas CASBIRT employed paraprofessional health educators for both screening and intervention. InSight’s follow-up rate of 66% was much higher than CASBIRT’s; therefore, large-scale imputation of follow-up values was not necessary for InSight.

For the purposes of potentially informing future SBIRT service delivery, changes by baseline risk and sociodemographic subgroups were examined. The present study found a fairly consistent pattern of greater change among men (than women), among those at relatively higher risk status (versus lower risk), and among those with only one substance of misuse (versus both alcohol and illicit drug misuse). No differential effect was observed by race/ethnicity and age, indicating similar affect of CASBIRT services across age and ethnoracial groups. To some degree, these results parallel those of the InSight study, which reported greater decreases in alcohol outcomes among those at higher risk (although this was not observed in Insight’s drug use outcomes).⁶ This finding in both studies of greater change among those with the greatest problem severity is somewhat surprising, given SBIRT’s primary focus on impacting the large number of non-dependent, relatively lower risk users.³ It may be that, in both studies, those patients with higher risk or severity were more receptive and motivated to make changes than those at relatively lower risk levels. In addition, regression to the mean cannot be ruled out.

As with any research that requires follow-up with patients with risky drug and alcohol use behaviors, there was a concern that patients lost to follow-up were significantly different from those who were contacted at follow-up. Patients lost to follow-up reported higher mean days of binge drinking and higher alcohol and drug risk levels, but were not significantly different with respect to all other variables examined. Therefore, while some patients lost to follow-up may have had slightly more risky behavior patterns, the groups were otherwise quite similar, a surprising finding considering the low response rate.

Although the existing protocol for provision of Screening and Brief Intervention services in San Diego County included some data collection, its design as a service, rather than research program lends itself to several limitations. Since it was a real-world service provision project, randomization to treatment and control groups was not feasible. Without a control group, it is impossible to know if patients’ substance use behaviors would have improved on their own, independent of services delivered. It is also possible that patients experienced test reactivity, whereby the measures themselves prompted the behavioral changes, rather than the interventions. Another potential explanation for decreased risky behaviors is the therapeutic effect of attention alone; that is, it may have simply been the time spent with a health educator that lead to behavioral changes, rather than how that time was spent. The baseline interview was conducted inperson, whereas the follow-up was a telephone interview. To the degree that these 2 methods differ with regard to veracity of reporting, a systematic bias could have been introduced. The large amount of missing follow-up data is a concern, and although imputation was conducted to reduce non-response bias, the use of last observation carried forward in analyses can introduce bias.²² Finally, this study lacks biological confirmation of claims of abstinence or reduction. All data recorded at baseline and follow-up were collected exclusively through patient self-reports. Without biological confirmation, it is likely that some patients exaggerated or misreported their reductions in drug and alcohol use.

Despite the limitations, the present study has many strengths. Although not population-based, CASBIRT attempted universal screening and the convenience sample came from a large, ethnically-diverse patient population with a wide range of medical needs. Linguistically appropriate screening, intervention, and follow-up assessment was available for Spanish speaking Latinos. Interventionists (paraprofessional health educators) were well trained and supervised on an ongoing basis, working side by side with a supervisory Health Educator who provided feedback on an ongoing basis. CASBIRT services were integrated into the acute care setting, and were well received by ED/trauma staff and administrators. The analysis method based on intention to treat principles gives confidence in the results, insofar as nonresponse bias was reduced.

CONCLUSION

Footnotes

Supervising Section Editor: Michael Menchine, MD, MPH
Submission history: Submitted December 7, 2011; Revision received May 4, 2012; Accepted September 6, 2012
Full text available through open access at http://escholarship.org/uc/uciem_westjem
DOI: 10.5811/westjem.2012.9.11551

Address for Correspondence: Susan I. Woodruff, PhD, San Diego State University, School of Social Work, 5500 Campanile Drive, San Diego, CA 92182-4119. Email: swoodruff@projects.sdsu.edu.

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.

REFERENCES

1. National Institute of Drug Abuse (NIDA) Drug abuse and addiction: one of America’s most challenging public health problems. June 2005. Available at:http://archives.drugabuse.gov/about/welcome/aboutdrugabuse/magnitude/. Accessed September 16, 2011.

2. US Department of Health and Human Services (USDHHS) Results from the 2007 national survey on drug use and health: national gindings. Substance Abuse and Mental Health Services Administration (SAMHSA) Office of Applied Studies (OAS). Available at:http://www.oas.samhsa.gov/nsduh/2k7nsduh/2k7results.cfm#Ch7. Accessed September 16, 2011.

3. Substance Abuse and Mental Health Services Administration (SAMHSA)/Center for Substance Abuse Treatment (CSAT) Guide to Substance Abuse Services for Primary Care Clinicians. 1997. Treatment Improvement Protocol (TIP) Series 24. DHHS Publication No. (SMA) 97–3139. Rockville, MD.

4. Babor TF, McRee BG, Kassebaum PA. Screening, brief intervention, and referral to treatment (SBIRT): Toward a public health approach to the management of substance abuse. Subst Abuse.2007;28(3):7–30. et al. [PubMed]

5. Madras B, Compton W, Avula D. Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: Comparison at intake and 6 months later. Drug Alcohol Depend. 2009;99(1):280–295. et al. [PMC free article] [PubMed]

6. InSight Project Research Group. SBIRT outcomes in Houston: Final report on InSight, a hospital district-based program for patients at risk for alcohol or drug use problems. Alcohol Clin Exp Res.2009;33(8):1374–1381. [PubMed]

7. Cunningham RM, Bernstein SL, Walton M. Alcohol, tobacco, and other drugs: future directions for screening and intervention in the emergency department. Acad Emerg Med. 2009;16(11):1078–1088. et al. [PubMed]

8. Cherpitel CJ, Moskalewicz J, Swiatkiewicz G. Screening, brief intervention and referral to treatment (SBIRT) in a Polish emergency department: three-month outcomes of a randomized controlled clinical trial. J Stud Alc Drugs. 2009;70:982–990. et al. [PMC free article] [PubMed]

9. Désy PM, Perhats C. Alcohol screening, brief intervention, and referral in the emergency department: an implementation study. J Emerg Nurs. 2008;34(1):11–19. [PubMed]

10. D’Onofrio G, DeGutis LC. Screening and brief intervention in the emergency department. Alcohol Res Health. 2005;28(2):63–72. [PubMed]

11. Daeppen JB, Gaume J, Bady P. Brief alcohol intervention and alcohol assessment do not influence alcohol use in injured patients treated in the emergency department: a randomized controlled clinical trial. Addiction. 2007;102(8):1224–1233. et al. [PubMed]

12. Havard A, Shakeshaft A, Sanson-Fisher R. Systematic review and meta-analyses of strategies targeting alcohol problems in emergency departments: interventions reduce alcohol-related injuries.Addiction. 2008;103(3):368–376. [PubMed]

13. Bernstein E, Bernstein J, Feldman J. The impact of screening, brief intervention, and referral to treatment on emergency department patients’ alcohol use. Ann Emerg Med. 2007;50(6):699–710. et al.[PubMed]

14. WHO ASSIST Working Group. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction. 2002;97(9):1183–1194. [PubMed]

15. Henry-Edwards S, Humeniuk R, Ali R. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): Guidelines for Use in Primary Care. Geneva: World Health Organization; 2003. et al. (Draft Version 1.1 for Field Testing)

16. Miller WR, Rollnick S. Motivational interviewing: Preparing people for change. 2nd ed. New York, NY: Guilford Press; 2002.

17. Rollnick S, Miller W. What is motivational interviewing? Behav Cog Psychoth. 1995;23:325–334.

18. Darby K, Kinnevy SC. GPRA and the development of performance measures. J Evid Based Soc Work. 2010;7:5–14. [PubMed]

19. Podrasky LA., Benton A. Evaluation lessons learned from a federal grant program that transcend funding agency. Eval Program Plan. 2005;28:355–357.

20. McLellan AT, Luborsky L, O’Brien CP. An improved diagnostic instrument for substance abuse patients: The Addiction Severity Index. J Nerv Ment Dis. 1980;168:26–33. et al. [PubMed]

21. Brown CH, Wang W, Kellam SG. Methods for testing theory and evaluating impact in randomized field trials: intent-to-treat analyses for integrating the perspectives of person, place and time. Drug Alc Depend. 2008;95(Suppl 1):S74–S104. et al. [PMC free article] [PubMed]

22. Mallinckrodt CH, Lane PW, Schnell D. Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Info J. 2008;42:303–319. et al.