Author	Affiliation
Lindsay M. Worth, MPA, MA, LMHC	University of New Mexico, Department of Psychiatric Research, Albuquerque, New Mexico
Wendy Macias-Konstantopoulos, MD, MPH, MBA	Harvard Medical School, Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts
Lauren Moy, PhD	New York University, New York, New York
Harold I. Perl, PhD	Independent Practice, Taos, New Mexico
Cameron Crandall, MD, PhD	University of New Mexico, Department of Emergency Medicine, Albuquerque, New Mexico
Roberta Chavez, MA	University of New Mexico Center on Alcoholism, Substance Use Disorder and Addictions, Albuquerque, New Mexico
Alyssa Forcehimes, PhD	The Change Companies, Carson City, Nevada
Raul Mandler, MD	National Institute on Drug Abuse Clinical Trials Network, Bethesda, Maryland
Michael P. Bogenschutz, MD	New York University, New York, New York

Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

In the United States, the emergency department (ED) is an important healthcare access point, especially for underinsured and underserved populations with reduced access to other sources of care.¹ In 2016 there were an estimated 145.6 million visits to non-federal hospital EDs in the United States,² and a report published in 2010 by the Substance Abuse and Mental Health Services Administration found that almost half of all ED visits were related to drug misuse or dependence.³ Because EDs serve a high volume of individuals with substance use disorders, ED visits present opportunities for screening, brief intervention, and referral to treatment (SBIRT).⁴

There are some distinctive barriers inherent in recruiting individuals with substance use disorders. The rate of recruitment in clinical trials for addiction research is linked to location and size of the recruitment site, the target population, the inclusion and exclusion criteria, and the perceived benefit to the participant of the treatment offered.⁵ The natural inclination for individuals to understate or hide highly stigmatized behaviors presents obstacles in both recruitment and data quality.^6-10 Additionally, patients may decline to participate because of a number of reasons including not feeling well, lack of interest, concerns about confidentiality, and the time-consuming nature of the study.^11-13 Obtaining a representative sample of the population of interest and agreement rates of 70% or more support generalizability of that population.^14-16

Participant compensation is another important consideration. Participants may perceive low compensation as patronizing, while excessive payment can compromise voluntary consent.¹⁷ Yet even though it is important to establish appropriate compensation for participation,¹⁶ it is not the most important factor in securing enrollment.^18-19 Study staff flexibility (eg, taking breaks from study assessments for medical interventions) and rapport-building (eg, expressing compassion) are considered two of the most important determinants in successful recruitment for ED-based clinical trials.^19-25

In medical settings, collecting data from patients with electronic devices, such as tablet or laptop computers, has proven to be an acceptable²⁶ and time-saving²⁷ method for gathering information. Allowing participants to complete behavior assessments electronically minimizes feelings of embarrassment and judgment and improves a sense of privacy compared to study staff interview methods.^28-30 Several studies suggest that electronic screening outperforms verbal screening in detecting adversity across a spectrum of potentially sensitive topics among ED patient populations. ^28-30

In addition to its role in data collection and data quality, technology has also proven useful with participant tracking in longitudinal studies. Both free and fee-based online search tools, online public records, and social networking sites are useful for locating participants.^21,31 Longitudinal ED-based research requires a variety of retention strategies including collecting adequate participant contact information; making repeated contact attempts for follow-up visit completion including in-person, phone calls, mailed letters, and web-based strategies; and allowing for flexibility in the location of follow-up completion.³²

Although extensive research has been done on SBIRT in alcohol use disorder, much less SBIRT research has been done with other substance use disorders.^1,33-34 To address this gap, we conducted a multisite trial “Screening, Motivational Assessment, Referral, and Treatment in Emergency Departments (SMART-ED)” through the National Institute on Drug Abuse Clinical Trials Network (NIDA CTN) to compare the effectiveness of 1) a brief motivational interviewing³⁶ intervention; 2) screening, assessment and referral; and 3) minimal screening only in an ED sample of patients with probable SUD. Conducting multisite clinical trials with complex behavioral interventions in the ED presents numerous challenges in recruitment and retention of participants. We describe our recruitment and retention experiences and the lessons learned while conducting this ED-based, multisite SBIRT study.

 

Population Health Research Capsule

What do we already know about this issue?

Clinical trial recruitment and retention of individuals who use substances are challenging in any setting and can be particularly difficult in ED settings.

What was the research question?

How can we maximize recruitment and retention of individuals who use substances who are patients in the ED?

What was the major finding of the study?

Recruitment goals were met: 1,285 were enrolled in the study and the 3-, 6-, and 12-month retention rates for this study were 89%, 86%, and 81%, respectively.

How does this improve population health?

Successful recruitment and retention allow for a better understanding of how an intervention in the ED impacts current and future substance use.

 

METHODS

Recruitment and initial baseline assessment for the SMART-ED study took place between October 2010–February 2012 in six urban academic EDs in the US, each of which partnered with a node of the NIDA CTN (Trial Registration www.clinicaltrials.gov Identifier: NCT01207791).³⁴ Three sites were on the East Coast and one in each of the Midwest, South, and Southwest regions (Table 1).³⁴ Site selection criteria for this study included the following: EDs that collectively had a patient population broadly representative of the US population; an adequate number of ED patients with SUD; ED research experience and infrastructure; access to a referral network for specialty addiction treatment; and EDs with the sufficient staff and willingness to participate and implement the study protocol.

For this study our goal was to enroll 1,285 participants across sites over a nine-month period and complete follow-up visits at three, six, and 12 months post baseline.

We used tablet computers for a number of project activities: 1) to screen and collect data; 2) access the electronic health record; and 3) collect participant contact information. In addition to eliminating the need for paper forms, tablet computers allowed study staff to receive immediate notification of participant eligibility and group randomization. Study staff approached potentially eligible patients after triage. The ED tracking boards helped to locate patients. Table 2 lists the complete inclusion and exclusion criteria. Every effort was made to meet with patients in a private room, although this was often a challenge. At one site, study staff placed a partition in the corner of the waiting room and used this space to screen patients for the study.

Once they were enrolled, we collected participant contact information including 1) residential and mailing address, 2) phone number(s), 3) email address, 4) Social Security number, 5) place of employment, and 6) contact information for two “locators” (ie, persons who would know how to contact the participants during the course of the study). If they were not able to provide sufficient contact information, they were not eligible to participate (Table 2). Although Social Security numbers were gathered as a part of the form used for this study, they were not used to track participants in this study. Participants were randomly assigned to one of three cohorts: 1) brief motivational interviewing intervention; 2) screening assessment and referral; or 3) minimal screening only.

Compensation for completing the baseline and each follow-up assessment was $50 and $75, respectively. Baseline assessments took between 60–120 minutes, and follow-up assessments ranged between 90–210 minutes to complete. At a separate location from the ED, staff (who were blinded to treatment assignment) conducted follow-up assessments. Appointment cards, maps, and study contact information were provided at the initial ED baseline visit, and reminder calls were made prior to each follow-up visit.

When a participant attended their follow-up study visit, staff were required to review and update all participant contact information. If a participant did not attend their follow-up visit, staff would, in order, do the following: attempt to reach the participant by varying times of call attempts; send email and text message; mail a letter to the participant; and contact the participant’s locators. If staff were unsuccessful in reaching the participant, they would conduct an internet search to try to obtain more current contact information. At one site, follow-up staff attempted to locate the participant in person at their home address. Across and within sites, there did not appear to be a single approach to locating participants and scheduling follow-ups that emerged as superior to another approach.

Follow-up staff documented all contact attempts, regardless of success, in the “Contact Log,” which included date and time of attempted contact and a description and result of the attempt (Figure). Documentation allowed staff to see what type of contacts had already been attempted. Unsuccessful tracking methods and bothering a participant or locator who may have been recently contacted were not repeated. In addition to using the participant contact information provided to help locate participants for follow up, other accommodations such as meeting at a more convenient location (depending on institutional review board [IRB] rules), varying times to meet, or a phone option for conducting follow-up were offered. Study staff also emphasized that participation in the study was voluntary.

Participant incarceration is an expected occurrence that poses challenges to completing follow-up. In anticipation of this reality, we obtained Office for Human Research Protections approval to conduct follow-up visits with participants who became incarcerated after enrollment, and the SMART-ED study sites pursued IRB approval. Ultimately, the ability of the study staff to follow up with the participant depended upon the study site’s IRB regulations, type of consent obtained from the participant, and the rules of the confining correctional facility.

Another challenge to retention was the occasional participant request for withdrawal when contacted to schedule follow-up visit appointments. In such cases, we honored the request and mailed the participant a letter confirming their decision to withdraw, providing the study’s contact information, reviewing the benefits of participation, and inviting them to contact the study should they change their mind.

Ongoing study staff training occurred throughout the study, emphasizing the importance of 1) recruitment study staff approaching all potentially eligible patients (post-triage) without regard to diagnosis, thus, improving the representativeness of the sample; and 2) follow-up study staff reviewing the methods for contacting participants. Additionally, weekly recruitment and retention calls with all sites provided a forum to discuss any recruitment and retention issues, clarify procedures, and troubleshoot unanticipated problems.

Over the course of the study, we made several adjustments to improve participant retention. These adjustments included decreasing assessment time at follow-up (ie, fewer assessments administered at follow-up), expanding the time windows for completing assessments, and providing incentive compensation to study staff at sites who achieved an 85% follow-up rate or higher. The original four-week time window for completing follow-up assessment (two weeks before and two weeks after the ideal follow-up date) was opened to allow a participant six weeks to complete follow-up visit (two weeks prior and four weeks post the ideal follow-up date). For example, if someone’s follow-up was due on February 14, they could be seen as early as February 1 or as late as March 14 for their follow-up.

This expanded follow-up time window offered participants increased flexibility and convenience when scheduling their visits without compromising follow-up data integrity. The incentive compensation offered to study staff who achieved 85% follow-up rates or higher was in the form of a $5 gift card to a coffee shop for each staff member involved in follow-ups at that site. This amount was felt to increase team motivation and promote friendly competition across the study sites to complete follow-up visits with participants, without encouraging coercive practices or dishonest reporting.

RESULTS

Sites recruited participants for this study over a 16-month period during which a total of 20,762 patients were approached for an initial screening. Of those, 15,224 (73.3%) patients gave verbal consent to anonymously complete an electronic screening questionnaire to determine eligibility. Based on eligibility, willingness to participate, and ability to continue, 1,285 patients were enrolled in this study, on target with recruitment projections. We excluded patients who had an incomplete screen (252), fell below the cutoff score of the Drug Abuse Screening Test for problematic drug use (12,888), failed to meet inclusion criteria (64), did not complete consent (725), or withdrew prior to randomization (10). Table 3 provides an overview of study participant characteristics.

Tracking and retention occurred over a 29-month period during which staff completed 3,179 follow-up assessments. The 3-, 6-, and 12-month retention rates for this study were 89%, 86%, and 81%, respectively. Follow-up rates did not vary by group assignment (Table 4). Aside from being unreachable for follow-up, other reasons for missed follow-ups included incarceration, study withdrawal, and death.

As many as 70 contact attempts were made for a few participants before they completed a follow-up. Follow-up staff made on average 26 contact attempts per participant to schedule a follow-up appointment. Contact attempts included making phone calls to participants and locators; texting; sending letters and email messages; conducting online searches to include searching obituaries and incarceration websites; visiting the participant’s home; and on occasion, if approved by the local IRB, sending private messages on Facebook. Phone calls were the most common method used to contact a participant. Varying the time of calling and the days when a participant was called increased the success of reaching and scheduling follow-ups with participants.

Of the 3,176 follow-up assessments completed, 2,918 (91.8%) were in person and 261 (8.2%) over the telephone. We identified 64 participants incarcerated at some point in the follow-up period. Comprehensive study results can be found in the author MB’s 2014 primary outcomes paper.³⁵

DISCUSSION

The population in this study included ED patients with SUD. Retention at the three-month follow-up was 89% and remained above 80% for subsequent follow-ups. Site selection based on predetermined criteria, inclusion and exclusion criteria that included criteria that increased the likelihood of successful follow-up with participants, adequate compensation for study visits, ongoing training and monitoring of recruitment and retention efforts, effective use of technology (eg, tablet computers), and flexibility in enrollment and conducting follow-up assessments were factors considered to support successful recruitment and retention of participants. Urban sites with a large ED census of patients and availability of substance use treatment programs were also key factors in site selection for this study (Table 1). Additionally, patient population characteristics were considered for generalizability of the study (Table 3).

We chose certain inclusion and exclusion criteria to support successful follow-up with participants (Table 2). Criteria for inclusion that contributed to ease of contacting participants for follow-up included access to a phone, residence within 50 miles from the location of follow-up visits, and ability to provide sufficient contact information (required to provide at least two reliable locators). “Locators” are individuals who may have contact information for the participant if the follow-up staff are not able to reach the participant. Although the same recruitment and retention guidelines were used across study sites, the success of using these guidelines varied; methods that worked well at one site were not always effective across sites. It was important to allow sites to adapt general study guidelines that best suited their population and environment.

Staff flexibility at enrollment (eg, meeting patients when they felt well enough to complete assessments and were not busy with medical care) and follow-up (eg, completing follow-ups by phone or in the community and when convenient to the participant), was the single most likely factor to have mediated the success in recruitment and retention. We did not gather data on the participant’s opinion of using tablet computers, but it is hoped that this minimized any feelings of embarrassment or perceived stigmatization in reporting sensitive drug use information. Similarly, compensation is presumed to have been acceptable as there were no complaints about compensation being too little or too much over the course of the study.

The average number of contact attempts was 26 and ranged up to 70 to reach a participant for a follow-up visit. Most commonly, participants or their locators were reached by phone or via letters sent, but conducting online searches and using social media (ie, Facebook) to connect with participants were important access points as well. Both the amount of time and effort this intense level of follow-up entailed and the potential for participants to feel harassed or coerced to participate must be seriously considered. To ensure participants do not feel harassed or coerced, it is important to emphasize that participation is voluntary. Additionally, documentation of contact attempts ensures that participants who have refused to participate are not contacted again and contact methods that have been unsuccessful are not repeated. The level of effort to contact participants is time-consuming, and it is important to appropriately plan for this. Likewise, thoughtful and strategic outreach to enrolled patients requires careful internal documentation and communication within the follow-up team, also requiring time and effort.

For future research in EDs we would recommend using wireless internet data cards rather than relying on wireless connections to the ED network. Losing internet connection became a point of frustration for both participants and enrollment study staff conducting interviews as they would sometimes lose data and be forced to repeat parts of the baseline assessment. A wireless internet data card allows users to access online information anytime and anywhere without getting disconnected from the network.

Obtaining participant consent upfront for texting, emailing, and searching for participants through publicly available data including social media networks such as Facebook is recommended. We implemented this midway through the study, and some study sites had difficulty in gaining permission from their IRBs to use these resources without participant consent. We would also recommend seeking IRB approval and participant consent to continue working with enrolled study participants who might become incarcerated during the study.

LIMITATIONS

Allowing follow-ups to occur outside the target follow-up date may have inflated retention results slightly, but because these windows were well-defined and narrow, the impact on data was minimal and we feel the benefit to follow-up rates and data collection justifies the approach.

CONCLUSION

Consistent with the research, we found that recruitment of ED patients with substance use disorder and retention of these participants in a longitudinal study required a multifaceted process. We found that certain methods for recruitment and retention were useful across sites (eg, exclusion criteria, consent for contact through social media, IRB approval of procedures to retain incarcerated patients), but it was also important to consider the location of a study site in tailoring and developing additional strategies.”

Footnotes

Section Editor: Wirachin Hoonpongsimanont, MD, MSBATS

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Lindsay M. Worth, MPA, MA, LMHC, University of New Mexico, Department of Psychiatric Research, 1101 Yale Blvd NE, Suite 2212, Albuquerque, NM 87106. Email: lindsayw@salud.unm.edu. 3 / 2023; 24:228 – 235

Submission history: Revision received April 19, 2022; Submitted November 2, 2022; Accepted November 16, 2022

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. This article submission is based on recruitment and retention results from the Screening, Motivational Assessment, Referral, and Treatment in Emergency Departments (SMART-ED) research study supported by the National Drug Abuse Treatment Clinical Trials Network (CTN) protocol (Trial Registration www.clinicaltrials.gov Identifier:NCT01207791). There are no conflicts of interest to declare.

REFERENCES

1. Cunningham RM, Bernstein SL, Walton M, et al. Alcohol, tobacco, and other drugs: future directions for screening and intervention in the emergency department. Acad Emerg Med. 2009;16(11):1078-88.

2. Rui P, Kang K, Ashman JJ. National Hospital Ambulatory Medical Care Survey: 2016. Available at: https://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2016_ed_web_tables.pdf. Accessed April 5, 2017.

3. The DAWN Report: Highlights of the 2010 Drug Abuse Warning Network (DAWN) Findings on Drug-Related Emergency Department Visits. 2012. Available at: https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_electronic_sources.html. Accessed December 12, 2013.

4. D’Onofiro BE. A promising approach for emergency departments to care for patients with substance use and behavioral disorders. Health Affairs. 2013;32(12):2122-8.

5. Wakim PG, Rosa C, Kothari P, et al. Relation of study design to recruitment and retention in CTN trials. Am J Drug Alcohol Abuse. 2011;37(5):426-33.

6. Cepeda A, Valdez A. Ethnographic strategies in the tracking and retention of street-recruited community-based samples of substance using hidden populations in longitudinal studies. Subst Use Misuse. 2010;45(5):700-16.

7. Rockett IRH, Putnam SL, Jia H, et al. Declared and undeclared substance use among emergency department patients: a population-based study. Addiction. 2006;101(5):706-12.

8. Vitale S, van de Mheen D. Illicit drug use and injuries: a review of emergency room studies. Drug Alcohol Depend. 2006;82(1):1-9.

9. Lee MO, Vivier PM, Diercks DB. Is the self-report of recent cocaine or methamphetamine use reliable in illicit stimulant drug users who present to the emergency department with chest pain?. J Emerg Med. 2009;37(2):237-41.

10. Schuckman H, Hazelett S, Powell C, et al. A validation of self-reported substance use with biochemical testing among patients presenting to the emergency department seeking treatment for backache, headache, and toothache. Subst Use Misuse. 2008;43(5):589-95.

11. Blow FC, Walton MA, Barry KL, et al. Alcohol and drug use among patients presenting to an inner-city ED: a latent class analysis. Addict Behav. 2011;36(8):793-800.

12. Booth BM, Weber JE, Walton MA, et al. Characteristics of cocaine users presenting to an emergency department chest pain observation unit. Acad Emerg Med. 2005;12(4):329-37.

13. Wilets I, O’Rourke M, Nassisi D. How patients and visitors to an urban emergency department view clinical research. Acad Emerg Med. 2003;10(10):1081-5.

14. Melberg HO, Humphreys K. Ineligibility and refusal to participate in randomized trials of treatments for drug dependence. Drug Alcohol Rev. 2010;9(2):193-201.

15. Broderick KB, Ranney ML, Vaca FE, et al. Study designs and evaluation models for emergency department public health research. Acad Emerg Med. 2009;16(11):1124-31.

16. Patel MX, Doku V, Tennakoon L. Challenges in recruitment of research participants. Advances in Psychiatric Treatment. 2003;9:229-38.

17. Evans E, Murphy DA, Grella CE, et al. Regulatory issues encountered when conducting longitudinal substance use disorder research. J Drug Issues. 2008;38(4):1003-26.

18. Farabee D, Hawken A, Griffith P. Tracking and incentivizing substance use disorders in longitudinal research. Results of a survey of National Institute on Drug Abuse-funded researchers. J Addict Med. 2011;5(2):87-91.

19. Felsen CB, Shaw EK, Ferrante JM, et al. Strategies for in-person recruitment: lessons learned from a New Jersey Primary Care Research Network (NJPCRN) study. J Am Board Fam Med. 2010;23(4):523-33.

20. Svendsen TS, Erga AH, Hagen E, et al. How to maintain high retention rates in long-term research on addiction: a case report. J Soc Work Pract Addict. 2017;17(4):374-87.

21. Clough A, Wagman J, Rollins C, Barnes J, et al. The SHARE project: maximizing participant retention in a longitudinal study with victims of intimate partner violence. Field Methods. 2011;23(1):86-101.

22. Walton MA, Ramanathan CS, Reischl TM. Tracking substance use disorders in longitudinal research: understanding follow-up contact difficulty. Am J Community Psychol. 1998;26(2):233-53.

23. Cottler LB, Compton WM, Ben-Abdallah A, et al. Achieving a 96.6 percent follow-up rate in a longitudinal study of drug abusers. Drug Alcohol Depend. 1996;41(3):209-17.

24. Marmor JK, Oliveria SA, Donahue RP, et al. Factors encouraging cohort maintenance in longitudinal study. J Clin Epidemiol. 1990;4(6):531-5.

25. Woolard RH, Carty K, Wirtz P, et al. Research fundamentals: Follow-up of subjects in clinical trials: addressing subject attrition. Acad Emerg Med. 2004;11(8):859-66.

26. Abernathy AP, Herndon JE, Wheeler JL, et al. Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv Res. 2008;43(6):1975-91.

27. Hess R, Santucci A, McTigue K, et al. Patient difficulty using tablet computers to screen in primary care. J Gen Intern Med. 2007;23(4):476-80.

28. Trautman DE, McCarthy ML, Miller N, et al. Intimate partner violence and emergency department screening: computerized screening versus usual care. Ann Emerg Med. 2007;49(4):526-34.

29. Gottlieb L, Hessler D, Long D, et al. A randomized trial on screening for social determinants of health: the iScreen study. Pediatrics. 2014;134(6):e1611-8.

30. Cullen D, Woodford A, Fein J. Food for thought: a randomized trial of food insecurity screening in the emergency department. Acad Pediatr. 2019;19(6):646-51.

31. Cotter RB, Burke JD, Loeber R, et al. Innovative retention methods in longitudinal research: a case study of the developmental trends study. J Child Fam Stud. 2002;11(4):485-98.

32. Roche JS, Clery MJ, Carter PM, et al. Tracking assault-injured, drug-using youth in longitudinal research: follow-up methods. Acad Emerg Med. 2018;25(11):1204-15.

33. Madras BK, Compton WM, Avula D, et al. Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: comparison at intake and 6 months later. Drug Alcohol Depend. 2009;99(1–3):280-95.

34. Babor TF, McRee BG, Kassebaum PA, et al. Screening, brief intervention, and referral to treatment (SBIRT): toward a public health approach to the management of substance use disorder. Subst Abuse. 2007;28(3):7-30.

35. Bogenschutz MP, Donovan DM, Mandler RN, et al. Brief intervention for patients with problematic drug use presenting in emergency departments: a randomized clinical trial. JAMA Intern Med. 2014;174(11):1736-45.

36. Miller WR, Rollnick S. Motivational Interviewing: Helping People Change. 2013.