Author	Affiliation
Gia Ciccolo, MPH	Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts
Alexa Curt, BA	Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts
Carlos A. Camargo, MD, DrPH	Harvard Medical School, Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts
Margaret Samuels-Kalow, MD, MPhil, MSHP	Harvard Medical School, Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts

Introduction
Methods
Results
Limitations
Conclusion

INTRODUCTION

With recent policy changes, including the movement toward accountable care organizations as health delivery systems, there has been an increasing priority placed on both screening for social risk factors,^1-3 (defined as the “adverse social conditions that are associated with poor health”)⁴ and assessing social needs, or the patient’s prioritization of social interventions.⁴ Although emergency department (ED) patient populations have a high prevalence of social risk,⁵ optimal strategies for identifying these factors within the busy and time-limited setting of the ED have yet to be described.

Currently, a major barrier to identifying and addressing social risk and social need in EDs is the wide variety of different tools used across studies⁵ and the lack of a “criterion standard” assessment. As stated in a systematic review of social needs in the ED, “a concise yet comprehensive material needs [social risk and social need] screening tool has not yet been created and validated for ED patients.”⁵ Efforts to standardize screening questions³ have been limited by the copyright restrictions on recommended questions and the total length of the survey, which limits the applicability of most tools in the ED. To date, studies using shorter tools have focused on screening for an individual social risk (eg, food insecurity) rather than assessing the multiple social risks that may affect patients’ health.^5-7 Few tools assess both social risks and social needs in one assessment. The lack of consistency in ascertainment techniques and screening tools presents an obstacle to researchers, policy makers, and health systems to design interventions to address social risk and social need in the ED patient population.

In developing a social screening tool, it is of critical importance to ensure the screening questions are easy to understand and interpreted in a consistent manner.⁸ Cognitive interviewing has been proposed as a method for improving the validity of response processes, by allowing the researcher to understand how participants interpret questions.⁷ The hybrid cognitive interview methodology involves two parts. The first is “think aloud” in which the intention of the interviewer is to guide the participant in providing verbal insight into his/her thought process and understanding while walking through the survey. The second is “verbal probing” in which the participant responds to specific probes concerning understanding of certain areas of the survey. Survey changes informed by this process ensure that the respondent is interpreting and responding to questions as intended in the survey.⁸ Survey changes based on information from cognitive interviewing data, such as those in this study, are used to clarify the intention of the question to the reader, improve survey comprehension,⁴ and have been used to optimize other self-report assessment tools.^5,9 Modifying a screening tool using this technique can thus increase the ability of the tool to assess risks and needs consistently.

The goal of this study was to develop and optimize a social risk and social need screening tool for ED patients that would be both brief and understandable to patients in order to connect them to potential interventions.

METHODS

We conducted a cognitive interview study with patients and parents of patients in the ED. The initial 16-question survey was developed in both English and Spanish, through a systematic review of existing social risk and social need screening tools using web-based searches and PubMed. Questions were included if they addressed one of the five core domains of the Accountable Health Communities screening tool: 1) food insecurity; 2) housing instability; 3) transportation needs; 4) utility needs; and 5) interpersonal safety.³ This tool was available in the public domain, without copyright restrictions.

A cognitive interview guide of open-ended questions (Online Supplement Table 1) was developed by the study team, piloted and refined. Edits were made to best capture patient understanding and feedback concerning the proposed questions. The guide was developed in English and Spanish, with interviews in the patient’s choice of language. Iterative cycles of cognitive interviews were performed and recorded. Transcripts were reviewed by investigators, the questionnaire was modified in response, and re-tested in a subsequent cycle of interviews until thematic saturation was achieved and no novel feedback was obtained.

 

Population Health Research Capsule

What do we already know about this issue?

Emergency department (ED) patient populations have a high prevalence of social risk yet optimal strategies for identifying these factors remain to be described.

What was the research question?

To develop and optimize a brief and understandable social risk and social need screening tool for ED patients.

What was the major finding of the study?

Patient informed changes supported a more concise, understandable, and dependable screening tool.

How does this improve population health?

This screening tool serves to assess social risk and social need in ED patients, enable linkage to relevant resources and improve overall health outcomes.

 

Patients were purposively sampled by language spoken (English or Spanish) as well as health literacy level (adequate or limited) in order to reduce bias in representation in the patient population and recruited from a large, urban ED. A bilingual research assistant screened patients for eligibility. Eligibility criteria included adult patients or parent/guardians of pediatric patients, fluency in either English or Spanish, provider approval for approach, and plans for discharge home. We excluded patients on an involuntary mental health hold or with active intoxication.

Patient participants completed a brief demographic survey and a health literacy assessment (Newest Vital Sign)¹⁰ in either English or Spanish, as well as the cognitive interview. Cognitive interviewing used the “think aloud” and “verbal probing” methods and was employed to understand the participant’s thought process, while going through the survey and comprehension of each survey question. The interviewer received cognitive interview training, and direct feedback following each interview, from a researcher trained in the technique. All interviews were recorded and professionally transcribed. All “think aloud” and “verbal probing” responses were reviewed and a cycle was complete when no new responses were given. All changes to the survey were made by consensus of the study team. The study was approved by the Institutional Review Board (IRB) of Partners HealthCare.

RESULTS

In total, 16 patients completed cognitive interviews over the course of three cycles. Of the 16 participants, four (25%) were primarily Spanish-speaking and five (31%) were categorized as having limited health literacy (Online Supplement Table 2). Based on participant feedback, the survey was reduced and simplified from 16 questions, concerning the five domains of social risk and social need to 10 questions concerning four of the five original domains; neighborhood safety was excluded (Table).

Questions concerning neighborhood safety were removed as participants did not have consensus on the meaning of “safety.” For example, some participants felt these questions were referencing crime in the surrounding area [“You can safely walk around your neighborhood without feeling endangered” (adequate health literacy)]. Others felt they were referring to domestic violence [“Que se refiere como que si alguien que vive con un hijo, me lo golpeara, me va a hacer un maltrato agresivo como ¿violencia doméstica?”] [That it refers to like if someone that lives with a son/daughter, were to hit him/her, [or] is going to aggressively mistreat me like domestic violence? (adequate health literacy)] and [“I would assume, there, that you are referring to something that would be more like domestic abuse” (adequate health literacy)]. The lack of consistency in definition caused difficulty in interpreting a positive answer, and determining the appropriate community resource for response. The alternative option of adding further questions to clarify the type of safety need would have made the survey excessively long for ED use and overlapped with existing ED screening protocols for intimate partner violence. For these reasons, in addition to the limited community resources available to address safety, we removed the domain of safety from the question set.

For other domains, participants mentioned confusion in the wording and subsequent description of response options [“I think the wording is a little confusing after you’ve just gone through questions that are more direct yes or no…And so I had to switch gears and be like, ‘Oh wait. Okay. So now it’s often true, sometimes true, never true thing…’ (Adequate health literacy)]. Responses were thus simplified to binary options for improved participant understanding and ease in taking the survey [“Again, I just don’t like those sometimes, nevers, often, always. I think people get thrown off with that.” (Adequate health literacy)]. Questions were also removed for similarity to one another [“3a, I guess it’s fine. 3B is fine as well. They’re both pretty similar” (Adequate health literacy)].

LIMITATIONS

Limitations of the study include recruitment limited to those who spoke English or Spanish. In this study, English- and Spanish-speaking patients had similar survey feedback; thus, all changes were made to both versions of the survey. The tool will need to be translated and tested in other languages. Interviews with participants with limited health literacy tended to be shorter with less feedback provided, suggesting that additional techniques to improve cognitive interviewing may be needed in this population. Social risk and social need screening results were not collected from the participants, so we cannot compare perceptions of the question to measured risk or need. We were only able to interview to thematic saturation across the categories of language and health literacy, which were chosen because they were believed to have the greatest impact on patient comprehension of the questions. Additionally, we do not have data on those who declined participation. Therefore, we were unable to compare those who did and did not participate in the study. Because there is no “criterion standard” for social risk and social need assessment,¹¹ a larger study to understand the performance of the questions compared to other measurements of socioeconomic status, social risk, and social need will be the next step to better understand performance of this screening tool.

CONCLUSION

The cognitive interviews provided important information concerning how to improve an assessment tool for measuring social risk and social need in the ED. After addressing a variety of the study participant concerns (including word choice, response categories, terminology, and question clarity), the final assessment tool (Online Supplement Final Survey) as compared to the original version, is more concise, understandable, and more likely to measure these factors as intended. Importantly, this tool includes both social risk and social need and was developed in both English and Spanish and among patients with a range of health literacy.

This short screening tool was developed to be of use to ED clinicians attempting to link patients to community resources, health system administrators developing programs to address adverse social determinants of health, and researchers working to improve care and outcomes for patients with social risk and social need. Given the importance and goal of integration of social determinant measures in clinical practice,^12,13 we encourage future work to focus on testing the tool across multiple EDs, comparison with population level data, as well as implementation-science work regarding best practices for screening patients, and connecting them to appropriate community resources to improve health outcomes.

Footnotes

Section Editor: Erik Anderson, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Margaret Samuels-Kalow, MD, MPhil, MSHP, Harvard Medical School, Massachusetts General hospital, Department of Emergency Medicine, 125 Nashua Street, Suite 920, Boston, MA 02114. Email: msamuels-kalow@partners.org. 9 / 2020; 21:1170 – 1174

Submission history: Revision received January 15, 2020; Submitted May 12, 2020; Accepted May 12, 2020

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. This work was supported by the Emergency Medicine Foundation. Dr. Samuels-Kalow is supported by the Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Advancing Translational Sciences, National Institutes of Health Award UL 1TR002541) and financial contributions from Harvard University and its affiliated academic healthcare centers. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic healthcare centers, or the National Institutes of Health. There are no conflicts of interest to declare.

REFERENCES

1. Garg A, Sheldrick RC, Dworkin PH. The inherent fallibility of validated screening tools for social determinants of health. Acad Pediatr. 2018;18(2):123-4.

2. Prioritizing social determinants of health in Medicaid ACO programs: a conversation with two pioneering states. 2018. Available at: https://www.chcs.org/prioritizing-social-determinants-health-medicaid-aco-programs-conversation-two-pioneering-states/. Accessed June 13, 2018.

3. Billioux A, Verlander K, Anthony S, et al. Standardized screening for health-related social needs in clinical settings: the accountable health communities screening tool. 2017. Available at: https://nam.edu/wp-content/uploads/2017/05/Standardized-Screening-for-Health-Related-Social-Needs-in-Clinical-Settings.pdf. Accessed August 14, 2019.

4. Alderwick H, Gottlieb LM. Meanings and misunderstandings: a social determinants of health lexicon for health care systems. Milbank Q. 2019;97(2):407-19.

5. Malecha PW, Williams JH, Kunzler NM, et al. Material needs of emergency department patients: a systematic review. Acad Emerg Med. 2018;25(3):330-59.

6. Bottino CJ, Rhodes ET, Kreatsoulas C, et al. Food insecurity screening in pediatric primary care: Can offering referrals help identify families in need?. Acad Pediatr. 2017;17(5):497-503.

7. Martel ML, Klein LR, Hager KA, et al. Emergency department experience with novel electronic medical record order for referral to food resources. West J Emerg Med. 2018;19(2):232-7.

8. Ryan K, Gannon-Slater N, Culbertson M. Improving survey methods with cognitive interviews in small- and medium-scale evaluations. Am J Eval. 2012;33(3):414-30.

9. Samuels-Kalow ME, Rhodes KV, Henien M, et al. Development of a patient-centered outcome measure for emergency department asthma patients. Acad Emerg Med. 2017;24(5):511-22.

10. Osborn CY, Weiss BD, Davis TC, et al. Measuring adult literacy in health care: performance of the newest vital sign. Am J Health Behav. 2007;31:S36-46.

11. Beck AF, Cohen AJ, Colvin JD, et al. Perspectives from the society for pediatric research: interventions targeting social needs in pediatric clinical care. Pediatr Res. 2018;84(1):10-21.

12. Seymour CK, Griffin C, Holmes SM, et al. Structural differential – a 32 year old man with persistent wrist pain. N Engl J Med. 2018;379(25):2385-8.

13. Metzi JM, Hansen H. Structural competency: theorizing a new medical engagement with stigma and inequality. Soc Sci Med. 2014;103:126-33.