| Author | Affiliation |
|---|---|
| Sheryl L. Heron, MD, MPH | Emory University School of Medicine, Department of Emergency Medicine, Atlanta, GA |
| Dahlia M. Hassani, MD | Emory University School of Medicine, Department of Emergency Medicine, Atlanta, GA |
| Debra Houry, MD, MPH | Emory University School of Medicine, Department of Emergency Medicine, Atlanta, GA |
| Tammie Quest, MD | Emory University School of Medicine, Department of Emergency Medicine, Atlanta, GA |
| Douglas S. Ander, MD | Emory University School of Medicine, Department of Emergency Medicine, Atlanta, GA |
ABSTRACT
Introduction:
To use 360-degree evaluations within an Observed Structured Clinical Examination (OSCE) to assess medical student comfort level and communication skills with intimate partner violence (IPV) patients.
Methods:
We assessed a cohort of fourth year medical students’ performance using an IPV standardized patient (SP) encounter in an OSCE. Blinded pre- and post-tests determined the students’ knowledge and comfort level with core IPV assessment. Students, SPs and investigators completed a 360-degree evaluation that focused on each student’s communication and competency skills. We computed frequencies, means and correlations.
Results:
Forty-one students participated in the SP exercise during three separate evaluation periods. Results noted insignificant increase in students’ comfort level pre-test (2.7) and post-test (2.9). Although 88% of students screened for IPV and 98% asked about the injury, only 39% asked about verbal abuse, 17% asked if the patient had a safety plan, and 13% communicated to the patient that IPV is illegal. Using Likert scoring on the competency and overall evaluation (1, very poor and 5, very good), the mean score for each evaluator was 4.1 (competency) and 3.7 (overall). The correlations between trainee comfort level and the specific competencies of patient care, communication skill and professionalism were positive and significant (p<0.05).
Conclusion:
Students felt somewhat comfortable caring for patients with IPV. OSCEs with SPs can be used to assess student competencies in caring for patients with IPV.
INTRODUCTION
Coker et al.1 reported 29% of women have experienced physical, sexual or psychological intimate partner violence (IPV) in their lifetime. Additionally, according to the Bureau of Justice Statistics 42% of women victimized by an intimate partner sustained injuries, with females comprising 58% of murder victims.2 Emergency departments (ED) and other primary care providers are often the entry point for individuals experiencing family violence.3–11 Therefore, healthcare providers should play a vital role in ensuring that their patients receive the assistance they need to address violence in the home.
Recently the Institute of Medicine (IOM) compiled a comprehensive review of the current state of family violence education in the IOM’s 2001 report, “Confronting Chronic Neglect: The Education and Training of Health Care Professionals on Family Violence.” In addition, the IOM made specific recommendations for family violence curricular development.4 As noted in the IOM report, basic competency skills for health providers, as recommended by the American Association of Colleges of Nursing, can serve as template for describing competency skills for all healthcare professionals.4 In our educational setting, there had been sporadic teaching of family violence in the M1 and M2 years. An informal survey of our course directors revealed three hours over four years dedicated to teaching family violence.
There is increasing commitment by the academic community to improve IPV education in the United States (U.S.). The Liaison Committee on Medical Education accreditation standards state “the curriculum must prepare students for their role in addressing the medical consequences of common societal problems, for example, [by] providing instruction in the diagnosis, prevention, appropriate reporting and treatment of violence and abuse.”12 Despite the importance, the literature is sparse with examples on how to teach this topic. Typically, the lectures are sporadic and interspersed between courses that may serve as the appropriate vector for teaching, i.e. teaching child maltreatment in pediatrics or IPV in the women’s health module.4 One university instituted formal lectures on IPV, including a mock patient evaluation, during the first year medical school curriculum.13
Observed Structured Clinical Examinations (OSCE) that use standardized patients (SP) have been used for over 30 years to assess clinical skills in increasingly high stakes settings.14 Both OSCE and SPs have been shown to be effective in training and assessing core components of clinical skills for topics typically considered difficult communications. With respect to IPV, SPs have been used in the OSCE format to teach IPV skills to fourth year students rotating in the primary care setting. This technique has been shown to be an effective modality to teach them core concepts in IPV and improve clinical skills with lasting effects.20 While competency-based evaluation is the goal on an emergency medicine (EM)clerkship, OSCE models in the emergency setting have not yet been widely developed.21 Our review of the literature indicates that there is only one research study regarding the reliability of OSCE when applied to EM resident training in the U.S.22 In addition, Johnson and Reynard23, assessed OSCE as an evaluative method in Great Britain in 1994 with undergraduate students in EM. Another study in Israel developed a national domestic violence (DV) experiential training program for physicians and noted that they improved their capabilities to manage DV cases.24
The purpose of this study was to implement and assess an IPV OSCE module using SPs for fourth year medical students during their EM rotation. Secondary objectives were to measure competency in IPV, comfort (self-efficacy), communication skills and professionalism.
METHODS
Study Design
This was a University Institutional Review Board (IRB) approved prospective, observational study. All students provided informed verbal consent in compliance with our university IRB.
Study Methodology
We assessed IPV knowledge and skills as part of an OSCE at the end of a four-week EM fourth year rotation. All students performed one IPV SP encounter during three separate OSCE evaluations.
Study Protocol
Students received interactive didactic training during a one-hour small group session on family violence and IPV with trained faculty. Prior to the didactic training, each student’s knowledge, attitudes and belief about IPV was assessed from a pre-test. The pre-test also assessed the student’s comfort level in caring for IPV patients. Within several weeks of this educational session students participated in an ED OSCE piloted as a summative evaluation at the end of a mandatory EM rotation. As an adjunct to the OSCE we conducted our study of IPV skills.
The SPs received two hours of IPV training by the grant principal investigator (PI) on the SP encounter, which was adapted from Reteguiz and Cornel-Avendano’s25 workbook on SP cases. Specific items on the SP checklist included: screening for IPV, assessing safety of the victim, addressing injuries, and providing resources and counseling.
Students, SPs, and grant personnel completed the same single evaluation of the student during the SP encounter. The evaluation included communication and competency skills.
EM grant personnel served as evaluators for the SP encounter. The grant PI and Co-PIs had an average of eight years of EM training and an average minimum of 40 hours of formal IPV training. Grant personnel created the SP case and reviewed the checklist reviewed during a grant personnel meeting. The role of the evaluator included observing the history and physical (H&P) and completing the check-off list relating to the H&P. In addition, individual feedback by grant personnel was given to the student immediately after the SP exercise. Feedback was based upon the checklist, grant personnel observation of the student’s communication skills, the student’s physical examination technique, and the student’s discussion of diagnosis and plan with the SP. The post-test was administered to assess any improvement in the student’s knowledge, recognition and awareness of family violence.
Measurements
Competency
The 360-degree evaluation form, which was developed by grant personnel to assess IPV skills of fourth year medical students in the IPV OSCE station, measured student competency. The evaluation instrument included a checklist of responses that would assist faculty trained in family violence education in making a final determination of the competency level of the student to appropriately manage IPV patients. There were 25 items specific to the SP encounter, three related to competency skills (patient care, communication and professionalism) and three global items (student’s overall comfort, confidence and informativeness). A trained faculty, the SP and the student also completed the same evaluation form providing a 360-degree assessment of the student’s performance.
Comfort
Pre- and post-test and the 360-degree evaluation measured student comfort. Grant personnel with IPV expertise developed and reviewed pre- and post-test content to ensure their face validity. The pre- and post-test were the same instrument. The 25-item checklist assessed the trainee’s communication skills and comfort in working with IPV patients. The pre- and post-test were not graded, but rather compared to see a difference in response by the trainee after the SP encounter.
Professionalism and Patient Car
Professionalism and patient care were measured by the 360-degree, 25-item checklist. Using a five point Likert scale, the SP and faculty rated the students on patient care, communication skills and professionalism.
Data Analysis
We imported the checklist information and assessment of competency from the SP, attending and student into an Excel database, and then computed frequencies, means and correlations using SAS version 8.0
RESULTS
Forty-one students participated in the SP exercise. The students’ pre-test was noted at mildly uncomfortable to comfortable (2.7), with minimal change on the post-test (2.9) after the SP encounter (p=NS).
In this cohort, which includes students from 29 states and four countries, approximately 13 % are underrepresented minorities and 49% are women. Table 1 exhibits H&P findings. Although 88% of the students screened for IPV and 98% asked about the patient’s injury, only 39% asked about verbal abuse and only 11% documented their findings in the chart.
SP, attending and student assessment for competency, professionalism and patient care show average to very good composite scores (Table 2 and and3).3). The majority of students ranged in their communication skills from good to very good (Table 2). We analyzed the correlations analyzed using three evaluators. Correlation between overall trainee comfort level and each of the competencies (patient care, empathetic communication skill and professionalism) were positive (Table 4). Correlations when stratified by evaluators were not universally statistically significant.
DISCUSSION
With the recent emphasis on clinical competency, the use of SPs is being recognized as a useful evaluation tool to assess competency in medical schools. Currently, almost half of medical schools use SPs in clerkships and in fourth-year examinations.26 Recently, specific classes have been developed to teach medical students these communication skills. One example is the Uniformed Services University of Health Sciences, which developed a pilot course as part of a psychiatry clerkship to improve medical student communication skills.27
Goals of an undergraduate EM rotation include the assessment of acutely sick and injured patients as well as experience in evaluating the undifferentiated patient.21, 28 Many IPV victims in the ED setting will not present with IPV as their chief complaint and therefore may go unrecognized, unless students are taught to inquire specifically about IPV. Further, many patients may not present to the ED with IPV as their primary complaint but would be willing to disclose IPV if asked directly by a physician.6 One way to assist students in considering the diagnosis of IPV is to introduce the subject early in medical student education. Using the SP encounter in IPV can be one such tool. This study sought to develop an IPV SP encounter and an evaluation tool for this educational goal.
SP encounters have several advantages. It allows for direct observation of the student, an opportunity for immediate feedback to the student by faculty, and a specific focus on the student’s ability to manage IPV. In addition, students have the opportunity to ask specific questions related to IPV and are encouraged to include it as part of their differential when caring for patients. Our use of the pre- and post-test after using the SP IPV scenario did not significantly improve student comfort; however, Boulet et al29 caution that relatively little research has been done to assess how to best set a passing standard on SP encounters and recommend that future research using SPs should include standard-setting methods. To assess or replicate research findings of published reports, journal editors, reviewers and authors should provide adequate detail when describing SP methodology.30
LIMITATIONS
Limitation of this study include not assessing students previous experience with IPV, either through clinical experience, didactics or personal experience prior to training. Second, due to the small sample size of this pilot study, we cannot extrapolate these findings to the entire medical school class and we don’t know how these findings would persist long term. Third, the students did not have demographic markers on the OSCE pre-test and post-test that determined ethnicity and gender; thus we were unable to infer any biases based on ethnicity and gender, although a larger study should look at those demographics and how they affect the results. Fourth, we did not achieve a 100% completion rate for all the forms, which resulted in missing data. Lastly, the SP case and form has not been validated and would warrant use and repetition in other student ED rotations to test its validity and reliability.
CONCLUSION
Effective methods for training medical students in IPV are needed. Using SPs in an OSCE setting could help trainees focus on the affective elements of training (such as attitudes) in caring for patients who experience IPV. This methodology should be further developed to use as a tool to measure students’ competencies in caring for patients with IPV. We found correlation among the trainee comfort level, patient care, communication skills and professionalism. In our investigation, the OSCE did not show a clinically significant improvement in students’ comfort level in caring for IPV victims. The SP OSCE did help us identify likely gaps in our educational curriculum. Further studies using SP with IPV should be conducted to determine effectiveness of this educational technique within the medical student curriculum.
Footnotes
This study was funded by the Emory University School of Medicine Innovation in Medical Education Grant.
Supervising Section Editor: Michael Epter, DO
Submission history: Submitted September 24, 2009; Revision Received December 22, 2009; Accepted March 20, 2010.
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Address for Correspondence: Sheryl L Heron, MD, MPH, Department of Emergency Medicine, 49 Jesse Hill Jr Dr SE, Atlanta, GA 30303
Email: sheron@emory.edu
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources, and financial or management relationships that could be perceived as potential sources of bias. The authors disclosed none.
REFERENCES
1. Coker AL, Davis KE, Arias I, et al. Physical and mental health effects of intimate partner violence for men and women. Am J Prev Med. 2002;23(4):260–8. [PubMed]
2. Durose MR, Harlow CS, Langan PA, et al. Family violence statistics: including statistics on strangers and acquaintances. U.S. Department of Justice. 2005. Retrieved fromhttp://www.ojp.udsoj.gov/bjs.
3. Abbott J, Johnson R, Koziol-McLain J, et al. Domestic violence against women: incidence and prevalence in an emergency department population. JAMA. 1995;273(22):1763–7. [PubMed]
4. Cohn F, Salmon M, Stobo J. Confronting Chronic Neglect: The education and training of health professionals on family violence; 2002.
5. Coker AL, Bethea L, Smith PH, et al. Missed opportunities: intimate partner violence in family practice settings. Prev Med. 2002;34(4):445–54. [PubMed]
6. Hayden SR, Barton ED, Hayden M. Domestic violence in the emergency department: how do women prefer to disclose and discuss the issues? JEM. 1997;15(4):447–51.
7. Heron SL, Thompson MP, Jackson E, et al. Do responses to an intimate partner violence screen predict scores on a comprehensive measure of intimate partner violence in low-income black women? Ann Emerg Med. 2003;42(4):483–91. [PubMed]
8. Horan DL, Chapin J, Klein L, et al. Domestic violence screening practices of obstetrician-gynecologists. Obstet Gynecol. 1998;92(5):785. [PubMed]
9. Krimm J, Heinzer MM. Domestic violence screening in the emergency department of an urban hospital. J Natl Med Assoc. 2002;94(6):484–91. [PMC free article] [PubMed]
10. Rodriguez MA. Clinicians, intimate partner violence, and opportunities. West J Med.2001;174(5):323–4. [PMC free article] [PubMed]
11. Wadman MC, Muelleman RL. Domestic violence homicides: ED use before victimization. Am J Emerg Med. 1999;17(7):689–91. [PubMed]
12. Liaison, Comittee on Medical Education Functions and Structure of a Medical School Liaison Committee on Medical Education. 2004. Jun, 2008. [Date accessed December 7, 2009]. Retrieved from: http://www.lcme.org/functionslist.htm#educationalprogam.
13. Ernst AA, Houry D, Weiss SJ, et al. Domestic violence awareness in a medical school class: 2-year follow-up. South Med J. 2000;93(8):772–6. [PubMed]
14. Hodges B. Validity and the OSCE. Med Teach. 2003;25(3):250–4. [PubMed]
15. Eid A, Petty M, Hutchins L, et al. “Breaking bad news”: standardized patient intervention improves communication skills for hematology-oncology fellows and advanced practice nurses. J Cancer Educ. 2009;24(2):154–9. [PubMed]
16. Garg A, Buckman R, Kason Y. Teaching medical students how to break bad news. Cmaj.1997;156(8):1159–64. [PMC free article] [PubMed]
17. Kahn MJ, Szerlip HM. Using standardized patients to teach end-of-life skills to clinical clerks.Acad Med. 1999;74(5):581. [PubMed]
18. Quest TE, Otsuki JA, Banja J, et al. The use of standardized patients within a procedural competency model to teach death disclosure. Acad Emerg Med. 2002;9(11):1326–33. [PubMed]
19. Singer PA, Cohen R, Robb A, et al. The ethics objective structured clinical examination. J Gen Intern Med. 1993;8(1):23–8. [PubMed]
20. Haist SA, Wilson JF, Pursley HG, et al. Domestic violence: increasing knowledge and improving skills with a four-hour workshop using standardized patients. Acad Med. 2003;78(10):S24–6.[PubMed]
21. Coates WC. The emergency medicine subinternship–an educator’s guide to planning and administration. Acad Emerg Med. 2005;12(2):129e1–4.
22. Burdick WP, Ben-David MF, Swisher L, et al. Reliability of performance-based clinical skill assessment of emergency medicine residents. Acad Emerg Med. 1996;3(12):1119–23. [PubMed]
23. Johnson G, Reynard K. Assessment of an objective structured clinical examination (OSCE) for undergraduate students in accident and emergency medicine. J Accid Emerg Med. 1994;11(4):223–6. [PMC free article] [PubMed]
24. Shefet D, Dascal-Weichhendler H, Rubin O, et al. Domestic violence: a national simulation-based educational program to improve physicians’ knowledge, skills and detection rates. Med Teach.2007;29(5):e133–8. [PubMed]
25. Reteguiz J, Cornel-Avendano B. Mastering the OSCE CSA. Vol. 1999. New York: McGraw-Hill; pp. 180–4.
26. Kasselbaum DG, Eaglen RH. Shortcomings in the evaluation of students’ clinical skills and behaviors in medical school. Acad Med. 1999;74(7):842–9. [PubMed]
27. Hall MJ, Graceanne A, Lisa M, et al. Use of standardized patients to enhance a psychiatry clerkship. Acad Med. 2004;79(1):28–31. [PubMed]
28. Burdick WP, Jouriles NJ, D’Onofrio G, et al. Emergency medicine in undergraduate education. SAEM Education Committee, undergraduate subcommittee, society for academic emergency medicine. Acad Emerg Med. 1998;5(11):1105–10. [PubMed]
29. Boulet JR, De Champlain AF, McKinley DW. Setting defensible performance standards on OSCEs and standardized patient examinations. Med Teach. 2003;25(3):245–9. [PubMed]
30. Howley L, Szauter K, Perkowski L, et al. Quality of standardised patient research reports in the medical education literature: review and recommendations. Medical Educ. 2008;42(4):350–8.