| Author | Affiliation |
|---|---|
| Alexis E. Pelletier-Bui, MD | Cooper Medical School of Rowan University, Department of Emergency Medicine, Camden, New Jersey |
| Benjamin H. Schnapp, MD, MEd | University of Wisconsin School of Medicine and Public Health, Department of Emergency Medicine, Madison, Wisconsin |
| Liza G. Smith, MD | University of Massachusetts Medical School – Baystate, Department of Emergency Medicine, Springfield, Massachusetts |
| Doug Franzen, MD | University of Washington Medical School, Department of Emergency Medicine, Seattle, Washington |
| Elizabeth Barrall Werley, MD | Penn State Health Milton S. Hershey Medical Center, Department of Emergency Medicine, Hershey, Pennsylvania |
| Erin McDonough, MD | University of Cincinnati College of Medicine, Department of Emergency Medicine, Cincinnati, Ohio |
| Melanie Camejo, MD, MHPE | University of Missouri – Kansas City/Truman Medical Center, Department of Emergency Medicine, Kansas City, Missouri |
The number of applications to individual emergency medicine (EM) residency programs has markedly increased over the past decade.1-3 As a result, residency programs have difficulty reviewing applications holistically and struggle to identify applicants who are truly interested in their program. These challenges were exacerbated by the COVID-19 pandemic: programs received more applications; and away-rotation restrictions limited EM applicants’ ability to express, and programs to identify, interest in a residency program or geographic region.2 Additionally, the Association of American Medical Colleges reported a concern for maldistribution of interview offers to the highest tier applicants, leaving other well-qualified students with a paucity of interviews – a trend that would threaten the success of the Match for all stakeholders.4
The graduate medical education community has made several proposals and implemented innovations in the residency application process in an attempt to help programs identify best fit applicants with the highest likelihood of matching into their program. Some of these innovations, such as the Standardized Video Interview in EM and the required secondary application essay in otolaryngology (ENT), proved ineffective.5,6 Other practices, such as application filter use, increasing costs per application, implementing caps on applications or interviews, and early or phased cycle matches potentially exacerbate existing inequities for applicants, particularly those who are under-represented in medicine, financially disadvantaged, or lacking mentorship.7-9 One new innovation, preference signaling (PS), has the potential to be fair and equitable for all applicants as well as low cost and low effort for both residency programs and applicants alike.
Preference signaling is a concept rooted in game theory and developed in labor economics to address the challenge of employers not being able to perform a detailed analysis of all potential applicants and aiding them with identifying high-yield employee prospects. Preference signaling allows applicants to assign virtual “tokens” to their most desired employers, providing applicants the opportunity to communicate their interest, and employers the ability to focus their attention on these most “serious” applicants.10
While used by the American Economic Association since 2006, PS only recently gained attention in the residency application process. It was first proposed in the orthopedics and ENT literature in 2017 and 2018, respectively, followed by a promising computer simulation model by ENT in 2019, and implementation by ENT during the 2020–2021 application season.11-14 During the 2020–2021 application season, each applicant to ENT was able to assign five allotted tokens to desired programs via the Otolaryngology Program Directors Organization (OPDO) website over a two-week period, after which their list was finalized. The OPDO then distributed the lists to individual programs on the same day that the Electronic Residency Application Service (ERAS) opened for application review by programs.15 Table 1 shows unpublished preliminary data from the ENT 2020–2021 trial.14,16 Urology implemented a similar PS program for the 2021–2022 residency application cycle via the American Urologic Association website.17 Internal medicine, general surgery, and dermatology have also implemented PS in the 2021–2022 residency application cycle as a component of a supplemental application through ERAS.18 Applicants and programs for all participating specialties have the ability to opt out of PS.15,17,18
Table 1Preliminary preference signaling data from the 2020–2021 otolaryngology application seasona.
| Program data | Applicant data |
|---|---|
| 100% participated (125 programs)
100% received signals
○ The top 10 programs comprised 21% of all tokens
○ Top 20: 38%
○ Top 30: 52%
90% of program directors would continue a similar process in the future (reported after match)
|
558/632 applicants participated
93% received an interview from ≥ 1 signaled program
○ 15% received interviews from all 5 signaled programs
○ 25% from 4
○ 21% from 3
○ 22% from 2
○ 10% from 1
○ 7% from 0
Overall, 18% interview offer rate
○ Non-signaled programs: 14% interview offer rate
○ Signaled programs: 58% interview offer rate
Lowest quartile of applicants demonstrated 33% increase in interview offers at signaled programs
Around 50% of signals are sent to programs from the same geographic region as their home program
Fall survey: 70% satisfied/10% dissatisfied
75% would continue a similar process in the future (reported after match)
|
aData sources can be found in manuscript references 14 and 16.
If managed by a reputable national organization, PS in EM could credibly increase transparency in a process that is high stakes for both applicants and programs, allowing applicants to define program and geographic preference and programs to identify more seriously interested applicants in an equitable manner.11,19,20 It has the potential to provide lower quartile applicants more visibility, when they may otherwise be filtered out of consideration due to low board scores, geography or other factors, and may over time curtail some of the overapplication behavior should applicants discover favorable responses at signaled programs.16,19 Preference signaling may also attract programs’ attention to applicants previously thought to be “out of their league” and not viable matches.21 Allowing the ability to signal preferences with the initial application might also cut down on the amount of time spent on extra applicant communication, such as time spent by applicants drafting emails to specific programs delineating interest, and time spent by program leadership and coordinators responding to those communications. Additionally, it stands to reason that as more applicants receive and accept their most desired interviews (rather than accept offers from less-desired programs), it would potentially relieve interview congestion, opening earlier interviews to middle- and lower- tier applicants.13 Similarly, programs could more efficiently assign interview invitations to higher probability matches, potentially reducing interview cancellations.11
Some reasonable concerns have been raised about PS. Preference signaling may not actually decrease the number of applications from students.21,22 By forcing an expression of preference early in the season, PS may disadvantage applicants who may be unclear regarding which programs are a good fit or wish to find their fit in programs during the interview process. Similarly, applicants may change their preferences during the season but would not have the opportunity to reassign their tokens. Programs may hold bias against applicants that do not assign them a token, potentially causing these programs to disregard applicants who may be a strong fit for their environment. Additionally, tokens are valuable due to their scarcity and may be unintentionally devalued by programs that receive a disproportionate number of tokens. It is also worth mentioning that there may be unforeseen challenges or consequences with the PS model for both applicants and programs that have yet to be discovered.
Some important practical considerations must be addressed before PS is implemented. The number of ideal signal tokens per applicant is unclear. The use of too many tokens risks diluting their value and raises the potential for token non-use to be a signal of disinterest. Too few tokens could require applicants to choose arbitrarily between their top programs and may leave programs with too small a pool of signaled applications to make a difference in their review approach. While ENT used five signals during their initial trial, they have decreased this to four for the 2021–2022 application cycle.14 Dermatology is using three signals for the 2021–2022 application cycle, whereas internal medicine, surgery, and urology are using five signals.17,18 Given each of these specialties varies from EM in the number of programs, available positions, number of applicants and average number of applications per applicant, it will be challenging to determine an ideal number of signals based on other specialties’ experience.1,2,23
Of the specialties implementing PS for the 2021–2022 match cycle, general surgery aligns most closely with EM with regard to the number of programs and applicants at 331 and 2908, respectively, compared to 273 and 3734 in EM, but the number of available positions is almost half that in surgery at 1569 compared to 2840 in EM, again making a comparison challenging.23. However, given that the use of five tokens is the most common initial start point, we would recommend the same for EM, with adjustments made in future years based on program and applicant feedback as well as Match data.
The best approach for assigning tokens is also unclear. While ENT suggests applicants divide tokens between “reach” programs and programs for which they are competitive, economics research suggests it is ideal to use all tokens on programs where an applicant is competitive.10,24 Finally, it has been suggested that a continuous-variable system may be more ideal than the current binary PS system, allowing applicants to signal degree of preference in a program by dividing 100 points among prospective programs (eg, all 100 to one program or 20 for each of five programs).22
While smaller specialties such as ENT and urology have had success with using their own websites and program director organizations to coordinate PS, we recognize that the much greater number of EM residency programs and applicants to EM may make this exceedingly challenging for the Council of Residency Directors in EM to coordinate and/or finance. Therefore, we would propose using the ERAS platform, as larger specialties like internal medicine and surgery have opted to do. We also support the ENT model of applicants finalizing signals prior to ERAS application opening for programs, which will allow programs to more effectively allocate time to holistic application review and identify high-yield applicants for interview.
Despite these uncertainties, the recurring challenges and current application climate provide a compelling case for trialing PS. We recommend exploring interest in PS from all EM residency application stakeholders, continuing to learn from the experiences of ENT, urology, dermatology, internal medicine, and surgery, and investigating methods for potential implementation of a PS pilot in EM for the 2022–2023 application season. While PS may not decrease the raw number of applications, it could address the largest flaw in the current system: the lack of ability for applicants to communicate, and programs to discern, genuine interest.19 By allowing applicants and programs to understand each other better, we believe that PS has the potential to allow for a more sustainable and equitable match process that might create more ideal matches for all candidates and programs with less friction along the way.
Footnotes
Section Editor: Jeffrey Love, MD and Danielle Hart, MD
Full text available through open access at http://escholarship.org/uc/uciem_westjem
Address for Correspondence: Alexis Pelletier-Bui, MD, Cooper Medical School of Rowan University, Department of Emergency Medicine, 1 Cooper Plaza, Camden, NJ 08103. Email: pelletier-bui-alexis@cooperhealth.edu. 1 / 2022; 23:72 – 75
Submission history: Revision received July 1, 2021; Submitted October 29, 2021; Accepted October 28, 2021
Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.
REFERENCES
1. ERAS statistics 2015–2020. 2020. Available at: www.aamc.org/media/39326/download. Accessed October 29, 2021.
2. ERAS statistics, preliminary data (ERAS 2021), residency. 2021. Available at: www.aamc.org/data-reports/interactive-data/eras-statistics-data. Accessed October 29, 2021.
3. Ramsay N. EM match 2020 by the numbers. Available at: https://www.emra.org/students/newsletter-articles/em-match-2020-by-the-numbers/. Accessed October 29, 2021.
4. Whelan A. Open letter on residency interviews. 2020. Available at: https://www.aamc.org/media/50291/download?utm_source=sfmc&utm_medium=Email&utm_campaign=ERAS&utm_content=Interviews. October 29, 2021.
5. Gallahue FE, Deiorio NM, Blomkalns A, et al. The AAMC standardized video interview: lessons learned from the residency selection process. Acad Med. 2020;95(11):1639-42.
6. Kramer S. Is the program-specific paragraph responsible for declining application numbers? A commentary. Otolaryngol Head Neck Surg. 2018;158(2):215-6.
7. Ward M, Pingree C, Laury AM, et al. Applicant perspectives on the otolaryngology residency application process. JAMA Otolaryngol Head Neck Surg. 2017;143(8):782-7.
8. Graboyes EM, Goebel JA. Reforming the otolaryngology-head and neck surgery match: should we embrace a consortia match?. JAMA Otolaryngol Head Neck Surg. 2016;142(8):728-30.
9. Chang CW, Erhardt BF. Rising residency applications: How high will it go?. Otolaryngol Head Neck Surg. 2015;153(5):702-705.
10. Coles P, Kushnir A, Niederle M. Preference signaling in matching markets. Am Economic J: Microeconomics. 2013;5:99-134.
11. Salehi PP, Benito D, Michaelides E. A novel approach to the national resident matching program: the Star System. JAMA Otolaryngol Head Neck Surg. 2018;144(5):397-8.
12. Bernstein J. Not the last word: Want to match in an orthopaedic surgery residency? Send a rose to the program director. Clin Orthop Relat Res. 2017;475(12):2845-9.
13. Whipple ME, Law AB, Bly RA. A computer simulation model to analyze the application process for competitive residency programs. J Grad Med Educ. 2019;11(1):30-5.
14. Otolaryngology preference signaling: Updates. 2021. Available at: https://opdo-hns.org/mpage/signaling-updates. Accessed October 29, 2021.
15. Preference signaling for otolaryngology – head and neck surgery interviews. 2020. Available at: https://cdn.ymaws.com/opdo-hns.org/resource/resmgr/signaling_microsite/signaling_updates/white_paper_signaling_v4.pdf. Accessed October 29, 2021.
16. The otolaryngology signaling experience. Virtual.
17. Introducing preference signaling pilot into the urology match. 2021. Available at: https://sauweb.org/match-program/signaling/guidance.aspx. Accessed October 29, 2021.
18. Supplemental ERAS application guide. 2021. Available at: https://students-residents.aamcorg/media/12326/download?attachment. Accessed October 29, 2021.
19. Salehi PP, Heiser A, Salehi P, et al. Ideas and innovations to improve the otolaryngology-head and neck surgery residency application and selection process. Otolaryngol Head Neck Surg. 2021;164(5):1001-10.
20. Chang CWD, Pletcher SD, Thorne MC, et al. Preference signaling for the otolaryngology interview market. Laryngoscope. 2021;131(3):E744-E5.
21. Fantasia J, Elsamra S, Thavaseelan S. Improving the match: use of preference signaling to optimize the urology match interview process. Urology. 2021;154:57-61.
22. Carmody B. The Sheriff of Sodium. Preference signaling: Winners and losers edition. 2020. Available at: https://thesheriffofsodium.com/2020/08/19/preference-signaling-winners-and-losers-edition/. Accessed October 29, 2021.
23. Results and data: 2021 main residency match. Available at: https://www.nrmp.org/wp-content/uploads/2021/05/MRM-Results_and-Data_2021.pdf. Accessed October 20, 2021.
24. Otolaryngology preference signaling: guidance for program directors and mentors. 2020. Available at: https://opdo-hns.org/mpage/signaling-guidance. Accessed October 29, 2021.
