On March 11th and 12th, representatives from the NBME, AMA, AAMC, Federation of State Medical Boards and the Educational Commission for Foreign Medical Graduates gathered in Philadelphia for InCUS — the Invitational Conference on USMLE Scoring — to discuss potential changes regarding how Step 1 is scored. InCUS came about as a result of a wave of student frustrations over the pressure from the increasing importance of this exam in garnering residency interviews.
The time spent preparing for Step 1 of the USMLE is one of the most stressful times of a medical student’s tenure in medical education. It is widely known and circulated amongst students that performance on Step 1 is one of — if not the most important factor for obtaining residency interviews. After all, in the most recent survey of program directors, 94 percent of respondents cited the test as an important factor, with an average rating of 4.1 out of 5, with 5 being the most important.
Yet so much of a medical student’s future prospects rely heavily on this one test. Failing is a red flag, and passing with a low score is almost certainly worse — unlike the MCAT, Step 1 cannot be retaken unless you fail. It should come as no surprise that this test is a major stressor for medical students.
How did we get here?
Take a look at the figure below.
As the applicant pool has continued to grow, Medicare funding for new PGY-1 positions has not maintained pace. The difference between total spots available and total applicants is now nearly 13,000. With this level of competition, it comes as no surprise that programs are getting swamped with applications. For instance, the average number of applications per internal medicine applicant in 2018 was 60, 46.4 for emergency medicine, and 52.7 for family medicine. This, of course, inundates programs with hundreds or even thousands of applications to comb through.
It is unreasonable to think that PDs, residents, and fellows have the time to holistically review every application without some form of screening tool, in addition to their clinical workloads, research, teaching, and other administrative responsibilities.
A pass-fail Step 1 exam sounds nice, right?
One of the most-discussed alternatives to the current USMLE scoring is a pass-fail exam, returning the exam to its original purpose as a board examination. The pros are obvious — less weight on one exam for students leads to less overall stress from studying for an exam that likely doesn’t correlate to being a better physician, while simultaneously allowing students to focus on other areas of academic or extracurricular interest to make themselves stand out to residency programs.
But these programs would almost certainly resort to another objective metric to take Step 1’s place. That could come in the form of shifting importance to Step 2 CK scores or adding in another standardized exam, such as a specialty-specific exam, akin to SAT subject tests. The former would effectively raise the bar for Step 2, which would not fix the problem at hand, while the latter would favor students who knew what specialty they were interested in pursuing earlier on in medical school, allowing them to prepare at an earlier stage.
In order to avoid this, the concept of limiting the number of applications that an applicant can submit to ERAS was also discussed at InCUS. The idea is that by decreasing applications, students are only applying to programs that they are most interested in, rather than a shotgun approach to avoid getting that dreaded email on the Monday of Match Week. The decreased number of applications per applicant would decrease the total number of applications that programs would receive, ostensibly allowing them additional time for a more extensive and holistic review of applicants.
I expect this would be a significant stressor for fourth-year medical students, be they U.S. MDs, DOs or IMGs. Marginal applicants would not be afforded the luxury to apply broadly, and top-tier students may have angst over applying to top-tier programs, lest they not match. Those who attend top-ranked medical schools would surely have a leg up without the great equalizer that is a numerically-scored Step 1.
In fact, 50 percent of PDs surveyed collectively rated “graduate of highly-regarded U.S. medical school” a 3.8 on the importance scale, which had a higher average rating of importance than both “demonstrat[ing] involvement and interest in research” (3.7), and the personal statement (3.7). Without a more objective assessment like Step 1, where you went and who you know, would surely become a more important factor.
Unintended consequences
This is bad news for osteopathic medical students and IMGs. Despite osteopathic students making up more than 25 percent of current U.S. students, only three DO students out of 28 spots filled matched dermatology in 2019, only four out of 231 matched neurosurgery, and only 15 out of 752 matched ortho. Rather than shattering the glass ceiling, this change would lower it.
In the absence of a screening tool, such as score cut-offs used by programs, more subjective metrics will be used in its stead. For instance, clerkship grades, which are already weighted heavily by PDs, would likely gain a larger role. I see this as problematic for a few reasons. To start, the score breakdown for clerkship grades varies between institutions, with some placing a larger percentage of the grade based on the Shelf/COMAT, and others placing a larger percentage of the grade based on the subjective evaluation by the attendings, fellows, or residents. The grading of clerkships is often arbitrary, with qualitative grades such as honors, high pass, pass, and fail. As medical students from SDN and r/medicalschool often joke, “medical student A was exceptional, performing at the level of a resident. Pass.” — some evaluators simply do not believe that a student can achieve higher than a pass. Then, there are others who will award honors to any student who shows up and merely tries his or her best.
In a world where Step 1 becomes pass-fail, pre-medical students would undoubtedly face increased pressure to perform well on the MCAT, in classes, and pump out even more extracurriculars, with hopes of getting into a top 20 medical school. Rather than remove the stress surrounding a standardized exam, changing Step 1 to pass-fail would merely shift the stress to the MCAT. A change to pass-fail would also pressure premeds into prioritizing a highly-ranked school over less pricey state schools. Part of InCUS’s mission was to look for ways to reduce racial and socioeconomic bias that is inherent to Step 1 scoring, however setting up a system that favors students from wealthier families would run counter to this mission.
As an osteopathic medical student, I have seen the challenges we face — greater difficulty landing away rotations at academic medical centers, less access to clinical research opportunities, and a majority of MD program directors, which have all contributed to DO students not matching into competitive specialties and institutions at the same rate as our MD counterparts. Not all of us attend the Best Medical School (BMS). A great equalizer should be in place to give osteopathic students, IMGs and even allopathic students at less-renowned institutions an even playing field.
Have your voice heard by filling out the InCUS survey.
Ryan Okonski is a medical student.
Image credit: Shutterstock.com
