Some medical schools have altered their admissions process by replacing the traditional applicant interview with the Multiple-Mini Interview (MMI). MMI resembles speed-dating: applicants rotate through numerous interview stations, where they act out scenarios and solve puzzles, sometimes alone and sometimes in groups. A July New York Times article presented a good overview of MMI, as implemented by Virginia Tech Carillon.

As you might expect, schools that have adopted MMI (UCLA and UC Davis among them) maintain its superiority to the traditional interview. For example, Stanford administrators praised the school’s MMI program in a recent editorial:

I’ve interviewed under both formats. While I see strengths in both, I see many reasons why it would be a mistake for MMI to entirely replace the traditional interview:\

Only the traditional interview deters applicants from embellishing their credentials. Interviewers often try to confirm that elements of an application are truthful: if an applicant claims fluency in Spanish, an interviewer might introduce himself in Spanish. And for good reason–there is clear evidence that some medical students cheat. For example, researchers at Brigham & Women’s Hospital established that at least 1 in 20 applicants to their most competitive residency programs had plagiarized their personal statements. MMI refuses to evaluate applicants in the context of their application, a huge drawback that encourages cheating.

MMI questions can be found out ahead of time, and MMI can probably be coached. Many schools purchase a bank of MMI questions from McMaster University in Canada and reuse their questions day to day and year to year. Some schools request that interviewees sign non-disclosure agreements (NDAs) to keep the MMI scenarios they witness a secret. Not all applicants abide by the NDAs and a well-connected student would have little trouble finding out the questions. For that matter, a publicly available fact-sheet put out by McMaster lays out two of the MMI scenarios that I encountered on the interview trail.My impression is that MMIs are coachable: most of the stations involved extemporaneous acting, and having taken improvisational acting as an undergraduate was a tremendous help. After all, if MMI weren’t coachable, why would schools need interviewees to sign NDAs? I’ve already noticed that colleges are putting on “mock MMIs” for their pre-meds. The coachability of MMI will increasingly limit its ability to objectively evaluate interviewees.

The main research studies on MMI are not as relevant as they might appear. The main studies on MMI come from McMaster University (the school that profits by licensing MMI questions, which strikes me as a substantial conflict-of-interest). Researchers asked applicants undergoing traditional interviews to volunteer to participate in a trial MMI. Neither the student’s performance in the MMI nor his decision whether to participate would be considered in the admissions process. The study looked within this sample of volunteers and found that higher scores on the MMI moderately correlated with better evaluations on clinical clerkships. Universities cite this study as evidence that MMI ought to replace the traditional interview. The fallacy in citing this study is that it only examined applicants who performed well enough on the traditional interview to be admitted. The study does not tell us that MMI by itself is better than a traditional interview at assessing candidates. Rather, it suggests that succeeding on both correlates more strongly to good clerkship evaluations than does succeeding on the traditional interview alone. It’s not clear that MMI alone is any better than the traditional interview, and indeed, MMI alone may be much worse. The study is analogous to my asking, “of the girls who I would have an excellent time with on a long date (traditional interview), does how much I enjoy talking to them at a cocktail party (MMI) correlate to how good of a couple we would make?” Even if the answer is yes, it doesn’t mean that when I seek out mates I should abandon dates and only attend cocktail parties.

I agree with the Stanford administrators that the traditional interview and MMI test different skill sets: the interview requires depth and is a one-on-one exploration of someone’s person and character; the MMI assesses how well someone communicates and improvises in different situations. It makes sense that the best doctors have both of these skills, but I have not seen evidence that one set of skills is better than the other.

In sum, I’m not convinced that MMI ought to replace traditional interviews. MMI’s utility is unproven. Should more schools adopt MMI, I would expect students to adapt by obtaining schools’ questions ahead of time and by practicing their acting skills. Lastly, because MMI does not evaluate an interviewee in the context of his application, it offers additional temptation to embellish one’s qualifications.

MMI might be a great supplement to the traditional application process, but it strikes me as a poor replacement.

“Reflex Hammer” is a medical student who blogs at The Reflex Hammer.

Submit a guest post and be heard on social media’s leading physician voice.