Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

What is the best way to choose residents?

Skeptical Scalpel, MD
Education
March 7, 2014
Share
Tweet
Share

On Twitter a while ago, a medical student asked me how surgical program directors select new residents. Then a discussion arose among some academic surgeons on the same topic. Someone suggested that medical school grades were the best way to tell whether an applicant would be a successful resident.

The fact is that we aren’t really sure what the best way to choose residents is.

First, here’s what we really do.

A 2011 paper from the Journal of Surgical Education reported on a survey of general surgery program directors, associate program directors, and department chairs, with 262 (65%) responding.

USMLE Step 1, used by 37% of programs, was the most common applicant screening criterion with USMLE Step 2 second at 24% and graduation from an LCME-accredited US med school third at 15%. The least important criteria were previous research experience and publications.

Final selection criteria were assessed using a Likert scale. The number one factor was the interview followed by the USMLE Step 1 score, letters of recommendation, and USMLE Step 2 score. The least important factor by far was whether an applicant had done a preliminary year. Research, publications, and a previous rotation at the institution also ranked near the bottom. Class ranking, the dean’s letter, and surprisingly, Alpha Omega Alpha status were in the middle.

Responses consisted of 49% from university programs, 38% from university affiliated hospital programs, and 13% from independent community hospital programs. The average number of applicants per program was 571.

The problem is that proof of the value of the above methods of selection is lacking. A paper from Academic Medicine in 2011 reviewed nine studies of USMLE scores and resident performance and found no correlation between those scores and the acquisition of clinical skills by students residents or fellows.

A meta-analysis of 80 studies and over 41,000 participants from the journal Medical Education in 2013 found that the USMLE Step 1 scores and medical school grades were associated with better resident performance. However, if you eliminate studies showing that better USMLE scores led to better scores on in-training exams and passing licensing tests, only two studies found that USMLE scores correlated well with subjective ratings of residents.

What about grades?

The authors pointed out that, “These data could potentially be more useful to program directors if grading systems across medical schools were standardized.”

I said the same thing in a previous post.

A study of 348 categorical general surgery residents at six residency programs on the west coast looked at resident attrition or need for remediation. The need for remediation was associated with receiving a grade of honors in the medical school surgery clerkship and with a slightly but statistically significantly lower USMLE Step 1 scores. For example, PGY-1 residents needing remediation averaged 225 on USMLE Step 1 vs. 232 for those not needing remediation.

A major issue is the fact that we don’t have good data on the clinical performance of surgical residents or graduates of training programs. I know from personal experience that a good USMLE score or a high score on the surgery in-training exam had a “halo effect” when it came time for faculty to evaluate overall resident performance.

ADVERTISEMENT

According to the Wall Street Journal, some businesses are asking job applicants of all ages “to provide SAT or ACT scores, results from graduate-school entrance tests and grade-point averages along with their work history.”

Google, a successful company by any measure, does not care much about grades. Here’s what Laszlo Bock, senior vice president of people operations, had to say in the New York Times about employee performance: “One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria [sic] for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation. Google famously used to ask everyone for a transcript and G.P.A.’s and test scores, but we don’t anymore, unless you’re just a few years out of school. We found that they don’t predict anything.”

But the relationship between college grades and performance at Google isn’t the same as the relationship between med school grades and performance as a surgeon. Or maybe it is.

I suppose one could argue that applicants for residencies, who are recent graduates of medical schools, would fall into Google’s “slight correlation” category.

I know one thing: I don’t know how to select applicants who will become good surgeons. Do you?

“Skeptical Scalpel” is a surgeon blogs at his self-titled site, Skeptical Scalpel.

Prev

The risks of treating high blood pressure in the elderly

March 6, 2014 Kevin 7
…
Next

Why ICD-10 concerns me deeply

March 7, 2014 Kevin 25
…

Tagged as: Residency, Surgery

Post navigation

< Previous Post
The risks of treating high blood pressure in the elderly
Next Post >
Why ICD-10 concerns me deeply

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Skeptical Scalpel, MD

  • The hospital CEO who made a surgical incision. What happened?

    Skeptical Scalpel, MD
  • Medical error is not the third leading cause of death

    Skeptical Scalpel, MD
  • Should speed-eating contests be banned?

    Skeptical Scalpel, MD

More in Education

  • Graduating from medical school without family: a story of strength and survival

    Anonymous
  • 2 hours to decide my future: How the SOAP residency match traps future doctors

    Nicolette V. S. Sewall, MD, MPH
  • What led me from nurse practitioner to medical school

    Sarah White, APRN
  • Bridging the rural surgical care gap with rotating health care teams

    Ankit Jain
  • Why tracking cognitive load could save doctors and patients

    Hiba Fatima Hamid
  • The hidden cost of becoming a doctor: a South Asian perspective

    Momeina Aslam
  • Most Popular

  • Past Week

    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • When did we start treating our lives like trauma?

      Maureen Gibbons, MD | Physician
    • Why male fertility needs to be part of every health conversation

      Hoag Memorial Hospital Presbyterian | Conditions
    • How home-based AI can reduce health inequities in underserved communities [PODCAST]

      The Podcast by KevinMD | Podcast
    • Inside human trafficking: a guide to recognizing and preventing it [PODCAST]

      The Podcast by KevinMD | Podcast
    • Graduating from medical school without family: a story of strength and survival

      Anonymous | Education
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • My journey from misdiagnosis to living fully with APBD

      Jeff Cooper | Conditions
    • Addressing the physician shortage: How AI can help, not replace

      Amelia Mercado | Tech
    • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

      Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO | Meds
  • Recent Posts

    • How home-based AI can reduce health inequities in underserved communities [PODCAST]

      The Podcast by KevinMD | Podcast
    • Adriana Smith’s story: a medical tragedy under heartbeat laws

      Nicole M. King, MD | Physician
    • What if medicine had an exit interview?

      Lynn McComas, DNP, ANP-C | Conditions
    • Why U.S. health care pricing is so confusing—and how to fix it

      Ashish Mandavia, MD | Physician
    • From survival to sovereignty: What 35 years in the ER taught me about identity, mortality, and redemption

      Kenneth Ro, MD | Physician
    • When doctors forget how to examine: the danger of lost clinical skills

      Mike Stillman, MD | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 10 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • When did we start treating our lives like trauma?

      Maureen Gibbons, MD | Physician
    • Why male fertility needs to be part of every health conversation

      Hoag Memorial Hospital Presbyterian | Conditions
    • How home-based AI can reduce health inequities in underserved communities [PODCAST]

      The Podcast by KevinMD | Podcast
    • Inside human trafficking: a guide to recognizing and preventing it [PODCAST]

      The Podcast by KevinMD | Podcast
    • Graduating from medical school without family: a story of strength and survival

      Anonymous | Education
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • My journey from misdiagnosis to living fully with APBD

      Jeff Cooper | Conditions
    • Addressing the physician shortage: How AI can help, not replace

      Amelia Mercado | Tech
    • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

      Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO | Meds
  • Recent Posts

    • How home-based AI can reduce health inequities in underserved communities [PODCAST]

      The Podcast by KevinMD | Podcast
    • Adriana Smith’s story: a medical tragedy under heartbeat laws

      Nicole M. King, MD | Physician
    • What if medicine had an exit interview?

      Lynn McComas, DNP, ANP-C | Conditions
    • Why U.S. health care pricing is so confusing—and how to fix it

      Ashish Mandavia, MD | Physician
    • From survival to sovereignty: What 35 years in the ER taught me about identity, mortality, and redemption

      Kenneth Ro, MD | Physician
    • When doctors forget how to examine: the danger of lost clinical skills

      Mike Stillman, MD | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

What is the best way to choose residents?
10 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...