Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Bias in top tier academic journals

Derek Lowe, PhD
Meds
June 23, 2011
Share
Tweet
Share

Venture-capital guy Bruce Booth has a provocative post, based on experience, about how reproducible those papers are that make you say, “Someone should try to start a company around that stuff”:

The unspoken rule is that at least 50% of the studies published even in top tier academic journals – Science, Nature, Cell, PNAS, etc… – can’t be repeated with the same conclusions by an industrial lab. In particular, key animal models often don’t reproduce. This 50% failure rate isn’t a data free assertion: it’s backed up by dozens of experienced R&D professionals who’ve participated in the (re)testing of academic findings. This is a huge problem for translational research and one that won’t go away until we address it head on.

Why such a high failure rate? Booth’s own explanation is clearly the first one to take into account – that academic labs live by results. They live by publishable, high-impact-factor-journal results, grant-renewing tenure-application-supporting results. And it’s not that there’s a lot of deliberate faking going on (although there’s always a bit of that to be found), as much as there is wishful thinking and running everything so that it seems to hang together just well enough to get the paper out. It’s a temptation for everyone doing research, especially tricky cutting-edge stuff that fails a lot of the time anyway. Hey, it did work that time, so we know that it’s real – those other times it didn’t go so smoothly, well, we’ll figure out what the problems were with those, but for now, let’s just write this stuff up before we get scooped.

Even things that turn out to be (mostly) correct often aren’t that reproducible, at least, not enough to start raising money for them. Booth’s advice for people in that situation is to check things out very carefully. If the new technology is flaky enough that only a few people can get it to work, it’s not ready for the bright lights yet.

He also has some interesting points on “academic bias” versus “pharma bias”. You hear a lot about the latter, to the point that some people consider any work funded by the drug industry to be de facto tainted. But everyone has biases. Drug companies want to get compounds approved, and to sell lots of them once that happens. Academic labs want to get big, impressive publications and big, impressive grants. The consequences of industrial biaes and conflicts of interest can be larger, but if you’re working back at the startup stage, you’d better keep an eye on the academic ones. We both have to watch ourselves.

Derek Lowe has worked for several major pharmaceutical companies and blogs at In the Pipeline.

Submit a guest post and be heard on social media’s leading physician voice.

Prev

2 changes to cut Federal health care expenditures

June 23, 2011 Kevin 8
…
Next

ACP: ACOs – On to the final rule

June 23, 2011 Kevin 1
…

Tagged as: Medications

Post navigation

< Previous Post
2 changes to cut Federal health care expenditures
Next Post >
ACP: ACOs – On to the final rule

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Derek Lowe, PhD

  • a desk with keyboard and ipad with the kevinmd logo

    The value of new drugs depends on your perspective

    Derek Lowe, PhD
  • a desk with keyboard and ipad with the kevinmd logo

    Potassium iodide won’t protect you from nuclear reactor radioactive isotopes

    Derek Lowe, PhD

More in Meds

  • Why retail pharmacies are the future of diverse clinical trials

    Shelli Pavone
  • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

    Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO
  • A world without antidepressants: What could possibly go wrong?

    Tomi Mitchell, MD
  • The truth about GLP-1 medications for weight loss: What every patient should know

    Nisha Kuruvadi, DO
  • The hidden bias in how we treat chronic pain

    Richard A. Lawhern, PhD
  • Biologics are not small molecules: the case for pre-allergy testing in an era of immune-based therapies

    Robert Trent
  • Most Popular

  • Past Week

    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why flashy AI tools won’t fix health care without real infrastructure

      David Carmouche, MD | Tech
    • Why Medicaid cuts should alarm every doctor

      Ilan Shapiro, MD | Policy
    • When the diagnosis is personal: What my mother’s Alzheimer’s taught me about healing

      Pearl Jones, MD | Conditions
    • Key strategies for smooth EHR transitions in health care

      Sandra Johnson | Tech
    • Reassessing the impact of CDC’s opioid guidelines on chronic pain care [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • How dismantling DEI endangers the future of medical care

      Shashank Madhu and Christian Tallo | Education
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
  • Recent Posts

    • Graduating from medical school without family: a story of strength and survival

      Anonymous | Education
    • Inside human trafficking: a guide to recognizing and preventing it [PODCAST]

      The Podcast by KevinMD | Podcast
    • Earwax could hold secrets to cancer, Alzheimer’s, and heart disease

      Sandra Vamos, EdD and Domenic Alaim | Conditions
    • Why male fertility needs to be part of every health conversation

      Hoag Memorial Hospital Presbyterian | Conditions
    • Why health care must adapt to meet the needs of older adults with disabilities

      Lynn A. Schaefer, PhD | Conditions
    • How doctors took back control from hospital executives

      Gene Uzawa Dorio, MD | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 6 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why flashy AI tools won’t fix health care without real infrastructure

      David Carmouche, MD | Tech
    • Why Medicaid cuts should alarm every doctor

      Ilan Shapiro, MD | Policy
    • When the diagnosis is personal: What my mother’s Alzheimer’s taught me about healing

      Pearl Jones, MD | Conditions
    • Key strategies for smooth EHR transitions in health care

      Sandra Johnson | Tech
    • Reassessing the impact of CDC’s opioid guidelines on chronic pain care [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • How dismantling DEI endangers the future of medical care

      Shashank Madhu and Christian Tallo | Education
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
  • Recent Posts

    • Graduating from medical school without family: a story of strength and survival

      Anonymous | Education
    • Inside human trafficking: a guide to recognizing and preventing it [PODCAST]

      The Podcast by KevinMD | Podcast
    • Earwax could hold secrets to cancer, Alzheimer’s, and heart disease

      Sandra Vamos, EdD and Domenic Alaim | Conditions
    • Why male fertility needs to be part of every health conversation

      Hoag Memorial Hospital Presbyterian | Conditions
    • Why health care must adapt to meet the needs of older adults with disabilities

      Lynn A. Schaefer, PhD | Conditions
    • How doctors took back control from hospital executives

      Gene Uzawa Dorio, MD | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Bias in top tier academic journals
6 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...