Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Bias in top tier academic journals

Derek Lowe, PhD
Meds
June 23, 2011
Share
Tweet
Share

Venture-capital guy Bruce Booth has a provocative post, based on experience, about how reproducible those papers are that make you say, “Someone should try to start a company around that stuff”:

The unspoken rule is that at least 50% of the studies published even in top tier academic journals – Science, Nature, Cell, PNAS, etc… – can’t be repeated with the same conclusions by an industrial lab. In particular, key animal models often don’t reproduce. This 50% failure rate isn’t a data free assertion: it’s backed up by dozens of experienced R&D professionals who’ve participated in the (re)testing of academic findings. This is a huge problem for translational research and one that won’t go away until we address it head on.

Why such a high failure rate? Booth’s own explanation is clearly the first one to take into account – that academic labs live by results. They live by publishable, high-impact-factor-journal results, grant-renewing tenure-application-supporting results. And it’s not that there’s a lot of deliberate faking going on (although there’s always a bit of that to be found), as much as there is wishful thinking and running everything so that it seems to hang together just well enough to get the paper out. It’s a temptation for everyone doing research, especially tricky cutting-edge stuff that fails a lot of the time anyway. Hey, it did work that time, so we know that it’s real – those other times it didn’t go so smoothly, well, we’ll figure out what the problems were with those, but for now, let’s just write this stuff up before we get scooped.

Even things that turn out to be (mostly) correct often aren’t that reproducible, at least, not enough to start raising money for them. Booth’s advice for people in that situation is to check things out very carefully. If the new technology is flaky enough that only a few people can get it to work, it’s not ready for the bright lights yet.

He also has some interesting points on “academic bias” versus “pharma bias”. You hear a lot about the latter, to the point that some people consider any work funded by the drug industry to be de facto tainted. But everyone has biases. Drug companies want to get compounds approved, and to sell lots of them once that happens. Academic labs want to get big, impressive publications and big, impressive grants. The consequences of industrial biaes and conflicts of interest can be larger, but if you’re working back at the startup stage, you’d better keep an eye on the academic ones. We both have to watch ourselves.

Derek Lowe has worked for several major pharmaceutical companies and blogs at In the Pipeline.

Submit a guest post and be heard on social media’s leading physician voice.

Prev

2 changes to cut Federal health care expenditures

June 23, 2011 Kevin 8
…
Next

ACP: ACOs – On to the final rule

June 23, 2011 Kevin 1
…

Tagged as: Medications

Post navigation

< Previous Post
2 changes to cut Federal health care expenditures
Next Post >
ACP: ACOs – On to the final rule

ADVERTISEMENT

More by Derek Lowe, PhD

  • a desk with keyboard and ipad with the kevinmd logo

    The value of new drugs depends on your perspective

    Derek Lowe, PhD
  • a desk with keyboard and ipad with the kevinmd logo

    Potassium iodide won’t protect you from nuclear reactor radioactive isotopes

    Derek Lowe, PhD

More in Meds

  • From stigma to science: Rethinking the U.S. drug scheduling system

    Artin Asadipooya
  • How drugmakers manipulate your health from diagnosis to prescription

    Martha Rosenberg
  • The food-drug interaction risks your doctor may be missing

    Frank Jumbe
  • Why retail pharmacies are the future of diverse clinical trials

    Shelli Pavone
  • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

    Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO
  • A world without antidepressants: What could possibly go wrong?

    Tomi Mitchell, MD
  • Most Popular

  • Past Week

    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • How New Mexico became a malpractice lawsuit hotspot

      Patrick Hudson, MD | Physician
    • Why doctors are reclaiming control from burnout culture

      Maureen Gibbons, MD | Physician
    • A world without vaccines: What history teaches us about public health

      Drew Remignanti, MD, MPH | Conditions
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • How the 10th Apple Effect is stealing your joy in medicine

      Neil Baum, MD | Physician
  • Recent Posts

    • From Founding Fathers to modern battles: physician activism in a politicized era [PODCAST]

      The Podcast by KevinMD | Podcast
    • From stigma to science: Rethinking the U.S. drug scheduling system

      Artin Asadipooya | Meds
    • The gift we keep giving: How medicine demands everything—even our holidays

      Tomi Mitchell, MD | Physician
    • The promise and perils of AI in health care: Why we need better testing standards

      Max Rollwage, PhD | Tech
    • From burnout to balance: a neurosurgeon’s bold career redesign

      Jessie Mahoney, MD | Physician
    • Healing the doctor-patient relationship by attacking administrative inefficiencies

      Allen Fredrickson | Policy

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 6 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • How New Mexico became a malpractice lawsuit hotspot

      Patrick Hudson, MD | Physician
    • Why doctors are reclaiming control from burnout culture

      Maureen Gibbons, MD | Physician
    • A world without vaccines: What history teaches us about public health

      Drew Remignanti, MD, MPH | Conditions
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • How the 10th Apple Effect is stealing your joy in medicine

      Neil Baum, MD | Physician
  • Recent Posts

    • From Founding Fathers to modern battles: physician activism in a politicized era [PODCAST]

      The Podcast by KevinMD | Podcast
    • From stigma to science: Rethinking the U.S. drug scheduling system

      Artin Asadipooya | Meds
    • The gift we keep giving: How medicine demands everything—even our holidays

      Tomi Mitchell, MD | Physician
    • The promise and perils of AI in health care: Why we need better testing standards

      Max Rollwage, PhD | Tech
    • From burnout to balance: a neurosurgeon’s bold career redesign

      Jessie Mahoney, MD | Physician
    • Healing the doctor-patient relationship by attacking administrative inefficiencies

      Allen Fredrickson | Policy

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Bias in top tier academic journals
6 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...