Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Bias in top tier academic journals

Derek Lowe, PhD
Meds
June 23, 2011
Share
Tweet
Share

Venture-capital guy Bruce Booth has a provocative post, based on experience, about how reproducible those papers are that make you say, “Someone should try to start a company around that stuff”:

The unspoken rule is that at least 50% of the studies published even in top tier academic journals – Science, Nature, Cell, PNAS, etc… – can’t be repeated with the same conclusions by an industrial lab. In particular, key animal models often don’t reproduce. This 50% failure rate isn’t a data free assertion: it’s backed up by dozens of experienced R&D professionals who’ve participated in the (re)testing of academic findings. This is a huge problem for translational research and one that won’t go away until we address it head on.

Why such a high failure rate? Booth’s own explanation is clearly the first one to take into account – that academic labs live by results. They live by publishable, high-impact-factor-journal results, grant-renewing tenure-application-supporting results. And it’s not that there’s a lot of deliberate faking going on (although there’s always a bit of that to be found), as much as there is wishful thinking and running everything so that it seems to hang together just well enough to get the paper out. It’s a temptation for everyone doing research, especially tricky cutting-edge stuff that fails a lot of the time anyway. Hey, it did work that time, so we know that it’s real – those other times it didn’t go so smoothly, well, we’ll figure out what the problems were with those, but for now, let’s just write this stuff up before we get scooped.

Even things that turn out to be (mostly) correct often aren’t that reproducible, at least, not enough to start raising money for them. Booth’s advice for people in that situation is to check things out very carefully. If the new technology is flaky enough that only a few people can get it to work, it’s not ready for the bright lights yet.

He also has some interesting points on “academic bias” versus “pharma bias”. You hear a lot about the latter, to the point that some people consider any work funded by the drug industry to be de facto tainted. But everyone has biases. Drug companies want to get compounds approved, and to sell lots of them once that happens. Academic labs want to get big, impressive publications and big, impressive grants. The consequences of industrial biaes and conflicts of interest can be larger, but if you’re working back at the startup stage, you’d better keep an eye on the academic ones. We both have to watch ourselves.

Derek Lowe has worked for several major pharmaceutical companies and blogs at In the Pipeline.

Submit a guest post and be heard on social media’s leading physician voice.

Prev

2 changes to cut Federal health care expenditures

June 23, 2011 Kevin 8
…
Next

ACP: ACOs – On to the final rule

June 23, 2011 Kevin 1
…

Tagged as: Medications

Post navigation

< Previous Post
2 changes to cut Federal health care expenditures
Next Post >
ACP: ACOs – On to the final rule

ADVERTISEMENT

More by Derek Lowe, PhD

  • a desk with keyboard and ipad with the kevinmd logo

    The value of new drugs depends on your perspective

    Derek Lowe, PhD
  • a desk with keyboard and ipad with the kevinmd logo

    Potassium iodide won’t protect you from nuclear reactor radioactive isotopes

    Derek Lowe, PhD

More in Meds

  • Tofacitinib: a lesson in heart-immune health

    Larry Kaskel, MD
  • The case for regulating, not banning, kratom

    Heidi Sykora, DNP, RN
  • How India-Pakistan tensions could break America’s generic drug pipeline

    Adwait Chafale
  • The unfair war on buprenorphine

    Brian Lynch, MD
  • Drug giants face suit over hidden cancer risks

    Martha Rosenberg
  • The diseconomics of scale: How Indian pharma’s race to scale backfires on U.S. patients

    Adwait Chafale
  • Most Popular

  • Past Week

    • The mental health workforce is collapsing

      Ronke Lawal | Conditions
    • The measure of a doctor, the misery of a patient

      Anonymous | Physician
    • A doctor’s struggle with burnout and boundaries

      Humeira Badsha, MD | Physician
    • The stoic cure for modern anxiety

      Osmund Agbo, MD | Physician
    • How therapy helps uncover hidden patterns that shape our lives [PODCAST]

      The Podcast by KevinMD | Podcast
    • The role of faith and culture in patient recovery

      Monzur Morshed, MD and Kaysan Morshed | Physician
  • Past 6 Months

    • Rethinking the JUPITER trial and statin safety

      Larry Kaskel, MD | Conditions
    • The ignored clinical trials on statins and mortality

      Larry Kaskel, MD | Conditions
    • How one physician redesigned her practice to find joy in primary care again [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why doctors must fight for a just health care system

      Alankrita Olson, MD, MPH & Ashley Duhon, MD & Toby Terwilliger, MD | Policy
    • The human case for preserving the nipple after mastectomy

      Thomas Amburn, MD | Conditions
    • IMGs are the future of U.S. primary care

      Adam Brandon Bondoc, MD | Physician
  • Recent Posts

    • How therapy helps uncover hidden patterns that shape our lives [PODCAST]

      The Podcast by KevinMD | Podcast
    • Focal therapy for prostate cancer: a new option

      Louis S. Liou, MD, PhD | Conditions
    • The power of ordinary joy for physician well-being

      Ben Reinking, MD | Physician
    • Reinforcing trust in AI: a critical role for health tech leaders

      Miles Barr | Tech
    • Leadership is about pulling others up

      Sagar Chapagain, MD | Physician
    • The opioid crisis in wealthy zip codes

      Carlos N. Hernandez-Torres, MD | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 6 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • The mental health workforce is collapsing

      Ronke Lawal | Conditions
    • The measure of a doctor, the misery of a patient

      Anonymous | Physician
    • A doctor’s struggle with burnout and boundaries

      Humeira Badsha, MD | Physician
    • The stoic cure for modern anxiety

      Osmund Agbo, MD | Physician
    • How therapy helps uncover hidden patterns that shape our lives [PODCAST]

      The Podcast by KevinMD | Podcast
    • The role of faith and culture in patient recovery

      Monzur Morshed, MD and Kaysan Morshed | Physician
  • Past 6 Months

    • Rethinking the JUPITER trial and statin safety

      Larry Kaskel, MD | Conditions
    • The ignored clinical trials on statins and mortality

      Larry Kaskel, MD | Conditions
    • How one physician redesigned her practice to find joy in primary care again [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why doctors must fight for a just health care system

      Alankrita Olson, MD, MPH & Ashley Duhon, MD & Toby Terwilliger, MD | Policy
    • The human case for preserving the nipple after mastectomy

      Thomas Amburn, MD | Conditions
    • IMGs are the future of U.S. primary care

      Adam Brandon Bondoc, MD | Physician
  • Recent Posts

    • How therapy helps uncover hidden patterns that shape our lives [PODCAST]

      The Podcast by KevinMD | Podcast
    • Focal therapy for prostate cancer: a new option

      Louis S. Liou, MD, PhD | Conditions
    • The power of ordinary joy for physician well-being

      Ben Reinking, MD | Physician
    • Reinforcing trust in AI: a critical role for health tech leaders

      Miles Barr | Tech
    • Leadership is about pulling others up

      Sagar Chapagain, MD | Physician
    • The opioid crisis in wealthy zip codes

      Carlos N. Hernandez-Torres, MD | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Bias in top tier academic journals
6 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...