Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Statistical significance doesn’t always warrant a miracle headline

Jennifer Gunter, MD
Conditions
March 28, 2014
Share
Tweet
Share

Interpreting studies is a dicey thing. Often I find what might be statistically significant translated into headlines that might not really get at the nuance of the study or the results.

Take these three for example:

  1. “Pine bark extract improves severe perimenopausal symptoms”
  2. “Two weeks of antibiotic therapy relieves IBS (irritable bowel syndrome)”
  3. “Study: ‘Female viagra flibanserin’ works”

The first line of the last article: “Need a boost to your sex life. The magic could be in a little pill?”

Let’s look at the studies referenced by these three headlines.

French maritime bark extract, in the dose studied, improved hot flashes for 35% of women who took the drug versus 29% of women who took placebo. Insomnia and sleep problems improved by 28% for the bark extract versus 21% for placebo. (In the 3 domains tested the bark extract “worked”, but it barely reached statistical significance.) Statistically more women who took the drug did better, but is 7 women out of a hundred getting benefit for hot flashes and insomnia from a daily medication a clinically significant or meaningful benefit?

The same goes for the IBS drug, an antibiotic called rifaximin. In the article a physician says that many participants ”say they are 80% improved, 90% improved, that kind of results …”

Looking at the studies, TARGET 1 and TARGET 2 we see that 41% in the drug group were responders versus 32% of the placebo group (which is pretty poor overall if you ask me as a known placebo improves the symptoms of IBS for 59% of patients)! Might a few of the responders felt dramatically better? Is that “many”? I guess it depends on your perspective, but understanding that only 9% of people were true responders puts the findings in a  different light.

And flibanserin? Again, the same type of numbers. The initial studies quoted as “magic” actually showed animprovement for 30-40% of women who took flibanserin versus 15-30% for placebo. Overall, there was an increase in 1-1.8 number of satisfying sexual encounters a month. In my very unscientific study (a poll from a couple of weeks ago) it appears that 78% of people didn’t find those numbers clinically meaningful either.

flibanserin

There is no doubt that each one of these studies has statistical significance. When studying any therapy that is obviously the first step, however, statistical significance is often parlayed into miracle headlines and the promise that a drug is truly helpful is tempered somewhat when you compare it with the placebo response rate.

It behooves everyone not to get caught up in the hype and to put the results in perspective. When a medication helps 65% of people versus 25% for placebo the medical decisions tend to be easier (and by the way, that is more along the lines of what I’d call a miracle response rate, not 40% versus 32%).

If no previous therapy has worked for a condition before then 7 responders out of 100 might truly be a miracle. If a patient has not responded to any previous therapy then 7 responders might be worthwhile. If the medication is extremely low-cost and has minimal side effects, then 7 responders out of a 100 might seem reasonable for many people. However, if the medication is expensive and/or has side effects then obviously the enthusiasm needs to be tempered. The response rate to a medication is part of the risk benefit ratio and every patient that will be different and is controlled by variables such as severity of illness, response to previous treatments, impact of the condition on their life, and previous experience with side effects and cost.

Statistical significance doesn’t mean something works for everyone and it doesn’t mean a therapy is a miracle, so reporters and health care providers need to stop intimating that it does. Statistical significance simply means the likelihood that the desired effect was achieved by chance. The next and far more important step is interpreting those results and applying them in a clinically meaningful way, but those kinds of discussions probably don’t generate sexy, amazing headlines.

Jennifer Gunter is an obstetrician-gynecologist and author of The Preemie Primer. She blogs at her self-titled site, Dr. Jen Gunter.

Prev

Apps used to monitor blood pressure need better design

March 28, 2014 Kevin 2
…
Next

Choosing a perfect residency: What goes into a rank list

March 28, 2014 Kevin 1
…

ADVERTISEMENT

Tagged as: Mainstream media, OB/GYN

Post navigation

< Previous Post
Apps used to monitor blood pressure need better design
Next Post >
Choosing a perfect residency: What goes into a rank list

ADVERTISEMENT

More by Jennifer Gunter, MD

  • The Ellen Show broadcasts potentially harmful information about ovarian cancer screening

    Jennifer Gunter, MD
  • Dear science: an appreciation

    Jennifer Gunter, MD
  • Are there too many female OB/GYNs?

    Jennifer Gunter, MD

More in Conditions

  • Clinical ghosts and why they haunt our exam rooms

    Kara Wada, MD
  • High blood pressure’s hidden impact on kidney health in older adults

    Edmond Kubi Appiah, MPH
  • How declining MMR vaccination rates put future generations at risk

    Ambika Sharma, Onyi Oligbo, and Katrina Green, MD
  • How one unforgettable ER patient taught a nurse about resilience

    Kristen Cline, BSN, RN
  • Why regular exercise is the best prescription for lifelong health

    George F. Smith, MD
  • When the weight won’t budge: the hidden physiology of grief, stress, and set point

    Sarah White, APRN
  • Most Popular

  • Past Week

    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • Why specialist pain clinics and addiction treatment services require strong primary care

      Olumuyiwa Bamgbade, MD | Conditions
    • Harassment and overreach are driving physicians to quit

      Olumuyiwa Bamgbade, MD | Physician
    • Why peer support can save lives in high-pressure medical careers

      Maire Daugharty, MD | Conditions
    • When a medical office sublease turns into a legal nightmare

      Ralph Messo, DO | Physician
    • Addressing menstrual health inequities in adolescents

      Callia Georgoulis | Conditions
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
  • Recent Posts

    • How shared language saved a patient from isolation

      Syed Ahmad Moosa, MD | Physician
    • The shocking risk every smart student faces when applying to medical school

      Curtis G. Graham, MD | Physician
    • Clinical ghosts and why they haunt our exam rooms

      Kara Wada, MD | Conditions
    • High blood pressure’s hidden impact on kidney health in older adults

      Edmond Kubi Appiah, MPH | Conditions
    • Deep transcranial magnetic stimulation for depression [PODCAST]

      The Podcast by KevinMD | Podcast
    • How declining MMR vaccination rates put future generations at risk

      Ambika Sharma, Onyi Oligbo, and Katrina Green, MD | Conditions

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 1 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • Why specialist pain clinics and addiction treatment services require strong primary care

      Olumuyiwa Bamgbade, MD | Conditions
    • Harassment and overreach are driving physicians to quit

      Olumuyiwa Bamgbade, MD | Physician
    • Why peer support can save lives in high-pressure medical careers

      Maire Daugharty, MD | Conditions
    • When a medical office sublease turns into a legal nightmare

      Ralph Messo, DO | Physician
    • Addressing menstrual health inequities in adolescents

      Callia Georgoulis | Conditions
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
  • Recent Posts

    • How shared language saved a patient from isolation

      Syed Ahmad Moosa, MD | Physician
    • The shocking risk every smart student faces when applying to medical school

      Curtis G. Graham, MD | Physician
    • Clinical ghosts and why they haunt our exam rooms

      Kara Wada, MD | Conditions
    • High blood pressure’s hidden impact on kidney health in older adults

      Edmond Kubi Appiah, MPH | Conditions
    • Deep transcranial magnetic stimulation for depression [PODCAST]

      The Podcast by KevinMD | Podcast
    • How declining MMR vaccination rates put future generations at risk

      Ambika Sharma, Onyi Oligbo, and Katrina Green, MD | Conditions

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Statistical significance doesn’t always warrant a miracle headline
1 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...