Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Everyone needs rudimentary statistical training

Christopher Johnson, MD
Physician
February 27, 2020
Share
Tweet
Share

Every day we get bombarded in the news with health statistics. Coffee causes cancer! Coffee cures cancer! And so on. Many of these are meant to grab headlines (and, these days, web page clicks), and the articles they accompany are often very poor at telling the reader what they mean. They often have statistics, and health statistics can be complicated. Sad to say, even many physicians are pretty poor and sorting out the hype from the helpful. This article is a very helpful guide to finding your way when you are reading the health news. It’s called, “Helping Doctors and Patients Make Sense of Health Statistics,” and it does just that. I’d bookmark it or even print it out for future reference, if you’re more old school. Don’t be put off by the somber-looking first page — it’s actually quite readable. I should point out here that, although I took statistics courses long ago and have used simple statistical tests in my own research career, I am by no means expert. I always consulted a real statistician before submitting any research for publication.

The article starts with the common problem one sees in media reports: the difference between absolute and relative risk. The authors used the example of a scare over birth control pills that happened in 1995 when the U.K. Committee on Safety in Medicines issued a warning about a newer version of pills. The committee sent a warning to all physicians that the newer pills were associated with a 100 percent rise in risk for serious blood clots. One hundred percent! Yikes! The warning led to many women stopping their pills, and there was a predictable rise in unwanted pregnancies, accompanied by an estimated 15,000 more abortions the following year. The effects lasted for years. What was the truth about this new risk?

The truth was that the newer pills were associated with a risk for serious blood clots of 2 per 7,000 women. For comparison, the earlier pills had been associated with a 1 per 7,000 women risk. And 2 is 100 percent more than 1, so that’s the increase in relative risk. But the increase in absolute risk was an additional 1 woman in 7,000. (It should be noted here that pregnancy itself is associated with an increased risk of blood clots.) I see this particular misunderstanding often in news reports regarding risks of medical procedures. When you read these stories, you need to examine not just the relative risk, which often makes for good headlines; you need to look at the actual number, not just the percent change.

The pill scare hurt women, hurt the National Health Service, and even hurt the pharmaceutical industry. Among the few to profit were the journalists who got the story on the front page.

There is also this helpful article, from the always readable Scientific American. I like it a lot, too. It reminds us how statistical significance doesn’t always mean real-life significance.

Imagine if there were a simple single statistical measure everybody could use with any set of data and it would reliably separate true from false. Oh, the things we would know! Unrealistic to expect such wizardry though, huh? Yet, statistical significance is commonly treated as though it is that magic wand. Take a null hypothesis or look for any association between factors in a data set and abracadabra! Get a “p value” over or under 0.05 and you can be 95 percent certain it’s either a fluke or it isn’t. You can eliminate the play of chance! You can separate the signal from the noise! Except that you can’t. That’s not really what testing for statistical significance does. And therein lies the rub.

The article points out what the vaunted, and too often venerated, p-value means is that it estimates the probability of getting roughly that result if the study hypothesis is assumed to be true. It can’t on its own tell you whether this assumption was right, or whether the results would hold true in different circumstances. It provides a limited picture of probability, taking limited information about the data into account and giving only “yes” or “no” as options.

If you are really interested in this topic, you should read a bit about what’s called Bayesian statistics, named after an 18th-century mathematician. The basic notion here is that we need to consider our prior knowledge about something before applying statistical tests, and that we should factor in this knowledge when we make our statistical comparisons. In other words, all possibilities are not intrinsically equal going into the analysis. The debates between Bayesian and what are termed “frequentist” statisticians go back and forth. But what we should take home from these debates is that the science of statistics, like other sciences, is subject to revision and change over time.

A final key point is to look at medical headlines of new medical breakthroughs and try to decide if the findings really are “significant” in real life. Is there only a tiny effect, really, even though the p-value is “significant” at the 0.05 level? Also, beware of what we call “data dredging,” in which multiple comparisons are made using the same data set. When you do that the chances of coming up with a significant, yet spurious association goes up.

All of this has made some people call for some rudimentary statistical training to be part of the standard mathematics curriculum at the high school level. I think this is a good idea. I didn’t get introduced to any statistical concepts in high school, and I took all the math available. That should change if we expect the mass of our citizenry to be competent to judge things for themselves. Medical journalists definitely need this knowledge because currently, many do a terrible job interpreting medical reports. The two articles I linked are a great place to start.

Christopher Johnson is a pediatric intensive care physician and author of Keeping Your Kids Out of the Emergency Room: A Guide to Childhood Injuries and Illnesses, Your Critically Ill Child: Life and Death Choices Parents Must Face, How to Talk to Your Child’s Doctor: A Handbook for Parents, and How Your Child Heals: An Inside Look At Common Childhood Ailments. He blogs at his self-titled site, Christopher Johnson, MD.

Image credit: Shutterstock.com

Prev

When an aging parent says no to much needed help

February 27, 2020 Kevin 0
…
Next

Core values were key to reclaiming my physician identity

February 28, 2020 Kevin 0
…

ADVERTISEMENT

Tagged as: Hospital-Based Medicine, Practice Management

Post navigation

< Previous Post
When an aging parent says no to much needed help
Next Post >
Core values were key to reclaiming my physician identity

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Christopher Johnson, MD

  • The success of Australian firearms regulation: What it could mean for children

    Christopher Johnson, MD
  • Do protocols and pathways improve care?

    Christopher Johnson, MD
  • Why are so many community hospitals transferring children to larger facilities?

    Christopher Johnson, MD

Related Posts

  • How the COVID-19 pandemic highlights the need for social media training in medical education 

    Oscar Chen, Sera Choi, and Clara Seong
  • Residency training, and training in residency

    Michelle Meyer, MD
  • Dealing with the pressures of learning as a physician-in-training

    Linda Nguyen
  • Why doctors-in-training need better nutritional education

    Abeer Arain, MD, MPH
  • The vulnerability of abortion access and training

    Shereen Jeyakumar
  • Why medical students need more continuity of care training

    Nathaniel Fleming

More in Physician

  • The broken health care system doesn’t have to break you

    Jessie Mahoney, MD
  • How a $75 million jet brought down America’s boldest doctor

    Arthur Lazarus, MD, MBA
  • The dreaded question: Do you have boys or girls?

    Pamela Adelstein, MD
  • When rock bottom is a turning point: Why the turmoil at HHS may be a blessing in disguise

    Muhamad Aly Rifai, MD
  • How grief transformed a psychiatrist’s approach to patient care

    Devina Maya Wadhwa, MD
  • Fear of other people’s opinions nearly killed me. Here’s what freed me.

    Jillian Rigert, MD, DMD
  • Most Popular

  • Past Week

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
    • Make cognitive testing as routine as a blood pressure check

      Joshua Baker and James Jackson, PsyD | Conditions
    • The dreaded question: Do you have boys or girls?

      Pamela Adelstein, MD | Physician
    • A world without antidepressants: What could possibly go wrong?

      Tomi Mitchell, MD | Meds
    • Rethinking patient payments: Why billing is the new frontline of patient care [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • The silent crisis hurting pain patients and their doctors

      Kayvan Haddadan, MD | Physician
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • What happened to real care in health care?

      Christopher H. Foster, PhD, MPA | Policy
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • The hidden bias in how we treat chronic pain

      Richard A. Lawhern, PhD | Meds
  • Recent Posts

    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • Why great patient outcomes don’t protect female doctors from burnout [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why ADHD in women is finally getting the attention it deserves

      Arti Lal, MD | Conditions
    • How a $75 million jet brought down America’s boldest doctor

      Arthur Lazarus, MD, MBA | Physician
    • Why ruling out sepsis in emergency departments can be lifesaving

      Claude M. D'Antonio, Jr., MD | Conditions
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 1 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
    • Make cognitive testing as routine as a blood pressure check

      Joshua Baker and James Jackson, PsyD | Conditions
    • The dreaded question: Do you have boys or girls?

      Pamela Adelstein, MD | Physician
    • A world without antidepressants: What could possibly go wrong?

      Tomi Mitchell, MD | Meds
    • Rethinking patient payments: Why billing is the new frontline of patient care [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • The silent crisis hurting pain patients and their doctors

      Kayvan Haddadan, MD | Physician
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • What happened to real care in health care?

      Christopher H. Foster, PhD, MPA | Policy
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • The hidden bias in how we treat chronic pain

      Richard A. Lawhern, PhD | Meds
  • Recent Posts

    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • Why great patient outcomes don’t protect female doctors from burnout [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why ADHD in women is finally getting the attention it deserves

      Arti Lal, MD | Conditions
    • How a $75 million jet brought down America’s boldest doctor

      Arthur Lazarus, MD, MBA | Physician
    • Why ruling out sepsis in emergency departments can be lifesaving

      Claude M. D'Antonio, Jr., MD | Conditions
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Everyone needs rudimentary statistical training
1 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...