Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Can flawed doctor ratings harm patients?

Skeptical Scalpel, MD
Physician
August 10, 2016
Share
Tweet
Share

Can you trust online physician ratings? Many people believe you can.

But here’s what a recent article in the Harvard Business Review (HBR) about the reliability of online user ratings regarding buying a product. The authors concluded that online ratings were generally not to be trusted for three important reasons:

  1. The reviews are usually based on a small sample of users. The authors said, “We can be more confident in an average star rating if the sample size is large and if the variability of the distribution of ratings is smaller (i.e., if different reviewers tend to agree).”
  1. People who leave reviews are not randomly selected to do so. “Consumers with extreme opinions are more likely to post reviews, which is referred to as a ‘brag-and-moan’ bias.” The distribution of ratings is often clustered at 5-star and 1-star with only a few 2s, 3s and 4s.
  1. Most people who post ratings have not purchased comparable products. Therefore, they have no way to test one product against another. “It is well-known that consumers’ quality evaluations are heavily biased by variables other than objective product performance, such as brand image, price, and physical appearance.”

These principles can be applied to physician ratings too.

The most important problem with online physician ratings is sample size. The HBR authors illustrated this with an example.

Take a product with an average rating of 4 stars. If the product was rated by at least 25 users, you can be 95 percent sure that the average is between 3.5 and 4.5.

So if two doctors have fewer than 25 ratings each, there may be no real difference between a doctor with a rating of 3.6 and one with a 4.4 score.

What would you guess is the average number of ratings for physicians online?

According to the National Research Corporation in 2015, “Most online provider ratings are based on an average of only 2.4 reviews per physician.”

When I blogged about a 2014 paper claiming online doctor ratings were reliable, the paper said its authors looked at ratings during a 9-year period and found that 21 percent of cardiac surgeons in Florida were each rated an average of 1.9 times. The 79 percent of surgeons who had no online ratings performed 79 percent of the total surgeries in 2012, the year they chose to look at patient outcomes.

The majority of surgery was done by surgeons who had no online ratings. Comparing surgeons under those conditions is no better than flipping a coin.

What about online sites like ProPublica’s Surgeon Scorecard?

I and others have written about its many shortcomings, but not as succinctly and scientifically as patient safety guru Peter Pronovost and co-authors in the July 2016 issue of Annals of Surgery.

Since the full text of the three-page paper is available here, I will simply quote its conclusions:

ADVERTISEMENT

“Patients and providers deserve valid and transparent performance measures, and hospitals and doctors should be accountable for the care they provide. Flawed measures, however, are not only meaningless but may actually harm patients, providers, and other stakeholders. The measurement enterprise must be held to the same high standards that we appropriately expect of health care providers.”

“Skeptical Scalpel” is a surgeon who blogs at his self-titled site, Skeptical Scalpel.  This article originally appears in Physician’s Weekly.

Image credit: Shutterstock.com

Prev

1 second of every day of the last year of medical school

August 10, 2016 Kevin 0
…
Next

A physician mother as the breadwinner. Here's how she did it.

August 10, 2016 Kevin 2
…

Tagged as: Primary Care

Post navigation

< Previous Post
1 second of every day of the last year of medical school
Next Post >
A physician mother as the breadwinner. Here's how she did it.

ADVERTISEMENT

More by Skeptical Scalpel, MD

  • The hospital CEO who made a surgical incision. What happened?

    Skeptical Scalpel, MD
  • Medical error is not the third leading cause of death

    Skeptical Scalpel, MD
  • Should speed-eating contests be banned?

    Skeptical Scalpel, MD

Related Posts

  • Why social media may be causing real emotional harm

    Edwin Leap, MD
  • Are patients using social media to attack physicians?

    David R. Stukus, MD
  • Patients made this doctor care about politics

    Chad Hayes, MD
  • You are abandoning your patients if you are not active on social media

    Pat Rich
  • Some patients are hesitant to see the doctor. Here’s how we can fix that.

    Arthur Guy
  • Physician Suicide Awareness Day: Where are the patients? 

    Jennifer M. Sweeney

More in Physician

  • Why the heart of medicine is more than science

    Ryan Nadelson, MD
  • How Ukrainian doctors kept diabetes care alive during the war

    Dr. Daryna Bahriy
  • How women physicians can go from burnout to thriving

    Diane W. Shannon, MD, MPH
  • Why more doctors are choosing direct care over traditional health care

    Grace Torres-Hodges, DPM, MBA
  • How to handle chronically late patients in your medical practice

    Neil Baum, MD
  • How early meetings and after-hours events penalize physician-mothers

    Samira Jeimy, MD, PhD and Menaka Pai, MD
  • Most Popular

  • Past Week

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • America’s ER crisis: Why the system is collapsing from within

      Kristen Cline, BSN, RN | Conditions
    • Why timing, not surgery, determines patient survival

      Michael Karch, MD | Conditions
    • How early meetings and after-hours events penalize physician-mothers

      Samira Jeimy, MD, PhD and Menaka Pai, MD | Physician
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
  • Recent Posts

    • Why the heart of medicine is more than science

      Ryan Nadelson, MD | Physician
    • How Ukrainian doctors kept diabetes care alive during the war

      Dr. Daryna Bahriy | Physician
    • Why Grok 4 could be the next leap for HIPAA-compliant clinical AI

      Harvey Castro, MD, MBA | Tech
    • How women physicians can go from burnout to thriving

      Diane W. Shannon, MD, MPH | Physician
    • What a childhood stroke taught me about the future of neurosurgery and the promise of vagus nerve stimulation

      William J. Bannon IV | Conditions
    • Beyond burnout: Understanding the triangle of exhaustion [PODCAST]

      The Podcast by KevinMD | Podcast

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 20 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • America’s ER crisis: Why the system is collapsing from within

      Kristen Cline, BSN, RN | Conditions
    • Why timing, not surgery, determines patient survival

      Michael Karch, MD | Conditions
    • How early meetings and after-hours events penalize physician-mothers

      Samira Jeimy, MD, PhD and Menaka Pai, MD | Physician
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
  • Recent Posts

    • Why the heart of medicine is more than science

      Ryan Nadelson, MD | Physician
    • How Ukrainian doctors kept diabetes care alive during the war

      Dr. Daryna Bahriy | Physician
    • Why Grok 4 could be the next leap for HIPAA-compliant clinical AI

      Harvey Castro, MD, MBA | Tech
    • How women physicians can go from burnout to thriving

      Diane W. Shannon, MD, MPH | Physician
    • What a childhood stroke taught me about the future of neurosurgery and the promise of vagus nerve stimulation

      William J. Bannon IV | Conditions
    • Beyond burnout: Understanding the triangle of exhaustion [PODCAST]

      The Podcast by KevinMD | Podcast

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Can flawed doctor ratings harm patients?
20 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...