Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Can flawed doctor ratings harm patients?

Skeptical Scalpel, MD
Physician
August 10, 2016
Share
Tweet
Share

Can you trust online physician ratings? Many people believe you can.

But here’s what a recent article in the Harvard Business Review (HBR) about the reliability of online user ratings regarding buying a product. The authors concluded that online ratings were generally not to be trusted for three important reasons:

  1. The reviews are usually based on a small sample of users. The authors said, “We can be more confident in an average star rating if the sample size is large and if the variability of the distribution of ratings is smaller (i.e., if different reviewers tend to agree).”
  1. People who leave reviews are not randomly selected to do so. “Consumers with extreme opinions are more likely to post reviews, which is referred to as a ‘brag-and-moan’ bias.” The distribution of ratings is often clustered at 5-star and 1-star with only a few 2s, 3s and 4s.
  1. Most people who post ratings have not purchased comparable products. Therefore, they have no way to test one product against another. “It is well-known that consumers’ quality evaluations are heavily biased by variables other than objective product performance, such as brand image, price, and physical appearance.”

These principles can be applied to physician ratings too.

The most important problem with online physician ratings is sample size. The HBR authors illustrated this with an example.

Take a product with an average rating of 4 stars. If the product was rated by at least 25 users, you can be 95 percent sure that the average is between 3.5 and 4.5.

So if two doctors have fewer than 25 ratings each, there may be no real difference between a doctor with a rating of 3.6 and one with a 4.4 score.

What would you guess is the average number of ratings for physicians online?

According to the National Research Corporation in 2015, “Most online provider ratings are based on an average of only 2.4 reviews per physician.”

When I blogged about a 2014 paper claiming online doctor ratings were reliable, the paper said its authors looked at ratings during a 9-year period and found that 21 percent of cardiac surgeons in Florida were each rated an average of 1.9 times. The 79 percent of surgeons who had no online ratings performed 79 percent of the total surgeries in 2012, the year they chose to look at patient outcomes.

The majority of surgery was done by surgeons who had no online ratings. Comparing surgeons under those conditions is no better than flipping a coin.

What about online sites like ProPublica’s Surgeon Scorecard?

I and others have written about its many shortcomings, but not as succinctly and scientifically as patient safety guru Peter Pronovost and co-authors in the July 2016 issue of Annals of Surgery.

Since the full text of the three-page paper is available here, I will simply quote its conclusions:

ADVERTISEMENT

“Patients and providers deserve valid and transparent performance measures, and hospitals and doctors should be accountable for the care they provide. Flawed measures, however, are not only meaningless but may actually harm patients, providers, and other stakeholders. The measurement enterprise must be held to the same high standards that we appropriately expect of health care providers.”

“Skeptical Scalpel” is a surgeon who blogs at his self-titled site, Skeptical Scalpel.  This article originally appears in Physician’s Weekly.

Image credit: Shutterstock.com

Prev

1 second of every day of the last year of medical school

August 10, 2016 Kevin 0
…
Next

A physician mother as the breadwinner. Here's how she did it.

August 10, 2016 Kevin 2
…

Tagged as: Primary Care

Post navigation

< Previous Post
1 second of every day of the last year of medical school
Next Post >
A physician mother as the breadwinner. Here's how she did it.

ADVERTISEMENT

More by Skeptical Scalpel, MD

  • The hospital CEO who made a surgical incision. What happened?

    Skeptical Scalpel, MD
  • Medical error is not the third leading cause of death

    Skeptical Scalpel, MD
  • Should speed-eating contests be banned?

    Skeptical Scalpel, MD

Related Posts

  • Why social media may be causing real emotional harm

    Edwin Leap, MD
  • Are patients using social media to attack physicians?

    David R. Stukus, MD
  • Patients made this doctor care about politics

    Chad Hayes, MD
  • You are abandoning your patients if you are not active on social media

    Pat Rich
  • Some patients are hesitant to see the doctor. Here’s how we can fix that.

    Arthur Guy
  • Physician Suicide Awareness Day: Where are the patients? 

    Jennifer M. Sweeney

More in Physician

  • How regulatory overreach is destroying innovation in U.S. health care

    Kayvan Haddadan, MD
  • The unseen cost of detachment in radiology

    Dr. Yesu Raju
  • I thought success was a destination. Then I became a doctor.

    Ryan Nadelson, MD
  • How subjective likability practices undermine Canada’s health workforce recruitment and retention

    Olumuyiwa Bamgbade, MD
  • Why judgment is hurting doctors—and how mindfulness can heal

    Jessie Mahoney, MD
  • Why evidence-based management may be an effective strategy for stronger health care leadership and equity

    Olumuyiwa Bamgbade, MD
  • Most Popular

  • Past Week

    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • The hidden health risks in the One Big Beautiful Bill Act

      Trevor Lyford, MPH | Policy
    • How New Mexico became a malpractice lawsuit hotspot

      Patrick Hudson, MD | Physician
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why health care leaders fail at execution—and how to fix it

      Dave Cummings, RN | Policy
    • How digital tools are reshaping the doctor-patient relationship

      Vineet Vishwanath | Tech
  • Past 6 Months

    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • The hidden health risks in the One Big Beautiful Bill Act

      Trevor Lyford, MPH | Policy
  • Recent Posts

    • How regulatory overreach is destroying innovation in U.S. health care

      Kayvan Haddadan, MD | Physician
    • Why the U.S. mental health care system is failing and how to fix it [PODCAST]

      The Podcast by KevinMD | Podcast
    • The unseen cost of detachment in radiology

      Dr. Yesu Raju | Physician
    • I thought success was a destination. Then I became a doctor.

      Ryan Nadelson, MD | Physician
    • Why psychotherapy works and why psychotherapy fails

      Peggy A. Rothbaum, PhD | Conditions
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 20 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

  • Most Popular

  • Past Week

    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • The hidden health risks in the One Big Beautiful Bill Act

      Trevor Lyford, MPH | Policy
    • How New Mexico became a malpractice lawsuit hotspot

      Patrick Hudson, MD | Physician
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why health care leaders fail at execution—and how to fix it

      Dave Cummings, RN | Policy
    • How digital tools are reshaping the doctor-patient relationship

      Vineet Vishwanath | Tech
  • Past 6 Months

    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
    • The hidden health risks in the One Big Beautiful Bill Act

      Trevor Lyford, MPH | Policy
  • Recent Posts

    • How regulatory overreach is destroying innovation in U.S. health care

      Kayvan Haddadan, MD | Physician
    • Why the U.S. mental health care system is failing and how to fix it [PODCAST]

      The Podcast by KevinMD | Podcast
    • The unseen cost of detachment in radiology

      Dr. Yesu Raju | Physician
    • I thought success was a destination. Then I became a doctor.

      Ryan Nadelson, MD | Physician
    • Why psychotherapy works and why psychotherapy fails

      Peggy A. Rothbaum, PhD | Conditions
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Can flawed doctor ratings harm patients?
20 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...