Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Can flawed doctor ratings harm patients?

Skeptical Scalpel, MD
Physician
August 10, 2016
Share
Tweet
Share

Can you trust online physician ratings? Many people believe you can.

But here’s what a recent article in the Harvard Business Review (HBR) about the reliability of online user ratings regarding buying a product. The authors concluded that online ratings were generally not to be trusted for three important reasons:

  1. The reviews are usually based on a small sample of users. The authors said, “We can be more confident in an average star rating if the sample size is large and if the variability of the distribution of ratings is smaller (i.e., if different reviewers tend to agree).”
  1. People who leave reviews are not randomly selected to do so. “Consumers with extreme opinions are more likely to post reviews, which is referred to as a ‘brag-and-moan’ bias.” The distribution of ratings is often clustered at 5-star and 1-star with only a few 2s, 3s and 4s.
  1. Most people who post ratings have not purchased comparable products. Therefore, they have no way to test one product against another. “It is well-known that consumers’ quality evaluations are heavily biased by variables other than objective product performance, such as brand image, price, and physical appearance.”

These principles can be applied to physician ratings too.

The most important problem with online physician ratings is sample size. The HBR authors illustrated this with an example.

Take a product with an average rating of 4 stars. If the product was rated by at least 25 users, you can be 95 percent sure that the average is between 3.5 and 4.5.

So if two doctors have fewer than 25 ratings each, there may be no real difference between a doctor with a rating of 3.6 and one with a 4.4 score.

What would you guess is the average number of ratings for physicians online?

According to the National Research Corporation in 2015, “Most online provider ratings are based on an average of only 2.4 reviews per physician.”

When I blogged about a 2014 paper claiming online doctor ratings were reliable, the paper said its authors looked at ratings during a 9-year period and found that 21 percent of cardiac surgeons in Florida were each rated an average of 1.9 times. The 79 percent of surgeons who had no online ratings performed 79 percent of the total surgeries in 2012, the year they chose to look at patient outcomes.

The majority of surgery was done by surgeons who had no online ratings. Comparing surgeons under those conditions is no better than flipping a coin.

What about online sites like ProPublica’s Surgeon Scorecard?

I and others have written about its many shortcomings, but not as succinctly and scientifically as patient safety guru Peter Pronovost and co-authors in the July 2016 issue of Annals of Surgery.

Since the full text of the three-page paper is available here, I will simply quote its conclusions:

ADVERTISEMENT

“Patients and providers deserve valid and transparent performance measures, and hospitals and doctors should be accountable for the care they provide. Flawed measures, however, are not only meaningless but may actually harm patients, providers, and other stakeholders. The measurement enterprise must be held to the same high standards that we appropriately expect of health care providers.”

“Skeptical Scalpel” is a surgeon who blogs at his self-titled site, Skeptical Scalpel.  This article originally appears in Physician’s Weekly.

Image credit: Shutterstock.com

Prev

1 second of every day of the last year of medical school

August 10, 2016 Kevin 0
…
Next

A physician mother as the breadwinner. Here's how she did it.

August 10, 2016 Kevin 2
…

Tagged as: Primary Care

Post navigation

< Previous Post
1 second of every day of the last year of medical school
Next Post >
A physician mother as the breadwinner. Here's how she did it.

ADVERTISEMENT

More by Skeptical Scalpel, MD

  • The hospital CEO who made a surgical incision. What happened?

    Skeptical Scalpel, MD
  • Medical error is not the third leading cause of death

    Skeptical Scalpel, MD
  • Should speed-eating contests be banned?

    Skeptical Scalpel, MD

Related Posts

  • Why social media may be causing real emotional harm

    Edwin Leap, MD
  • Are patients using social media to attack physicians?

    David R. Stukus, MD
  • Patients made this doctor care about politics

    Chad Hayes, MD
  • You are abandoning your patients if you are not active on social media

    Pat Rich
  • Some patients are hesitant to see the doctor. Here’s how we can fix that.

    Arthur Guy
  • Physician Suicide Awareness Day: Where are the patients? 

    Jennifer M. Sweeney

More in Physician

  • The backbone of health care is breaking

    Grace Yu, MD
  • Why doctors must ask for help before burnout escalates

    Diane W. Shannon, MD, MPH
  • Why medicine is like a jazz solo

    Arthur Lazarus, MD, MBA
  • Why so many patients ask for drips—and what doctors wish they knew

    Dr. Akintola Aminat Olayinka
  • Reproductive care after Roe: Why silence is not an option

    Christine Petrin, MD, MPH and Susan Thompson Hingle, MD
  • When your identity is your job: Why it’s dangerous in medicine

    Brooke Buckley, MD, MBA
  • Most Popular

  • Past Week

    • Health equity in Inland Southern California requires urgent action

      Vishruth Nagam | Policy
    • Could antibiotics beat heart disease where statins failed?

      Larry Kaskel, MD | Conditions
    • How restrictive opioid policies worsen the crisis

      Kayvan Haddadan, MD | Physician
    • Why palliative care is more than just end-of-life support

      Dr. Vishal Parackal | Conditions
    • My improbable survival of stage 4 cancer

      Kelly Curtin-Hallinan, DO | Conditions
    • How Ukrainian doctors sustained diabetes care during the war [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why transgender health care needs urgent reform and inclusive practices

      Angela Rodriguez, MD | Conditions
    • COVID-19 was real: a doctor’s frontline account

      Randall S. Fong, MD | Conditions
    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • Why pain doctors face unfair scrutiny and harsh penalties in California

      Kayvan Haddadan, MD | Physician
    • mRNA post vaccination syndrome: Is it real?

      Harry Oken, MD | Conditions
  • Recent Posts

    • How Ukrainian doctors sustained diabetes care during the war [PODCAST]

      The Podcast by KevinMD | Podcast
    • The backbone of health care is breaking

      Grace Yu, MD | Physician
    • Why doctors must ask for help before burnout escalates

      Diane W. Shannon, MD, MPH | Physician
    • Why medicine is like a jazz solo

      Arthur Lazarus, MD, MBA | Physician
    • Why so many patients ask for drips—and what doctors wish they knew

      Dr. Akintola Aminat Olayinka | Physician
    • Reproductive care after Roe: Why silence is not an option

      Christine Petrin, MD, MPH and Susan Thompson Hingle, MD | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 20 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Health equity in Inland Southern California requires urgent action

      Vishruth Nagam | Policy
    • Could antibiotics beat heart disease where statins failed?

      Larry Kaskel, MD | Conditions
    • How restrictive opioid policies worsen the crisis

      Kayvan Haddadan, MD | Physician
    • Why palliative care is more than just end-of-life support

      Dr. Vishal Parackal | Conditions
    • My improbable survival of stage 4 cancer

      Kelly Curtin-Hallinan, DO | Conditions
    • How Ukrainian doctors sustained diabetes care during the war [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why transgender health care needs urgent reform and inclusive practices

      Angela Rodriguez, MD | Conditions
    • COVID-19 was real: a doctor’s frontline account

      Randall S. Fong, MD | Conditions
    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • Why pain doctors face unfair scrutiny and harsh penalties in California

      Kayvan Haddadan, MD | Physician
    • mRNA post vaccination syndrome: Is it real?

      Harry Oken, MD | Conditions
  • Recent Posts

    • How Ukrainian doctors sustained diabetes care during the war [PODCAST]

      The Podcast by KevinMD | Podcast
    • The backbone of health care is breaking

      Grace Yu, MD | Physician
    • Why doctors must ask for help before burnout escalates

      Diane W. Shannon, MD, MPH | Physician
    • Why medicine is like a jazz solo

      Arthur Lazarus, MD, MBA | Physician
    • Why so many patients ask for drips—and what doctors wish they knew

      Dr. Akintola Aminat Olayinka | Physician
    • Reproductive care after Roe: Why silence is not an option

      Christine Petrin, MD, MPH and Susan Thompson Hingle, MD | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Can flawed doctor ratings harm patients?
20 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...