Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Can chatbots help choose the right doctor or hospital?

Michael L. Millenson
Tech
June 15, 2023
Share
Tweet
Share

While ChatGPT and Bard cannot provide reliable medical or treatment advice, there is a question of whether these chatbots can assist patients in selecting the right doctor or hospital. Surprisingly, both chatbots occasionally suggest individual physicians and hospitals, providing detailed provider information. However, the accuracy and usefulness of this information are difficult to assess. As I highlighted in a recent Forbes.com column, chatbots like Microsoft and OpenAI’s ChatGPT and Google’s Bard blend truth with “truthiness,” a term coined by comedian Stephen Colbert to describe information that feels true without evidence to support it.

For example, Bard confidently provided me with statistics on the number of knee replacement surgeries conducted by major Chicago hospitals in 2021, along with their infection rates and the national average. It even named the Chicago surgeon with the highest number of knee surgeries and provided their infection rate. When I inquired about heart bypass surgery, Bard furnished both the mortality rate for certain local hospitals and a national average.

While Bard occasionally cited sources like the American Hospital Association, the Centers for Medicare & Medicaid Services, and the Joint Commission, it also referred to its own knowledge. Aware that generative AI can produce false information, I fact-checked the data with these organizations, but none of them had the data attributed to them.

With chatbots, the wording of the query matters. When I asked Bard about the surgeon performing the most knee replacements in Chicago, it provided one name. However, when I broadened the question to encompass the “Chicago area,” Bard listed seven highly skilled and experienced surgeons with a history of successful outcomes and compassionate care.

Although Bard included cautionary language in its responses, such as acknowledging the risks associated with surgery, it still unequivocally recommended scheduling a consultation with one of the seven surgeons for knee replacement. On the other hand, ChatGPT did not make direct recommendations but did provide a list of four top knee replacement surgeons based on their expertise and patient outcomes.

The methods used by Bard and ChatGPT to generate these answers remain unknown. We are uncertain why they provide one, four, or seven names. To assess the validity of the physician recommendations, I conducted a basic test by asking Bard to recommend good restaurants in the Chicago area. The responses included reputable establishments, ranging from a Michelin Guide three-star restaurant in Lincoln Park to a gastropub renowned for its beer selection in the West Loop. Alternative phrasing yielded a list of nine restaurants, including four from the initial list, along with three well-regarded local pizza chains. ChatGPT produced ten restaurant names in response to the first query and eight in the second, with some overlap with Bard’s suggestions.

Ideally, the best way to address these questionable responses from chatbots would involve surgeons and hospitals disclosing accurate and standardized data on procedure volumes and complication rates promptly. While waiting for this transparency (which may take a while), I believe it is essential for physicians to advocate for collaborative efforts involving organizations like the American Medical Association and the American Hospital Association, with the inclusion of patient groups, to develop responsible solutions for this problem.

In the meantime, the public is left to wonder whether the doctors listed by chatbots as heart specialists are akin to a satisfying pizza or chefs who, with three Michelin stars, are among the best in the world.

Michael L. Millenson is president, Health Quality Advisors, LLC and can be reached on his self-titled site, Michael L. Millenson.

Prev

Syndicators and risk management: Educating investors on multifamily syndication

June 15, 2023 Kevin 0
…
Next

Unlocking the secrets to aging well [PODCAST]

June 15, 2023 Kevin 0
…

Tagged as: Health IT

Post navigation

< Previous Post
Syndicators and risk management: Educating investors on multifamily syndication
Next Post >
Unlocking the secrets to aging well [PODCAST]

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Michael L. Millenson

  • My totally wrong expert predictions for health care in 2025

    Michael L. Millenson
  • The hidden benefits of your health insurance plan can save your life

    Michael L. Millenson
  • Your bone fracture, my cash flow: the consequences of private equity in health care

    Michael L. Millenson

Related Posts

  • Don’t judge when trainees use dating apps in the hospital

    Austin Perlmutter, MD
  • How to choose the right rehab option after a hospital stay

    Edward Hoffer, MD
  • Are hospital CEOs responding to the realities of health care?

    Ammura Hernandez, MD
  • Are negative news cycles and social media injurious to our health?

    Rabia Jalal, MD
  • How social media can help or hurt your health care career

    Health eCareers
  • Sharing mental health issues on social media

    Tarena Lofton

More in Tech

  • Why interoperability is key to achieving the quintuple aim in health care

    Steven Lane, MD
  • How Mark Twain would dismantle today’s flawed medical AI

    Neil Baum, MD and Mark Ibsen, MD
  • 9 domains that will define the future of medical education

    Harvey Castro, MD, MBA
  • Key strategies for smooth EHR transitions in health care

    Sandra Johnson
  • Why flashy AI tools won’t fix health care without real infrastructure

    David Carmouche, MD
  • Why innovation in health care starts with bold thinking

    Miguel Villagra, MD
  • Most Popular

  • Past Week

    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • When did we start treating our lives like trauma?

      Maureen Gibbons, MD | Physician
    • First impressions happen online—not in your exam room

      Sara Meyer | Social media
    • How Gen Z is transforming mental health care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Dedicated hypermobility clinics can transform patient care

      Katharina Schwan, MPH | Conditions
    • Why ADHD in adults is often missed—and why it matters [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • Why recovery after illness demands dignity, not suspicion

      Trisza Leann Ray, DO | Physician
    • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

      Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO | Meds
    • Why medical students are trading empathy for publications

      Vijay Rajput, MD | Education
  • Recent Posts

    • How Gen Z is transforming mental health care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Nurses aren’t eating their young — we’re starving the profession

      Adam J. Wickett, BSN, RN | Conditions
    • Why wanting more from your medical career is a sign of strength

      Maureen Gibbons, MD | Physician
    • U.S. health care leadership must prepare for policy-driven change

      Lee Scheinbart, MD | Policy
    • Why the pre-med path is pushing future doctors to the brink

      Jordan Williamson, MEd | Education
    • Why the fear of being forgotten is stronger than the fear of death [PODCAST]

      The Podcast by KevinMD | Podcast

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

Leave a Comment

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • When did we start treating our lives like trauma?

      Maureen Gibbons, MD | Physician
    • First impressions happen online—not in your exam room

      Sara Meyer | Social media
    • How Gen Z is transforming mental health care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Dedicated hypermobility clinics can transform patient care

      Katharina Schwan, MPH | Conditions
    • Why ADHD in adults is often missed—and why it matters [PODCAST]

      The Podcast by KevinMD | Podcast
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • Why recovery after illness demands dignity, not suspicion

      Trisza Leann Ray, DO | Physician
    • Why does rifaximin cost 95 percent more in the U.S. than in Asia?

      Jai Kumar, MD, Brian Nohomovich, DO, PhD and Leonid Shamban, DO | Meds
    • Why medical students are trading empathy for publications

      Vijay Rajput, MD | Education
  • Recent Posts

    • How Gen Z is transforming mental health care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Nurses aren’t eating their young — we’re starving the profession

      Adam J. Wickett, BSN, RN | Conditions
    • Why wanting more from your medical career is a sign of strength

      Maureen Gibbons, MD | Physician
    • U.S. health care leadership must prepare for policy-driven change

      Lee Scheinbart, MD | Policy
    • Why the pre-med path is pushing future doctors to the brink

      Jordan Williamson, MEd | Education
    • Why the fear of being forgotten is stronger than the fear of death [PODCAST]

      The Podcast by KevinMD | Podcast

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Leave a Comment

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...