Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Slow thinking and machine learning in medicine

Drea Burbank, MD
Tech
November 9, 2018
Share
Tweet
Share

Recently, several high-profile institutions have called attention to the issue of inclusion and equity when artificial intelligence (AI) algorithms are applied in medicine. Leaders from the law, medicine, social sciences and computer sciences are speaking out about the challenges of using smart algorithms to solve social problems.

While this topic might easily be dismissed as anti-progress by the tech community, it should be reexamined as a genuine offer to partner leading human slow-thinkers with intuitive fast-thinking machines. Collaboration can improve the impact of AI on society.

How cognitive algorithms can be biased

The terms “slow-thinking” and “fast-thinking” were coined by psychologist Daniel Kahneman who shared a 2002 Nobel Prize for identifying routine cognitive biases in humans — including academics trained in statistics. Kahneman’s work replaced prevailing theories of humans as fully logical, utilitarian decision-makers with a more compassionate view of choices made using flawed, altruistic and often irrational cognitive algorithms.

The takeaway should not be that humans are stupid, but rather to acknowledge that rational thought and cognitive biases coexist in all of us — despite our best intentions. We need to design independent safeguards to protect our decision-making from sloppy heuristics.

Recently, leading scientists at the Stanford Presence Center’s AI in Medicine: Inclusion and Equity (AiMIE) symposium pointed out that this process applies to machines as well as humans.

Current controls on human cognition

Intuition evolved to be the quick-and-dirty substitute for thought when speed was necessary for survival.

Although efficient, intuition is also subconscious, illogical and biased by experience. The Implicit Association Test from Harvard identifies subconscious biases from microsecond delays in choice. It was publicized on the web so scientists, public servants and policymakers can be more aware of and compensate for prejudice. A societal example of formalized processes for the same includes research ethics boards whose members are recruited from diverse sectors in society to analyze the ethical implications of proposed research projects.

So how is this relevant to AI?

We might assume that machine learning is better at solving problems with emotionless intuition. However, it is designed to mimic human cognitive processes, which makes it as susceptible to fast-thinking bias as human thought.

“I think we need to move beyond assuming that technology can inherently solve social or economic problems,” says Dr. Sanjay Basu an Assistant Professor of Medicine at Stanford University, “And instead ask why these problems persist and whether our technology or other tools are really being designed to reinforce or challenge these problems.”

How machine learning algorithms can be biased

ADVERTISEMENT

At the AiMIE symposium, several luminaries weighed in on potential sources of AI bias that need to be intentionally corrected:

Access to data. Because there is a digital divide in access to smartphones and computers, disadvantaged populations are persistently underrepresented in public datasets. Just like human intuition, if the data is biased, so are the algorithms. As Dr. Nirav Shah, Stanford professor and former COO of clinical operations for Kaiser Permanente in Southern California artfully suggested, “Maybe data should be considered a determinant of health.”

Failure in algorithm design. One of the values of lean computer programming is to “release early, release often,” but executing prototypes in social sciences can be disastrous. As Dr. Mark Cullen, director of the Stanford Population for Health Studies, dryly commented, “Intelligence rather than artificial intelligence might be more useful in reducing disparities in the short run.”

Positive-feedback loops. Flaws in AI design may compound. As Judge Mariano Cuellar, a California Supreme Court justice explains: “If we train the machines only on the data we have today we will end up with machines that deepen the inequities of our system.”

Failure to adjust. While humans can comprehend outliers and tailor solutions, machines are universally merciless. “Personalization can very easily become persecution.” explains Virginia Eubanks, a professor of political science at the University of Albany and author of the recent book Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor which illustrated several recent injustices from poorly-designed AI algorithms used in disadvantaged populations.

So how do we prevent bias?

Machine bias can be addressed with the same mechanisms we use to compensate for human bias: structured thought, written logic and checking assumptions with diverse representation.

In conclusion, we need humans and AI

Just like human thought, intuitive decision-making and strategic decision-making coexist and each has benefits in different situations.

While we need the analytical tools that AI and machine-learning can give us, we also need human capacities for messy debate, structured thought and the willingness to face uncomfortable realities.

Humans trained in this type of cognition often work in public roles such as social sciences, law and medicine. These partners are willing and able to collaborate with industry — often without compensation. These collaborations may take time and effort and require clear negotiation of competing interests up front; however, they make innovative products more robust, valuable and available to a broader audience.

Perhaps what we need is not more human thinking or more AI, but better integration of the two heuristics so that we can achieve a cognitive stereovision.

In closing, we offer a quote by Glenn Cohen, JD, a Harvard law professor and another speaker at AiMIE. “William Gibson once said something along the lines of ‘The future is already here, it is just not very evenly distributed.’ If the use of AI in health care is going to improve care for everyone, we have to make sure its starting assumptions, training sets and cascade effects reduce not exacerbate existing divisions.”

Drea Burbank is a physician-entrepreneur. Ayo Roberts and Chandi Broadbent also contributed to this article.

Image credit: Shutterstock.com

Prev

8 tips to land the residency of your dreams

November 9, 2018 Kevin 4
…
Next

MKSAP: 64-year-old man seen after an intraoperative liver biopsy

November 10, 2018 Kevin 0
…

Tagged as: Public Health & Policy

Post navigation

< Previous Post
8 tips to land the residency of your dreams
Next Post >
MKSAP: 64-year-old man seen after an intraoperative liver biopsy

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Drea Burbank, MD

  • How doctors can regain control of their software

    Drea Burbank, MD
  • How doctors should organize

    Drea Burbank, MD
  • How physicians are mishandling technology

    Drea Burbank, MD

Related Posts

  • The difference between learning medicine and doing medicine

    Steven Zhang, MD
  • When learning medicine is not enough

    Hanna Saltzman
  • Medicine, fast and slow

    Claire Brown
  • The excitement of clinical rotations: Not just learning medicine but doing medicine

    Orly Farber
  • 3 lessons I’m learning about practicing medicine

    Klaus Kessel
  • How social media can advance humanism in medicine

    Pooja Lakshmin, MD

More in Tech

  • In medicine and law, professions that society relies upon for accuracy

    Muhamad Aly Rifai, MD
  • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

    Harvey Castro, MD, MBA
  • Why fearing AI is really about fearing ourselves

    Bhargav Raman, MD, MBA
  • Health care’s data problem: the real obstacle to AI success

    Jay Anders, MD
  • What ChatGPT’s tone reveals about our cultural values

    Jenny Shields, PhD
  • Bridging the digital divide: Addressing health inequities through home-based AI solutions

    Dr. Sreeram Mullankandy
  • Most Popular

  • Past Week

    • How dismantling DEI endangers the future of medical care

      Shashank Madhu and Christian Tallo | Education
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions
    • Do Jewish students face rising bias in holistic admissions?

      Anonymous | Education
    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
    • Residency as rehearsal: the new pediatric hospitalist fellowship requirement scam

      Anonymous | Physician
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • The hidden bias in how we treat chronic pain

      Richard A. Lawhern, PhD | Meds
  • Recent Posts

    • Antimicrobial resistance: a public health crisis that needs your voice [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why a fourth year will not fix emergency medicine’s real problems

      Anna Heffron, MD, PhD & Polly Wiltz, DO | Education
    • Why shared decision-making in medicine often fails

      M. Bennet Broner, PhD | Conditions
    • Do Jewish students face rising bias in holistic admissions?

      Anonymous | Education
    • She wouldn’t move in the womb—then came the rare diagnosis that changed everything

      Amber Robertson | Conditions
    • Rethinking medical education for a technology-driven era in health care [PODCAST]

      The Podcast by KevinMD | Podcast

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

Leave a Comment

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • How dismantling DEI endangers the future of medical care

      Shashank Madhu and Christian Tallo | Education
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions
    • Do Jewish students face rising bias in holistic admissions?

      Anonymous | Education
    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
    • Residency as rehearsal: the new pediatric hospitalist fellowship requirement scam

      Anonymous | Physician
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • The hidden bias in how we treat chronic pain

      Richard A. Lawhern, PhD | Meds
  • Recent Posts

    • Antimicrobial resistance: a public health crisis that needs your voice [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why a fourth year will not fix emergency medicine’s real problems

      Anna Heffron, MD, PhD & Polly Wiltz, DO | Education
    • Why shared decision-making in medicine often fails

      M. Bennet Broner, PhD | Conditions
    • Do Jewish students face rising bias in holistic admissions?

      Anonymous | Education
    • She wouldn’t move in the womb—then came the rare diagnosis that changed everything

      Amber Robertson | Conditions
    • Rethinking medical education for a technology-driven era in health care [PODCAST]

      The Podcast by KevinMD | Podcast

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Leave a Comment

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...