Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

AI in health care: the black box of prior authorization

P. Dileep Kumar, MD, MBA
Tech
April 14, 2025
Share
Tweet
Share

Amidst the recent news about the murder of UnitedHealthcare CEO, the issue of prior authorization has come to the forefront again. Insurance companies use prior authorization as an easy tool to control the costs and incorporate it as a genuine step of the health care delivery process. The recent trend is to use AI (artificial intelligence) for prior authorizations. AI offers several advantages over human workers, further saving time and money.

Using AI for prior authorizations opens an entirely novel type of Pandora’s box in health care. A reduction of 50–75 percent of the manual task was achieved by using automated AI for prior authorization in 2022. Natural language processing (NLP) can be used to extract relevant information from the myriads of data submitted to the insurance companies and formulate a decision. AI programs can also be trained on previous customer data and previous decision patterns of the company. Algorithms can be tweaked to approve or deny treatment or medication requests from physicians. This could be a purely financial decision, nothing to do with health care delivery. AI can also ostensibly route queries to the appropriate reviewer faster. A federal class action lawsuit filed in 2023 against UnitedHealthcare Group alleges a 90 percent error rate in AI programs used for prior authorizations. The lawsuit also alleges that only 0.2 percent of the health policyholders are likely to appeal the denials because many consider it a hassle and a fruitless effort. The company apparently used a prediction model called “nH Predict” to determine coverage for Medicare Advantage patients in a post-acute care setting. According to the lawsuit, the model uses rigid and unrealistic predictions of recovery, often overruling the treating physicians’ determinations. Seemingly, “nH Predict” uses data from a few million patients compiled over years. The details of the machine learning algorithm used in the program are proprietary and hence not publicly known.

This issue brings the “black box” problem of AI programs. While transparency is important for trust building, AI companies encourage minimizing transparency because of various reasons such as maintaining competitiveness, preserving copyright and intellectual property, reducing exposure to lawsuits, and preventing malicious attacks. As such, almost all the current AI programs are a black box, and it is not entirely clear why they produce the results they do. Regulation seems to be the only way to force AI companies to increase the transparency and accountability of AI programs. Even though organizations such as the European Union are making some headway, globally, AI is likely to be a black box in the near future.

Ethical issues surrounding AI programs have been addressed by initiatives such as the Asilomar AI principles. In short, these principles embrace human values, privacy, liberty, and the common good. Asilomar principles are faced with considerable problems in practice because of the varied interpretation across different nations and political spectrums. In the current environment, commercial goals overtake any ethical concerns of AI programs. One would assume that AI programs used in such a vital arena such as health care are ethical in delivering their conclusions.

Racial bias in AI-enabled programs in health care has already been documented by Obermeyer et al. in a paper published in the journal Science. Health systems use commercial prediction algorithms to identify and help patients with complex health needs. The authors found that a widely used program was biased against Black patients, giving them a similar risk score as White patients even though, generally, they were sicker.

AI programs also face instability concerns, delivering unpredictable results. Large language models are prone to “hallucinations.” Using high-quality training data, setting clear boundaries for the use of AI models, and continuous refining and revision are suggested as possible solutions to decrease hallucinations. However, such rigorous measures are lacking in the current AI models used in health care.

A new bill in Congress, S. 4532 aiming to improve seniors’ timely access to care is seeking more transparency for prior authorizations under Medicare Advantage plans. As per the bill, the absolute numbers and percentages of denial of the prior authorization requests must be available to the public from health insurers and other agencies contracting health care services from federal or state governments. In the future, this type of transparent data dissemination might force health insurance companies to tweak their algorithm and, hopefully, reduce prior authorization denials.

Even though CMS has released guidelines for using AI in procedures such as prior authorizations and utilization management, as of now, the rule lacks rigor. While the federal and state authorities struggle to regulate and direct AI ventures, the FDA has recently stepped onto the arena. In May 2024, the FDA released a list of 882 approved AI or machine learning–enabled medical devices, which include predictive software programs. Because of the proliferation of such AI programs and their widespread use in U.S. health care, monitoring and ensuring quality and safety are going to be a challenge for any regulatory agency. But it needs to be done.

P. Dileep Kumar is a board-certified practicing hospitalist specializing in internal medicine. Dr. Kumar is actively engaged with professional associations such as the American College of Physicians, Michigan State Medical Society, and the American Medical Association. He has held a variety of leadership roles and has authored more than 100 publications in various medical journals and a book on rabies (Biography of Disease Series). Additionally, he has presented more than 50 papers at various national and international medical conferences. Several of his papers are widely cited in the literature and referenced in various textbooks.

Dr. Kumar has been involved in various hospital committees with advanced knowledge of Centers for Medicare & Medicaid Services (CMS) initiatives such as meaningful use, value-based purchasing, and Accountable Care Organizations.

Furthermore, Dr. Kumar has served as a scientific peer reviewer for various medical journals, including the British Medical Journal, Annals of Internal Medicine, American Journal of Cardiology, Physician Leadership Journal, and European Journal of Clinical Microbiology & Infectious Diseases.

Prev

From the other side of the table: a plea for empathy

April 14, 2025 Kevin 1
…
Next

How to build a culture where physicians feel valued [PODCAST]

April 14, 2025 Kevin 2
…

Tagged as: Health IT

Post navigation

< Previous Post
From the other side of the table: a plea for empathy
Next Post >
How to build a culture where physicians feel valued [PODCAST]

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by P. Dileep Kumar, MD, MBA

  • Sweden’s controversial COVID-19 strategy: lessons from higher mortality rates

    P. Dileep Kumar, MD, MBA
  • The collapse of rural health care: Why small-town hospitals are closing

    P. Dileep Kumar, MD, MBA
  • The Alaska Air Flight 1282 door blowout and patient safety

    P. Dileep Kumar, MD, MBA

Related Posts

  • Prior authorization reform for health care coverage takes center stage

    Afua Aning, MD
  • Black health care professionals are in mourning and deserve to be entirely heard

    Ellelan Degife
  • Programs that recruit and retain Black and Latinx students in health care fields are essential to address racial health disparities

    Alia Richardson
  • Why the health care industry must prioritize health equity

    George T. Mathew, MD, MBA
  • Difficulties navigating the health care system are causing many Black and brown kids to fall through the cracks

    Jeffrey Shahidullah, PhD
  • Proactive care is the linchpin for saving America’s health care system

    Ronald A. Paulus, MD, MBA

More in Tech

  • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

    Harvey Castro, MD, MBA
  • Why fearing AI is really about fearing ourselves

    Bhargav Raman, MD, MBA
  • Health care’s data problem: the real obstacle to AI success

    Jay Anders, MD
  • What ChatGPT’s tone reveals about our cultural values

    Jenny Shields, PhD
  • Bridging the digital divide: Addressing health inequities through home-based AI solutions

    Dr. Sreeram Mullankandy
  • Staying stone free with AI: How smart tech is revolutionizing kidney stone prevention

    Robert Chan, MD
  • Most Popular

  • Past Week

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • Make cognitive testing as routine as a blood pressure check

      Joshua Baker and James Jackson, PsyD | Conditions
    • The dreaded question: Do you have boys or girls?

      Pamela Adelstein, MD | Physician
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • What happened to real care in health care?

      Christopher H. Foster, PhD, MPA | Policy
    • Residency as rehearsal: the new pediatric hospitalist fellowship requirement scam

      Anonymous | Physician
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
  • Recent Posts

    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
    • The invisible weight carried by Black female physicians

      Trisza Leann Ray, DO | Physician
    • A female doctor’s day: exhaustion, sacrifice, and a single moment of joy

      Dr. Damane Zehra | Physician
    • Addressing America’s reliance on psychotropic medication [PODCAST]

      The Podcast by KevinMD | Podcast
    • The hidden cost of malpractice: Why doctors are losing control

      Howard Smith, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

Leave a Comment

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • The broken health care system doesn’t have to break you

      Jessie Mahoney, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician
    • Make cognitive testing as routine as a blood pressure check

      Joshua Baker and James Jackson, PsyD | Conditions
    • The dreaded question: Do you have boys or girls?

      Pamela Adelstein, MD | Physician
    • The hidden cost of delaying back surgery

      Gbolahan Okubadejo, MD | Conditions
  • Past 6 Months

    • What’s driving medical students away from primary care?

      ​​Vineeth Amba, MPH, Archita Goyal, and Wayne Altman, MD | Education
    • Internal Medicine 2025: inspiration at the annual meeting

      American College of Physicians | Physician
    • What happened to real care in health care?

      Christopher H. Foster, PhD, MPA | Policy
    • Residency as rehearsal: the new pediatric hospitalist fellowship requirement scam

      Anonymous | Physician
    • Are quotas a solution to physician shortages?

      Jacob Murphy | Education
    • A faster path to becoming a doctor is possible—here’s how

      Ankit Jain | Education
  • Recent Posts

    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
    • The invisible weight carried by Black female physicians

      Trisza Leann Ray, DO | Physician
    • A female doctor’s day: exhaustion, sacrifice, and a single moment of joy

      Dr. Damane Zehra | Physician
    • Addressing America’s reliance on psychotropic medication [PODCAST]

      The Podcast by KevinMD | Podcast
    • The hidden cost of malpractice: Why doctors are losing control

      Howard Smith, MD | Physician
    • How scales of justice saved a doctor-patient relationship

      Neil Baum, MD | Physician

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Leave a Comment

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...