Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

AI in health care: the black box of prior authorization

P. Dileep Kumar, MD, MBA
Tech
April 14, 2025
Share
Tweet
Share

Amidst the recent news about the murder of UnitedHealthcare CEO, the issue of prior authorization has come to the forefront again. Insurance companies use prior authorization as an easy tool to control the costs and incorporate it as a genuine step of the health care delivery process. The recent trend is to use AI (artificial intelligence) for prior authorizations. AI offers several advantages over human workers, further saving time and money.

Using AI for prior authorizations opens an entirely novel type of Pandora’s box in health care. A reduction of 50–75 percent of the manual task was achieved by using automated AI for prior authorization in 2022. Natural language processing (NLP) can be used to extract relevant information from the myriads of data submitted to the insurance companies and formulate a decision. AI programs can also be trained on previous customer data and previous decision patterns of the company. Algorithms can be tweaked to approve or deny treatment or medication requests from physicians. This could be a purely financial decision, nothing to do with health care delivery. AI can also ostensibly route queries to the appropriate reviewer faster. A federal class action lawsuit filed in 2023 against UnitedHealthcare Group alleges a 90 percent error rate in AI programs used for prior authorizations. The lawsuit also alleges that only 0.2 percent of the health policyholders are likely to appeal the denials because many consider it a hassle and a fruitless effort. The company apparently used a prediction model called “nH Predict” to determine coverage for Medicare Advantage patients in a post-acute care setting. According to the lawsuit, the model uses rigid and unrealistic predictions of recovery, often overruling the treating physicians’ determinations. Seemingly, “nH Predict” uses data from a few million patients compiled over years. The details of the machine learning algorithm used in the program are proprietary and hence not publicly known.

This issue brings the “black box” problem of AI programs. While transparency is important for trust building, AI companies encourage minimizing transparency because of various reasons such as maintaining competitiveness, preserving copyright and intellectual property, reducing exposure to lawsuits, and preventing malicious attacks. As such, almost all the current AI programs are a black box, and it is not entirely clear why they produce the results they do. Regulation seems to be the only way to force AI companies to increase the transparency and accountability of AI programs. Even though organizations such as the European Union are making some headway, globally, AI is likely to be a black box in the near future.

Ethical issues surrounding AI programs have been addressed by initiatives such as the Asilomar AI principles. In short, these principles embrace human values, privacy, liberty, and the common good. Asilomar principles are faced with considerable problems in practice because of the varied interpretation across different nations and political spectrums. In the current environment, commercial goals overtake any ethical concerns of AI programs. One would assume that AI programs used in such a vital arena such as health care are ethical in delivering their conclusions.

Racial bias in AI-enabled programs in health care has already been documented by Obermeyer et al. in a paper published in the journal Science. Health systems use commercial prediction algorithms to identify and help patients with complex health needs. The authors found that a widely used program was biased against Black patients, giving them a similar risk score as White patients even though, generally, they were sicker.

AI programs also face instability concerns, delivering unpredictable results. Large language models are prone to “hallucinations.” Using high-quality training data, setting clear boundaries for the use of AI models, and continuous refining and revision are suggested as possible solutions to decrease hallucinations. However, such rigorous measures are lacking in the current AI models used in health care.

A new bill in Congress, S. 4532 aiming to improve seniors’ timely access to care is seeking more transparency for prior authorizations under Medicare Advantage plans. As per the bill, the absolute numbers and percentages of denial of the prior authorization requests must be available to the public from health insurers and other agencies contracting health care services from federal or state governments. In the future, this type of transparent data dissemination might force health insurance companies to tweak their algorithm and, hopefully, reduce prior authorization denials.

Even though CMS has released guidelines for using AI in procedures such as prior authorizations and utilization management, as of now, the rule lacks rigor. While the federal and state authorities struggle to regulate and direct AI ventures, the FDA has recently stepped onto the arena. In May 2024, the FDA released a list of 882 approved AI or machine learning–enabled medical devices, which include predictive software programs. Because of the proliferation of such AI programs and their widespread use in U.S. health care, monitoring and ensuring quality and safety are going to be a challenge for any regulatory agency. But it needs to be done.

P. Dileep Kumar is a board-certified practicing hospitalist specializing in internal medicine. Dr. Kumar is actively engaged with professional associations such as the American College of Physicians, Michigan State Medical Society, and the American Medical Association. He has held a variety of leadership roles and has authored more than 100 publications in various medical journals and a book on rabies (Biography of Disease Series). Additionally, he has presented more than 50 papers at various national and international medical conferences. Several of his papers are widely cited in the literature and referenced in various textbooks.

Dr. Kumar has been involved in various hospital committees with advanced knowledge of Centers for Medicare & Medicaid Services (CMS) initiatives such as meaningful use, value-based purchasing, and Accountable Care Organizations.

Furthermore, Dr. Kumar has served as a scientific peer reviewer for various medical journals, including the British Medical Journal, Annals of Internal Medicine, American Journal of Cardiology, Physician Leadership Journal, and European Journal of Clinical Microbiology & Infectious Diseases.

Prev

From the other side of the table: a plea for empathy

April 14, 2025 Kevin 1
…
Next

How to build a culture where physicians feel valued [PODCAST]

April 14, 2025 Kevin 2
…

Tagged as: Health IT

Post navigation

< Previous Post
From the other side of the table: a plea for empathy
Next Post >
How to build a culture where physicians feel valued [PODCAST]

ADVERTISEMENT

More by P. Dileep Kumar, MD, MBA

  • Sweden’s controversial COVID-19 strategy: lessons from higher mortality rates

    P. Dileep Kumar, MD, MBA
  • The collapse of rural health care: Why small-town hospitals are closing

    P. Dileep Kumar, MD, MBA
  • The Alaska Air Flight 1282 door blowout and patient safety

    P. Dileep Kumar, MD, MBA

Related Posts

  • Prior authorization reform for health care coverage takes center stage

    Afua Aning, MD
  • Black health care professionals are in mourning and deserve to be entirely heard

    Ellelan Degife
  • Programs that recruit and retain Black and Latinx students in health care fields are essential to address racial health disparities

    Alia Richardson
  • Why the health care industry must prioritize health equity

    George T. Mathew, MD, MBA
  • Difficulties navigating the health care system are causing many Black and brown kids to fall through the cracks

    Jeffrey Shahidullah, PhD
  • Proactive care is the linchpin for saving America’s health care system

    Ronald A. Paulus, MD, MBA

More in Tech

  • Why trust and simplicity matter more than buzzwords in hospital AI

    Rafael Rolon Rivera, MD
  • ChatGPT in health care: risks, benefits, and safer options

    Erica Dorn, FNP
  • Why AI must support, not replace, human intuition in health care

    Rafael Rolon Rivera, MD
  • Why health care reform must start with ending monopolies

    Lee Ann McWhorter
  • AI can help heal the fragmented U.S. health care system

    Phillip Polakoff, MD and June Sargent
  • Why GenAI pilots fail in health care—and how to fix it

    Kedar Mate, MD
  • Most Popular

  • Past Week

    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • How federal actions threaten vaccine policy and trust

      American College of Physicians | Conditions
    • Are we repeating the statin playbook with lipoprotein(a)?

      Larry Kaskel, MD | Conditions
    • Why transgender health care needs urgent reform and inclusive practices

      Angela Rodriguez, MD | Conditions
    • mRNA post vaccination syndrome: Is it real?

      Harry Oken, MD | Conditions
  • Past 6 Months

    • COVID-19 was real: a doctor’s frontline account

      Randall S. Fong, MD | Conditions
    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • Confessions of a lipidologist in recovery: the infection we’ve ignored for 40 years

      Larry Kaskel, MD | Conditions
    • A physician employment agreement term that often tricks physicians

      Dennis Hursh, Esq | Finance
    • Why taxing remittances harms families and global health care

      Dalia Saha, MD | Finance
  • Recent Posts

    • The silent cost of choosing personalization over privacy in health care

      Dr. Giriraj Tosh Purohit | Uncategorized
    • How IMGs can find purpose in clinical research [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why the U.S. Preventive Services Task Force is essential to saving lives

      J. Leonard Lichtenfeld, MD | Policy
    • Medicaid lags behind on Alzheimer’s blood test coverage

      Amanda Matter | Conditions
    • The unspoken contract between doctors and patients explained

      Matthew G. Checketts, DO | Physician
    • AI isn’t hallucinating, it’s fabricating—and that’s a problem [PODCAST]

      The Podcast by KevinMD | Podcast

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

Leave a Comment

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • How federal actions threaten vaccine policy and trust

      American College of Physicians | Conditions
    • Are we repeating the statin playbook with lipoprotein(a)?

      Larry Kaskel, MD | Conditions
    • Why transgender health care needs urgent reform and inclusive practices

      Angela Rodriguez, MD | Conditions
    • mRNA post vaccination syndrome: Is it real?

      Harry Oken, MD | Conditions
  • Past 6 Months

    • COVID-19 was real: a doctor’s frontline account

      Randall S. Fong, MD | Conditions
    • Why primary care doctors are drowning in debt despite saving lives

      John Wei, MD | Physician
    • New student loan caps could shut low-income students out of medicine

      Tom Phan, MD | Physician
    • Confessions of a lipidologist in recovery: the infection we’ve ignored for 40 years

      Larry Kaskel, MD | Conditions
    • A physician employment agreement term that often tricks physicians

      Dennis Hursh, Esq | Finance
    • Why taxing remittances harms families and global health care

      Dalia Saha, MD | Finance
  • Recent Posts

    • The silent cost of choosing personalization over privacy in health care

      Dr. Giriraj Tosh Purohit | Uncategorized
    • How IMGs can find purpose in clinical research [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why the U.S. Preventive Services Task Force is essential to saving lives

      J. Leonard Lichtenfeld, MD | Policy
    • Medicaid lags behind on Alzheimer’s blood test coverage

      Amanda Matter | Conditions
    • The unspoken contract between doctors and patients explained

      Matthew G. Checketts, DO | Physician
    • AI isn’t hallucinating, it’s fabricating—and that’s a problem [PODCAST]

      The Podcast by KevinMD | Podcast

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Leave a Comment

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...