Every argument for AI mental health chatbots eventually arrives at the same line: It’s better than no care at all. Sometimes that’s true. But a quieter, more consequential question is starting to surface behind it, one that has less to do with what AI can do for patients, and more to do with what it does to the system patients depend on.
A recent Forbes Technology Council analysis raised exactly this concern: As insurers and employers invest heavily in AI-powered mental health platforms, they may be doing so at the direct expense of the human clinicians those platforms are supposedly meant to support. Reimbursement rates already fail to keep pace with rising costs and administrative demands. Therapists are limiting insurance participation, moving to private-pay models, or leaving certain markets altogether. If AI investment quietly shifts financial value away from the clinicians the mental health system depends on, rather than strengthening the economics of human-delivered care, “better than nothing” stops being a temporary bridge and starts becoming the permanent standard for anyone who can’t afford to opt out of it.
This isn’t hypothetical. As CalMatters recently reported, a union representing health care workers has filed a formal complaint with California state regulators against a large integrated health system over its alleged use of an automated algorithm in mental health triaging. The complaint surfaced alongside a new state bill, SB 903, that would bar companies from advertising chatbots as therapy. Whatever the outcome of either, the complaint itself is a signal: The concern that AI is being deployed as a gatekeeper, deciding who gets access to a human clinician and who gets routed to a chatbot, isn’t a theoretical worry anymore. It’s playing out inside one of the country’s largest health systems right now.
For patients managing schizophrenia, bipolar disorder, or other serious mental illness, this matters more than it might for the general population. These are conditions where an AI triage layer’s failure mode isn’t inconvenience. It’s a missed relapse signal, a chatbot’s reassuring tone standing in for a clinician’s trained judgment, or a sycophantic response reinforcing exactly the kind of thinking a human clinician would gently challenge. The population with the least room for error in this system is often the population most likely to be routed through its cheapest layer.
Real physician voices, twice a week
Free, and one click to unsubscribe.
None of this is an argument against AI in mental health care. Used well, it can extend real clinical capacity: administrative tools that give clinicians back time with patients, passive monitoring that flags a warning sign between appointments, triage support that helps a human make a faster, better-informed decision. The distinction that matters is whether a given AI tool is designed to extend human care or designed to substitute for it when human care becomes too expensive to sustain.
That distinction should show up in how health systems evaluate these tools, not just in how vendors market them. A useful question for any organization considering an AI mental health platform: Does this tool measurably increase the time and financial value available to your human clinicians, or does it quietly draw down the number of clinicians in-network over time? Those two outcomes look identical in a press release and completely different five years later.
Patients and families deserve the same clarity. If a chatbot is genuinely a bridge to human care (available at 2 a.m. when nothing else is, a starting point before a scheduled appointment), that’s a real, valuable role. If it’s becoming the default endpoint for anyone whose insurance network has quietly thinned out around them, that’s a different thing wearing the same interface, and it deserves to be named as such.
This complaint won’t be the last of its kind. As more insurers and employers lean on AI to manage mental health costs, the same question will keep resurfacing in different states, against different companies: Was this tool built to strengthen the system patients depend on, or to make it cheaper to need less of it?
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
