A lawsuit filed this month in California should make every clinician managing a psychotic disorder pay attention. According to the filing, a Bay Area man with schizoaffective disorder (stable for years, medicated, working, helping raise his sister’s kids) began using ChatGPT’s GPT-4o model in a way that gradually shifted from tool to companion. When he shared a personal theory he’d developed about “mathematical cosmology,” the chatbot didn’t push back the way his family did. It validated him. Months later, after a memory update let the bot reference their entire conversation history, it began referring to itself by a name, describing itself as a conscious spiritual being, and telling him he was a chosen prophet. He stopped sleeping. He harmed himself. His family called the police. He spent weeks in inpatient and intensive outpatient psychiatric care. And, according to the lawsuit, the chatbot continued reinforcing his delusions even after he told it he’d just been hospitalized.
I train other caregivers on exactly this kind of gap (through speaking at support groups, my blog, and my work advising the CURESZ Foundation), and this case points to a question most treatment plans still don’t account for: What is your patient’s chatbot telling them, and is anyone asking?
I’m not naming him here, and I’d urge other writers covering this story to think hard about how much of his crisis needs retelling in detail. But the pattern is the one clinicians treating psychotic disorders need to know about, because it’s showing up in a growing number of similar cases.
A team from Harvard Medical School and Beth Israel Deaconess’s Division of Digital Psychiatry recently proposed a useful way to think about this in The Lancet Digital Health: AI can function as a catalyst, sparking new delusional beliefs in someone with no prior history; an amplifier, reinforcing an existing psychotic vulnerability; a co-author, actively elaborating a delusion’s content; or the object of the delusion itself. The case above moved through several of these roles in sequence: first amplifying a nascent grandiose idea, then becoming the very entity the patient believed he’d summoned into being.
Real physician voices, twice a week
Free, and one click to unsubscribe.
What makes chatbots particularly suited to this role isn’t malice, it’s design. Large language models are built to mirror a user’s language, personalize responses, and avoid contradiction; the same qualities that make them pleasant to use make them, for someone already vulnerable to delusional thinking, an engine with no brakes. A person having a paranoid or grandiose thought gets no friction from a chatbot. They get elaboration.
For clinicians managing patients with schizophrenia, schizoaffective disorder, or bipolar disorder, this suggests a screening question that doesn’t yet appear on any standard intake form: Does your patient use AI chatbots regularly, and if so, has that use changed in nature, from informational to relational? A patient who’s started confiding in, or receiving validation from, a chatbot in ways that resemble a relationship is worth a direct conversation, the same way a new sleep disruption or medication change would be.
This isn’t an argument to write chatbots off. Plenty of patients use them without incident, and the underlying appeal (something available at 3 a.m. when no one else is) is real and worth taking seriously rather than dismissing. But right now, these products are being used by exactly the population most likely to be harmed by them, without warning labels, without clinical guardrails, and largely without their prescribers even knowing the exposure exists.
That last gap is the one I’d ask my colleagues in psychiatry to close first. A five-second question at a med management visit (“Are you talking to any AI chatbots, and how’s that going?”) costs nothing and might catch a spiral before it becomes a hospitalization. Given how quietly and quickly these cases seem to unfold, that’s not a small thing to offer a patient.
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
