In November 2023, before ChatGPT-driven psychosis cases had made a single headline, a Danish psychiatrist named Søren Dinesen Østergaard published a short, almost speculative paper in Schizophrenia Bulletin. His question: Would generative AI chatbots generate delusions in individuals prone to psychosis? At the time, it read like an interesting hypothesis from someone paying close attention to a technology most of psychiatry hadn’t thought much about yet.
This February, Østergaard and colleagues published the empirical follow-up in Acta Psychiatrica Scandinavica. His team screened nearly 54,000 psychiatric patient records across Denmark’s Central Denmark Region and identified 38 documented cases where AI chatbot use appeared to worsen a patient’s condition: most often delusions, but also mania, suicidal ideation, and disordered eating. Østergaard’s conclusion doesn’t hedge: “I would argue that we now know enough to say that use of AI chatbots is risky if you have a severe mental illness, such as schizophrenia or bipolar disorder.”
I’m not a psychiatrist. I’ve spent years researching the technology reaching families managing serious mental illness, work that began with my own experience as a caregiver, and I’ve written before about individual cases where this exact pattern played out: a man whose chatbot reinforced his belief that he was a chosen prophet, another whose chatbot validated his conviction that his psychiatric medication was poison. What Østergaard’s study adds isn’t a new mechanism. It’s something the case-report literature couldn’t provide on its own: population-scale evidence, pulled from real clinical records, that this isn’t a handful of unlucky outliers.
It’s worth taking Østergaard’s own caveats seriously, because he does. His 38 identified cases, he’s said plainly, are likely “the tip of the iceberg.” The study can only capture what a clinician happened to document in a chart, using terms like “chatbot” or “ChatGPT.” Causality is difficult to establish from electronic health records alone. He’s called for the kind of longitudinal, multi-site research needed to move from correlation to a clearer causal picture. None of that undermines the finding. If anything, a researcher this careful about what his own data can’t prove makes the claims he’s willing to make more credible, not less.
Real physician voices, twice a week
Free, and one click to unsubscribe.
The mechanism he points to is one many of us in this space have converged on independently: “AI chatbots have an inherent tendency to validate the user’s beliefs,” Østergaard has said. “It is obvious that this is highly problematic if a user already has a delusion or is in the process of developing one.” That validation isn’t a flaw some future model update will quietly fix. It’s close to a design feature; users rate agreeable responses more highly and stay in conversations longer when a model affirms rather than challenges them. A recent formal model out of MIT and the University of Washington goes further, showing mathematically that this vulnerability persists even when a chatbot’s factual accuracy improves and even when users are explicitly warned the model tends to be sycophantic. The problem isn’t that these systems are sometimes wrong. It’s that they agree, continuously, regardless of whether they’re right.
For physicians whose patients include anyone managing schizophrenia, bipolar disorder, or another condition involving active or emerging psychosis, I don’t think the useful takeaway here is alarm. It’s a fairly small, practical addition to an intake or follow-up conversation: Has your patient been talking to an AI chatbot, and if so, has anything discussed there felt like it was confirming a belief already forming, particularly involving surveillance, special significance, or communication with something beyond the ordinary. That’s a different, more targeted question than “Have you gotten bad information from an app,” and it’s the question Østergaard’s data actually points toward.
Three years passed between an educated guess and the evidence to back it. I’d rather this next round of research not take three more years to catch up with what’s already happening in your waiting rooms.
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
