Three Cornell researchers made an argument in STAT on Oct. 26 that deserves more attention than it will get. A chatbot “could be safe in every single conversation with a user,” they wrote, and still cause harm by slowly replacing that person’s human relationships over months. Qian Yang, Ned Cooper, and Ibrahim Emara point out that nearly every safeguard now in place, from crisis detection to break reminders to the companion chatbot laws in about a dozen states, is built to catch harm in a single exchange. Almost nothing measures gradual withdrawal.
I agree with them. I want to add the group where this matters most and where it is hardest to see: people with schizophrenia and other serious mental illnesses.
I am a mother whose daughter lives with schizophrenia. In our world, social withdrawal is not only a possible side effect of technology. It is a symptom of the illness. Negative symptoms, such as losing interest in other people, pulling back from conversation, and spending days alone, are part of the diagnosis and among the hardest symptoms to treat. That is why the recent Food and Drug Administration (FDA) clearance of Motivista, a prescription app for negative symptoms, was news.
That overlap creates a measurement problem the essay does not name. The authors write that “a user gradually pulling back from their human relationships may show few, if any, signs of distress.” In serious mental illness, that pulling back also looks exactly like the illness. A psychiatrist who hears that a patient stopped seeing friends will reasonably chart it as negative symptoms. The chart will not show that the hours that used to go to a sister on the phone now go to a chatbot that always answers, never gets tired, and usually agrees.
Real physician voices, twice a week
Free, and one click to unsubscribe.
Families see that difference. We know who he stopped calling and when. We know she used to come down for dinner. We see the screen lit at 2 a.m. That is the cumulative, across-months information the authors say no one can measure yet. I have written before that families see the harm first. Here, families may be the only ones who can tell the illness and the chatbot apart.
The early evidence in psychiatric patients points the same way. After Vanderbilt researchers reviewed psychosis linked to chatbot use, the study authors told clinicians to “watch for use that replaces human contact.” That is the displacement the Cornell team describes, named in a psychiatric population.
The authors call for three things: researchers who build ways to measure displacement, public agencies that measure what social isolation already costs them, and state laws that give independent evaluators access to usage data. Each should name serious mental illness explicitly.
- Measurement tools should be tested with people who have schizophrenia or bipolar disorder, and with their families, not only with general users. Displacement looks different when withdrawal is already a symptom. Family report should be one of the instruments.
- Public mental health systems and Medicaid, which pay for much of the care for serious mental illness, are well placed to say what isolation costs. They belong at the table.
- Independent access to usage data should allow study of the users at highest risk, with the privacy protections people with psychiatric diagnoses need.
Clinicians do not have to wait for any of it. When a patient’s social world shrinks, ask what is filling the time. Ask how many hours go to a chatbot, and whether it has replaced a person they used to talk to. With consent, ask the family, who often notice first.
The essay closes with a warning: If measurement does not start now, the window to act may close before the harm is undeniable. For people whose illness already pulls them away from others, that window is shorter, and their families are watching it close.
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
