A recent KevinMD piece by an emergency physician laid out what AI scribes get wrong and how to catch it before signing. I read it as a mother of someone living with serious mental illness, and one error type stopped me cold: The scribe can take a diagnosis the doctor was only thinking out loud about and record it as fact.
In the emergency department, that might mean a ruled-out pulmonary embolism showing up as a diagnosis. In psychiatry, it means something else. A clinician says, “We need to rule out schizophrenia,” and the chart says schizophrenia. That word follows a young person to every future clinician, every ER visit, every insurance form, every disability or custody question. Psychiatric labels carry a weight that most diagnoses don’t, and they are much harder to remove than to add.
The numbers are worth knowing. A study published October 8 in BMJ Innovations tested a commercial ambient scribe on more than 5,700 real consultations. With its safety layers turned off, about 3.3 percent of note content had no support in the conversation, and about 2.9 percent was content that could change a clinical decision. With safety checks on, those figures fell to 1.6 percent and 1.2 percent. The authors work for the company that makes the scribe, so independent numbers may look different.
One to three percent sounds small. But a psychiatric note is not a list of vitals. It is built from the conversation itself, and psychiatric conversations are full of exactly the material a scribe can misread.
Real physician voices, twice a week
Free, and one click to unsubscribe.
Clinicians ask hard questions out loud: about voices, about suicide, about safety. A question is not an answer, and a scribe that blurs the two can put a risk in the chart that was never there. Clinicians reason through a differential in front of the patient: Could this be bipolar disorder, a substance-induced episode, a medical cause? Each possibility is a chance for a guess to become a diagnosis. And families are often in the room, describing frightening episodes from their own point of view. A scribe that doesn’t track who said what can record a parent’s worry as the patient’s own report.
This matters more now because patients and families can read the note. Under the 21st Century Cures Act, most patients can see their clinical notes online, and many people with serious mental illness share portal access with a parent or caregiver. I have read notes about my own child that didn’t match what was said in the room. Before AI, that was rare. With an AI scribe in every appointment, it doesn’t have to be.
The emergency physician’s checklist is a good start: Read the high-risk sections first, delete anything you didn’t say, and watch for a differential turning into a diagnosis. Psychiatry needs a version of its own.
- Check the psychiatric high-risk sections first: Diagnoses, risk assessments, safety statements, and anything about substance use deserve a line-by-line read before signing.
- Track who said it: Collateral from a family member should be labeled as collateral, not merged into the patient’s history.
- Ask before recording: Patients and families should know an AI is listening and be able to say no, especially in a first visit or a crisis.
- Give patients and families a fast way to flag errors: A correction request should not take weeks, and a family member who spots an error should know whom to tell.
- Measure psychiatry separately: Health systems that track scribe errors after go-live should report psychiatric notes on their own, because the cost of an error there is different.
AI scribes can give psychiatrists back time with their patients, and I want that for every clinician who has ever typed through a session. But in psychiatry, the note isn’t just a record of the visit. For many patients, it becomes part of who the system believes they are. That deserves a second look before anyone signs.
Nicole Drapeau Gillen is a mother, advocate, and author who translates the fast-moving landscape of technology in serious mental illness (SMI) care into guidance families and clinicians can use. Thrust into caregiving for a loved one with SMI, with no direction on how to help, she turned that experience into a mission, writing two books and building an ongoing effort to bring families and clinicians into the conversation.
Her first book, Schizophrenia and Related Disorders: A Handbook for Caregivers, is a reference for every stage of caregiving, endorsed by Dr. E. Fuller Torrey as a must-read for SMI caregivers. Her second, Connected Care: A Practical Guide to Technology for Serious Mental Illness, maps apps, artificial intelligence tools, telepsychiatry, and brain-based treatments for a field moving faster than anyone can track. Dr. Akira Sawa, director of the Johns Hopkins Schizophrenia Center, has said the book “directly addresses” significant gaps.
