It’s 3 a.m. There’s a septic patient in room 6 and someone intoxicated yelling in the hallway. A 78-year-old on eleven medications is going home, and her daughter needs discharge instructions she can actually follow. The hospital’s approved AI tool sits behind two logins and only runs on a desktop. The chatbot on your phone opens with one tap.
Most clinicians I know have felt that pull. Some have given in.
Late last month, WYPR in Baltimore reported that an employee at Johns Hopkins Bayview Medical Center had uploaded a spreadsheet with information on nearly 3,200 patients to a public AI tool the hospital had not approved. The file held treatments, medications, provider information, account numbers, and medical record numbers. Hopkins says it had the worker delete the thread, asked the AI company to delete the data, and is reinforcing training. Patients got letters and a year of credit monitoring.
I don’t know why that employee did it, and I won’t guess. Hopkins has said what it knows. What I do know, after about twenty years of ER shifts, is why clinicians reach for these tools. Malice almost never has anything to do with it. Friction does.
Real physician voices, twice a week
Free, and one click to unsubscribe.
The pattern shows up again and again when I talk with hospital teams. The sanctioned tool was bought for one narrow job, usually notes. It’s slow to load. It can’t summarize a 400-page outside record or turn discharge instructions into plain Spanish at a sixth-grade reading level. A free chatbot does both in seconds. So the work drifts to wherever the speed is.
A policy memo won’t change that, and neither will a scary module people click through once a year. When the fast tool is forbidden and the safe tool is slow, tired people pick fast. Calling that a discipline problem lets the people who chose the tools off the hook.
Here is what I think would help, starting from the floor.
Ask staff what they’re pasting, and ask anonymously. You’ll learn more about your real AI needs from that survey than from any vendor demo. If half your nurses use a chatbot to translate discharge papers, that’s your biggest use case and your biggest risk, in the same sentence.
Give people a safe version of the fast thing. A secure, logged chatbot inside your network that opens on any workstation in one click will beat a ban. Compare its price with the cost of one breach notification.
Make “Is this tool approved?” answerable in ten seconds. One intranet page, every approved tool, one line on what each is for. If a resident has to email IT and wait two days, they won’t ask.
Teach one rule. No names, no record numbers, no dates of birth in any tool the hospital hasn’t approved. People can hold one rule in their heads at 3 a.m. They can’t hold a 30-page policy.
Treat reports as near misses. If the first person who admits pasting a chart into a chatbot gets fired, nobody else will admit it, and you’ll be flying blind. Aviation learned this decades ago. Medicine already knows how to run blame-free safety reporting. We just haven’t pointed it at AI.
Patients have a stake in this, too. That 78-year-old doesn’t care which software wrote her instructions. She cares that they’re right, and that her medication list isn’t sitting on a server she never agreed to. Both are on us.
I’m optimistic about AI in emergency medicine. I wrote a book about it. But in a hospital, the tool that gets used is the one that’s easiest at 3 a.m., and right now, in a lot of places, that isn’t the safe one.
Harvey Castro, known as DR GPT™, is an emergency physician, author, keynote speaker, and physician futurist. He is advisor to the CEO of Phantom Space. He writes and speaks about artificial intelligence in health care, digital health, space medicine, and the future of human-centered care. Explore his speaking topics and availability.
He is the author of Bing Copilot and Other LLM: Revolutionizing Healthcare With AI, Solving Infamous Cases with Artificial Intelligence, The AI-Driven Entrepreneur: Unlocking Entrepreneurial Success with Artificial Intelligence Strategies and Insights, ChatGPT and Healthcare: The Key To The New Future of Medicine, ChatGPT and Healthcare: Unlocking The Potential Of Patient Empowerment, Revolutionize Your Health and Fitness with ChatGPT’s Modern Weight Loss Hacks, Success Reinvention, and Apple Vision Healthcare Pioneers: A Community for Professionals & Patients.
He shares updates on LinkedIn, YouTube, Facebook, Instagram, and X. #DRGPT
