Hospitals and health systems have spent the past several years building AI governance programs. They have formed committees, written policies, reviewed vendors, and debated where these tools belong in clinical care. That work matters.
But as a physician, I worry that too much of the conversation still stops short of the exam room. A health system can approve a tool, complete a compliance review, and satisfy a governance checklist while leaving clinicians with too little guidance on how AI should shape the decisions they make with patients.
AI is already appearing in documentation, decision support, patient education, and clinical search tools. Patients are using it too, often before they speak with a clinician. In many encounters, AI is now an unseen participant in how questions are framed, risks are understood, and choices are weighed. That makes point-of-care governance as important as enterprise governance.
Too often, AI governance is treated as a technology process. Clinicians need to know the source of the information they are using, what evidence supports it, and what assumptions may be embedded in the output. They also need training, and permission, to challenge it.
Real physician voices, twice a week
Free, and one click to unsubscribe.
I have used these tools in practice and don’t raise these concerns because I am skeptical of AI. I see real promise in tools that are grounded in trusted evidence and designed to support the way clinicians actually work. Used well, AI can help clinicians sort through complex information more quickly and help patients participate more actively in their care.
Patient engagement should always be welcomed. Many people now arrive with questions shaped by online searches or AI-generated summaries of symptoms, diagnoses, and treatment options.
Medicine depends on context: the patient in front of you, the quality of the evidence, the tradeoffs between options, and the uncertainty that often remains. Clinicians help patients make sense of all of that. If AI is part of the conversation, clinicians need to interpret it, not simply react to it.
There is another concern that deserves more attention: the risk of clinical deskilling. Physicians are trained to gather information, weigh competing explanations, recognize patterns, and question their own assumptions. If AI routinely generates differentials, summarizes evidence, and suggests next steps, some of that cognitive work may become less practiced over time.
That does not mean AI should be avoided. Every major technology changes clinical practice. Electronic health records changed documentation. Digital references changed how we retrieve information. AI is different because it can reach into the reasoning process itself.
The danger is not that AI will replace physicians. It is that physicians may gradually use less of the judgment that makes medicine more than pattern recognition. If clinicians become accustomed to accepting AI outputs without testing them, they may lose opportunities to build the diagnostic intuition that helps them recognize when an answer is incomplete, outdated, or simply wrong.
Health systems should treat AI literacy as part of clinical competency. Clinicians do not need to become data scientists, but they should understand how AI tools are built, where limitations can arise, how bias can enter a system, and how to validate an output against trusted evidence and a patient’s circumstances.
That training should be practical, repeated, and tied to real clinical scenarios. Clinicians should be expected to ask: What is this based on? What might be missing? Does it fit this patient? What would make me question it?
Organizations also need to be selective about the tools they bring into care. AI that surfaces clinical information should be grounded in expert-reviewed content, transparent about its evidentiary basis, and supported by governance that keeps pace as evidence changes.
The future of clinical AI will not be judged by how many tools a health system deploys. It will be judged by whether those tools help clinicians think more clearly, communicate more effectively, and make better decisions with patients.
Physicians will not compete with AI in that future. They will need to become better evaluators of evidence, better interpreters of information, and better guides for patients trying to make sense of an increasingly crowded information landscape.
AI can support that work, but it will not do it on its own. Better engagement, better access to evidence, and better decisions will depend on whether organizations build the habits and expectations that keep clinicians in the driver’s seat.
That is why AI governance cannot remain an IT function, a compliance exercise, or a boardroom discussion. It has to become part of clinical practice.
The organizations that get this right will not simply approve AI for use. They will teach clinicians how to question it, explain it, and use it with judgment at the point of care, where the stakes are highest.
This essay is cited in the KevinMD record on artificial intelligence.
Amanda Heidemann is the senior clinical content consultant for clinical effectiveness at Wolters Kluwer Health. She is board-certified in family medicine and clinical informatics, bringing her extensive experience to Wolters Kluwer and the UpToDate® team to work with health administrators and care organizations seeking clinical transformation and optimization of technology solutions. She is passionate about using technology to enhance the patient-physician relationship and to deliver high-quality health care. More about her professional background can be found on her LinkedIn profile.
