A patient sits across from her physician and asks that the AI-assisted reading of her scan not be the final word. She isn’t opposed to technology. She remembers reading about an AI system generating false clinical information at another hospital, and the story stayed with her. Her physician understands the hesitation. He has become more cautious himself after a documentation tool fabricated part of a medication history, an error he caught only because he knew enough to question what the system had produced.
Neither is a technophobe. Both are responding rationally to a genuine problem in health care AI. We have spent enormous energy developing increasingly capable AI systems. We have spent considerably less thinking about how clinicians and patients are supposed to develop justified trust in them.
The benefits and risks are both real
Any serious conversation about health care AI should begin by acknowledging what is working. AI-assisted imaging is demonstrating the potential to help clinicians identify abnormalities that might otherwise be missed. Predictive systems can identify patterns of clinical deterioration earlier than conventional processes in some settings. Ambient documentation tools are beginning to reduce some of the administrative burden that has consumed physician time and contributed to burnout. Used well, AI can augment clinical judgment, expand physician capacity, and potentially improve care.
Real physician voices, twice a week
Free, and one click to unsubscribe.
But acknowledging those benefits does not require minimizing the risks. AI systems can produce information that is plausible, confident, and wrong. Their reasoning can be difficult for physicians to interrogate, and their performance can vary across patient populations and clinical environments. Patients face another problem: Many may not know when AI has influenced their care, much less understand how much weight it carried in a diagnosis, risk assessment, or treatment recommendation.
Then there is accountability. When an AI-supported decision contributes to patient harm, who owns the outcome? The physician who relied on the system? The health system that purchased it? The vendor that developed it? The people who implemented it? If those questions remain unresolved until something goes wrong, trust is already on unstable ground.
Why the current response falls short
The instinctive response to an AI failure is usually local. A hospital conducts an internal review. A policy gets rewritten. A disclosure requirement is added. Another layer of oversight appears. These responses may be entirely appropriate, but they remain fragmented. An AI failure at one institution can damage public confidence far beyond that institution, while what the institution learns from that failure may never meaningfully inform another health system. Every hospital should not have to rediscover the same risks independently.
Health care needs shared approaches to AI incident reporting, meaningful transparency at the point of care, and clearer accountability among clinicians, health systems, and technology vendors. A physician should be able to understand when AI materially influenced a clinical recommendation. A patient should be able to know when it affected his or her care. Responsibility for failures should be established before they occur rather than negotiated afterward. None of this requires another technological breakthrough. It requires coordination across institutions that still too often approach AI governance as an internal organizational responsibility. That is the immediate task: coordinated governance.
But governance is only the beginning
Governance can reduce risk and establish accountability. It cannot, by itself, create the kind of trust that health care AI ultimately requires. Most AI is still being introduced as a tool layered onto clinical processes that were designed before the technology existed. The technology changes, while the underlying human system largely does not. Trust consequently becomes something we try to repair after a failure rather than something designed into the relationship between clinician, technology, and patient. Trust should be part of the architecture from the beginning.
This is where I think health care needs to move beyond the familiar idea of simply keeping a “human in the loop.” The Human-AI Cognitive Co-evolution, or HACC, framework I have been developing starts from a different premise. The objective should not be to create increasingly capable AI while asking clinicians to supervise it from the sidelines. It should be to create sustained, recursive interaction in which human and machine capabilities develop together.
That changes the physician’s role. The physician is not merely checking the machine’s work. The physician questions it, challenges it, corrects it, interprets it in the context of an actual patient, and learns where its capabilities and limitations lie. The AI system, in turn, should learn from those interactions. Over repeated cycles, the objective is not simply better AI output. It is better co-judgment.
That distinction matters because there are two very different paths health care can take. If physicians increasingly transfer cognitive work to AI, their own judgment may stagnate or weaken even while the technology becomes more capable. What looks like trust can gradually become dependence. But if clinicians remain cognitively engaged, AI can serve a very different function: exposing patterns, stimulating questions, challenging assumptions, and expanding what physicians themselves are capable of seeing and reasoning through.
The goal should therefore not be a future in which AI becomes steadily smarter while physicians become increasingly reliant on it. It should be one in which the interaction makes both sides of the relationship more capable.
Trust has to be engineered
Health care AI will not earn lasting trust through better public relations, reassuring slogans, or another disclosure form. Nor should trust depend on AI becoming perfect. Medicine has never demanded perfection from physicians, technologies, or institutions. It has demanded mechanisms for recognizing uncertainty, detecting error, learning from failure, and assigning responsibility. AI should be governed according to the same principle.
In the near term, that means coordinated governance: shared incident reporting, meaningful transparency, continuous evaluation, and clear accountability. But the longer-term challenge is larger. We have to design clinical environments in which physicians and AI systems can learn from each other without surrendering human judgment in the process. If we do that, trust becomes more than confidence in the accuracy of a particular algorithm. It becomes confidence in a system capable of questioning itself, correcting itself, and improving through continued human-AI interaction.
Governance can keep trust from collapsing. Human-AI co-evolution can make that trust resilient.
Health care will need both.
This essay is cited in the KevinMD record on artificial intelligence.
Matt Hasan is an economist, AI strategist, and founder of aiRESULTS. He advises health systems, payers, and life sciences organizations on the strategic implications of artificial intelligence, digital transformation, and emerging technologies. Over a career spanning more than four decades, he has held leadership and advisory roles with organizations including AT&T, IBM, Deloitte, Capgemini, and Citigroup, and previously served on the faculty of New York University’s Stern School of Business.
Dr. Hasan’s work focuses on the intersection of technology, institutions, and human decision making, with particular emphasis on how AI is reshaping medicine, governance, leadership, and professional practice. He is the founder of The AI Humanist Movement and an advocate for Human-AI Synergy, a framework that views AI not merely as a tool, but as a cognitive partner capable of extending human capabilities.
His writing includes “A Profession at the AI Frontier: Medicine Must Reinvent Itself or Cede Ground,” published in Health Affairs Forefront. He shares updates on LinkedIn and Medium.
