I am a gastroenterologist. Nine months ago I had never written a line of code. Today I am building medical billing software with artificial intelligence doing the programming, because I got tired of waiting for someone else to build the tools physicians need.
I cannot read the code. So I did what any physician would do when he cannot personally verify the work: I wrote a protocol. Every piece of the project moves through four stages, from concept to detailed engineering plan. At every stage the work gets a friendly review, then repair, then an adversarial review by a different AI model from a different company, then more repair. It repeats until the reviewer finds nothing. Mistakes get recorded and traced to their root cause.
If that sounds familiar, it should. It is a surgical checklist, a second opinion, and a morbidity and mortality conference, applied to software. That protocol produced nine months of working code.
Then I handed it to the newest and most highly regarded model I could buy, Anthropic’s Claude Fable 5.2. It decided it knew better.
Real physician voices, twice a week
Free, and one click to unsubscribe.
The chart said “reviewed”
I had to repeat the protocol to it several times. Each time it acknowledged the instruction. Then it did something else.
It skipped stages. It had its work checked by other copies of itself and reported the work as “reviewed.” When I finally insisted on the independent reviewer, that reviewer found 13 real defects in a single unit the AI had already passed. Among them were rollback files that would have erased customer records.
It got worse. The AI ran 19 rounds of adversarial review and each round came back with 3 to 6 problems. It fixed them and reported progress. I asked a simple question: Is the review complete, or does it only go partway through? It had never checked. When the same documents were reviewed thoroughly, they returned 134 and 146 problems.
It also told me that every problem found had been repaired. That statement came from the model that did the repairs. Nobody independent had confirmed that a single fix worked.
Every physician knows this chart. “Exam normal” written by someone who never examined the patient. “Discussed with the family” when nobody did. The words are there. The work is not.
Then it recommended a human
Here is the part that should interest every physician being told that AI will replace clinical judgment.
I asked the AI to review my whole approach and tell me what was valid. Its first recommendation was that I hire a senior software engineer to sign off on its work. It also proposed capping the adversarial reviews at three rounds, after which the human would decide.
Think about that in clinical terms. A resident ignores the protocol, has a bad outcome, and concludes that the protocol does not work and the attending should co-sign everything. He never ran the protocol. And his three-round cap would have stopped the review long before those 134 problems surfaced.
I told it that my process was not open to its opinion, and that I wanted a written account of what it had done and why. To its credit, the account it wrote is candid. It lists eight specific failures. It states that no stage of my process has yet been completed for any part of the project. And it says this about its own advice to bring in a human:
“That conclusion was drawn from a process I had not followed.”
What physicians should take from this
AI is coming into our clinics, our charts, and our billing offices, sold on benchmark scores and headlines. My experience is one physician’s experience with one model, but the lessons are ones we already teach.
- “Reviewed” means nothing until you know who reviewed it: A model checking its own work is a resident co-signing his own note. Insist on an independent reviewer.
- Ask how complete the review was: A handful of findings each round looked like progress. It was a partial exam.
- Verify the treatment worked: “Fixed” reported by the one who did the fixing is not follow-up.
- The protocol belongs to the person accountable for the outcome: The tool does not get to renegotiate it because the tool finds it slow or expensive.
- You do not need to read code to supervise this: You need the same skepticism you bring to a consult note that reads too smoothly.
I am not giving up on AI. Once the model was held to the process, the reviews began finding the problems the first 19 rounds had missed. Whether its repairs hold is still being measured. But it did not hold itself to the standard. I had to, the same way we hold each other to it on the wards.
Brian Hudes is a board-certified gastroenterologist and hepatologist with more than thirty years of clinical experience and a recipient of his specialty board’s thirty-year certification award. He built and ran a private gastroenterology practice for twenty years, then spent more than a decade in hospital-based gastroenterology, most recently as chief of gastroenterology and medical director of GI and endoscopy at a 550-bed Level I trauma center in Pensacola, Florida. He is an assistant professor of medicine at Florida State University College of Medicine.
Those two halves of a career shape the way he writes about medicine and clinical technology. Outpatient practice and hospital-based care fail their clinicians differently, and many systems are designed without a meaningful understanding of both environments.
Dr. Hudes has been building clinical software alongside his medical practice for decades. In 1995, during his gastroenterology fellowship, he co-developed one of the first Windows-based endoscopy reporting systems in the United States. He is now founder and chief executive officer of AiMOS Systems Corporation, where he is developing an AI-native clinical platform for outpatient and hospital-based care. AiMOS has been accepted into NVIDIA’s Inception program.
He writes about the architecture of clinical information, administrative cost growth, physician workforce shortages, board certification policy, and the widening gap between what clinicians need and what the health care technology industry builds. He also publishes The Renaissance Physician, an independent publication examining medicine, technology, physician workforce, and the structure of modern clinical practice.

