Last month I wrote here about how AI is changing what patients expect their clinicians to know. Medicine does not have an information problem. It has an interpretation problem. The chart is full. What is missing is someone with the time and the tools to ask why.
In the weeks since, the conversation has moved, and it moved fast. Building clinical decision support has made me ask a related question. When should the software stop and hand the decision back to the clinician?
We are no longer only debating whether AI can suggest a diagnosis. Health systems are already using software that can take actions such as scheduling and routing, and the stakes rise when those actions affect clinical care. In a September survey commissioned by Imprivata, 28 percent of respondents said their organizations had agentic AI in production, and 72 percent reported that AI tools or agents were deployed without formal IT approval. In October, the American Medical Association (AMA) released a governance framework built around four questions. The last one keeps me up at night. Who is responsible for the risks and outcomes? And Sen. Bill Cassidy has circulated a discussion draft, the PATIENT AI Act, that would clarify when AI tools fall under Food and Drug Administration (FDA) or Office of the National Coordinator for Health Information Technology (ONC) oversight and give the FDA authority to review software updates on a rolling basis.
The test for all of it is what a clinician can see, question, and override in the middle of a shift. Think about the doctor reviewing labs at 9 a.m. on a Monday or the nurse practitioner charting at midnight. They will live inside whatever gets built.
Real physician voices, twice a week
Free, and one click to unsubscribe.
So let me offer a view from a different chair. My team and I have spent years building The Dr Z LabDX, software that helps clinicians interpret complex laboratory results. The most important design decision we ever made was what it would refuse to do.
It does not decide.
TDZ LabDX brings together the patient’s history, symptoms, and hundreds of lab markers, then pulls that information into a draft care plan. It’s the computational half of what I call the bionic practitioner. It can keep more details in view than any one person could reliably hold in their head. The practitioner brings the discernment to decide which findings matter, what to address first, and what the patient can realistically do.
That’s the point. Give the software the tedious work of pulling the case together so the practitioner has more time to do their job as lead clinical strategist. We designed it that way on purpose.
Here is why. Governance is usually discussed as a policy layer you add on top of a tool after it ships. Committees, audits, approval workflows. Those matter. But one of the most important governance decisions gets made much earlier, when builders decide what a clinician can review and what the software can do on its own.
A tool that shows the basis for a recommendation gives the clinician a chance to disagree. If it acts before the clinician can review it, that chance comes after the fact.
And putting a human in the loop only helps if that person can do a meaningful review. Studies of AI-generated emergency department summaries have found errors and omissions. The practical question is whether the clinician reviewing a polished draft can easily check it against the chart. If the loop is a rubber stamp, there is no loop. The design has to make it easy to see what the software used and to stop or change what it proposes.
I come at this personally. In my last piece, I wrote about spending years collecting diagnoses before an infection was found that helped explain what was happening to me. When I think about what generic AI would have done in that situation, I think about how easily it could have moved me to the next treatment on the same plan.
What finally helped was asking why. You cannot delegate that question to software that is not built to ask it.
I am not against AI. I run a technology company, and I believe the future of health care belongs to clinicians who use technology well. But “well” has to mean something. Before your organization turns on the next autonomous feature, ask four questions.
Can I see why it did what it did? Can I say no? Does my no actually change anything? And when it is wrong, whose name is on it? If a vendor can’t answer the first three, ask what you’re actually being asked to sign off on.
While regulators work out the rules, builders are already deciding what their software can do. Clinicians should demand a seat in that conversation, because we are the ones signing the chart.
A tool that cannot tell you why should not be allowed to act on its own.
Brandy Zachary, known as Dr. Z, is a chiropractic physician, an Institute for Functional Medicine Certified Practitioner, and the founder and CEO of TDZ Functional Medicine Academy (TDZ-FMA), LabDX, and DiagnoseThis. She built TDZ-FMA with no outside investment into a 2026 Inc. 5000 company with 2,095 percent three-year growth, and she is the inaugural Gold Stevie Award winner for Best Female Entrepreneur.
Zachary graduated cum laude from the Honors College at Loyola Marymount University with a degree in communications and earned her Doctor of Chiropractic degree in California in 2001, graduating magna cum laude as salutatorian. After reactivating her chiropractic license, she opened Body Love Cafe in Walnut Creek, California, in fall 2016. The award-winning holistic health clinic, with several practitioners, earned Diablo Magazine’s Best of the East Bay recognition across multiple years during her ownership.
She is the lead instructor for TDZ-FMA, which is jointly accredited through bodies including the Accreditation Council for Continuing Medical Education and the American Nurses Credentialing Center, and she developed the academy’s FMACP certification. She has taught for or been interviewed by the Institute for Functional Medicine, Rupa Health, Designs for Health, 3×4 Genetics, and SIBO SOS. She also knows the health care system as a patient, having traveled to Princeton, Stanford, Harvard, and the University of California, San Francisco, in pursuit of her own diagnosis.
Zachary is the author of eight books, six of them Amazon No. 1 bestsellers. They include How to Read a Client from Across the Room (McGraw-Hill, 2012), which won a Bronze Axiom Business Book Award and introduced the Character Code System; Middle-Aged Mama in a Muumuu, a marketing and business playbook for practitioners; and A Practitioner’s Guide to Mastering Functional Medicine Lab Values, a four-volume series used as required reading in other functional medicine training programs. Her writing has also appeared in Entrepreneur, and she hosts the Functional Medicine 2.0 Podcast. She shares updates from the academy on Instagram.
