A colleague showed me a tool she was thinking about bringing into her practice. She pasted in a patient summary and got back a tidy set of next steps. Ten minutes later, to show me how it worked, she pasted the same summary in again. The answer came back different. The gap was small, but if the two of us had been covering for each other that week, we would have built two different plans for the same patient from the same data.
She closed the laptop. I did not blame her.
Most physicians have met that kind of AI by now. It writes well. It sounds certain. It cannot tell you where anything came from. Backing away from a tool you cannot verify is sound clinical judgment, the same instinct that makes you pull up the lab yourself before you act on what someone told you it said.
But backing away does not solve the problem that made these tools appealing in the first place. Patients now arrive carrying more data than a visit can hold: years of labs, a watch that has recorded their sleep and heart rate every night for two years, a genetic report from a company they used once, a supplement list typed into their phone in the parking lot. The volume climbs every year. The number of minutes in a clinic day does not. Somebody has to read all of it before I walk into the room, and it is not going to be me at eleven at night.
Real physician voices, twice a week
Free, and one click to unsubscribe.
So the work is figuring out which tools have earned a place that close to a patient. Four questions sort them fast.
Show me the work
Every output should link back to the study, guideline, or protocol it came from, and I should be able to open that link in one click. If a tool flags that a patient’s cholesterol pattern and family history may warrant earlier imaging, I want to see the threshold that triggered the flag and the source that set the threshold.
The reason is what happens after the flag appears. The patient will ask why. A colleague reviewing the chart will ask why. “The system suggested it” is not an answer I can give to either one. A tool that cannot show its work puts the physician in the position of defending reasoning they were never allowed to see.
Will it say the same thing tomorrow?
Same inputs, same output, every session. This sounds obvious until you watch a tool fail it, the way my colleague’s did over coffee.
Reproducibility is what makes coverage possible. I go to a conference, my partner opens the same patient file, and the picture on the screen is the one I saw: the same flagged risks, the same open questions. A tool whose output drifts between sessions cannot support continuity of care, no matter how good any single answer looks.
Does it know what it doesn’t know?
A patient uploads a panel from a clinic overseas that uses different reference ranges. A general-purpose model will quietly convert the numbers and hand back something clean. A tool built for clinical use flags the ambiguity and asks the physician to confirm the source before it goes any further.
This is the failure mode that worries me most, because it is invisible. Missing data, contradictory data, and data outside what the system can speak to should all arrive on my screen as gaps, marked as gaps. Fluent language poured into a hole is worse than no answer, since nothing about it looks wrong.
Does it interrupt me only when it matters?
We have already run this experiment with monitors and inbox alerts. A system that flags every out-of-range value teaches clinicians to stop reading it.
Escalation should be tied to a defined threshold: a pattern that holds across several markers, a real change in a trend, a meaningful shift in risk. And when something does reach me, it should come with the context that makes it interpretable, not a lone number and an exclamation point.
What the good version protects
These tools are usually sold on speed, which is the least interesting thing about them. What a well-built one gives back is the hour spent assembling a coherent picture from eight different sources. Nobody trained for a decade to reconcile a spreadsheet.
The decision stays with the physician. The tool compresses the review and surfaces what looks relevant, along with the evidence behind it. The physician reads it, argues with it, and decides what to act on, what to retest, and what to raise with the patient. If anything a system produces can reach a patient without a physician signing off on it, that is a design failure, and it should disqualify the tool.
None of this requires understanding how the model works. Ask the four questions. Ask for a click-through on one flagged output. Ask what the tool does when you feed it an incomplete chart. Companies that built for clinical practice will have those answers ready. The rest will change the subject.
Neil Panchal is a board-certified emergency physician and the cofounder and chief medical officer of Longevitix, where he builds the clinical and AI infrastructure behind precision longevity medicine.
He trained in emergency medicine at Mount Sinai in New York City and completed agentic AI training with the Harvard Data Science Institute and Stanford Medicine. His clinical affiliations include Yale New Haven Health, Hackensack Meridian Health, and Atlantic Health. He has spent his career where acute medicine meets prevention, treating the downstream failures of chronic disease that earlier risk stratification could have prevented.
At Longevitix, he directs the clinical evidence engine. He translates clinical research, biomarkers, wearables, and multimodal data into personalized protocols, and he sets the standards that govern how AI applies to physician workflows and patient care. His work spans healthspan optimization, early disease detection, and clinical decision support, pairing frontline medical judgment with health informatics.
He writes and speaks on evidence-based longevity, preventive medicine, and the responsible deployment of AI in clinical practice, with recent work including peer-reviewed research, commentary on physician oversight of agentic AI, and writing on KevinMD. He also publishes on the Longevitix blog and shares updates on LinkedIn, where Longevitix maintains a company page as well.
