Picture a busy ED at 1 a.m. A CT pulmonary angiogram comes back from the scanner. The worklist shows no AI flag for pulmonary embolism. The radiologist has 30 studies in the queue. The patient’s heart rate is still 112.
What does the missing flag mean? A new study says: less than most of us assume.
On Oct. 21, researchers at Emory published a look at two FDA-cleared PE detection algorithms in npj Digital Medicine. The tools ran across 17 facilities for two years. On more than 30,000 CT angiograms, the PE model caught 86.8 percent of emboli. Its FDA clearance had reported 94.86 percent.
The gap got wider where it hurts. Saddle emboli, the ones you can see from across the room, were caught 99.3 percent of the time. Subsegmental clots dropped to 72.9 percent. On routine chest and abdominal CTs, the incidental PE tool caught 73.5 percent, against a cleared figure of 89.7 percent. In a group of patients who later had dedicated PE imaging, it missed 19 of 61 clots on the first scan.
Real physician voices, twice a week
Free, and one click to unsubscribe.
Specificity was excellent. When the tool flags something, it is usually right. Silence is the problem.
I worked ER shifts for about twenty years. I know how a worklist changes behavior. A red flag moves a study to the top. No flag lets it sit. Nobody decides to trust the silence. It just happens, one busy night at a time.
So here is what I’d tell an ED group or a radiology chair this week.
Treat the flag as a triage tool, not a read. The authors say the same thing: It’s an adjunct, not a replacement. Write that into the department’s policy in one sentence, so a new hire hears it on day one.
Ask for your own numbers. Clearance data comes from a test set. Your scanners, your protocols, and your patients are different. Ask the vendor or your informatics team for sensitivity at your site, split by clot location. If nobody can produce it, that’s your answer.
Watch the version. The Emory team could not confirm the deployed model matched the one the FDA cleared. If your hospital can’t tell which version is running, it can’t tell when performance changes.
Keep the clinical picture in charge. Tachycardia, hypoxia, a swollen leg, a Wells score that worried you before the scan. None of that goes away because a box stayed unchecked.
None of this is an argument against imaging AI. These tools speed up the reads that matter, and a 99 percent specificity flag is a gift on a crowded night. It’s an argument for honest labels. A tool that misses about one in four small clots should be described that way to the people relying on it.
The FDA’s digital health chief said last week that draft guidance on generative AI devices is coming. PE detection isn’t generative AI, but the lesson carries over. Clearance is the start of the evidence, not the end. Real-world performance has to be measured after go-live, by the hospitals using the tool.
Until that’s routine, the safest rule is the old one. Read the patient, then the scan, then the flag.
Harvey Castro, known as DR GPT™, is an emergency physician, author, keynote speaker, and physician futurist. He is advisor to the CEO of Phantom Space. He writes and speaks about artificial intelligence in health care, digital health, space medicine, and the future of human-centered care. Explore his speaking topics and availability.
He is the author of Bing Copilot and Other LLM: Revolutionizing Healthcare With AI, Solving Infamous Cases with Artificial Intelligence, The AI-Driven Entrepreneur: Unlocking Entrepreneurial Success with Artificial Intelligence Strategies and Insights, ChatGPT and Healthcare: The Key To The New Future of Medicine, ChatGPT and Healthcare: Unlocking The Potential Of Patient Empowerment, Revolutionize Your Health and Fitness with ChatGPT’s Modern Weight Loss Hacks, Success Reinvention, and Apple Vision Healthcare Pioneers: A Community for Professionals & Patients.
He shares updates on LinkedIn, YouTube, Facebook, Instagram, and X. #DRGPT
