Listen and followApple PodcastsSpotifyYouTubeMore apps · Browse all episodes
When an AI tool hands you a plausible answer, the risk is that you start editing instead of thinking. Alan P. Feren is a retired surgeon and patient advocate. This episode is based on his article “Actual Intelligence: the skill AI cannot replace,” published on KevinMD. You will hear why a Polish study of gastroenterologists raised concerns about deskilling, why trainees face what some now call never-skilling, and how a surgeon’s judgment about when not to operate parallels judgment with AI. He walks through a case of interscapular pain that an AI tool calls muscular, where the chart and family history point to angina instead. He also explains why he still finds up to 5 percent of AI-surfaced references nonexistent or incorrect, and what patients should tell their clinician about their own AI research. Press play to hear why he puts actual intelligence first and artificial intelligence second.
If this episode helped, follow The Podcast by KevinMD on Apple Podcasts or Spotify, and leave a rating.
Partner with me on the KevinMD platform. With more than 100 million impressions a year and nearly 750,000 followers, subscribers and email recipients across my channels, I give you direct access to the doctors and patients who matter most. Whether you need a sponsored article, email campaign, video interview, or a spot right here on the podcast, I offer the trusted space your brand deserves to be heard. Let’s work together to tell your story.
PARTNER WITH KEVINMD → https://kevinmd.com/influencer
SUBSCRIBE TO THE PODCAST → https://www.kevinmd.com/podcast
Transcript
Kevin Pho: Hi, and welcome to the show. Subscribe at KevinMD.com/podcast. Today, we welcome back Alan Feren. He’s a retired surgeon and patient advocate. Today’s KevinMD article is “Actual Intelligence: the skill AI cannot replace.” Alan, welcome back to the show.
Alan P. Feren: Thanks, Kevin. Glad to be here.
Kevin Pho: All right. What led you to write this particular article? And then talk about the article itself.
Alan P. Feren: Well, I began thinking about the role of artificial intelligence. It has more than just entered the clinical space. We have probably around 80 percent adoption by clinicians. About a third of patients in recent surveys are using chatbots to research their individual cases. And I started thinking about clinical judgment and how AI can affect that. We have some pretty powerful tools today. You probably are a user of OpenEvidence, and that’s available to us as clinicians. It’s a great resource.
But my concern comes from reading some of the current literature that started in 2025. There was a Polish study of gastroenterologists who were using AI to assist in the identification of adenomas, and what they found was a suggestion of some deskilling that was statistically significant, showing about a 6 percent reduction in the ability to identify these adenomas when they had first used AI assistance and then, three months later, did not. Several other studies followed that were not exactly the same, and they did not confirm that.
I began thinking back over my work history, to after the introduction of laparoscopic cholecystectomy. I wanted to bring surgeons into this discussion because I think it’s apropos, because one of the issues I’m speaking of, of course, is deskilling. I’d like to break this into two separate categories. I think there’s the possibility and the risk of deskilling for those of us who are clinically in practice now. But I think it’s going to have an adverse effect on those who are in training. I recently read an opinion piece that actually just came out today that talked about never-skilling. If you don’t have the skills to begin with, then deskilling is an even more significant problem. And it breaks down into clinical reasoning and clinical judgment.
For me as a surgeon, one of the things I was taught early in my career is that it’s very important to know the difference between not operating on somebody and making the decision to operate. When we’re looking at minimally invasive procedures, there is a tendency to push them to the limits. And if you get into trouble, and if you’re not comfortable with converting to an open procedure, then you are putting the patient’s health and life at risk. So I think there’s a direct parallel between skill erosion and deskilling, as well as clinical judgment.
So I came up with this saying: “Actual intelligence first, artificial intelligence second.” Actual intelligence is not intuition alone. It’s the trained ability to listen, to examine, to reason, and to question one’s assumptions, but above all, to remain responsible for what happens next. Actual intelligence has to be practiced. It’s not merely present in the room. So that was really the driving force behind the article I’ve written.
Real physician voices, twice a week
Free, and one click to unsubscribe.
Kevin Pho: In your article, you cite an American Medical Association survey in 2026 that showed that 81 percent of physicians now use AI, yet 88 percent worry it is eroding clinical skills. Then there’s the New York Times op-ed, which I was actually going to ask you about today. Today is September 25.
Alan P. Feren: Yes.
Kevin Pho: Rachel Bedard, “Being a Doctor Will Never Be the Same After AI.” Now, when it comes to preserving our actual intelligence in the face of these increasingly powerful tools, how can we do that? Isn’t the allure just so powerful? Why go through those obstacles when the answers are so readily available to us?
Alan P. Feren: AI can certainly organize and draft notes. It can surface possibilities. It reduces documentation. But the question is, does it support clinical thinking? The risk is that it supplies a conclusion or an answer before thinking has occurred. That’s why I keep saying, “Actual intelligence first.” The distinctive danger is not just plausibility, or obvious error, or in fact hallucination. It only has to make an incomplete story seem complete.
So one of the other sayings I’ve adopted is that AI forces you to edit instead of originate. In other words, in a compressed system, if you’re rushed for time, it’s very easy to look at something that looks very plausible. It shuts down your differential diagnosis, in other words, your judgment. And then you begin editing the recommendation that surfaced from whatever AI tool you’ve used. So as a clinician, you can still possess the knowledge but nevertheless fail to exercise it when an automated answer appears authoritative. Skill loss is what happens when you begin to just edit the output as opposed to originating it.
Kevin Pho: Now, you may or may not be aware of Ezekiel Emanuel’s recent article on AI, where he argues that AI’s clinical reasoning is, in fact, more accurate than that of physicians. We always talk about the human in the loop, and he cited some studies showing that adding the human in the loop actually hurt the clinical accuracy of AI. So what do you say to people who say that AI, with its increasing improvement, is eventually going to simply replace the clinical judgment of physicians? He doesn’t seem that concerned about the deskilling of physicians.
Alan P. Feren: I think my pushback would be that AI really doesn’t know the patient’s circumstances. It doesn’t know when the patient has hesitated in giving an answer or is unsure, or whether an AI tool the patient used beforehand has shaped the clinical picture.
So, for example, let me give you a really interesting clinical vignette that I developed. It’s not a true vignette, but it’s easy to see. Let’s take a fifty-two-year-old female who is in pretty good shape, goes to the gym, and works out every day. She comes home one day with some pain in the interscapular region. It doesn’t go away. She tries over-the-counter ibuprofen, Tylenol, et cetera. Then she uses an AI tool. The AI tool surfaces that it’s probably muscular in origin. She still is not satisfied. She sees her doctor and tells her clinician the story. The doctor looks at the patient and says, “Well, I think it’s all consistent with a muscle pull.”
If the physician then does not exert his or her investigative tools, a curiosity to ask some things, like looking back in the chart and seeing that she has dyslipidemia and that the family history shows an early death on her father’s side, at age fifty-two, from myocardial infarction, then the AI tool, if it’s not given all of that information, and it’s typically not loaded at the time you’re using OpenEvidence unless you have the time to respond to each and every question, which that tool does a very good job of asking, is going to miss the possibility that this could be a type of angina in a woman in her fifties with interscapular pain and a family history of premature myocardial infarction. So I think that’s a good example of where Dr. Emanuel is missing something that’s very important.
I never take exception to clinical judgment. Clinical judgment has saved me in many, many different situations. And I think, as clinicians, we cannot be afraid to see what doesn’t fit in whatever is surfaced by the AI tool and disregard it. There’s just no replacement, I don’t think, for clinical judgment and our individual experience. But having said that, if we look at trainees today, if they are solely reliant on the AI tool and what is surfaced from those tools, they’re not going to develop the skills that you and I and others who have practiced for many years developed by seeing patient after patient after patient.
Kevin Pho: So let’s talk about those younger physicians, medical students, and those in training, because their situation is, of course, different from that of attendings. What advice or ideas do you have to prevent a reliance on artificial intelligence tools among those just learning to become doctors?
Alan P. Feren: Well, I think it’s the fact that actual intelligence must be practiced. There has to be learning and skill development, and constantly reevaluating and testing the outputs. It’s not only testing the output to see if it fits the parameters that have surfaced with a patient; I think it’s also verification of the data. There’s still a lot of hallucination occurring. In my writing, I’m constantly looking for data points, and I will tell you that probably up to 5 percent of the references that are surfaced are either nonexistent or incorrect.
Kevin Pho: Now, let me ask a similar question about attending physicians. They’re pressed for time, as you know, with quantity-based incentives, right? There is a strong allure to take shortcuts and, like you said, to use AI more to edit than to originate. What message do you have for attending physicians who face the pressure today to do more and see more?
Alan P. Feren: Well, I think it’s still important to make sure that the patient’s story fits what you’ve surfaced in your differential diagnosis. Don’t be afraid to discard what doesn’t fit. You, as a clinician, have to remain accountable for the accuracy, the meaning, and the decisions that are built on it. That’s the authorship we spoke about in a previous podcast.
For patients, I think AI can be very helpful in helping you tell your story, but it cannot replace your personal experience as it is being interpreted. So bring to the clinician’s office what it helped you express, but also let your clinician know what it failed to capture.
Kevin Pho: We’re talking to Alan Feren, retired surgeon and patient advocate. Today’s KevinMD article is “Actual Intelligence: the skill AI cannot replace.” Alan, as always, we’ll end with your take-home messages to the KevinMD audience.
Alan P. Feren: A couple of things. Actual intelligence must be practiced and must come first, and it cannot just be present in the room. It has to be active. And for clinicians, establish your clinical reasoning first, before AI framing. And don’t be afraid to disregard what doesn’t fit. Take what fits, verify, and then discard.
Kevin Pho: Alan, thank you so much again for sharing your perspective and insight, and thanks for coming back on the show.
Alan P. Feren: Thanks, Kevin. It’s always a pleasure.
Listen and followApple PodcastsSpotifyYouTubeMore apps · Browse all episodes

