Subscribe to The Podcast by KevinMD. Watch on YouTube. Catch up on old episodes!

We are joined by Fady Chamoun, an internal medicine physician, to explore the transformative potential of artificial intelligence (AI) in health care. We delve into the art of prompt engineering and its six essential elements: clear task definition, detailed context, exemplar guidance, persona understanding, structured format, and appropriate tone. Fady shares his insights on how mastering these elements can enhance patient care, streamline medical education, and ultimately improve patient outcomes.

Fady Chamoun is an internal medicine physician.

He discusses the KevinMD article, “Mastering prompt engineering to elevate AI in clinical practice.”

Our presenting sponsor is Nuance, a Microsoft company.

Together, Microsoft and Nuance are leveraging their rich digital technology and advanced AI capabilities to tackle some of health care’s biggest challenges. AI-driven technology promises to revolutionize patient and provider experiences with clinical documentation that writes itself.

The Nuance Dragon Ambient eXperience, or DAX for short, is a voice-enabled solution that automatically captures patient encounters securely and accurately at the point of care. DAX Copilot combines proven conversational and ambient AI with the most advanced generative AI in a mobile application that integrates directly with your existing workflows.

Physicians who use DAX have reported a 50 percent decrease in documentation time and a 70 percent reduction in feelings of burnout, and 85 percent of patients say their physician is more personable and conversational.

Discover AI-powered clinical documentation that writes itself. Visit https://nuance.com/daxinaction to see a 12-minute DAX Copilot demo.

VISIT SPONSOR → https://nuance.com/daxinaction

SUBSCRIBE TO THE PODCAST → https://kevinmd.com/podcast

RECOMMENDED BY KEVINMD → https://kevinmd.com/recommended

GET CME FOR THIS EPISODE → https://kevinmd.com/cme

I’m partnering with Learner+ to offer clinicians access to an AI-powered reflective portfolio that rewards CME/CE credits from meaningful reflections. Find out more: https://kevinmd.com/learnerplus

Transcript

Kevin Pho: Hi, and welcome to the show. Subscribe at KevinMD.com/podcast, and get CME for this episode by clicking on the CME link in the show notes. Today we welcome Fady Chamoun. He’s an internal medicine physician and hospitalist. Today’s KevinMD article is “Mastering prompt engineering to elevate AI in clinical practice.” Fady, welcome to the show.

Fady Chamoun: Good morning, thank you, thank you for having me.

Kevin Pho: So let’s start by briefly sharing your story and journey.

Fady Chamoun: Yeah, I’m originally a graduate from the American University in Beirut. Came to Minnesota for an externship as a fourth year medical student at Mayo Clinic, and I just loved the experience. Then I really liked the health care in Minnesota, so I moved here for residency, did my residency back in late 90s, and been here since. Joined the VA after that, and then I’ve been with M Health Fairview, where I still work now, since 2005. I started as an emergency room, hospital medicine doc with the VA, and then been a hospitalist since 2005.

Back in 2007, basically what we all started to see, you know, more denials at that time related to observation, inpatient, et cetera. I was trying to come up with some solution for that frustration. I was, like anyone else at that time, kind of frustrated with people that are from a different state, they really don’t know the practice here. A lot of times surgical subspecialties trying to tell the hospitalist if an AFib should be inpatient or not.

So I started the internal physician advisor group in our organization, and basically since then it’s been very successful. We are 10 hospitals, 200 plus clinics, so it’s a big health care system, almost 50,000 employees. So we moved to cover all the hospitals. At that time ended our relationship with all these outside companies that were doing this consultation for utilization review, and started designing and having observation units across the system.

And back in 2012 I was the system medical director in the health care system, and recently in the last year I’m the vice president of medical practice. So I’m always looking for new things, new ideas, to work with. And I’ve always been an early adopter, technology a little bit too much if you ask my family, especially in the artificial intelligence world.

And when we started seeing the first GPTs, even before ChatGPT 3, that in November 2022 became kind of, you know, the whole world knew about it, the models started looking like what they call the knee of the curve. It started looking like the exponential growth is going to start pretty soon. Back maybe in late 2018, 2019, looks like that growth is changing from linear to really going to start ticking up, and things are going to really get very interesting.

So GPT-3 comes out, fascinating results, everybody is just surprised. A lot of people that were predicting when we’re going to get, and we can talk about that, the general intelligence, or AGI, were predicting years from now. And then now everybody is like, oh my God, this is really happening fast. And then the definition is really more a spectrum for the AGI, it’s not like you’re going to reach a certain bullet point and now you’re at AGI. It’s like we’re more close to AGI today than we were six months ago, et cetera. This is how I’d like to look at it.

So then, from my background, from my work, I started to explore the implication in health care, and what can we do in health care to use that technology. It was clear to me that this is a big deal, this is internet or even more in scale of what’s going on. And a lot of it brings with it, and we can talk about safety and responsibility too.

You know, I still remember the first Thanksgiving around the time where the first GPT came out. I had people over, and one of my nephews is like, you think this is going to, because his wife works for the government and they were working on a bill for foster kids. And we’re just playing around with GPT, and I was like, OK, what are you working on? And she told me, and then, OK, let’s ask it to design a bill that would be, because we want it to pass, that would be appealing to both parties, that would address these issues. And the thing, you know, we’re talking the first models here, just spit out the whole bill in the way it’s written for the legislators in Minnesota. And she looked at me, it’s like, this is three months of work.

So the question that came up then was like, well, is this going to replace us, is this going to take our job, which is a common question. And I still remember clearly, I said, I don’t know if it will replace us or not, but I’m very sure that the people who know how to use it will replace you if you don’t get on board and learn how to use it.

So that’s really what I’m seeing now with more implementation in health care. There is a lot of fear, and I’m usually, I’m a believer, you’re not going to be able to stop change. You better adjust and see how you can make it safer, better, advancing it, rather than trying to just be a block, because it’s not going to work. Change is going to happen. This is how we survive as a species, we continue to change and hopefully improve. Although sometimes we bring with that the risk of going backward, you know, some of the social media stuff that happened that a lot of people talk about right now.

So this is how I got interested. I basically, as I did in utilization review, in a short period of time became an expert, because I just kept reading. You know, that’s all what I’m doing, watching YouTube videos on AI, a lot of interesting stuff out there by knowledgeable people, newsletters, et cetera.

And then I was on a panel at a hackathon at Rutgers sponsored by Johnson and Johnson last fall. They had like 20 teams coming up with ideas that they’ll bring to the market for AI in health care. The stuff that we discussed, and these younger folks designed, it’s just unbelievable.

Like, some of the ideas that came out, how to just have a 1-800 number, like, basically families, a lot of them sometimes want to talk to the doctor, they want to understand what is that angiogram you’re doing, et cetera. And then have your note digested by AI. And we have a lot of models now that are doing a great job basically in the voice, similar to a human. And then the family will have a 1-800 number with a code that the patient will provide, and they can listen in, sons, daughters, et cetera, for the plan today. And then maybe have, because the AI has a lot of time, interact, like, what does it mean to do a HIDA scan? And a lot of ideas like that, and much bigger projects.

So it became clear like this is really good. Like some of the ambient dictation, I’ve tried some of these solutions where you literally put the phone and tell the patient, you know, I have a software, you’re helping me write my notes, if you’re OK with that. And you just do your interview normally, and it is mind-blowing, some of the stuff. Like the patient during just normal conversation would say, I had a colonoscopy in 2003, and then in the past procedure history you get colonoscopy, the result. So it became clear the potential is huge here.

So we started basically, well, again, people are busy, they’re not keeping up with the stuff, they hear about it. And I’m using it a lot outside the patient care, because you’ve got the HIPAA issues, and this is very important, to be very careful not to share specific patient information. But in my administrative job, organizing ideas, organizing emails, I found it extremely helpful in digesting a lot of stuff in a CMS memo or publications, multiple pages, and getting to the point, and then interact with the models on, what does that mean, did it mention the inpatient only list or not, et cetera.

So it really became very clear that this is the way the future is going. And I would really encourage physicians, instead of fighting it, which we do, because we’re busy, because we’re burned out, we do a lot of that, and then what happens, other people start making decisions for us, and then we start complaining.

Similar to what happened, like, my job in finance is revenue cycle, I mean, VP of medical practice in revenue cycle, because most physicians basically complain about all the finance stuff and the clinical documentation improvement. Which, as an internal medicine doctor, actually I said it in a meeting last week, like, the clinical documentation, we always complain that we’re not paid for using our brain, right, as an internal medicine doc. Well, the way you are paid in this system is through clinical documentation. So when you drop the ball there and you don’t respond to CDI, and this is how you’re making money for the system, this is how the system reimburses you, so if you don’t write it down they don’t know what you did or how you’re thinking.

So it’s really, I think, instead of fighting the progress, getting on board, guiding it, deciding what is safe, what is still out there, be careful, et cetera. Physician input is extremely important in that. And if we try to keep fighting stuff, and other people come in and make decisions on our behalf, then we have to be careful later on and not to complain.

Kevin Pho: Yeah. Let me ask you, your KevinMD article is “Mastering prompt engineering to elevate AI in clinical practice,” and I’ve had several episodes of course talking about that intersection between AI and health care. You mentioned that you’re using AI outside of clinical practice to summarize ideas and summarize documents, but within clinical practice it’s still relatively new. And I think something that you said really resonated, is that the physicians who are comfortable with AI are going to eventually replace those who aren’t comfortable with AI, right?

So right now, how are you seeing physicians use AI clinically? You know, we have ambient notes, but other than ambient notes, how are you using, or how are other physicians clinically using AI?

Fady Chamoun: Right. So before we start any discussion, we’re going to have to assume that you’re using a solution that your organization is basically accepting as a secure solution with HIPAA compliance. So this is an assumption.

And the stuff I’ve used, we’ve experimented with a few things in the organization, we’re still not sure what to use yet to implement on a large scale. AI today is extremely good in summarizing. This is where, after the implementation of electronic medical record, this became a big problem, that literally led to a lot of burnout. Too much information, and you don’t have time to review it, and it’s literally in the thousands and thousands, sometimes hours of reading on some patients, and you just have like 10 minutes if you want to start rounding.

So clinical application. I’m a hospitalist with my administrative job, round maybe on a weekend, because I have a full-time job, and I don’t have a lot of time to go through 16 charts when I start in the morning and understand every one of them. The model takes the last progress note, the last two progress notes, summarizes in a few lines, with the right prompt, which is not that difficult. And in the article we talk about what to do for the prompt to be the right prompt.

So I’m a hospitalist rounding on these patients, and some of the models know from, because they keep, basically what’s your job, et cetera. But you put, as we discussed in the article, you put a context. You know, you’re helping me write a brief summary on this patient, providing you with the last progress note, I want you to be clear, no repetition, keep the active issues that are still going on and the relevant past medical history that would interfere or affect this hospital stay.

You take a list, and then when I have my list with all the names, on the back of the paper I print three lines on each one of them. Because I’m rounding in a community hospital, we don’t have a resident, it’s not like everyone has a few line summary for a sign out. I’m doing kind of my own sign out before I start rounding. Because you go to the room and it’s your first day and you don’t remember, this is the patient with AFib and pneumonia, or the AFib and heart failure, right?

So summarizing it is extremely good. And I’ll tell you, like, in the last few models we’ve had, you almost never find an error, almost never in summarizing find an error. And if there is something that is not right, almost universally you go back to the original note that you put in there, and the attending putting a note at 2 a.m. in the morning did a mistake there, said something, or was dictating and was misunderstood through the voice recognition.

So summarization is extremely important, because there is a lot of record and summarizing it. Or changing, basically take your note and change it to patient language. I’ve had people over the years literally in the patient survey saying, I trust this man, he saved my life, and I trust he’s very smart, but I couldn’t understand his medical jargon, right? And you do your best to try to explain heart failure and pulmonary edema, you’re not going to get as close to this.

So you tell the model, and I literally sometimes just to play around, because I speak Arabic and French too, this patient speaks French or Arabic and he’s a mechanical engineer, explain to them why eating salt would make their heart failure worse. And it’s mind-blowing how it comes up with these examples of the pipes and the engine, that will never cross my mind. So patient education also extremely helpful.

I still don’t like it when it starts giving me, and I don’t let it give me, the plan. Summarizing the assessment does a great job, with a differential extremely good. But sometimes you start putting the plan, it’s scary good, but I don’t, because it almost goes through the book, and as a physician, like, I don’t want to do this panel on a 92-year-old, et cetera. So I leave the plan, at least for now, for myself.

Kevin Pho: I certainly want to emphasize this, because for those who aren’t familiar or don’t use AI regularly, really the output is only as good as the prompt that you put in, right? And your article talks about that, you gave some wonderful examples of prompts.

You know, to address your example, sometimes you’re right, it does include the plan, so I have to specifically say only use the information that I type, don’t add any added information, and that way it won’t make up anything. Because I think the strength of AI is really to summarize and manipulate English, but it’s not as strong coming up with its own ideas, and the information that it gives up sometimes isn’t the most accurate. So I think that that is certainly important to keep in the back of your mind.

Give us some tips in terms of how we can generate effective prompts for the AI.

Fady Chamoun: What you’re saying is extremely true. These are language models, they’re extremely good in fabricating stuff, literally, or summarizing stuff. So you have to be careful. They’re so convincing, that’s why in the politics it’s like all the fake stuff is really kind of concerning. It will say it in a way that is so convincing, and it will articulate that in a way that the human basically cannot tell if this is true or not. That’s the scary part. Like, is this based on research or you just came up with this?

You have to remember, these are language models. They’re getting better now with the reasoning, et cetera, but they’re language models, that’s how they’re built. So you need to guide it. It has everything almost written, and how did they train these models, on everything on the internet.

So if you don’t guide it, and you basically say you’re a physician, it doesn’t know you’re not talking about geology or archaeology. You know what I’m saying? It has the library of everything. So you need to guide it.

I’ve had people saying, in the prompt, give me explanation, what are the treatment or the diet restriction for heart failure? OK, give you explanation for a doctor, for a medical student, for a nurse? Who are you, who are you giving the document to? You didn’t put any context. The more you guide, you’re narrowing to this section of the library, that the terms mean this thing. Because terms in health care finance totally mean something totally different in other industries.

So you need to guide the model on who you are, what you’re trying to achieve, you want to be assertive, clear, what tone you have, in which format, you want an email, you want a coding, et cetera. So that’s why these prompts.

So they’re language models, and the scary part, like I use it in utilization review sometimes, and the scary part is you still as a physician need to make the decision if someone is inpatient or not. Because this thing is going to put in words whatever you ask it to do, and sometimes it’s like, oh my God, this is convincing but it’s totally wrong.

So you have to be very specific. The more you say, the more you give examples, and then go back and correct what I mean by that. So I’ve had people saying, oh, it just gave me like 20 lines. OK, how many lines you want? Well, five. Go back and tell it, in five lines can you do it again? These are, basically they’ll put an answer in like a few seconds. So whatever you don’t like, or you say like, well, the tone is too informal, I’m emailing my chief executive here, please make, just go back and say that instead. Like, don’t get frustrated and complain to someone else. This is a computer software, a language model. Just give the command on what you want to improve.

Kevin Pho: Now for those physicians who are newer to AI, are there any resources that you could recommend in terms of where they can start learning about optimized prompts, or is this purely a trial and error exercise?

Fady Chamoun: There are. I usually don’t like, although some people will love that, like, I don’t like to just say, oh, these are, you know, Anthropic Claude has a library of prompts, they’re not health care related. I really would encourage people to say what you want. It’s not that difficult now.

So I’ll give you an example. This short summary for a hospitalist that rounds for a few days only can take too long, right, six weeks of hospitalization, et cetera. So I’ll develop a prompt, and I encourage people to articulate what they want. Now sometimes they’ll come up with something that they really like, save it, put it in a form. But it’s not that difficult, I really encourage people to troubleshoot themselves.

So this short summary, I would say, I’m going to provide you with the last two progress notes from the hospitalist, the infectious disease or the consultant notes, I will basically say the specialty, and the patient has been hospitalized for the last six weeks. I want you to give me a summary of the history of present presentation, hospital course, followed by follow-up recommendations, and then specialty recommendations. And that’s really most of what you need.

And it comes back and you read it and it’s like, oh my God, I will never, because when I’m dictating I sometimes would repeat something, because it comes to your brain, you would say it, and you already talked about heart failure and now you’re talking about Lasix again, et cetera. It will package it in a way that is, and then you say the tone, be formal, be clear, no repetition, and then followed by a list of diagnoses that are compliant with the coding requirements. And it will do that. It won’t just tell you heart failure, it will say acute and chronic congestive heart failure.

So it is really, you need to do it to be confident, and then also to know where you have to be careful. Like, you can’t, don’t go read a book, just do it. You know what I’m saying?

But there are, you know, the article is really, it’s two minutes to read, three minutes to read. I started a YouTube channel, AI for Doctors, and it’s like a few subscribers, just to help people like my friends and our staff, if they want to know a little bit more on the different platforms out there.

Because where do you start? There are a few of them that are big. There’s Anthropic Claude, now they have this 3.5, it’s a pretty good model, it’s free. GPT, OpenAI, it’s pretty good, I still have the Plus subscription, but you can have free subscription too if you don’t use it a lot. I didn’t like so far what’s coming out of Google, but they have their models and they started a lot of this DeepMind work.

There is a model out there in an app called Pi. It is amazing for interaction, for back and forth. Like you’ll be driving and you’re going to a place and you start asking questions, and it answers. Or you’re watching a movie and you want a little bit more context without spoiling it. It is mind-blowing. I think Microsoft actually bought the company.

So there are a few things, but go out there, experiment, look at a couple of people on YouTube that have newsletters to give updates. But again, use it, and then you’ll be more familiar with what it can do and where it needs improvement.

Kevin Pho: We’re talking to Fady Chamoun. He’s an internal medicine physician and hospitalist. Today’s KevinMD article is “Mastering prompt engineering to elevate AI in clinical practice.” Fady, we’ll end with some of your take-home messages that you want to leave with the KevinMD audience.

Fady Chamoun: Yeah, thanks, Kevin. This is so exciting, and as everything new that is exciting, and at this scale, can be scary. I get it. We’re not going to be able to stop it. We should get on board, guide it, so it is safe for our patients, for us. And it can be a potential huge help in decreasing burnout, and decreasing the part of the job that we don’t like, which is spending a lot of time with electronic medical record rather than patients.

Kevin Pho: Fady, thank you so much for sharing your perspective and insight, and thanks again for coming on the show.

Fady Chamoun: Thank you, thanks.