AI-powered health care messaging [PODCAST]

Listen and followApple PodcastsSpotifyYouTubeMore apps · Browse all episodes

Join physician executive Justin Norden as he delves into the world of health care messaging and how large language models (LLMs) like ChatGPT are transforming patient communication. Discover how LLMs can streamline patient messaging workflows, improve response quality, and alleviate the burden on health care professionals. Explore the challenges and opportunities in integrating LLMs, and gain insights into the future of health care innovation.

Justin Norden is a physician executive.

They discuss the KevinMD article, “Can generative artificial intelligence help clinicians better manage patient messages?”

Our presenting sponsor is Nuance, a Microsoft company.

Together, Microsoft and Nuance are leveraging their rich digital technology and advanced AI capabilities to tackle some of health care’s biggest challenges. AI-driven technology promises to revolutionize patient and provider experiences with clinical documentation that writes itself.

The Nuance Dragon Ambient eXperience, or DAX for short, is a voice-enabled solution that automatically captures patient encounters securely and accurately at the point of care. DAX Copilot combines proven conversational and ambient AI with the most advanced generative AI in a mobile application that integrates directly with your existing workflows.

Physicians who use DAX have reported a 50 percent decrease in documentation time and a 70 percent reduction in feelings of burnout, and 85 percent of patients say their physician is more personable and conversational.

Discover AI-powered clinical documentation that writes itself. Visit https://nuance.com/daxinaction to see a 12-minute DAX Copilot demo.

VISIT SPONSOR → https://nuance.com/daxinaction

SUBSCRIBE TO THE PODCAST → https://kevinmd.com/podcast

RECOMMENDED BY KEVINMD → https://kevinmd.com/recommended

GET CME FOR THIS EPISODE → https://earnc.me/8LHpol

Powered by CMEfy.

Transcript

Kevin Pho: Hi, and welcome to the show. Subscribe at KevinMD.com/podcast. Get CME for this episode by clicking on the CME link in the show notes. Today we welcome Justin Norden. He’s a physician executive. Today’s KevinMD article is “Can generative artificial intelligence help clinicians better manage patient messages?” Justin, welcome to the show.

Justin Norden: Kevin, thanks so much for having me here.

Kevin Pho: So we’ll talk about your article in a little bit. Let’s start by briefly sharing your story and journey.

Justin Norden: Perfect. So, Justin Norden, physician and computer scientist by background. I started out on the technical side, actually doing undergrad in computer science and a master’s in computational biology focused on machine learning, and then came to Stanford for med school and business school. For me, what was interesting was coming in with this technical background, seeing what technology could do, and I felt like I kept banging my head against the wall: “I can’t believe we do it this way. We could do better on differential diagnosis. We could do better on so many aspects of technology in patient care.” That’s been the thread that’s kept me going ever since.

I started to take a number of clinical detours. I was part of the team helping launch the Stanford Center for Digital Health; we were doing some of the first telemedicine visits out of Epic. I jumped over to Apple on the health care team, helping to grow and see some of the clinical projects there, and then launched a company called Trustworthy AI, where, as someone who’s been fascinated by technology and what computers could do, I knew there was going to be something different needed if we were going to do this in a health care setting. It ended up going from there. I now work in venture capital, working with startups, but I also teach at Stanford, on faculty in bioinformatics. I teach on digital health and AI in medicine.

And, as with everyone, and I’m sure you’ve had a lot of guests already talk about what’s coming, I’m completely enthralled by what’s happening. The key question for me is how we bring this to our clinical champions around the world safely.

Kevin Pho: Absolutely. And as you alluded to, the promise of generative AI is wide open right now. It’s a Wild West. I interview so many physicians about potential applications of large language models, so I’m interested in hearing your take on it. You talk about that in your KevinMD article, “Can generative artificial intelligence help clinicians better manage patient messages?” So tell us about your article and your approach.

Justin Norden: Absolutely. So in very short, yes, these tools will have a transformative impact across all of health care. But then the next part is, are they having that impact today? The answer is no. We don’t yet know exactly how they perform. These tools are new. We haven’t figured out a way to manage the risks and understanding, but we see the potential, and it’s going to take all of us working together to take those next steps.

So the purpose of this article: I co-wrote it with Spencer Dorn, who’s a physician and vice chair of medicine out at UNC, someone who’s written, and we’ve talked a lot together. He’s written a lot himself as well on what are potential ways we can start to use these tools. And so what we wrote in this article was just expanding a little bit more on how we might use these tools for in-basket messaging, getting rid of that pajama time, which is something that has just been a burden on so many physicians. While lots of people talk about maybe just drafting a note or a response to a patient, there are so many other applications, as we go before that even gets to a clinician’s inbox, and after: to triage beforehand, help label it so it goes to the right person, draft that response, and bring in the right pieces of information. So while people are most used to this chatbot interface of ChatGPT, the potential for these tools comes in so many other non-chatbot-related features.

Kevin Pho: Absolutely. I’m super interested in hearing that. I’m a primary care internal medicine physician, and when you say things like inbox and patient messages, especially in Epic, it just keeps on growing and growing and growing. You could never get back to inbox zero. So hearing about these AI applications to make our lives easier is certainly something that many clinicians are interested in. So tell us how this can happen.

Justin Norden: Absolutely. So what we outline, and how this will happen in the future, I’m sure, will be totally different than what we’re outlining here today, but we just wanted to take a stab at a few steps for how these could happen. There are a number of different points where we could use these tools to aid in the experience.

Part one is when the patient is actually sending the message to a clinician. We’ve all seen messages of varying quality and length and completeness. One of the things where a large language model could help is in actually editing and understanding that message. Let’s say we were missing key pieces of detail. They mention a medication but don’t mention a dose. There’s a step before it even gets sent out, or away from the patient, where that message could be revised to make it more complete. In certain circumstances, maybe even if it’s a medication refill, something super basic, you could actually even skip needing to go all the way through all these other steps, which we’re starting to see in other applications as well.

But this first part is before it even gets to the physician. Let’s see, what can we add to make the message more complete? Or is there anything we can handle, information to give them? “Hey, I heard you mention this piece, a question about your medication. Here’s a piece of information. Did this solve your question? No? Fantastic. How do we refine your message and keep moving?” So that’s a first step of how we can get the patient the right information and/or get them to complete a more complete message, so we can actually resolve that.

Kevin Pho: Now, would that tool go into the patient’s chart? Like, you gave the medication analogy. Would it go into the patient’s chart, look at their medications, and maybe fill in dosages, or would it simply just reword a message to make it more polished?

Justin Norden: Both are options. Both are options. Candidly, I think the first version, from a technical perspective, is far easier just to do the second, of just, “Hey, missing things in this, and let’s make this more polished,” as we move forward. And I think as we go forward over these months and years, we’ll see far deeper integrations of these tools, where it is pulling up all that information from the chart and populating it.

Kevin Pho: Perfect. All right, how else can it help? Step two?

Justin Norden: So right, this is at the beginning, working with the patient beforehand. Step two is, OK, a message does need to be sent. It’s a little bit more complete than it would have been before. Now it’s actually, where should it go? Where should this message get routed to? Is this an automatic thank-you note, where we can send a response and we really don’t need to waste your precious time, as a primary care physician, looking through this and sending that? Should this go to your MA? Should this go to a different person on your staff? Is this a billing question? What does this message type look like, and how can I prevent as much as possible from going to you, so you’re just able to practice at the top of your license in your time?

This has been a dream. A number of academic centers across the country have tried this previously with many tools, and now, with the tools today, the key difference is we are so much better at this that it is going to add utility across the board. With previous attempts, it just hasn’t been good enough. Messages weren’t going to the right spot. But the technology is good enough now where we can route to a much better spot. Never perfect. These tools will never be perfect, and I think this is the key message. But we can get it to the right place, and if it’s wrong, there’s still a human in the loop to redirect it where it needs to go. So this is a big step: Is this urgent, non-urgent, clinical, non-clinical? And then who actually needs to receive this message?

Kevin Pho: So, completely automated, because in my institution, for instance, we have a person going through inboxes to triage the urgency of messages. So you’re saying with these large language models, we have the capability to do that automatically, potentially?

Justin Norden: Exactly. And from some of the early studies I’ve seen, from talking to partners, we can actually do a better job than most of those people today, because it’s not a clinician, usually, who’s reviewing this message. Maybe it’s someone who’s making a best attempt, and they’re not perfect today, right? You’ll get some messages that you didn’t need to see, that could have gone somewhere else. And likewise, these large language models will also miss occasionally and give you the wrong message. What’s different across these tools now is that they’re probably better than that human triage in some cases. That’s what really allows these tools to finally, actually give value, because for years and years we’ve seen AI fail again and again and again in health care: the promises of Watson, the promises of everything before. What’s different is the capability of these tools today. Even though they’re not perfect, they’re now better in many cases than human performance.

Kevin Pho: All right. You’re making the future sound pretty promising from a primary care standpoint.

Justin Norden: Hope so. So the last step, I’ll just fly through, just so we can finish this. One thing that’s really challenging is when you actually get this message. Now a message was refined, and it was deemed that it needs to go to you, Kevin. Now you see this message. You’re dealing with tens, hundreds of these. “Now wait, who is this patient? What is their chart? What do I need to know?” Instead of coming in blind, let’s use these LLMs to summarize key parts from their medical record, to bring up what is relevant, or most likely relevant, to that message, so that rather than all the clicks going around searching for that piece of information, “Can I have this message here, and can I have a summary snapshot of the patient?” So this is the next key step, information retrieval from the chart, to get you what you need. And then finally, now let’s draft you a message, or a handful of messages, to look at.

Kevin Pho: So when you say large language models, are we talking about something proprietary? Are you talking about something like GPT? As you know, even within the world of large language models, there are many. So what kind of engine are we talking about behind the scenes?

Justin Norden: Great question. So large language models is, right, the umbrella term. It’s this type of model. Sometimes people will call it a foundation model, as these often take in images or other forms of media, and we’ll actually see a lot more of that this year from all the big companies. Which one should health care use? Should they use OpenAI’s GPT-4? Should they use Google’s Med-PaLM? Should they use Anthropic’s Claude? The jury’s out. My expectation is health care will use a combination of these, depending on your application. Certain parts of these models will perform better, and certain ones will be closed source, where we don’t know. OpenAI isn’t sharing every detail under the hood of GPT-4.

The other class of models that will get used in health care are the open-source models: Meta building things like Llama, there’s Falcon, there are other models. Instead of having to rely on OpenAI or Google, for example, health care systems will soon be able to download these open-source models, fine-tune them themselves, and deploy them themselves, where they can control every step of that process. So the jury’s out. There’s this open-source versus closed-source debate that is happening across AI broadly, and I expect, for the foreseeable future, health care will start to use both. Over time, maybe health care and clinicians will demand, “We actually only want to use open-source models, so we can control more of these steps.” But we’re not there yet. The field is evolving too fast to say for sure which direction we’re going.

Kevin Pho: So how far are we from the world that you’re describing? I just looked at my inbox before this interview: over 100 messages. So how long do we have to wait for your vision of automating the triage of patient messaging with AI?

Justin Norden: So from a technical perspective, this is possible today. All of the steps I described are absolutely possible today, and I have seen companies working on every one of them, or seen research projects working on every one of them. What is soonest, and what physicians are really starting to play with now, is what’s available in Epic at a few institutions where they’re testing this, mostly this last step of drafting physician responses. So there are studies ongoing. UC San Diego is a pilot site; Stanford’s another pilot site where this is happening today. All of these steps will take longer. That’s the key question, and a big part of it is going to be how much we, as a group, as physicians who are overburdened, demand these things to happen.

The earliest application of generative AI that I would say is really catching on, more than this in-basket example, is ambient documentation. Every prediction is wrong, but this is the thing that has already seen the fast adoption, because clinicians are demanding it: “Wow, this can help me save time on my note. I’m now going up the chain. Hey, I need this now. My friends are using this. Why don’t I have this?” And that’ll be an application where, as we start to see that more and more, adoption of these tools will start to follow.

So technically this is possible today, but being a realist, before this is available for you, Kevin, in your daily practice, it may be a year, it may be three years. Historically, whenever you try to make an early prediction with technology in health care, you’re wrong by years or decades. So I will certainly do everything in my power to figure out how we can get these tools trusted and in your hands faster.

Kevin Pho: And specifically with Epic, are they using a proprietary model? Are they using GPT with their AI ventures?

Justin Norden: Yeah. So Epic has actually done something pretty amazing. It just announced a big partnership with Microsoft, something we really haven’t seen before, even bringing Satya Nadella, Microsoft CEO, to the actual Epic developer conference, talking about the work that they’re doing. So they are developing all sorts of things, thinking about all sorts of things, and we’ll see what gets launched, and when. Epic is typically very conservative; it usually isn’t the fastest-moving organization. But candidly, we’ve seen them move faster and partner more here than they have in other areas. So I’m cautiously optimistic, and they know that this is an important area for clinicians and for use across the country. Not that much is deployed today.

Kevin Pho: So one of the cautions I hear about these large language models is things like patient privacy, and using some of this patient information to learn for future models. So talk more about that.

Justin Norden: Yeah. So this is a big issue. Large language models, and OpenAI, essentially train these tools on the whole internet and have basically used that to learn associations to deliver the capabilities that amaze so many people today. The fear is that clinicians will put in patient information, or others, and that someone will find, on the next release of GPT-4.5, “Oh my gosh, wow, I just found my entire patient record,” because someone put this in the public models, which they use to train on, and then spit back out.

So this is happening. I’ve just had conversations with people from interns to residents to attendings all across the country. People are using maybe their OpenAI app on their phone to record a message, record a patient note, put in PII, even though they shouldn’t, or they maybe try to avoid it, and some of this is going to end up in the next training run for these models. That should not happen. Let’s be very clear: That is not allowed by our data standards, and we should not be using these public sources that train on their data.

With that said, there are private versions where you can start to use these tools, and it doesn’t involve leaking out training data to the public. So there are tools. They’re not available at most institutions; they’re not there yet. It will be possible to use these tools in a secure way, where you’re not leaking patient information out to be trained on for the next release.

Kevin Pho: We’re talking to Justin Norden. He’s a physician executive. Today’s KevinMD article is “Can generative artificial intelligence help clinicians better manage patient messages?” Justin, let’s end with some of your take-home messages you’d like to leave with the KevinMD audience.

Justin Norden: Absolutely. I think, keep playing with these tools. This really is the future for where things are going. Generative AI is the most transformative piece of technology I’ve seen in my career, and it will be available over these coming months and years in different respects across health care. The best way to learn is to start using these tools in your daily life in a different way. Figure out how to use this tool to write better emails, how to do it in non-clinical scenarios, which are far safer, first, because this is going to be part of our daily lives.

Kevin Pho: Justin, thank you so much for sharing your perspective and insight, and thanks again for coming on the show.

Justin Norden: Thanks so much for having me, Kevin.

Listen and followApple PodcastsSpotifyYouTubeMore apps · Browse all episodes