Subscribe to The Podcast by KevinMD. Watch on YouTube. Catch up on old episodes!

Join us for an engaging discussion on the future of health care diagnostics with physician, health care consultant, and serial entrepreneur Harvey Castro. In this episode, we delve into the transformative potential of Apple Vision Pro in disease detection and diagnostics. From its integration of AI to its impact on early intervention, we explore the nuances of this cutting-edge technology and its implications for health care systems globally. Tune in as we uncover the challenges, opportunities, and ethical considerations surrounding the adoption of AI-driven diagnostic tools.

Harvey Castro is a physician, health care consultant, and serial entrepreneur with extensive experience in the health care industry. He can be reached on his website, harveycastromd.info, Twitter @HarveycastroMD, Facebook, Instagram, and YouTube.

He discusses the KevinMD article, “Apple Vision Pro rebooting medical diagnosis.”

Our presenting sponsor is Nuance, a Microsoft company.

Together, Microsoft and Nuance are leveraging their rich digital technology and advanced AI capabilities to tackle some of health care’s biggest challenges. AI-driven technology promises to revolutionize patient and provider experiences with clinical documentation that writes itself.

The Nuance Dragon Ambient eXperience, or DAX for short, is a voice-enabled solution that automatically captures patient encounters securely and accurately at the point of care. DAX Copilot combines proven conversational and ambient AI with the most advanced generative AI in a mobile application that integrates directly with your existing workflows.

Physicians who use DAX have reported a 50 percent decrease in documentation time and a 70 percent reduction in feelings of burnout, and 85 percent of patients say their physician is more personable and conversational.

Discover AI-powered clinical documentation that writes itself. Visit https://nuance.com/daxinaction to see a 12-minute DAX Copilot demo.

VISIT SPONSOR → https://nuance.com/daxinaction

SUBSCRIBE TO THE PODCAST → https://kevinmd.com/podcast

RECOMMENDED BY KEVINMD → https://kevinmd.com/recommended

GET CME FOR THIS EPISODE → https://earnc.me/uQ9nWk

Powered by CMEfy.

Transcript

Kevin Pho: Hi, and welcome to the show. Subscribe at KevinMD.com/podcast, get CME for this episode by clicking on the CME link in the show notes. Today we welcome back Harvey Castro. He’s an emergency physician, health care consultant, and entrepreneur. We’re going to talk about the Apple Vision Pro and how it reboots medical diagnosis. Harvey, welcome back to the show.

Harvey Castro: Hey, thanks for having me.

Kevin Pho: So Harvey’s been on multiple times. Go to KevinMD.com/podcast to search his name and hear his story and prior episodes. But today let’s jump right into a recent one about the Apple Vision Pro and how it reboots medical diagnosis. For those who didn’t get a chance to read your article, tell us what this one’s about.

Harvey Castro: You know, honestly, at the end of the day it’s pretty simple. We have this new tool called the Apple Vision Pro, and I see this as really helping us in health care. And the article just basically goes through different scenarios, and I’d love to share those, kind of different ways we can use this Apple Vision Pro in health care.

Kevin Pho: Yeah. So for those who aren’t familiar with it, or didn’t see the keynote, or haven’t gone to an Apple store to demo it, just give us a brief description about what it’s about, and what makes this different from a technology standpoint.

Harvey Castro: What makes this device different from all the other ones that are out there, like Meta glasses, is the ability to see through this device. Usually when you put on these devices you really can’t see well through it. And so, for example, there’s neurosurgeries being done right now with the Apple Vision Pro, because you can see that clearly through it.

Kevin Pho: So why is that important? So the fact that you can see clearly through it, and I know we have a lot of these augmented reality, VR type glasses, so why is that important?

Harvey Castro: Yeah, because the image is so clear, and you can literally walk through an OR and do a procedure with the device on, and there’s really no lag time. And I say really, there’s nanoseconds, but our brain really doesn’t see that lag.

And so, why it’s so important, imagine I’m doing surgery on a patient but I need to pull up their X-ray or their CT scan report, and being able to visualize it while sterile and in the operating field.

The other part, imagine I’m about to do surgery, and let’s pretend I’m doing surgery on you, my friend, and I create a digital twin of you. Now I can do the procedure on your digital twin using the Apple Vision Pro, and so making training so much better.

Kevin Pho: All right. So I think the Apple Vision Pro came out within the last few months. So tell us about some of the health care applications of the device.

Harvey Castro: As far as health care, I had a lot of fun taking a look at Epic. I know a lot of people aren’t crazy about Epic, but I was really impressed with their product.

What they did is, imagine putting on the Apple Vision Pro, and as an ER doctor I need information right away. So if a lab result comes up I want to know immediately. And so imagine having this on and having the Epic application on, where I can start seeing the labs as they’re coming in. And say I need to see your X-ray, for example, then I can literally pull up the X-ray as I’m having these on.

But the nice part is, with the Apple Vision Pro I can blow it up and it could be a 100 inch screen, bigger than my PACS machine. And then, to make it great for our patients, I can mirror this to the TV that’s in the room with the patient, and now they can see that X-ray or the lab or what I’m talking about.

Kevin Pho: So for those who haven’t tried it, gone to an Apple store, made an appointment to try it, what’s it like wearing it, and what exactly are you seeing? So when you say that you can see through the device and it’s more defined than you could in real life, just give a description. What exactly are you seeing?

Harvey Castro: Yeah, so right off the bat, say I’m in my office right now and say I had them on, I would be able to see you directly with the vision, and then say I needed to pull up an X-ray, I can literally put it up here in the virtual world and I would be literally seeing it.

And it’s so clear that sometimes, I have it on for several hours, I’ll forget that I have it on. And I know that sounds silly, but for example, say you were giving a presentation and I’m looking at your slide, sometimes I’ll take a picture of the screen. And one time I caught myself taking a picture of the virtual world thinking it was a slide, and I wanted to take a picture of that slide. And that’s how real this thing is.

Kevin Pho: All right. So you talk about innovations in surgery, medical records. What are some of the other case studies or examples?

Harvey Castro: Yeah, I’m going blank on the name of a company, but I’ve seen two different applications in the mental care space, and it’s basically a combination of VR, virtual reality, say the Apple Vision Pro, plus artificial intelligence.

And in that artificial intelligence, what the user can do, this one is mental issues in the sense that if they’re having depression and they’re able to have a conversation with this virtual bot, they’ll see it, they can put calming music, they can white everything out. So now, like I mentioned, that see-through, you can create an environment where, say your favorite warm place is the ocean or whatever that is, and it creates that environment, so you’re literally immersed in this environment. And now you’re having a conversation with this bot to go over your symptoms, and it’s recording.

And then all that information obviously goes to your health care professional, that can go through and say, OK, this is where they’re at on their therapy. Think of therapies like PTSD. And so they’re able to go through different scenarios, and I’m really excited.

The other one that’s really kind of small, but I thought that was interesting, they use a Kung Fu Panda avatar, and it’s to help you relax and meditate. And I thought, you know what, these are great tools for our patients. Some individuals may need this more than others. And I thought, man, this is a simple app that they can go through and immerse themselves in this environment.

Kevin Pho: One of the things that you touched upon is how the Apple Vision Pro can improve early diagnosis of conditions. So what are some examples of that?

Harvey Castro: Yeah. So having this, now, this is a little bit futuristic, and obviously some things need to be FDA approved. But imagine the ability to pair this VR with AR, and being able to have experts like yourself, and say you and I were consulting on a patient, and now I have you with the VR, I’m in with the VR, and we’re seeing a patient, and now I can take advantage of your expertise, and you’d be like, Harvey, this is actually this disease, have you considered it?

Imagine doing grand rounds with this thing on. Imagine not limiting ourselves to just, say, California and Texas, but now we can take the world’s experts and bring them into VR and help us diagnose patients.

So it’s a combination of using AI, VR, being able to have this pass through effect, that now literally we could be walking through the hospital and seeing our patients, and then leveraging technology, leveraging AI, and obviously the human in the loop, someone like you, an expert that I can say, hey, what do you think of XYZ?

Kevin Pho: Now, you mentioned some real life scenarios that are happening now. Epic, you said, has an application for it. Surgeons are really using this as we speak. This is not something that’s going to happen in the next few years, but this is actually happening right now.

Harvey Castro: Yeah, what’s crazy is, literally on day one a neurosurgeon started using this device on day one to do neurosurgery, which I was shocked that they were doing something as delicate as neurosurgery.

Last month I read an article, Fox News, that basically talked about an augmented reality abdominal surgery done not with Apple Vision Pro, but they actually used the Meta glasses, which don’t have as good resolution.

And so doctors are using it. And then anecdotally I’ve noticed on LinkedIn doctors putting surgeries that they have done using the Apple Vision Pro online, and just showing how they’re using it.

One quick example, wound care. Say you had a patient and you had to classify that wound. Imagine being able to use augmented reality that has a 3D camera on it, being able to take a picture of the wound, analyze it with AI, and then tell you how that’s progressing, and then being able to share that with others. I mean, that’s live application.

Someone’s in surgery able to do that right now. And we all know that sometimes our surgeons, depending on what they’re doing, they’re using separate glasses. Imagine being able to blow up the surgical field, or the lesion that’s so small, making it this big, that now the resolution is really clear and they can jump in and act on that particular lesion.

Kevin Pho: Now, for those who aren’t familiar with these terms, just give us a brief contrast between AR, augmented reality, and VR, virtual reality.

Harvey Castro: Yeah, so that’s an excellent question. So think of virtual reality, that literally this is my office, but now I want to make this, I’m in the middle of the beach, so I can convert this office into the middle of the beach.

But say I’m in the medical field and I want to do augmented reality, where I’m having a conversation with you, and say I need to pull up your X-ray, I can pull up this virtual world of augmented reality and show your X-ray. And then, since I have the device on, I can mirror it on the TV so that you can see it. Or ideally, if you had a device and I had a device, then together we’re able to see it.

I think medical education is going to be huge. Imagine the pathophysiology, trying to teach a patient XYZ, having them put on the glasses and say, hey, let’s walk through your heart, let’s walk through your X-ray together, and for them to understand what’s going on with their pathology. And that way they know, you know what, I do need to take my medicine.

Kevin Pho: Now, do you see sometimes the risk of wearing these glasses as just another technological barrier that separates the physician and the patient? You mentioned a scenario in the emergency department. If you came in wearing these glasses to a patient, do you see that acting as a barrier that may impede the relationship you have with that patient?

Harvey Castro: For sure. It’s unfortunate, patients don’t know, especially people who have never heard of the Apple Vision Pro. Sometimes I wear it on the airplane, people look at me like I’m crazy. So it is a factor, it is a barrier.

So obviously, for HIPAA reasons and ethical reasons, we should ask permission from our patients to see if they agree or don’t, and then spend the time with them so that they understand what we’re doing. And I really think it’s important to mirror what we’re seeing on that screen, so that way the patient can be like, OK, I see exactly what he’s seeing, and get them involved and see how this is actually beneficial for their health. But if they don’t see it that way, then obviously I would respect the patient’s wishes and not use it.

Kevin Pho: Now, is that common practice, like patients seeing what you’re seeing, and having what you’re seeing mirrored on a screen? Is that common practice?

Harvey Castro: You know, this is such a new technology. I wore this thing at HIMSS on stage, and people were happy to be able to see what I was seeing, to be able to be transparent. The big thing right now with AI is transparency, and so the more transparent we can be with our data, with what we’re doing, the better for our patients. So I believe this is, or will be, a common practice.

Kevin Pho: Now let’s talk about the technology itself. So are we there yet with all the scenarios and case studies that you mentioned? Is the technology there yet with Apple Vision Pro, or does it still need to have different iterations and future versions?

Harvey Castro: I think we’re to some degree there. This thing weighs, I think, two pounds. I really believe that the future is near, and I’ve seen some studies that are close, these are Meta glasses, and basically I can have a conversation with you and use AI.

But what I’m getting at is, the Apple Vision Pro eventually will just be on the screen of this lens. And so instead of being in this virtual world, I can actually be out, and then I can see everything I’m saying on the actual screen of that lens, which will be amazing. So I think we’ll be there in the near future.

Kevin Pho: So a couple years ago, or maybe a decade ago, we had Google Glass, right? And we had a lot of these case studies and scenarios kind of similar to what we’re talking about now. So what makes this different? What makes the Apple Vision Pro different from Google Glass, and why is it not just a fad?

Harvey Castro: Yeah, no, that’s a great question. Obviously this thing is really expensive, so it can’t be given out to the common consumer, because it costs a lot of money. And honestly, in the hospital system I’m noticing that it’s the departments that have more in their budgets, so like the surgery department, neurosurgery departments, are carrying this.

What makes this different is, the chips that we have today are 10 times better, or more actually, than what Google Glass had. The resolution is there. That lag that I mentioned, it’s really not that much of a lag, you can’t see it, whereas before you could kind of notice it.

So all these technology differences are making it, plus culturally, we’re so many years out from the Google Glass, now we’re starting to accept these things into our workflow, we’re starting to accept it as a patient. And so I think we’re getting there now, as opposed to, that was really early on.

Kevin Pho: Now, you mentioned cost. So in general, how much does an Apple Vision Pro headset cost?

Harvey Castro: Yeah, it’s about $3,500, and that’s a lot of money. And obviously that’s just the lowest model, and then it all goes by memory, so the more memory the more expensive. And then if you buy the warranty, you could easily hit five grand pretty easily if you tried just buying everything that they have.

Kevin Pho: So what are some immediate foreseeable steps, just in the next, I know we’re talking about technology advancing so quickly, say in the next six months or so? What do you see, the adoption? Are you seeing medical institutions and medical schools adopting this? What kind of uptake are you seeing?

Harvey Castro: Yeah, it’s interesting. Health care is a little slow on some things, and the culture is stuck on doing certain things a certain way. But I do see education actually being one of the first things that we will start seeing.

Being able to dissect a human, being able to do it with the Apple Vision Pro, that is huge. Imagine, how many times were you and I in the lab trying to dissect when we were in med school, and we only could go in certain hours, and if we wanted to go at night we had to get a pass? Now I can literally put these on and go through the anatomy and I can learn so much quicker.

So I really see medical schools using this for anatomy lab. The famous saying is, see one, do one, teach one. This is exactly that. Why not do this a 100 times on your Apple Vision Pro before you actually do it on a real patient?

Kevin Pho: Now, has Apple themselves, have they touted some of the health care applications of the Apple Vision Pro through their promotional materials?

Harvey Castro: I haven’t seen anything. Interesting enough, I’ve actually seen third party companies starting to adapt this technology. They’ve done it for the other VR headsets that are out there, and promoting it on the Apple Vision Pro because of that see-through ability that I had mentioned.

Kevin Pho: So you mentioned that you wore this at HIMSS during one of your talks. So at HIMSS, what kind of companies were utilizing that technology, and what kind of applications do you think are through the pipeline?

Harvey Castro: At HIMSS, I went through the exhibit real quickly, I didn’t see any particular company. I’ve just seen different organizations post that they’re using it, the mental care space, a lot of surgery applications for this device.

And then I’ve seen different hospital systems, I’m going blank on the name of it, in California, that are actually using this device. And I’ve seen some ophthalmologists starting to use this device as well. I know this is looking at your eyes, and so instead of using a mouse, every time you turn your eyes around or you look around, that’s where you hit enter. And so I believe with that type of technology, it’s able to look at your eyes and be able to diagnose different things. Obviously that would require FDA approval, but I think the technology is there now, we just need to make sure that FDA approves those types of scenarios.

Kevin Pho: And when you say the technology is there, it’s stable enough for surgery? It’s not going to glitch out, it’s not going to pause in the middle of a procedure or something like that?

Harvey Castro: As far as I have read and seen and spoken to some of these surgeons, they have had zero issues with that. As far as battery life, you are able to connect different batteries to the main battery, so you’re able to extend it for 8, 12 hour surgeries.

Kevin Pho: Any downsides you see to increased uptake and utilization of the Apple Vision Pro in health care?

Harvey Castro: Obviously for me is the cost and the weight. I have read some case studies of individuals saying that they’re having neck pain and issues or headaches from wearing this device. I personally haven’t had any headaches or neck pain from it, but I think this device should get smaller, like I mentioned earlier, maybe it should be just glasses. But I think those are, and obviously the cost, another big barrier.

Kevin Pho: How long can you wear a headset, the Apple Vision Pro, during one sitting?

Harvey Castro: I mean, honestly, I’ve worn it as long as eight hours, and the only reason I stopped is because that was the end of my work day. So I think at the end of the day it’s physiological, each person will have a different tolerance to it.

There are some interesting early case studies that are saying that if you’re wearing this too long, you will have trouble driving. There’s not that many studies, but I did see one, and so I found that interesting. So if that is the case, maybe there might be a scenario that says, OK, once you wear this you’re not allowed to drive for like an hour, so that your brain readjusts.

Kevin Pho: So have you worn this during one of your emergency department shifts?

Harvey Castro: I have not. Right now I’m doing a lot of consulting, so I’ve stepped away from actually being in the trenches. But I’ve been working with different colleagues that are actually using it. The main thing with them, when they want to use it, is the administration is telling them, hey, we’re not sure about this technology yet, do not wear these.

Kevin Pho: We’re talking to Harvey Castro. He’s an emergency physician, health care consultant, and we’re talking about how the Apple Vision Pro reboots medical diagnosis. Harvey, as always, we’ll end with some of your take-home messages that you want to leave with the KevinMD audience.

Harvey Castro: At the end of the day, it’s all about the patient. And just be sensitive to what their needs are, and together let’s use our clinical gestalt and technology to give them better health care.

Kevin Pho: Harvey, once again, thank you so much for sharing your perspective and insight. Thanks again for coming back on the show.