The Trust Problem Holding Back Health Care AI w/ Sundar Subramanian, CEO, Zyter

We did this study. Our data science lead did this in a clinical setting along with Mayo. It's published. A set of clinicians were using an AI, which was pretty good, right? And were making recommendations. And, uh, the clinicians were under, uh, you know, overriding the AI 84% of the time. 84% of the time, right? They may be overriding it, and sometimes the AI is right, and they may be overriding it. So sometimes they may be tweaking it some- but it's, it doesn't always… it wasn't always point is it wasn't always the underlying capability of the AI itself, but for whatever reason, they were overriding it. So we did this study. We introduced a framework, and the overrides for the same samples went down below 30%. But then when you adjust for the AI shouldn't be recommending X, it should only be recommending high-confidence recommendations, and low-confidence recommendations should be filtered, the overrides went down to, guess what? 1.6%. Welcome back to "Healthcare Rounds," where we give you smart conversations with the people moving healthcare forward. I'm John Marchica, CEO of Darwin Research Group. A quick reminder, if you have a question or comment for me, please send it to healthcare rounds at darwinresearch.com. And if you're not receiving our weekly newsletter, you can subscribe also at darwinresearch.com. Today, I'm speaking with Sundar Subramanian, CEO of Xydor, whose TrueCare platform runs care and utilization management for 45 health plans covering more than 44 million members. And before taking the helm last year, Sundar spent 16 years at TW- at PwC, where he led Strategy& and the firm's digital transformation business. And before that, he led Booz & Company's healthcare services and Medicaid and Medicare work with earlier roles at WellCare and McKinsey. Sundar, welcome to "Healthcare Rounds." Thank you, John. Glad to be here. So most of today we're gonna be talking about AI, but I've gotta ask upfront, what's the biggest difference between running a company and then being at all these blue-chip, uh, consulting firms? What's-- How, how is it different for you? Oh, a great question. I think, uh, the primary difference I would say is, you know, in these other places you're advising or you wouldn't have a lot of, uh, I'd say you could- you had the luxury of focusing on specific things. Here, the switching cost is high, right? Because in this job, you have to go from one topic, like this podcast, to a, you know, looking at a cybersecurity plan, to how is our financial doing, to a board update, to… So it's the switching between so many topics that is, uh, is new, but is also exciting. Yeah. Yeah. Well, um, before we get into your company, 'cause I, I, I do wanna have you spend some time describing what it is that you do. Um, but I wanna ask another kind of big picture question. Obviously, seems like every day that, like, we're all focused on AI, and how is AI gonna disrupt business, and how is it gonna make business better and all of that. But if you were just to, from your perspective, looking at where healthcare is right now, um, give us your top-line assessment, you know, sort of the good, the bad. Just set expectations for where we are today in healthcare and AI. It, it's a great framing. I'd say if you've s-- In, in healthcare, I'd say we are at a significant inflection point, right? Because all of a sudden, what happened is that intelligence became really abundant, right? There's this pervasive cognitive layer that has arrived from frontier models, but also other models, right? And it's extraordinarily capable of answering really deep expertise-based You know, questions in a very fast way, right? That did not exist before. And so AI has moved, I think, extraordinary speed from like, you know, hey, knowing like we were having AI models doing predictions, risk scores, et cetera, years back to, you know, assisting, which is copili- copilot, uh, kind of in clinical settings or even, um, you know, transcribing like Ambient AI to now really doing. I think we have finally arrived at a moment where like, you know, complex workflows, you can stitch together things to start doing, and it's not about, it's not gonna be anymore about another dashboard telling us what is wrong. It's about a system that can help us make it right. And given the power of the cognition and intelligence that exists, and I think there's only gonna be marginal value in making it even more intelligent, right? You don't need like, you know, uh, triple PhD level to discern once you've arrived at. So it's now about how do you stitch together and make it right and make it like, you know, doing things that everybody can trust, and it doesn't has- hallucinate 10% of the times, and it's about auditability, it's about… So it's all about governance, auditability, doing, right? Not just about the level of intelligence. And I think that's a good, exciting place to be, but it doesn't still, it, it means there's a lot of work to be done still. Sure, sure. Well, I've recently read, um, uh, I don't know if you've seen it, Eric Larson's, uh, Healthcare's Oppenheimer moment. Um, actually, if I'm truthful, I, I read a good chunk of it and then had to skim the rest 'cause it's, it's really long, but it's, it's terrific. But in, in your mind, again, back to that framing, how do you bucket things? Is it in terms of, you know, as, as you said, now we're getting to the doing. Do you think of it in terms of AI for workflow or workflow, or AI for clinical decision-making, or is it more about, um, giving you that background intelligence versus agentic? Like how, just help me understand how you would view the healthcare market and its applications for, for AI. It, it's, it's a really good question to start. I would offer that s- one of the things that I am fascinated by is many executives start with like, "Hey, where can I apply AI in healthcare?" Right? And- And I almost think that's the wrong place to start. And just to be you know, a little counter, uh, uh, intuitive here. The, the reason is, you know, it's a, it-- A- a- and it, it's understandable why people started that question just to pay homage to it for a second because like, oh my God, we look around and of course the intelligence is like, you know, multiplying, and of course the cognitive power is enormous. Of course these frontier models are conquering new and new domains. Of course these benchmarks are being broken. So it's only natural to ask where can I apply it? And I think a better question to start would be what outcomes am I trying to drive end to end? And the reason I say this is when you look at prior technology waves When people are start with like, how can I embed the technology, you know, a natural human tendency happens right away, right? That is, I start to embed technology on my current state processes or operating model. It, it happened when electricity came about, right? How many ever years ago? When industrial revolution happened, 40 years it took for productivity to improve. And it's a puzzle that's been studied for, by so many, uh, you know, economists. And essentially, I took away it's because people forgot to redesign the process. They just operated in the same current process, but they embedded elec- you know, like electricity instead of local power generation, right? Like, and of course, results didn't arrive. I think it's helpful to be contextual on what could AI do and keep it in the back of our mind, but start with what is it that you want, you wanna achieve as the outcome, and how do you redesign that end to end so you can solve for the business process debt. And you can very clearly see efforts that are launched by companies that, you know, start out with a certain framing, go down a path. You could almost clearly predict that they will get some, some value out of it, but, you know, they'll get some convenience out of it, but it's not really going to make a big difference because they're just gonna end up automating a few tasks here and there and, uh, maybe they, you know, improve convenience for some of their employees, but it doesn't do anything transmitter because they start out asking the wrong question, and they go down a path very quickly of automating current state. And they may and may not say this in their mission statements, but it's what ends up happening practically. Right. Yeah. So, yeah. That makes a lot of sense. And so if I'm, um, if I'm a CEO of a health system, this is like maybe one application, maybe the first thing that I say is, "Well, it's all about patients and serving patients," and, and like having that patient lens, right? But then right behind that, 'cause the CFO is, she's sitting right next to me, it's like, "Our margins are terrible." And, you know, we've, we've been talking about this for years where we've pressure from the government and CMS and payers and all this. We gotta do better on our margins. So these two things, elements of the triple aim or quintuple aim, we've, we've got to focus on having better care for patients, and we, we've got to focus on shoring up our margins. If that were your lens, if those two things were, you know, kind of right out of the gates, then how would you think about structuring? And I'm, I'm, I bet a lot of the, maybe the C-suite that are listening to this now, they, they can be reflective on what their process was, you know, 18 months ago. But if that, if that were your lens, what would you be thinking about? So those are really good lenses. Like, so let's start with you started defining an outcome with a triple aim kind of outcome, which includes experience for the customer. Then you obviously have financial outcomes for yourself in, as a company. And I think along the way, in order to add, you, you ha- you know, humans at the end of the day have to adopt AI, right? It doesn't diffuse by itself. So something that's more embedded as a metric you sh- I would, I would submit to add to your list is trust. How do you measure that somebody's trusting what it's gonna say in order for the adoption to happen? And let's take that set of like, you know, uh, end business metrics and financial metrics and, uh, you know, process metrics like trust, and how do you really embed it in the process? You start out with those, right? And then I would, I would say take, take an end-to-end process that is core mission to, you know, something that happens across different stakeholders. For instance, right, if you take prior authorization as a process, right? It, it, it goes through a lot of steps, right? It goes first to the clinical diagnosis happen, but somebody has to do the coding. Somebody has to s- you know, prepare the documents and download the charts, and then that has to be submitted. And then on the other hand, whoever is underwriting the risk is going to look at that and verify eligibility, verify if the provider is the right provider. Then they have to go through their own set of processes for applying some medical policy guideline and finally determine if it is really the appropriate level of care, et cetera, whatever that job is that's being done, right? So boiling it down to the different jobs to be done. And at the end of this pro- you know, end-to-end process, the members are waiting for the care to be arriving, and the providers and the nurses that are reviewing these, uh, have their own experiences, right? So if you lay that out, instead of starting with like, hey, let's just embed AI to automate a step or two, if you start to lay this out and look at it end to end, it really-- We've done this. And when we do this, it boils down to really, you know, from w- take one side of it, it, it, like take the, the insurer side of it, it boils down to three jobs to be done. Three jobs to be done. That's what is doing, being done in this process. First job to be done is validating that the coverage and the benefit is actually appropriate and is, is there, right? The second job to be done is that it is a medically necessary service given the context of the case, right? And at the right level. The third job to be done is communication related, is making sure everybody understands that outcome and is actually moving forward with the care. Those are the three jobs to be done Instead of going step by step, if you now re-architect the process for those jobs to happen as close as possible to real time, newer possibilities arrive. Why not the rule set of the benefit and guideline administration is set forward in a way of a common layer that actually is very evident and upfront, right? And why not the, uh, medical necessity reviews is parameterized in a way that it can be adjudicated real time? Why not that, uh, with the help of the, this ubiquitous intelligence layer, you can extract any unstructured information that is not, you know, easy to discern in, in, in the point of service and give back a result. If you, if you, if you start to push us, yourself around that, you could actually see that the end-to-end process could be real time. You, you can, you can see that it's much more reliable and consistent, um, so much higher quality, and extraordinary, uh, patient satisfaction because guess what? You're not waiting for days. You, you-- While you're at the point of service, you're getting a real-time response, and you're getting a suggestion on if something is missing, and you take care of it right there. And of course, it's gonna be good to margins because you're doing it in a much more efficient way, but you're also, you know, creating a phenomenal experience that can set you up for growth. So I'm so glad that you picked this example because this is one that I wrestle with. Obviously, I'm not in the business that you're in or in the prior auth world, but, but, but in my mind, it's the humans, you know. So for example, um, I have a surgery coming up, and my doctor wants me to take some drug before the surgery, and it's expensive because maybe it prevents XYZ from happening during the surgery. Um, that there's this fight that goes on that we're all familiar with. So there are things that need, like you said, that, that, checkboxes that need to happen that'll trigger a prior auth. But a lot of times it's that conversation with, "Well, we need to talk to your doctor first because this isn't on formulary, and this is, of course, prior auth." And that the human element is what ends up taking a lot of time. So I guess flipping it back to you is, how does the AI, or do-- are we getting into the agentic world here, right? How does the AI really collapse that time? Because that's what upsets patients."I've gotta wait for two weeks to get a drug. I can't. I, I've gotta wait a week, and my sche- my surgery is scheduled two days from now, and my doctor can't get ahold of him, the payer," and da da. So talk to that, that, that point specifically. Yeah, I think there are two questions you asked, and those are, I think, connected but separate questions What, what contributes, what contributes to the delays in this process on like it taking days and the care not being rendered at the point like it, it needs to be, right? And the job to be done on a prior auth's validating so the coverage can be, uh, you know, paid for. But it takes days, like you said. And second is the human element. Is it really that, you know, it's… Is it, is it-- Where is the human element? What is the purpose of the human element? And, you know, is it contributing to the delay or contributing to the care? I think let's, let's unpack each of it, right? So the first one I would s- I would submit a lot of what happens today is the administrative getting in the way of the clinical that's adding to the delays. And so for instance, with all good intent in this instance, maybe somebody from the, from your doc's office maybe submitted the prior auth. But they, they pushed a certain set of buttons and something's submitted, and, you know, it's in, it, it's in… A lot of these cases there's a l- heavy, unstructured, handwritten medical record or a clinical that comes with it that somebody's got to sit there and look at and understand the full context of the case. And, and then they-- it takes, you know, it doesn't take minutes, it sometimes takes hours. And then they decide, "Oh my God, XYZ information I'm not able to find, and my criteria says I need that information to validate if it's a blah, blah, blah," right? So then they'll have to go back and ask for that information. So all of this adds up more working time, but elapsed time, and all of this going back and forth. So absolutely, agentic AI can help with reducing those timeframes. You-- The job has to be done. Nobody disputes that somebody's gotta verify whatever, right? But why does it take so much time? Why does… So you could, you could use extraction tools to, um, you know, very thick PD, 600, 700-page PDFs. You could have extraction models read these even handwritten notes very fast, and surface the summaries in a way that helps the clinician. So there's a lot of things that AI could help with in this process, even when there's a… The second part of it, I'd say the human element. In fact, I would argue that if done correctly, it's not a human in the loop as a rubber stamp. It's that the human element actually is magnified to have much more leverage and advantage because it's, it's this, it's you have the agents working for the human. That should be the mindset, right? So I'll give another example of… So, so if you notice, we all went to like, you know, years back, we all went to whatever, right? Like, uh- departments to, we used to stand in, or a pharmacy, we used to stand in a line for checking out, and then a human used to check it out, right? Like physically actually check us out, right? But now most places have self-service checkouts. Six, so, so there's a human involved, but notice the UI and UX for the human is now that's involved is very different. There are six or seven self-service checkouts. When something is not working, there's a red bulb that goes off, right? And so that human knows to pull them, that counter has a problem. They have a different UI in which they override or something and then come and help. We should be thinking about clinicians' experience like that. The agents working for the clinicians doing these kinds of extraction jobs, the summarization jobs and all that, and the human can really think about it and say, "Ah, that's what is going on. I think this makes sense. Let's just approve." And they just approve, and ev- all the communication happen from there, everything happens. Or if they have a question, they interactively ask the question, and these agents are able to go query these records, et cetera, answer these questions. So human potential can go up way more than what we do today because you're unlocking that portion of it which was stranded in administrivia, right? Yeah. And it, what, what I think of is, you know, on, on that side of things, imagine, you know, a large oncology group, let's say. And I, and I would imagine that in that space there's a lot of prior auth that goes on, right? Um, just to the nature of the expense of some of the drugs and pathways and all those kinds of things. And the pathways that have been set, at least at some level, should be structured in such a way to avoid as much of this sort of prior auth, right? But if I'm looking at, now I'm not the payer, but I'm looking at this now from the, the group practice or even you could say make it bigger, an IDN, right? Is, is, is my view going to be different from the payer? And then how am I able to use agentic or other AI tools to be able-- 'cause that, the clinician, that's where all that frustration comes in, right? We talk about the patient, but getting, getting to that point where the AI can push things through faster and I, I c- I can't think of a place where I'd want that to happen more quickly than in cancer. So how, how does that play out? Uh, yeah, exactly. So the more-- I think the potential is even more. Fi-first of all, there's tremendous potential for simpler cases to be automated, right? Everybody can see that kind of like in your hierarchy of things that, um, you know, when there's unstructured information involved, even in simpler cases that are not captured, that can be automated through AI, that increases the potential. But in the com-- more complex cases like oncology or other specialty kind of, um, UM or the, uh, end-to-end care and case management, I think AI is gonna be really differentiated. Um, for the, for a few-- Like, I'll give you a couple of examples, right? So even in co-- So, so in those situations, both from an IDNO health system perspective or from a payer perspective, first of all, understanding the complexity of the full treatment protocol, all of the, you know, uh, variables, all of the different things that needs to be laid out together, that can be simplified using agents picking up… AI is very good. You know, if you set up the right symbolic guardrails around it and, uh, you know, with the right auditability, it, it's very good at dealing with lots of information. It's very good at, you know, summarizing the information. It can be very good at predicting, you know, like, what could be the right, you know, next thing to do. And here's the-- And, and just, and, and when you get into more care management and how do you really help the patient with it, next best action is a phenomenal use case for AI. Because what happens is individually there can be different predictive models and pathways, but putting it all together at a member level and figuring out what is the right course of action and presenting the evidence that supports that, and, you know, dynamically working with a human, uh, you know, who's, who's responsible for the treatment to, like, iteratively create a care plan. I mean, all of those things can give tremendous amount of leverage for a clinician that's actually taking care of a cancer patient. So along those lines, I mean, I, I know we do have some pharma, biotech/life sciences people that listen to this, and we've been writing about agreements that have been happening for drug development, but just kind of describing the surface layer. Do you have any sense, this may be out-outside of your field, or maybe it is, any sense of the role of AI in drug discovery and some of the things either hypothetically, um, where we can see the benefits or real benefits that some organizations are reporting already? Yeah. So I am not the expert on this, but, um, but given, um, in the field of healthcare and what… I think there are a few things that sort of is starting to emerge that I think we're gonna see phenomenal progress on. First is accelerating the drug discovery itself because of the ability to process through lots of different variations and combinations and figuring out. So what took a lot of dollars, um, and from my prior job, uh, observing some of the especially big pharma in the life science, there's a lot of, uh, you know, effort and dollars that are spent on figuring out the, you know, the, the right things to bet on and, and then going through the approval process and, and then discovering at the later stage that, you know, you sort of… You, you know, it's, it's high stakes poker, right? You have to take a path, you have to invest with conviction, and then at the end it may or may not get approved. And that's also why drug prices are so high. And if there was a magic wand that could actually play this out, first of all, you know, figure out what are the right things to pick and, uh, figure out how… You know, it's almost like you can create a digital twin at each stage and see, play it out, like through the different stages of drug discovery and see what happens, and that improves not only your efficacy, but also shortens the time that something takes to come to market. And I, I think those are the areas where, um, it, it's, it… I mean, even more than like administrative or clinical processes, these things are more dependent on the level of intelligence of the node. And, uh, so that's what I meant, like, you know, in, in, in, in payer-provider world or health services- You know, you, you don't need more intelligence, you need more coordination, you need more, you know, orchestration. But in drug discovery, the more the intelligence is, is actually better off because you're gonna get to more, uh, precise in the nature of prediction, and that's gonna help you save a lot of costs down the line, but accelerate the drug development and discovery itself. And that is turning out to be the case, right? And I think that there's tremendous amount of results we'll see on, um, uh, over the coming years on this acceleration of the drug development and the discovery and the phases of this. Yeah. I think a lot of people talk to, you know, what is that, a billion dollars, right, to develop a drug. S- the number moves around depending on who you talk to. Um, certainly is a lot more today than it was 10 or 20 years ago. But I think a lot of people focus on the expense of the clinical trials, right? And, 'cause they are expensive. They take time, the multiple locations, often worldwide locations, that kind of thing. But what really gets fed into that billion-dollar number, that's the time spent, that phase one or prior to that, with all the failures. Like, right, the analogy that had been drug development that is, it may or may not be apt, but it's like drilling for oil, right? All of these failures, and then you're- Exactly … trying to improve your hit rate. And that's where I see, you know, you're-- there's not much that you can do about the cost of clinical trials. There's probably some area there where, you know, whether in this recruiting patients or managing the data that maybe there can be some improvements there. But it's really on that front end where a lot of the costs go up because you're, you're paying for all the failures. When they, when, when that billion dollar plus number is given for what did it cost to develop this drug, it's all the other ones that didn't make it, uh, as well. Um, so, uh, tell me about Xydar. Uh, we've been talking, um, process here with, you know, applications of AI and how you see things, but tell me about your organization. Zyder is, um, is a, a technology and AI company, but I'd like to think of as a company that helps companies achieve outcomes. So I would say more of an outcome company. And essentially, we are setting out to-- So you introduced us as, you know, we work with lots of, um, health plans and providers and, um, you know, over 45 million members are in our platform. So we have scale, but what we, uh, are known for and wanna be known for and continue to improve is, like, how do we help clients achieve outcomes? So that is the, is the problem statement I started with, with, which is, like, you know, from the day of electricity, we put technology in, but the end-to-end business process problem doesn't get solved. It repeats. Every industry has this issue. And so I took this job because, uh, we don't wanna just be this, like, you know, put one model here or solve for one use case here, but t-t-take an area, look at the process end-to-end and help, help drive outcomes, the triple aim, the margins, and create trust along the way, right? So we are really about how do you create an orchestration control plane? How do you create, like, lineage, auditability and all of that, that provides the trust, but most fundamentally, like, helps clients, uh, reinvent the process so they can be, uh, practicing at the top of their license or, you know, as a business being set up to succeed at the top of their potential. So the processes and the stuff that should be taken care of by AI or, you know, business roles is getting taken care of, so they are set up for success. Yeah. I wanna c- that, that's-- Thank you for that, uh, that overview. I wanna come back to something now you've said a couple of times, and you said it in the beginning, and that's that, that trust word. And it's funny how many contexts that on, on this show, like, we, we talk about trust. It just, it just happens, right? And this lack of, for whatever w- whatever the reason. But I think in the world of AI, there's a lot of skepticism and, you know, even if you're using a chatbot, right? Uh, and who am I talking to? Or I know I'm not talking to a person, but do I trust what information is coming to me? Or, um, if I'm a clinician, do I trust this AI agent that's helping me diagnose a pro- a, a difficult patient that I'm having problems? Whatever that is, there's that trust factor. And so in your business, how are you consciously… Because as I said, you've mentioned it a couple of times, how are you consciously thinking about that application and building trust through these processes? Yeah. It's, uh, it's not a trivial question. So because, uh, the in- like I said, intelligence is everywhere, right? The cognition power has improved, but yet outcomes are not there, which means that it alludes to something is missing in the middle. And why aren't the recommendations consistently trusted? I think there is a lot to unpack there. I will give a stat based on a study we did. So to start unpacking this, and, and I would say we are s- like a student here in learning in this topic, but we are also seeing results. So, um, we did this study and, um, you know, uh, our data science lead did this in a clinical setting along with Mayo. It's published. And, um, you know, a set of clinicians were using an AI, which was pretty good, right? And were making recommendations. And, uh, the clinicians were under, uh, you know, overriding the AI 84% of the time. 84% of the time, right? And, uh, they may be overriding it, and sometimes the AI is right, and they may be overriding it. So sometimes they may be tweaking it somewhere, but it's, it doesn't always, it wasn't always point is it wasn't always the underlying capability of the AI itself, but for whatever reason, they were overriding it. So we did this study. We introduced a framework, and the overrides for the same samples went down below 30%. But then when you adjust for the AI shouldn't be recommending X, it should only be recommending high-confidence recommendations, and low-confidence recommendations should be filtered, the overrides went down to, guess what? 1.6%. So, and so that's a startling difference, right? So then we started to unpack why did this happen? I mean, yes, there was a framework. It's got five components, et cetera. But, like, the way I understood what was really going on is the following. Really, the, there were two things that were really leading to that. And, you know, in, in current terminology, I'll, I'll give the answer and then explain why it was the case, right? The current terminology, it's explainability and it's confidence scoring. And why does this matter? So first, explainability is, you know, peop- it, the onus on making sure that the evidence is there and is clear and the lineage is clear is not on the user, is on the system So everybody thinks that, hey, their, their AI is good, they can do research citations on it and some paper, et cetera. No, no, no. You have to present it in the experience within three seconds, right? That's how people know, ah, okay, got it.'Cause that's how quickly people are making a judgment, right? So the onus on experience, uh, explainability is also linked to what you talked about experiences, is like the end user experience has to surface the explainability in a way that's very intuitive very quickly. The second thing that matters, confidence scoring. The way I internalized it is if the AI asks for help when it doesn't know, it increases trust, right? So if it is guessing, if the handwriting is not clear, it's guessing. Say so, that you're guessing, and it will increase trust because what happens as a result, because then, you know that example in the, the supermarket I talked about, the red, red bulbs. Somebody's asking for help, then they know that this is where I need to focus my attention. So when this happens, the human on the other side, two things happen, right? First is they don't have to go through the calculus or whatever recommendation themselves in their head to see if it matches with the AI in order to say that I trust the AI. Because the lineage is being presented so evidently they follow the reasoning without having to redo the thinking. Second, they know that it will ask when it needs help, and over time that creates a lot of trust. So with those two things, the trust factor goes up quite a bit in the system Right? But these are not trivial things, right? And it keeps changing, so you have to keep working on the experience, and you have to keep working on making sure that, you know, it's very, very high fidelity, and when it's not, it's asking for help. It seems like what you're saying, Sundar, that there, that maybe this is, you know, however many 24 months, 36 months, whatever, how- however far we are into this, at least the general population, that this may be a, a newer realization. And when I'm thinking about my initial experience, um, well over a, a year ago with ChatGPT, probably 18 months. It was a while ago. And I'm kind of an early adopter, so whenever it was available, I started playing around with it. And I think everybody's had a similar kind of experience, right? You… It makes you feel good. It gi- you know, it hits just like the dopamine hits that you get with the likes on a, on a LinkedIn post or on a Facebook post, right? Same kind of thing. And then you realize it doesn't take very long, or at least if you're paying attention, this thing, this thing is kind of bullshitting you a little bit, right? And y- it wants to make you feel good, but it's not… It's almost as if it doesn't want to admit that it needs to ask more questions to get a better answer for you, right? Right. And, and so that was… So the early experience was it, it was like this six-week joyful period, and then after that, once I started getting a sense of it, I was like, "I'm not us- I'm not gonna use this. I, I don't like this." And then we switched- … we switched to Claude- Yeah … here, here at Darwin. So we switched to Claude, which is a kind of a different approach Um, I don't know that it is readily asking me questions when it doesn't understand something. I think it still shows that it's the expert or tries to, you know. But I, but my meter is, my BS meter is up. And so I wonder how much of… I guess my-- This is a long way of saying, I wonder how much of this trust factor comes down to, you know, sort of anthropomorphizing the, the AI product to try to make it be friendly, make it s- you know, very conversational. So there's that aspect of it. But you hit on something critical, which is I gain more confidence with Claude when it says something to the effect of,"Well, that's an interesting question. I'm gonna ask you a series of questions now to help me, you know, hone in on whatever response that I'm gonna give." Now I feel like the AI is working. I'm trusting it more, in other words. It's giving me more options to think through. So in my head, I'm using this word thinking, right? It's thinking better because it's, it's even pushing back on my question. Does any of that makes, make sense? It, it makes a lot of sense. You hit on something I wanna pick up on. So it's a really… It, it's a, it's a really evolving field, and if people say this has been solved, they're not, like, they I don't think they've been. So here's the thing. First of all, like, the, the, the different applications of AI, chatbot, like, you know, experiences, you know, creative topics. There's no wrong answer. You, you create, you write, et cetera. Those are different kind of things. What I generally am dealing with is, like, production workflows in which outcomes matter, right? You can't get it-- You can't say, "I got it wrong," like five times out of 100. Like wait, that's, that's not acceptable. So, um, in those kinds of processes, you're he- you, you're having AI help humans make decisions faster, right? So it… You're, you're from this kind of chat domain to decision domain. And the fundamental construct of the intelligence that's buried, the generative AI, is still The same fundamental architecture that came out from 2017, the paper, which is really still token is guessing the next word, right? It- that's what is going on, right? So it's, it's more… A- and now there are other methods that are evolving. JEV and TypeCVI has come up with this approach about, you know, better, better about like specific construct and which decisions are being made. How do you, how do you be more confident? But the question on decisions in order to gain trust, fundamentally, you want uncertainty to be lowered. Yeah. So fundam-- But, but, but AI, it's all probabilistic. So, uh, that's a, that's a big issue, right? Because anything fundamentally statistical or probabilistic is not definitive, right? And generally, wherever you can be business rules driven, wherever you can be deterministic, you want to be deterministic. You don't want to substitute it with AI, because first of all, you're increasing in- influence cost, but more importantly, you're substituting something that was definitive with something that's, uh, statistical and probabilistic, and you don't want that, right? And so, but then, but then it does add a lot of value in unstructured documents and these kinds of things. So then the question to ask is how can you get it to perform as closely as possible to deterministic answers so that you can trust it, right? So which is sort of your experience. You're saying like, "Hey, it's bullshitting me." It, it, it, it's just trying to guess the next word that pleases you. It's not trying to solve the underlying problem because it doesn't know what the underlying problem was or is. So now… But then there are, uh, techniques that are evolving now, uh, JEV is one, on like better constructs for decision-making that are more certain versus introducing this uncertainty. We have experimented with one thing. Uh, we, we introduced this neuro-symbolic logic gates and layers around the AI. So it's sort of like saying like, "Hey, I'm giving you this policy context, and this policy context is to be interpreted with like think of it like an if-then-else statement of these kinds of, you know, logic layers," right? So, so you are, you are letting the AI extract et cetera with, with power, but it's still gated with this kind of an logic layer that makes it more deterministic, right? That sort of, you know, I-- In that example of the statistic I gave, the second dimension I gave was it's able to say when it's not right and confidence, so that's not an easy problem to solve. And the way we have been evolving is by surrounding it with non-AI rules, right? And, uh, so that way it's very gated and, uh, when, when it's n- when the rules are not met or when there's a very specific definition, then the confidence score should be lower, right? And, uh, so that's ways that you can sort of create determinism and certainty, which increases confidence and trust, right, with these kinds of decisions. Sure, sure. Um, well, Sundar, I've got one last question that's sort of the, the, the forecasting question or the looking into your crystal ball question. But I'm not gonna go five years out or 10 years out 'cause it's ridiculous with AI, right? But I just wanna know, what are you looking for, you know, over the next, the next three to six months? And if you're running a health system or if you're, you know, over at, uh, Cigna or something, like, what are the signals or trends, things that, that you're gonna be paying attention to over the next three to six months, knowing how rapidly that AI is, is improving and escalating? Yeah. So I think I, I would say three things, right? Um, first is, so far the adoption of AI has been, like, directly correlated with, like, adoption of the chatbot, right? Like, in spite of the, the significant kind of milestones and progress on… So, so put differently, I'm not, like, waking up thinking about what other benchmark did AI beat today, which has been our life last year, right? So it's not that that's gonna be making the next leap of difference as much as I expect that it's gonna keep, you know, keep improving. I think the kind of things with my clients I'm looking forward is, like, you know, those announcements that show that very key end-to-end processes are getting close to real time. Right? Is PriorArt fully real time? Like across what? Across 95% of what, what needs to happen is it real time. And the 5% and humans are engaged, like the example you gave, where it's, it's very close collaboration is enabled, right? How many end-to-end processes can we make real time? Um, so that's what I would look for. The second thing I'd look for is, are there extraordinary, like you talked about triple aim, right? Are there extraordinary patient outcomes that are coming through because of how AI is collaborating with humans, right? So we talked about the cancer and oncology example. If when, if an AI-- if, if a human care manager, caregiver is collaborating with a set of agents and is able to like take the next best action, and AI helps take the next best action, et cetera, and creating and driving a phenomenal experience that's improving longevity, satisfaction, right, uh, clinical outcomes that are studied and published, I think that would be very exciting because that's, that's what we are looking for. It's not about the benchmarks, uh, models beating benchmarks anymore. It's about like, let's, let's start measuring real world outcomes, right? So it's that. And then the third thing is, um, this factor of trust we are just stepping into, right? It's an extraordinarily powerful technology, but, uh, we have not, as humans in society, had enough time to work with it. I think there's a lot of people that are technically technical geniuses, but like may lack the wisdom coming out of many of these technology shops. And only society will decide where this lands in the end. And for that to happen, I firmly believe that a human has to adopt. If something has to be automated, a human has to adopt the technology. And so that's the third thing I look for, which is like, what, what signals extraordinary trust and adoption? So, uh, yeah, those are the three things I look for. Well, Sundar, this has been fantastic. Um, I've really enjoyed, you know, when tackling a topic like AI, especially, you know, in looking at what your organization does. Even preparing for this interview, I was like, "Well, there's a gazillion questions I could ask this guy," but how do we stay, you know, somewhat narrowly focused into, um, a workable set of topics? And I, I hope we accomplished that. But, uh, thank you so much. I really do appreciate, um, your time. And then if anybody wants to reach you or find your company, is any, any last words? Yeah, zyda.ai is our, uh, website. You can, uh, check us out. And John, thanks for having me on, and it was a really great conversation. I appreciate it, and I also appreciate how you, uh, picked up on specific, like, friction points in the system and how to do things differently, and how to create trust in this, in the way that we do it. So really enjoyed the conversation. Thanks for having me. Thank you for joining us on Healthcare Rounds. If you enjoyed this episode, forward it to a friend and rate, review, and subscribe on your favorite platform. Positive reviews, of course, help other people find the show. And remember, if you have a question you'd like me to answer in an upcoming episode, or if you have a comment about my conversation with Sundar, please send it to healthcarerounds@darwinresearch.com. And if you're not receiving our Take Darwin Research Group's weekly newsletter, you can subscribe at darwinresearch.com. I'm John Marchica. We'll see you next round.