This theCUBE Research conversation examines industry approaches to deploying generative artificial intelligence AI, agents and AI operating models in banking and enterprise. Sri Ambati of H2O.ai, chief executive officer and co‑founder, and Ranil Boteju of Commonwealth Bank of Australia, chief artificial intelligence officer, discuss model governance, model evaluation and operationalization with host John Furrier of theCUBE Research.
Ambati highlights H2O.ai's benchmark work and model evaluation efforts and they emphasize cataloging agents strong governance and robust evaluations. Boteju shares Commonwealth Bank's experience scaling machine learning into production and they report the bank uses generative AI to build customer companions fight fraud saving roughly $1 billion and deploy coding agents that drive nearly three times faster engineering delivery. The discussion covers small language models SLMs versus frontier large language models LLMs agent deployment talent and the emerging AI operating model. The theCUBE Research hosts underscore the need for CEO-led mandates common platforms and sovereign context as an enterprise moat.
Key topics include generative AI agents model governance AI operating models machine learning MLOps model evaluation data sovereignty and practical deployment strategies for banking and enterprise.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For theCUBE + NYSE Wired: Mixture of Experts Series
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for theCUBE + NYSE Wired: Mixture of Experts Series.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: Mixture of Experts Series
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: Mixture of Experts Series. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Sri Ambati, H2O.ai & Ranil Boteju, Commonwealth Bank of Australia
This theCUBE Research conversation examines industry approaches to deploying generative artificial intelligence AI, agents and AI operating models in banking and enterprise. Sri Ambati of H2O.ai, chief executive officer and co‑founder, and Ranil Boteju of Commonwealth Bank of Australia, chief artificial intelligence officer, discuss model governance, model evaluation and operationalization with host John Furrier of theCUBE Research.
Ambati highlights H2O.ai's benchmark work and model evaluation efforts and they emphasize cataloging agents strong governance and robust evaluations. Boteju shares Commonwealth Bank's experience scaling machine learning into production and they report the bank uses generative AI to build customer companions fight fraud saving roughly $1 billion and deploy coding agents that drive nearly three times faster engineering delivery. The discussion covers small language models SLMs versus frontier large language models LLMs agent deployment talent and the emerging AI operating model. The theCUBE Research hosts underscore the need for CEO-led mandates common platforms and sovereign context as an enterprise moat.
Key topics include generative AI agents model governance AI operating models machine learning MLOps model evaluation data sovereignty and practical deployment strategies for banking and enterprise.
Sri Ambati, H2O.ai & Ranil Boteju, Commonwealth Bank of Australia
Ranil Boteju
Chief AI officerCommonwealth Bank of Australia
Sri Ambati
CEO & Co-founderH2O.ai
search
(INTRO)
John Furrier
>> Hello, I'm John Furrier, host of theCUBE. We are here at theCUBE's NYSE Wired studio in New York City. Of course, we're a Palo Alto theCUBE studio connecting Silicon Valley and Wall Street. This is the NYSE Wired program's Mixture of Experts series. We've talked to leaders who are making it happen, this transition to AI. We have theCUBE alumni back. Sri Ambati, founder and CEO of H2O.ai. Great to see you again, Sri. Thank you, Ranil Boteju, Chief AI Officer, Commonwealth Bank of Australia, customer of yours, but also pioneering. Thanks for coming on all the way, flying in from Sydney.
Ranil Boteju
>> It was a whole day.
John Furrier
>> All day. You slept on the plane. Good. Looking good. All right, let's get into this. What's happening, Sri? The AI100 again, another designation for you. Congratulations.
Sri Ambati
>> Thanks for running, Ranil, for making the long—
John Furrier
>> explain the relationship between you guys. And the bank.
Sri Ambati
>> Yeah, one of the aspects of AI is you discover, especially with open source, global markets. One of the things that we found out was this bank from Australia that was downloading our open source and beginning to build a partnership. And fast forward 10 years, it's one of the most advanced AI banks in the world, able to deploy across every sphere of the bank, create sort of thousands of use cases. But top 25 super valuable AI use cases, whether it's fighting fraud, scams, or helping operate, do better business across their commercial entities to across every sphere of the bank. They've deployed AI. So, and we are privileged for the partnership that Commonwealth
John Furrier
>> Bank—Well, I want to dig into some of the specifics, but I want to ask, you're downloading open source software as a bank. You're not allowed to.
Ranil Boteju
>> That was the while it kicked off the relationship. So that was probably back in the, pre-2021 We really formalized our relationship in '21 and we had a lot of success working with H2O.ai, particularly areas like Driverless AI and massively scaled machine learning. Now that we've started to move towards, large language models, agents, GenAI, again, we're building on the existing relationship and lots of areas of partnership.
John Furrier
>> Talk about your innovation strategy because obviously pre-Generative AI machine learning, great for fraud detection. A lot of financial services are using it. What's been the big change in the past 4 years for you?
Ranil Boteju
>> Yes. So I would say that we— so we've been doing AI, most banks have, for at least a decade. we've been scaling machine learning models even pre sort of GenAI. We had hundreds, if not more than 1,000. We've got more than 2,000 now. The big change for us, though, was, the advent of generative AI where you can actually, rather than just predict things, you can actually generate new content. And then you've also got agents that can actually take actions on behalf. So we had a look at those new capabilities and decided that there were basically 4 areas that we wanted to really focus this on. So the first was using these capabilities we can reimagine how we interact with customers. So we built customer companions that interact with customers, help them achieve their goals. That's one area. The second area is frauds and scams. So we've really seen the promise of AI in the context of frauds and scams. Last year, I think we saved about $1 billion of our customers' money going to the hands of fraudsters. And it's a function of all of the AI alerts, all of the generative capabilities we have. And then two other ones. One of them, which I think we're relatively mature, is the use of coding agents and AI tools to really help our engineers. So we have about 9,000 engineers across the Commonwealth Bank, 8,000 of them are using coding agents every day. We're getting almost a 3x improvement in the speed of delivering code. And that's resulted in much faster ability to deliver change. And then the last area is reimagining work for our people, right? So think about knowledge workers, fantastic opportunity to use LLMs, generative agents to remove joyless work. So for sort of being polite. Yeah, grunt work is terrible. All the repetitive tasks and really free up people to focus on using judgment. Yeah, deep synthesis or coaching their teams, spending time with clients. It's really freeing up our people to do much more higher order work.
John Furrier
>> All right. So, Sri, talk about this AI 100. What was the criteria? What was the success? Was it the relationship with these guys? Was it something specific? Can you explain?
Sri Ambati
>> So AI 100 is celebrating the unsung heroes of AI. If you think about its actual value delivery, that last mile is coming from our customers, our prospects, or in general worldwide practitioners who have taken AI and solved something really valuable, whether in personal life, AI for good, or in professional life, in banks and telcos and healthcare. Most of the people obviously Silicon Valley, San Francisco, China, all the AI makers globally are celebrated, right? You yourself interviewed so many superstars in AI all the way from Nobel Prize winners to Yann LeCun in open source to the latest Fei-Fei Li who picked up an amazing thing. So deep learning, you have all these great phenomenal pioneers. But what is lesser known is that the last mile of AI is actually people in the banks, people in the telcos, people who are actually taking this AI and actually applying it for solving real valuable problems for their companies. And we have one from Australia who basically saved his pet's life using AI. And we recognized him as an AI for Good winner. And then there's regulators, the regulators of AI. So
John Furrier
>> saving a dog and pet, that's like rock star status. Everyone can relate to that. I got a tear in my eye already. Question for you guys, because, this is a serious pioneers, but the question when it goes to business is how do I inject intelligence into my organization and how do I encode my business into AI? And that speaks to the journey, right? You mentioned the history with machine learning. What's the key journey elements? Because people are rushing super fast now. The acceleration is months, years, which is a decade?
Ranil Boteju
>> Yeah, look, I'm happy to share with you some of our learnings. So we've been on this journey for many years now. It's a multi-year journey and, we've learned by doing. So I would say one of the first things is the agenda. It really does need to be a CEO agenda. And so getting your CEO on board, really leading it, embracing it, that is almost the critical factor. And we were recently benchmarked by Evident AI, and if I look, we actually came out as fourth in the world, which was, quite pleased about. But when I look at all of the top-ranking banks, what stands out is number one, right? AI in those organizations is very much a CEO mandate. That's probably the first thing. AI is definitely something that you can't delegate. So not just the CEO, all of the leaders across an organization have to embrace it. You've got to use it. You've actually got to touch and feel, use the tools yourself, there's no other way. And that's how you actually build adoption.
Sri Ambati
>> Being on the frontier is
John Furrier
>> the—the frontier
Ranil Boteju
>> intelligence.Yeah, but you've actually got to use it yourself. Using it. Yeah. You can't delegate it to a center of
John Furrier
>> excellence.I want to bring this to a good
Sri Ambati
>> point.But I was going to also add some superstars on Ranil's team and in the bank. Yeah. Are also AI 100 winners. They're empowered. They're also AI 100 winners. So might as well. Isn't kind of— they
John Furrier
>> have a stable of talent.
Ranil Boteju
>> it's something— investing in talent, right? So one of the things that I'm really, again, quite proud about at CommBank is our talent density. So AI is an area where you absolutely need to have the right skills and you've got to have them across an organization, not just in one team. So having the highly skilled people across every part of your organization. That's a really big driver of success as well.
John Furrier
>> But also it's multi-threaded. And I want to bring up the CEO question because it's almost 3 tiers to the progression that we're seeing in the business model transformation. The technical side is great. There's a lot of IT innovation. You mentioned your journey, but when you say CEOs can say inject AI into our business, a lot of companies I talk to, the complaints I hear from the back channel is my CEO couldn't spell AI. And so if the leaders don't know AI, absolutely, that's a huge problem.
Ranil Boteju
>> Yep.
John Furrier
>> And then they won't understand the workflow impact, which is in the line of business.
Ranil Boteju
>> You want your leaders to really embrace it, use it every day, and actually lead from the front. I'd say the second part is kind of linked to that is AI is no longer a technology transformation. So that maybe in the early years getting it started. Now to transform an organization, AI is really about transforming your business model, transforming your ways of working, your org design. So that aspect is again, it's a much broader mandate than just technologists and data
John Furrier
>> scientists.All right. So I want to take it to the next level because now we're seeing— we saw AI Factory. We've been covering AI factories for years. We've been following that. That's obviously full throttle. NVIDIA's got their GTC in Berlin. The stock's up. Everything's back to normal. There's demand. Okay. AI factories produce intelligence. Yeah, people get them on-prem, get some intelligence being generated. They're an entire system producing intelligence. Okay, so that's the output with tokens. Harnessing that intelligence is where we're talking about how you make it work. So CEO understands it, they understand the business model opportunity transitioning. Then you get into the HR operations people. What's the impact on people, talent? And then you get to the operations, CTO, CIO, CISO line of business where it's like, now we're gonna get results. They're all kind of interthreaded cuz they're not silos. We're reporting. We've been reporting that for about a year now and we're starting to see the emergence of an AI operating model.
Ranil Boteju
>> Yes,
John Furrier
>> absolutely.Okay. So,
Ranil Boteju
>> yeah, factory.
John Furrier
>> Yeah.Now operating model sits on top. Now it's like, okay, we're running our business. What's your thoughts on that? Because this is net new discussion because they're dependent, they're layered on
Ranil Boteju
>> top.Yeah. So it's absolutely critical to take that approach. So number one, you can't really transform if you don't do that. You'll just get islands of AI. If you want to transform and reimagine your organization, you have to take that approach. Critical to that though is you need to have a very high degree of coordination, right? So you need to have a very clear, across those different areas, how does it all link together? What are the priorities? What are the use cases you picked? But then it's got to be underpinned by some common capabilities. So for example, what you don't want is your HR team, your marketing team, your tech team, all using slightly different stacks, different patterns, different guardrails. So what you must have is a common platform to build and deploy agents and generative AI. You need to have really consistent guardrails and you need to really quarterback that whole ecosystem activity such that it's sequenced and that you can build once and deploy many. I love that. That last part is something that I think many organizations at the start, thousand flowers bloom and they realize islands of AI, islands of AI, right? But the coordination part is absolutely critical.
Sri Ambati
>> The super The core of AI is actually feedback loops, right? So how do you get the fast feedback loops inside your organization so you increase the learning rate of the organization, right? And over time, intelligence has a nonlinear effect. It goes on
John Furrier
>> each—but you got to have the connected islands because if you have islands of AI, you said you're going to have agents now.
Ranil Boteju
>> They won't get off my island.
John Furrier
>> Absolutely.You have a lot of blockers built into that model. So you have to— AI is horizontal. It's a data model, right? So what is that? Data has to be frictionless horizontally with guardrails Governance, intelligence, trust, boundary crossing.
Sri Ambati
>> And one of the things we think is super important is for an organization, it needs to own its thinking, needs to own the sovereign part of AI. We think it's very important because you want to learn from your prompt responses and over time get your own asset.
John Furrier
>> So sovereignty brings up a lot of questions because if you have an AI factory producing intelligence and Let's just use a car analogy. If in France they're producing Mistral cars and France has got Cohere cars, if you have country models that are not that good, I'm not saying they're not good, but if they're not as good, you want to separate the models. We're seeing that. And then it brings up the question of rules. Agents are rule-based. And my daughter said to me in a conversation a couple weeks ago, when Americans had the revolution, they didn't want the British rule of law. They created their own laws. And so you see that kind of like, what if agents start forming their own laws because they don't like the laws they were living under. Now, if you say IT was an old regime and now we're a new regime, I bring this up as a metaphor to open the question of governance. What are the rules of engagement? This is a rhetorical question, but it brings up a good point of if agents are going to be acting on their own and then talking to other agents, that coordination has to be rules of the road.
Ranil Boteju
>> Absolutely. So within an organization, it's absolutely critical that you have a single catalog and register of all of your agents and all of your AI. That must exist. The starting point is every single bit of AI and agent has to be cataloged and registered. The second part is all of the observability about what they're doing. That needs to be common. You want to have a common set of evaluation criteria, guardrails, so that you can actually evaluate these things in the round and actually compare them. Without those two things, I don't think it's possible.
John Furrier
>> And context becomes relationships and context. Right.
Sri Ambati
>> Context is a great one. So writing down What are the obligations of the bank? What are the obligations of an entity organization and all the kind of SOPs versus the processes that are actually being implemented and the whole action and looking at your organization as almost like from zero, right? Sort of day zero, rebuild everything from first principle, but never has in history of the world writing down What is the intent more important, has been more important.
John Furrier
>> Intent and where the data sits. Now, Saurabh is like, okay, control, where's the data sitting? That's intelligence.
Ranil Boteju
>> You want to manage that content, that context and knowledge. Really, that's really your differentiator, right? So in a world where everyone's using the same models, we've all got the same models. What will differentiate us is how good is the context and knowledge that your models are pointed to. So that's an area where organizations are, that's the moat is the knowledge and context that is specific.
John Furrier
>> The domain expertise.
Ranil Boteju
>> Absolutely.
John Furrier
>> That brings up the long tail use cases. So in the power law of models, at the head of the power law, you got all the frontier general intelligence. But as you move down the curve, depending on how you argue the slope, whether it's a fat belly or long tail, you don't need all that horsepower. Not for everything. If you got a use case and your domain expertise, I might want to use an OpenAI— 100%. Yeah. Low cost, high performance.
Ranil Boteju
>> So look, I guess the strategy is have a broad range of models. So you want to use frontier models, you want to have the smaller flash models, you want to have access to a broad range of open source, obviously from trusted jurisdictions. And then you also want to have your small language models. And really the goal is if you've got a suite of different models, you can then really optimize them for the right outcome. Which is a lot more cost effective and tuning them, making sure they're good.
John Furrier
>> Reinforced learning is becoming more popular.
Sri Ambati
>> We have a product Cascade that actually fine-tunes this continuously. And CBA is building an SLM factory as well, right? So to the truly tight loop, all of that. We actually think the small language— you saw Javabench, H2O is number 1 on Javabench now with generalized classification. So you're seeing the rise of small language models. There's a
John Furrier
>> tabler—who would have predicted that? Oh, we did. 3, 4 years ago we were talking about
Sri Ambati
>> that.4 years
John Furrier
>> agowas a little early,
Sri Ambati
>> but the world didn't believe in that even until a few quarters ago when token costs were still manageable and the models were
John Furrier
>> exactly—yeah, they weren't too early, but it was pretty obvious.
Sri Ambati
>> Absolutely.
John Furrier
>> Multi-modelsintegrating with each
Sri Ambati
>> otherand purpose-built models are better. You can manage them with bigger models, right? So the bigger models you asked about, you almost touched on governance and agent safety. Yeah. The larger models are good at keeping the smaller models in check.
John Furrier
>> Yeah, right. What I love about being my age is that I've seen many waves. And when I see AI, I see the same picture— infrastructure, middleware, application. Not to oversimplify, but we know what that means. But this intelligence layer is replacing the old control plane, semantic layer, whatever you want to— harmonize it. It's been called many things.
Ranil Boteju
>> It's an updated version of that.
John Furrier
>> It's an updated version of that. And that becomes critical if you decouple the models from the underlying infrastructure. You don't have the ultimate flexibility. Do you believe in that principle? What's your thoughts on how people should think about their architecture? Because I want to have the best model at any given time. If someone's got a great engine, I want my car to run it.
Ranil Boteju
>> Yes. So I guess the concept that many organizations are now working through is model routing, model optimization. So how do you ensure that you've got basically the ability to constantly optimize which model you're using to solve what task, right? So that's effectively, everyone's now focused on trying to solve that.
John Furrier
>> So as architects on bringing joy to the world, removing joyless work, you're removing joy, but joy by default increases joy. But you take joy, the anti-joy away, the pain, I'm joyful. So what is joy for a user? That would be self-actualizing in the moment. Of their work.
Ranil Boteju
>> Yes, so joy would be you get to spend a lot more time doing deep synthesis, deep thinking. You could spend more time, everyone can actually have more time to be an individual contributor. You get more smart time spent with your clients, but also time with your team coaching, right? So those aspects will be freed up. The more repetitive tasks, just attending governance meetings, all of those sorts of things, you can now start to shrink. All of these tools actually shrink that.
Sri Ambati
>> Andso intelligence has become ubiquitous like water, right? Like H2O. But judgment has become scarce. So every time something escalates now, it's no longer a simple problem. Yeah, right. In the past, you could delegate something. It would take 2 weeks to finish. Now it's back in 20 minutes.
John Furrier
>> I love that.
Sri Ambati
>> So you need to judge.
John Furrier
>> Well, judgment, original thinking, problem solving, these— I won't call them soft skills, but the deep thinking.
Ranil Boteju
>> You're actually using your slow brain a lot more. So just personally, I find that over the last, I'd say probably 3 or 4 months, my day has really changed. So I'm now doing a lot more deep thinking, slow thinking, problem solving, which is great. Turns out it's also a lot more taxing as well, right? So I need to—
John Furrier
>> actually, what are you thinking about these days? What's on your mind as you have time to reflect? Are you looking at the landscape differently? What's been your change for how you do your job?
Ranil Boteju
>> So it's for me, it's about context, context, context, context. It's about enabling all of the tools that I use. How do they have access to the most context about me and my job and what I'm doing? The more I focused on that, the more basically useful all of my productivity tools have been. So on a personal level, that's been a really big unlock. But weirdly, though, as I was doing that, I was spending a lot of time creating context for my tools. I also realized actually I've got a team of humans. How much time am I spending?
John Furrier
>> What are they doing in their life?
Ranil Boteju
>> No, but I need to share more context with them, right? So that was this kind of epiphany.
John Furrier
>> I find a similar situation. I'm spending a lot more time systematic, look at the system, and then now you got to go into the social interactions.
Ranil Boteju
>> You got to do both. Yeah, you got to do both.
Sri Ambati
>> One contra, almost contrarian thing that's emerging, right, for some of my customers is What used to be a bane, like bureaucracy was a bane in the past, right? But if you look at some countries like Singapore, where bureaucracy is very efficient, it's actually safer. It's actually better to do your finance there. So in a weird way, AI can enforce bureaucracy faster, so it's no longer slower. So writing down the rules of the engagement, as you rightfully said, in the bureaucracy is
John Furrier
>> a feature,
Sri Ambati
>> not a bug,
John Furrier
>> becauseit's
Sri Ambati
>> beingautomated.
John Furrier
>> Suddenly it's being abstracted away. That's abstraction layer.
Sri Ambati
>> That's exactly right.
John Furrier
>> Well, this is where rules come in. That's why coding is so easy for AI, because code, it's all there. It's the easiest thing to get first.
Sri Ambati
>> But, but Ranil brings up a different part, right? So thinking fast, thinking slow. Yeah. Kahneman, legendary book. The thinking slow part takes a little longer.
John Furrier
>> Our videos would be very boring if we were thinking slow. We're thinking fast right now.
Sri Ambati
>> Yeah, but, but AI is— I mean, the last two months, last month. Yeah. Like 700 papers that have 300 years of
John Furrier
>> math.Well, I think, remember the phrase burst mode was a feature in the processors, like inside the semiconductors. We kind of have a burst mode capability now. I can be real time or I can be slow thinking. That's a lot of what AI is doing. If you look at the Pareto curves and some of the things around multi-step reasoning, it's very interesting. It's similar brain activity. You want some hard problems, you have to think longer.
John Furrier
>> Yeah.And that's an AI cost. But also you don't want to make someone think about something that's not critical to the organization. If they're planning their summer vacation around the world, may not be a good use of—
Sri Ambati
>> what we are beginning to see, right, sort of capturing this generalized general-purpose AI into domain-specific value unlock, right? If you can go deeper into a banking application all the way down, you can now start building world models in these specific domains. AI for biology, which is a huge movement about to start and take off, try to kind of really fight cancer. Maybe all started doing AI years ago. That kind of promise of AI is—
John Furrier
>> well, the other thing we're seeing with agents, I'd love to get your thoughts quickly as we wrap up. But, one of the things about token budgets and all these things are being kind of capped and rethought is that you can actually look at who's higher performance and give them more budget if they're putting stuff into production. That's a metric versus someone who's just using tokens. And it's in pilot mode. So you're starting to get these kind of business outcome thinking like, okay, oops.
Ranil Boteju
>> Yeah, no, that's really, you hit a really critical point there. So basically the whole need for really good evaluations is critical. And so if you just look at who's spent the most tokens, not really telling much.
John Furrier
>> Not the right report card.
Ranil Boteju
>> Not the right report card. But if you also look at who's committed code, who's done pull request reviews, delivered the most features, there's a number of outcome metrics that are really hard data, if you can actually build your evals around that, that actually gives you a really solid set. But look, in general, evals is like everything is now— Yeah, it's great. You've got to evaluate it, right? So otherwise you don't know whether you're spending your time on the right things. And so the focus
John Furrier
>> is—You've got to know what eval looks like successfully.
Sri Ambati
>> Eval will live longer than LLMs, right? Yeah. So, and we are doing some
John Furrier
>> work—To your point about judgment, if I'm going to evaluate something, agent, or outcome. I got to know what success looks like. As the old expression goes, don't play chess if you don't know what checkmate looks like. So that's kind of the world we're living in. That's where the opportunity for executives is to lean back and what are we trying to do? And you're kind of orchestrating burst mode when you need it.
Ranil Boteju
>> Yes. Particularly as you're freeing up time. So, I mentioned we're removing joyless work that does give people more time. We want to make sure is that time being used on the most productive things? And that's again where there's an opportunity to evaluate.
Sri Ambati
>> It's a great point, right? Sort of saving time was the original promise, right? So if you save time, you give more joy. AI should make life better, right? So that's kind of the long-term play here.
John Furrier
>> Well, Sri, thank you for coming on. Congratulations. Great practitioner example of executing. And we're going to live— AM Opera is about execution results.
Sri Ambati
>> I was going to say something. To end. We have on predictive, generative, and agentic. H2O has the leading benchmarks in the space which are lead— they lead into the eval world. But I think the lead-in here is how do you start integrating all of this converging into one AI?
John Furrier
>> All right. Well, congratulations and continued success. Love having you guys on. Thank you. I'm John Furrier for theCUBE. We're here all day getting all the data, a lot of water, a lot of H2O. H2O.ai is here. Again, with a real example of real-world AI intelligence in action, with the World Bank here just doing a global business, all doing all the right things. Thanks for watching.
Sri Ambati, H2O.ai & Ranil Boteju, Commonwealth Bank of Australia
search
(INTRO)
John Furrier
>> Hello, I'm John Furrier, host of theCUBE. We are here at theCUBE's NYSE Wired studio in New York City. Of course, we're a Palo Alto theCUBE studio connecting Silicon Valley and Wall Street. This is the NYSE Wired program's Mixture of Experts series. We've talked to leaders who are making it happen, this transition to AI. We have theCUBE alumni back. Sri Ambati, founder and CEO of H2O.ai. Great to see you again, Sri. Thank you, Ranil Boteju, Chief AI Officer, Commonwealth Bank of Australia, customer of yours, but also pioneering. Thanks for coming on all the way, flying in from Sydney.
Ranil Boteju
>> It was a whole day.
John Furrier
>> All day. You slept on the plane. Good. Looking good. All right, let's get into this. What's happening, Sri? The AI100 again, another designation for you. Congratulations.
Sri Ambati
>> Thanks for running, Ranil, for making the long—
John Furrier
>> explain the relationship between you guys. And the bank.
Sri Ambati
>> Yeah, one of the aspects of AI is you discover, especially with open source, global markets. One of the things that we found out was this bank from Australia that was downloading our open source and beginning to build a partnership. And fast forward 10 years, it's one of the most advanced AI banks in the world, able to deploy across every sphere of the bank, create sort of thousands of use cases. But top 25 super valuable AI use cases, whether it's fighting fraud, scams, or helping operate, do better business across their commercial entities to across every sphere of the bank. They've deployed AI. So, and we are privileged for the partnership that Commonwealth
John Furrier
>> Bank—Well, I want to dig into some of the specifics, but I want to ask, you're downloading open source software as a bank. You're not allowed to.
Ranil Boteju
>> That was the while it kicked off the relationship. So that was probably back in the, pre-2021 We really formalized our relationship in '21 and we had a lot of success working with H2O.ai, particularly areas like Driverless AI and massively scaled machine learning. Now that we've started to move towards, large language models, agents, GenAI, again, we're building on the existing relationship and lots of areas of partnership.
John Furrier
>> Talk about your innovation strategy because obviously pre-Generative AI machine learning, great for fraud detection. A lot of financial services are using it. What's been the big change in the past 4 years for you?
Ranil Boteju
>> Yes. So I would say that we— so we've been doing AI, most banks have, for at least a decade. we've been scaling machine learning models even pre sort of GenAI. We had hundreds, if not more than 1,000. We've got more than 2,000 now. The big change for us, though, was, the advent of generative AI where you can actually, rather than just predict things, you can actually generate new content. And then you've also got agents that can actually take actions on behalf. So we had a look at those new capabilities and decided that there were basically 4 areas that we wanted to really focus this on. So the first was using these capabilities we can reimagine how we interact with customers. So we built customer companions that interact with customers, help them achieve their goals. That's one area. The second area is frauds and scams. So we've really seen the promise of AI in the context of frauds and scams. Last year, I think we saved about $1 billion of our customers' money going to the hands of fraudsters. And it's a function of all of the AI alerts, all of the generative capabilities we have. And then two other ones. One of them, which I think we're relatively mature, is the use of coding agents and AI tools to really help our engineers. So we have about 9,000 engineers across the Commonwealth Bank, 8,000 of them are using coding agents every day. We're getting almost a 3x improvement in the speed of delivering code. And that's resulted in much faster ability to deliver change. And then the last area is reimagining work for our people, right? So think about knowledge workers, fantastic opportunity to use LLMs, generative agents to remove joyless work. So for sort of being polite. Yeah, grunt work is terrible. All the repetitive tasks and really free up people to focus on using judgment. Yeah, deep synthesis or coaching their teams, spending time with clients. It's really freeing up our people to do much more higher order work.
John Furrier
>> All right. So, Sri, talk about this AI 100. What was the criteria? What was the success? Was it the relationship with these guys? Was it something specific? Can you explain?
Sri Ambati
>> So AI 100 is celebrating the unsung heroes of AI. If you think about its actual value delivery, that last mile is coming from our customers, our prospects, or in general worldwide practitioners who have taken AI and solved something really valuable, whether in personal life, AI for good, or in professional life, in banks and telcos and healthcare. Most of the people obviously Silicon Valley, San Francisco, China, all the AI makers globally are celebrated, right? You yourself interviewed so many superstars in AI all the way from Nobel Prize winners to Yann LeCun in open source to the latest Fei-Fei Li who picked up an amazing thing. So deep learning, you have all these great phenomenal pioneers. But what is lesser known is that the last mile of AI is actually people in the banks, people in the telcos, people who are actually taking this AI and actually applying it for solving real valuable problems for their companies. And we have one from Australia who basically saved his pet's life using AI. And we recognized him as an AI for Good winner. And then there's regulators, the regulators of AI. So
John Furrier
>> saving a dog and pet, that's like rock star status. Everyone can relate to that. I got a tear in my eye already. Question for you guys, because, this is a serious pioneers, but the question when it goes to business is how do I inject intelligence into my organization and how do I encode my business into AI? And that speaks to the journey, right? You mentioned the history with machine learning. What's the key journey elements? Because people are rushing super fast now. The acceleration is months, years, which is a decade?
Ranil Boteju
>> Yeah, look, I'm happy to share with you some of our learnings. So we've been on this journey for many years now. It's a multi-year journey and, we've learned by doing. So I would say one of the first things is the agenda. It really does need to be a CEO agenda. And so getting your CEO on board, really leading it, embracing it, that is almost the critical factor. And we were recently benchmarked by Evident AI, and if I look, we actually came out as fourth in the world, which was, quite pleased about. But when I look at all of the top-ranking banks, what stands out is number one, right? AI in those organizations is very much a CEO mandate. That's probably the first thing. AI is definitely something that you can't delegate. So not just the CEO, all of the leaders across an organization have to embrace it. You've got to use it. You've actually got to touch and feel, use the tools yourself, there's no other way. And that's how you actually build adoption.
Sri Ambati
>> Being on the frontier is
John Furrier
>> the—the frontier
Ranil Boteju
>> intelligence.Yeah, but you've actually got to use it yourself. Using it. Yeah. You can't delegate it to a center of
John Furrier
>> excellence.I want to bring this to a good
Sri Ambati
>> point.But I was going to also add some superstars on Ranil's team and in the bank. Yeah. Are also AI 100 winners. They're empowered. They're also AI 100 winners. So might as well. Isn't kind of— they
John Furrier
>> have a stable of talent.
Ranil Boteju
>> it's something— investing in talent, right? So one of the things that I'm really, again, quite proud about at CommBank is our talent density. So AI is an area where you absolutely need to have the right skills and you've got to have them across an organization, not just in one team. So having the highly skilled people across every part of your organization. That's a really big driver of success as well.
John Furrier
>> But also it's multi-threaded. And I want to bring up the CEO question because it's almost 3 tiers to the progression that we're seeing in the business model transformation. The technical side is great. There's a lot of IT innovation. You mentioned your journey, but when you say CEOs can say inject AI into our business, a lot of companies I talk to, the complaints I hear from the back channel is my CEO couldn't spell AI. And so if the leaders don't know AI, absolutely, that's a huge problem.
Ranil Boteju
>> Yep.
John Furrier
>> And then they won't understand the workflow impact, which is in the line of business.
Ranil Boteju
>> You want your leaders to really embrace it, use it every day, and actually lead from the front. I'd say the second part is kind of linked to that is AI is no longer a technology transformation. So that maybe in the early years getting it started. Now to transform an organization, AI is really about transforming your business model, transforming your ways of working, your org design. So that aspect is again, it's a much broader mandate than just technologists and data
John Furrier
>> scientists.All right. So I want to take it to the next level because now we're seeing— we saw AI Factory. We've been covering AI factories for years. We've been following that. That's obviously full throttle. NVIDIA's got their GTC in Berlin. The stock's up. Everything's back to normal. There's demand. Okay. AI factories produce intelligence. Yeah, people get them on-prem, get some intelligence being generated. They're an entire system producing intelligence. Okay, so that's the output with tokens. Harnessing that intelligence is where we're talking about how you make it work. So CEO understands it, they understand the business model opportunity transitioning. Then you get into the HR operations people. What's the impact on people, talent? And then you get to the operations, CTO, CIO, CISO line of business where it's like, now we're gonna get results. They're all kind of interthreaded cuz they're not silos. We're reporting. We've been reporting that for about a year now and we're starting to see the emergence of an AI operating model.
Ranil Boteju
>> Yes,
John Furrier
>> absolutely.Okay. So,
Ranil Boteju
>> yeah, factory.
John Furrier
>> Yeah.Now operating model sits on top. Now it's like, okay, we're running our business. What's your thoughts on that? Because this is net new discussion because they're dependent, they're layered on
Ranil Boteju
>> top.Yeah. So it's absolutely critical to take that approach. So number one, you can't really transform if you don't do that. You'll just get islands of AI. If you want to transform and reimagine your organization, you have to take that approach. Critical to that though is you need to have a very high degree of coordination, right? So you need to have a very clear, across those different areas, how does it all link together? What are the priorities? What are the use cases you picked? But then it's got to be underpinned by some common capabilities. So for example, what you don't want is your HR team, your marketing team, your tech team, all using slightly different stacks, different patterns, different guardrails. So what you must have is a common platform to build and deploy agents and generative AI. You need to have really consistent guardrails and you need to really quarterback that whole ecosystem activity such that it's sequenced and that you can build once and deploy many. I love that. That last part is something that I think many organizations at the start, thousand flowers bloom and they realize islands of AI, islands of AI, right? But the coordination part is absolutely critical.
Sri Ambati
>> The super The core of AI is actually feedback loops, right? So how do you get the fast feedback loops inside your organization so you increase the learning rate of the organization, right? And over time, intelligence has a nonlinear effect. It goes on
John Furrier
>> each—but you got to have the connected islands because if you have islands of AI, you said you're going to have agents now.
Ranil Boteju
>> They won't get off my island.
John Furrier
>> Absolutely.You have a lot of blockers built into that model. So you have to— AI is horizontal. It's a data model, right? So what is that? Data has to be frictionless horizontally with guardrails Governance, intelligence, trust, boundary crossing.
Sri Ambati
>> And one of the things we think is super important is for an organization, it needs to own its thinking, needs to own the sovereign part of AI. We think it's very important because you want to learn from your prompt responses and over time get your own asset.
John Furrier
>> So sovereignty brings up a lot of questions because if you have an AI factory producing intelligence and Let's just use a car analogy. If in France they're producing Mistral cars and France has got Cohere cars, if you have country models that are not that good, I'm not saying they're not good, but if they're not as good, you want to separate the models. We're seeing that. And then it brings up the question of rules. Agents are rule-based. And my daughter said to me in a conversation a couple weeks ago, when Americans had the revolution, they didn't want the British rule of law. They created their own laws. And so you see that kind of like, what if agents start forming their own laws because they don't like the laws they were living under. Now, if you say IT was an old regime and now we're a new regime, I bring this up as a metaphor to open the question of governance. What are the rules of engagement? This is a rhetorical question, but it brings up a good point of if agents are going to be acting on their own and then talking to other agents, that coordination has to be rules of the road.
Ranil Boteju
>> Absolutely. So within an organization, it's absolutely critical that you have a single catalog and register of all of your agents and all of your AI. That must exist. The starting point is every single bit of AI and agent has to be cataloged and registered. The second part is all of the observability about what they're doing. That needs to be common. You want to have a common set of evaluation criteria, guardrails, so that you can actually evaluate these things in the round and actually compare them. Without those two things, I don't think it's possible.
John Furrier
>> And context becomes relationships and context. Right.
Sri Ambati
>> Context is a great one. So writing down What are the obligations of the bank? What are the obligations of an entity organization and all the kind of SOPs versus the processes that are actually being implemented and the whole action and looking at your organization as almost like from zero, right? Sort of day zero, rebuild everything from first principle, but never has in history of the world writing down What is the intent more important, has been more important.
John Furrier
>> Intent and where the data sits. Now, Saurabh is like, okay, control, where's the data sitting? That's intelligence.
Ranil Boteju
>> You want to manage that content, that context and knowledge. Really, that's really your differentiator, right? So in a world where everyone's using the same models, we've all got the same models. What will differentiate us is how good is the context and knowledge that your models are pointed to. So that's an area where organizations are, that's the moat is the knowledge and context that is specific.
John Furrier
>> The domain expertise.
Ranil Boteju
>> Absolutely.
John Furrier
>> That brings up the long tail use cases. So in the power law of models, at the head of the power law, you got all the frontier general intelligence. But as you move down the curve, depending on how you argue the slope, whether it's a fat belly or long tail, you don't need all that horsepower. Not for everything. If you got a use case and your domain expertise, I might want to use an OpenAI— 100%. Yeah. Low cost, high performance.
Ranil Boteju
>> So look, I guess the strategy is have a broad range of models. So you want to use frontier models, you want to have the smaller flash models, you want to have access to a broad range of open source, obviously from trusted jurisdictions. And then you also want to have your small language models. And really the goal is if you've got a suite of different models, you can then really optimize them for the right outcome. Which is a lot more cost effective and tuning them, making sure they're good.
John Furrier
>> Reinforced learning is becoming more popular.
Sri Ambati
>> We have a product Cascade that actually fine-tunes this continuously. And CBA is building an SLM factory as well, right? So to the truly tight loop, all of that. We actually think the small language— you saw Javabench, H2O is number 1 on Javabench now with generalized classification. So you're seeing the rise of small language models. There's a
John Furrier
>> tabler—who would have predicted that? Oh, we did. 3, 4 years ago we were talking about
Sri Ambati
>> that.4 years
John Furrier
>> agowas a little early,
Sri Ambati
>> but the world didn't believe in that even until a few quarters ago when token costs were still manageable and the models were
John Furrier
>> exactly—yeah, they weren't too early, but it was pretty obvious.
Sri Ambati
>> Absolutely.
John Furrier
>> Multi-modelsintegrating with each
Sri Ambati
>> otherand purpose-built models are better. You can manage them with bigger models, right? So the bigger models you asked about, you almost touched on governance and agent safety. Yeah. The larger models are good at keeping the smaller models in check.
John Furrier
>> Yeah, right. What I love about being my age is that I've seen many waves. And when I see AI, I see the same picture— infrastructure, middleware, application. Not to oversimplify, but we know what that means. But this intelligence layer is replacing the old control plane, semantic layer, whatever you want to— harmonize it. It's been called many things.
Ranil Boteju
>> It's an updated version of that.
John Furrier
>> It's an updated version of that. And that becomes critical if you decouple the models from the underlying infrastructure. You don't have the ultimate flexibility. Do you believe in that principle? What's your thoughts on how people should think about their architecture? Because I want to have the best model at any given time. If someone's got a great engine, I want my car to run it.
Ranil Boteju
>> Yes. So I guess the concept that many organizations are now working through is model routing, model optimization. So how do you ensure that you've got basically the ability to constantly optimize which model you're using to solve what task, right? So that's effectively, everyone's now focused on trying to solve that.
John Furrier
>> So as architects on bringing joy to the world, removing joyless work, you're removing joy, but joy by default increases joy. But you take joy, the anti-joy away, the pain, I'm joyful. So what is joy for a user? That would be self-actualizing in the moment. Of their work.
Ranil Boteju
>> Yes, so joy would be you get to spend a lot more time doing deep synthesis, deep thinking. You could spend more time, everyone can actually have more time to be an individual contributor. You get more smart time spent with your clients, but also time with your team coaching, right? So those aspects will be freed up. The more repetitive tasks, just attending governance meetings, all of those sorts of things, you can now start to shrink. All of these tools actually shrink that.
Sri Ambati
>> Andso intelligence has become ubiquitous like water, right? Like H2O. But judgment has become scarce. So every time something escalates now, it's no longer a simple problem. Yeah, right. In the past, you could delegate something. It would take 2 weeks to finish. Now it's back in 20 minutes.
John Furrier
>> I love that.
Sri Ambati
>> So you need to judge.
John Furrier
>> Well, judgment, original thinking, problem solving, these— I won't call them soft skills, but the deep thinking.
Ranil Boteju
>> You're actually using your slow brain a lot more. So just personally, I find that over the last, I'd say probably 3 or 4 months, my day has really changed. So I'm now doing a lot more deep thinking, slow thinking, problem solving, which is great. Turns out it's also a lot more taxing as well, right? So I need to—
John Furrier
>> actually, what are you thinking about these days? What's on your mind as you have time to reflect? Are you looking at the landscape differently? What's been your change for how you do your job?
Ranil Boteju
>> So it's for me, it's about context, context, context, context. It's about enabling all of the tools that I use. How do they have access to the most context about me and my job and what I'm doing? The more I focused on that, the more basically useful all of my productivity tools have been. So on a personal level, that's been a really big unlock. But weirdly, though, as I was doing that, I was spending a lot of time creating context for my tools. I also realized actually I've got a team of humans. How much time am I spending?
John Furrier
>> What are they doing in their life?
Ranil Boteju
>> No, but I need to share more context with them, right? So that was this kind of epiphany.
John Furrier
>> I find a similar situation. I'm spending a lot more time systematic, look at the system, and then now you got to go into the social interactions.
Ranil Boteju
>> You got to do both. Yeah, you got to do both.
Sri Ambati
>> One contra, almost contrarian thing that's emerging, right, for some of my customers is What used to be a bane, like bureaucracy was a bane in the past, right? But if you look at some countries like Singapore, where bureaucracy is very efficient, it's actually safer. It's actually better to do your finance there. So in a weird way, AI can enforce bureaucracy faster, so it's no longer slower. So writing down the rules of the engagement, as you rightfully said, in the bureaucracy is
John Furrier
>> a feature,
Sri Ambati
>> not a bug,
John Furrier
>> becauseit's
Sri Ambati
>> beingautomated.
John Furrier
>> Suddenly it's being abstracted away. That's abstraction layer.
Sri Ambati
>> That's exactly right.
John Furrier
>> Well, this is where rules come in. That's why coding is so easy for AI, because code, it's all there. It's the easiest thing to get first.
Sri Ambati
>> But, but Ranil brings up a different part, right? So thinking fast, thinking slow. Yeah. Kahneman, legendary book. The thinking slow part takes a little longer.
John Furrier
>> Our videos would be very boring if we were thinking slow. We're thinking fast right now.
Sri Ambati
>> Yeah, but, but AI is— I mean, the last two months, last month. Yeah. Like 700 papers that have 300 years of
John Furrier
>> math.Well, I think, remember the phrase burst mode was a feature in the processors, like inside the semiconductors. We kind of have a burst mode capability now. I can be real time or I can be slow thinking. That's a lot of what AI is doing. If you look at the Pareto curves and some of the things around multi-step reasoning, it's very interesting. It's similar brain activity. You want some hard problems, you have to think longer.
John Furrier
>> Yeah.And that's an AI cost. But also you don't want to make someone think about something that's not critical to the organization. If they're planning their summer vacation around the world, may not be a good use of—
Sri Ambati
>> what we are beginning to see, right, sort of capturing this generalized general-purpose AI into domain-specific value unlock, right? If you can go deeper into a banking application all the way down, you can now start building world models in these specific domains. AI for biology, which is a huge movement about to start and take off, try to kind of really fight cancer. Maybe all started doing AI years ago. That kind of promise of AI is—
John Furrier
>> well, the other thing we're seeing with agents, I'd love to get your thoughts quickly as we wrap up. But, one of the things about token budgets and all these things are being kind of capped and rethought is that you can actually look at who's higher performance and give them more budget if they're putting stuff into production. That's a metric versus someone who's just using tokens. And it's in pilot mode. So you're starting to get these kind of business outcome thinking like, okay, oops.
Ranil Boteju
>> Yeah, no, that's really, you hit a really critical point there. So basically the whole need for really good evaluations is critical. And so if you just look at who's spent the most tokens, not really telling much.
John Furrier
>> Not the right report card.
Ranil Boteju
>> Not the right report card. But if you also look at who's committed code, who's done pull request reviews, delivered the most features, there's a number of outcome metrics that are really hard data, if you can actually build your evals around that, that actually gives you a really solid set. But look, in general, evals is like everything is now— Yeah, it's great. You've got to evaluate it, right? So otherwise you don't know whether you're spending your time on the right things. And so the focus
John Furrier
>> is—You've got to know what eval looks like successfully.
Sri Ambati
>> Eval will live longer than LLMs, right? Yeah. So, and we are doing some
John Furrier
>> work—To your point about judgment, if I'm going to evaluate something, agent, or outcome. I got to know what success looks like. As the old expression goes, don't play chess if you don't know what checkmate looks like. So that's kind of the world we're living in. That's where the opportunity for executives is to lean back and what are we trying to do? And you're kind of orchestrating burst mode when you need it.
Ranil Boteju
>> Yes. Particularly as you're freeing up time. So, I mentioned we're removing joyless work that does give people more time. We want to make sure is that time being used on the most productive things? And that's again where there's an opportunity to evaluate.
Sri Ambati
>> It's a great point, right? Sort of saving time was the original promise, right? So if you save time, you give more joy. AI should make life better, right? So that's kind of the long-term play here.
John Furrier
>> Well, Sri, thank you for coming on. Congratulations. Great practitioner example of executing. And we're going to live— AM Opera is about execution results.
Sri Ambati
>> I was going to say something. To end. We have on predictive, generative, and agentic. H2O has the leading benchmarks in the space which are lead— they lead into the eval world. But I think the lead-in here is how do you start integrating all of this converging into one AI?
John Furrier
>> All right. Well, congratulations and continued success. Love having you guys on. Thank you. I'm John Furrier for theCUBE. We're here all day getting all the data, a lot of water, a lot of H2O. H2O.ai is here. Again, with a real example of real-world AI intelligence in action, with the World Bank here just doing a global business, all doing all the right things. Thanks for watching.
John Furrier