KubeCon + CloudNativeCon EU 2025 | Holly Cummins & Vincent Caldeira, Red Hat

Clips
More from KubeCon + CloudNativeCon EU 2025

Holly Cummins

Senior Principal Software Engineer

Red Hat

Vincent Caldeira

APAC CTO

Red Hat

play_circle_outline Evolution of AI technology from monolithic to distributed models.

play_circle_outline Overcoming Challenges in Adopting Kubernetes for AI and Machine Learning Orchestration in Stateful Environments

play_circle_outline Collaboration with ecosystem partners for enterprise-grade AI solutions.

play_circle_outline Focus on minimizing costs and maximizing performance in AI deployment.

play_circle_outline Leveraging CNCF standards for energy efficiency and metric consistency in AI operations.

Info
Transcript

Holly Cummins & Vincent Caldeira, Red Hat

Holly Cummins

Senior Principal Software Engineer Red Hat

Vincent Caldeira

APAC CTO Red Hat

In this episode of theCUBE, we delve into the insightful discussion featuring Holly Cummins, esteemed Senior Principal Software Engineer, and Vincent Caldeira, Chief Technology Officer of Financial Services for Red Hat, as they share their knowledge and perspectives at KubeCon+CloudNativeCon EU 2025.

Join our hosts Savannah Peterson and Rob Strechay as they interview the experts. Cummins and Caldeira provide an engaging dialogue on the evolution of artificial intelligence, particularly the shift from monolithic to distributed systems and the parallels ... Read more

explore Keep Exploring

what are the benefits of moving from monolithic AI approaches to smaller models and why is orchestration necessary in this new approach? add

What are the challenges in making AI and stateless technologies work together in a cloud-native environment, particularly in the context of Kubernetes? add

What are some areas of contribution for Red Hat in the AI application space, particularly in terms of working with partners and supporting customers in running data stacks in a resilient and distributed way? add

What are some of the challenges companies face when trying to implement AI-enabled infrastructure for experimentation purposes in the enterprise environment? add

What is the role of the CNCF in helping to converge on standards for system energy efficiency? add

bolt Powered by CUBE AI

Holly Cummins & Vincent Caldeira, Red Hat

search

Savannah Peterson

>> Good morning, open source fans, and a welcome back to KubeCon CloudNativeCon. We're here on day two of our three days of live coverage on theCUBE here in London, England. My name's Savannah Peterson, managing these masterful conversations here with Rob Strechay all week. Rob, this is such a fun show and you are the keynote king. I know you had the opportunity to see our next guests and I am super excited about it. Are you feeling ready for this next segment?

Rob Strechay

>> I feel energized and ready. I love this topic and I think, again, who better to have on when you're talking open source than a Red Hat?

Savannah Peterson

>> Exactly. Without further ado, Vincent Caldeira and Holly Cummins, thank you so much for taking the time to be here with us today.

Vincent Caldeira

>> Thank you.

Holly Cummins

>> Thank you.

Savannah Peterson

>> Busy day for you. Y'all did the keynote. Can you give us a little bit of a recap in case the audience hasn't had a chance to check it out? Holly, I'll start with you.

Holly Cummins

>> Yeah, so what we were talking about is the fact that AI is coming, AI has arrived, and the way we were doing AI in the old days, like two whole years ago, we had this really monolithic approach, and we learned from normal computing that monolithic approaches maybe aren't the best. They don't necessarily scale, they're not the most sustainable, and so really what we're looking at is moving to smaller models. And as soon as you have smaller models, then you need orchestration. And as soon as you have orchestration, then you're really looking at distributed computing. And now what we're looking at as well is this move to agents. And so then that's distributed plus plus where you have different systems interacting with lots of other different parts of the system. And then if you sort of step back a bit and you kind of draw the architecture diagram, you're like, "Wait a minute, I've seen this before. Where have I seen this before? It's exactly like Kubernetes." We've already figured out how to solve all these problems with Kubernetes. And so then you sort say, "Okay, this is awesome. We're done, let's go home." But of course it doesn't work like that.

Savannah Peterson

>> I was going to say that's a novel idea. Yeah.

Holly Cummins

>> Yeah, problem solved. No new technology. Okay.

Savannah Peterson

>> for one.

Holly Cummins

>> Yeah, if only. All of these patterns, they almost look like the same patterns that we figured out with Kubernetes. But when we were doing cloud-native computing, really the mental model that we had and the paradigm was we want to be stateless. The more stateless we can make the more things, the more scalable we're going to be, the better, the more cloud-native we're going to be. And as soon as you put stateless and AI in the same sentence, it sort of starts to feel a little bit uncomfortable because AI is all about state. And the way AI is evolving, the way we're getting sort of answers that are more tailored to our domain, is we bring in more and more state and we shovel all this state into the context and then we shovel it back and forth and back and forth. And so we can make it work with Kubernetes, but there's just a little bit of an impedance mismatch. There's some things that we need to do in the community to better make those two models that are kind of opposites fit together to solve the problems that we need to solve.

Rob Strechay

>> So I mean, that was a big theme just from this morning in general, I think, from the keynotes. And you guys had some announcements and you talked about some different places that you're going and where you're contributing. Why don't you bring us up to speed on some of that too? There was the Envoy and the AI Gateway and some other things that you were talking about earlier today.

Vincent Caldeira

>> So first I think it's important to understand we contribute, but we also work very broadly with the ecosystem. So not everything that we have covered today is necessarily a primary area of contribution for Red Hat. Very often we work with partners, especially in the AI application space, to actually achieve our objective. And I think the biggest example of this is data. Fundamentally, we don't really deal in Red Hat around data technology, but obviously to support our customers, we have to make sure that they are able to run in a cloud-native and distributed way data stacks. And obviously enterprise-grade as well, there is this concept of resilience that is extremely important for people to manage. Again, Kubernetes is very good at this, at orchestration, resilience, recovery, but it's not necessarily natively good at handling like the recovery of the state. And in a business it's very important. So of course we have to use our contribution into the native, cloud-native ecosystem and build those capabilities to support the type of stateful workload. But at the same time, to be honest, it's also a lot about contributing and supporting the rest of the ecosystem because all this provider in the data and Kubernetes space from the Red Hat perspective are actually partner of ours.

Savannah Peterson

>> And that's vital. Only more and more data coming with AI and everything that's going on, 50% of data at the edge. So having those partners is really mission-critical for figuring out how to navigate this. You, first of all, you covered a lot in that keynote. Even I'm just sitting here, I'm like, wow, how did you do that?

Holly Cummins

>> We spoke really fast.

Savannah Peterson

>> I'm impressed with both of you, frankly.

Vincent Caldeira

>> Five minutes.

Savannah Peterson

>> Yeah, yeah.

Rob Strechay

>> Exactly. See this is like an eternity for you. So now we can unpack this. That's the good part.

Savannah Peterson

>> Very impressive. So you get to work with a variety of different contributors, people, companies, a varying scale and size. One of the things that Rob and I have noticed a lot in the conversations that we've had is there's still a lot of folks who are just getting started and trying to figure out how to ramp up, whether that be on-prem, at the edge, whatever that might be, but beginning to deploy and experiment with multiple things before they make these massive spends to do this at scale. What would be your advice for someone watching the show today and who's just getting started or someone who's feeling this kind of ominous anxiety around, oh my goodness, am I going to make the right decisions about my tech stack? Holly, you can start.

Holly Cummins

>> I mean, the nature of tech stacks is am I going to make the right decisions about my tech stacks? Almost certainly not, right? Because there's so many different options and the requirements are always evolving. So I think if you sort of start out saying that there's a right and a wrong and I need to get into the right, and then once I'm in the right, I'm never going to change, then you've already-

Savannah Peterson

>> Then I'll be right forever. Right?

Holly Cummins

>> Yeah. You're already wrong. It's about evolving and it's about having that sort of mental resiliency to cope with the fact that sometimes we make the wrong stack decisions. We all do. And there isn't a right answer. And it would be lovely if we could just sort of go and get the recipe and go, "Yeah, you get this, this and this," and then it meets all requirements forever, but of course it doesn't.

Rob Strechay

>> I think one of the things that's really interesting about that, and you just hit on it a little bit, and it kind of brings in the whole AI thing, is that the AI to us and what we've been seeing and the small language models, distribution of them out to the edge, so inference at the edge. What are you seeing from a technology stack happening within Kubernetes and within the communities that really is aiding in that? Because to me that is a perfect place for Kubernetes because, from a control plane perspective.

Vincent Caldeira

>> So maybe an interesting thought because to me what's very interesting is to see what people focus on in tech conference versus actually what people do in the real enterprise scenario.

Savannah Peterson

>> That's a great point.

Vincent Caldeira

>> There's a huge gap right now happening, right?

Savannah Peterson

>> Yes!

Vincent Caldeira

>> I've been on quite a few session talking about how we do huge scale distributed tuning of model. I mean, I learned that now we are going to have two new orchestrators contributed by NVIDIA and Google, and we already have three of them. So I'm like, oh my God. But right now in terms of enterprise adoption, people I think are more fundamentally focused on how do I give an easy access to AI-enabled infrastructure to the people who are starting to experiment with it? And that's actually a platform engineering conversation as well as an AI conversation because it's down to the level of, can I actually use Kubernetes as a construct to provide access to a virtual machine with GPU or just provide my developer a way to serve a model? So I talk a lot right now with enterprise around GPU as a service model, as a service type of construct, and it's kind of really all the way down to the question of how do I get some of my people to start the journey?

Savannah Peterson

>> How do you get them to start and how do you get them to start realizing that value too once you are started, and quantifying that and what's working here versus what will work across the entire organization or this particular vertical or whatever that might be? I mean, you bring up a really good point. Ooh, this is a fun one. How do you determine, because you're helping guide so many people on this journey, how are you giving yourself the confidence in these steps to know that you are sharing best practices? I would imagine there's a lot of dialogue with the community and your partners and everybody else. I went to Holly last time, so I'll go to you, Vincent, on this one.

Vincent Caldeira

>> Look, so I think, first, I don't think there is one way of going about the deployment of AI infrastructure in general. In certain case, we have to look at extremely hybrid model that actually benefit from on-prem private infrastructure typically where you have all this sensitive data that you don't even want to take out to an external cloud provider because then the amount of security compliance, due diligence that you have to do just for this is just tremendous. At the same time, you want to have this kind of way of getting access to Elastic Compute that obviously a lot of US infrastructure provider provide. So depending on the type of the business, the sensitivity of the data, which use case you actually deal with, there can be pretty extreme variations on how you would start. But as an architect, I like the concept of pattern. In fact, there is something we promote in my team now for the past couple of years, so that we call validated pattern. And validated pattern is essentially a GitOps implementation of good architectural practice. So we codify the architecture deployment of the components and we basically start the conversation with our customer with the pattern. So we tell people we build the pattern this way for these reasons, but we understand that in your environment we'll have to do adaptation. And because an adaptation is a change in code, obviously you can just play along and refine your strategy as you go along. It is a very good way as well to go around this big problem, which is the parallelizes by analyze this type of issue. But because I want to have this perfect platform, I'm just going to be for the next month, if not years, just architecting. And there is no such thing as a perfect architecture coming out of the books.

Savannah Peterson

>> No.

Rob Strechay

>> No. I think that to me-

Savannah Peterson

>> Perfectionism is a myth, period, in general, across the board. I'm actually really glad we're talking about this as a theme is the smartest people in our industry, many of which who are here, we're all learning and iterating and exploring together right now at a pretty wild velocity no less. Go ahead, Rob.

Rob Strechay

>> Yeah, I mean I think that to me, you kind of hit on a couple things there where you look at the validated platforms, but there's also this struggle between cost and performance. And then you have the third leg of the stool, which is sustainability or how do you do this? And because there's not enough power in the world for these AI machines and things of that nature, and especially when you hit on the data sovereignty aspect of it as well. So when you see all of these things kind of coming together, especially sustainability to me, 'cause I will say that that is probably, you're one of the few people who've actually talked about it this week. So I'll give you kudos for that because I think it's a big problem and I think being more sustainable will actually lead to lowering costs. And so I'm interested to hear kind of, Holly, where your head's at on this.

Holly Cummins

>> Yeah, it's something that I think Vincent and I both care a lot about, and I think we're at the right place now in the evolution of the technology. Because I think any new technology, you sort of go through a phase of innovation where you're just figuring out what you can do, and then the next phase is the optimization. And I think now we're moving into the optimization phase with these AI technologies where we're sort of stepping back and saying, "That's really cool. Can we do it for a lower cost? Can we do it with less electricity? Because actually, it was really cool, but the cost was too high, the electricity was too high." And what's really nice is, and I think again we're seeing it with AI, is it's not like a trade-off. It's not like we sort of say, "Oh, well we could do the thing that we want to do or we could lower the costs." What we're actually doing is we're seeing better results with lower costs, which is really cool. So, for example, with something like a smaller model-

Savannah Peterson

>> I was just going to say with smaller models as ... Yep.

Holly Cummins

>> Yeah, often it's faster, the costs are lower, and it's more tailored to your domain, so it actually gives you better results. So we fed all the data in and now we're kind of stepping back and going, actually, what if we fed more targeted data in? That would be cool.

Savannah Peterson

>> And you have more flexibility on location when you're doing that as well.

Holly Cummins

>> Yeah, exactly. So it's like this double win or this triple win of on every single metric that you care about, you're better off.

Savannah Peterson

>> It's like when you were talking about the excitement around, I had the visual of you throw the spaghetti against the cupboard to see if it's done or throw the noodles against the wall versus the beautifully plated Italian meal that might come to at a restaurant. It's we're navigating all of that in the kitchen right now in our collective technical kitchens. If you could tell the entire world something about what's going on in our AI revolution right now, or it's not a revolution as much as it is a evolution, what would you tell them? What would you folks who are a little apprehensive, for example, about this journey right now, and not just from a tech stack perspective, but from a emotional and cultural perspective? Because you're out here leading on the forefront, playing with all the things and doing this stuff. Holly, I'll go to you first on this one.

Holly Cummins

>> I mean, it is a period of enormous change. I think revolution maybe isn't the wrong word because it is, technology's always changing, but even when you're used to the pace of change of technology, this is a big change. But I think the fundamentals are the same as we've seen with other new technologies, which is that you need to keep that critical thinking. So you need to say, "Well, this is cool, but what problem am I trying to solve? Is this the right solution? How do I know if it's solving my problem?" So get in those, metrics isn't maybe the right word 'cause it makes it sound really formal, but you need to have that ability to say, "What problem am I trying to solve? Is it solving this problem? How do I know if I've succeeded? What other solutions did I have? Can I combine these in a new way that's maybe even going to give me a better result?" So instead of having the one huge model that you hope will solve your entire business processes, maybe you keep some of the baby along with the bathwater and you combine the old and the new, and then that gives you the best results.

Rob Strechay

>> I like that.

Savannah Peterson

>> I like that a lot. We were talking about electricity, but also greater sustainability conversation and sustainability of, well, AI inference at scale. How does the CNCF ecosystem help power that? Is it the iteration of the new projects that you're always getting to integrate? I mean, beyond the data that you mentioned earlier, I mean, which is obviously a thing, but there's got to be more to it than that. How often are you tapping into the folks here walking around behind us to help make sure that your solution delivery continues to stay awesome?

Vincent Caldeira

>> Yeah. So maybe let's start with the fundamentals, right? I think to me, the biggest role of the CNCF is to help us converge on standards way to actually operate. Having, for example, a single CNCF product that is able to provide me energy efficiency of the system and promoting this across the stack is a supremely important aspect of it. If we all have different metrics to compare how efficient we are, we are never going to converge, right? We are not looking at the same problem, the same-

Savannah Peterson

>> You're not even speaking the same language to a degree.

Vincent Caldeira

>> Yeah, exactly, right?

Savannah Peterson

>> Yeah.

Holly Cummins

>> My apple is greener than your orange.

Vincent Caldeira

>> Exactly. Right?

Savannah Peterson

>> Right. Exactly. Yeah.

Vincent Caldeira

>> Exactly. And that kind of explain why one of our first contribution with IBM and Intel back a few years ago was Kepler Project, because Kepler give us the metrics. And I remember having this conversation with an engineer, we have a lot of fancy use case, we want to run AI ops and like automated auto-scaling of Cluster, and we all converge to the thing that we don't have a way to measure everything is working, so we've got to fix that part first. But it's a very good example of how having the committee like the CNCF kind of putting those standards across and getting industry to adopt them across. Because there could be multiple projects benefiting from Kepler, it doesn't matter. The fact that we're all starting from the same language, the same objective is very important.

Savannah Peterson

>> From the same bowl of apples.

Vincent Caldeira

>> Yeah. And in AI right now is going to be the same. There's multiple layer. I mean, if you have a chance to attend my talk tomorrow, I'll be presenting with IBM Research our thoughts around how do we build a composable architecture for efficient inferencing? And we are not pushing people in specific CNCF projects, so that's very important. We are actually advocating for a composable architecture and people really building component that can work with each other. So we are really allowing for upstream contribution and innovation. We don't want to actually limit people to specific project, but we want to have this kind of agreement about these are actually the four or five layers of optimization, the levers that we can use. If we can at least agree that those are the levers, this is how we measure, and this is how the interoperability works, then as an ecosystem we can quickly solve these problems. We can get faster adoption. And that goes back to your earlier question. We don't expect the majority of the consumer to have to understand the complexity and have to deal with it, 'cause that will be crazy.

Savannah Peterson

>> Yeah. And overwhelming.

Vincent Caldeira

>> Yeah.

Savannah Peterson

>> Yeah. They're keeping you real busy. You're on keynote, you're here on this stage, and you're on stage tomorrow. My goodness, you're running all over the show floor.

Vincent Caldeira

>> Yeah, but the real value is actually between the keynotes is actually catching up with other company about kind of leveling our understanding of what technology seems to be working or is the right area of investment and how we can collaborate. Actually, that's where a lot of behind the scenes opportunities of networking, they will probably result in the next collaboration or-

Rob Strechay

>> Yeah. The hallway track.

Savannah Peterson

>> I was ... Yeah.

Vincent Caldeira

>> Yeah.

Holly Cummins

>> And this kind of event is such a magnificent vehicle for collaboration and for discovering sort of the common interests that you didn't necessarily even know you had. So for example, with sustainability, when you say it, then people sort of poke their head up and go, "Oh yes, I'm really interested in that as well." And so there's the tag for sustainability. And it's just such a great way to make those connections and then drive the technology further across boundaries.

Savannah Peterson

>> Oh yeah, I think that's such a good point, Holly. And we all missed each other. This community is all about community. The open source community is such a people first, human, first group of individuals that I think it's always really powerful and especially compelling when we all get to come together. Holly and Vincent, thank you so much for taking the time in your busy day.

Holly Cummins

>> Oh, thank you.

Vincent Caldeira

>> Pleasure.

Savannah Peterson

>> Well, busy week rather. This is always a blast. And thank you, Rob. Fantastic insights as always.

Rob Strechay

>> All fun.

Savannah Peterson

>> All fun. Hopefully you're having all fun, wherever you might be. We're here in London, England at KubeCon and CloudNativeCon. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.