KubeCon + CloudNativeCon NA 2025 | James Harmison & Jennifer Vargas, Red Hat

Clips
More from KubeCon + CloudNativeCon NA 2025

James Harmison

Sr Principal Technical Marketing Manager

Red Hat AI

Jennifer Vargas

Sr Principal Marketing Manager

Red Hat AI

play_circle_outline KubeCon coverage from Atlanta, focusing on AI and Kubernetes intersection.

play_circle_outline Red Hat AI and OpenShift AI distinction explained, clarifying product capabilities and implementations.

play_circle_outline Red Hat AI 3: Enhancing Enterprise Agility with Scalable Inference and Innovations in Open-Source for AI Operations

play_circle_outline Community contributions to CNCF discussed, including Bootc and its significance for enterprise AI.

Info
Transcript

James Harmison & Jennifer Vargas, Red Hat

James Harmison

Sr Principal Technical Marketing Manager Red Hat AI

Jennifer Vargas

Sr Principal Marketing Manager Red Hat AI

In this conversation at KubeCon + CloudNativeCon North America 2025, theCUBE’s Rob Strechay and Savannah Peterson sit down with Red Hat’s Jennifer Vargas and James Harmison to unpack the Red Hat AI 3 announcement and what it means for AI on Kubernetes. They break down how Red Hat AI 3 aligns the broader Red Hat AI portfolio (including OpenShift AI and RHEL AI) to deliver AI inference at scale, help enterprises build agentic AI, and take advantage of innovations such as LLMD, Bootc and support for diverse hardware accelerators across existing Kubernetes enviro... Read more

explore Keep Exploring

What was announced by Red Hat in October related to AI, and why is it significant? add

What is the relationship between Red Hat AI and OpenShift AI, and how do they fit into the broader vision and strategy for AI products? add

What innovations are being made to help enterprises run large language models more efficiently? add

What recent addition was donated and how is it foundational to the AI portfolio? add

bolt Powered by CUBE AI

James Harmison & Jennifer Vargas, Red Hat

search

Savannah Peterson

>> Good morning, open source fans, and welcome back to the South. We're here in Atlanta, Georgia, rolling through day two of our three days of live coverage at KubeCon. My name's Savannah Peterson, bringing you a fantastic next segment with the Rob Strechay. Rob, I think you're the most fluent on the company we have up next of anyone at our company.

Rob Strechay

>> Probably true. I would hazard a guess that yes, having them in here, and this is an exciting topic. I've spent a lot of time with them this week and leading up to this, and there's a lot of interesting stuff in the area we're about to talk to about as well.

Savannah Peterson

>> Absolutely.

Rob Strechay

>> If I could get that out of my mouth.

Savannah Peterson

>> We're going to get there, Rob, we're going to get there. Jennifer, welcome back to the show. So nice to see you.

Jennifer Vargas

>> Thank you so much.

Savannah Peterson

>> And James, wonderful to have you on board with us as well.

James Harmison

>> Hey, thank you. Nice to be here.

Savannah Peterson

>> Yeah, I'm excited. So y'all had, the whole show is about the intersection of AI and Kubernetes and how Kubernetes is powering our AI future. Y'all had a cool announcement back in October, Red Hat AI 3. I'm hoping you can tell us a little bit about that, what that means, and why it's relevant to the community. Jennifer, I'll start with you, but James, I'd love for you to chime in.

Jennifer Vargas

>> Awesome. Yes, back in October we launched Red Hat AI 3, which is our AI platform, we're very excited about it. It does mark a change for us, it's a significant evolution of what we've been doing in AI. We basically, at this time, we announced two key elements for us, which is one, AI inference at scale, which is what everyone wants today. They just want to run large language models at scale. And the second one is helping enterprises build agent AI. So, we're building our foundation to make sure that we can build agents or enterprises can build agents.

Rob Strechay

>> Yeah, I was going to say, but help people understand what Red Hat AI 3 is, because I think people, no offense to Red Hat, but sometimes they get confused of what's in OpenShift AI versus Red Hat AI 3-

Savannah Peterson

>> That's a good point, Rob....

Rob Strechay

>> and things like that. So, help people understand where they should be looking.

James Harmison

>> Yeah.

Jennifer Vargas

>> Do you want to?

James Harmison

>> Yeah, I'll go ahead and answer. So, Red Hat AI is the overall brand for our portfolio of AI products, whereas OpenShift AI is a specific product with a specific set of implementation details and components. Red Hat AI is the sort of vision that we have strategically across a couple of different product segments that help people address the types of challenges they're having. So overall, Red Hat AI 3, we're synchronizing the releases of those products and we expect it to be a little bit more coupled together, in terms of capability delivery across one product impacting the features in another, and things like that.

Savannah Peterson

>> So what are some of the, everyone talks about AI at scale, getting in and out of MVP stage, pretty much the entire industry across verticals is right there, right now, it feels like. So, what are some of the big advantages, the big benefits of this announcement, Jennifer? How is that going to make it easier?

Jennifer Vargas

>> So basically, one of the things that we're at is, how can we help enterprises run large language models, but still being able to lower the cost and still deliver the best response times? And so, one of the things that we have done is done a lot of innovations in open source projects, of course. One of them is called LLMD, and what that promises is basically provide enterprises with more operational agility so they can actually run these models faster, easier, in a better way. And as well, we give them the flexibility to deploy these models using diverse hardware accelerators. And one of the things that we're doing with our inference at scale, one of the benefits is we want to give the opportunity to people that already has Kubernetes to run AI workloads now, right? Most of them have these big enterprise deployments with Kubernetes, but running large language models in a stable way, understanding the performance, it's complex. So, that's what we're investing on to make sure that they can do it at scale and at an enterprise level.

Rob Strechay

>> Yeah. I mean, even Jonathan Bryce was up on stage, and we had a little analyst Q&A with him yesterday and he was talking about VLLM and LLMD and how that open source and compatibility with things like OpenAI and things like that. We had Robert Shaw, one of the maintainers on, actually leading up to KubeCon as well, just to even talk through some of that. What are some of the other things that have been really critical, because you donate a lot back to the CNCF. What are some of the other things that have been going back that really helps bring the community together?

James Harmison

>> Yeah, I'll say one of the recent additions, recently donated was Bootc in a current, I think it's in a sandbox state right now. Bootc is foundational to one of the elements of our AI portfolio. So RHEL AI, Red Hat Enterprise Linux AI is a Bootc-based image, and one of the challenges with doing AI in an enterprise environment is versioning things like your kernel and the driver and getting the appropriate Python libraries. And it's really challenging to get everything to work well together and to do it at scale and to keep it updated. And so, we're using that Bootc technology that we recently donated to CNCF in order to deliver RHEL AI, to make it easier for enterprises to get actionable inference right away. If you've got accelerators and bare metal or VMs, we can get you to inference really quickly using that technology. So, making sure it's getting broad community adoption and support in CNCF is critical to making it something that's going to be easier for enterprises to deploy at scale.

Savannah Peterson

>> How do you decide, I'm curious because Red Hat does such a good job of integrating open source community, donating projects, contributing to projects, and also scaling a big enterprise while you're at it. How do you decide internally what's going to go back to the community? What's going to go, how do you distribute all the goods, so to speak? James, I'll ask you.

James Harmison

>> I could talk about that a bit, yeah, yeah.

Savannah Peterson

>> Yeah, I'm curious. It's got to be kind of fun.

James Harmison

>> One of the key... We build everything in the open from the get-go to start with. So when we talk about donating something to CNCF, it's not because we weren't building it in the open already. We usually were, and we're hoping to get community contributions and people interested and excited about the technology. We generally hit this sort of critical mass, where enough people become really interested in a technology that we know it's a hit in the open source community and people are finding value in it. And those are the things that we say, "Okay, you know what? This really needs governance beyond just Red Hat's interests." And those are the kinds of projects that we make sure to try to get donated and in the hands of some larger foundation where more people can provide input to how they're running.

Savannah Peterson

>> I love that. So just in case a community member, a new community member here in open source land is listening, this hype, this excitement that we get around these projects really does drive business value, business decisions, and investment at a real level to help make sure that what's built is the best it can possibly be. I love that.

James Harmison

>> Yeah, there's no such thing as too much input, right? If enough people have that problem, we know that it's something that's worth addressing.

Savannah Peterson

>> Yeah, yeah.

Rob Strechay

>> So, help us understand some of the top use cases that you're seeing as you go around, because you guys are out talking to customers a lot and seeing things. What's going through their head and what are they trying to build now?

Jennifer Vargas

>> Yeah, so what we're seeing is some mix between your regular machine learning and your generative AI. So, we still see machine learning. We have large enterprises that are using prediction for operations or anything that has to do with maintenance operations or customer service or fraud detection, that still keeps going on, we see a lot of those use cases. On the agentic AI side, we see a lot, well, sorry, on the generative AI side, we see a lot of your virtual assistants, your knowledge-based use cases. Those are across every enterprise. So, we see it's very interesting use cases. We have use cases where cities are using it to maintain languages or to connect between people in a city so you can serve more people with translating services. We also see situations where virtual assistants that can help out customers connect better with their enterprises. There's a variety of them. I don't know, James, you work closer with the customers, so you might have some-

James Harmison

>> Yeah, yeah. One of the big customer challenges I've seen come up over and over again is where human analysts have to make rational decisions about areas that have large volumes of information. So in financial services doing real estate investment, the volume and diversity of real estate data is so high that typically analysts that are making real estate decisions are having to pour through so much content in order to be able to make decisions. And having something that is able to parse all of that stuff and that unstructured, raw sort of human text format and accelerate their decisions is real dollars on the line in terms of capability there.

Savannah Peterson

>> I think that's a great example. I like the real estate one because you think of cops, right? There's not always comparable projects in the world, as well, there's not always comparable real estate data depending on what's going on. And when you said that, I just saw all of the tabs that I actually quite literally currently have open on my laptop, merging into one, where I could then know what I needed to prioritize and do and see a lot clearer. It's almost like an elevated observability at that point where you're able to make decisions a lot faster. I mean, everything's moved fast in tech for a long time, but it's quite quick right now. How are you keeping up and knowing that, say the features, I mean, it's this community feedback loop, but I'm wondering if there's other discussions internally around this. How are you keeping up with the velocity and continuing to serve at the rate of innovation that you are? Jennifer, I'll start with you on this one.

Jennifer Vargas

>> Yeah, that's interesting. I mean, having this open source communities helps a lot because one of the things that we try to do is bring customers as well. Right? We incentivize customers to be part of the community. So from the get-go, we can understand the challenges and the problems they're having, so that helps a lot. But at the same time, we try to involve a lot of research universities, be very connected with researchers, universities, where the people are really doing the innovation. So, when we talk about these open source community, it's a healthy mix of partners of the larger companies, of our engineers, the customers, and as well the researchers. And that gets us a healthy mix where we can understand where the needs are, where the challenges are coming up, are surging, and then how we can close that gap or that bridge between what enterprises have today, because at the end of the day, it needs to work for the enterprise and where innovation is going. So, we're trying to constantly close that gap, so we make sure that people can keep using their investments, but still can adopt technologies like AI, which are new, which for some people can be challenging, it can look risky. Right? So you really want them to adopt the technology fast, but understand how they can mitigate the risk. So, using a combination of all of those items or activities is how we combine everything to make the decisions where the roadmap is going to go next.

Savannah Peterson

>> What a cool experience for the researchers on that side. I feel the integration of academia and research and high-tech are really coming in this super cool Venn diagram right now where we're able to leverage so much that's been happening there, particularly on the HPC side, but really bring it into the fold of these business solutions and solving real world problems, curing cancer, all this cool stuff.

James Harmison

>> Yeah, you see in the VLM project is a great example of that, where Red Hat is the number one commercial contributor to code in the VLM project, but the number one overall contributor is UC Berkeley research students.

Savannah Peterson

>> I love that.

James Harmison

>> So, there's so much cool science happening in this commercial enterprise software space.

Rob Strechay

>> Yeah, and I was going to-

Savannah Peterson

>> That's neat....

Rob Strechay

>> circle back to that because I think that's an important thing where you guys, it's not just the ones that you bring to the community, you're also contributing across. I mean, yeah, you brought KServe, for instance to CNCF, but you also are big with Kubeflow, and you're also in with us, obviously VLLM and LLMD, but you have TrustyAI and you have a number of other things. Help people understand how that also translates back to things like Red Hat AI 3 and things like that, because I think to me, I know, dirty little secret, I just go to your GitHub and I look at what's been posted and I know what's coming. Right? Because, not that I'm not supposed to say that, probably I'm going to get yelled that now by you guys, but anyway, when you start to look at it-

James Harmison

>> That's where I go.

Rob Strechay

>> Anyway, but exactly, it's where most-

Savannah Peterson

>> ....

Rob Strechay

>> Dirty little secret, right?

James Harmison

>> Yeah.

Rob Strechay

>> But again-

Savannah Peterson

>> Well, that's building in public though. I mean that's the deal. Yeah.

Rob Strechay

>> Right. So, help people understand how that's really helping customers now with Red Hat AI 3.

James Harmison

>> Yeah. One of the keys is, we contribute to a lot of projects that never make it into Red Hat products. So, our engineers in general have a lot of freedom to collaborate in the community and make decisions in those communities and explore ideas and try things out. So, you hear a lot about things like we just got KServe into the CNCF, but we've been working on the KServe project for years now, and it was never always clear that KServe belongs in CNCF. So, one of the things that happens at Red Hat is that we work on lots of open source projects, they never make it into a product, they never make it into a CNCF contribution. But by just having people feeling the pulse of what the community's doing out in the wild in so many spaces, you can't help but succeed and have something that's really going to be successful and valuable to our customers. So, our products are this amalgamation of the things that worked and the things that clicked the most, and then life-cycled and supported over the long term like our enterprise customers need. But it has to have that spark out in the community first.

Rob Strechay

>> Right, it's also contributing to the things as part of the ecosystem as well that, like you said, may not be part of it, like model registries and things of that nature, which yes, you have yours, but there's also open source ones that are out there that may or may not be the same one and people can then bring them together with the commercial parts and the open source parts. I mean, to me, that's the power of what you guys do, especially in this AI realm, which is, how do you make it easy? That had to have been one of the focuses of Red Hat AI 3 is, how do we just give people an easy button from that perspective?

Jennifer Vargas

>> Yeah, exactly. Yeah, because at the end of the day, we believe customers need two things. One, the flexibility to mix and match, which is what you're saying. It's something that we carry in every layer of our stack, which is like, this is our opinionated way of the tools that we think you need but if you want to bring your own, we're totally open for you to bring the open source technology that works for you or the framework that works for you. Right? But on the RSI, one of the things that we do is to make sure that we give customers interoperability, that they're able to make these pieces work together, right? And we need them to work together, they need them to make sure that they're integrated because at the end of the day, they need to support these tools in the long term. You need these tools for production environments, for critical applications, right? So you need applications or technologies that work together, that are resilient, that are reliable, and that's the part where we put the spark on our products, as you say. That's when we make them work together and we make them enterprise-grade. They're supported, but you get the transparency, you get the flexibility, and that's the value I think Red Hat AI 3 brings to the picture when it comes to AI. When everyone's looking for an AI that's transparent, that is trustworthy, that is reliable, and not only that, but it also feels that we all belong to it. That's almost like we tend to say, democratize AI. It's pretty much that, it's how we make AI for everyone.

Savannah Peterson

>> It's collaborative at least. Yeah.

Jennifer Vargas

>> Yes.

Rob Strechay

>> Yeah.

Savannah Peterson

>> No, I think that's wonderful, what a fun conversation. All right, I have one final question to close us out. Since you're a regular Jennifer and we love hanging out with you, James is about to be a regular, what do you hope to be able to say at KubeCon in Salt Lake City next year, 2026, that you cannot say yet today?

Jennifer Vargas

>> Wow.

Savannah Peterson

>> Jennifer, I'll start with you.

Jennifer Vargas

>> So, I think one of the things that's been happening with AI is that everyone is trying to look for the ROI, right? It's really expensive to deploy AI at the scale that everyone wants. On the enterprise, enterprises need to demonstrate ROI very quickly. And I think agentic AI is probably are key to that, because it will help enterprises match their workflows. Enterprises already understand automation, how to automate processes. What they need it to take it to the next level. Can we make it autonomous? Can we make it work with AI? And I think that's what agentic AI is going to bring to us. So hopefully next year in Salt Lake City, we are ready to show you guys more and more capabilities around agentic AI for enterprises.

Savannah Peterson

>> Can't wait to see, it Jennifer. What about you, James?

James Harmison

>> Yeah, I think key to that, I want to build on that because I don't want just to be able to deliver agentic AI capabilities to our customers. I want to have customers sharing where they've actually demonstrated value to their organization using those agentic AI capabilities. We need to have some concrete success stories in the space where people can give other people ideas.

Savannah Peterson

>> Yes, please, yeah.

James Harmison

>> And I think we're early days still, and so there's not a ton of people that are willing to go out and talk about exactly how much they've made because all the accounting isn't done yet on everything, but.

Savannah Peterson

>> Well, and if you have an early mover advantage, you're not exactly shouting from the rooftops, "Hey, this is how we just saved $2 million a month on whatever."

James Harmison

>> Exactly. We don't want to spin everyone else's treadmills up yet, so yeah. But for sure, I think that by next year we'll definitely have a lot of people saying, "Oh no, this is real and this is how I was able to deliver it, and these are the capabilities that were key." And those lessons learned are going to go into developing the next generation of new capability.

Savannah Peterson

>> Awesome. Well, we cannot wait to tell those stories, maybe some customer examples here with us on stage or projects that have really taken off. James and Jennifer, thank you both so much.

Jennifer Vargas

>> Thank you for having us.

Savannah Peterson

>> Just a joy, and thank you, Rob.

Rob Strechay

>> Thank you.

Savannah Peterson

>> And thank all of you for tuning into our three days of continuous coverage here from Atlanta, Georgia at KubeCon. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.