Red Hat Summit + AnsibleFest 2025 | Stephen Watt, Red Hat

Clips
More from Red Hat Summit + AnsibleFest 2025

Stephen Watt

Vice President, Distinguished Engineer, Office of the CTO

Red Hat

play_circle_outline Red Hat's belief in AI models and applications as the future

play_circle_outline Role of openness and transparency in AI innovation

play_circle_outline Red Hat's role in open source community and AI innovation

play_circle_outline Productivity and developer support initiatives at Red Hat

play_circle_outline Innovative AI Trends and Projects with Distributed Systems Expertise in Open Source Inferencing

Info
Transcript

Stephen Watt, Red Hat

Stephen Watt

Vice President, Distinguished Engineer, Office of the CTO Red Hat

Stephen Watt Discusses AI Evolution at Red Hat Summit 2025

Stephen Watt, vice president and distinguished engineer, office of the Chief Technology Officer at Red Hat, joins hosts Rob Strechay, director and principal analyst at theCUBE Research, and Rebecca Knight from SiliconANGLE Media at the Red Hat Summit 2025. The discussion explores the transformative role of artificial intelligence models and applications in enterprise environments.

During the session, Watt elaborates on Red Hat's focus on the future of AI, highlighting the importance ... Read more

explore Keep Exploring

What areas of applications are being explored with large language models in the context of AI technology development? add

What are your thoughts on the future of AI in relation to open source and transparency? add

What analogy is often used to describe the rate of innovation happening in AI compared to other technology developments? add

What are some ways in which your company plans to streamline the deployment and use of different software personas and models within the AI stack? add

What aspect of Red Hat's background and expertise aligns well with the transition to AI and distributed inferencing? add

bolt Powered by CUBE AI

Stephen Watt, Red Hat

search

Rebecca Knight

>> Welcome back everyone to theCube's live coverage of Red Hat Summit AnsibleFest here in beautiful Boston, Massachusetts. I'm your host, Rebecca Knight, along with Rob Strechay, my co-host and analyst. Rob, the room is filling up. The conference is really just getting started.

Rob Strechay

>> Yeah, this is the day zero that's just about to explode. We have some great stuff going on tonight. Keynotes galore tomorrow morning and Wednesday morning. It's going to be a lot of fun still to come.

Rebecca Knight

>> Lots of great announcements tomorrow-

Rob Strechay

>> Tomorrow....

Rebecca Knight

>> but right now, I would like to welcome our next guest. He is Stephen Watt, Vice President, Distinguished Engineer, Office of the CTO at Red Hat. Thank you so much for coming on theCube, Stephen.

Stephen Watt

>> Glad to be here.

Rebecca Knight

>> I should say returning to theCube, you are a longtime Cube alum. Exactly.

Rob Strechay

>> Yeah. Back to almost, if not the first one, one of the first ones that we had.

Stephen Watt

>> Yes, absolutely.

Rebecca Knight

>> So let's start out with just some broad brushstrokes about how Red Hat believes AI models and applications really are the future of it.

Stephen Watt

>> Yeah, well, I think it all starts with large language models. I think we had this sort of era of predictive AI and now with generative AI, I think there's a whole lot of really new applications, and I think there is interesting use cases in three different areas, right? Training, fine-tuning, and inference. Last year, we announced InstructLab, which was a democratizing fine-tuning models. And with our Neural Magic acquisition, we've got a lot more into inference and that's about serving models and creating value for applications in the enterprise.

Rob Strechay

>> Yeah, I mean we even talked about, because people want to broaden those applications to be more agentic, which is not just, hey, the gen AI chatbot that everybody knows and loves to go and use, but it really talks about being very specific to a task and helping with a task. How do you see what things evolving? Because you brought up Neural Magic and vLLM, there's a lot of open-source technology that goes into a lot of these pieces. How do you guys... they're putting it together and saying, "Here's how it all aligns"?

Stephen Watt

>> Yeah, I would say it's all about context, and so there's retrieval augmentation, RAG, and then RAFT, which is applying RAG with fine-tuning. We've got a emerging story around that with the upstream Llama Stack project, where we've just done a lot of work upstream to enable all of that. The second piece of context around that is enabling services and your large language models to be able to call out to those services through the model context protocol. So we've done a lot of work around that also in Llama Stack. And so when you put those together, you're able to basically augment what the large language model knows through perhaps documents that you have, as well as services that you can integrate into those. I'm quite excited about that. I think that's going to add a lot of new capabilities that we can leverage through vLLM.

Rebecca Knight

>> So how can companies prepare their applications to work for both traditional users and the AI agents?

Stephen Watt

>> Well, I think there is two specific areas going back into context again. One is vector databases. So these are a new addition to our standard enterprise data management stories, but it's a way of, you take those ETL pipelines and you chunk all your documents back into those vector databases. So that's the first step. Once you do that, you're able to basically take what you institutionally know within your organization, add it into something that's accessible from the large language model. The second thing that's really interesting is sort of like the evolution of service-oriented architectures. So we took all of our applications, decomposed those into services, but they were historically distinct from the large language model, which was always created at a point in time. So to be able to take those and those capabilities that you've built out over time and hook those into the large language model, I think those two things together are really exciting.

Rob Strechay

>> I think what's fun, being here and we get to go to a lot of different places, like we were at KubeCon, CloudNativeCon over in London and back in Salt Lake and seeing how things are progressing, it would seem that AI and the future of AI is open. How do you see it? Because I think again, you guys are at the heart of open source, you contribute to so many different projects upstream. How do you see it?

Stephen Watt

>> I think that's a great observation. It is open. I think with open things, you get transparency and agency, which really tends to accelerate innovation. It's interesting you mentioned Kubernetes, CloudNative Computing Foundation. I would say the evolution of that as well is sort of the PyTorch foundation and the ecosystem that's growing around that, there's a nucleus around that. The way Kubernetes is in the CNCF, PyTorch, we've got vLLM inside that foundation now, and it's growing out from there. So I think that's a key community about how we reason about where open work is happening. The next I would say is on AI and open source is quite interesting around definitions of open with AI. They vary quite a lot. I would say with us, our minimum definition of open is open model weights plus an open ecosystem software stack. We would like to see that evolve into an open pipeline and open training sets as well, and we would like to see those aforementioned communities and foundations spread out into those spaces as well.

Rebecca Knight

>> I would love you to riff a little bit on why you see openness and transparency as so critical in AI development.

Stephen Watt

>> Well, I think a number of different vectors. One is sunlight is the best disinfectant. So for security, it's pretty helpful. It's useful from an indemnification standpoint. So if I'm building my business around a model, which there's a whole new class of applications and businesses that are being derived from generative AI, you want to make sure that you're not going to get into legal issues based on how that model was trained. And so one, you want some level of comfort around the assets that you're using around that. So there's security, there's indemnification, and then there's also a rate of acceleration around innovation. So when people are publishing, like say the DeepSeek paper, where prior to that you had what I would broadly classify as brute force training of generative models where we just throw more compute and throw more data at it. And when people publish new techniques where in that paper people could see, oh, well if we take a bit more care at every phase, we can actually produce much greater yields in performance. And so there's a steel-sharpened steel effect with the community around that, which I think is important as well.

Rob Strechay

>> I think one of the things that people always look towards Red Hat is open source. You have a unique vantage point within the organization. Where do you see open source now that Red Hat's at the stage of life that it is? Linux is still out there going strong. There's a lot going on with that, and we'll be talking about the rest of this week. And there's a lot going on within Kubernetes, like you were talking about. And then you have these different levels and different organizations, like you talked about PyTorch. How do you see, and what do you think Red Hat's role is going forward in open source?

Stephen Watt

>> With so much innovation happening in AI, the internal analogy we use a lot when we talk about it is the speed of the treadmill. So if you look at RHEL and the Linux kernel, maybe the treadmill was running at two or three out of 10. From an innovation standpoint, Kubernetes came along with OpenShift, five out of 10. With AI, it feels like it's running at 11 out of 10. The rate of innovation, new project, the creative destruction that's happening, our role is to basically create a steady pipeline that businesses can use to consume software where it's stabilized and safe. They can actually continue to evolve and build their businesses on a platform that's stable and evolving incrementally. So we play a sort of a cherry-picker role upstream where we spend a lot of time figuring out which foundations to invest in, which open source projects to invest in. And we look at those across a number of different community metrics. We have a pretty sophisticated open source program office that is able to really think very thoughtfully about communities and foundations and which ones we invest in and which ones we don't. I think that can't be understated because of the, just massive amounts of innovation that's happening in AI right now. I would say it's probably the biggest thing.

Rebecca Knight

>> No, Stephen, I actually want to ask about that because you are in the Office of the CTO and a lot of what we're talking about here, which is the hard tech, but a lot of it is also about change management and it's about culture and creating this culture of transparency and openness and also innovation. So when you are talking to organizations about how they use AI and open source, what are the messages that you recommend senior leaders push down to the organization and as well as the end bandolist within the organization? What is your best advice there?

Stephen Watt

>> The first is get started. This isn't a fad. The second is start with a problem that you know have a customer for, an internal customer for. In my own organization, we do a lot of experimentation, and the thing that separates a successful project and an internal project is adoption internally around applied AI. We've had a number of different projects where we're like, "Look, we can do this amazing thing using large language models," but there's no one willing to actually pick it up and use it. When we actually start, it seems obvious, but when we start with a pain point, like a CVE comes out, we write a patch for a CVE, well, we've got all these different streams of Red Hat Enterprise Linux that need to get patched, that's a nightmare. Could you figure out how to retroactively take this patch and apply it across all of those and solve a major problem for a bunch of engineers? That has a real consumer internally within it. And those things, when we build them, they get picked up and deployed right away. So it's get started and then start with the problem.

Rob Strechay

>> So I think just leveraging off of that, because I think one of the interesting things that is going on is that it's AI everywhere to a certain extent. And when I think of compute and infrastructure everywhere, one of the things I think about is Red Hat because there's just so many different deployment models from the far edge to the near edge to the data center up to the cloud. How do you see that? Because that's so many moving parts, especially coming from the CTO's office, how do you really, for lack of better, how do you grok that to understand, okay, these are the pieces that really need to flow together?

Stephen Watt

>> Yeah, I think it's an interesting question. There's two pieces to that. I would say there's the data center, the mothership, and then there's the far edge. Even in Far Edge, it's segmented. There's different types of far edge. There's the device edge, there's large single-node servers. Large language models have a specific resource budget that they need. So you do have to think quite thoughtfully about what you can deploy to the edge versus what needs to live at the mothership, in the data center. We have Red Hat Edge Manager that allows us to do fleet control of applications, the inference and models out to the far edge Advanced Cluster Manager vLLM that allows you to deploy inside the data center. But essentially we're looking to build a single sort of control plane that allows you to figure out what you want to deploy where, that's appropriate for the resource footprint where you're trying to deploy it.

Rob Strechay

>> Yeah, we see that people are bringing AI to the data versus the data to AI because they don't want to move petabytes of data or hey, maybe it's a latency question, is that some of the things that you're hearing, is the customers are coming to you and saying, "Hey, I understand that this is not a perfect fit, but I'm doing this because it's part of... I mean, you have multiple car companies who use OpenShift and Linux in the cars and stuff like that. We were talking about that at MWC and on the telco space and CES in the auto space. How do you see customers making those trade-offs?

Stephen Watt

>> It's interesting you mentioned that. In some sense, technology is a bit like fashion. Moving the compute to the data was what we were doing with Hadoop and big data back in the 2000s, and then it's sort of as cloud, the advent of cloud, it sort of went the opposite direction and now it's coming back again. I think it depends what you're trying to do. So if you're doing pre-training or fine-tuning, you really do need to do the computation and the pipeline closest to where the data is because it's just not feasible to move that much data around. When you get to the far edge, it's trickier, you need a smaller model. If you're trying to provide context to the model, maybe you can do that through APIs calling externally, but if you're trying to do connected to a vector database, it depends what footprint it's running on. So in some sense, we're back to very sophisticated data pipelines.

Rebecca Knight

>> So finally, Red Hat has an audacious goal of achieving net-zero emissions by 2030. I want to ask you how close you are to that and what you're doing to work towards your sustainability goals?

Stephen Watt

>> That's a great question. So we're actually doing quite a lot. There's two different ways to look at sustainability. One is the green perspective. The second is infrastructure is incredibly expensive. They're actually a shared incentive of how can we run things so that they consume less power. And so we have a project called Kepler, which is instrumenting the service from the bare metal all the way up to the top of the stack. And basically from that, you can extrapolate how to do scheduling of applications so that you can create policies to really drive the power consumption down based on your time to solution needs. I would say that as a general approach is a pattern that's sort of used across all of our different sustainability objectives.

Rob Strechay

>> Yeah. Again, we've talked about Kepler. I think about a year and a half ago we had some of the folks on from Red Hat at the Open Source Software Summit in Vancouver. And I think part of that was really about also, it's not just sustainability for sustainability's sake, and I think that was part of it. You can help to schedule to different power grids and run in different data centers, but it also can be tied to KPIs around what's needed and things like that. There's also a lot of focus because I would say you're going to have not only a platform engineers here, you're going to have SREs here, you have DevOps here, you have developers here this week. How do you see productivity and where Red Hat's helping? Because you have Developer Hub, for instance, which is based off of upstream of backstage and things of that nature. From the CTO's office, is that a major focus where you're sitting here, because Kubernetes wasn't easy to begin with and now it's gotten easier, but there's probably further we can go, and I know you're doing that with OpenShift and things of that nature.

Stephen Watt

>> Yeah, I think for any of our software, we'll build golden templates, deployment patterns where sort of with the push of a button, you can roll out the whole stack. So certainly your inference stack, OpenShift AI, what we're able to do is look at different personas that use different software and basically create golden templates around... create a library of depending on which persona, what your company allows you to deploy. You can kind of pick those things, deploy those, and then as part of that, you can also pick what models you want to serve. An interesting bit of new research that we're doing in the spaces. There's a horses for courses type thing when it comes to models, some models are better at scientific things. Some models are better at history, some models are better at literature, et cetera. When we're making intelligent decisions around which models to use for what, we've actually got an exploratory project right now, a semantic router that actually is able to, depending on what you're asking it, figure out where to route the request to and then figure out how to serve that thing to you. That's kind of a fun emerging technologies project we've got going on right now.

Rebecca Knight

>> Well, I was just going to ask you about what are the most exciting things that you're seeing? I mean, and there's obviously that, emerging technology project. Are there any other trends or without giving too much away happening tomorrow that you see on the horizon?

Stephen Watt

>> I think the thing I'm really most excited about that really works well with Red Hats background and expertise, which is, if you look at our story, right with RHEL scale up, and then the world went scale out with OpenShift during that decade that we were building Kubernetes and OpenShift, we became a distributed systems powerhouse. And then you have this sort of third chapter around AI. And what's really great about inferencing and distributed inferencing is that we're able to take all that muscle memory we built around distributed systems and inject that into inferencing. And I am really excited around where we're going over there. Stay tuned.

Rebecca Knight

>> Excellent. Well, great.

Rob Strechay

>> It was great.

Rebecca Knight

>> Great note to end on. Steve, thank you so much for coming back on theCube.

Stephen Watt

>> My pleasure.

Rebecca Knight

>> Always a pleasure having you.

Rob Strechay

>> Yep.

Stephen Watt

>> Yep. Thank you.

Rebecca Knight

>> I'm Rebecca Knight for Rob Strechay, tuned for more of theCube's live coverage of Red Hat Summit AnsibleFest, 2025. You're watching theCube. Be a leader in enterprise tech news and analysis.