Red Hat Summit 2024 | Sam Heywood | Run:ai

Clips
News
More from Red Hat Summit 2024

Sam Heywood

Vice President Product Marketing

Run:ai

Run:ai integrates into OpenShift for efficient AI resource management

As artificial intelligence induces the same infrastructure sprawl previously seen with cloud technologies, AI resource management has emerged as a crucial challenge for enterprises.Organizations are now inundated with vast sets of static and in-motion data resources, raising the question: what solutions can effectively manage AI resources and infrastructure contained within OpenShift, while supporting companies on their transformation journeys? By integrating with OpenShift, Run:ai provides a robust solution, enabling seamless management of AI resources and infrastructure.Run:ai’s Sam Heywood discusses the integration with OpenShift AI for operational seamlessness

play_circle_outline Run:ai and Red Hat Collaborate for Efficient AI Infrastructure Management and Teamwork Advocacy

Info

Sam Heywood | Run:ai

Sam Heywood

Vice President Product Marketing Run:ai

search

>> Hello and welcome back to Red Hat Summit and AnsibleFest here in Denver. We're really excited to have you here watching theCUBE, where we're going to really dig in and unpack what's going on in the ecosystem this afternoon with Red Hat and how that is really impacting how their customer's journey to AI. I'm joined by Paul Gillen. I'm Rob Strechay. Rebecca Knight is floating around here. We got just a wall-to-wall coverage coming at you all afternoon. So today we're going to really just dive into, and who better to have on here, but Run:ai? Run:ai, been in the news. We're not going to get into that stuff. But really, when you start to look at it, we got Sam Hayward who leads up marketing for Run:ai, and we're going to really delve into, because you guys had an announcement. So welcome on board, Sam.

Sam Heywood

>> Oh, thank you, Rob. Really pleased to be here.

>> So why don't you take us through the announcement that you had with Red Hat this week?

Sam Heywood

>> Sure. So Run:ai, AI infrastructure management, GPU orchestration. We help organizations accelerate their AI workloads. We've had long and deep partnerships with Nvidia, Red Hat, several others, and just last week, we announced acquisition by Nvidia.

>> Right. And that's still pending in all of that fun jazz.

Sam Heywood

>> Yeah, there's always some regulatory that you got to clear, but yes.

>> So when you look at a partner like Red Hat, how do you work together? How do you meet in the field? How do people work with Red Hat and work with Run?

Sam Heywood

>> Yeah, so I think taking one step back, and then I'll get to that. If you think about everything that's going on in AI, it's a team sport. Right? What we're trying to do, it's big and it's going to require the whole industry to help make that transition. With Red Hat in particular, really strong opportunities to partner because the Run:ai solution we run on top of Kubernetes and OpenShift Kubernetes is widely adopted within enterprise. So we leverage that. And then if you think about the development and the MLOps and all those types of things, OpenShift AI, we just announced a new collaboration with Red Hat on that. On the top, you've got OpenShift AI, you've got Run:ai doing all the GPU orchestration on top of OpenShift below. It's a match made in heaven from a product perspective. We're all targeting the same accounts, and we just work together to help move the customer forward, which is one of the things that's so exciting about the Nvidia acquisition. Nvidia's saying is the customer is the mission. That's the same view we take. So we meet in the middle, try to find opportunities in the field to solve customer problems and help drive this whole AI thing forward.

Paul Gillin

>> So what will OpenShift customers gain from this collaboration that they wouldn't get from simply buying Run:ai off the shelf?

Sam Heywood

>> So what would OpenShift customers gain by their collaboration with Red Hat?

Paul Gillin

>> Yeah.

Sam Heywood

>> Each party brings a lot of different perspective and expertise to it. So when you start talking about AI within the enterprise, at the end of the day, the products and the technology need to deliver the promised capabilities. But what I'm seeing is enterprises also looking for deep expertise. And so sure, you could go buy Run:ai. Sure, you could go OpenShift, but bringing the two teams together and providing the opportunity for joint solutioning around the customer problem, that's a big value add.

>> So there's more integration at that level because I think one of the things that Run:ai brings that people are always talking about, especially with sustainability in mind, is fractional GPUs-

Sam Heywood

>> Sure.

>> And being able to really bring that to OpenShift. Is that some of where these integrations and how the working together is helping?

Sam Heywood

>> Yeah, for sure. And I'll expand that a little bit for the audience. Run:ai, the people we work with, they typically show up with these two diametrically opposed problems. On the one hand, they have GPU infrastructure, and when they put their scope on it, they see the utilization is really low. Right? On the other hand, when they talk to their data scientists and their researchers, what they hear is they can't quite get access to the resources they need. And so what that speaks to is the difficulty and the complexity of scheduling and provisioning compute to support AI workloads. So what's one of the things that contributes to that? Massive over-provisioning. So let's say you have two, four, eight, 16, 32, whatever, a whole bunch of data scientists, and they want interactive environments to support what they're doing in OpenShift AI. You could give each of them an entire GPU, but that's a little bit overkill. Let's say for their development work, they need about four gigs. You've got a bunch of H-100s with 80 gigs of memory. What we can do is take that one physical GPU and divide it into up to 20 logical GPUs, and then give each of those logical GPUs to the developer.

Paul Gillin

>> I imagine the cost savings would be massive on that. In addition to savings on cost, what other payoffs do customers get from that fractional GPU allocation?

Sam Heywood

>> Sure. So the number one thing is acceleration of their critical AI initiatives. Most organizations, they're dealing with AI everywhere all at once. Right? It's driving business strategy changes, organizational changes and infrastructure changes. There's five key challenges in AI infrastructure. We won't go through them all, but the combined solution helps accelerate those critical AI initiatives, which is the whole goal. We're not there to help them save money. We're not there to actually help them drive higher utilization. What we're there to do is ensure that they have proper support for the critical initiatives and that they're confident that the way that infrastructure is being used maps back to their most highest priorities.

Paul Gillin

>> You work both in the cloud and on-prem, is that right?

Sam Heywood

>> That's exactly right. So the easiest way to think about it is Run:ai is a bring your own GPU, bring your own Kubernetes solution. And if those GPUs are long-term leases in AWS or a DGX BasePOD or a DGX SuperPOD and a data center, we span all of that. And that's part of what makes the OpenShift collaboration so fantastic is they also bring that hybrid approach.

>> Is there certain use cases that you see organizations really embracing Run:ai? Is it sustainability? Is it like Paul was getting at, cost savings? What are the main drivers where... You brought up the one with the data scientists.

Sam Heywood

>> Right.

>> What are some of the other use cases that people, they think and they go, "Hey, yeah, this would be great. There's a certified operator with OpenShift AI. Let me go down this road?"

Sam Heywood

>> So the main thing that we're seeing in terms of use cases, it's not industry-specific or anything like that, which is not what you're suggesting, but it's how do we take this GPU infrastructure and provide it as a broad service back to the business, that makes it accessible to most of the business and supports the entire AI lifecycle, including interactive environments for development, including training, small training, massive distributed training in support of generative AI, and including inference, right? We support that entire AI lifecycle, and what we see is enterprises wanting to use their GPU infrastructure to deliver all that, as opposed to having some infrastructure for dev, some infrastructure for training and some infrastructure for inference.

Paul Gillin

>> Do you see training moving toward the cloud in a big way, or perhaps even the other way? There's been a lot of discussion about whether AI training will migrate more resources back on premises. What do you see?

Sam Heywood

>> It's a fantastic question. I think it's too early to call what the trend will be, but what I'll do is I'll... I had the privilege of selling a company too and spending a long time with Cloudera and got to spend a lot of great time with Mike Olson. And one of the things Mike Olson says is, "Data has gravity." Right? Training is very data-intensive. So I think that data gravity is going to have a significant influence on where the training workloads take place.

>> So when you look at this and you see... Again, Run:ai has been in the market for quite some time, and again, you've been around this market for quite a while. When you look at how people are embracing AI, do you see it not only for the generative AI, but you see it for what we're now calling, I guess traditional or heritage AI use cases as well?

Sam Heywood

>> So yes. I think the interesting thing, Nvidia's CEO, Jensen Huang, he refers to generative AI is the ChatGPT moment. What's happened with generative AI is it's driven AI into the kind of headspace, if you will, of everybody. But the fact is machine learning has been around for quite a long time, right? Financial services were early adopters, obviously a lot of defense use cases, a lot of consumer use cases for targeting and online and things like that. So what's happening today with the generative AI is really a foundational transformation. It really is, but it's built on all of the work that's gone before it in terms of more traditional ML, right? Machine learning is not new. It's been out there for a long time. It's just what's happened in this last phase is pretty amazing.

Paul Gillin

>> What's involved in GPU fractioning? It seems like you'd have to get pretty low level into the hardware to provide that capability. Do you do that?

Sam Heywood

>> Sure. So I'll answer at the level I can understand it. What's involved in GPU fractioning really terms of what we're trying to get done is to maximize usage of that GPU. How do we do it? Nvidia GPU operator is required. Beyond that, in terms of explaining all the technology, I'd love to have a follow-up with you at the bar.

Paul Gillin

>> But you do more than just that. You also do workload orchestration. You do actually some Kubernetes type functions as they relate to ML workloads specifically.

Sam Heywood

>> Yeah. So if you go back again to those two diametrically opposed problems, we've got less utilization on the GPUs than we thought, and we have teams saying they can't get access to GPUs. That is problem number one. And we solved that really well, and we solved it, I think, in a unique value driving way. But we've wrapped an entire platform around that. So if you think about the five main challenges when you're talking about AI everywhere, all at once and dealing with the infrastructure, the first thing is, again, how do you support those data scientists and make sure they have always on access for interactive environments to do the development? The second thing is most organizations are dealing with a little bit of GPU sprawl. Right? They've got stuff in AWS, they have stuff in Lambda. Right? They're running Cloud DGX. They've got stuff in and a data center. How do you bring all that together under a single pane of glass? Once you have that, how can you ensure that those resources are being used in a way that aligns to your critical business priorities? So we have an enormously powerful policy engine to drive all that. And then once you have all that in place, you think about the volume, variety of velocity of AI workloads, getting those scheduled and ensuring right fit compute, that real critical problem. We provide all of that.

>> So when I look at it, and we've had a lot of partners on, you're I think the first one we've had on this week that is... Assuming the acquisitions, you will not be so small anymore, but pending, but you're obviously smaller than Red Hat. How has Red Hat been to work with?

Sam Heywood

>> Oh, my gosh. It's pretty amazing. I don't mean to get all giddy about it, but coming into this and preparing for the OpenShift AI announcement, it's been literally fantastic. Right? Again, we're a smaller company. We have to focus on mission-critical stuff, so some of the details can slip a little bit. So some of the things that needed a little bit of attention, the Red Hat team was absolutely fantastic at circling back and saying, "We need to get this done. Let us know how we can help." It's been amazing.

Paul Gillin

>> What form do you believe the integration will ultimately take? Will we see Run:ai functionality integrated directly into OpenShift, or do you expect to remain a separate SKU?

Sam Heywood

>> So here's what I would say. It's a team sport. If you think about all of the technologies and all the vendors that are required to pull this off, it's a team sport. We're all still trying to figure out where the puck's going. The mission is make the customer successful and do what's needed to get there. As we continue to learn and grow, those answers about what the integration looks like in the future of it all will become more clear.

>> Yeah, I think what's interesting is that you guys are right at the nexus, and I know you were at KubeCon and think there's a lot going on in the open area of this as well, and that you've actually contributed back into that. Do you see that as also a place that you and Red Hat actually come together as well?

Sam Heywood

>> Yeah. The open source community is so vital to everything that's going on. If you think about all the innovation through the years that has been driven, including Linux itself, open source is absolutely critical and open source in terms of how do you participate in the open source communities, again, to drive mission for the customer is absolutely something we think about all the time?

Paul Gillin

>> So the model training world isn't just about GPUs. There are about a dozen companies that are working on hardware that's specific to training, inferencing edge applications. Are you working with those companies as well?

Sam Heywood

>> So we take a broad view in terms of the technology that we support. I can't get into roadmap right now, but we definitely are wanting to support as much as the ecosystem at the silicon level as we can.

Paul Gillin

>> So it's not going to be Nvidia only?

Sam Heywood

>> I can't get into the roadmap right now.

Paul Gillin

>> Got it.

>> Well, Sam, we really appreciate you being on board. I think, again, we look forward to seeing what colors you're wearing next time we have you on board, and maybe a follow-up with you guys to get a little deeper into how all the fractioning happens and how that works and how the operator works. But really appreciate you coming on board here on theCUBE. Thank you.

Sam Heywood

>> Yeah, no, thank you both. It's been my pleasure.

>> And thank you for joining us for Red Hat Summit AnsibleFest, day two here on theCUBE, the leader in tech analysis and news. Stay tuned. We'll be right back.

Sam Heywood | Run:ai

search

>> Hello and welcome back to Red Hat Summit and AnsibleFest here in Denver. We're really excited to have you here watching theCUBE, where we're going to really dig in and unpack what's going on in the ecosystem this afternoon with Red Hat and how that is really impacting how their customer's journey to AI. I'm joined by Paul Gillen. I'm Rob Strechay. Rebecca Knight is floating around here. We got just a wall-to-wall coverage coming at you all afternoon. So today we're going to really just dive into, and who better to have on here, but Run:ai? Run:ai, been in the news. We're not going to get into that stuff. But really, when you start to look at it, we got Sam Hayward who leads up marketing for Run:ai, and we're going to really delve into, because you guys had an announcement. So welcome on board, Sam.

Sam Heywood

>> Oh, thank you, Rob. Really pleased to be here.

>> So why don't you take us through the announcement that you had with Red Hat this week?

Sam Heywood

>> Sure. So Run:ai, AI infrastructure management, GPU orchestration. We help organizations accelerate their AI workloads. We've had long and deep partnerships with Nvidia, Red Hat, several others, and just last week, we announced acquisition by Nvidia.

>> Right. And that's still pending in all of that fun jazz.

Sam Heywood

>> Yeah, there's always some regulatory that you got to clear, but yes.

>> So when you look at a partner like Red Hat, how do you work together? How do you meet in the field? How do people work with Red Hat and work with Run?

Sam Heywood

>> Yeah, so I think taking one step back, and then I'll get to that. If you think about everything that's going on in AI, it's a team sport. Right? What we're trying to do, it's big and it's going to require the whole industry to help make that transition. With Red Hat in particular, really strong opportunities to partner because the Run:ai solution we run on top of Kubernetes and OpenShift Kubernetes is widely adopted within enterprise. So we leverage that. And then if you think about the development and the MLOps and all those types of things, OpenShift AI, we just announced a new collaboration with Red Hat on that. On the top, you've got OpenShift AI, you've got Run:ai doing all the GPU orchestration on top of OpenShift below. It's a match made in heaven from a product perspective. We're all targeting the same accounts, and we just work together to help move the customer forward, which is one of the things that's so exciting about the Nvidia acquisition. Nvidia's saying is the customer is the mission. That's the same view we take. So we meet in the middle, try to find opportunities in the field to solve customer problems and help drive this whole AI thing forward.

Paul Gillin

>> So what will OpenShift customers gain from this collaboration that they wouldn't get from simply buying Run:ai off the shelf?

Sam Heywood

>> So what would OpenShift customers gain by their collaboration with Red Hat?

Paul Gillin

>> Yeah.

Sam Heywood

>> Each party brings a lot of different perspective and expertise to it. So when you start talking about AI within the enterprise, at the end of the day, the products and the technology need to deliver the promised capabilities. But what I'm seeing is enterprises also looking for deep expertise. And so sure, you could go buy Run:ai. Sure, you could go OpenShift, but bringing the two teams together and providing the opportunity for joint solutioning around the customer problem, that's a big value add.

>> So there's more integration at that level because I think one of the things that Run:ai brings that people are always talking about, especially with sustainability in mind, is fractional GPUs-

Sam Heywood

>> Sure.

>> And being able to really bring that to OpenShift. Is that some of where these integrations and how the working together is helping?

Sam Heywood

>> Yeah, for sure. And I'll expand that a little bit for the audience. Run:ai, the people we work with, they typically show up with these two diametrically opposed problems. On the one hand, they have GPU infrastructure, and when they put their scope on it, they see the utilization is really low. Right? On the other hand, when they talk to their data scientists and their researchers, what they hear is they can't quite get access to the resources they need. And so what that speaks to is the difficulty and the complexity of scheduling and provisioning compute to support AI workloads. So what's one of the things that contributes to that? Massive over-provisioning. So let's say you have two, four, eight, 16, 32, whatever, a whole bunch of data scientists, and they want interactive environments to support what they're doing in OpenShift AI. You could give each of them an entire GPU, but that's a little bit overkill. Let's say for their development work, they need about four gigs. You've got a bunch of H-100s with 80 gigs of memory. What we can do is take that one physical GPU and divide it into up to 20 logical GPUs, and then give each of those logical GPUs to the developer.

Paul Gillin

>> I imagine the cost savings would be massive on that. In addition to savings on cost, what other payoffs do customers get from that fractional GPU allocation?

Sam Heywood

>> Sure. So the number one thing is acceleration of their critical AI initiatives. Most organizations, they're dealing with AI everywhere all at once. Right? It's driving business strategy changes, organizational changes and infrastructure changes. There's five key challenges in AI infrastructure. We won't go through them all, but the combined solution helps accelerate those critical AI initiatives, which is the whole goal. We're not there to help them save money. We're not there to actually help them drive higher utilization. What we're there to do is ensure that they have proper support for the critical initiatives and that they're confident that the way that infrastructure is being used maps back to their most highest priorities.

Paul Gillin

>> You work both in the cloud and on-prem, is that right?

Sam Heywood

>> That's exactly right. So the easiest way to think about it is Run:ai is a bring your own GPU, bring your own Kubernetes solution. And if those GPUs are long-term leases in AWS or a DGX BasePOD or a DGX SuperPOD and a data center, we span all of that. And that's part of what makes the OpenShift collaboration so fantastic is they also bring that hybrid approach.

>> Is there certain use cases that you see organizations really embracing Run:ai? Is it sustainability? Is it like Paul was getting at, cost savings? What are the main drivers where... You brought up the one with the data scientists.

Sam Heywood

>> Right.

>> What are some of the other use cases that people, they think and they go, "Hey, yeah, this would be great. There's a certified operator with OpenShift AI. Let me go down this road?"

Sam Heywood

>> So the main thing that we're seeing in terms of use cases, it's not industry-specific or anything like that, which is not what you're suggesting, but it's how do we take this GPU infrastructure and provide it as a broad service back to the business, that makes it accessible to most of the business and supports the entire AI lifecycle, including interactive environments for development, including training, small training, massive distributed training in support of generative AI, and including inference, right? We support that entire AI lifecycle, and what we see is enterprises wanting to use their GPU infrastructure to deliver all that, as opposed to having some infrastructure for dev, some infrastructure for training and some infrastructure for inference.

Paul Gillin

>> Do you see training moving toward the cloud in a big way, or perhaps even the other way? There's been a lot of discussion about whether AI training will migrate more resources back on premises. What do you see?

Sam Heywood

>> It's a fantastic question. I think it's too early to call what the trend will be, but what I'll do is I'll... I had the privilege of selling a company too and spending a long time with Cloudera and got to spend a lot of great time with Mike Olson. And one of the things Mike Olson says is, "Data has gravity." Right? Training is very data-intensive. So I think that data gravity is going to have a significant influence on where the training workloads take place.

>> So when you look at this and you see... Again, Run:ai has been in the market for quite some time, and again, you've been around this market for quite a while. When you look at how people are embracing AI, do you see it not only for the generative AI, but you see it for what we're now calling, I guess traditional or heritage AI use cases as well?

Sam Heywood

>> So yes. I think the interesting thing, Nvidia's CEO, Jensen Huang, he refers to generative AI is the ChatGPT moment. What's happened with generative AI is it's driven AI into the kind of headspace, if you will, of everybody. But the fact is machine learning has been around for quite a long time, right? Financial services were early adopters, obviously a lot of defense use cases, a lot of consumer use cases for targeting and online and things like that. So what's happening today with the generative AI is really a foundational transformation. It really is, but it's built on all of the work that's gone before it in terms of more traditional ML, right? Machine learning is not new. It's been out there for a long time. It's just what's happened in this last phase is pretty amazing.

Paul Gillin

>> What's involved in GPU fractioning? It seems like you'd have to get pretty low level into the hardware to provide that capability. Do you do that?

Sam Heywood

>> Sure. So I'll answer at the level I can understand it. What's involved in GPU fractioning really terms of what we're trying to get done is to maximize usage of that GPU. How do we do it? Nvidia GPU operator is required. Beyond that, in terms of explaining all the technology, I'd love to have a follow-up with you at the bar.

Paul Gillin

>> But you do more than just that. You also do workload orchestration. You do actually some Kubernetes type functions as they relate to ML workloads specifically.

Sam Heywood

>> Yeah. So if you go back again to those two diametrically opposed problems, we've got less utilization on the GPUs than we thought, and we have teams saying they can't get access to GPUs. That is problem number one. And we solved that really well, and we solved it, I think, in a unique value driving way. But we've wrapped an entire platform around that. So if you think about the five main challenges when you're talking about AI everywhere, all at once and dealing with the infrastructure, the first thing is, again, how do you support those data scientists and make sure they have always on access for interactive environments to do the development? The second thing is most organizations are dealing with a little bit of GPU sprawl. Right? They've got stuff in AWS, they have stuff in Lambda. Right? They're running Cloud DGX. They've got stuff in and a data center. How do you bring all that together under a single pane of glass? Once you have that, how can you ensure that those resources are being used in a way that aligns to your critical business priorities? So we have an enormously powerful policy engine to drive all that. And then once you have all that in place, you think about the volume, variety of velocity of AI workloads, getting those scheduled and ensuring right fit compute, that real critical problem. We provide all of that.

>> So when I look at it, and we've had a lot of partners on, you're I think the first one we've had on this week that is... Assuming the acquisitions, you will not be so small anymore, but pending, but you're obviously smaller than Red Hat. How has Red Hat been to work with?

Sam Heywood

>> Oh, my gosh. It's pretty amazing. I don't mean to get all giddy about it, but coming into this and preparing for the OpenShift AI announcement, it's been literally fantastic. Right? Again, we're a smaller company. We have to focus on mission-critical stuff, so some of the details can slip a little bit. So some of the things that needed a little bit of attention, the Red Hat team was absolutely fantastic at circling back and saying, "We need to get this done. Let us know how we can help." It's been amazing.

Paul Gillin

>> What form do you believe the integration will ultimately take? Will we see Run:ai functionality integrated directly into OpenShift, or do you expect to remain a separate SKU?

Sam Heywood

>> So here's what I would say. It's a team sport. If you think about all of the technologies and all the vendors that are required to pull this off, it's a team sport. We're all still trying to figure out where the puck's going. The mission is make the customer successful and do what's needed to get there. As we continue to learn and grow, those answers about what the integration looks like in the future of it all will become more clear.

>> Yeah, I think what's interesting is that you guys are right at the nexus, and I know you were at KubeCon and think there's a lot going on in the open area of this as well, and that you've actually contributed back into that. Do you see that as also a place that you and Red Hat actually come together as well?

Sam Heywood

>> Yeah. The open source community is so vital to everything that's going on. If you think about all the innovation through the years that has been driven, including Linux itself, open source is absolutely critical and open source in terms of how do you participate in the open source communities, again, to drive mission for the customer is absolutely something we think about all the time?

Paul Gillin

>> So the model training world isn't just about GPUs. There are about a dozen companies that are working on hardware that's specific to training, inferencing edge applications. Are you working with those companies as well?

Sam Heywood

>> So we take a broad view in terms of the technology that we support. I can't get into roadmap right now, but we definitely are wanting to support as much as the ecosystem at the silicon level as we can.

Paul Gillin

>> So it's not going to be Nvidia only?

Sam Heywood

>> I can't get into the roadmap right now.

Paul Gillin

>> Got it.

>> Well, Sam, we really appreciate you being on board. I think, again, we look forward to seeing what colors you're wearing next time we have you on board, and maybe a follow-up with you guys to get a little deeper into how all the fractioning happens and how that works and how the operator works. But really appreciate you coming on board here on theCUBE. Thank you.

Sam Heywood

>> Yeah, no, thank you both. It's been my pleasure.

>> And thank you for joining us for Red Hat Summit AnsibleFest, day two here on theCUBE, the leader in tech analysis and news. Stay tuned. We'll be right back.