Red Hat Summit 2024 | Steven Huels | Red Hat & Justin Boitano | NVIDIA

Clips
News
More from Red Hat Summit 2024

Steven Huels

GM, AI Business Unit

Red Hat

Justin Boitano

Vice President, Enterprise AI

NVIDIA

Three insights you might have missed from theCUBE’s coverage of Red Hat Summit

There’s no shortage of excitement around the possibilities that surround AI and open-source technology. Automation is playing a huge role for AI, including when it comes to bridging the world of hybrid cloud.Exploring the future of hybrid cloud and open-source technology within Red Hat Inc.’s ecosystem was a central focus of the recent Red Hat Summit. This was also the second year that AnsibleFest was held in the same place as the summit.The overall takeaway from the event was that it represented a journey, according to Bob Laliberte, principal analyst at theCUBE Research, in an analysis of the event. There’s no one product that’s going to be a destination, he noted

Red Hat and Nvidia join forces to help users develop AI applications

Two artificial intelligence powerhouses, Red Hat Inc. and Nvidia Corp., are partnering together to better serve enterprise and business users with AI microservices.“Everyone knows AI is going to play a critical role in their enterprise going forward,” said Steven Huels (pictured, right), vice president and general manager of the AI Business Unit at Red Hat. “The challenges they run into have to do with a lot of the tooling, the ease of integration, the deployment and consumption. And the partnership here is really targeted at accelerating that and mak[ing] it as easy to adopt as possible

play_circle_outline Red Hat and NVIDIA Collaborate to Streamline AI Deployment and Address Tooling Challenges

play_circle_outline NIMS role in performance optimization and securing runtime for models

play_circle_outline The importance of partnerships in staying ahead in the competitive AI market

play_circle_outline The evolving role of AI in hybrid environments, including edge device deployments

Info

Steven Huels | Red Hat & Justin Boitano | NVIDIA

Steven Huels

GM, AI Business Unit Red Hat

Justin Boitano

Vice President, Enterprise AI NVIDIA

search

>> Hello and welcome back to theCUBE at Red Hat Summit in AnsibleFest. We're so excited to be here in Denver today really unpacking all of the announcements and all of the partnerships and everything that's going on within the Red Hat environment. I think what's been a lot of fun is that I'm Rob Strechay, joined by Paul Gillin, and we got Rebecca Knight floating around here who will be back up soon. We get to dig in into what does it mean when we say AI? And how do you bring this together? Because I look at it and it takes a village. And note, that village is full of partners. And when you start to look at that, Paul, one of the ones that always comes to mind is NVIDIA. And so we're joined by Steven Huels, who's the VP and GM for the AI business unit at Red Hat, and Justin Boitano, who's the vice president of Enterprise AI at NVIDIA. Couldn't have two better people partnering up and coming here to talk about this. So welcome.

Steven Zipperman

>> Thanks for having us.

Justin Bougher

>> Thanks for having us.

Steven Zipperman

>> This is awesome.

>> Yeah, I think everything I've seen here today is really good about how the thread is being pulled throughout from how you get started to how you go to production with AI. And I think, again, some of the discussions that you've had around OpenShift AI and the partnership with NVIDIA and NIMS really makes a lot of sense about that last mile and getting to production and really getting things up and running and how you can do that efficiently. So why don't you guys... Steven, why don't you kick us off with, Hey, what was the announcement about? And Justin can jump in there too.

Steven Zipperman

>> No, that's great. It's a really exciting announcement for us. Probably a lot of folks don't know, when we started our first AI strategy at Red Hat, NVIDIA was in the founding partner group when we launched, and we've just continued to build on that relationship. What we announced today is with OpenShift AI, we provide that enterprise-grade MLOps platform that allows customers to do their development, tuning, deployment and scaling and life-cycling of models. And we've integrated the power of NVIDIA NIMS into that to provide customers with an easy-to-start point with NVIDIA optimized NIMS so they have access to different models that they can then integrate into applications and scale those consistently across the OpenShift platform. It's really focused on the easing, the consumption of AI, which is we hear a lot from customers, right? That at this point, everyone knows AI is going to play a critical role in their enterprise going forward. The challenges they run into have to do with a lot of the tooling, the ease of integration, the deployment and consumption. And the partnership here is really targeted at accelerating that and make it as easy to adopt as possible.

>> And go a little deeper on what NIMS brings to the party here.

Justin Bougher

>> Yeah. So we first announced NIMS at GTC in mid-March. And at our core, we're an accelerated computing company. We've been at this now for, I don't know, 25 years, accelerating first applications and then compute. And what we realized is as this open model ecosystem flourishes, last summer, it really started, Meta put out Llama 2. There's been 30 million downloads of Llama 2. They just put out Llama 3. There's Mistral and Cohere and a bunch of other model builders getting into the game. And so where we start in this is our value is really to do the performance acceleration of all those models. Every model builder in the world wants their models to run as performant and as efficiently from a power standpoint on our processors as possible because this turns into real cost savings for running those models in production and better data center power efficiency. So we really start there in the NIMS performance optimization. The NIMS then require, I'll call it open source model, serving back ends and some of our libraries. And so we secure that runtime, we clear it of all vulnerabilities. Next, exposures so that security teams can be confident they can take this containerized microservice with the state-of-the-art models in a secured way and connect it to their enterprise data on OpenShift in any cloud or data center. So really that's where we're starting with NIMS and our contribution, I think, to this overall solution.

Paul Gillin

>> So if I'm just getting started with training LLMs, what do I have to do to bring OpenShift AI and NIMS into the equation?

Steven Zipperman

>> Yeah. This one's the great part, right? So I think when generative AI really started to enter the market and take off, customers were looking for ways to take advantage of it. And everyone sort of natural progression went from, "Hey, maybe we'll tune our own or build our own LLM," which most people don't have the infrastructure for or the skills for, the data for. Then it migrated into tuning, like, "We'll figure out ways to infuse data into these models themselves." That still required a lot of data expertise, required resources, and you were still committing yourself to a specific model. Then you started to see RAG take off where it gave you a more transient way of integrating data with LLMs. With NIMS, we've actually taken that a step further where you don't have to stand up the model or train it at all yourself, right? From a couple of clicks of the button, you're able to stand up an LLM within your infrastructure. It's already running. You can scale it. So if you're doing RAG, you can then integrate a vector database. So your focus can be on how do I encode and embed my data into the Vector database? I can test out different models from NIMS to find out which one suits my use case the best and I'm into production a lot faster. So we've removed a lot of the requirements around getting started to really just a couple of button clicks.

>> Yeah, I think that to me is the key. And I think, again, it's been the story here with all the announcements from InstructLab and Podman all the way up to OpenShift AI, RHEL AI, and somewhere in the middle there. And when I look at this and how you go to market with this and bring this to the customers, to me it's about that optionality, right? It's about the open source model. I mean, again, you had the granite models being open source and the code generation also. And I think to me, Open generally wins. Are you seeing that from the NVIDIA side where you're bringing all these models and models serving together? You have your own models, but you also bring together to open source ones.

Justin Bougher

>> Yeah, I mean, look, we are just advocates of people building the world's best models, whether they're open or honestly proprietary. We love them all. I think they all serve different purposes as you start to build applications. What we're seeing more is these applications are moving into these agent-based architectures where you basically a user asset question, a model like a big model, we usually decompose that into a plan on how to go collect the right information from across the enterprise. One model might have to be fine-tuned to understand a specific data source. Another model, a standard embedding model off the shelf is okay to go select data from a vector database. And so these applications become these ensemble of models. And you're fine-tune when you need to drive up accuracy for domain-specific things. But everybody wants to start with, "Well, how far can I get with the off-the-shelf problem? Because generative AI is no good if it's not in production." So if you sit there and play with it and you're trying to tune models and curate it, it doesn't help you. Typically, what you want to do is figure out how to drive the accuracy up to get a business outcome going, get it into production, and then you can figure out how to shrink the models and cost-reduce it and make it more cost-effective. But ultimately, that's where we see customers going.

Paul Gillin

>> Okay. What is the attitude toward open source within NVIDIA right now? Is that evolving at all? I mean, you have a lot of proprietary software. Is that attitude changing?

Justin Bougher

>> Oh, man, it's funny. I feel like we should actually do more in our open source marketing to make it clear how much open source we do. We've created projects in the data science community called Rapids. It's all open source. It's for doing data, preparation, and curation in potentially a pre-training phase. We take all the, I'll call it the training frameworks and fine-tuning frameworks. TensorFlow, PyTorch, we downstream them. We performance optimize them. We upstream that. We work in the Kubernetes ecosystem with our friends to figure out how do you do a node feature discovery, how do you detect devices in a server? How do you use operators to automatically detect those devices, install the runtimes of our GPUs, install monitoring and management tools. So we do a ton in the open source. I think sometimes that gets lost, so we're huge advocates-

Paul Gillin

>> Proprietary isn't bad.

Justin Bougher

>> Yeah, no, but we're huge advocates of the open source. The times where we're not open source is when we need to move quickly, right? And so that's the tension I think we have uniquely as a company. We need to invent new hardware and bring new hardware capabilities to market at a speed that we found the community can't absorb. So we do that stuff out of tree. And then over time we work to get what we can back upstream to work with the community. So we try and play our part, that's for sure.

Paul Gillin

>> What's unique about the relationship with Red Hat?

Justin Bougher

>> I mean, our teams work together in a ton of open source projects naturally, but really on the go-to-market side I think is where we see this one plus one equaling three. We can basically activate our entire demand generation of NIMS. If people don't know how to experience them, they can go to ai.nvidia.com and from a browser you can prompt any model. So what's great about this is we're tracking all those open source models in the day they launch, people can prompt them and interact with them and experience. You can export an API key, you can start to build that into applications to experiment with these multi-model microservices calling into the hosted endpoints. But then when you want to basically ensure you've got data privacy and security and you want to move it into your own data center so you protect your prompts and your intellectual property, you can export those NIMS and then run them on OpenShift. And so I think that's where I think a lot of customers are asking for us to meet, is in their clouds, in their cloud VPCs or in their data centers because they're not quite yet comfortable with their prompts and the IP through retrieval augmented generation, all being sent to these models as services. So they want to control that and make sure it all kind of stays on their network.

Steven Zipperman

>> And Red Hat's always been an ecosystem play. Even from the early days of RHEL, we provided all the core bits you needed for security, user management, networking, all of that. But then we relied on our ecosystem really to build and manage their applications on top. It's no more true anywhere else other than AI as well. In AI space at this point, there's no single vendor that's going to be able to give you the full end-to-end system. And so Red Hat, again, we focus on our strength, which is giving the core components you'll need in an AI management and system. But then we rely on our partners to go deep in specific areas. And that's everything from the accelerator later, making sure they have access to the components they need to be able to do their experiments all the way through the usability stack. And so for us, this has been a natural extension of motions we've been doing for decades at this point.

>> So I think you kind of hit on the agent aspect of it. And I think there's also the whole mixture of experts, the MOE type of thing. Where do you see organizations as they look to... So you have the ones that are just getting started, and okay, they're going, "Hey, maybe I'm doing RAG to get out of the gate because it's easy for me to hook in my data and kind of train on it with the vector embeds and tagging and all of that." How do you see them progressing up to something like wanting to do a mixture of experts and things of that nature?

Steven Zipperman

>> Yeah, so I think in general, AI as a whole isn't necessarily new in the technology realm. Before generative AI, we had predictive analytics. In a lot of that space, those models were built and targeted towards specific use cases. So if you were in financial services, maybe it was a risk assessment, maybe it was a forecast of some sort. When we start looking at generative AI, what changed now is I think enterprises know they have to have an enterprise-wide AI system and that users across all the lines of business are going to interact with AI. But I think the core trend that they want models that are going to be specific to their use cases that provide business value is going to continue. So I think we're going to continue to see customers' journey towards smaller models, more efficient models, targeted specific use cases. And they're going to probably look in a lot of cases to start with augmenting places that are already getting value. So there will be things that branch off into new domains, but they're probably going to look to complement traditional predictive AI with generative AI and expand those systems. And that's a lot of the agent architecture that Justin was talking about here.

Paul Gillin

>> Justin, the NVIDIA course, the Wall Street darling of last year, big target painted on your back right now. A lot of competitors coming after NVIDIA. What role do partnerships play in your staying ahead?

Justin Bougher

>> Yeah, look, as a company, we realize everybody wants a part of this market. It's amazingly new market that's being created. As Jensen would say. There's a new industry being born through generative AI. And so I mean, honestly, we got to keep running faster. We got to work more with our partners to help customers achieve success quickly with generative AI, which is part of the reason for the work we do. And I think beyond the direct partnership, there's also the indirect partners and working with GSIs and helping them understanding these application patterns that we're starting to see emerge and then seeding them with some of these use cases that are really easy to stand up, provide really a quick time to value and help them get going. So another open source contribution is on our GitHub page, we call it NVIDIA generative AI examples, We have workflows that touch on common RAG patterns or common agent patterns that people can use to get going as application frameworks. Our SAs use those with customers, our GSIs use those with customers, and then people contribute back there. So I think that's also a flourishing ecosystem that people can tap into for kind of tools to get going. And actually what's great about there is the API's being called from there, they're all hosted through ai.nvidia.com so that people can use the NIMS. And when they're ready, they can export it and run in their data center.

>> Yeah, I think that to me is one of the big things. And I know you have some different sessions going on today and tomorrow around how AI is not just in a data center, it's not just in the cloud, it is hybrid. I think that's a place that really is key because these models have to be trained. We actually have a power law around gen AI. And when you look at the long tail on there, we think most of them when they're SMLs or small or segmented language models that are deployed for inference out wherever and things of that. Is that where a focus for OpenShift AI as well?

Steven Zipperman

>> So it's definitely something we're working in that direction. And we did some announcements today on how we're supporting single node OpenShift. You can kind of anticipate what's coming next as far as edge deployments. So in a lot of cases, again, this is merging very, very quickly. Customers are now there doing that. They're talking about putting generative AI into edge device locations. So when we look at OpenShift AI, it's already a composable system. What we're looking at is then how do we take individual components and be able to push those to specific platforms. So you can push just our serving runtime to an edge layer, put an LLM there, but you can still have that model phone home, all of its metrics, telemetry, so you can monitor your entire fleet from a single pane of glass. This is really valuable in use cases like factory deployments where you have multiple factories, multiple devices, single model may have hundreds of instances deployed and you want to see how they're all performing. So these are all things that we're designing for our customers. Yes.

>> Yeah, no, I think that's a great place for us to leave it. I think, again, it gives everybody an idea of how to start and how to get to production, which is I think what we're digging into here, because I think AI has been seen as mystical and so complicated, but I think these types of partnerships are really helping to bring it and simplify it for that kind of big middle group of people.

Justin Bougher

>> And we haven't said it enough, the easy button is ai.nvidia.com. And I'm telling you, all the hosted models are there. Super easy to get going.

>> We love that. So thank you both for coming on board and really appreciate it.

Justin Bougher

>> Thank you so much.

Steven Zipperman

>> Thanks for having us.

>> Yeah. And thank you. Stay tuned to more from Red Hat Summit Ansiblefest here on VQ, where we're the leader in tech analysis and news. Thank you.

Steven Huels | Red Hat & Justin Boitano | NVIDIA

search

>> Hello and welcome back to theCUBE at Red Hat Summit in AnsibleFest. We're so excited to be here in Denver today really unpacking all of the announcements and all of the partnerships and everything that's going on within the Red Hat environment. I think what's been a lot of fun is that I'm Rob Strechay, joined by Paul Gillin, and we got Rebecca Knight floating around here who will be back up soon. We get to dig in into what does it mean when we say AI? And how do you bring this together? Because I look at it and it takes a village. And note, that village is full of partners. And when you start to look at that, Paul, one of the ones that always comes to mind is NVIDIA. And so we're joined by Steven Huels, who's the VP and GM for the AI business unit at Red Hat, and Justin Boitano, who's the vice president of Enterprise AI at NVIDIA. Couldn't have two better people partnering up and coming here to talk about this. So welcome.

Steven Zipperman

>> Thanks for having us.

Justin Bougher

>> Thanks for having us.

Steven Zipperman

>> This is awesome.

>> Yeah, I think everything I've seen here today is really good about how the thread is being pulled throughout from how you get started to how you go to production with AI. And I think, again, some of the discussions that you've had around OpenShift AI and the partnership with NVIDIA and NIMS really makes a lot of sense about that last mile and getting to production and really getting things up and running and how you can do that efficiently. So why don't you guys... Steven, why don't you kick us off with, Hey, what was the announcement about? And Justin can jump in there too.

Steven Zipperman

>> No, that's great. It's a really exciting announcement for us. Probably a lot of folks don't know, when we started our first AI strategy at Red Hat, NVIDIA was in the founding partner group when we launched, and we've just continued to build on that relationship. What we announced today is with OpenShift AI, we provide that enterprise-grade MLOps platform that allows customers to do their development, tuning, deployment and scaling and life-cycling of models. And we've integrated the power of NVIDIA NIMS into that to provide customers with an easy-to-start point with NVIDIA optimized NIMS so they have access to different models that they can then integrate into applications and scale those consistently across the OpenShift platform. It's really focused on the easing, the consumption of AI, which is we hear a lot from customers, right? That at this point, everyone knows AI is going to play a critical role in their enterprise going forward. The challenges they run into have to do with a lot of the tooling, the ease of integration, the deployment and consumption. And the partnership here is really targeted at accelerating that and make it as easy to adopt as possible.

>> And go a little deeper on what NIMS brings to the party here.

Justin Bougher

>> Yeah. So we first announced NIMS at GTC in mid-March. And at our core, we're an accelerated computing company. We've been at this now for, I don't know, 25 years, accelerating first applications and then compute. And what we realized is as this open model ecosystem flourishes, last summer, it really started, Meta put out Llama 2. There's been 30 million downloads of Llama 2. They just put out Llama 3. There's Mistral and Cohere and a bunch of other model builders getting into the game. And so where we start in this is our value is really to do the performance acceleration of all those models. Every model builder in the world wants their models to run as performant and as efficiently from a power standpoint on our processors as possible because this turns into real cost savings for running those models in production and better data center power efficiency. So we really start there in the NIMS performance optimization. The NIMS then require, I'll call it open source model, serving back ends and some of our libraries. And so we secure that runtime, we clear it of all vulnerabilities. Next, exposures so that security teams can be confident they can take this containerized microservice with the state-of-the-art models in a secured way and connect it to their enterprise data on OpenShift in any cloud or data center. So really that's where we're starting with NIMS and our contribution, I think, to this overall solution.

Paul Gillin

>> So if I'm just getting started with training LLMs, what do I have to do to bring OpenShift AI and NIMS into the equation?

Steven Zipperman

>> Yeah. This one's the great part, right? So I think when generative AI really started to enter the market and take off, customers were looking for ways to take advantage of it. And everyone sort of natural progression went from, "Hey, maybe we'll tune our own or build our own LLM," which most people don't have the infrastructure for or the skills for, the data for. Then it migrated into tuning, like, "We'll figure out ways to infuse data into these models themselves." That still required a lot of data expertise, required resources, and you were still committing yourself to a specific model. Then you started to see RAG take off where it gave you a more transient way of integrating data with LLMs. With NIMS, we've actually taken that a step further where you don't have to stand up the model or train it at all yourself, right? From a couple of clicks of the button, you're able to stand up an LLM within your infrastructure. It's already running. You can scale it. So if you're doing RAG, you can then integrate a vector database. So your focus can be on how do I encode and embed my data into the Vector database? I can test out different models from NIMS to find out which one suits my use case the best and I'm into production a lot faster. So we've removed a lot of the requirements around getting started to really just a couple of button clicks.

>> Yeah, I think that to me is the key. And I think, again, it's been the story here with all the announcements from InstructLab and Podman all the way up to OpenShift AI, RHEL AI, and somewhere in the middle there. And when I look at this and how you go to market with this and bring this to the customers, to me it's about that optionality, right? It's about the open source model. I mean, again, you had the granite models being open source and the code generation also. And I think to me, Open generally wins. Are you seeing that from the NVIDIA side where you're bringing all these models and models serving together? You have your own models, but you also bring together to open source ones.

Justin Bougher

>> Yeah, I mean, look, we are just advocates of people building the world's best models, whether they're open or honestly proprietary. We love them all. I think they all serve different purposes as you start to build applications. What we're seeing more is these applications are moving into these agent-based architectures where you basically a user asset question, a model like a big model, we usually decompose that into a plan on how to go collect the right information from across the enterprise. One model might have to be fine-tuned to understand a specific data source. Another model, a standard embedding model off the shelf is okay to go select data from a vector database. And so these applications become these ensemble of models. And you're fine-tune when you need to drive up accuracy for domain-specific things. But everybody wants to start with, "Well, how far can I get with the off-the-shelf problem? Because generative AI is no good if it's not in production." So if you sit there and play with it and you're trying to tune models and curate it, it doesn't help you. Typically, what you want to do is figure out how to drive the accuracy up to get a business outcome going, get it into production, and then you can figure out how to shrink the models and cost-reduce it and make it more cost-effective. But ultimately, that's where we see customers going.

Paul Gillin

>> Okay. What is the attitude toward open source within NVIDIA right now? Is that evolving at all? I mean, you have a lot of proprietary software. Is that attitude changing?

Justin Bougher

>> Oh, man, it's funny. I feel like we should actually do more in our open source marketing to make it clear how much open source we do. We've created projects in the data science community called Rapids. It's all open source. It's for doing data, preparation, and curation in potentially a pre-training phase. We take all the, I'll call it the training frameworks and fine-tuning frameworks. TensorFlow, PyTorch, we downstream them. We performance optimize them. We upstream that. We work in the Kubernetes ecosystem with our friends to figure out how do you do a node feature discovery, how do you detect devices in a server? How do you use operators to automatically detect those devices, install the runtimes of our GPUs, install monitoring and management tools. So we do a ton in the open source. I think sometimes that gets lost, so we're huge advocates-

Paul Gillin

>> Proprietary isn't bad.

Justin Bougher

>> Yeah, no, but we're huge advocates of the open source. The times where we're not open source is when we need to move quickly, right? And so that's the tension I think we have uniquely as a company. We need to invent new hardware and bring new hardware capabilities to market at a speed that we found the community can't absorb. So we do that stuff out of tree. And then over time we work to get what we can back upstream to work with the community. So we try and play our part, that's for sure.

Paul Gillin

>> What's unique about the relationship with Red Hat?

Justin Bougher

>> I mean, our teams work together in a ton of open source projects naturally, but really on the go-to-market side I think is where we see this one plus one equaling three. We can basically activate our entire demand generation of NIMS. If people don't know how to experience them, they can go to ai.nvidia.com and from a browser you can prompt any model. So what's great about this is we're tracking all those open source models in the day they launch, people can prompt them and interact with them and experience. You can export an API key, you can start to build that into applications to experiment with these multi-model microservices calling into the hosted endpoints. But then when you want to basically ensure you've got data privacy and security and you want to move it into your own data center so you protect your prompts and your intellectual property, you can export those NIMS and then run them on OpenShift. And so I think that's where I think a lot of customers are asking for us to meet, is in their clouds, in their cloud VPCs or in their data centers because they're not quite yet comfortable with their prompts and the IP through retrieval augmented generation, all being sent to these models as services. So they want to control that and make sure it all kind of stays on their network.

Steven Zipperman

>> And Red Hat's always been an ecosystem play. Even from the early days of RHEL, we provided all the core bits you needed for security, user management, networking, all of that. But then we relied on our ecosystem really to build and manage their applications on top. It's no more true anywhere else other than AI as well. In AI space at this point, there's no single vendor that's going to be able to give you the full end-to-end system. And so Red Hat, again, we focus on our strength, which is giving the core components you'll need in an AI management and system. But then we rely on our partners to go deep in specific areas. And that's everything from the accelerator later, making sure they have access to the components they need to be able to do their experiments all the way through the usability stack. And so for us, this has been a natural extension of motions we've been doing for decades at this point.

>> So I think you kind of hit on the agent aspect of it. And I think there's also the whole mixture of experts, the MOE type of thing. Where do you see organizations as they look to... So you have the ones that are just getting started, and okay, they're going, "Hey, maybe I'm doing RAG to get out of the gate because it's easy for me to hook in my data and kind of train on it with the vector embeds and tagging and all of that." How do you see them progressing up to something like wanting to do a mixture of experts and things of that nature?

Steven Zipperman

>> Yeah, so I think in general, AI as a whole isn't necessarily new in the technology realm. Before generative AI, we had predictive analytics. In a lot of that space, those models were built and targeted towards specific use cases. So if you were in financial services, maybe it was a risk assessment, maybe it was a forecast of some sort. When we start looking at generative AI, what changed now is I think enterprises know they have to have an enterprise-wide AI system and that users across all the lines of business are going to interact with AI. But I think the core trend that they want models that are going to be specific to their use cases that provide business value is going to continue. So I think we're going to continue to see customers' journey towards smaller models, more efficient models, targeted specific use cases. And they're going to probably look in a lot of cases to start with augmenting places that are already getting value. So there will be things that branch off into new domains, but they're probably going to look to complement traditional predictive AI with generative AI and expand those systems. And that's a lot of the agent architecture that Justin was talking about here.

Paul Gillin

>> Justin, the NVIDIA course, the Wall Street darling of last year, big target painted on your back right now. A lot of competitors coming after NVIDIA. What role do partnerships play in your staying ahead?

Justin Bougher

>> Yeah, look, as a company, we realize everybody wants a part of this market. It's amazingly new market that's being created. As Jensen would say. There's a new industry being born through generative AI. And so I mean, honestly, we got to keep running faster. We got to work more with our partners to help customers achieve success quickly with generative AI, which is part of the reason for the work we do. And I think beyond the direct partnership, there's also the indirect partners and working with GSIs and helping them understanding these application patterns that we're starting to see emerge and then seeding them with some of these use cases that are really easy to stand up, provide really a quick time to value and help them get going. So another open source contribution is on our GitHub page, we call it NVIDIA generative AI examples, We have workflows that touch on common RAG patterns or common agent patterns that people can use to get going as application frameworks. Our SAs use those with customers, our GSIs use those with customers, and then people contribute back there. So I think that's also a flourishing ecosystem that people can tap into for kind of tools to get going. And actually what's great about there is the API's being called from there, they're all hosted through ai.nvidia.com so that people can use the NIMS. And when they're ready, they can export it and run in their data center.

>> Yeah, I think that to me is one of the big things. And I know you have some different sessions going on today and tomorrow around how AI is not just in a data center, it's not just in the cloud, it is hybrid. I think that's a place that really is key because these models have to be trained. We actually have a power law around gen AI. And when you look at the long tail on there, we think most of them when they're SMLs or small or segmented language models that are deployed for inference out wherever and things of that. Is that where a focus for OpenShift AI as well?

Steven Zipperman

>> So it's definitely something we're working in that direction. And we did some announcements today on how we're supporting single node OpenShift. You can kind of anticipate what's coming next as far as edge deployments. So in a lot of cases, again, this is merging very, very quickly. Customers are now there doing that. They're talking about putting generative AI into edge device locations. So when we look at OpenShift AI, it's already a composable system. What we're looking at is then how do we take individual components and be able to push those to specific platforms. So you can push just our serving runtime to an edge layer, put an LLM there, but you can still have that model phone home, all of its metrics, telemetry, so you can monitor your entire fleet from a single pane of glass. This is really valuable in use cases like factory deployments where you have multiple factories, multiple devices, single model may have hundreds of instances deployed and you want to see how they're all performing. So these are all things that we're designing for our customers. Yes.

>> Yeah, no, I think that's a great place for us to leave it. I think, again, it gives everybody an idea of how to start and how to get to production, which is I think what we're digging into here, because I think AI has been seen as mystical and so complicated, but I think these types of partnerships are really helping to bring it and simplify it for that kind of big middle group of people.

Justin Bougher

>> And we haven't said it enough, the easy button is ai.nvidia.com. And I'm telling you, all the hosted models are there. Super easy to get going.

>> We love that. So thank you both for coming on board and really appreciate it.

Justin Bougher

>> Thank you so much.

Steven Zipperman

>> Thanks for having us.

>> Yeah. And thank you. Stay tuned to more from Red Hat Summit Ansiblefest here on VQ, where we're the leader in tech analysis and news. Thank you.