HPE Discover Barcelona 2023 | Manuvir Das | NVIDIA

theCUBE.net

Clips
News
More from HPE Discover Barcelona 2023

Manuvir Das

VP, Enterprise Computing

Nvidia Corp.

Gen AI and beyond: A glimpse into the future of enterprise technology

Generative artificial intelligence is a transformative force set to redefine how enterprises approach productivity and innovation.The transition from cloud-based AI to on-premises deployment is being addressed with turnkey solutions to ensure data security and intellectual property protection. Nvidia Corp.’s commitment to sustainability is driving innovations in power-efficient GPUs, reinforcing its dedication to environmental responsibility, according to Manuvir Das (pictured), vice president of enterprise computing at Nvidia. As AI continues to reshape the tech industry, Nvidia’s gen AI and sustainable hardware solutions are at the forefront of empowering enterprises for the future

play_circle_outline The Vital Role of AI: Unveiling a groundbreaking supercomputing solution for revolutionary AI models

play_circle_outline Discussion on the deployment of AI and the edge-to-cloud strategy

Info

Manuvir Das | NVIDIA

Manuvir Das

VP, Enterprise Computing Nvidia Corp.

search

(upbeat music)

>> Hello everyone, and welcome back to "theCUBE's" live coverage of HPE Discover Barcelona 2023. I'm your host, Rebecca Knight, along with my co-host and analyst, Rob Strechay. As we've said, this is day two. This is it. We're in this conference.

>> We're definitely in. And I think this is exciting because I think, if you hadn't been asleep for the last year, you know, AI is really the big thing that's going on, and there was a lot of announcements made from the main stage, and I'm excited that we have one of the speakers-

>> Indeed, indeed. >> Who was up there

with Antonio. >> A main stage celebrity.

>> Exactly. And we can get deeper into that.

>> Yeah. >> Because I think there was some really neat stuff being done.

>> So I'd like to welcome Manuvir Das. He is the VP Enterprise Consulting at NVIDIA Back to "theCUBE", you've been on here many times, a "CUBE" veteran, Manuvir, welcome.

>> A few times, yes. >> Yeah.

>> It's a pleasure to be here, though.

>> Thank you. Well, as Rob was just saying, you were up on the main stage, you did a terrific job. You were making announcements with Antonio. Can you tell our viewers a little bit more about some of the most exciting news from the main stage this morning?

>> Yeah, I think there were really three things that we announced. The first thing was, you know, what people have traditionally thought about with AI, that you're building these big models and you need this super computer style infrastructure to build big models. And so the announcement was really that because of Gen AI, there's more and more companies that now want to build their own models. And so they need that same kind of infrastructure. And so they need a more complete solution. Because, you know, when there's just a few, there's a lot of DIY, right? I'll stitch things together, it's okay. I've hired the few people in the world who really know how to do it, and they'll stitch it all together for me. But when you have thousands of companies trying to do this, you need a more prepared solution so that, if you don't have one of those five people, you can still get it done, right. So that was the first announcement, right. Which was the supercomputing solution for building models. The second announcement was the enterprise solution, which is, if you really think about the miracle of generative AI for enterprise companies, I'll put it to you this way, right? Like my company and I, we've been working with users of AI for years now, right. And I've had so many meetings where it's been, okay, what does your company do? Which industry are you in? Here's an interesting AI use case for you that could change your business. And then you kind of go through this conversation and they have to believe it. Okay, so it was like a one customer at a time. Explain to them why AI might be helpful to them and then get them over the line. But you know, what's changed with generative AI is that, no matter what you do as a company, there's a use case for you. Because it's just about human productivity, okay. So, you know, if I could, the simplest way I'd put it is, you think about any person who's a professional worker, you're doing a job in a company, right. Maybe you write press releases. Okay, I used that example earlier. Or maybe, you know, you're in sales and you're generating sales reports, right? How cool would it be if you had a free intern, right? Somebody who's in college, they're a hot shot from Harvard, but they sit with you and they do 80% of the work upfront for you, and then you use your expertise to finish it up. And that's basically what Gen AI is. That's what these models are able to do. And so now every company wants to do this, but the question is, am I going to' have the giant infrastructure to train a model? No, but what I can do is I can take a model somebody else already built, a foundation model, I can start from that. And so now the infrastructure I need is only for the rest of the thing. Somebody gave me a foundation model, I'm going to' fine tune it with my data, and then I'm going to' deploy my application. So the second solution was really for that use case, okay, you've already got a model, how are you going to' fine tune it and deploy it?

>> Yeah, I think that's the important part is also the complexity of setting up a system. And I think that was loud and clear from the announcement this morning is, how do you have a turnkey solution for bringing AI? Because, as you said, maybe they're going to use your, you have some foundation models that you've put out. There's Llama 2, there's a number of other, you know, you can go to Hugging Face and use that as your, you know, or Anthropic, or what have you. Are you seeing that people, so many people want to' get started, but the reason you had to go turnkey is they just don't know where to start putting it together.

>> Yeah, yeah. You know what's been happening, Rob, is that, as with many things, the cloud is a natural place to start. Because you just have APIs for these things and you use it. And so I would say in almost every company, people who started doing the first work with Gen AI were just using the cloud and using the services that already exist in the cloud. Because you don't have to deploy infrastructure. It's already sitting there underneath. And you don't need to bring software in because it's hidden in those services. You just use it. And now we are at the point where these companies, they've got a few projects, okay, this is real, I'm getting productivity. I want to actually do this seriously, but now I need to do it on my own. Because, you know, when you use the cloud services, every time you prompt the model, what are you actually doing? You're sending your company's IP over there. Where is it going, how is it being used? And so that's why now you need the turnkey, because if every company had to redo for themselves what an OpenAI has done, or what Anthropic has done, that's too hard. Somebody has to package it up and give it to them so it's easy for them to use.

>> I think, again, and not to cut you off, but I think following up on that, 'cause I think on our research side, on "theCUBE" research side, what we've come up with is really a power law of AI, of Gen AI in particular. Where, you know, again, you have this long tail and what we've actually said is there's bumps in the tail and it's size of model versus variants across different verticals.

>> Correct. >> And specificity across those. And I think what we're seeing is that there's going to' be a really narrow band of people building their own foundation models. But there's going to' be a very long tail-

>> Yeah, yeah. >> Of people having private AI.

As you would say. >> Private AI, I think we like to say it as, you want to' own your own model and you want to' carry it in your briefcase with you, so you can deploy it wherever you want. But the beauty of it is to own your own model, you don't have to do all the work. Because you pick up one of these prebuilt ones, whether it's Llama 70b, or whatever. And then you fine tune a little bit, customize a little bit, and now it's your variant and it belongs to you. So I think that's very important. And then I think to your other point, you know, because there's so many different use cases, the interesting thing about these models is, I can take an off the shelf model and for the thing I want to do, it's really not very good, but if I do a little fine tuning on it, it becomes really good and I don't need the giant GPT model. And so you're going to' have a very large number of models. And so I think for every company, you're going to' have hundreds of difference of models that you are using within your company. They're all coming from the same foundation, like a Llama 70b, or NVIDIA's Nemotron models or whatever. But you're going to' have like a whole library of these.

>> I mean the example that you used on the main stage of writing press releases, I'm a journalist, it really resonated with me because I understand that that takes knowledge, and a dash of creativity, and an understanding of how things work to have to do those things. And to describe the AI doing sort of the intern work, you know, getting you the basics. And then the human in the loop comes in and says, okay, we all know what the kind of gobbledygook ChatGPT can can churn out. And, you know, let's make this more accessible. Let's make this have a little bit of more-

>> Yeah I think because you need the actual human expert to produce the final result. You know, and, Rebecca, this is also why I think in our interactions with enterprise customers, this is by far the dominant use case right now of these assistance. Because as long as I'm using Gen AI to create assistance for my employees, the risk profile is very low. Because the AI is not producing an end result that is exposed to the world. It's just a hidden assistant. As opposed to Gen AI is doing the customer conversation or something like that. Now it's like, what am I really saying to the customer? What data am I exposing? So I think this assistant use case is really powerful. People use terms like copilot, for example, is a term I think that's good. I think every professional employee in every company can do with an assistant. And here's Gen AI to be that assistant.

>> Yeah, I think it is about productivity and I think that beyond Gen AI, your hardware actually helps people with other AI, as well. Which we used to call ML, but you know, not splitting-

>> And that hasn't gone away. >> No, exactly.

>> But also, when the world changes, you have to be aware of it. And so, you know, NVIDIA has worked on all the other kinds of AI for years, but we knew Gen AI was coming. We were the ones working with OpenAI for years on all the infrastructure. So of course all the other AI use cases are continuing, but we are very clear on the impact of Gen AI. And so that's why NVIDIA is such a big focus, and that's why we did these solutions with HPE.

>> And I think it's also how you deploy because they have their edge to cloud strategy and being hybrid. And are you seeing more as they go around this, looking for that turnkey type solution because they want to' put stuff closer to where the data is at the edge?

>> Yes, absolutely. You know, I'll put it to you this way, right? So NVIDIA, we are a platform company. We work with everybody, right? Our technology is in each of the clouds. And the clouds all have great offerings. So you could ask yourself, why is NVIDIA, why do we produce the whole stack? And the reason is because we consistently hear from customers that they want the choice. They want to choose, I'm running some workload on this cloud, some workload on that cloud. And we are the ones who say, well, our hardware is available everywhere. Our software is the same software everywhere. So you get a consistent experience, right? So now how does that dovetail with HPE? I think HPE made a bet a few years ago with GreenLake to say, on the infrastructure side, we want to' be the people who say to the customer, "We'll meet you wherever you want to be met." And so there's a very natural sort of, you know, synergy between them and us, right? Because we want our technology to be everywhere, and they want people to consume infrastructure everywhere. And so it just fits, so, yeah.

>> You know, you made the point that you do work with a lot of different providers. And here you are at HPE, but there's other technology conferences going around and your boss Jensen is flying all over the world. How does that work internally? Because we know that having a partnership, it really does matter so much, in terms of your alignment on values, and priorities, and your commitment to sustainability, and things like that. How are you as NVIDIA, as a company, making sure that you are melding with all of these different-

>> Rebecca, that's such a great question. I think there's two answers. The first is, we take a platform approach where we build one stack that works in lots of different places. So that's what makes it practical to work with many partners. But the other word that I think is much more important is empathy. So the way we choose to work with a partner, let's say HPE, is what does HPE stand for? What is their customer base? Why do HPE's customers go to HPE? Okay, we'll work with you to produce a solution that fits into that model. On the other hand, if you're working with a different partner, they may have a completely different set of customers who think differently, who consume the technology differently. And in that case, we'll work with that partner in a different way. So we are not trying to take our square peg and put it into everybody's round holes. We go to each of them and say, "What does your round hole look like?" And we'll make our peg the right round for you. So that's kind of' how we look at it.

>> No, that's really interesting.

>> I think, again, we had Justin on yesterday and we were talking about Grace Hopper and what they're doing there. I think there seems to be more alignment than just at the hardware level. And I think sustainability is one of those places where NVIDIA takes it pretty seriously.

>> I think it's huge, you know, because I think for many years now, with the advent of the cloud, Rob, people have been very focused on the economic comparison between the public cloud and doing your own. And you know, there's the saying that if you're willing to do your own and you have the expertise, it'll be more cost effective than renting. But there's also that logic now for, you know, power consumption. And the thing about that we are really proud of at NVIDIA is these GPUs, you know. If you look at one GPU, it consumes more power than one CPU. But one GPU does the work of like 1,000 CPUs. So when you're actually running your workload, the most power efficient, energy efficient way to run your workload is on GPUs. And the reason we have these new generations, like Grace Hopper, is because we're constantly making the GPUs more and more power efficient. So we love what Justin's doing because he's actually delivering what looks to be on one unit, a more power hungry product, but on a workload level is actually more power efficient and it's going to' save the world a lot of money and a lot of energy consumption. So yeah, it's exciting.

>> It is, exciting times. Manuvir, thank you so much for coming on, for returning to "theCUBE", I should say.

>> Of course, it was my pleasure.

Until next time, I guess. >> Until next time, yeah. See you in Barcelona in 2024.

>> Likewise.

>> I'm Rebecca Knight. Stay tuned for more of "theCUBE's" live coverage. For Rob Strechay, you are watching "theCUBE", the leader in high tech enterprise coverage. (upbeat music)

Manuvir Das | NVIDIA

search

(upbeat music)

>> Hello everyone, and welcome back to "theCUBE's" live coverage of HPE Discover Barcelona 2023. I'm your host, Rebecca Knight, along with my co-host and analyst, Rob Strechay. As we've said, this is day two. This is it. We're in this conference.

>> We're definitely in. And I think this is exciting because I think, if you hadn't been asleep for the last year, you know, AI is really the big thing that's going on, and there was a lot of announcements made from the main stage, and I'm excited that we have one of the speakers-

>> Indeed, indeed. >> Who was up there

with Antonio. >> A main stage celebrity.

>> Exactly. And we can get deeper into that.

>> Yeah. >> Because I think there was some really neat stuff being done.

>> So I'd like to welcome Manuvir Das. He is the VP Enterprise Consulting at NVIDIA Back to "theCUBE", you've been on here many times, a "CUBE" veteran, Manuvir, welcome.

>> A few times, yes. >> Yeah.

>> It's a pleasure to be here, though.

>> Thank you. Well, as Rob was just saying, you were up on the main stage, you did a terrific job. You were making announcements with Antonio. Can you tell our viewers a little bit more about some of the most exciting news from the main stage this morning?

>> Yeah, I think there were really three things that we announced. The first thing was, you know, what people have traditionally thought about with AI, that you're building these big models and you need this super computer style infrastructure to build big models. And so the announcement was really that because of Gen AI, there's more and more companies that now want to build their own models. And so they need that same kind of infrastructure. And so they need a more complete solution. Because, you know, when there's just a few, there's a lot of DIY, right? I'll stitch things together, it's okay. I've hired the few people in the world who really know how to do it, and they'll stitch it all together for me. But when you have thousands of companies trying to do this, you need a more prepared solution so that, if you don't have one of those five people, you can still get it done, right. So that was the first announcement, right. Which was the supercomputing solution for building models. The second announcement was the enterprise solution, which is, if you really think about the miracle of generative AI for enterprise companies, I'll put it to you this way, right? Like my company and I, we've been working with users of AI for years now, right. And I've had so many meetings where it's been, okay, what does your company do? Which industry are you in? Here's an interesting AI use case for you that could change your business. And then you kind of go through this conversation and they have to believe it. Okay, so it was like a one customer at a time. Explain to them why AI might be helpful to them and then get them over the line. But you know, what's changed with generative AI is that, no matter what you do as a company, there's a use case for you. Because it's just about human productivity, okay. So, you know, if I could, the simplest way I'd put it is, you think about any person who's a professional worker, you're doing a job in a company, right. Maybe you write press releases. Okay, I used that example earlier. Or maybe, you know, you're in sales and you're generating sales reports, right? How cool would it be if you had a free intern, right? Somebody who's in college, they're a hot shot from Harvard, but they sit with you and they do 80% of the work upfront for you, and then you use your expertise to finish it up. And that's basically what Gen AI is. That's what these models are able to do. And so now every company wants to do this, but the question is, am I going to' have the giant infrastructure to train a model? No, but what I can do is I can take a model somebody else already built, a foundation model, I can start from that. And so now the infrastructure I need is only for the rest of the thing. Somebody gave me a foundation model, I'm going to' fine tune it with my data, and then I'm going to' deploy my application. So the second solution was really for that use case, okay, you've already got a model, how are you going to' fine tune it and deploy it?

>> Yeah, I think that's the important part is also the complexity of setting up a system. And I think that was loud and clear from the announcement this morning is, how do you have a turnkey solution for bringing AI? Because, as you said, maybe they're going to use your, you have some foundation models that you've put out. There's Llama 2, there's a number of other, you know, you can go to Hugging Face and use that as your, you know, or Anthropic, or what have you. Are you seeing that people, so many people want to' get started, but the reason you had to go turnkey is they just don't know where to start putting it together.

>> Yeah, yeah. You know what's been happening, Rob, is that, as with many things, the cloud is a natural place to start. Because you just have APIs for these things and you use it. And so I would say in almost every company, people who started doing the first work with Gen AI were just using the cloud and using the services that already exist in the cloud. Because you don't have to deploy infrastructure. It's already sitting there underneath. And you don't need to bring software in because it's hidden in those services. You just use it. And now we are at the point where these companies, they've got a few projects, okay, this is real, I'm getting productivity. I want to actually do this seriously, but now I need to do it on my own. Because, you know, when you use the cloud services, every time you prompt the model, what are you actually doing? You're sending your company's IP over there. Where is it going, how is it being used? And so that's why now you need the turnkey, because if every company had to redo for themselves what an OpenAI has done, or what Anthropic has done, that's too hard. Somebody has to package it up and give it to them so it's easy for them to use.

>> I think, again, and not to cut you off, but I think following up on that, 'cause I think on our research side, on "theCUBE" research side, what we've come up with is really a power law of AI, of Gen AI in particular. Where, you know, again, you have this long tail and what we've actually said is there's bumps in the tail and it's size of model versus variants across different verticals.

>> Correct. >> And specificity across those. And I think what we're seeing is that there's going to' be a really narrow band of people building their own foundation models. But there's going to' be a very long tail-

>> Yeah, yeah. >> Of people having private AI.

As you would say. >> Private AI, I think we like to say it as, you want to' own your own model and you want to' carry it in your briefcase with you, so you can deploy it wherever you want. But the beauty of it is to own your own model, you don't have to do all the work. Because you pick up one of these prebuilt ones, whether it's Llama 70b, or whatever. And then you fine tune a little bit, customize a little bit, and now it's your variant and it belongs to you. So I think that's very important. And then I think to your other point, you know, because there's so many different use cases, the interesting thing about these models is, I can take an off the shelf model and for the thing I want to do, it's really not very good, but if I do a little fine tuning on it, it becomes really good and I don't need the giant GPT model. And so you're going to' have a very large number of models. And so I think for every company, you're going to' have hundreds of difference of models that you are using within your company. They're all coming from the same foundation, like a Llama 70b, or NVIDIA's Nemotron models or whatever. But you're going to' have like a whole library of these.

>> I mean the example that you used on the main stage of writing press releases, I'm a journalist, it really resonated with me because I understand that that takes knowledge, and a dash of creativity, and an understanding of how things work to have to do those things. And to describe the AI doing sort of the intern work, you know, getting you the basics. And then the human in the loop comes in and says, okay, we all know what the kind of gobbledygook ChatGPT can can churn out. And, you know, let's make this more accessible. Let's make this have a little bit of more-

>> Yeah I think because you need the actual human expert to produce the final result. You know, and, Rebecca, this is also why I think in our interactions with enterprise customers, this is by far the dominant use case right now of these assistance. Because as long as I'm using Gen AI to create assistance for my employees, the risk profile is very low. Because the AI is not producing an end result that is exposed to the world. It's just a hidden assistant. As opposed to Gen AI is doing the customer conversation or something like that. Now it's like, what am I really saying to the customer? What data am I exposing? So I think this assistant use case is really powerful. People use terms like copilot, for example, is a term I think that's good. I think every professional employee in every company can do with an assistant. And here's Gen AI to be that assistant.

>> Yeah, I think it is about productivity and I think that beyond Gen AI, your hardware actually helps people with other AI, as well. Which we used to call ML, but you know, not splitting-

>> And that hasn't gone away. >> No, exactly.

>> But also, when the world changes, you have to be aware of it. And so, you know, NVIDIA has worked on all the other kinds of AI for years, but we knew Gen AI was coming. We were the ones working with OpenAI for years on all the infrastructure. So of course all the other AI use cases are continuing, but we are very clear on the impact of Gen AI. And so that's why NVIDIA is such a big focus, and that's why we did these solutions with HPE.

>> And I think it's also how you deploy because they have their edge to cloud strategy and being hybrid. And are you seeing more as they go around this, looking for that turnkey type solution because they want to' put stuff closer to where the data is at the edge?

>> Yes, absolutely. You know, I'll put it to you this way, right? So NVIDIA, we are a platform company. We work with everybody, right? Our technology is in each of the clouds. And the clouds all have great offerings. So you could ask yourself, why is NVIDIA, why do we produce the whole stack? And the reason is because we consistently hear from customers that they want the choice. They want to choose, I'm running some workload on this cloud, some workload on that cloud. And we are the ones who say, well, our hardware is available everywhere. Our software is the same software everywhere. So you get a consistent experience, right? So now how does that dovetail with HPE? I think HPE made a bet a few years ago with GreenLake to say, on the infrastructure side, we want to' be the people who say to the customer, "We'll meet you wherever you want to be met." And so there's a very natural sort of, you know, synergy between them and us, right? Because we want our technology to be everywhere, and they want people to consume infrastructure everywhere. And so it just fits, so, yeah.

>> You know, you made the point that you do work with a lot of different providers. And here you are at HPE, but there's other technology conferences going around and your boss Jensen is flying all over the world. How does that work internally? Because we know that having a partnership, it really does matter so much, in terms of your alignment on values, and priorities, and your commitment to sustainability, and things like that. How are you as NVIDIA, as a company, making sure that you are melding with all of these different-

>> Rebecca, that's such a great question. I think there's two answers. The first is, we take a platform approach where we build one stack that works in lots of different places. So that's what makes it practical to work with many partners. But the other word that I think is much more important is empathy. So the way we choose to work with a partner, let's say HPE, is what does HPE stand for? What is their customer base? Why do HPE's customers go to HPE? Okay, we'll work with you to produce a solution that fits into that model. On the other hand, if you're working with a different partner, they may have a completely different set of customers who think differently, who consume the technology differently. And in that case, we'll work with that partner in a different way. So we are not trying to take our square peg and put it into everybody's round holes. We go to each of them and say, "What does your round hole look like?" And we'll make our peg the right round for you. So that's kind of' how we look at it.

>> No, that's really interesting.

>> I think, again, we had Justin on yesterday and we were talking about Grace Hopper and what they're doing there. I think there seems to be more alignment than just at the hardware level. And I think sustainability is one of those places where NVIDIA takes it pretty seriously.

>> I think it's huge, you know, because I think for many years now, with the advent of the cloud, Rob, people have been very focused on the economic comparison between the public cloud and doing your own. And you know, there's the saying that if you're willing to do your own and you have the expertise, it'll be more cost effective than renting. But there's also that logic now for, you know, power consumption. And the thing about that we are really proud of at NVIDIA is these GPUs, you know. If you look at one GPU, it consumes more power than one CPU. But one GPU does the work of like 1,000 CPUs. So when you're actually running your workload, the most power efficient, energy efficient way to run your workload is on GPUs. And the reason we have these new generations, like Grace Hopper, is because we're constantly making the GPUs more and more power efficient. So we love what Justin's doing because he's actually delivering what looks to be on one unit, a more power hungry product, but on a workload level is actually more power efficient and it's going to' save the world a lot of money and a lot of energy consumption. So yeah, it's exciting.

>> It is, exciting times. Manuvir, thank you so much for coming on, for returning to "theCUBE", I should say.

>> Of course, it was my pleasure.

Until next time, I guess. >> Until next time, yeah. See you in Barcelona in 2024.

>> Likewise.

>> I'm Rebecca Knight. Stay tuned for more of "theCUBE's" live coverage. For Rob Strechay, you are watching "theCUBE", the leader in high tech enterprise coverage. (upbeat music)