HPE Discover 2024 | Neil MacDonald and Bob Pette discuss the significant expansion of the HPE and NVIDIA partnership

Clips
News
More from HPE Discover 2024

Neil MacDonald

Executive Vice President, Compute, HPC & AI

HPE

Bob Pette

Vice President and General Manager, Enterprise Platforms

NVIDIA

Three insights you might have missed from theCUBE’s coverage of HPE Discover

Artificial intelligence took center stage at HPE Discover last week. For three days, company executives captivated audiences with keynote sessions focused on Hewlett Packard Enterprise Co.’s cutting-edge AI solutions and the robust infrastructure it’s building to support them.New solutions for AI deployment and how HPE’s enterprise customers can take advantage of them were discussed at length by theCUBE, SiliconANGLE Media’s livestreaming studio, through interviews with company executives, clients and industry analysts at HPE’s signature event

HPE partners with Nvidia to support hybrid AI architecture for enterprises

Without a doubt, the winner of the generative artificial intelligence boom thus far has been Nvidia Corp. Now the company is joining forces with Hewlett Packard Enterprise Co. to give enterprises the complete AI stack. HPE and Nvidia have paired the former’s expertise in on-premises solutions with the latter’s cutting-edge hardware. Since enterprises often keep their private data on-prem, industry-specific models such as Nvidia inference microservices are the future, according to Neil MacDonald (pictured, right), executive vice president and general manager of the Compute Business Unit at HPE

play_circle_outline Revolutionizing Enterprise Productivity: NVIDIA and HPE Join Forces for Generative AI Deployment Success

play_circle_outline Maximizing Enterprise Transformation Through On-Prem Data and Industry-Specific Models in the Age of AI

play_circle_outline Ensuring CUDA Compatibility: Financial Options for Smooth Enterprise AI Expansion

play_circle_outline Importance of larger GPUs in the AI world

play_circle_outline Partnership and go-to-market strategy between NVIDIA and HPE

Info

Neil MacDonald and Bob Pette discuss the significant expansion of the HPE and NVIDIA partnership

Neil MacDonald

Executive Vice President, Compute, HPC & AI HPE

Bob Pette

Vice President and General Manager, Enterprise Platforms NVIDIA

search

>> Welcome back, everyone, to the theCUBE's live coverage here in Las Vegas for HPE Discover 2024. I'm John Furrier, host of theCUBE with Dave Vellante, my co-host, co-founder. It's our 14th year covering HP and HPE Discover. Got two great guests here, unpacking the big news with NVIDIA and HP, the strategic relationship, a deep relationship. Neil MacDonald, executive vice president HPC and AI for HPE. And Bob Pette, vice president and general manager for enterprise platforms at NVIDIA. Gentlemen, thank you for coming on and I think we had a great preamble. It wasn't on camera, but I think we had some good content coming out. Thanks for coming on.

Neil MacDonald

>> Yeah, glad to be here.

>> So, I got to say, this is unprecedented times, the world's being changed by generative AI, the new platform shift. It's different. Tokens, neural networks, vectors, these are the terms that Jensen's talking about, NIMs, a whole nother architecture's here. And you guys announced the partnership between NVIDIA. Neil, this is significant. This is not a barney deal, this is not a deal that's just a strategic relationship. There's profound impact. Go-to-market, there's technology. Take us through real quick the relationship and then we'll get into it.

Neil MacDonald

>> So, the announcement that we made this week is NVIDIA AI Computing by HPE, and our focus is really accelerating the next wave of deployment of generative AI. We have this technology that's going to change the world and impact all kinds of processes inside businesses, because fundamentally, what it's about is productivity. We believe that every enterprise is going to become, to some extent, driven by generic AI or they're going to become obsolete because they're going to be beaten by the competitors who do. So, the challenge is that, as great as all of this amazing innovation and technology is right now, enterprises have different needs. There are lots of solutions out there that are really reference architectures. That puts a huge burden on enterprise customers to curate and integrate different pieces of technology at the hardware level. And then, to curate and integrate all of the layers in the data stack and in the model stack before they can ever get to the experimentation on use cases and capture the competitive benefits of productivity. So, the wonderful thing that we are engaged on with our great partners here at NVIDIA is delivering a co-developed portfolio of products and solutions that enable enterprises to accelerate that journey because we take care of that for them. They can take these technologies, deploy them in a few clicks in less than 30 seconds, and get going on their journey of discovery and transformation, leveraging generative AI.

>> Bob, talk about the NVIDIA piece because we've been covering you guys for a while. And your conference, the AI systems front and center. I mean it's a data center in a box. It's a huge monster machine. But the appetite for generative AI is not just in the broad bigger market that goes down into the enterprise and it's different, but there's still huge demand.

Bob Pette

>> Massive demand. Historically, it's been in the hyperscaler market because that's where things have been trained, large language models have been trained, ChatGPT and the like. But the enterprise is where that intelligence will be gathered and obtained, and they require a different model. There's data privacy things to consider, they're GDPR things to be considered data governance and the like. As you move from hyperscaler technology into the enterprise, you'll still do fine-tuning. And so, our GPU and our CPU-GPU assortment covers, and what we announced here as part of the HPE Private Cloud for AI, is a slew of products from small, medium, large and extra large to address these different use cases. There'll be some training done, there'll be fine-tuning done, where you take an existing LLM or small LLM and you fine tune it for your particular industry or your particular corporation. Probably, most importantly is where you get into retrieval augmented generation, where you add your specific data to it. And then, the final step is in that inference process is true latency optimized inferencing for customer assistance, digital avatars, AI copilots for code-assist. But the Hopper systems and the Grace Hopper systems that we've announced here with HPE are state-of-the-art, will be timed to market with HPE when we're shipping Blackwell next year. Jensen's already talked about Rubin in 2026, and we're looking forward to the next couple of years with our partners here at HPE.

>> Well, one-year cycle. John points out, and John's the only person I know who times Jensen at events. And other than GTC where he spoke for hours, he was 25 minutes in the sphere, which is a new record, double roughly what we see at other shows. But the thing about Jensen is he always brings something new to these shows and we watch all his videos and it's like, wow, he's always got a new take. And this time it was three ingredients needed for AI, the model, the compute, and the data. And then, he talked about NIMs and NeMo and of course CUDA and other software, all three layers. So, really, again, always brings new concepts. How do those fit into what you guys announced?

Neil MacDonald

>> Well, any form of generative AI relies on data and enterprise data is on-prem. It's on-prem for reasons of confidentiality or protecting IP, it's on-prem for regulatory reasons, it's on-prem for any number of concerns. And so, any generative AI endeavor that's really going to transform business processes in the enterprise has a natural gravity to on-prem for part of that architecture. And as you know, we've long held that the world is going to be hybrid, and it's been borne out. And nowhere is it being borne out more than in generative AI. So, the challenge then becomes how do you make that easy for the enterprise to deal with? And that's where the three pillars come in. When you think about the computing stack, this isn't about taking an accelerator and a server and putting them together, and then putting that together with network and all the other pieces. It's not a kit that has to be assembled, like so many other offers in the market today. What we've done is co-developed a turnkey integrated solution that integrates that compute stack, including the storage that's needed and the ability to ingest data across a disparate enterprise into gen AI initiatives, leveraging our data fabric. But it's more than that. You need to be able to observe this infrastructure. You need to be able to curate and lifecycle manage it. So, we've done all of that too, but that's only the compute stack. When you think about the data that feeds into it, that's also part of what we've delivered. And most critically for the enterprise, it's about the models. So, Bob, maybe you can add a little bit about the model stack and how we've integrated profoundly into what we've done with Private Cloud AI.

Bob Pette

>> Absolutely. As we move from this instruction-led computing world that we've been in into intention-led computing, those three stacks are key. The compute stack, again, not just GPU, but an integrated approach that looks at latency to data and latency between CPU and GPU. Neil's covered that greatly. The data stack partnership, GreenLake for storage, all of the great storage that we have, integrated NeMo retriever. The slowest horse in the race is the one that's going to determine the length of the race, and if that's a query that someone has and timely feedback. So, knowing where the data is, making sure that it's guardrail protected, that's part of the data stack. All of that is built into NVIDIA AI Enterprise that they've incorporated into their stack. And then, the model stack is really based around NVIDIA Inference Microservices. These essentially are the runtime for models. We have a slew of models covering a variety of enterprise use cases across multiple industries, which is what Jensen refers to this as the next industrial revolution. We started out with the ability to detect, moved into healthcare and then trying to detect cancer, look for differences. Now, we're actually able to take multiple data sources and predict, and that's where that more and more intelligence is extracted. So, these NIMs, these containerized models can come from anywhere. People can download them from wherever they are, but we have a nightly NIM AI factory at NVIDIA that we continue to build these models, tune, test, optimize. We have replicated all of the servers and the systems that are part of NVIDIA AI Computing by HPE. We test them out there, so our enterprise customers are collective customers, know they're getting the latest state-of-the-art models. And so, models are the key. They need to get to the data and they need to run on the infrastructure that's going to let them truly at performance for time to insight, time to value.

>> I want to dig into that because this is a really nuanced point, but it's super important. NIMs from NVIDIA, that's your inference microservice, which is great. That means that clients could build their own models.

Bob Pette

>> Yes.

>> So, this is a key part. Although there are a lot of model opportunities out there, open source whatnot. So, that's important to point out. I want to bring this up because all the people we talk to in our research, in our conversations says they use the proprietary and or large open source model for things, some content, some queries, some retrieval. But when you talk about their data, they want a simple and easy way to stand up infrastructure. Kind of like the old cloud days. "I want to do a SaaS app. I just spin up credit card and I get some services." They want the same thing for their data because that's their crown jewel.

Bob Pette

>> Yep, exactly.

>> And so, take us through what you guys see there because that is different. It might be smaller, it might be different sizes, but that's a key part of their IP and the workloads are end to end, so you know a couple things. I know my data and I know my workload. How does that factor into this?

Neil MacDonald

>> Well, that's kind of the key for the enterprise because a large language model with some level of general intelligence is one thing, but if you apply that in your business processes, you're not going to get the results you want in terms of quality because it doesn't have the specific intelligence about your business or your industry. So, this is why for enterprises to be successful with generative AI, they have to pursue either fine-tuning a model based on their own data, which they're going to do on-prem, or leveraging retrieval augmented generation where the generation process is leveraging that data from inside the enterprise. But to do that, enterprise data is very fragmented. And so, that's why we've integrated into private cloud AI our data fabric that enables you to pull that together from those disparate sources in order to underpin that work. But that tuning, or RAG, or the development of small specialized models in that company or industry is really critical to get the level of impact that's needed.

Bob Pette

>> And I think that's the transformation that we're seeing. You're not going to use ChatGPT to optimize your supply chain. It might give you some hints, but there are hints that it's scraped from the internet.

Neil MacDonald

>> And everything on the internet's true, right?

>> Yeah, everything.

Bob Pette

>> You want your data, you want your SAP system, Oracle system, your Salesforce system. You want all of your information to be a part of that model. And some of that is done via RAG, but a lot of it is done to an industry-specific model that is developed that we see this today in climate and geospatial large language models, in financial large language models. And when I say large, I'm not talking about trillions of parameters, but billions or hundreds of billions of parameters. By industry, you'll see that and we are seeing that happen. To your point, that's the transformation. So, I take that industry-specific one, then I make it corporation-specific with my semantics, my words, so that when it goes to get data that is in my databases, it can make the correlation. It has the same semantics, the same language, the same set of information. So, that's the big transformation. And that's part of what NVIDIA AI Enterprise, HP AI Essentials and the NIMs are meant to do is help companies go from these truly large language models down to models that they're going to work with on a regular basis to improve their business.

Neil MacDonald

>> Yeah. You mentioned RAG, obviously retrieval augmentation generations is what it means for the folks who don't know what it means, but that's a lot of text. There's also tokens can be used for multimodal, so you got images, PDFs, X-rays, DNA,

Bob Pette

>> Yeah, full multimodal text.

>> So, that's an important point. But again, that's part of the new system, which brings up next question, which is can you guys highlight or illuminate the difference between old-school data processing, which I could do in the cloud by the way. Cloud's great for data processing. Put data in the cloud and process it. This is not data processing. It's neural network. It's a different data source, either fragmented or different data states. Scope that problem and opportunity. How do you guys look at that?

Bob Pette

>> Yeah, I'll tee it up here if I can. This gets back to this, I don't want to call it a soundbite, but it's really important. Instruction-led computing, means the programmers have an idea of what they're trying to find and they go through the code in an order of if, then, else, to see what's there with the hopeful outcome of coming up with something important, and that's served us well to date. Intention-led computing goes well beyond that. And so, it's not so much about how I'm going to crunch data, it's about what I'm trying to do. And AI learns that based on the questions that you're probing the model with, based on the data that you're exposing it to. It will not only answer your questions, but it understands what your second question will likely be. It's learning to write code on its own, and so, it knows your intentions. And we've seen this time and time again for the years that we've been working with AI and NVIDIA, we are constantly surprised at what it can do. We may train it to do one thing and we turn it onto a different modality and we're shocked. We're shocked. And so, that's the true power is not just getting the answers that you want, but getting the answers that maybe you should be asking, knowing that your intended goal is to save lives, improve the climate, improve the supply chain, increase sales, whatever it may be.

Neil MacDonald

>> Think about an enterprise example. You've got a B2B business. You're responding to request for proposals. In an instruction-led world, the way we would run all of our IT in the past, you're trying to come up with a perfect algorithm to write all of that. In an intentional-led world, you leverage HPE Private Cloud AI as part of the NVIDIA AI Computing by HPE, and you take your database of all the responses you've ever generated in your business, and then you have the system generate your next response. So, by doing that, what have you done? You've made your sales teams and your response teams, your pursuit teams vastly more productive, so you can cover more opportunities. You've gotten the best practices out of what's worked for you in the past, the deals you've won, the deals you've lost, and you've factored that intelligence in a systematic way into the business. So, that's transformational. If you're competing with another organization who's trying to do this all by hand, you're going to be faster to respond, you're going to be better at leveraging your best practice, you're going to win more. And that's the power of transformation. The challenge is how do you enable enterprises to do that without having to become gurus in the compute stack, the data stack and the model stack? Because the architecture is so different from traditional enterprise IT-

Bob Pette

>> Very complex.

Neil MacDonald

>> And that's why we're so excited about what we're doing together with NVIDIA.

>> The reason why this is so important, this message that you're sending about solutions is that cycles are compressing, everything's compressing. And I want to get your perspective on this, kind of a multi-part question, but I'll start with one. So, we're seeing a one-year cadence rhythm, I think Jensen calls it, right? H100s are becoming a little bit more plentiful and then can't wait to get Blackwell. We already see Rubin announced. What does that mean from your standpoint in terms of managing the lifecycles, which you used to two, maybe it was even extended tape outs were stretching out to two to three, sometimes even four years. Now, it's like this one-year rhythm. What does that mean for you?

Neil MacDonald

>> Well, the first thing it means is aligning the amazing engineering teams at HPE rigorously with NVIDIA's cadence. And we're thrilled to have announced our first Gen 12 servers here, then the HPE ProLiant line, which are based on NVIDIA Superchips. So, what you can expect to see moving forward is incredibly tight alignment on that development, and we're committed to ensuring that HP and NVIDIA are together at time to market on all of those transitions. So, that's the first thing that it means. The second thing it means is thinking beyond the system. Sometimes in traditional enterprise IT, we've thought of it like a server, it's a box. The architecture is not about the individual nodes anymore. It's not even about the individual accelerators anymore. It's about the whole system architecture that brings together all of the capabilities in that compute stack, the data stack and the model stack that you need to be effective with generative AI. So, another shift here, both for our partnership and our collaboration, but also, for our enterprise customers, is thinking at that system level in a much, much more profound and cohesive way. Bob?

Bob Pette

>> Yeah, and I think what will tie these architectures together in a way that customers don't feel like they're going to be missing out is CUDA compatibility. We'll guarantee that what works today will work on Blackwell, will work on Rubin and run faster, do more, do better. And in working in concert, early on before we were releasing Blackwell and Rubin systems, we'll be testing out all of the software, not just our models and CUDA, but also all the HPE software. And from a lifecycle management standpoint, one of the reasons we partner with HPE, besides their broad reach, they've got financial options to make it easy for people to transition. Some people might want Rubin today, you always want the latest thing, but do you want to improve your business today or do you want to wait till 2026? And so it's important that A, they know the code's going to run. The code will run it's compatible, will continue to optimize, and they know that there are financial options to not only expand. I doubt that they'll replace those Hopper systems in the Rubin timeframe, they'll just add Rubin.

>> Well, we have new law, we had Moore's law, now we have Jensen's law, which is spend more, save more. But I want to ask you, so after GTC, we took the chart that Jensen had and we did the calculation. 1,000X improvement in eight years, we superimposed Moore's law, which was 100X in 10 years, and you guys actually stole that in Taiwan, our chart. It wasn't really our chart. You did your own version.

Bob Pette

>> Yeah, it was a great chart.

>> The reason I ask that... And we understand, thank you, we understand the importance of systems, but it does come back to that core, that GPU, we need bigger GPUs. And when you look at Blackwell and what you did with NVLink and to keep that lead. Everybody's saying, "Okay, they're the most valuable company in the world. They're going to get all this competition. How long is that lead?" When you look at Blackwell and the things that you did, I'd love to ask you, if you can answer, most people from NVIDIA are super smart guys, as we say in Boston. So, maybe you can answer this, maybe you can't. But we noticed reading about it, learning about it, that you had a little trick in there. You dialed down the floating point to four point granularity as opposed to eight point, which saves power, it saves money. Brilliant. And it's the same TSM process. So, we're like, "Wow, what happens when NVIDIA moves to a new node and then dials things back up?" So, you have a lot of flexibility there. What can you tell us about what's coming that roadmap without obviously giving away too much?

Bob Pette

>> Yeah, I'm not going to give away too much, but the level whether you need N4, N8, FP8, FP16, FP32 depends on the workload. Our R&D group is doing dramatic results. We're getting what used to require FP16 now runs in N4 at the same level of accuracy. This is important because now I can fit more data, larger models into a frame buffer size that I need to run that model. So, I may not need the hyperscaler level of model. I can put it into an extra large HPE Private Cloud for AI system. So, we'll make trade-offs every generation, we do this historically. The need for a floating point in supercomputing was significant, and the need to reserve space for FP64, which we'll always have or FP32 calculations we'll always have. When you get into the tensor world, the AI world, you want to try to squeeze as far as you can. If we could get to an inch one level-

>> Wow, .

Bob Pette

>> I'm not saying we would, but if we could, we would. Why wouldn't you, right? If we could make the same prediction in the same or faster timeframe with less precision using up less memory, you're going to do that and we'll continue to innovate that way. Will there be some things that hope that some of that floating point was there? Yes, but they'll still run faster. It's just the AI will run 100 times faster.

>> And so, Neil, you have to obviously understand the roadmap and put the right strategic fit for your portfolio, and you got to do that in a much faster timeframe. It's a fascinating dynamic.

Neil MacDonald

>> Well, it is a wonderful time to be in the systems world because it's such an incredible rate of innovation coming from our partners, NVIDIA, and we're thrilled to be there, time to market with every one of these transitions. The key though is if we languish in the detail of exactly what's in a piece of silicon, we're having the wrong conversation with enterprises because we shouldn't be burdening customers with having to understand the whole technical detail of the stack. That's the power of our focus and our approach. They are just going to have access to more performant, more capable systems that lets them deploy a larger number of generative AI use cases inside their enterprise.

Bob Pette

>> Yeah, we don't want them thinking about N4-

Neil MacDonald

>> You've got to extract that, but we still want to know. But I think the other thing that we need to talk about though is that not just is the architecture chaining and the structure of how these systems put together changing, but we're also heading into a very, very different power regime in addition to the efficiency. So, maybe Bob, you should share some of the color on that.

Bob Pette

>> Yeah, so much like we looked to optimize model accuracy by reducing the number of bytes required, we're taking power up. This is the way it's going to work. We need bigger and bigger power systems. You saw that from Hopper to Blackwell, let's assume that that will continue. That doesn't mean that we're going to run out of power because while we may be taking power up 2X or 3X, if the performance is going up well beyond that, it still is a more efficient system. So, if we can improve, say Llama-3-70B 5X with our NIMs running on a system today at a certain power level, when that power goes up and you get that 5X improvement, it still is going to be saving you money. The more you buy it, the more you save. So, in the same power envelope, you're going to get more done. But with power constantly going up to produce more intelligence from the data that it's being fed, we do know that there's only so much space in so many data centers. And so, that's one of the main reasons we partner with HPE on this announcement is they're experts in liquid cooling, whether it's 70% liquid cooled or direct liquid cooled. So, we're confident in taking the power up because it's still going to be a profitable formula for our customers. They're still going to save money, and especially with the liquid cooling piece, they'll save even more. From a power standpoint, you guys can reuse that heat that's pulled out and heath buildings in winters and the like. So, we're confident that people say GPUs are power hungry, I think customers are information hungry.

Neil MacDonald

>> They're intelligence hungry, right?

Bob Pette

>> Intelligence hungry, and so we're feeding that intelligence hungry with our-

>> They're outcome hungry. They want the outcomes they want.

Bob Pette

>> They're outcome hungry.

Neil MacDonald

>> Well, exactly. And the key to this at the end of the day is we're moving into a regime that's going to have to be 100% liquid cooled. And in that world where you're multi-kilowatt sockets, there's an alternative. At HPE, we're blessed to have all of the experience from decades of Cray building 100% liquid-cooled systems. You can see them here on the show floor at Discover with the Venado system at Los Alamos, the Jensen and Antonio inaugurated just a few short weeks ago. So, we're really thrilled that all of that experience is coming to bear right at the heart of this enterprise generative AI journey.

>> Dare I say democratization of supercomputing? But I won't say it. Well, I think that last conversation about getting into the details proves that the engineering's different, but by design, has to be different. We're in a different world. You guys take care of that on behalf of the customer. So, I have to ask on our last minute we have so left, on the go-to-market together, this is a significant part of the relationship-

Neil MacDonald

>> Yep, very much so....

>> could you quickly talk about how that's going to work? Antonio was here on theCUBE, the CEO of HP said, "We're going to go as one company." Jensen was texting him last night. They had a great time on stage. What is the go-to-market? Obviously, we saw the simplicity, four clicks, you're up and running. What is the message? What's the comp? Who's getting paid? You guys holding hands together? Is it a joint solution? What's the go-to-market?

Neil MacDonald

>> So, the first thing that we're doing is we're training and certifying our entire enterprise Salesforce on NVIDIA technologies as part of NVIDIA AI Computing by HPE, so that we can go bring that benefit of Private Cloud AI to all of our enterprise customers, but we're not stopping there. We're also enabling certification and training in our channel community, which is such a core set of partners for both of us in expanding reach and being able to bring these competitive benefits, in a democratized way, to the broad sweep of enterprises. And then, of course, we're working with NVIDIA in the field. Bob?

Bob Pette

>> Yeah, we started with the product group. So, my product group is actually just focused on the enterprise and just focused on the OEMs because of that huge opportunity, most of the AI is going to be there. So, we'll continue to make sure we're innovating at that level versus being distracted with trying to serve a bunch of different communities via one product group. The text that Antonio and Jensen shared, Jensen sent this morning and he said, "I want a tiger team pulled together." He sent it to our head of sales, myself, a couple of my peers, and said, "Let's get a tiger team together and figure out the next 90 days. I want to know every go-to-market activity." And so, he's a genius at pulling that together and holding us accountable. There'll be joint go-to-market campaigns that we do together. We'll be making sure that their customer briefing centers are fully equipped with our technology. Our sales force, which is much, much smaller than their sales force, we'll be working with them on Fortune 1,000 accounts, but it'll be a detailed plan that Jensen wants to review on a regular basis and I'm sure with Antonio.

>> Awesome. That's great.

>> Being in the valley for 25 years, I can tell you personally been following Jensen's career going back, and I know a lot of the dynamics. He's an entrepreneur. I think he really respects, and I saw a little bit on stage, to HP's culture. And he's a systems guy. HP's a technical company. I think it's a good fit. And congratulations on the relationship.

Bob Pette

>> Yeah, thank you.

Neil MacDonald

>> Well, we're excited to be working with the best partner in the space, and excited to bring that value to our customers.

Neil MacDonald

>> And we're timing the CEOs on keynotes. We're going to see how they do around the track. They're tech athletes. We want to see their speed. Dave, we're going to track it. All right, you watching theCUBE here. Bringing all the action. NVIDIA and HP, a really monumental partnership. We're going to keep tracking it here on the theCUBE. We're bringing all you the data here. We got our own NIM, it's called theCUBE data. Stay with us for more after this short break.

Neil MacDonald and Bob Pette discuss the significant expansion of the HPE and NVIDIA partnership

search

>> Welcome back, everyone, to the theCUBE's live coverage here in Las Vegas for HPE Discover 2024. I'm John Furrier, host of theCUBE with Dave Vellante, my co-host, co-founder. It's our 14th year covering HP and HPE Discover. Got two great guests here, unpacking the big news with NVIDIA and HP, the strategic relationship, a deep relationship. Neil MacDonald, executive vice president HPC and AI for HPE. And Bob Pette, vice president and general manager for enterprise platforms at NVIDIA. Gentlemen, thank you for coming on and I think we had a great preamble. It wasn't on camera, but I think we had some good content coming out. Thanks for coming on.

Neil MacDonald

>> Yeah, glad to be here.

>> So, I got to say, this is unprecedented times, the world's being changed by generative AI, the new platform shift. It's different. Tokens, neural networks, vectors, these are the terms that Jensen's talking about, NIMs, a whole nother architecture's here. And you guys announced the partnership between NVIDIA. Neil, this is significant. This is not a barney deal, this is not a deal that's just a strategic relationship. There's profound impact. Go-to-market, there's technology. Take us through real quick the relationship and then we'll get into it.

Neil MacDonald

>> So, the announcement that we made this week is NVIDIA AI Computing by HPE, and our focus is really accelerating the next wave of deployment of generative AI. We have this technology that's going to change the world and impact all kinds of processes inside businesses, because fundamentally, what it's about is productivity. We believe that every enterprise is going to become, to some extent, driven by generic AI or they're going to become obsolete because they're going to be beaten by the competitors who do. So, the challenge is that, as great as all of this amazing innovation and technology is right now, enterprises have different needs. There are lots of solutions out there that are really reference architectures. That puts a huge burden on enterprise customers to curate and integrate different pieces of technology at the hardware level. And then, to curate and integrate all of the layers in the data stack and in the model stack before they can ever get to the experimentation on use cases and capture the competitive benefits of productivity. So, the wonderful thing that we are engaged on with our great partners here at NVIDIA is delivering a co-developed portfolio of products and solutions that enable enterprises to accelerate that journey because we take care of that for them. They can take these technologies, deploy them in a few clicks in less than 30 seconds, and get going on their journey of discovery and transformation, leveraging generative AI.

>> Bob, talk about the NVIDIA piece because we've been covering you guys for a while. And your conference, the AI systems front and center. I mean it's a data center in a box. It's a huge monster machine. But the appetite for generative AI is not just in the broad bigger market that goes down into the enterprise and it's different, but there's still huge demand.

Bob Pette

>> Massive demand. Historically, it's been in the hyperscaler market because that's where things have been trained, large language models have been trained, ChatGPT and the like. But the enterprise is where that intelligence will be gathered and obtained, and they require a different model. There's data privacy things to consider, they're GDPR things to be considered data governance and the like. As you move from hyperscaler technology into the enterprise, you'll still do fine-tuning. And so, our GPU and our CPU-GPU assortment covers, and what we announced here as part of the HPE Private Cloud for AI, is a slew of products from small, medium, large and extra large to address these different use cases. There'll be some training done, there'll be fine-tuning done, where you take an existing LLM or small LLM and you fine tune it for your particular industry or your particular corporation. Probably, most importantly is where you get into retrieval augmented generation, where you add your specific data to it. And then, the final step is in that inference process is true latency optimized inferencing for customer assistance, digital avatars, AI copilots for code-assist. But the Hopper systems and the Grace Hopper systems that we've announced here with HPE are state-of-the-art, will be timed to market with HPE when we're shipping Blackwell next year. Jensen's already talked about Rubin in 2026, and we're looking forward to the next couple of years with our partners here at HPE.

>> Well, one-year cycle. John points out, and John's the only person I know who times Jensen at events. And other than GTC where he spoke for hours, he was 25 minutes in the sphere, which is a new record, double roughly what we see at other shows. But the thing about Jensen is he always brings something new to these shows and we watch all his videos and it's like, wow, he's always got a new take. And this time it was three ingredients needed for AI, the model, the compute, and the data. And then, he talked about NIMs and NeMo and of course CUDA and other software, all three layers. So, really, again, always brings new concepts. How do those fit into what you guys announced?

Neil MacDonald

>> Well, any form of generative AI relies on data and enterprise data is on-prem. It's on-prem for reasons of confidentiality or protecting IP, it's on-prem for regulatory reasons, it's on-prem for any number of concerns. And so, any generative AI endeavor that's really going to transform business processes in the enterprise has a natural gravity to on-prem for part of that architecture. And as you know, we've long held that the world is going to be hybrid, and it's been borne out. And nowhere is it being borne out more than in generative AI. So, the challenge then becomes how do you make that easy for the enterprise to deal with? And that's where the three pillars come in. When you think about the computing stack, this isn't about taking an accelerator and a server and putting them together, and then putting that together with network and all the other pieces. It's not a kit that has to be assembled, like so many other offers in the market today. What we've done is co-developed a turnkey integrated solution that integrates that compute stack, including the storage that's needed and the ability to ingest data across a disparate enterprise into gen AI initiatives, leveraging our data fabric. But it's more than that. You need to be able to observe this infrastructure. You need to be able to curate and lifecycle manage it. So, we've done all of that too, but that's only the compute stack. When you think about the data that feeds into it, that's also part of what we've delivered. And most critically for the enterprise, it's about the models. So, Bob, maybe you can add a little bit about the model stack and how we've integrated profoundly into what we've done with Private Cloud AI.

Bob Pette

>> Absolutely. As we move from this instruction-led computing world that we've been in into intention-led computing, those three stacks are key. The compute stack, again, not just GPU, but an integrated approach that looks at latency to data and latency between CPU and GPU. Neil's covered that greatly. The data stack partnership, GreenLake for storage, all of the great storage that we have, integrated NeMo retriever. The slowest horse in the race is the one that's going to determine the length of the race, and if that's a query that someone has and timely feedback. So, knowing where the data is, making sure that it's guardrail protected, that's part of the data stack. All of that is built into NVIDIA AI Enterprise that they've incorporated into their stack. And then, the model stack is really based around NVIDIA Inference Microservices. These essentially are the runtime for models. We have a slew of models covering a variety of enterprise use cases across multiple industries, which is what Jensen refers to this as the next industrial revolution. We started out with the ability to detect, moved into healthcare and then trying to detect cancer, look for differences. Now, we're actually able to take multiple data sources and predict, and that's where that more and more intelligence is extracted. So, these NIMs, these containerized models can come from anywhere. People can download them from wherever they are, but we have a nightly NIM AI factory at NVIDIA that we continue to build these models, tune, test, optimize. We have replicated all of the servers and the systems that are part of NVIDIA AI Computing by HPE. We test them out there, so our enterprise customers are collective customers, know they're getting the latest state-of-the-art models. And so, models are the key. They need to get to the data and they need to run on the infrastructure that's going to let them truly at performance for time to insight, time to value.

>> I want to dig into that because this is a really nuanced point, but it's super important. NIMs from NVIDIA, that's your inference microservice, which is great. That means that clients could build their own models.

Bob Pette

>> Yes.

>> So, this is a key part. Although there are a lot of model opportunities out there, open source whatnot. So, that's important to point out. I want to bring this up because all the people we talk to in our research, in our conversations says they use the proprietary and or large open source model for things, some content, some queries, some retrieval. But when you talk about their data, they want a simple and easy way to stand up infrastructure. Kind of like the old cloud days. "I want to do a SaaS app. I just spin up credit card and I get some services." They want the same thing for their data because that's their crown jewel.

Bob Pette

>> Yep, exactly.

>> And so, take us through what you guys see there because that is different. It might be smaller, it might be different sizes, but that's a key part of their IP and the workloads are end to end, so you know a couple things. I know my data and I know my workload. How does that factor into this?

Neil MacDonald

>> Well, that's kind of the key for the enterprise because a large language model with some level of general intelligence is one thing, but if you apply that in your business processes, you're not going to get the results you want in terms of quality because it doesn't have the specific intelligence about your business or your industry. So, this is why for enterprises to be successful with generative AI, they have to pursue either fine-tuning a model based on their own data, which they're going to do on-prem, or leveraging retrieval augmented generation where the generation process is leveraging that data from inside the enterprise. But to do that, enterprise data is very fragmented. And so, that's why we've integrated into private cloud AI our data fabric that enables you to pull that together from those disparate sources in order to underpin that work. But that tuning, or RAG, or the development of small specialized models in that company or industry is really critical to get the level of impact that's needed.

Bob Pette

>> And I think that's the transformation that we're seeing. You're not going to use ChatGPT to optimize your supply chain. It might give you some hints, but there are hints that it's scraped from the internet.

Neil MacDonald

>> And everything on the internet's true, right?

>> Yeah, everything.

Bob Pette

>> You want your data, you want your SAP system, Oracle system, your Salesforce system. You want all of your information to be a part of that model. And some of that is done via RAG, but a lot of it is done to an industry-specific model that is developed that we see this today in climate and geospatial large language models, in financial large language models. And when I say large, I'm not talking about trillions of parameters, but billions or hundreds of billions of parameters. By industry, you'll see that and we are seeing that happen. To your point, that's the transformation. So, I take that industry-specific one, then I make it corporation-specific with my semantics, my words, so that when it goes to get data that is in my databases, it can make the correlation. It has the same semantics, the same language, the same set of information. So, that's the big transformation. And that's part of what NVIDIA AI Enterprise, HP AI Essentials and the NIMs are meant to do is help companies go from these truly large language models down to models that they're going to work with on a regular basis to improve their business.

Neil MacDonald

>> Yeah. You mentioned RAG, obviously retrieval augmentation generations is what it means for the folks who don't know what it means, but that's a lot of text. There's also tokens can be used for multimodal, so you got images, PDFs, X-rays, DNA,

Bob Pette

>> Yeah, full multimodal text.

>> So, that's an important point. But again, that's part of the new system, which brings up next question, which is can you guys highlight or illuminate the difference between old-school data processing, which I could do in the cloud by the way. Cloud's great for data processing. Put data in the cloud and process it. This is not data processing. It's neural network. It's a different data source, either fragmented or different data states. Scope that problem and opportunity. How do you guys look at that?

Bob Pette

>> Yeah, I'll tee it up here if I can. This gets back to this, I don't want to call it a soundbite, but it's really important. Instruction-led computing, means the programmers have an idea of what they're trying to find and they go through the code in an order of if, then, else, to see what's there with the hopeful outcome of coming up with something important, and that's served us well to date. Intention-led computing goes well beyond that. And so, it's not so much about how I'm going to crunch data, it's about what I'm trying to do. And AI learns that based on the questions that you're probing the model with, based on the data that you're exposing it to. It will not only answer your questions, but it understands what your second question will likely be. It's learning to write code on its own, and so, it knows your intentions. And we've seen this time and time again for the years that we've been working with AI and NVIDIA, we are constantly surprised at what it can do. We may train it to do one thing and we turn it onto a different modality and we're shocked. We're shocked. And so, that's the true power is not just getting the answers that you want, but getting the answers that maybe you should be asking, knowing that your intended goal is to save lives, improve the climate, improve the supply chain, increase sales, whatever it may be.

Neil MacDonald

>> Think about an enterprise example. You've got a B2B business. You're responding to request for proposals. In an instruction-led world, the way we would run all of our IT in the past, you're trying to come up with a perfect algorithm to write all of that. In an intentional-led world, you leverage HPE Private Cloud AI as part of the NVIDIA AI Computing by HPE, and you take your database of all the responses you've ever generated in your business, and then you have the system generate your next response. So, by doing that, what have you done? You've made your sales teams and your response teams, your pursuit teams vastly more productive, so you can cover more opportunities. You've gotten the best practices out of what's worked for you in the past, the deals you've won, the deals you've lost, and you've factored that intelligence in a systematic way into the business. So, that's transformational. If you're competing with another organization who's trying to do this all by hand, you're going to be faster to respond, you're going to be better at leveraging your best practice, you're going to win more. And that's the power of transformation. The challenge is how do you enable enterprises to do that without having to become gurus in the compute stack, the data stack and the model stack? Because the architecture is so different from traditional enterprise IT-

Bob Pette

>> Very complex.

Neil MacDonald

>> And that's why we're so excited about what we're doing together with NVIDIA.

>> The reason why this is so important, this message that you're sending about solutions is that cycles are compressing, everything's compressing. And I want to get your perspective on this, kind of a multi-part question, but I'll start with one. So, we're seeing a one-year cadence rhythm, I think Jensen calls it, right? H100s are becoming a little bit more plentiful and then can't wait to get Blackwell. We already see Rubin announced. What does that mean from your standpoint in terms of managing the lifecycles, which you used to two, maybe it was even extended tape outs were stretching out to two to three, sometimes even four years. Now, it's like this one-year rhythm. What does that mean for you?

Neil MacDonald

>> Well, the first thing it means is aligning the amazing engineering teams at HPE rigorously with NVIDIA's cadence. And we're thrilled to have announced our first Gen 12 servers here, then the HPE ProLiant line, which are based on NVIDIA Superchips. So, what you can expect to see moving forward is incredibly tight alignment on that development, and we're committed to ensuring that HP and NVIDIA are together at time to market on all of those transitions. So, that's the first thing that it means. The second thing it means is thinking beyond the system. Sometimes in traditional enterprise IT, we've thought of it like a server, it's a box. The architecture is not about the individual nodes anymore. It's not even about the individual accelerators anymore. It's about the whole system architecture that brings together all of the capabilities in that compute stack, the data stack and the model stack that you need to be effective with generative AI. So, another shift here, both for our partnership and our collaboration, but also, for our enterprise customers, is thinking at that system level in a much, much more profound and cohesive way. Bob?

Bob Pette

>> Yeah, and I think what will tie these architectures together in a way that customers don't feel like they're going to be missing out is CUDA compatibility. We'll guarantee that what works today will work on Blackwell, will work on Rubin and run faster, do more, do better. And in working in concert, early on before we were releasing Blackwell and Rubin systems, we'll be testing out all of the software, not just our models and CUDA, but also all the HPE software. And from a lifecycle management standpoint, one of the reasons we partner with HPE, besides their broad reach, they've got financial options to make it easy for people to transition. Some people might want Rubin today, you always want the latest thing, but do you want to improve your business today or do you want to wait till 2026? And so it's important that A, they know the code's going to run. The code will run it's compatible, will continue to optimize, and they know that there are financial options to not only expand. I doubt that they'll replace those Hopper systems in the Rubin timeframe, they'll just add Rubin.

>> Well, we have new law, we had Moore's law, now we have Jensen's law, which is spend more, save more. But I want to ask you, so after GTC, we took the chart that Jensen had and we did the calculation. 1,000X improvement in eight years, we superimposed Moore's law, which was 100X in 10 years, and you guys actually stole that in Taiwan, our chart. It wasn't really our chart. You did your own version.

Bob Pette

>> Yeah, it was a great chart.

>> The reason I ask that... And we understand, thank you, we understand the importance of systems, but it does come back to that core, that GPU, we need bigger GPUs. And when you look at Blackwell and what you did with NVLink and to keep that lead. Everybody's saying, "Okay, they're the most valuable company in the world. They're going to get all this competition. How long is that lead?" When you look at Blackwell and the things that you did, I'd love to ask you, if you can answer, most people from NVIDIA are super smart guys, as we say in Boston. So, maybe you can answer this, maybe you can't. But we noticed reading about it, learning about it, that you had a little trick in there. You dialed down the floating point to four point granularity as opposed to eight point, which saves power, it saves money. Brilliant. And it's the same TSM process. So, we're like, "Wow, what happens when NVIDIA moves to a new node and then dials things back up?" So, you have a lot of flexibility there. What can you tell us about what's coming that roadmap without obviously giving away too much?

Bob Pette

>> Yeah, I'm not going to give away too much, but the level whether you need N4, N8, FP8, FP16, FP32 depends on the workload. Our R&D group is doing dramatic results. We're getting what used to require FP16 now runs in N4 at the same level of accuracy. This is important because now I can fit more data, larger models into a frame buffer size that I need to run that model. So, I may not need the hyperscaler level of model. I can put it into an extra large HPE Private Cloud for AI system. So, we'll make trade-offs every generation, we do this historically. The need for a floating point in supercomputing was significant, and the need to reserve space for FP64, which we'll always have or FP32 calculations we'll always have. When you get into the tensor world, the AI world, you want to try to squeeze as far as you can. If we could get to an inch one level-

>> Wow, .

Bob Pette

>> I'm not saying we would, but if we could, we would. Why wouldn't you, right? If we could make the same prediction in the same or faster timeframe with less precision using up less memory, you're going to do that and we'll continue to innovate that way. Will there be some things that hope that some of that floating point was there? Yes, but they'll still run faster. It's just the AI will run 100 times faster.

>> And so, Neil, you have to obviously understand the roadmap and put the right strategic fit for your portfolio, and you got to do that in a much faster timeframe. It's a fascinating dynamic.

Neil MacDonald

>> Well, it is a wonderful time to be in the systems world because it's such an incredible rate of innovation coming from our partners, NVIDIA, and we're thrilled to be there, time to market with every one of these transitions. The key though is if we languish in the detail of exactly what's in a piece of silicon, we're having the wrong conversation with enterprises because we shouldn't be burdening customers with having to understand the whole technical detail of the stack. That's the power of our focus and our approach. They are just going to have access to more performant, more capable systems that lets them deploy a larger number of generative AI use cases inside their enterprise.

Bob Pette

>> Yeah, we don't want them thinking about N4-

Neil MacDonald

>> You've got to extract that, but we still want to know. But I think the other thing that we need to talk about though is that not just is the architecture chaining and the structure of how these systems put together changing, but we're also heading into a very, very different power regime in addition to the efficiency. So, maybe Bob, you should share some of the color on that.

Bob Pette

>> Yeah, so much like we looked to optimize model accuracy by reducing the number of bytes required, we're taking power up. This is the way it's going to work. We need bigger and bigger power systems. You saw that from Hopper to Blackwell, let's assume that that will continue. That doesn't mean that we're going to run out of power because while we may be taking power up 2X or 3X, if the performance is going up well beyond that, it still is a more efficient system. So, if we can improve, say Llama-3-70B 5X with our NIMs running on a system today at a certain power level, when that power goes up and you get that 5X improvement, it still is going to be saving you money. The more you buy it, the more you save. So, in the same power envelope, you're going to get more done. But with power constantly going up to produce more intelligence from the data that it's being fed, we do know that there's only so much space in so many data centers. And so, that's one of the main reasons we partner with HPE on this announcement is they're experts in liquid cooling, whether it's 70% liquid cooled or direct liquid cooled. So, we're confident in taking the power up because it's still going to be a profitable formula for our customers. They're still going to save money, and especially with the liquid cooling piece, they'll save even more. From a power standpoint, you guys can reuse that heat that's pulled out and heath buildings in winters and the like. So, we're confident that people say GPUs are power hungry, I think customers are information hungry.

Neil MacDonald

>> They're intelligence hungry, right?

Bob Pette

>> Intelligence hungry, and so we're feeding that intelligence hungry with our-

>> They're outcome hungry. They want the outcomes they want.

Bob Pette

>> They're outcome hungry.

Neil MacDonald

>> Well, exactly. And the key to this at the end of the day is we're moving into a regime that's going to have to be 100% liquid cooled. And in that world where you're multi-kilowatt sockets, there's an alternative. At HPE, we're blessed to have all of the experience from decades of Cray building 100% liquid-cooled systems. You can see them here on the show floor at Discover with the Venado system at Los Alamos, the Jensen and Antonio inaugurated just a few short weeks ago. So, we're really thrilled that all of that experience is coming to bear right at the heart of this enterprise generative AI journey.

>> Dare I say democratization of supercomputing? But I won't say it. Well, I think that last conversation about getting into the details proves that the engineering's different, but by design, has to be different. We're in a different world. You guys take care of that on behalf of the customer. So, I have to ask on our last minute we have so left, on the go-to-market together, this is a significant part of the relationship-

Neil MacDonald

>> Yep, very much so....

>> could you quickly talk about how that's going to work? Antonio was here on theCUBE, the CEO of HP said, "We're going to go as one company." Jensen was texting him last night. They had a great time on stage. What is the go-to-market? Obviously, we saw the simplicity, four clicks, you're up and running. What is the message? What's the comp? Who's getting paid? You guys holding hands together? Is it a joint solution? What's the go-to-market?

Neil MacDonald

>> So, the first thing that we're doing is we're training and certifying our entire enterprise Salesforce on NVIDIA technologies as part of NVIDIA AI Computing by HPE, so that we can go bring that benefit of Private Cloud AI to all of our enterprise customers, but we're not stopping there. We're also enabling certification and training in our channel community, which is such a core set of partners for both of us in expanding reach and being able to bring these competitive benefits, in a democratized way, to the broad sweep of enterprises. And then, of course, we're working with NVIDIA in the field. Bob?

Bob Pette

>> Yeah, we started with the product group. So, my product group is actually just focused on the enterprise and just focused on the OEMs because of that huge opportunity, most of the AI is going to be there. So, we'll continue to make sure we're innovating at that level versus being distracted with trying to serve a bunch of different communities via one product group. The text that Antonio and Jensen shared, Jensen sent this morning and he said, "I want a tiger team pulled together." He sent it to our head of sales, myself, a couple of my peers, and said, "Let's get a tiger team together and figure out the next 90 days. I want to know every go-to-market activity." And so, he's a genius at pulling that together and holding us accountable. There'll be joint go-to-market campaigns that we do together. We'll be making sure that their customer briefing centers are fully equipped with our technology. Our sales force, which is much, much smaller than their sales force, we'll be working with them on Fortune 1,000 accounts, but it'll be a detailed plan that Jensen wants to review on a regular basis and I'm sure with Antonio.

>> Awesome. That's great.

>> Being in the valley for 25 years, I can tell you personally been following Jensen's career going back, and I know a lot of the dynamics. He's an entrepreneur. I think he really respects, and I saw a little bit on stage, to HP's culture. And he's a systems guy. HP's a technical company. I think it's a good fit. And congratulations on the relationship.

Bob Pette

>> Yeah, thank you.

Neil MacDonald

>> Well, we're excited to be working with the best partner in the space, and excited to bring that value to our customers.

Neil MacDonald

>> And we're timing the CEOs on keynotes. We're going to see how they do around the track. They're tech athletes. We want to see their speed. Dave, we're going to track it. All right, you watching theCUBE here. Bringing all the action. NVIDIA and HP, a really monumental partnership. We're going to keep tracking it here on the theCUBE. We're bringing all you the data here. We got our own NIM, it's called theCUBE data. Stay with us for more after this short break.