HPE Discover 2024 | AI ready your Datacenter

Clips
News
More from HPE Discover 2024

Marty Poniatowski

Presales Director: East, Central, Canada

AMD

AMD answers the demand surge for AI-ready GPUs with its Instinct series

The rapid acceleration of enterprise artificial intelligence usage has brought a dramatic demand surge for advanced chips, such as graphics processing units. In answering that demand, Advanced Micro Devices Inc.’s GPU product family, AMD Instinct, is designed to cater to the high demands of AI training.AMD’s Marty Poniatowski talks with theCUBE about updates on the Instinct MI300X AI-focused GPU.“At AMD, we developed a GPU family called Instinct,” said Marty Poniatowski (pictured), presales director of East and Central Canada at AMD. “The latest version is the MI300X, and it has 192 [gigabytes] of memory, which means you can run against it very large Llama 2 models

play_circle_outline AMD's Instinct GPU family, MI300X, in high demand for AI training.

play_circle_outline Customized Xilinx FPGA for Efficient Edge AI Workloads in Shorter Cycles

play_circle_outline AMD's EPYC Processors and Instinct Accelerators Dominate Market with Efficient GPU Feeding

Info

AI ready your Datacenter

Marty Poniatowski

Presales Director: East, Central, Canada AMD

search

>> Welcome back everyone to theCUBE's live coverage here in Las Vegas for HPE Discover 2024. I'm John Furrier, your host of theCUBE, with my co-host, Dave Vellante, co-founder of theCUBE with Rob Strechay, Rebecca Knight, the whole team's here, SiliconANGLE and the team coverage here. Got a great guest here from AMD. Marty Poniatowski. Welcome to theCUBE.

Marty Poniatowski

>> Thank you very much, John.

>> Guys, we've been talking about AMD for years on theCUBE just from sustainability, but now more than ever, the demand for chips, GPUs, XPUs is high, but it's not just that. It's what's around the chips. There's connectivity, there's switches. And you're seeing all these new clustered systems come out. You're seeing these new cloud providers pop up. Take us through what you guys are talking about. What's the big story?

Marty Poniatowski

>> Okay, great. Thank you, John. So, first of all, you're right on the money. There's a whole ecosystem around it. So, number one, I'm in New York. I work with a lot of mega customers and they like to have alternatives for any technology that they buy, they don't single source anything. So, from a GPU accelerator standpoint in the AI world, they want to make sure that they have more than one supplier. And at AMD, we developed a GPU family called Instinct. The latest version is the MI300X, and it has 192 gig of memory, which means you can run against it, very large Llama 2 models. So, it's made for really high-end training and it's a great alternative because no customer wants to buy just from one manufacturer. So, our Instinct product's in very high demand now. And in fact, Microsoft Azure has an MI300X environment that's optimized with Hugging Face. So, a lot of my customers are trying this out. They like to get them on-premises, but they can try them out right in Azure.

>> You're doing a lot of pre-sales, the pre-sales director, obviously, which is your sales, you're targeting big players. They're worried about lock-in. Can you share your thoughts on this and what is the concern? What's the specific nuance there on lock-in? Is it just supply? Is it overall diversity of workloads, flexibility? I mean, what is your take on that? I mean, I hear that a lot, but I don't even know what it means. Can you share your thoughts?

Marty Poniatowski

>> Yes. So, lock-in for any technology is the kiss of death for my customers because then they'll be solely reliant on a manufacturer. So, in the accelerator world, we are a great alternative, very high performance. That's true of software as well. In the AI world, there's a software development environment that you could get locked into. So, something is developed called Framework. And PyTorch and JAX are two examples. So, with these frameworks, you can develop in kind of an open source world as it were, and then you could have a target GPU. In our case, PyTorch and JAX have a ROCm plugin, that's our software. But if you're a PyTorch or JAX user, you don't even know it. There's Python code that brings in ROCm. So, now you have multiple GPUs that you can run, and you have the software stack that'll support any GPU. So, now my customers feel protected. They're not locked in from a hardware standpoint or from a software standpoint.

>> So, AMD, obviously has made a great business by being an alternative choice. The history is interesting. When you shed the fab business that took, I would say a monkey off your back, and allowed you to do some things and get more agile, and it obviously has worked. I got to invoke two icons. One, Jack Welch who said, "You got to be number one or number two in your industry." And Scott McNealy, who said, "There's always the two of everything." And I said to him one day, "Ketchup, Heinz, dominant." He goes, "Del Monte, Dave." So, he had me. So, you made a great business there. You're talking about software. I would like to hear more about software because it's not just the GPU hardware that has popularized this business. It's the ability to apply software. You said you run software on any GPU. Can you explain that In more detail?

Marty Poniatowski

>> Yes. So, first of all, thank you for saying that by the way. From a processor standpoint, I mentioned Instinct, our accelerator family. EPYC is our processor family, and we're up to about a third market share in the... So, that is another alternative that customers can look at. From a software standpoint, these frameworks I mentioned earlier have really taken over, meaning the lock-in if you use one of these frameworks, no longer exists. In fact, a lot of my customers are very concerned that if they develop in an AI environment with software that's lock-in, then that means they're going to have to use that software environment and that GPU. With the frameworks, they're clear to use any accelerators that they like. And it's a very good point, Dave. It's a very big part of the world now. It's not just the accelerator, the software is equally important.

>> Marty, I want to get your thoughts because AMD obviously has had a great business and x86, still does. Some will say, "Hey, maybe they took the eye off the ball on GPUs and FPGAs because of the x86." Not true. That's been clarified. We've been documenting that on theCUBE. But also, you had success in gaming. I mean the history of AMD and gaming, ray tracing. There's a lot of algorithms involved in gaming, and so we've seen that the market developers always been led by software developers. So, can you share the importance of software development process when you're talking about the combining or using or mixing and matching GPUs and FPGAs and how that relates to AI specifically?

Marty Poniatowski

>> Yep. Fantastic question, John, because when I describe AI to my customers, there is the build process, that is where you're training your own model. Typically, you go to Hugging Face, you find a model that's close to what you like, and then you feed your data into it and you train that model. You're building your own model. At the edge, you have the buy. There's commercial off-the-shelf software as an example, that runs, and sometimes you don't even need a GPU to run this. Some of the software runs on a CPU at the edge, and we have clients based on another family called Ryzen. And what's happening here is that AI is being pushed to the edge. You will see applications like Copilot later this year that'll have a big part of the AI software running on the client. So, yes, you're always going to have these big models that you train, but you're going to want your client to be able to support this AI software at the edge. And again, there's software that companies buy and it runs at the edge, and there's software called WaitTime, and it uses typically existing cameras. And what it does is crowd control. So, you buy WaitTime off-the-shelf software as an example, and it runs on a CPU. You don't even necessarily need a GPU for it. This industry's changing so fast. Hopefully, I'm here next year and you'll be asking me questions that are completely different because things will change-

>> We hope. We hope because there's a huge demand. Once the infrastructure performance levels get performing enough, we think that the software interactions between data are going to be big. And you mentioned the edge. So, I want to ask you which workloads work best under AMD's accelerated FPGA or GPU combos? Is it streaming data? What specifically do you see as this? Because we're hearing in the general GPU market that not everything needs a GPU. In some cases, we might have idle capacity laying around in these clusters that might bring in a new era of virtualization like performance where they overbuild. This is really nuanced conversation. What applications specifically do you see you guys honing in on around the FPGA and the GPUs?

Marty Poniatowski

>> I'm glad you asked this because we bought Xilinx, that's our FPGA family. And with respect to AI, what you can do is take one of these field programmable gate arrays and customize it such that it runs the AI software on a semi-custom chip. So, if you have, in a vehicle, for instance, an entertainment system, rather than have a full suite of software and components, the FPGA can do the lion's share of this. You've customized it, and it runs in the car in the entertainment system.

>> So, that's custom hardware?

Marty Poniatowski

>> Semi-custom. Correct.

>> Okay, that's great for edge smaller devices, maybe. So, you're streaming data, you're managing different things. Okay, what other use cases besides auto, do you see this working?

Marty Poniatowski

>> Oh, it's unlimited. I was with a healthcare customer here earlier today, and in their imaging systems, they're considering this as well. There's so much experimentation going on with this because so many new AI workloads are being developed, and now you're saying to yourself, "What am I going to run this on at the edge?" And the FPGA in a lot of cases is perfect. It's custom, it's efficient, and it'll run that software at the edge for.

>> So, you're infusing intelligence now into all applications in the data center, you're modernizing the data center, new hardware architectures, and you've connected the edge now and you're injecting intelligence into the edge. Do you see those two worlds coming together?

Marty Poniatowski

>> They are coming together lightning fast. I'm glad you mentioned the modernization in the data center, because I talked about our market share earlier. Our EPYC CPUs have gained huge market share. And what we're doing is taking what I call technology debt, old environments, we're modernizing them to much smaller efficient environments, less floor space, power cooling, licensing. Pretty much every customer here is worried about licensing today. And what some of my customers are doing is taking that savings and applying it to their AI projects. Money's not always falling out of the trees, but if you modernize your environment and you realize this huge savings with EPYC, now you can apply that to AI.

>> One of the things we talk a lot with customers and also companies who give us their conjecture and their story, and sometimes you got to match it up to where the customers are and what they know or think, and sometimes there's a mismatch. Many reasons. Could be hype, could be overplaying their hand. But certainly in this market, there's an appetite for demand and appetite for a generative AI in the enterprise and consumer. Let's say the enterprise. One thing that's holding it back is the data privacy, the security, and then the multitude of the data estate is fragmented, but they're still going forward. Now, they're thinking, okay, we don't want lock-in. You've talked about that. But they also don't want to go back to the days where you had proprietary monolithic systems, whether it's many compete at gnosis or anything that locks them in like a rack that could be limited. We don't yet see that, but there's fear for a heterogeneous environment that will be preserved. Where do you guys see the battle lines forming in terms of the right answer on making that happen? Because you're seeing workloads have different needs. Some people do more training and only do inference. You only inference and reinforce learning. You've got neural networks, it's got vectors, tokens, and embeds. Okay, great. Okay. That's just a storage problem. Maybe some custom silicon. So, the world seems to be morphing into shorter cycles, custom silicon, custom packaging, purpose-built stuff. What's the ideal scenario? Keeping it more open. Where should people be looking at to stare at this?

Marty Poniatowski

>> Well, John, you hit the nail on the head. I have a series of lunch panel events I run, and we invite customers in for a lunch, and I have a little expert panel, we interview them. The first question that's asked invariably is related to data, because people have data at the edge, they have it in the public cloud and they have it on-premises. So, now I need to get all that data and feed it into my model to train it. How am I going to do that? And you're talking about data integrity, in the AI world, there's this concept of hallucinations. I feed in all this data, but some of it's erroneous. I don't know it, right? I'm feeding in so much data, I can't even measure it. So, now when you train the model, it's possible to get output that is, let's just call it incorrect, and that's referred to the AI world as hallucinations. So, dealing with this data problem is the very first step in figuring out how am I going to train my model, so that I know that I have good integrity with the data that I'm using because it's everywhere?

>> So, I want to ask you, you've got x86 architecture. You've got ARM-based architectures now. So, you're in both of those worlds, obviously very mega-high volume because of this stuff. Where do FPGAs fit? Why do I need them today? And what's the primary use cases there?

Marty Poniatowski

>> Okay, let me position the architectures one more time. So, Instinct, the accelerator, as we call it in the AI world. EPYC, which is the processor, and you mentioned ARM, right? There's a use for ARM as well. But EPYC, our market share's skyrocketing. We have a Ryzen architecture for the client, and then the FPGAs, so all of these. Instinct obviously for training and the very high end. And EPYC's important in AI, Dave, because you need to feed these GPUs efficiently. And then, as I mentioned at the client, you're going to want to run some of the AI locally. I guarantee you, by the end of this year when you're buying your holiday gifts, people are going to be asking for laptops for clients that will run AI software. And then, the FPGAs, the Xilinx field programmable gate arrays that allow you to take whatever you've created and implement them for a specific workload. So, there's a place for all of them.

>> I think the market needs... We saw this emerge when these custom GPU clouds, I call them tier-two clouds, whatever you want to call them, they're specialty clouds, because workloads can be scoped in, and this is a boon for either the data centers and/or the hosters. So, as you see more cloud services coming and potentially custom services, assume chip sales normalize and some supply, and then people can win on their performance. Cloud services and custom racks and hardware might be needed. I mean, I love what HP's doing. Hey, you push a button, three clicks you're in, you manage your own data to do some RAG, do some stuff. But you're going to see this shift when it starts to get real. More cloud services and more custom stuff, on-prem or edge. Can you share your thoughts on how you see that playing out if the dots connect? And they will, you guys are going to get better. You get the x86 and the GPU side and the accelerators. Okay, great. What's next?

Marty Poniatowski

>> Right. Well, first of all, I'm glad you mentioned RAG, retrieval augmented generation. This is where you take a model typically from Hugging Face, bring it in, and that's where you do your training. But what's happening now is that going back to my modernization and using EPYC to save money, what we're doing now is we are bringing into the data center very powerful accelerators that consume a lot of power, right? This is not for the faint of heart. So, direct liquid cooling is coming into play. The cooling technologies are very important to making this environment work. For instance, I'm in New York, a lot of my customers have 10, 12 kilowatt power drops in their data centers. That's not going to power a lot of accelerators.

>> Yeah, you can get all the GPUs you want, but you've got the power them.

Marty Poniatowski

>> You got to power them, you have to cool them, John, right? So, this is where the direct, there's other technologies, but direct-

>> Or make a better chip that has less power requirements, right?

Marty Poniatowski

>> First of all, I think that's going to happen, but right now, everybody's so power hungry with accelerators, "Just give me more, Marty. Give me more." It's kind of like a car. You want to go faster, can't always... That trade-off's tricky there, right? You want to go lightning fast-

>> That's a nice pre-sales motion. What do you need? Stand in line.

Marty Poniatowski

>> Now, you know why I'm pretty good at my job.

>> Well, I'm just kidding. So, I know you guys got a lot of orders and backlog probably, but your customers are doing systems design thinking. It's a systems revolution with these clustered systems that we've been reporting on. And it's the same game, just different shapes. It's just larger scale. You got spines and leaves, racks look the same, but different. And this is where the action is. This is what people in the enterprise are trying to figure out is, "Okay, I know what a data center is. I know why it exists, but I have a goal. I got an outcome. I got an app, end-to-end workload. I got to scope it." And it might be different by company, so custom is huge.

Marty Poniatowski

>> I couldn't agree more. I've been doing this a long time. My customers used to be highly-specialized. There was somebody that knew one technology, another tech... Everybody's become so sophisticated overnight because AI's not for the faint of heart, I need to understand everything about the environment. And we have, working with HPE, their optimized systems here, the Cray XD675 has our GPUs in it. People are like, "I love that. Everything's integrated. It's fast." But you're right. How many of those can you put in a rack in my data center? How am I going to power that? So, people have to become savvy in a lot of different technologies, and AMD's here to help them obviously. But this is very challenging and it's changing fast, and it's fun as heck. It's fun.

>> Well, it's changing fast. It's changing fast. We should definitely follow up because we've been chronicalizing and documenting this wave for many years now. In fact, we saw the first... Four years ago at Supercomputing, which we have never covered because it was a HPC event, but we saw all the people there. AMD, all your competition, Amazon. That's where they were recruiting engineers. Like this is an AI show. This is coming fast. We've been there every year, so it's happened. So, let's keep in touch. Great to have you guys working on-

>> Thank you, Marty.

>> We need more power.

Marty Poniatowski

>> Thank you very much. Thanks for having me.

>> This was great. AMD is here on theCUBE. We are here at HPE Discover, and this is theCUBE Live. We're closing out day two. Tomorrow, day three continues. I'm John Furrier, Dave Vellante, Rebecca Knight, Rob Strechay, Bob Laliberte, and the whole team is here. Thanks to the production crew here. Great show again. Continue to scale up the content. Of course, bringing the data to you, streaming live. Thanks for watching.

AI ready your Datacenter

search

>> Welcome back everyone to theCUBE's live coverage here in Las Vegas for HPE Discover 2024. I'm John Furrier, your host of theCUBE, with my co-host, Dave Vellante, co-founder of theCUBE with Rob Strechay, Rebecca Knight, the whole team's here, SiliconANGLE and the team coverage here. Got a great guest here from AMD. Marty Poniatowski. Welcome to theCUBE.

Marty Poniatowski

>> Thank you very much, John.

>> Guys, we've been talking about AMD for years on theCUBE just from sustainability, but now more than ever, the demand for chips, GPUs, XPUs is high, but it's not just that. It's what's around the chips. There's connectivity, there's switches. And you're seeing all these new clustered systems come out. You're seeing these new cloud providers pop up. Take us through what you guys are talking about. What's the big story?

Marty Poniatowski

>> Okay, great. Thank you, John. So, first of all, you're right on the money. There's a whole ecosystem around it. So, number one, I'm in New York. I work with a lot of mega customers and they like to have alternatives for any technology that they buy, they don't single source anything. So, from a GPU accelerator standpoint in the AI world, they want to make sure that they have more than one supplier. And at AMD, we developed a GPU family called Instinct. The latest version is the MI300X, and it has 192 gig of memory, which means you can run against it, very large Llama 2 models. So, it's made for really high-end training and it's a great alternative because no customer wants to buy just from one manufacturer. So, our Instinct product's in very high demand now. And in fact, Microsoft Azure has an MI300X environment that's optimized with Hugging Face. So, a lot of my customers are trying this out. They like to get them on-premises, but they can try them out right in Azure.

>> You're doing a lot of pre-sales, the pre-sales director, obviously, which is your sales, you're targeting big players. They're worried about lock-in. Can you share your thoughts on this and what is the concern? What's the specific nuance there on lock-in? Is it just supply? Is it overall diversity of workloads, flexibility? I mean, what is your take on that? I mean, I hear that a lot, but I don't even know what it means. Can you share your thoughts?

Marty Poniatowski

>> Yes. So, lock-in for any technology is the kiss of death for my customers because then they'll be solely reliant on a manufacturer. So, in the accelerator world, we are a great alternative, very high performance. That's true of software as well. In the AI world, there's a software development environment that you could get locked into. So, something is developed called Framework. And PyTorch and JAX are two examples. So, with these frameworks, you can develop in kind of an open source world as it were, and then you could have a target GPU. In our case, PyTorch and JAX have a ROCm plugin, that's our software. But if you're a PyTorch or JAX user, you don't even know it. There's Python code that brings in ROCm. So, now you have multiple GPUs that you can run, and you have the software stack that'll support any GPU. So, now my customers feel protected. They're not locked in from a hardware standpoint or from a software standpoint.

>> So, AMD, obviously has made a great business by being an alternative choice. The history is interesting. When you shed the fab business that took, I would say a monkey off your back, and allowed you to do some things and get more agile, and it obviously has worked. I got to invoke two icons. One, Jack Welch who said, "You got to be number one or number two in your industry." And Scott McNealy, who said, "There's always the two of everything." And I said to him one day, "Ketchup, Heinz, dominant." He goes, "Del Monte, Dave." So, he had me. So, you made a great business there. You're talking about software. I would like to hear more about software because it's not just the GPU hardware that has popularized this business. It's the ability to apply software. You said you run software on any GPU. Can you explain that In more detail?

Marty Poniatowski

>> Yes. So, first of all, thank you for saying that by the way. From a processor standpoint, I mentioned Instinct, our accelerator family. EPYC is our processor family, and we're up to about a third market share in the... So, that is another alternative that customers can look at. From a software standpoint, these frameworks I mentioned earlier have really taken over, meaning the lock-in if you use one of these frameworks, no longer exists. In fact, a lot of my customers are very concerned that if they develop in an AI environment with software that's lock-in, then that means they're going to have to use that software environment and that GPU. With the frameworks, they're clear to use any accelerators that they like. And it's a very good point, Dave. It's a very big part of the world now. It's not just the accelerator, the software is equally important.

>> Marty, I want to get your thoughts because AMD obviously has had a great business and x86, still does. Some will say, "Hey, maybe they took the eye off the ball on GPUs and FPGAs because of the x86." Not true. That's been clarified. We've been documenting that on theCUBE. But also, you had success in gaming. I mean the history of AMD and gaming, ray tracing. There's a lot of algorithms involved in gaming, and so we've seen that the market developers always been led by software developers. So, can you share the importance of software development process when you're talking about the combining or using or mixing and matching GPUs and FPGAs and how that relates to AI specifically?

Marty Poniatowski

>> Yep. Fantastic question, John, because when I describe AI to my customers, there is the build process, that is where you're training your own model. Typically, you go to Hugging Face, you find a model that's close to what you like, and then you feed your data into it and you train that model. You're building your own model. At the edge, you have the buy. There's commercial off-the-shelf software as an example, that runs, and sometimes you don't even need a GPU to run this. Some of the software runs on a CPU at the edge, and we have clients based on another family called Ryzen. And what's happening here is that AI is being pushed to the edge. You will see applications like Copilot later this year that'll have a big part of the AI software running on the client. So, yes, you're always going to have these big models that you train, but you're going to want your client to be able to support this AI software at the edge. And again, there's software that companies buy and it runs at the edge, and there's software called WaitTime, and it uses typically existing cameras. And what it does is crowd control. So, you buy WaitTime off-the-shelf software as an example, and it runs on a CPU. You don't even necessarily need a GPU for it. This industry's changing so fast. Hopefully, I'm here next year and you'll be asking me questions that are completely different because things will change-

>> We hope. We hope because there's a huge demand. Once the infrastructure performance levels get performing enough, we think that the software interactions between data are going to be big. And you mentioned the edge. So, I want to ask you which workloads work best under AMD's accelerated FPGA or GPU combos? Is it streaming data? What specifically do you see as this? Because we're hearing in the general GPU market that not everything needs a GPU. In some cases, we might have idle capacity laying around in these clusters that might bring in a new era of virtualization like performance where they overbuild. This is really nuanced conversation. What applications specifically do you see you guys honing in on around the FPGA and the GPUs?

Marty Poniatowski

>> I'm glad you asked this because we bought Xilinx, that's our FPGA family. And with respect to AI, what you can do is take one of these field programmable gate arrays and customize it such that it runs the AI software on a semi-custom chip. So, if you have, in a vehicle, for instance, an entertainment system, rather than have a full suite of software and components, the FPGA can do the lion's share of this. You've customized it, and it runs in the car in the entertainment system.

>> So, that's custom hardware?

Marty Poniatowski

>> Semi-custom. Correct.

>> Okay, that's great for edge smaller devices, maybe. So, you're streaming data, you're managing different things. Okay, what other use cases besides auto, do you see this working?

Marty Poniatowski

>> Oh, it's unlimited. I was with a healthcare customer here earlier today, and in their imaging systems, they're considering this as well. There's so much experimentation going on with this because so many new AI workloads are being developed, and now you're saying to yourself, "What am I going to run this on at the edge?" And the FPGA in a lot of cases is perfect. It's custom, it's efficient, and it'll run that software at the edge for.

>> So, you're infusing intelligence now into all applications in the data center, you're modernizing the data center, new hardware architectures, and you've connected the edge now and you're injecting intelligence into the edge. Do you see those two worlds coming together?

Marty Poniatowski

>> They are coming together lightning fast. I'm glad you mentioned the modernization in the data center, because I talked about our market share earlier. Our EPYC CPUs have gained huge market share. And what we're doing is taking what I call technology debt, old environments, we're modernizing them to much smaller efficient environments, less floor space, power cooling, licensing. Pretty much every customer here is worried about licensing today. And what some of my customers are doing is taking that savings and applying it to their AI projects. Money's not always falling out of the trees, but if you modernize your environment and you realize this huge savings with EPYC, now you can apply that to AI.

>> One of the things we talk a lot with customers and also companies who give us their conjecture and their story, and sometimes you got to match it up to where the customers are and what they know or think, and sometimes there's a mismatch. Many reasons. Could be hype, could be overplaying their hand. But certainly in this market, there's an appetite for demand and appetite for a generative AI in the enterprise and consumer. Let's say the enterprise. One thing that's holding it back is the data privacy, the security, and then the multitude of the data estate is fragmented, but they're still going forward. Now, they're thinking, okay, we don't want lock-in. You've talked about that. But they also don't want to go back to the days where you had proprietary monolithic systems, whether it's many compete at gnosis or anything that locks them in like a rack that could be limited. We don't yet see that, but there's fear for a heterogeneous environment that will be preserved. Where do you guys see the battle lines forming in terms of the right answer on making that happen? Because you're seeing workloads have different needs. Some people do more training and only do inference. You only inference and reinforce learning. You've got neural networks, it's got vectors, tokens, and embeds. Okay, great. Okay. That's just a storage problem. Maybe some custom silicon. So, the world seems to be morphing into shorter cycles, custom silicon, custom packaging, purpose-built stuff. What's the ideal scenario? Keeping it more open. Where should people be looking at to stare at this?

Marty Poniatowski

>> Well, John, you hit the nail on the head. I have a series of lunch panel events I run, and we invite customers in for a lunch, and I have a little expert panel, we interview them. The first question that's asked invariably is related to data, because people have data at the edge, they have it in the public cloud and they have it on-premises. So, now I need to get all that data and feed it into my model to train it. How am I going to do that? And you're talking about data integrity, in the AI world, there's this concept of hallucinations. I feed in all this data, but some of it's erroneous. I don't know it, right? I'm feeding in so much data, I can't even measure it. So, now when you train the model, it's possible to get output that is, let's just call it incorrect, and that's referred to the AI world as hallucinations. So, dealing with this data problem is the very first step in figuring out how am I going to train my model, so that I know that I have good integrity with the data that I'm using because it's everywhere?

>> So, I want to ask you, you've got x86 architecture. You've got ARM-based architectures now. So, you're in both of those worlds, obviously very mega-high volume because of this stuff. Where do FPGAs fit? Why do I need them today? And what's the primary use cases there?

Marty Poniatowski

>> Okay, let me position the architectures one more time. So, Instinct, the accelerator, as we call it in the AI world. EPYC, which is the processor, and you mentioned ARM, right? There's a use for ARM as well. But EPYC, our market share's skyrocketing. We have a Ryzen architecture for the client, and then the FPGAs, so all of these. Instinct obviously for training and the very high end. And EPYC's important in AI, Dave, because you need to feed these GPUs efficiently. And then, as I mentioned at the client, you're going to want to run some of the AI locally. I guarantee you, by the end of this year when you're buying your holiday gifts, people are going to be asking for laptops for clients that will run AI software. And then, the FPGAs, the Xilinx field programmable gate arrays that allow you to take whatever you've created and implement them for a specific workload. So, there's a place for all of them.

>> I think the market needs... We saw this emerge when these custom GPU clouds, I call them tier-two clouds, whatever you want to call them, they're specialty clouds, because workloads can be scoped in, and this is a boon for either the data centers and/or the hosters. So, as you see more cloud services coming and potentially custom services, assume chip sales normalize and some supply, and then people can win on their performance. Cloud services and custom racks and hardware might be needed. I mean, I love what HP's doing. Hey, you push a button, three clicks you're in, you manage your own data to do some RAG, do some stuff. But you're going to see this shift when it starts to get real. More cloud services and more custom stuff, on-prem or edge. Can you share your thoughts on how you see that playing out if the dots connect? And they will, you guys are going to get better. You get the x86 and the GPU side and the accelerators. Okay, great. What's next?

Marty Poniatowski

>> Right. Well, first of all, I'm glad you mentioned RAG, retrieval augmented generation. This is where you take a model typically from Hugging Face, bring it in, and that's where you do your training. But what's happening now is that going back to my modernization and using EPYC to save money, what we're doing now is we are bringing into the data center very powerful accelerators that consume a lot of power, right? This is not for the faint of heart. So, direct liquid cooling is coming into play. The cooling technologies are very important to making this environment work. For instance, I'm in New York, a lot of my customers have 10, 12 kilowatt power drops in their data centers. That's not going to power a lot of accelerators.

>> Yeah, you can get all the GPUs you want, but you've got the power them.

Marty Poniatowski

>> You got to power them, you have to cool them, John, right? So, this is where the direct, there's other technologies, but direct-

>> Or make a better chip that has less power requirements, right?

Marty Poniatowski

>> First of all, I think that's going to happen, but right now, everybody's so power hungry with accelerators, "Just give me more, Marty. Give me more." It's kind of like a car. You want to go faster, can't always... That trade-off's tricky there, right? You want to go lightning fast-

>> That's a nice pre-sales motion. What do you need? Stand in line.

Marty Poniatowski

>> Now, you know why I'm pretty good at my job.

>> Well, I'm just kidding. So, I know you guys got a lot of orders and backlog probably, but your customers are doing systems design thinking. It's a systems revolution with these clustered systems that we've been reporting on. And it's the same game, just different shapes. It's just larger scale. You got spines and leaves, racks look the same, but different. And this is where the action is. This is what people in the enterprise are trying to figure out is, "Okay, I know what a data center is. I know why it exists, but I have a goal. I got an outcome. I got an app, end-to-end workload. I got to scope it." And it might be different by company, so custom is huge.

Marty Poniatowski

>> I couldn't agree more. I've been doing this a long time. My customers used to be highly-specialized. There was somebody that knew one technology, another tech... Everybody's become so sophisticated overnight because AI's not for the faint of heart, I need to understand everything about the environment. And we have, working with HPE, their optimized systems here, the Cray XD675 has our GPUs in it. People are like, "I love that. Everything's integrated. It's fast." But you're right. How many of those can you put in a rack in my data center? How am I going to power that? So, people have to become savvy in a lot of different technologies, and AMD's here to help them obviously. But this is very challenging and it's changing fast, and it's fun as heck. It's fun.

>> Well, it's changing fast. It's changing fast. We should definitely follow up because we've been chronicalizing and documenting this wave for many years now. In fact, we saw the first... Four years ago at Supercomputing, which we have never covered because it was a HPC event, but we saw all the people there. AMD, all your competition, Amazon. That's where they were recruiting engineers. Like this is an AI show. This is coming fast. We've been there every year, so it's happened. So, let's keep in touch. Great to have you guys working on-

>> Thank you, Marty.

>> We need more power.

Marty Poniatowski

>> Thank you very much. Thanks for having me.

>> This was great. AMD is here on theCUBE. We are here at HPE Discover, and this is theCUBE Live. We're closing out day two. Tomorrow, day three continues. I'm John Furrier, Dave Vellante, Rebecca Knight, Rob Strechay, Bob Laliberte, and the whole team is here. Thanks to the production crew here. Great show again. Continue to scale up the content. Of course, bringing the data to you, streaming live. Thanks for watching.