Name: The Architecture of "And": Uniting CPU & GPU for Enterprise AI
Uploaded: 2026-07-23T18:30:00.000Z
Duration: 47 min 28 s
Description: Supermicro and AMD leaders discuss how balanced CPU, GPU, networking and rack-scale architectures are reshaping enterprise AI infrastructure. They also explain how modernization, liquid cooling and joint engineering can improve efficiency, accelerate deployment and support agentic AI workloads.

Clips
More from Beyond the GPU: The Full-Stack Fight for Enterprise AI

Derek Dicker

CVP, Enterprise Business Group

AMD

Vik Malyala

Chief Business Officer

Supermicro

play_circle_outline AMD Supermicro engineering partnership and co-development of solutions

play_circle_outline Rising demand for AI infrastructure and data center modernization

play_circle_outline Consolidation benefits using Turin-based servers: footprint, power, TCO improvements

play_circle_outline Balancing CPUs and GPUs as Agentic AI Inference Drives Growing Enterprise CPU Demand

play_circle_outline Importance of system balance: memory, PCIe Gen6, I/O, networking, storage

play_circle_outline AMD Venice (6th gen EPYC) capabilities and H15 deployments

play_circle_outline Design tradeoffs: thermal, form factor, reliability, cost, and performance

play_circle_outline Helios Rack-Scale Architecture — 72 GPUs, 18 Venice CPUs: Enabling Open Standards and Customer Choice

play_circle_outline Retrofitting Existing Data Centers for AI: Liquid Cooling, Rack Density, and Modular Building-Block Solutions

play_circle_outline Need for rapid POCs and time-to-value for IT decision makers

53:10

Info
Transcript

The Architecture of "And": Uniting CPU & GPU for Enterprise AI

Derek Dicker

CVP, Enterprise Business Group AMD

Vik Malyala

Chief Business Officer Supermicro

Supermicro and AMD leaders discuss how balanced CPU, GPU, networking and rack-scale architectures are reshaping enterprise AI infrastructure. They also explain how modernization, liquid cooling and joint engineering can improve efficiency, accelerate deployment and support agentic AI workloads.

The Architecture of "And": Uniting CPU & GPU for Enterprise AI

(INTRO)

Dave Vellante

>> Hi everybody, welcome to this special theCUBE Power Panel, supporting Advancing AI. My name is Dave Vellante and the topic today is the architecture of AMD, uniting CPU and GPU for enterprise AI. And I have a little setup and then we're going to introduce the guests. So let me tell you about the angle that we're going to take today. AI infrastructure is very often discussed as though it were just exclusively a decision around GPUs. But in production, you have CPUs, accelerators, memory. You've got networking, storage, cooling, software, and rack integration. All of these have to work as one system in balance. So this becomes even more important as trusted enterprise systems of record, they're converging with AI -driven and agentic applications, the probabilistic nature of these. And today we're going to examine how AMD and Supermicro are combining roadmaps and engineering to translate Venice into the H15 family and broader rack scale infrastructure and how that impacts performance efficiency and time to value. Venice, of course, is the code name for AMD's sixth generation EPYC processor. It's the workhorse for heavy duty cloud and data center and agentic AI workloads. And the H designation is reserved for the AMD powered Supermicro data center servers. And with me to discuss that is Derek Dicker, who's the Corporate Vice President for Enterprise Business Group at AMD. Derek, thank you, by the way, for hosting us here at this EBC.

Derek Dicker

>> Thank you for coming and for having us. We're looking forward to the conversation.

Dave Vellante

>> And Vik Malyala, who's the Chief Business Officer at Supermicro. Vik, good to see you.

Vik Malyala

>> Same here. Thank you.

Dave Vellante

>> Okay. So let's start with the partnership. What is the unique role that each of your companies play in this partnership? How would you describe that? Why don't we start with you, Derek?

Derek Dicker

>> First off, the only way that somebody can build technology that's required today to address all the workloads that you described before is to have incredibly close partnerships between the companies in the chain of delivering products to the end. And with respect to Supermicro, an incredible relationship since the beginning and inception of EPYC. I think, Vik, we've had five generations of product launched on time at launch together.

Vik Malyala

>> Absolutely, yeah.

Derek Dicker

>> And the only way that happens is by, not just having a relationship between the two of us, but all the way down to the engineers, thinking through what's coming next, aligning our requirements, designing products together, co -innovating together. And it's been just a fantastic relationship so far.

Dave Vellante

>> How about from your perspective, Vik? I think EPYC, we go back to probably 2017, 2018 timeframe, but Genoa was probably 2022 timeframe and really been on a clear cadence since then. What's your perspective on the partnership?

Vik Malyala

>> I think the most important thing is two engineering centric companies that are collaborating and co-developing platforms and solutions together. AMD obviously is focusing on the silicon. Our role at Supermicro was to see how best to design a platform or a solution that takes advantage of every one of those features and making it available for customers. So basically making it real at a server level for the customers to use. and it goes very deep from the very beginning of the concept. You mentioned the first generation onwards. First generation when AMD introduced it was all about maximum number of cores and at that time the maximum memory that you can pack in a system with AMD on that one and it had 128 lanes of PCI Express which was unheard of, right? It started with that and we never looked back. Every generation, there is tremendous improvement, generation to generation, and features and functionality get added at a silicon level and to support that from Supermicro at a system level, now at a rack scale level or a data center level, we are working towards getting the right platforms out together.

Dave Vellante

>> And I want to get into the deep engineering that you guys do, but let's start with the demand. Everybody's excited about AI. What can you share, Derek, in terms of what you're seeing for demand, and then Vik, maybe you can take it closer to the end customer, because we really want to understand what's driving this next generation enterprise infrastructure. Where is demand? obviously demand is through the roof, but how would you describe it?

Derek Dicker

>> Yeah, I would actually attack it from a couple of different perspectives. The first is, every single enterprise customer is being asked by their management, by their board, how are they going to go and address AI? and utilize it in their infrastructure and day -to -day activities. That actually spurs two different sets of activities. The first is inside of data centers today, most of them are bursting at the seams and doing so with really legacy technology, call it three, four years back server infrastructure. In order to be able to put AI into that infrastructure, they actually have to be able to create room. And so there's quite a bit of activity that we see before the agentic world wrapped around modernization, data center modernization, that's one level of demand that's come on top. Now when you modernize, you actually give yourself the opportunity to compress the size of the footprint, reduce power, et cetera, but that really is in the pursuit of freeing up room for AI infrastructure. The next piece comes in of how are enterprises deploying enterprise AI and what are they gonna build out for that and that's driving the next layer of demand. And the only way to service those is by having an incredibly tight partnership with companies like Supermicro to build that next generation of infrastructure. That's fueling a large amount of CPU demand as well as GPU demand too today.

Dave Vellante

>> So Vik, that actually brings up an interesting point because traditionally, infrastructure has been about refresh cycles, but there seems to be something else happening here that's driving demand in the next generation of infrastructure. And that's really, I'll call it scaling without labor. Organizations are realizing that if they spend more on tech, they can actually scale faster with the same amount of labor. And so we predict that the amount of spending, the percentage of revenue that is spent on technology, on IT generally, but technology specifically, is going to go up because it's driving productivity and revenue, so it's not just a refresh cycle now every three or four or five years. It's more than that. Your perspective on demand.

Vik Malyala

>> So common knowledge, I think you talk to anyone in the industry, whether it's a CPU or GPU or anything that connects all these things together, everyone is saying they're sold out for the next several quarters. Why? You take a look at the demand that's insatiable. So in the case of AI, it all started with training clusters. You're talking about massive, massive training clusters are being built, but now in the last, let's say eight to 10 months, and then as we see the demand coming up, a lot more is to do with inferencing and inferencing specific to agentic AI that we are talking about now. The RAG models and everything was there, the gen AI models came up, and now you're talking about all these agentic AI workloads that are completely transforming the requirement for all the customers. The traditional workloads as they are, they are great, but in addition to that, people are trying to adopt the AI into their workflows in order to improve the efficiencies and productivity for the companies, and that is driving the entire decision making process. And the idea again here is to take something that they feel comfortable with and using it to enhance their workloads on top of that to get the better business value. That is what's driving the demand. And on top of it, the efficiencies and everything coming into picture, right? not necessarily in the same order, but as people start deploying these, these are deployed at scale, not like onesie, twosie, but you're talking about clusters and clusters of it. Now people need to figure out how best to optimize their cost function and how best to optimize the total cost of ownership and bring out the efficiencies. So a lot of these things are happening, and that's driving the demand.

Dave Vellante

>> When you think back to the cloud era, which is now the new legacy, you had the public cloud workloads, and you had the on-prem workloads, that was the state of the art hybrid. They really weren't working together. Maybe there was some bursting going on. It seems that's changing. You've clearly got a lot of activity going on in the public cloud. We had the neocloud panel, which people should check out. And so there's a lot of activity there. And then you've got organizations saying, hey, we want to bring the AI to the data that's on-prem. We're not going to move that all into the cloud. So you're seeing this true hybrid situation. So my question is where do they put all this stuff? They're bursting at the seams. So they've got to look to companies like yours to consolidate the existing infrastructure. What are you seeing in terms of the ability to consolidate servers down from where they are today to a better ratio?

Derek Dicker

>> Well maybe just one data point that we can offer up is in the partnership between our companies. As we've gone out and we've done case study related work within customers, we've actually built up a consolidation playbook. And within that, what we reflect is based on using a Turin, a fifth generation EPYC processor as the baseline inside a 2P server today, you can actually take three to four year old technology on legacy infrastructure and you can replace eight of those legacy servers with a single Turin-based server. Now, what does that equate to? that gets you down to substantially smaller footprint being used, it gets you down to 70 % less power, 71 % improved total cost of ownership, freeing up space for the AI workloads. That's one baseline fact. Now that carries a consistent set of performance across the two different compares. And we're seeing different, depending on what customers are looking for, different levels of consolidation, but it's very real and it's happening on a repeated basis. That's driving some of the demand you talked about. I'd say the other piece, by the way, is that on the AI side of the house, one of the things that we're experiencing more and more is a desire for people to actually put what historically would be token -related workloads out of the cloud, looking at how do they run on open source models inside of infrastructure on top of x86 servers. And that's the next level that we actually expect to see.

Dave Vellante

>> That's a good point.Customers, Vik, they want to be their own token generators, don't they?

Vik Malyala

>> Absolutely. I was trying to see where to start. But what is happening is it's expensive, right? talking about anything with AI, that is an additional expense on top of it, whatever the infrastructure spend that people already have. And what's happening is initially the easiest way is to just go to any a public cloud service provider and then take these tokens and start using. Everyone is happy until they see the bill. And then all of a sudden, now what do they do? There are definite benefits. So people are trying to figure out how to bring it on-prem. And that means they need to have the data centers, they need to probably retrofit the data centers, or whichever way that they are going with it. Now what happens? So if I pause and take a look at it, the amount of power that is going to be available per rack, that is being revised as they want to bring this AI infrastructure into that. Now that that is already taken care of, they can bring very high -dense compute right next to it. So what we are seeing is that we can make the best use of available power and space by bringing both AI infrastructure, which is a GPU, as well as the compute. And what we are seeing, people are more receptive to the idea of using liquid cooling. And when they bring liquid cooling into that, I can increase the density like anything. I don't know if you can say that, but we're talking about, what, 40 ,000 plus cores per rack?

Derek Dicker

>> 40 ,000 plus cores per rack.

Vik Malyala

>> When was the last time you heard about that a density in a rack level for CPU? We're not talking about GPUs, I'm talking about CPUs here.

Dave Vellante

>> Right.Okay, well let's get into it. Let's explain the whole CPU, GPU and why it's an and story and not an or story. the business news will cover it. Lisa, I think, was pretty explicit on the last earnings call. I think she said it used to be 8 to 1, it's going to be now 4 to 1. we think it's soon going to be 2 -to -1, maybe even 1 -to -1, maybe even flip. What's driving that? When you let agents loose in the organization, they're accessing different tools, they're accessing different documents, and they're spending time reasoning, and that requires, obviously, a lot of compute. But help us understand specifically what's going on under the hood.

Derek Dicker

>> Yeah, I'll take a first crack at it. It would be really great for Vik to pile on. If you look at the work that's being done in agentic workflows themselves and workloads, it's a lot of leveraging existing applications that are in place, but it's an orchestration that's occurring across them. So you have the preservation of the applications that we use, know and love today getting connected into, but there needs to be orchestration across all of those. There needs to be tool calls that take place and also a bunch of work on the data itself. All of these things fit really, really well to CPU architecture, and in particular, x86 architecture. So that combination of activities is what's driving the demand that we see on x86 CPUs in the agentic era. And it's a non -trivial explosion in consumption and use that we're seeing coming out over the horizon.

Vik Malyala

>> I think just to add to that, to start with, GPUs are the most expensive part of hardware, and we want to keep the GPUs busy doing the actual work, which basically means anything that can be done to support the GPUs and make sure that they are fed properly, that is where the computing started. With agentic workloads, to your point, a lot of these things can be done on a CPU far better than doing it on an expensive GPU. That's number one. Second thing is so many small batches coming in with these agents running on that one with more number of processor cores packed in, like 256 cores per CPU type of thing, with a lot of memory and I/O bandwidth being added to that, a lot of stuff can be done. I think I was reading something about what's the total latency impact by running these things in the CPU architecture. You're talking about 50 % to 80 % of the latencies of these agents can be addressed in a CPU's point of view. So not just for the pre-compute that CPUs used to be used for in the past. Now we are talking about a lot of agentic AI workloads, whether it's the database lookups or the tool callings and everything that we are talking about, all the orchestration that happens, all this is happening on the CPU. The CPU counts are increasing considerably in inference -specific workloads for enterprise customers. And coupling that and properly balancing it with the GPU architecture is the one that's going to provide the right recipe for customers. Am I correct?

Dave Vellante

>> You're correct. It's not just a CPU problem. You've got networking storage, IO. It's a balanced system to keep that GPU busy and get the most efficient use out of the system.

Vik Malyala

>> Right. I think that's the reason. You're absolutely right. That's the reason the CPU to GPU balance comes into the picture, because you take a look at the latest generation processor. Now you're talking about PCI Express Gen 6. You're talking about 16 channels of memory with 8 ,000 plus megahertz of memory. So now you're talking about how much memory, what bandwidth that you can get from the memory, and what I/O that you can get because of the PCI Express being faster, and how many GPUs to a specific CPU we are mapping. And if that memory and if that storage is not sufficient, what other storage you can add on top of it to create more of a balanced compute GPU and storage network with very fast, I would say, interconnect?

Derek Dicker

>> that's exactly like a Pensando, for example, in this case, what I would add to that is the beauty of the relationship between the two companies is the fact that we can sit down and look at mutual experiences within customers, and translate that to what are the requirements at a system level. And then between the two of us leveraging their system design expertise, as well as our ability to design silicon in a flexible manner, to go and address these workloads, we now have GPU centric technology, we have CPU -centric technology, we have networking technology, and rack scale architectures that we're developing jointly together, it leverages the knowledge of the end customers, mapped to what the art of the possible is, but it also leaves a little room for innovation, and that's the beauty of the relationship between the companies.

Dave Vellante

>> in my discussions with AMD folks, listening to Lisa on various calls, Vik was talking about the preponderance of training, and then we saw reasoning save the AI day there for a while and now we're into agentic. Did you see that coming? my point is AMD always has said inference is going to be the larger opportunity. I think you would agree with that. But you had to make a bet there and things changed so fast. Did you see that coming? How did you see around that corner? And what's happening next is really what I want to understand.

Derek Dicker

>> Yeah, no, that's totally fair. First off, what I would say is that the beauty of the company's evolution, if you just look at the overall share gain and the place that AMD plays in the market, I think the company's done just an amazing job of working with all the leaders in the industry. And you see the announcements that are out there right now between OpenAI, between Meta, a number of enterprise customers. Those discussions afford us the ability to see through their eyes where the biggest business opportunities are and where the potential business outcomes are. You couple that with what Supermicro sees in their engagements and discussions, and it starts to put together a picture of what the future may be. You go back in time and trace when we started talking about Agentech, it was back last year as we were having our Financial Analyst Day. We were talking about the impact on the CPU market. We were actually talking about what our CPU technology could look like in the future. But interestingly enough, and GPU, by the way, all of those pieces together, when you looked at it, actually mapped really, really well on top of agentic workflows of the future that we're talking about. So did we see around corners perfectly? No, of course not. But did we have footprints in the sand that we were following as we were developing solutions? Yes. And was that coupled by work that we were doing with Supermicro to understand just the complexity of the market over time? Absolutely.

Dave Vellante

>> So Vik, what were the signals you were seeing at the time and what are you seeing now?

Vik Malyala

>> So one of the things that working with AMD and as they define the processor SKUs, right? Because in a given thermal design, you can pack less number of cores with higher frequency or more number of cores with less frequency, and how the memory mapping should happen, and how the MCM designs, when they're designing, how it fits into the workloads. That's something that we always look at as we design our platforms. And when we do that, again, we would not know whether people would be focusing on these systems to use accelerators as a part of it, or they using it for storage, or they using it for network, or they using it for high performance compute. We would not know. But what we do know is by giving those options in the most optimized manner and be able to build these systems not just at a system level and conceptualize it at a rack level, how we are able to bring a cluster and how we are able to deliver that. That's what the main emphasis is. And it becomes an easier transition for us as we start looking into the AI centric workloads where what GPUs you use and how many GPUs you use, what it takes for you to keep the GPU busy and what new workloads you want to run on it. So again, going back to the point that traditional workloads are not going anywhere. They are still running on it. But the same systems that you can actually use to tune a little bit further in terms of the system configuration to be able to run all these new agentic and inference workloads, that's where the difference is. Yeah, no one has a crystal ball to say, if that was the case, people would have talked about all these agentic optimized CPUs like five years ago. And then why is it only being heard now? Because things are changing. It's just that we need to be ready with the right tools to be able to deliver the solution at the right time.

Derek Dicker

>> I would add just one other thing to that. The magic of a relationship like this is where we can communicate together on things we see coming, but be able to pivot, execute, and if we need to, make modifications. And the beauty is that Supermicro has a great set of technologies that allows them to build different types of systems. Within our silicon architecture, our engineers have thought forward to what are some of the things that we may want to modulate on. That's where the chiplet architecture comes in really well at a CPU level.we can adjust solutions over time.

Vik Malyala

>> Think of enterprise workloads, right? And people want security and sovereignty is one of the parts. CPU architecture already has the trusted execution in that one, which actually gives an enterprise workload the security aspect of it. It's not an afterthought, it's there. It just becomes the right tool available for this -

Derek Dicker

>> It's core to the architecture from the beginning.But to your point of seeing out into the future, There are certain things you can look at, follow themes, you build in capability, you're not exactly sure how it's going to be used, you provision it sufficiently enough, you work with system providers to make sure it can be taken advantage of and delivered to market when it's ready.

Dave Vellante

>> And like you said, that is a benefit, one of the many benefits of the chiplet architecture, it gives you the ability to flex. What is materially new about Venice? And I'm curious, is it specifically designed for this CPU and GPU agentic world? Was it just luck of the draw? And then how does that translate into H15 deployability for customers? So that's where I want to go here.

Derek Dicker

>> Yeah, I'll talk a little bit about the silicon side of things. Venice itself was created with a number of different applications in mind, but really it was the next generation, the sixth generation in the EPYC family. And by the way, the first five were delivered on time with quality with Supermicro standing there right with us at the launch, that only happens through incredible collaboration. Venice itself, the way I would describe it is it's the highest performance solution that is going to be on the market on the x86 side. Generation over generation, we'll see a 70 % performance increase in the product itself. We're featuring PCIe Gen 6 time to market with an ecosystem that we're building wrapped around it, again in partnership with Supermicro. And a lot of the use cases that were originally envisioned were, how do you marry up flavors of this product? By the way, it's 256 down to the base of the stack from a core count perspective. But for GPU -centric applications, how do we make sure that we design a system leveraging Venice in such a way that when you build something like Helios, which we haven't talked about yet, you get an optimized solution out of it? So a lot of work was done envisioning what the application sets could be, from an enterprise platform to a very high performance training rack scale architecture?

Vik Malyala

>> I think you hit the nail right on the head. When we started looking at Venice, one of the things that we started looking at what are the environments that it is going to be deployed in besides the traditional way. For example, we have standard 1U, 2U pizza boxes with a single or dual socket. that's been there, like our CloudDC, Hyper, these platforms. Then we extended that into a dense compute, like FlexTwin, which is liquid cooled platforms, and a 2U4 node a thing. And then we have integrated switch into that for those who wanted to have a dense compute, but in a, let's say, smaller form factor, like a SuperBlade. And the product line extends, one after the other. But then we also have seen people use either air or liquid because things are now being built at a rack scale, especially in a larger deployment arena. So what we did was look at what is the power delivery going to be in these systems, right? traditionally, you have 60 amp or 100 amp power drops coming in, and then you put PDUs, and then you put all these systems together. That's all great. But now we started looking at what is it that we can do with an OCP enhanced or OCP inspired designs where ORv3 rack with power delivery use the DC bus bar. We have transformed the product line not just for the traditional ones but also to take advantage of these things. So things are being considered at a rack scale or a data center scale. What is it that we need to do in a given operating environment and how do we basically pack as much punch in a smaller footprint? You figure out where it would be needed and how do we get the most optimal solution there? So that's the thought process. And we've been working with AMD on how best to put these things together so that it will be relevant for customers.

Dave Vellante

>> And you mentioned OCP, so you're probably, with your liquid cooling, you're probably running warmer water, right, or warmer liquid. And so that affects the environmental. When you think about advances in things like cores, memory, I/O, driving efficiency, what does that mean for customer outcomes?

Vik Malyala

>> Well, so to start with, your workloads are changing. Some of these could be requiring more memory. Some of them could be use take advantage of faster memory. Some of them could be requiring very fast interconnect as systems cannot just handle all the workload in a single system and it needs to be looked at in multiple systems clustered together. So the idea behind this is to give that option for the customers and see how different applications will behave and make sure that we are ready for that.

Dave Vellante

>> Let's get into the joint engineering because a lot has to happen between when you get to high yields and an OEM ready to deploy them. So what is the engineering that occurs between your two companies in between that cycle?

Derek Dicker

>> I think you almost have to go back even before that. You have to go back to where we're talking about what are the products that we want to build and what are the end outcomes that a customer that we want to enable. and that begins with the concept of the actual silicon that's being designed so the relationship that we have is one where we come in and we spend time going through what are the concepts that we're contemplating building for next generation looking at what are the specs associated with the silicon itself and why most importantly what's driving the features that we're putting inside the silicon that's the beginning of the dialogue it then goes beyond that too once we instantiate a design and put a design team on it and they're down a path when we get silicon back as we're prepping to get silicon back we're working very closely together on the system design complexities that exist. We may have to make adjustments along the way based on feedback as they're trying to build a system based on the silicon that we're delivering. Post that, you end up getting into validation, and that's an incredibly tight relationship where if issues are being found, when issues are being found, how do we get a very tight relationship for tight turnaround? Because, of course, we're trying to move very quickly to launch product, and ultimately then, there's validation work that they're doing, testing our silicon and their applications with ISVs on top. all that compatibility work needs to be done in coordination between our two companies also. So it goes all the way out to the point of launch and then even beyond launch, Dave, we're spending time talking through issues that a customer may find out in the field. And so it's from the day that we start thinking about what we might want to build to the day we sunset the product on both sides, their servers and infrastructure and our products.

Dave Vellante

>> And you have to optimize for performance, you have to optimize for reliability, the environmentals, obviously cost comes in. It sounds like you have to make a lot of tradeoffs and then find that perfect balance. Maybe you could describe that process.

Vik Malyala

>> So I think what we are seeing is silicon is getting hotter and we need to figure out a way to optimize the thermal design. So we need to pick the right type of form factor and the solutions associated with it. It could be air cooled or it could be liquid cooled. Because not everyone has liquid cooling in their data centers. So we need to have platforms that can accommodate and provide the highest performance using air cooling. And what does that system look like? So thermal design becomes very important for that. And the second part of it is the interoperability. So obviously people are not just going to use that for a specific workload. Some put accelerators in it. These accelerators especially could be inferencing or network optimization or any of those things, from a security point of view. All these things need to be validated within those designs. And the networking options, whether people want to use Ethernet or InfiniBand or whatnot, and how best you actually put it together, what storage you want to be supporting in that? Because now it's not just about the capacities, but it's also about the form factors too, E1S, E3S, and you're talking about U.2, U.3. And if somebody wants to use Compute Express Link as a part of these devices, and if somebody wants to try UALink as a part of these systems, all these things that we start looking at and say, is AMD going to support it? And if so, what needs to be done? What are the BIOS optimizations that we need to come up with? And what let's say, firmware also that we need to make the system run or operate in a certain way. All these things are the ones that we work hand in hand with AMD. And the designs of the systems are also a function of what we discuss with AMD and also what we discuss with customers. If customer says, I'm looking for a single socket and I'm looking for one DIMM per channel, it's quite a bit different from I'm looking for the most dense compute doing blah, blah, blah. I think the idea again here is talking to customers and getting a pulse of that and looking at what technology that we need to make sure that we'll be validating because we don't know whether it's going to be popular or not, but we still need to make sure that the platform is designed. Otherwise, we'll be late to the market. We want to be a launch partner. We want to bring as many products like the broadest product portfolio that we can bring, air or liquid, rack scale or a single node. All these things are what we are looking at how to bring. And in order to do that, we need to have every bit of the design checked twice. What Supermicro does and AMD engineering comes in and says, hey, if you do this, is the signal integrity going to be OK, is it going to work on this? What needs to be done from our AGESA code for you to do a code drop for your BIOS and things like that? A lot to be done. And a lot is being done. And prior to launch and after launch, I think we're going to continue to refine to get the best tuned solutions for our customers.

Dave Vellante

>> Because you're obviously optimizing for time to market, so a lot of that pre -work is critical in terms of being able to hit that time to market for customers. What does it mean for a customer when you can hit that time to market benchmark? What does it mean? What's the value for them?

Vik Malyala

>> Time to value. I think the whole idea is that you want to take systems that are fully validated and to the extent that one can we run synthetic benchmarks and we run ISV qualifications. Ultimately, it's the customer that is going to take it and use their applications, their workloads to go do something with it. We want to minimize that pain by bringing, time to value or time to get online. Make the systems, bring them to the customer as quickly as possible so that they can have a time -to -market advantage and they can see the benefits quickly.

Dave Vellante

>> What changes, Derek, from your standpoint, when we move from the server to the rack as the key unit? I'm interested in where Helios fits into this equation. Like you said, we haven't talked about it much, but what changes?

Derek Dicker

>> I would maybe take a step back and look at what have the customers been telling us mutually over time? One of the single biggest themes that we get on a regular basis is open, deep interest in having choice in what customers want to be able to buy. And so a lot of the strategy that you'll hear all of AMD talking about is criticality of open, whether it's open source software that we deliver, whether it's open standards that we're a part of. The whole idea is to build an ecosystem of players that can add innovation at the appropriate levels. And so that's something that we're very much involved in and driving ourselves. And Vik mentioned it with the Ultra Ethernet Consortium, UALink, all the work we do around PCIe, just developing standards as a part of that work. All of that put together essentially translates into systems that customers can depend upon, but also get different levels of innovation from different players in the market. So Helios, to your point, is a rack scale architecture that AMD is developing. We're doing it in concert with a number of different players in the market at different levels of the stack. It's essentially a large scale GPU-dense system, 72 GPUs paired with 18 Venice CPUs, all stitched together, with networking from AMD also, our Vulcano NICs coming together. And it's really designed to be validated as a complete system and then done in partnership with Supermicro and others in the industry to go deliver instantiations of this design.

Vik Malyala

>> I call it A plus A plus A. We want to use the AMD EPYC with Instinct as a GPU as well as the connectivity coming from either Pollara NICs or you're talking about Volcano going forward. Why that matters is because I just want to make sure that interoperability is fully and thoroughly done, and it's easiest when it is under a single roof, right? So AMD has been optimizing on their end, and we validate from our end. We still give the flexibility for customers to choose their own architecture for the overall solution, but the idea again here is to validate all these things together and make it readily available for the customers to deploy. Again going back to our initial thoughts, if you are putting Helios, that's fantastic, but what is the right size of compute that you want to put in front of it? And what is the connectivity that we need to be putting? What network topology that we need to bring in order to get the maximum throughput from this one? So you can throw the most expensive equipment and you can actually connect them with the fastest fabric and then you can give all the power and cooling you want and you can get the best performance. But CFOs are not going to like it, right? The idea here is again giving a balanced architecture that is well orchestrated and how we can provide the value to customers. And that's what we are trying to do, especially when working with AMD on this compute plus GPU together to provide an optimal solution, whether it's training or inferencing.

Dave Vellante

>> So when the server is the unit that you're bringing in versus the rack, the customers have to think about that differently, don't they? everything you just mentioned, they have to make those tradeoffs. And then these things are heavy. They can't just, sometimes they can maybe, but they've got to make sure that the infrastructure around that can support it. What do the customers have to think about in that transition from server to rack?

Vik Malyala

>> So the good thing is that I think dense compute is getting more and more common, mainly because of how the industry is trending. If we take a look at our existing MI355X type of systems, Instinct MI350 Series, we are able to put like eight to ten systems in a rack, getting to like 150 kilowatts per rack. Currently, people are already using it in their data centers. So it's not going to be a total surprise to start with. But at the same time, when you're deploying it at scale, oftentimes a traditional data center may not even fly. And we are getting strong customer demand on these platforms. And they know what this platform is going to look like. And based on that, they're actually retrofitting their data center or building a brand new data center. And since we are getting that pulse, what we are also able to do is know how we can optimize these compute platforms to go along with it so it doesn't look like an oddball. We want to make sure that the compute systems are also thought about at a rack scale. Think about the power delivery, the cooling and everything. That way, you don't have to start creating parallel environments with different dynamics in it. That's something that we are working with the customers. Customers are quite open and happy about it because now we are really thinking in terms of what they are trying to do and taking their feedback and start putting them in our platform designs. So is it difficult? Yes. And is it something that needs to be done? Absolutely. Are we doing it? Absolutely.

Dave Vellante

>> Liquid cooling is state of the art. It's going to give you the best performance, the best price performance, the lowest cost per token, the best cost per token per watt. But most data centers today, most enterprises, aren't designed for liquid cooling. Do you think that in the fullness of time, and I use an Andy Jassy statement, that that will change? Will organizations, because of the attractiveness of what they can get out of liquid cooling, rethink their data center infrastructure? Or will they go to the cloud and the neo -clouds?How do you see it?

Derek Dicker

>> I think that the reality is any transition of this nature takes time. There are going to be some customers that value the business outcome and have the capital to deploy to go create greenfield infrastructure to be able to deploy liquid cooling. And that's happening today. We see that en masse in a number of different places. In terms of existing infrastructure where people can't retrofit, maybe it's going to take some time before they get to a point where they're able to even afford to go into a retrofit of their existing data centers. But I think it's undeniable, at least the value, as you suggested, that comes associated with liquid cooling. The reality is for us, between our two companies, we're going to offer both and help monitor the transition over time so that we can give the customers what they need when they need it in time. That's how I simplistically look at it.

Vik Malyala

>> Yeah, I'm not a golfer, but you don't take a putter for a driver. That's right. The idea, again, here is to give the optimal solution for the customer. So if people are doing small scale inferencing workloads, the Instinct MI350X, MI355X, and also the PCIe version, the MI350P, these things are going to just fit fine in an air cooled environment for customers. If they want to go with liquid cooling in a small scale, they can still do the MI355X that can do that. And as for people who are looking for the best in class and the beast, per se, if you want to have it, Helios is there. So the idea is whether it's air cooled or liquid cooled, rack scale or at node level, could be one GPU to eight GPU type of systems, we want to give them the option. And that's what the industry wants and that's what we are trying to do so that we don't want to slow down the innovation. We want to accelerate together.

Dave Vellante

>> That's exactly right.Okay, I want to ask you guys about the, you have a lot of experience obviously with general purpose infrastructure, x86, AMD has just done a remarkable job over the last decade. What happens to the traditional workloads that are running out there, the trillions of dollars of investments that are there? Does that get absorbed? Do the functions that are running on that legacy software estate and the infrastructure around it, the offloads to storage and networking and all those functions in x86 land, do they get absorbed into a new AI infrastructure? How do you see that playing out? Where is that crystal ball?

Vik Malyala

>> If you have it, could you share it? But you're right. I think it's a very good point because traditional workloads, they're going to evolve, right? People are trying to figure out a way to optimize that to the extent possible because now all of a sudden, let's take an example of two generations ago systems where if you want to run an application that requires certain memory footprint, it may not even fit in a given system. So now they need to look at how it's going to run across four or five systems put together and connect with whatever the network was at that time. So now look at 16 channels of memory, 256 cores in a single system, put in a dual socket, you are able to get what, 512 cores or maybe double that number of threads and whatnot. The entire architecture can be redefined in terms of how the software is going to run. So the idea again here is, if people are running certain things, they can do at least the same, if not multiple times better traditional workloads. But at the same time, when they are doing that, they want to take advantage of what is happening with AI. Right, every company is looking to see what is it that they can do to improve the decision making. How do they run agentic workloads and how do they improve the customer experience? Think of customer support calls or internal knowledge base, database queries and whatnot. Every one of these things will be done far better whether you use a traditional workload as is or you optimize it further using some agentic workloads. Both of them can be supported with the same system. So it's prudent for anyone to take a look at what they are trying to do and how they can take advantage of these new let's say systems that are bringing the value to them

Dave Vellante

>> and I think you would agree that obviouslya startup is going to write code that is AI native that could create some interesting competitive dynamics in industries with existing software companies and many of those existing software companies will likely refactor their code for AI and it's a transition it's not an either or necessarily but when we hit these waves you tend to see these disruptions occur new companies come out of the woodwork that we've never heard of before and it is a crystal ball question I don't know if you have anything to add to

Derek Dicker

>> this

>> I would just agree with what Vik said

Dave Vellante

>> I think there's going to be a transition and every

Derek Dicker

>> single enterprise customer is gonna have a decision to make in terms of how quickly they wanna migrate and the way that they wanna migrate. The good news is that existing applications today can be coordinated into an agentic workflow with some level of additive technology. Then for those who wanna go and really invest even more heavily and have the capital to do so, reimagining their entire operation is entirely possible too. I think the ones that are gonna have the most difficult time are those who choose not to do anything and wait and see. I think that's the most important thing that we see right now.

Dave Vellante

>> Yeah, I think that's a key point. You're not going to run your agentic enterprise on legacy infrastructure. It's just that's a failure mode. You shouldn't even try.

Vik Malyala

>> Right. I think going back, I was thinking through, and one of the things that we have seen is, especially the enterprise customers or the people who already have data centers. They are struggling to see how to ensure that they are able to take advantage of the latest gear that is coming in. What Supermicro is doing is to figure out a way to lessen the pain. We have started investing in so -called Data Center Building Block Solutions, or DCBBS, where we are going to help them to retrofit their data centers by bringing liquid cooling into that. Could be like water towers and things like that, or could be power shelves that need to be put in, or change the power delivery to the systems. This way, one doesn't have to go and build a brand new data center, but can take advantage of all the latest within the existing data center by making certain modifications. If someone wants to start brand new, we can certainly help them on that front as well. But to just wrap up the thoughts on that one, if the customers want to take advantage of the latest, they are not alone. We can actually certainly help them on a system level as well as the deployment level.

Dave Vellante

>> All right, let's close with a single action item for IT decision makers. Derek, you can start and then Vik, you can bring us home.

Derek Dicker

>> I think probably the most important thing that I would offer up is it's not a scenario where anyone can afford to wait. I think the infrastructure is changing so fast, the capabilities that are available are changing so fast, and really the most important thing that one can do is understand where they want to head from a workload perspective, put infrastructure in place and go partner with folks like Supermicro and ourselves to the extent that there's an interest. We spend a lot of our time working closely with end customers in partnership with Supermicro to help them evaluate options, to conduct POCs, to take advantage of some of the transitionary technologies and offerings that Supermicro has, and to put new infrastructure in place to take advantage of this exciting time that's coming at us.

Vik Malyala

>> Balanced platforms and architecture is the key. What we are doing together is whether you want to use the standard compute, you want to use accelerated compute, how to put these things together to make sure that it meets or exceeds someone's expectations. In order to do that, one has to do the proof of concept quickly. We can give as much information as possible that we have already done together, but ultimately it boils down to customers, using it as a POC, and very quickly roll it into production. And that's what I think people have to start with.

Dave Vellante

>> Guys, it's been really interesting to learn more about the partnership over these three panels that we've had, going deeper into some of the engineering work that you're doing, the roadmap, I can't thank you enough for spending some time with us.

Derek Dicker

>> Thank you so much for the opportunity, it's been great. Thank you.

Dave Vellante

>> Yeah, you're very welcome, and thank you for watching. I'll just leave you with this, to Derek's point, don't wait. You got to get on that AI curve, and your legacy infrastructure is not going to support your future agentic infrastructure. So you need to think about that and plan for the future. This is Dave Vellante for theCUBE. Check out the panels that we're doing here on theCUBE.net. This is in support of Advancing AI where theCUBE is there as well. Check out SiliconANGLE for all the news and thank you for watching. Thank you.