Name: GPU Access to Business Outcomes: How the Neocloud Ecosystem is Advancing AI
Uploaded: 2026-07-23T18:30:00.000Z
Duration: 1 h 1 min 26 s
Description: Industry leaders from Supermicro, AMD, TensorWave and Crusoe discuss how the neocloud ecosystem is moving AI infrastructure beyond GPU access toward integrated systems spanning compute, networking, storage, power and software. The panel also examines open architectures, cost-per-token economics and the partnerships needed to accelerate AI deployment at scale.

Clips
More from Beyond the GPU: The Full-Stack Fight for Enterprise AI

Ilya Tabakh

VP of Innovation

TensorWave

Umair Piracha

Sr. Director Supply Chain and Strategic Sourcing - Crusoe Cloud

Crusoe

Linda Yang

Director, AI Product Management

Supermicro

Ted Marena

Dir. Business Development for DCGPU

AMD

pause_circle_outline NeoCloud ecosystem enabling integrated AI infrastructure from GPUs to business outcomes

play_circle_outline System co-design: compute, storage, networking, cooling tuned to workloads

play_circle_outline Demand surge and supply constraints for GPUs, CPUs, NAND, networking components

play_circle_outline Crusoe's energy-first, vertically integrated data center strategy with liquid cooling

play_circle_outline Shift from GPU-centric to CPU+GPU balance driven by agentic AI workloads

play_circle_outline AMD roadmap and Helios MI455X: more compute, memory, HBM, memory bandwidth

play_circle_outline Open-Architecture AI Alliance: AMD, Supermicro, Crusoe Cloud Partner with TensorFlow, ROCm Supporting PyTorch and JAX

play_circle_outline Networking/interconnect bottlenecks and need for standards-based Ethernet approaches

play_circle_outline Beyond GPU-Hours: Tokenomics — Cost per Million, Tokens/sec, Tokens/Watt, and Time-to-Value Optimization

play_circle_outline Importance of ecosystem partners and openness to future-proof AI investments

play_circle_outline NeoClouds vs hyperscalers: specialization, higher utilization, faster speed-to-market

47:28

Info
Transcript

GPU Access to Business Outcomes: How the Neocloud Ecosystem is Advancing AI

Ilya Tabakh

VP of Innovation TensorWave

Umair Piracha

Sr. Director Supply Chain and Strategic Sourcing - Crusoe Cloud Crusoe

Linda Yang

Director, AI Product Management Supermicro

Ted Marena

Dir. Business Development for DCGPU AMD

Industry leaders from Supermicro, AMD, TensorWave and Crusoe discuss how the neocloud ecosystem is moving AI infrastructure beyond GPU access toward integrated systems spanning compute, networking, storage, power and software. The panel also examines open architectures, cost-per-token economics and the partnerships needed to accelerate AI deployment at scale.

GPU Access to Business Outcomes: How the Neocloud Ecosystem is Advancing AI

(INTRO)

>> Hi everybody, welcome to this special theCUBE Power Panel. We're here in AMD's Executive Briefing Center, and this is concurrent with AMD's Advancing AI event. And the theme here is from GPU access to business outcomes, how the NeoCloud ecosystem is advancing AI. And I have a little opening statement before I introduce our panel, so let me get into it. Gaining access to GPUs, that was of course a big theme in 2024 and even most of 2025, but today, enterprises, they got to worry about CPUs, accelerators, things like networking, storage, cooling, software portability, utilization of those resources, and of course, time to production as one integrated system. And so today, we're going to examine how the NeoCloud ecosystem brings those pieces together and how the specific alliance between AMD, Supermicro, and two NeoCloud leaders can help enterprises and developers build the next generation of AI applications, both safely and cost -effectively. To my right is Linda Yang, who's the Director of AI Solutions Product Management at Supermicro. Ted Marena is the Business Development Director for OEM, ODM, Compute, and Enterprise AI at AMD. Thank you, Ted, for hosting us here in your lovely briefing center. Umair Piracha is Senior Director for Procurement and Supply Chain at Crusoe Cloud, somebody we've featured in our NYSE Wired programs, and Ilya Tabakh, a VP of Innovation of TensorFlow, another company that we've featured. We've seen the rise of the NeoClouds, it's super exciting, specialized AI clouds that are really designed to power the next generation of workloads and applications. So thank you all for being here, it's great to have you.

>> Yeah, happy to be here.

>> Ilya, I'm going to start with you. Talk about why your company was founded, What was the problem that you set out to solve? And I'm interested in how that's changed since the founding of the company.

>> Yeah, absolutely. So TensorFlow was founded in 2024 with the goal of bringing an alternative to the ecosystem. TensorFlow was the first AMD exclusive scale cloud. And so as lots of demand was coming to the market the founders really were thinking about, do we become the 200th just neocloud or do we do something different? And they made a bet, which was pretty interesting at the time, to be AMD exclusive and bring AMD GPUs to the market. And so there's a lot of need for GPU supply and the way that you deliver it with the different types of GPU processors. There's some trade -offs and differences and things like that. And so they committed multiple years ago now and has been a pretty exciting journey since.

>> And we're going to get into that and explain what's different than a hyperscale cloud. But Umair, when I first met Chase Lochmiller, you guys were building data centers. Now you've advanced. Talk about your founding vision and where you've taken it.

>> So like other NeoClouds, we were a Bitcoin mining company. We saw the problem, Crusoe saw the problem, our founders saw the problem that energy is going to be the bottleneck. So they went to clean energy, stranded energy, and they were powering the Bitcoin mining business. coming two years later, we are now in AI compute, we have a totally vertically integrated story, we create our own power, we build our own data centers, we put Crusoe Cloud, and then we have managed inference on top of it. So I can sum it up that we didn't just add AI to a cloud, we built a cloud because of AI, and specifically because of the energy problem at the center of it, that's Crusoe's story.

>> Well, what does it take to stand up a data center these days? You need land, power, water, you got to deal with local politics and the sentiment, not in my backyard.

>> Don't forget about money.Yeah.

>> Well Ted, that doesn't seem to be a model these days, does it?

>> But yeah. Knock on wood.

>> You definitely need capital knock on wood for sure. You got to love these waves, you got to ride them when they're there. But it's a complicated situation, so maybe describe that a little bit.

>> Yeah, ultimately if you think about it, maybe a decade ago a huge data center was like 30 megawatts, right? And in data center world, when people talked about this, everybody was like, wow, that's crazy, right? But 30 megawatts in most areas is not the majority or even a significant amount of power if you think about a metropolitan area, right? It's not insignificant, but it's not a lot. Now we're talking about facilities that are significantly bigger or distributed facilities that are at that scale but in fleets, right? So you were really talking about closer to an industrial ask that's operating with compute and connectivity and power, obviously. And so you were talking about standing up and connecting with the grid or doing some things in power to be able to operate and function and be a good citizen on the grid. then you were talking about putting together a very performant and efficient network cooling and power infrastructure in order to be able to connect various computing elements together. And maybe more importantly, then you were talking about tuning that machine for the application you were trying to run. And so whether it's any of the AI workloads like training, fine tuning, inference, they all have different requirements on storage, access time, and things like that. And so you've got to build the car, tune the car, and then figure out what race you were running. And we're highly specialized in this type of vehicle and not a general fleet operator to stretch that analogy a little bit.

>> And you've obviously got to rely on partnerships, and we're going to get into that. I want to talk about demand. So Al Shugart was one of the OGs of disk drives, and disk drives are a very cyclical industry. It was up and down and up and down. And if you had overcapacity, you got killed. But Al Shugart said to me one time, look, when there's demand, you have to lean into it. You have to be there when the demand is there or else you miss a cycle and you miss the wave. So that's worse than having overcapacity, if you will. So I want to frame that because everybody's asking, well, when does the demand dry up? But it's an irrelevant question. You have to go at it and try to drive value. So what are you all seeing from demand? Linda, how would you describe the demand you were seeing? And I want to go around the table.

>> Yeah, sure. I can start. if we look at the industry this year specifically, it's a seller's market. So that's where I think we are in the best time of the AI era where the actual demand is surpassing the supply. So we've seen a lot of constraint, not only from the manufacturer side, also from different components that earlier we talked about the NAND the power the networking every piece of this infrastructure is in high demand so we are in the best position to be able to deliver to the fast-growing industry the best solution as Supermicro I think take ourselves as an example we've seen the latest technology from various vendors we've also seen the demand surpass the rate, the speed we can supply. we are also continuously pushing ourselves to the next level to see how we can shorten the time to build, the time to deliver, the time to operate those systems for our partners such as Crusoe Cloud partners here so that we can be successful in this way.

>> And thank you for that. And Ted, if you can get a GPU and a CPU, you'll sell it these days. And I think people were really surprised by the CPU shortage. What are you guys seeing?

>> Yeah, so just like everybody is saying, the demand is insatiable at the current time. What's remarkable, I think, is that a lot of the requirements are obviously model builders, people making AI solutions, and enterprises are just starting to dip their toe in the water. So as you look out over the horizon, it looks like we're going to be in a situation for at least a couple of years where we're at. And certainly we started to see at the end of last year where CPUs were becoming more significant as agentic really started to take off. It had a requirement where significantly more data needed to be accessed and the CPUs were the tool to organize and go fetch that data and then feed it to the GPUs so that's where it's at today and of course we also have all the networking to connect that together and to really provide it at scale and so we saw obviously training was driving a lot of the demand and then reasoning happened and now agenticis happening.

>> And so this is driving more workloads, but what are you seeing, Umair and Ilya as well, in terms of the demand, the profile, and what's driving that demand?

>> So the demand dynamics have changed, I think in the last 12 to 18 months. So people were just asking, where are my GPUs? Now the customers are asking more sophisticated questions, like what is my total cost per token? What is the latency under load? Can I run PyTorch workloads on AMD without retooling? So people are getting very, very complex and complicated demand. And I see the demand is just going to go above and beyond what we are seeing. I think we are just scratching the surface right here. So Crusoe's story is literally the vertically integrated story helps us because we are controlling each and every stack like Jensen and AMD. Lisa mentioned that we have a five -layer cake. So we literally have that five -layer cake. We want to focus on having liquid -cooled, ready -for -AMD next -generation GPUs. We are building AI -purpose -built data centers, and we are controlling power, networking, compute, and everything.

>> How would you describe it?What are you seeing at TensorFlow?

>> Yeah, so I think there's still plenty of demand at the frontier and scaling startup level. And what's interesting is you were starting to see folks start to train smaller models to operate further at the edge. And so some of the model builders that Ted was talking about are training smaller built for purpose models. We've done quite a bit of work in helping folks think through how do you reduce time to value. And so we have a framework that we support called ScalarLM that combines inference and training. And the whole idea is to get around this conversation of, hey, let's build a data center and connect it to the IT network and things like that. And more on focusing on, can I tune and train an application, a model to my specific use case? And by minimizing that time to value, organizations are able to wrap their heads around what is it going to take and baseline their workload. And the other thing that we're seeing is we worked with a major telecom to develop and support the development of a set of tools and models for the telecom industry called OTel. And what's interesting there is they're open sourcing models that are fine tuned for specific telecom applications. And so they're really lowering the bar for the entire industry, not just for themselves. And it's really interesting to see that conversation around how would you deploy this and what would it look like to operationalize it inside of my organization, but then have a real industry discussion on how would we do this in a way that would move the industry. And I think you were going to see initiatives like that pop up more often and really accelerate the whole field.

>> There's an interesting debate going on in our community about the frontier model vendors. They have tons of capital. They want to rule the world. and eat up the whole software AI stack. And then you have this long tail of innovators, specialized, whether it's telco or other industries, that they also have capital, maybe not as much as the frontier models, but they're driving demand. I'm interested in your thoughts on agentic. We talked about the shift from training to the reasoning and now agentic. How that affects how you all build systems and silicon and compile it into data centers? How is agentic affecting that demand generally and specifically the architectures that you have to build?

>> Yeah, totally. I think I will start from the system perspective. So indeed, two years ago, I think a lot of questions all around GPU, how we interconnect the GPUs, make it useful for the training workload. But as agentic AI comes in, we've seen a lot of CPU demand itself. For example, I was talking with Vellum, a startup company in Singapore. They were literally asking us as to how we are going to redesign the infrastructure for them. Used to be GPU -centric. Now we're talking about one -to -one ratio between CPU as well as GPU. And over time, she mentioned that, okay, probably they need infrastructure that's many -to -many, meaning many CPUs potentially to one GPU or many CPUs to many GPUs. So that ratio is drastically changing because of the agentic AI. This is just one example from a startup. But then I think there are lots of AI startups besides the Frontier Model Lab. They are driving new architecture. They are driving new infrastructure design, and then a lot of innovation is going to happen there as well.

>> So, Ted, what does that mean from a silicon perspective? You've got to get data in and out. You've got to connect all these GPUs and CPUs. That puts new challenges on your architecture.

>> Yeah, absolutely. And if you take a look at the transition that we're making, our current architecture class, state -of -the -art is our MI355X GPUs, and we have eight of those that are tightly connected in a pod. As we move to our next generation, MI455X, we're scaling that up to where you have 72 GPUs now, tightly interconnected. And so that Helios architecture is really intended to deliver better performance per watt, much higher capabilities from both training and inferencing, and to really attack the problem in a more system -level approach. So the scale -up and the scale -out network significantly increased to allow the communication to happen more quickly and to deliver the answers and the responses in a more timely manner. So it's more profitable for the NeoClouds and for the customers.

>> Right. And so when you think about what you guys have to deal with, the networking obviously comes into play. I know in talking to various NeoCloud players, you want all flash, at the same time, NAND prices are going up, so maybe there's a role for spinning disk. Although VAST would say, hey, we can show you that the TCO is better if you look at how you can optimize, but you have to take a systems perspective, right? Ilya, what does that mean from a standpoint of architecting, particularly the key pieces, the compute, the storage, the networking, where again, that is the new bottleneck, isn't it? The network?

>> Yeah. In some ways for us, it's always been a systems level problem. We've had the good fortune to work with some of the innovators in the field as they come to building a new solution, new application and scaling. And because of our exclusive focus on AMD, we've become very specialized on what can you do at the kernel level and other things to make the GPU side performant. And we're excited. Ted mentioned the larger GPU universe, but there's also some flexibility in what compute versus GPU trade-offs in the new architecture. And so we have some new degrees of freedom. But I think taking a couple steps back, it's always been a systems level problem. And as folks introduce these new concepts of deep thinking or agentic or things like that that change the GPU to CPU balance, a lot of it is what are you trying to do? What's your actual workload and how can we tune both the hardware and the software stack? And maybe even think about how you deploy this thing, right? Where is your data? Does it sit in a cloud instance somewhere else? Are we coming closer to you and how does that work? And ultimately, if we don't solve all of those things, we can talk about tokens, but ultimately people are interested in value creation. Sometimes they call it maybe throughput, right? And I think that's -

>> Useful tokens.

>> Yeah, yeah, exactly. Right. Right.

>> And so ultimately, at the end of the day, it's like I put a watt or a dollar or whatever in. Am I getting the thing that I expected out? And throughout the history of the organization, we've been focused on that. And this is a new chapter there.

>> Double click on your statement about your bet on AMD. Why that bet? It's counter to what a lot of folks would do in the industry. Why did you make that bet? What does it bring you?

>> Yeah, customers I think today sitting talking to you and all the things that are going to happen around Advancing AI Day, it seems a pretty obvious bet. I think at the beginning a few years ago, at the beginning of the company, it was less obvious. But ultimately, if you were going to have something as ubiquitous and important as maybe electricity, we're talking about intelligence, but it's a utility that folks are going to use in a mass amount, having one supplier and then also very fine control on optimizing these parts of the stack is just not the way to go forward, right? Ultimately, if there is only one GE I think we'd be having a pretty crazy conversation about electricity today. And people make different architectural decisions. And so GPUs aren't exactly the same across all the different vendors. And a TPU, for example, has made some choices that don't happen in the GPU world. And because of that, you have some applications where these chips and these tools are more relevant, more performant, and can make different decisions. And ultimately competition and variability drives better outcomes, and we're excited to be able to contribute to that.

>> If I can add that the goal isn't to standardize on one piece of technology. The goal is to standardize on outcomes while maintaining the freedom of choice.So I think that should be the view.

>> Thank you for that. And then now, when I think about Crusoe, you started as a data center builder, and then you expanded your offerings right around the same time as GenAI was coming in. So you could look at that as, okay, that's maybe an advantage because you could start with a clean sheet of paper. The other thing is you were later to that market. So how did you make that transition and the decision to do that? And what is unique about your capabilities in terms of that system design?

>> So system design is open. The reason we are with AMD is because of the open architecture and being a sourcing and procurement specialist, giving me more options is key to success in my world. And the systems with Supermicro, which really helps us, they are upgradable. We don't have to, we can swap things out. We can swap compute out. This is where the partnership with Supermicro and AMD is very, very critical for this kind of, it's not just a vendor and OEM relationship, which we have with these guys. I think the ROCm maturity has improved. So everything on the networking topology, it opens up doors. And I just don't want to be single sourced to anything. So that's first and foremost specifically coming from a sourcing and procurement person. I need options.

>> Okay, cool. So Linda, let's talk about open architecture.This is the mantra of Supermicro. What does it mean to have an open architecture? What does it mean for a company like AMD in terms of your partnership? and then I want to understand from the NeoCloud players why that's important to you. So what is an open architecture? What are the salient aspects of a so-called open architecture?

>> Yeah, let me try. So when I joined Supermicro two years back, I'm very fortunate to be able to focus on AMD product. So when I look at AMD product portfolio, I'm like, oh my God, there's so much to learn. So AMD not only offers the leading CPU, but also GPU. There's FPGA system. There's also smart NIC and accelerators for financial services companies. So if you look at the portfolio, they open up the big ecosystem where you have different vendors contribute to it, different builders contribute, different architectures that support different software stacks that can optimize and leverage the hardware itself. So when I think about open ecosystem or open architecture, it's where you have the playground that allows different architectures, different software stacks that can run freely as well as most efficiently on that playground. And on top of that, if we extend that open ecosystem to a bigger, let's say, AI ecosystem here, look at our partner customers, Crusoe, TensorFlow, we work closely together to define what we mean by open ecosystem, meaning, okay, we are working closely with each other, we help each other define what's the best for our common customer together. So we also work together with our partner and ecosystem vendor AMD we co-engineer what's out there for the open system together again ultimately Ifeel open system is where an architecture

>> can continuously evolve depending on customer demand so Ted what does that mean to AMD and I'm specifically interested from a developer perspective you hear a lot of talk about CUDA but how do you guys address that and why is that important for developers?

>> Yeah, so there's the open side on the hardware and then the open side on the software. Let's talk about software for a minute because a lot of what we've been focused on is the hardware side and of course we can always go back. So the software side I think this is something that many people in the marketplace are really starting to open their eyes and understand that ROCm is the only other tool that's upstreamed for the popular frameworks the way people are writing the majority of models today. So everybody knows that CUDA can support it, but ROCm is completely upstream and supported for PyTorch, TensorFlow, JAX, those tools. And so that's really important that organizations understand if they're writing in these frameworks for their AI solution, for the model that they're developing, then it will work out of the box, you just use ROCm and it runs on AMD Instinct GPUs and our servers overall are optimized for both the NICs and the CPUs with the GPU tightly integrated. So that open architecture allows choice. It allows potential innovations to occur on the networking, whether there's different types of switches that vendors want to utilize, although AMD makes our own NICs, we also validate third -party NICs as well. So that's what I like to focus on when we talk about an open architecture.

>> Yeah, and ROCm, of course, is your open developer platform for GPUs to be able to program them. So, okay, so for Umair, from Crusoe's perspective, how important is open? What does open mean? We've been using that term since, I remember Unix used to be open systems. So what is open to you and why is it important to your customers?

>> there are literally four layers of openness. from silicon to system to software to networking topology. And I, like Crusoe said, we are experts on each and every layer. That is why tokens are what matter for us because we are controlling pricing on each and every aspect. So that is why openness, as I said, from a strategic sourcing perspective and a procurement perspective opens up doors for me. And literally this partnership with AMD and Supermicro will reap a lot of benefits for Crusoe.

>> So what does it mean for your customers? How do you think about openness? Is it all about not getting locked in? Is it about choice for workloads? What is it?

>> Yeah, it's not a lot of things. I would start with Ted pointed out designing these systems and interconnecting them. We've been fortunate enough to be a first go to market partner on a couple of AMD platforms. And what's fun is from our perspective, we get to have this conversation on, hey, this is a scaled workload on your fabric, on this application. And that's a pretty dynamic relationship. And that openness allows us to bring other partners into that discussion. Ted mentioned having a lot of vLLM and other things fully upstreamed. That wasn't always the case. And we were involved in that process of how do we think about that and how do we demonstrate that this is at parity or better? And that's a dynamic conversation. So that open conversation allows folks that write compilers for kernels on the GPU, essentially the software that runs the code on the GPU itself, that can all of a sudden be an open conversation and it's not in other places. The other part is an acute example of this is in the scale up network, that is a significant percentage of the cost of an installation. And we believe and we're seeing that by open and standards -based approaches to that network, you get more vendors, more competition, and better performance, better pricing. And if that becomes a non -trivial part of your overall installation and an opportunity to squeeze out performance, that's good for everybody. And so those are open in how we can build a type of vehicle for our race team if you will, that analogy again. And then the other part is it really does offer a lot of, especially as enterprises and folks that have diverse workloads and diverse fleets, they can start to think about, can I optimize my workload and work with these special assets and special type of computing elements? And that's harder to do if you only get it from your vendor, because your vendor may not be focusing on it. There may be some strategic shift, and those things are much less controlled in an open setting. So there's a sourcing aspect of this.

>> I'm interested in one of the comments you made. The Ultra Ethernet Consortium, Ethernet is a well-known standard that created the interconnect, but of course you have a lot of discussion in the market. There's InfiniBand, there's Alternative, there's special Ethernets. How important is that open networking standard to someone like a Crusoe or a TensorFlow?

>> Right. Yeah.

>> Why is that so important? Just a sourcing choice?

>> Sourcing choices, networking is a big bottleneck. You see right now, getting compute or GPUs was the bottleneck a year ago, but now getting optical transceivers, that is the biggest bottleneck right now, 12 month lead times. And openness gives me freedom of choices. And partners like Supermicro opens up doors for me because they have relationship with all these vendors on the networking side as well. So yes, from a sourcing perspective, yes.

>> open source is, like somebody says, many times they've said, don't bet against open source, and it's proven to be true. open source Linux was the way...

>> Cisco learned the hard way. Yeah.

>> look, Linux was the way in which Windows the stranglehold that Windows had on the market. But there's a counterargument that says, as well if a closed system can deliver better value, better performance, et cetera, et cetera, you don't buy that that's changing. That open will remain the lever that it's always proven to be over the last 20, 30 years.Can you comment on that?

>> I think for me, and I think just thinking through this, we're in a world of great change, both accelerating demand, shifting workflows, and this is not the first time we've seen this in computing and definitely not the first time we've seen this in technology. Optionality, agility many of these things are really important. And I think that open approaches preserve a lot of what we're talking about in a way that you have some control and some ability to interject in your own roadmap, right? And for a thing that is a strategic core ask, those are all the things that you need before you can procure. But from a performance and an application tuning standpoint, there's a lot of degrees of freedom there also. And so we're not so defined that here's the workload everybody's running. Here's the reference I definitely think maybe that was a starting point for some things, but we're branching and diverging in a pretty active way and optionality, agility, all of these things are very important in order to be able to move more quickly and in a way that you can be sure of the ground that you were standing on.

>> Well, especially as the market changes so quickly. I mean I don't, I certainly didn't predict things like mythos coming out and it caught all your customers by surprise. They are trying to implement AI and agents and governance and all of a sudden they have to worry about all this security stuff and things change so quickly and so it's hard to predict the workloads. Did you have a comment?

>> Yeah, I want to add to earlier the practical aspect of the Ethernet and openness in terms of interconnect. So we all agree that interconnect is super important. And one of the new trend we've seen is enterprise is trying to adopt the AI. When any new technology comes in, it's great technology, but then only we can use it unless, we cannot use it until everybody can adopt it. So if we look at today's enterprise deployment and the interconnect is still purely Ethernet -based. So if you want to bring your advanced AI infrastructure into the existing broadly adopted technology, you have to base that on the Ethernet, which means you can fit your latest technology into the existing infrastructure and be able to seamlessly integrate them and deliver value, especially for the enterprise customer.

>> All right, I want to turn the conversation to how IT decision makers should think about the economics of AI. We hear a lot about tokenomics and cost per token, tokens per watt, et cetera. What matter, somebody I think, Ilya, might have said, or maybe it was Umair, what matter is the value above. We're seeing the whole software stack changing, people building new applications, trying to figure out how to redesign their processes, but we love to talk about infrastructure. So what are the metrics that customers should be thinking about that matter? And what are the ones that you guys optimize for?

>> Yeah, lots of things to point to where do I start. And I think where it's the most important is minimizing time to value and really understanding what this stuff looks like in your context, not in somebody else's context. We talk about benchmarking, we talk about all these things. A generic benchmark that is not representative of your workload is not super valuable. And so really understanding from a talent compute and the entire mix, how can we minimize time to value and really understand how this works in our context is super important. And then ultimately differentiating between exploration and scale. Once you were scaling and as you were thinking about your roadmap on when you scale these solutions and the things that you've built, what does that mean? There's a lot of things about value, throughput, total cost of ownership, but also agility, optionality, things like that that you need to think about. And so a lot of the time people will talk price. They'll say, what is it per hour for GPU? And we've seen at the frontier and in some of the other places in the demand world, that just doesn't work when you don't get support, when you don't have a partner that enables you to do the things that you need to do. And so really focusing on that end-to-end, what does it take us to deliver at scale, is critically important.

>> Okay, but everybody's freaking out about token costs and token maxing right now. Is that the wrong conversation?

>> No, it's not. Most IT buyers are still thinking in GPU hours, which as an analogy is like evaluating a car price based on the price of the engine.

>> So, we need to be talking about cost per million.

>> Yes, cost per million tokens, tokens per second, and then tokens per watt, which is Crusoe's home turf, because we take facilities cost, sustainable energy, and then literally can point out what token per watt cost is. and Crusoe's success story so far is that we are purpose building the AI infrastructure based upon how we control each and every layer of the stack and then the token prices reflect that decision making what we do on each and every layer.

>> I would imagine if energy were plentiful and free you might rethink how you design data centers and systems but it's not and so how do you address this challenge, Ted?

>> Yeah, everything that these guys have been saying, in terms of the market demand and what the requirements are, it boils down for us fairly simply. It's like every generation from the GPU, just specifically, we got to deliver more compute, we have to deliver more memory, and we have to deliver more memory bandwidth. That is just a requirement. If you look at the introduction over the last several years, we've been on a yearly cadence where we have been doing that. Every GPU that we come out with, MI300X, MI325X, MI355X, and now Helios MI455X. It's more compute, more memory, more memory bandwidth. And that, for a given application, we're doing it more efficiently. So even though each GPU generation is increasing in power, it's more efficient when you look at it on a per watt type of basis.

>> Yeah.

>> And so that's one aspect. And then the other aspect is obviously the networking and the CPU part of it and having that all as a system. And so as we build up these clusters and as we build up larger pod sizes, delivering that allows these guys at Supermicro to be able to offer those solutions that are valuable for customers.

>> Stay on the roadmap for a minute, if you will. Can you walk us through, paint a picture of the AMD roadmap? You talked about 40 to 50 % improvement. Is that annually? Is that typical 18 to 24 month Moore's Law? Help us understand better that roadmap for EPYC CPUs, GPUs, and the performance improvements that we can expect.

>> Sure, sure. So if you look at a lot of these models, the number of parameters just continue to explode and grow. They found that the larger these models, the larger the parameters, the better accuracy. And so one of the big benefits is having more HBM. So that's the high bandwidth memory that's tied to the GPU. So this is just a simple indicator. If you go back to 2023 with MI300X, it had 192 gig, right? The latest generation, which will be the MI455X, has 432. So it's like more than doubling in the three year period there, right? So as we deliver, and we have been on an annual cadence of introducing architectures, that's been the demands that the market has put on us to bring to market. And with the CPUs, we're increasing the number of cores. right so it's the same story where the business units are working together now both our networking group our CPU group and the GPU business units where we're figuring out okay if the CPUs can hand the GPUs the data faster lower latency then it improves the GPU performance same thing on the networking side leveraging Ultra Ethernet leveraging these other faster throughputs on the NICs so the GPUs don't wait for data. And so all of those are the factors of how the roadmap has evolved. And honestly, at the end of the day, I think Ilya talked about how it doesn't matter if you benchmark, but if you want indicators, you can go to MLPerf. You can look at a lot of public data about how AMD is competitive and often better than the alternatives that are out there.

>> And to what extent are you compressing your time to market, time to tape out, time to get high yields, and then you have to make these systems deployable. So to what degree is the time scale compressing and how are you keeping up?

>> Yeah, so sometimes I work very closely with Supermicro.

>> A little bit of pressure on Supermicro, not much.

>> Yeah, and they are a terrific partner. I do want to say, honestly, it is not easy. A yearly cadence is brutal, just to cut to the bare bones. And we can only really work with partners that are able to run. People have to, we have to sprint to be able to offer these solutions on that type of cadence. So I'll let Linda address the question, but really what we're doing at AMD, we have a chiplet strategy and we're able to do a lot of stacking with a lot of different types of architectures. And so that's one of the key differences that allows us to deliver on this cadence.

>> Yeah, I want to add to that, but getting back to your earlier question to take a step back when we look at the cost, it's all about the tokenomics. it's all about from product management 101. It's always ROI, return on investment. So if companies are going to spend money inventing new chiplets, inventing new systems, and even, have new deployment, it's all about what's the return on your investment. So to accelerate or increase the return on investment, what we do is we shorten the design and development cycle. we also shorten the integration cycle, validation cycle, we shorten the deployment cycle, and then on top of that, we'll be able to operate most cost -effectively at scale. So I still remember one of my practical customers said, oh, Linda, okay, this GPU, although you tell me the best performance per watt and most performance for my dollar, but it turned out to be the most expensive GPU I ever bought. Why? She doesn't have the power to power that GPU on. imagine you have such an expensive investment sitting idle there for a month, that becomes the most expensive investment. So that's why I tell her, listen, why don't you go to Neocloud?

>> I think the token cost will fall significantly, but the organizations or neo clouds that are building measurement infrastructure and operational discipline now will basically reap the benefits at every efficiency gain which is provided by AMD in the future. So the companies or organizations which are waiting for costs to fall before investing, they will always be chasing.

>> And for us, speed to market and intentionality has been the DNA of our company, right? And there's a lot of connected parts, obviously sourcing, the sites, integration, the feedback cycle of, is it doing what you think it's going to do and where are some opportunities to improve? And also bringing other partners. But being able to understand what the degrees of freedom are and where you can interject. And for us specializing around AMD, Ethernet, thinking about these systems gives us a lot of clear insight to where the bottlenecks are and the ability to control that, not always end to end, but intentionally. And so we intend to continue to bring some of that experience and maintain, this breakneck speed as Ted's talking about, because speed is the moat.

>> I was going to just say one other thing. Dave, your question about how are you delivering? How are you working with partners like Supermicro? One of the things that we were very grateful when we were developing MI355X, Supermicro had sent us hundreds of servers that AMD uses internally to do all of our validation, significant percentage of our testing. And so without having a partner that helps us do those type of things, this yearly annual cadence would basically be impossible to deliver.

>> Yeah, I want to add why I said earlier I want to cry. Because what happens at AMD they innovate at really lightning speed. So start with the MI300X, then MI325X, MI350X, MI355X. And almost every nine months, there's a new generation coming in. Many of our competitors, they have to skip versions, meaning they have to skip intermediate ones to be able to catch up. But Supermicro, we're stubborn. And we say, okay, we're going to deliver every generation with AMD product whenever they have the innovation. That's where we can supply that earlier generation and also leverage those to develop the newer generation and find value in the newer generation.

>> Yeah, not everybody needs the latest, and having N -1 is sometimes okay.

>> And having said that, I want to put a finer point on it, you were right, token costs are going to continue to plummet, but if you were waiting for that, That is, well, our research has clearly shown.

>> Not a good strategy.

>> no, it's not a good strategy because it's so much more than just price per token. Yes. It's the degree to which you can learn and adopt AI in your organization, refactor your business processes.

>> It's constant evolution happening in real time here.

>> The faster you get on the AI curve, the more value you were going to produce. And we have predicted, if you look at the average spending on IT across the board, around the world, that's about 4 % of firm revenues are spent on technology. That is going to explode. So you need to be prepared for that. Now, a lot of people don't want to hear that message.

>> Certainly a lot of vendors don't.

>> No, no, no.But think about it for a minute. You're going to be able to double your revenue, let's say, with the same number of employees and maybe spend, I don't know, 10, 20, maybe even double to take it from 4 % to 8%. So it'll go from a billion to 2 billion, but your labor costs are going to stay at $1 billion instead of going to $2 billion. That's real ROI, and that's what we're starting to see, more than glimpses of that in the marketplace. I want to ask you guys about what happens. the story of AMD is remarkable. Jerry Sanders famously said real men have fabs. And then you see what Lisa Su has done, and the whole fabless environment has just changed the game. what AMD has achieved in x86 excellence has been remarkable. Now we're in a new era. My question is, what do you think happens to general purpose computing? Does it get absorbed into AI computing? Do the functions that the offloads, the storage management, the security, all the stuff that x86 has traditionally done, does that get absorbed into AI computing? Is there a bridge? How do you think about that?

>> Yeah, I'll take a stab at that. I think that there have been so many applications written over such a long period of time that that's likely to continue. Now, you could argue that with a lot of the AI tools coming on, some of those could be converted over, but I think you were going to continue to have quite a percentage of the market that will continue with those applications as they are. AI, in many cases, is going to help accelerate it. It's going to provide innovation on top of those applications. And for sure, some applications will likely be replaced. But I think a lot of these software companies, what the market has been struggling with, if you look at the valuations of what's going on, like, oh, they're all going to go away. The reality is a lot of those software companies have a lot of data. They have access to a lot of information and knowledge that AI just doesn't understand because it's not public. And so I think there's always going to be that space available. And the software companies will use AI to grow and innovate.

>> So it's an interesting debate that's going on in our community, because if I turn to the NeoClouds, if I can develop, AI native applications on your clouds, and they're better, less expensive, faster, et cetera, that's going to create gravity and people are going to gravitate to that. How do you guys think about that? Because you were the white sheet of paper, the green field.

>> Yeah. I think it's a little bit of a broader question. If you bolt on these AI widgets into the thing you've always been doing, then what you've been doing is going to look very similar to what you've done before. And being fortunate enough to work with some of the innovators in the field, the frontier folks and things like that, they are not rebuilding a thing that they've been doing for 20 or 30 years. And so as folks apply these new things, and I think there is a debate, can you create new models, whether they're world models or foundation models that take much bigger bites out of the traditional compute model. And I think there's a lot of aspiring folks and a lot of physical compute and things that are still coming. That's going to be tested.

>> Yeah. And we're going to be talking about it very actively over the next couple of years.

>> But I think that the optimal choice has been delay of the commitment to the balance between these things. Even before the latest Cambrian era of AI explosion, there was this debate within Google, for example, and before on do we go CPU or TPU? And I've talked to a couple of folks that were at the table and it was a very active debate and everybody's like, oh no, we got to go these special accelerators. And it turns out that the optimal decision then was to delay the commitment as far as you could because it preserved the most flexibility because their paradigm was changing so quickly. And that seems to be the case again agentics is just making it a little bit more acute in this last cycle. Because if everybody would have committed all GPU all the time, no CPUs, right, then we would have a pretty different conversation today. And I think at this pace of change and the diversity of these applications, I don't think it's so easy of a question. But again, what are you trying to do end to end? And how do you maintain speed, agility, and flexibility is probably what I would recommend.

>> I can add on. The premise of the question could be, how do we future-proof today's investment? This is where AMD's open source comes in, this is where Supermicro's modular system design comes in, standard networking comes in. And I think the architecture that future-proofs you isn't the one with the best specs today, it's the one with the most freedom to adopt better specs tomorrow. So that's I think what the future holds.

>> So that's a good lead. Did you have a comment?

>> Yeah, I do want to add, we do learn a lot from history. We also build today based on history. So over the years, the industry has built so many things on the x86, on the CPU. Fundamentally GPU is smaller core, a lot more cores, CPU has bigger cores. So they focus on different workloads. They solve different problems. Imagine we have database today, they're all running on CPU -based infrastructure. There's no database running on GPU yet, maybe in the future, but in the short term, we don't see that. But then we all know almost the society running on top of database, whatever database it is, we're running on top of database, even AI. So in a way, I do think there will be a long time where they will coexist between different infrastructure. That's where the openness is to bring choice.

>> I personally think it's going to take the better part of a decade, but I do think that software companies are going to rewrite to ROCm, too, because I think they're going to get better performance, better economics, and I think there's going to be a lot of energy around that, and we'll see. But I want to come back to the, I want to pushing the NeoClouds a little bit. Why, it's an obvious question, and you can answer this. but why do we need a NeoCloud versus, say, hyperscale clouds? What do you do specifically that means that we need a company like TensorFlow or Crusoe?

>> Yeah, I'll jump in and start. Ultimately, hyperscalers have a lot of applications. They have software, they have different types of loads, and they're very good generalists, right? And they've been doing it for a while. And in many cases, they've served the enterprise well. The Neo Cloud and really a dense AI data center is a new type of ask. And the basic physics of this is that you need more GPUs, more closely connected. And if you were going to go for that density and if scaling law holds, you were going to need more cooling, more power and bigger scale up networks. And that's all new at the scale and the pace that's happening. And so being a generalist doesn't serve you well because you have an existing fleet. You have a pretty wide mix of expertise, and many of those things are not applicable in this high density, more of an industrial type setting. And so that's why TensorFlow really focuses on how do you put all these pieces together and then tune them for the race that you were running. And that's not often a question that the hyperscaler gets asked specifically. And so as this becomes a more relevant and common ask, maybe that will change. But today, speed, expertise, and especially with an AMD, an open silicon strategy, you have to also understand all of the software, hardware, co -design questions. And that's just a very small group of folks that are experts. So that you don't have the technical debt, essentially.

>> Yeah, and hyperscalers are optimized for breadth, not depth, as he mentioned, because they have massive diverse customers. They have workloads of different types. And so they are targeting the median customer. Crusoe or neo clouds, and specifically Crusoe, we are an energy -first company. We think about each and every component layer, clean energy, stranded energy. We go where the, we take the compute where the energy is. And that is, I think, the future of all the neo clouds who are going to be successful and to solve a model which the hyperscalers are not able to. The speed is the moat, as we already discussed. Our pricing reflects because we are controlling each and every layer of the stack.

>> So history would suggest that's a defensible position. Yeah. Because a specialist who's focused gets a foot in the market and achieves escape velocity and then good things happen. What happens, however, when the massive demand that you were seeing now and supply come into balance because while the hyperscalers are generalists, they also have other capabilities up the stack. They have their agent builder capabilities, their new work surface, and so they've gotten some other capabilities. Do you see yourselves, how do you future-proof the neocloud? Do you see yourselves getting into that type of business?

>> From the Crusoe perspective, we are building data centers for hyperscalers. So we have a big project, Stargate, Abilene. We are building for Oracle; OpenAI. we are building now for Microsoft, we have a couple of deals with Meta. So that is one side of the business for Crusoe and then we are building for ourselves. I think whoever controls the energy and the space and the power has the future in the AI race which we think Crusoe will have a clear advantage because of the vertically integrated story which we have.

>> How about you, Ilya?

>> Yeah, some things are similar. I think the real difference is we're seeing, back to this, are organizations going to do more of what they've done versus do net new? The utilization on these things is way higher than a traditional hyperscale data center. traditional hyperscale 1 .0 was built for e -commerce, right? Twice a year, you need your infrastructure to scale 10x. And I'm not trying to reduce their business to being so simple, but that was the initial driver, right? That it really was a scale up, scale down, but a relatively small utilization. What we're seeing from some of the innovative companies in this space is it becomes a top two or top three cost line item. And the way that you go to market there and think about is that core and how you manage your cost is very different. And I was mentioning earlier that everybody was bickering about dollar per GPU hour. That's not how you manage your core infrastructure. And ultimately, if some of the bigger problems that we were talking about get figured out by going to an AI approach or an AI hybrid approach, then that utilization pattern's gonna continue to hold. And then making sure that you were picking the best partners that are specialists in this field is gonna be a more important conversation, not a less important conversation.

>> So just to put a finer point on that, you were talking about utilization. For example, the utilization of a GPU, you can get a much higher utilization because you were specialized. Is that correct? Or are you talking about this entire infrastructure?

>> For inference, for example, that can be somewhat bursty based on what your user demand is. But if you have this deep think or batch type stuff, you may get to the point where you were using 80 % or 90 % of your GPU capacity continuously, right? If you think about the value that's created and what costs you would have to maintain. In a cloud setting, that means that I have to pay essentially for baseload provisioning, right? And that's really hard to do in a traditional hyperscale setting. Now, at the same time, we serve the hyperscalers, we have those conversations as well. But just generally from a business standpoint, think about how you would manage your top two or top three cost line items in your company. And that's why a partner and specialist becomes an important part.

>> Now, you guys are arms dealers. You sell to the hyperscalers. You sell to the service providers. So what is it about the relationship with the NeoClouds? Obviously, NeoClouds is a real thing, and you want to be in that business. It's helping drive your income statements and demand. But what is the uniqueness of your relationship with the NeoClouds that future-proofs them?

>> Yeah, to us, at least to myself indeed, I feel whatever comes to the success ultimately is the people. So to me, whoever can build the strongest ecosystem and who can help us deploy the latest technology, the fastest and operationalize them most cost effectively at scale. So those are the metrics that support the success for any technology, not only AI. I'm pretty sure there'll be new technology coming down the road, but that's always the principle I've been following.

>> First to market, be able to scale it, be able to deploy it, multiple data centers across the globe, I think is a story for success.

>> Ted, for you, having a lot of successful customers is a good thing.

>> Yeah, 100%. And we obviously look at the market, it's not homogeneous, right? So that's why we have a range of CPUs with all kinds of cores, right? That's why if you look at our GPU offering, We have everything from PCIe cards for enterprises to dip their toe in the water, right, all the way to the 8 -way servers and then AMD Helios, right? So we're up and down the food chain because we know some people are at different points on the journey.

>> Okay, last question. Go around the horn. Linda, we'll start with you and go around. What is the one action item that you would recommend to IT decision -makers as it relates to AI in the future?

>> Have the strongest ecosystem partners, including those at the table.

>> Yeah, my answer would be basically dictate that your AI journey is going to be open, right? So driving a requirement where, for example, frameworks are used to allow optionality for different software stacks, right? as well as the architecture itself. If you have the Ethernet as the infrastructure, you'll, again, have optionality and choice.

>> Yeah, AI infrastructure has moved from GPU access to a full -stack systems challenge. And I think this is exactly where the AMD Supermicro Crusoe ecosystem is designed to deliver because we are purpose -building our AI infrastructure, open at every layer, optimized for economics that matter. So my call to action is for everybody to run a cost per token analysis on current AI spend. And you can learn a lot. Yeah.

>> And for me, it's not a lot of thought on how do you minimize time to value, increase your velocity of understanding and exploration, but understand what that translation means to scale. And don't do it myopically on cost per token unit or whatever. It really is an end-to-end problem. And you have to cover the, how do I create value in my organization? And then understand what does that mean as that transforms and changes the trajectory of the org. And obviously, with partners like AMD and Supermicro and others in the ecosystem, TensorFlow is excited to continue to be first to market with a lot of these.

>> Well, I have to say, Linda, Ted, Umair, and Ilya, it's humbling for us at theCUBE to be able to document and participate in watching the incredible innovation and the hard work that's going on. It's just remarkable to see how fast the vision of your founders and all the people in your organizations and the hard work that they're doing. So thank you so much for spending some time with us. Demand is there. We just need to execute.

>> That's the hard part. Everybody can say that. That's the hard part. Yes, speed is the moat. Yes, and that's why these relationships are very crucial. I appreciate it. Strategic partnership, yes. Thank you.

>> And I'll leave you with this. if you were already leaned in, go faster. And if you were waiting, stop waiting. Because the sooner you get on that AI curve, the sooner you were going to be able to drive value for your organization. Thank you for watching this special Power Panel around advancing AI. I'm Dave Vellante, and you were watching theCUBE. Thank you. Thank you.