Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud | Corey Sanders & Peter Salanki, CoreWeave

Clips
More from Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud

Corey Sanders

Senior Vice President Product

CoreWeave

Peter Salanki

Chief Technology Officer

CoreWeave

Dave Vellante

Co-Founder & Co-CEO

SiliconANGLE Media, Inc.

play_circle_outline CoreWeave's 45+ Site, 1GW+ Data Center Buildout Fuels Rapid GPU Cluster Deployment in Weeks

play_circle_outline End of Row Liquid Cooling to 45°C Using Primary and Secondary Loops with Valvey Valves and Racky Manager

play_circle_outline Mission control: global observability, automated remediation, customer job tracking, topology-aware orchestration

play_circle_outline Intra-rack copper backplane delivering ~260 TB/s; scale-out via InfiniBand/Spectrum-X multi-planar networking

play_circle_outline KV caching and object storage to feed GPUs and serve models from global capacity pockets

play_circle_outline CoreWeave differentiation: AI-native hyperscaler with 90%+ GPU utilization and different resilience engineering

Info

Corey Sanders & Peter Salanki, CoreWeave

Corey Sanders

Senior Vice President Product CoreWeave

Peter Salanki

Chief Technology Officer CoreWeave

Dave Vellante

Co-Founder & Co-CEO SiliconANGLE Media, Inc.

HOST

search

Dave Vellante

>> Thank you, John, and thank you Palo Alto. This is Dave Vellante here at our NYSE Wired CUBE Studios. We're going to talk about going from silicon to production. Really, CoreWeave's full stack behind Vera Rubin. The industry is a buzz with the first Vera Rubin rack shipping. Corey Sanders is here as the senior vice president of products at CoreWeave. And Peter Salanki is the CTO of CoreWeave. Gentlemen, welcome.

Peter Salanki

>> Thank you for having us.

Corey Sanders

>> Thank you.

Dave Vellante

>> Thanks so much for coming into our studio. And Peter, I want to start with you if I can. You guys have made extraordinary investments, 15, 20 plus billion dollars in infrastructure. Where do you stand today in terms of data center, data center build-out, the capacity that you have, and what does the backlog of demand look like?

Peter Salanki

>> The backlog is insatiable, right? And I don't know what the current public number is of data centers that we have live, about 45 of them, maybe more. And we have over a gigawatt of power live and we've got a lot more in the pipeline. We're building as fast as we can. We could build 2X to 3X as fast as we would. It's kind of what we're trying to do. It's very hard. We have gotten down the number of days from, we cannot get power on GPUs in hand of customers to almost single digits, which we're very proud of. And we've done this through over six years of learning how to build these things both fast and reliable, and how to build a stack that is purpose built for AI versus general hyperscaler type computes.

Dave Vellante

>> Yeah. I think you're actually up over 45 now. You said a gigawatt. People talk about building data centers. It takes 30, 40, 50 billion dollars to build a gigawatt data center. You're there in total capacity. The other thing that I've read about CoreWeave and one of your differentiators is the speed with which you can get these centers up and running. It's weeks versus typically many, many months or more.

Peter Salanki

>> Yeah, exactly. I think that we're pressing below weeks now as well. And a lot of that is building it right from the ground up and everything around it. People ask me, "What is your secret sauce? What is the one component?" And there isn't really one component. It's everything from how we design the data centers, how we make modular components. So if we need to late bind and shift around what type of capacity we put in there, we can do so. And then we go into, how do we do the physical cabling, right? What patterns, what methodologies have we built, what tools do we give all of the... Sometimes we have 1,000 contractors there pulling the hundreds of miles of cables. In 100,000 GPU cluster, you will have over a million connections between all these GPUs and components. So how do we build the tools? We have people running around with iPads, color code things, all the systems we put in place so we can do the physical part quickly. And then when the physical part is done, this is really where all the exciting software kicks in and all the automation where we can take all these components fresh out the factory, configure them, test them, burn them in, as they call, together and deliver them as a fully working, 100% reliable cluster to a customer within a couple of days. And without a lot of software innovation, and honestly, a lot of hardworking, so hard learnings over the last couple of years, that would not be possible.

Dave Vellante

>> Well, for years, the industry marched to the cadence of Moore's law, basically doubling performance every couple of years. Now we're seeing that concept of extreme co-design that Jensen talks about. You guys, certainly Dell brings that to its engineering. You guys are deeply embedding that concept in your engineering with the hardware and software. And Corey, everybody's excited about Vera Rubin. The incremental improvements are, they're not incremental. They're orders of magnitude better. I'm interested as you bring Vera Rubin to the CoreWeave portfolio, what kind of workloads we can expect people running that they couldn't necessarily run in previous generations?

Corey Sanders

>> Yeah, absolutely. Well, so obviously the significant amount of customers we have today, the IBMs, the Coheres, the Mistrals running today on the Blackwell fleet that we have, the massive fleet that we have, they're doing frontier cutting edge training, right? They're building the next generation of model and building next generation solutions on top of it. What I'm really excited about... And they'll continue to do that. I think they'll continue to grow and develop on that. What I'm excited about as we look at Vera Rubin is a couple of things. I think one, certainly they'll be able to do more with a smaller amount. A lot of this is around better utilization, cost savings-

Peter Salanki

>> That's a big-

Corey Sanders

>> ... being able to be able to deliver the outcome with much less cost. And then I also think that the exciting part about Vera Rubin is the inference side of the house. As we see, the inference market has grown exponentially over even just the last couple years. And I think the opportunity for Vera Rubin to now play this really interesting role of both supporting this massive buildup of training while also supporting a huge opportunity for massive inferencing at a cost and a performance that I think before would have been impossible. And so brand new workloads, I think are coming to life as part of it. You can see I'm very excited.

Dave Vellante

>> And I'll tell you, our whole industry is excited, and it's the advancements. I remember one of my first supercomputing events that I ever went to, I was looking at the hoses and the liquid cooling infrastructure and it was this spaghetti mess and I was like, "Ooh, what happens when one of those hoses leaks?" It's not good.

Peter Salanki

>> Very green.

Dave Vellante

>> .

Peter Salanki

>> very quickly.

Corey Sanders

>> Yeah.

Dave Vellante

>> We've talked about the underappreciated things like connection integrity on hoses and the ability, what you guys are bringing in, you've got these new software capabilities that you've built. I think patents or patent pending innovations for Vera Rubin, Valvey and Racky-

Corey Sanders

>> Best names in the industry.

Dave Vellante

>> ... which is, I love it. Very cute. So novel software to deal with monitoring, observing. Explain what those innovations are and how they fit.

Peter Salanki

>> Sure. So traditional compute, like your laptop here, your computer at home is air cooled, right? You get air in, there's a fan that blows it. And it works fine. And it's not very efficient, but it's easy and there's no risk that anything gets wet. With the Blackwell generation of compute, we and the broader market started introducing liquid cooling. We have been extremely aggressive deploying liquid cooling. Pretty much everything we've done Blackwell and forward is liquid cooled. We're doing liquid cooled networking gear and CPU now as well. We think that going forward, everything's going to be liquid cooled. But liquid cooled comes with more challenges. Now all of a sudden you have a completely separate fluid system which is a lot more complex than air, which is also fluid. So when we look at the previous generation, Grace Blackwell, it was the first rack scale system, the first that really needed to be liquid cooled, we were working with technology called in-rack CDUs. This means that every rack has its own heat changer and the secondary loop, as it's called, is pretty simple. You have service one rack. And this has some benefits as a relatively easy way to start out. It's self-contained, but it's not necessarily as redundant as a larger scale secondary loop. Kind of awkward to try to fit all of that heat exchange power into one rack. So what we do with Vera Rubin is what is called end-of-row liquid cooling. So that means that you have one loop that runs in the building, that's what we call the primary loop that comes from the main chillers outside. And also with Vera Rubin, we're able to push up the primary loop temperature as high as 45 degrees Celsius, which means we can use outside air with very simple heat exchangers, even without compressors to cool the facility, which is great from a PUE, environmental efficiency standpoint. So that's the primary loop in the building. And then the secondary loop now that serves the individual GPUs, the individual racks, that will now serve, let's call it seven to eight to 10 racks. And then you have a much beefier heat exchanger, it's called an end-of-row CDU, much more redundant system. But this adds so much complexity because now you have a bunch of different systems, but a bunch of different servers, which will do different things, have different temperatures and need service, might break at a different time all tied together in this very critical, what we call secondary loop. So what we built to manage all of this is what we call Valvey, which is our own valve assembly. It's basically a bunch of pipes kind of similar to what you have in your house, in your basement, in your plumbing, but much bigger and then a whole bunch of sensors, much bigger. And then a whole bunch of sensors and valves all tied together and all automatically controlled in real time. Millisecond response times on these things. So it means that we can respond to demand from a GPU churning up. We can respond to demand, feed more fluid into that one in real time. And then you have the rack manager, Racky the rack manager. It's responsible for the bigger picture within this, what we call a row or a pod. It takes in telemetry from the GPUs itself, takes in telemetry from the power systems, takes in telemetry from different leak sensors and from the buildings, building management system and allows us to tie all these things together. It's still deployed locally in the pod because there's a lot of physical cables, different sensors, and we want it to be distributed and self-contained. And then it will then interact with Valvey, it will interact with other systems upstream and downstream to make decisions. How much power do we need here? Are we overloading something? We can do things like oversubscription of a building where I say that the building is specked to host 1,000 GPUs, but we can actually put 120,000 GPUs in there because we know that everything is not going to be active at the same time. Sometimes something is going over an upgrade or being debugged or a research is sitting on a GPU but they're not actually using it. And now with Racky and Valvey together, we can redirect more power and the liquid cooling fluid where it's needed and get a much better efficiency at our data center without risking leak events or power overlooks.

Dave Vellante

>> So this just underscores the engineering work that CoreWeave has done. What you described is very complicated. You've got the primary fluid network, you've got a secondary fluid network, you've got the direct liquid cooling right onto the chip. And so plumbers are becoming artisans.

Peter Salanki

>> Yeah.

Dave Vellante

>> We know they're in short supply.

Peter Salanki

>> Right.

Dave Vellante

>> So amazing work. So if I understand it right, Racky is sort of the ex officio, of the umbrella and it shows the health of the system. Valvey goes deep -

Peter Salanki

>> Yeah. Valvey is a liquid cooling component of it.

Dave Vellante

>> Yeah, amazing.

Peter Salanki

>> And on top of that, it's our mission control software stack. There's overarching, which then talks to Racky that talks to Valvey. And then mission control handles everything from tracking customers, either inference jobs or training jobs, seeing, are they progressing? Is there a performance degradation that we can detect, and can we tie that back to anything wrong from all these signals and both remediate it? It might be, okay, we see that this component, this GPU is underperforming for some reason. Then we want to take it out automatically without interrupting the customer and without them having to do something. So mission control system looks at it globally and can isolate different pieces right around underperforming components, which will then signal Valvey and Racky to do their thing.

Dave Vellante

>> And that enables you to use ambient air, as you said, essentially warm water, which OCP challenged the industry to step up to several years ago. So that just translates into lower cost.

Peter Salanki

>> Yep, lower cost and higher reliability, right?

Dave Vellante

>> Absolutely.

Peter Salanki

>> That's all .

Dave Vellante

>> Thank you for making that point. Now, I know coming off the GPU, the performance of these, the bandwidth is amazing, but the intra rack bandwidth is 260 terabytes per second.

Peter Salanki

>> Yeah, .

Dave Vellante

>> That's just a single NVL72 rack. But so what happens when you scale out and you got to stitch these things together?

Peter Salanki

>> You have to scale up bandwidth, that's what is within a single rack. And you can also ask yourself like, "Okay, why can't we scale that out?" So one of the tricks with a scale-up with a single rack is that on the back of this Grace Blackwell and Vera Rubin racks, it's all copper connecting these components together. And copper is very efficient. Unlike optics which are used in a fiber network, it doesn't use power, it doesn't create a bunch of heat. And it's a pretty reliable signal. So connecting a lot of bandwidth together is easiest and best done with copper without wasting 50% of your power budget in just heat loss to turn electrons into light. So when you want a lot of bandwidth, you can do it with copper. But copper, with its difference from light, is that it has a lot of signal loss over distance. So you can really only get as far as one rack reliably with copper. And rack takes up less space on this table. It's this much space and you'll see it this . Why would you put so much power into this rack? And if you haven't followed the data center evolution, like a rack, a normal server rack, 2019 when I got into this game, it was 16 kilowatts. So many data centers that were all purpose built for this, you could fit two of these racks and then the rest of your room would be empty. So why do you build it like that? It's to shorten the signal path so you can use these copper back planes to connect them together. So that's your scale-up network. That's what we get in one rack. That's why we can have these insane numbers . But as you know, AI labs and enterprises are not happy with the one just rack, the one rack. Then my business would be so easy. . So we put thousands of these in huge data centers, we need to connect them together. That's where what we call a scale-out network is. That's where we would use either InfiniBand or Rocky. We work with an NVIDIA Quantum InfiniBand or NVIDIA Spectrum-X Rocky. For Vera Rubin, there will be a lot of long Spectrum-X Rocky on the Spectrum-6 chip. And it's what is called a multiplanar design, gets very technical, but you basically build multiple parallel networks that are not interconnected. And this allows you to keep the networks very compact. Your port utilization on the switches is very efficient. So we can scale until you get to another GPU. So one GPU to a GPU on another side of your football sized building, football field sized building, you only have two hops for two different switches. So this keeps your latency down and you can have really efficient communication even over far distances as you scale out .

Dave Vellante

>> And it's two hops because if the primary path that you want to go to is blocked, you can go to an alternative path, essentially.

Peter Salanki

>> It's not really blocking. That's a great question. It's more around switches have fixed number of ports. They have 256 ports. So you have to stagger them together in a tree so that you can reach everything. The topology is constrained by number of ports you have per switch and that's why you have to make it that way. But by building these parallel networks, we can constrain it only to two levels versus before we needed to do three or four levels.

Dave Vellante

>> Got it.

Corey Sanders

>> And for our customers, the exposure is really quite important. Our software layers, our Kubernetes layer and our sunk layer, which is our Slurm on Kubernetes layer, are topology aware. So part of what we offer to our customers is not just this innovative approach to how we set up the networking and how we do the rack design, but actually how we expose it to customers so their jobs can take full advantage of that difference in performance at a rack level versus across rack level automatically based on the nature of the job.

Dave Vellante

>> We're going to talk about this more in a moment, but this is not your granddaddy's virtualization stack. You had to throw that away and reinvent for accelerated computing. Corey, so we're going from, as I referenced before, Moore's law to call Jensen's new law, buy more, save more.

Peter Salanki

>> .

Dave Vellante

>> Make more, right?

Corey Sanders

>> Jensen's law. Yeah.

Dave Vellante

>> And so you're talking about one 10th the cost per million tokens. Talk about the economics and specifically what that's going to mean for your customers.

Corey Sanders

>> Yeah, absolutely. One of the biggest things from a workload perspective in the inference space today and the AI application space today has been a limit of what you're willing to put behind it from a cost perspective. The beautiful thing about AI is that its limit is human ingenuity, but its secondary limit is actually the cost, right? How expensive is it going to take for me to be able to do these workloads? And that limits people. They end up running them for shortened amount of times or they take a smaller subset of the data and run it through their inferencing platform versus saying they're going to take the full corpus of data and run it through. And I think when you look at this one 10th number as part of Vera Rubin, you start saying, "Now it may be cost-effective for me to take every single piece of data I have and run through my inferencing stack," which of course is going to result in better experiences, better applications, better solutions. And I think on the CoreWeave side, we're pretty excited about that and we're excited about people being able to do that at any layer in our platform, whether it's at the Kubernetes layer, that they're taking advantage of it directly, or with our inference platform where they can take advantage of it on our behalf. And both of those I think are going to drive really new innovative solutions from our customers.

Dave Vellante

>> And you've got Jevons Paradox in play, prices go down, people buy more. Of course prices are going up now even on the older generation-

Peter Salanki

>> And people are buying more.

Corey Sanders

>> People are buying more.

Dave Vellante

>> Well, why is that? It's because the economics are so attractive. If you've got a fixed power envelope, then you're just dropping money to your bottom line. So it's critical. Okay. Peter, I want to come back to what you can do with Vera Rubin that you couldn't do with Blackwell. I recall at GTC this year, Jensen's, his Pareto curve on the vertical axis he had throughput, which is all what training guys want and then the horizontal axis, they added the LPUs and said, "Now we're extending this line before the experience tanks." And he basically said, "You have to figure, every business has to figure out where you are in that Pareto curve. Why? Because you're going to be tapping into intelligence in the form of tokens and then monetizing that." So what are some of the things that customers, you expect are going to be doing with Vera Rubin that they weren't able to do with previous generations?

Peter Salanki

>> Yeah. And when you're running, for instance, an eternal trade-off between throughput and latency, basically, how quickly do you get tokens or response out? And there's a lot of technologies that builds around this that we built in our stack to draw out that curve in both directions. And this comes back into the GPU depreciation and what can you do with all the generations of GPUs. So if you go back a couple of years, it was very simple. You deployed the GPU, deployed the H100 and that ran everything. You had a replica of your model on, say, eight H100s. Or if you went crazy, you had 64 H100s and everything was there. Now, the ensembles look a lot more interesting. You have what we call a speculative decoding model, which is a smaller model that runs first and tries to guess what might happen and it's really quick. And if it guesses right, it will drop your token cost and drop your latency because the smarter model could guess right. And it also knows if it guesses wrong, so then you can go back to the bigger, slower model. And we can use different, older generation SKUs and so on for the speculative decoding models. And we're also getting into disaggregating the components more and more. So you have, the Vera Rubin now can have a full trillion dollar parameter model easily on it and serve that in high speed. And then you can plug in things like the Grok or a separate Vera Rubin rack to help you disaggregate the decode phases of inference and so on, to break up the pipeline in smaller pieces where you have specific hardware, or you have a specific configuration of the same hardware that is the best at doing that. And combining these things, combining also the twice as much GPU to GPU bandwidth as you have in the Vera Rubin allows you to run bigger models faster and a more interesting mix of models and topologies that you set up. And I think the last thing, as we touch into different GPUs is really what people are missing and where the technology is evolving that is not just, okay, numbers are twice as big as how you can slice and dice things when you have more resources, and also the technical evolution around it that makes it really exciting and how you really get... So you see some of these Jensen graphs where there's a hockey stick inefficiency and that comes from I'm taking my model and I'm not just doubling the flops going into it, I can now disaggregate it over different components in a much smarter way.

Dave Vellante

>> And you mentioned the depreciation cycles. I just wanted to add for the audience, there was a lot of buzz in the media maybe around a year ago now. And you guys, I think depreciate on a six-year basis and people are saying, "Well, that's too long. These things only last two years." The market got it completely wrong. You're taking depreciated assets that are five, six years old and you're still able to serve customers. Why is that? Is that because the workloads are getting so diverse? Maybe you could explain that.

Peter Salanki

>> I don't necessarily think the workloads have gotten any more diverse. They're already pretty diverse.

Dave Vellante

>> Okay.

Peter Salanki

>> And it's more A, the broader market is starting to realize that this is how it works, and the fact that what I talked about that you can now disaggregate the pipeline quite a bit and use... You don't need Vera Rubin for every single thing. You can still use a different SKU, either less powerful SKU an older SKU as an efficient part of your pipeline, be it speculative decoding, be it for batch inference, be it for prefill. You can disassemble the pipeline into different pieces and put different things in there. And some of those things that you put in there are older generation components without even compromising speed. And then we talk about getting your token costs down. If you're doing things like batch inference and so on, those might be perfect to run on something like an A100 where the throughput and the throughput per dollar is really good, but it might not be able to do it at the absolutely lowest latency.

Dave Vellante

>> Yeah. And we talk about agentic all the time and the impact that it's going to put on the network and everything else, but sometimes you just actually do want to do a single shot prompt and you just want to ask it a question and get a simple answer. You don't necessarily need the latest and greatest infrastructure behind that. And I know OpenAI has done some others to innovation to actually route the query and the prompt to the most appropriate infrastructure. That's just benefit for customers.

Peter Salanki

>> Yeah. There's a lot of stuff that if you look at CoreWeave's inference stack we're doing with query routing, we're routing you to the best infrastructure. And compute scarcity is other part of this as well. A bit of a tangent, but not only do you need to find the right GPUs. We're building as fast as we can, right? I mentioned that we're building official number on how many data centers we're building parallel. It's a lot. And we're building as much as we can, humanly possible, and we still can't satisfy the demand. And that means that basically every single of our GPUs we bring online is sold out usually before it's brought online, but if we didn't sell it, it's going to sell that immediately. So then a lot of the things that we're building and Corey and my teams are building on is, how do we find pockets of capacity around the world, route your request to a pocketable capacity can serve it now efficiently using our CoreWeave object storage, move your model weights over there in real time. We're also launching some really cool things with distributed KV caching here so you can truly serve your model from anywhere in the world and we can find you these pockets of capacity to serve your inference need any time without you having to sign a huge five-year commitment for 10,000 to 200,000 GPUs.

Dave Vellante

>> I'm glad you brought up KV cache because we haven't talked about storage at all. You're seeing a hierarchy of memory now evolve in a big way. The reason I ask this is because we have a premise at the CUBE Research that the traditional general purpose functions that are so ubiquitous today with X86, they're ultimately going to be absorbed into the AI factory. We think that things like storage and obviously compute, networking, and software will ultimately have to be rewritten to take advantage of accelerated computing. It's kind of a self-serving question for you guys, but do you believe that premise? Do you think that has merit?

Peter Salanki

>> I'd like you to start, Corey.

Corey Sanders

>> Yeah. I think when we think about the full workload and the requirements of that full workload, whether it be on the training side or on the AI app side, it brings in all of the components that you're talking about, right? This is not just a, "Hey, I need GPUs and I want to go run this app." It is, "I need GPUs. Now I need to be able to feed those GPUs with the right storage." And this is where we have some unique technology or a lot of cache that can feed the GPUs with more storage than many in the market at a rate that then allows their GPUs to continue to perform at top performance. But then when you think about AI applications and the concept of an AI loop to be able to build these AI applications, you start adding things like sandboxes and evaluations that are going to take a whole lot of CPU power to be able to go do these types of assessments or run these tools as supporting your agent. And so the inferencing component running on the GPU is a big factor, but it's a part of a whole for an application that's going to include storage and going to include managed databases and going to include a whole lot of CPU to be able to deliver the new application of the future.

Dave Vellante

>> Well, that is not just GPU. It's CPU. And it's a tight integration.

Corey Sanders

>> Exactly.

Dave Vellante

>> You guys are really a new type of hyperscaler, call it an AI hyperscaler, but people want to know, well, eventually the hyperscalers are going to have their capabilities as well. But essentially, I have great respect for the hyperscalers, but they're... Sorry. They're bolting on AI into their, what is now a modern legacy infrastructure. People want to know though... Well, by the way, this is why, if I understand it correctly, you are able to achieve very high levels of utilization for extremely expensive resource, the GPU. Well north to 90% as I understand it. Whereas enterprises today, not only are they not liquid cooled, they just don't have that level of engineering expertise. That's not what they've been doing for the last 10 or 15 years. But I want... Right now demand is through the roof. Far exceeds supply. When demand and supply come together at an equilibrium, and it will likely happen at some point in time, I can't predict when, maybe it's next decade, whatever, what is CoreWeave's unique advantage as the AI hyperscaler? Let's close there.

Peter Salanki

>> So the traditional hyperscalers are built for a different purpose. And you have to realize that it's not that, oh, their stack is bad and they should have done it like we do. It's built for different purpose. You're built for running something like this where you have mission-critical infrastructure, you have a bank, you have a stock exchange where it's okay to throw away half of your compute in the name of redundancy. Now, if you go take the same pitch to someone running inference or training a huge model and say, "Hey, I'm going to take half of your compute. I'm going to throw it away for redundancy, but it will be reliable. You will not have a single interruption," they will not be happy about that, to use not too strong language. And then they try to fit this square peg into a round hole where they're now trying to push these components to a much more intense level than their stack was built for. When we built our stack, we looked at initially, how are supercomputers built? Okay, you build things to push it to the max and then you embrace that some things will fail. At some point, one of these million connections that I told you about, they will fail and they do. And when that failure happens, how do we detect it? How do we route around it? How do we make it so that the customer experience is still great, they still get super high utilization numbers, but they also can use 100% of the compute because we don't throw away so much in the name of preventing a single failure? And that's really where the difference comes from. And that's a difference in philosophy, it's a difference in how we build our data centers, how we build the company, how my teams operate. I think that's why it's been so hard for them to pivot and actually scale to this challenge.

Dave Vellante

>> But to put a finer point on it, you're not sacrificing that resilience, you're just engineering it differently.

Peter Salanki

>> We're engineering differently.

Dave Vellante

>> Right.

Corey Sanders

>> That's right.

Peter Salanki

>> We say that it's okay if there's an interruption because we can restart you so quickly and we can identify the failing components so quickly that it doesn't matter.

Corey Sanders

>> And to add to that, I think, Peter explained at the beginning this, the liquid cooling design is a good example. To me, that's coming from a hyperscaler and my history that the design and the implementation there is very unique and it creates this level of both quality but also non-redundancy in that quality. And when you think about the hyperscalers, you could say, "Well, yeah, they could always build that," but we've been doing it for years now. And in the age of AI, data becomes the actual currency of being able to deliver great service. And so we have this observability data that is now feeding our mission control service that Peter also mentioned and making customers' experiences better, which anyone who starts today and builds it this way won't have any of that historical data to be able to then continue to make the service better and better.

Dave Vellante

>> All right, guys. We got to leave it there. Thanks so much. I really appreciate your time-

Corey Sanders

>> Thank you.

Dave Vellante

>> ... and sharing with our audience and thank you for watching. We're generating tokens and intelligence here, bringing it out to you. We'll be right back. Right after this short break, theCUBE's NYSE Wired coverage of the next generation of AI infrastructure.

Corey Sanders & Peter Salanki, CoreWeave

search

Dave Vellante

>> Thank you, John, and thank you Palo Alto. This is Dave Vellante here at our NYSE Wired CUBE Studios. We're going to talk about going from silicon to production. Really, CoreWeave's full stack behind Vera Rubin. The industry is a buzz with the first Vera Rubin rack shipping. Corey Sanders is here as the senior vice president of products at CoreWeave. And Peter Salanki is the CTO of CoreWeave. Gentlemen, welcome.

Peter Salanki

>> Thank you for having us.

Corey Sanders

>> Thank you.

Dave Vellante

>> Thanks so much for coming into our studio. And Peter, I want to start with you if I can. You guys have made extraordinary investments, 15, 20 plus billion dollars in infrastructure. Where do you stand today in terms of data center, data center build-out, the capacity that you have, and what does the backlog of demand look like?

Peter Salanki

>> The backlog is insatiable, right? And I don't know what the current public number is of data centers that we have live, about 45 of them, maybe more. And we have over a gigawatt of power live and we've got a lot more in the pipeline. We're building as fast as we can. We could build 2X to 3X as fast as we would. It's kind of what we're trying to do. It's very hard. We have gotten down the number of days from, we cannot get power on GPUs in hand of customers to almost single digits, which we're very proud of. And we've done this through over six years of learning how to build these things both fast and reliable, and how to build a stack that is purpose built for AI versus general hyperscaler type computes.

Dave Vellante

>> Yeah. I think you're actually up over 45 now. You said a gigawatt. People talk about building data centers. It takes 30, 40, 50 billion dollars to build a gigawatt data center. You're there in total capacity. The other thing that I've read about CoreWeave and one of your differentiators is the speed with which you can get these centers up and running. It's weeks versus typically many, many months or more.

Peter Salanki

>> Yeah, exactly. I think that we're pressing below weeks now as well. And a lot of that is building it right from the ground up and everything around it. People ask me, "What is your secret sauce? What is the one component?" And there isn't really one component. It's everything from how we design the data centers, how we make modular components. So if we need to late bind and shift around what type of capacity we put in there, we can do so. And then we go into, how do we do the physical cabling, right? What patterns, what methodologies have we built, what tools do we give all of the... Sometimes we have 1,000 contractors there pulling the hundreds of miles of cables. In 100,000 GPU cluster, you will have over a million connections between all these GPUs and components. So how do we build the tools? We have people running around with iPads, color code things, all the systems we put in place so we can do the physical part quickly. And then when the physical part is done, this is really where all the exciting software kicks in and all the automation where we can take all these components fresh out the factory, configure them, test them, burn them in, as they call, together and deliver them as a fully working, 100% reliable cluster to a customer within a couple of days. And without a lot of software innovation, and honestly, a lot of hardworking, so hard learnings over the last couple of years, that would not be possible.

Dave Vellante

>> Well, for years, the industry marched to the cadence of Moore's law, basically doubling performance every couple of years. Now we're seeing that concept of extreme co-design that Jensen talks about. You guys, certainly Dell brings that to its engineering. You guys are deeply embedding that concept in your engineering with the hardware and software. And Corey, everybody's excited about Vera Rubin. The incremental improvements are, they're not incremental. They're orders of magnitude better. I'm interested as you bring Vera Rubin to the CoreWeave portfolio, what kind of workloads we can expect people running that they couldn't necessarily run in previous generations?

Corey Sanders

>> Yeah, absolutely. Well, so obviously the significant amount of customers we have today, the IBMs, the Coheres, the Mistrals running today on the Blackwell fleet that we have, the massive fleet that we have, they're doing frontier cutting edge training, right? They're building the next generation of model and building next generation solutions on top of it. What I'm really excited about... And they'll continue to do that. I think they'll continue to grow and develop on that. What I'm excited about as we look at Vera Rubin is a couple of things. I think one, certainly they'll be able to do more with a smaller amount. A lot of this is around better utilization, cost savings-

Peter Salanki

>> That's a big-

Corey Sanders

>> ... being able to be able to deliver the outcome with much less cost. And then I also think that the exciting part about Vera Rubin is the inference side of the house. As we see, the inference market has grown exponentially over even just the last couple years. And I think the opportunity for Vera Rubin to now play this really interesting role of both supporting this massive buildup of training while also supporting a huge opportunity for massive inferencing at a cost and a performance that I think before would have been impossible. And so brand new workloads, I think are coming to life as part of it. You can see I'm very excited.

Dave Vellante

>> And I'll tell you, our whole industry is excited, and it's the advancements. I remember one of my first supercomputing events that I ever went to, I was looking at the hoses and the liquid cooling infrastructure and it was this spaghetti mess and I was like, "Ooh, what happens when one of those hoses leaks?" It's not good.

Peter Salanki

>> Very green.

Dave Vellante

>> .

Peter Salanki

>> very quickly.

Corey Sanders

>> Yeah.

Dave Vellante

>> We've talked about the underappreciated things like connection integrity on hoses and the ability, what you guys are bringing in, you've got these new software capabilities that you've built. I think patents or patent pending innovations for Vera Rubin, Valvey and Racky-

Corey Sanders

>> Best names in the industry.

Dave Vellante

>> ... which is, I love it. Very cute. So novel software to deal with monitoring, observing. Explain what those innovations are and how they fit.

Peter Salanki

>> Sure. So traditional compute, like your laptop here, your computer at home is air cooled, right? You get air in, there's a fan that blows it. And it works fine. And it's not very efficient, but it's easy and there's no risk that anything gets wet. With the Blackwell generation of compute, we and the broader market started introducing liquid cooling. We have been extremely aggressive deploying liquid cooling. Pretty much everything we've done Blackwell and forward is liquid cooled. We're doing liquid cooled networking gear and CPU now as well. We think that going forward, everything's going to be liquid cooled. But liquid cooled comes with more challenges. Now all of a sudden you have a completely separate fluid system which is a lot more complex than air, which is also fluid. So when we look at the previous generation, Grace Blackwell, it was the first rack scale system, the first that really needed to be liquid cooled, we were working with technology called in-rack CDUs. This means that every rack has its own heat changer and the secondary loop, as it's called, is pretty simple. You have service one rack. And this has some benefits as a relatively easy way to start out. It's self-contained, but it's not necessarily as redundant as a larger scale secondary loop. Kind of awkward to try to fit all of that heat exchange power into one rack. So what we do with Vera Rubin is what is called end-of-row liquid cooling. So that means that you have one loop that runs in the building, that's what we call the primary loop that comes from the main chillers outside. And also with Vera Rubin, we're able to push up the primary loop temperature as high as 45 degrees Celsius, which means we can use outside air with very simple heat exchangers, even without compressors to cool the facility, which is great from a PUE, environmental efficiency standpoint. So that's the primary loop in the building. And then the secondary loop now that serves the individual GPUs, the individual racks, that will now serve, let's call it seven to eight to 10 racks. And then you have a much beefier heat exchanger, it's called an end-of-row CDU, much more redundant system. But this adds so much complexity because now you have a bunch of different systems, but a bunch of different servers, which will do different things, have different temperatures and need service, might break at a different time all tied together in this very critical, what we call secondary loop. So what we built to manage all of this is what we call Valvey, which is our own valve assembly. It's basically a bunch of pipes kind of similar to what you have in your house, in your basement, in your plumbing, but much bigger and then a whole bunch of sensors, much bigger. And then a whole bunch of sensors and valves all tied together and all automatically controlled in real time. Millisecond response times on these things. So it means that we can respond to demand from a GPU churning up. We can respond to demand, feed more fluid into that one in real time. And then you have the rack manager, Racky the rack manager. It's responsible for the bigger picture within this, what we call a row or a pod. It takes in telemetry from the GPUs itself, takes in telemetry from the power systems, takes in telemetry from different leak sensors and from the buildings, building management system and allows us to tie all these things together. It's still deployed locally in the pod because there's a lot of physical cables, different sensors, and we want it to be distributed and self-contained. And then it will then interact with Valvey, it will interact with other systems upstream and downstream to make decisions. How much power do we need here? Are we overloading something? We can do things like oversubscription of a building where I say that the building is specked to host 1,000 GPUs, but we can actually put 120,000 GPUs in there because we know that everything is not going to be active at the same time. Sometimes something is going over an upgrade or being debugged or a research is sitting on a GPU but they're not actually using it. And now with Racky and Valvey together, we can redirect more power and the liquid cooling fluid where it's needed and get a much better efficiency at our data center without risking leak events or power overlooks.

Dave Vellante

>> So this just underscores the engineering work that CoreWeave has done. What you described is very complicated. You've got the primary fluid network, you've got a secondary fluid network, you've got the direct liquid cooling right onto the chip. And so plumbers are becoming artisans.

Peter Salanki

>> Yeah.

Dave Vellante

>> We know they're in short supply.

Peter Salanki

>> Right.

Dave Vellante

>> So amazing work. So if I understand it right, Racky is sort of the ex officio, of the umbrella and it shows the health of the system. Valvey goes deep -

Peter Salanki

>> Yeah. Valvey is a liquid cooling component of it.

Dave Vellante

>> Yeah, amazing.

Peter Salanki

>> And on top of that, it's our mission control software stack. There's overarching, which then talks to Racky that talks to Valvey. And then mission control handles everything from tracking customers, either inference jobs or training jobs, seeing, are they progressing? Is there a performance degradation that we can detect, and can we tie that back to anything wrong from all these signals and both remediate it? It might be, okay, we see that this component, this GPU is underperforming for some reason. Then we want to take it out automatically without interrupting the customer and without them having to do something. So mission control system looks at it globally and can isolate different pieces right around underperforming components, which will then signal Valvey and Racky to do their thing.

Dave Vellante

>> And that enables you to use ambient air, as you said, essentially warm water, which OCP challenged the industry to step up to several years ago. So that just translates into lower cost.

Peter Salanki

>> Yep, lower cost and higher reliability, right?

Dave Vellante

>> Absolutely.

Peter Salanki

>> That's all .

Dave Vellante

>> Thank you for making that point. Now, I know coming off the GPU, the performance of these, the bandwidth is amazing, but the intra rack bandwidth is 260 terabytes per second.

Peter Salanki

>> Yeah, .

Dave Vellante

>> That's just a single NVL72 rack. But so what happens when you scale out and you got to stitch these things together?

Peter Salanki

>> You have to scale up bandwidth, that's what is within a single rack. And you can also ask yourself like, "Okay, why can't we scale that out?" So one of the tricks with a scale-up with a single rack is that on the back of this Grace Blackwell and Vera Rubin racks, it's all copper connecting these components together. And copper is very efficient. Unlike optics which are used in a fiber network, it doesn't use power, it doesn't create a bunch of heat. And it's a pretty reliable signal. So connecting a lot of bandwidth together is easiest and best done with copper without wasting 50% of your power budget in just heat loss to turn electrons into light. So when you want a lot of bandwidth, you can do it with copper. But copper, with its difference from light, is that it has a lot of signal loss over distance. So you can really only get as far as one rack reliably with copper. And rack takes up less space on this table. It's this much space and you'll see it this . Why would you put so much power into this rack? And if you haven't followed the data center evolution, like a rack, a normal server rack, 2019 when I got into this game, it was 16 kilowatts. So many data centers that were all purpose built for this, you could fit two of these racks and then the rest of your room would be empty. So why do you build it like that? It's to shorten the signal path so you can use these copper back planes to connect them together. So that's your scale-up network. That's what we get in one rack. That's why we can have these insane numbers . But as you know, AI labs and enterprises are not happy with the one just rack, the one rack. Then my business would be so easy. . So we put thousands of these in huge data centers, we need to connect them together. That's where what we call a scale-out network is. That's where we would use either InfiniBand or Rocky. We work with an NVIDIA Quantum InfiniBand or NVIDIA Spectrum-X Rocky. For Vera Rubin, there will be a lot of long Spectrum-X Rocky on the Spectrum-6 chip. And it's what is called a multiplanar design, gets very technical, but you basically build multiple parallel networks that are not interconnected. And this allows you to keep the networks very compact. Your port utilization on the switches is very efficient. So we can scale until you get to another GPU. So one GPU to a GPU on another side of your football sized building, football field sized building, you only have two hops for two different switches. So this keeps your latency down and you can have really efficient communication even over far distances as you scale out .

Dave Vellante

>> And it's two hops because if the primary path that you want to go to is blocked, you can go to an alternative path, essentially.

Peter Salanki

>> It's not really blocking. That's a great question. It's more around switches have fixed number of ports. They have 256 ports. So you have to stagger them together in a tree so that you can reach everything. The topology is constrained by number of ports you have per switch and that's why you have to make it that way. But by building these parallel networks, we can constrain it only to two levels versus before we needed to do three or four levels.

Dave Vellante

>> Got it.

Corey Sanders

>> And for our customers, the exposure is really quite important. Our software layers, our Kubernetes layer and our sunk layer, which is our Slurm on Kubernetes layer, are topology aware. So part of what we offer to our customers is not just this innovative approach to how we set up the networking and how we do the rack design, but actually how we expose it to customers so their jobs can take full advantage of that difference in performance at a rack level versus across rack level automatically based on the nature of the job.

Dave Vellante

>> We're going to talk about this more in a moment, but this is not your granddaddy's virtualization stack. You had to throw that away and reinvent for accelerated computing. Corey, so we're going from, as I referenced before, Moore's law to call Jensen's new law, buy more, save more.

Peter Salanki

>> .

Dave Vellante

>> Make more, right?

Corey Sanders

>> Jensen's law. Yeah.

Dave Vellante

>> And so you're talking about one 10th the cost per million tokens. Talk about the economics and specifically what that's going to mean for your customers.

Corey Sanders

>> Yeah, absolutely. One of the biggest things from a workload perspective in the inference space today and the AI application space today has been a limit of what you're willing to put behind it from a cost perspective. The beautiful thing about AI is that its limit is human ingenuity, but its secondary limit is actually the cost, right? How expensive is it going to take for me to be able to do these workloads? And that limits people. They end up running them for shortened amount of times or they take a smaller subset of the data and run it through their inferencing platform versus saying they're going to take the full corpus of data and run it through. And I think when you look at this one 10th number as part of Vera Rubin, you start saying, "Now it may be cost-effective for me to take every single piece of data I have and run through my inferencing stack," which of course is going to result in better experiences, better applications, better solutions. And I think on the CoreWeave side, we're pretty excited about that and we're excited about people being able to do that at any layer in our platform, whether it's at the Kubernetes layer, that they're taking advantage of it directly, or with our inference platform where they can take advantage of it on our behalf. And both of those I think are going to drive really new innovative solutions from our customers.

Dave Vellante

>> And you've got Jevons Paradox in play, prices go down, people buy more. Of course prices are going up now even on the older generation-

Peter Salanki

>> And people are buying more.

Corey Sanders

>> People are buying more.

Dave Vellante

>> Well, why is that? It's because the economics are so attractive. If you've got a fixed power envelope, then you're just dropping money to your bottom line. So it's critical. Okay. Peter, I want to come back to what you can do with Vera Rubin that you couldn't do with Blackwell. I recall at GTC this year, Jensen's, his Pareto curve on the vertical axis he had throughput, which is all what training guys want and then the horizontal axis, they added the LPUs and said, "Now we're extending this line before the experience tanks." And he basically said, "You have to figure, every business has to figure out where you are in that Pareto curve. Why? Because you're going to be tapping into intelligence in the form of tokens and then monetizing that." So what are some of the things that customers, you expect are going to be doing with Vera Rubin that they weren't able to do with previous generations?

Peter Salanki

>> Yeah. And when you're running, for instance, an eternal trade-off between throughput and latency, basically, how quickly do you get tokens or response out? And there's a lot of technologies that builds around this that we built in our stack to draw out that curve in both directions. And this comes back into the GPU depreciation and what can you do with all the generations of GPUs. So if you go back a couple of years, it was very simple. You deployed the GPU, deployed the H100 and that ran everything. You had a replica of your model on, say, eight H100s. Or if you went crazy, you had 64 H100s and everything was there. Now, the ensembles look a lot more interesting. You have what we call a speculative decoding model, which is a smaller model that runs first and tries to guess what might happen and it's really quick. And if it guesses right, it will drop your token cost and drop your latency because the smarter model could guess right. And it also knows if it guesses wrong, so then you can go back to the bigger, slower model. And we can use different, older generation SKUs and so on for the speculative decoding models. And we're also getting into disaggregating the components more and more. So you have, the Vera Rubin now can have a full trillion dollar parameter model easily on it and serve that in high speed. And then you can plug in things like the Grok or a separate Vera Rubin rack to help you disaggregate the decode phases of inference and so on, to break up the pipeline in smaller pieces where you have specific hardware, or you have a specific configuration of the same hardware that is the best at doing that. And combining these things, combining also the twice as much GPU to GPU bandwidth as you have in the Vera Rubin allows you to run bigger models faster and a more interesting mix of models and topologies that you set up. And I think the last thing, as we touch into different GPUs is really what people are missing and where the technology is evolving that is not just, okay, numbers are twice as big as how you can slice and dice things when you have more resources, and also the technical evolution around it that makes it really exciting and how you really get... So you see some of these Jensen graphs where there's a hockey stick inefficiency and that comes from I'm taking my model and I'm not just doubling the flops going into it, I can now disaggregate it over different components in a much smarter way.

Dave Vellante

>> And you mentioned the depreciation cycles. I just wanted to add for the audience, there was a lot of buzz in the media maybe around a year ago now. And you guys, I think depreciate on a six-year basis and people are saying, "Well, that's too long. These things only last two years." The market got it completely wrong. You're taking depreciated assets that are five, six years old and you're still able to serve customers. Why is that? Is that because the workloads are getting so diverse? Maybe you could explain that.

Peter Salanki

>> I don't necessarily think the workloads have gotten any more diverse. They're already pretty diverse.

Dave Vellante

>> Okay.

Peter Salanki

>> And it's more A, the broader market is starting to realize that this is how it works, and the fact that what I talked about that you can now disaggregate the pipeline quite a bit and use... You don't need Vera Rubin for every single thing. You can still use a different SKU, either less powerful SKU an older SKU as an efficient part of your pipeline, be it speculative decoding, be it for batch inference, be it for prefill. You can disassemble the pipeline into different pieces and put different things in there. And some of those things that you put in there are older generation components without even compromising speed. And then we talk about getting your token costs down. If you're doing things like batch inference and so on, those might be perfect to run on something like an A100 where the throughput and the throughput per dollar is really good, but it might not be able to do it at the absolutely lowest latency.

Dave Vellante

>> Yeah. And we talk about agentic all the time and the impact that it's going to put on the network and everything else, but sometimes you just actually do want to do a single shot prompt and you just want to ask it a question and get a simple answer. You don't necessarily need the latest and greatest infrastructure behind that. And I know OpenAI has done some others to innovation to actually route the query and the prompt to the most appropriate infrastructure. That's just benefit for customers.

Peter Salanki

>> Yeah. There's a lot of stuff that if you look at CoreWeave's inference stack we're doing with query routing, we're routing you to the best infrastructure. And compute scarcity is other part of this as well. A bit of a tangent, but not only do you need to find the right GPUs. We're building as fast as we can, right? I mentioned that we're building official number on how many data centers we're building parallel. It's a lot. And we're building as much as we can, humanly possible, and we still can't satisfy the demand. And that means that basically every single of our GPUs we bring online is sold out usually before it's brought online, but if we didn't sell it, it's going to sell that immediately. So then a lot of the things that we're building and Corey and my teams are building on is, how do we find pockets of capacity around the world, route your request to a pocketable capacity can serve it now efficiently using our CoreWeave object storage, move your model weights over there in real time. We're also launching some really cool things with distributed KV caching here so you can truly serve your model from anywhere in the world and we can find you these pockets of capacity to serve your inference need any time without you having to sign a huge five-year commitment for 10,000 to 200,000 GPUs.

Dave Vellante

>> I'm glad you brought up KV cache because we haven't talked about storage at all. You're seeing a hierarchy of memory now evolve in a big way. The reason I ask this is because we have a premise at the CUBE Research that the traditional general purpose functions that are so ubiquitous today with X86, they're ultimately going to be absorbed into the AI factory. We think that things like storage and obviously compute, networking, and software will ultimately have to be rewritten to take advantage of accelerated computing. It's kind of a self-serving question for you guys, but do you believe that premise? Do you think that has merit?

Peter Salanki

>> I'd like you to start, Corey.

Corey Sanders

>> Yeah. I think when we think about the full workload and the requirements of that full workload, whether it be on the training side or on the AI app side, it brings in all of the components that you're talking about, right? This is not just a, "Hey, I need GPUs and I want to go run this app." It is, "I need GPUs. Now I need to be able to feed those GPUs with the right storage." And this is where we have some unique technology or a lot of cache that can feed the GPUs with more storage than many in the market at a rate that then allows their GPUs to continue to perform at top performance. But then when you think about AI applications and the concept of an AI loop to be able to build these AI applications, you start adding things like sandboxes and evaluations that are going to take a whole lot of CPU power to be able to go do these types of assessments or run these tools as supporting your agent. And so the inferencing component running on the GPU is a big factor, but it's a part of a whole for an application that's going to include storage and going to include managed databases and going to include a whole lot of CPU to be able to deliver the new application of the future.

Dave Vellante

>> Well, that is not just GPU. It's CPU. And it's a tight integration.

Corey Sanders

>> Exactly.

Dave Vellante

>> You guys are really a new type of hyperscaler, call it an AI hyperscaler, but people want to know, well, eventually the hyperscalers are going to have their capabilities as well. But essentially, I have great respect for the hyperscalers, but they're... Sorry. They're bolting on AI into their, what is now a modern legacy infrastructure. People want to know though... Well, by the way, this is why, if I understand it correctly, you are able to achieve very high levels of utilization for extremely expensive resource, the GPU. Well north to 90% as I understand it. Whereas enterprises today, not only are they not liquid cooled, they just don't have that level of engineering expertise. That's not what they've been doing for the last 10 or 15 years. But I want... Right now demand is through the roof. Far exceeds supply. When demand and supply come together at an equilibrium, and it will likely happen at some point in time, I can't predict when, maybe it's next decade, whatever, what is CoreWeave's unique advantage as the AI hyperscaler? Let's close there.

Peter Salanki

>> So the traditional hyperscalers are built for a different purpose. And you have to realize that it's not that, oh, their stack is bad and they should have done it like we do. It's built for different purpose. You're built for running something like this where you have mission-critical infrastructure, you have a bank, you have a stock exchange where it's okay to throw away half of your compute in the name of redundancy. Now, if you go take the same pitch to someone running inference or training a huge model and say, "Hey, I'm going to take half of your compute. I'm going to throw it away for redundancy, but it will be reliable. You will not have a single interruption," they will not be happy about that, to use not too strong language. And then they try to fit this square peg into a round hole where they're now trying to push these components to a much more intense level than their stack was built for. When we built our stack, we looked at initially, how are supercomputers built? Okay, you build things to push it to the max and then you embrace that some things will fail. At some point, one of these million connections that I told you about, they will fail and they do. And when that failure happens, how do we detect it? How do we route around it? How do we make it so that the customer experience is still great, they still get super high utilization numbers, but they also can use 100% of the compute because we don't throw away so much in the name of preventing a single failure? And that's really where the difference comes from. And that's a difference in philosophy, it's a difference in how we build our data centers, how we build the company, how my teams operate. I think that's why it's been so hard for them to pivot and actually scale to this challenge.

Dave Vellante

>> But to put a finer point on it, you're not sacrificing that resilience, you're just engineering it differently.

Peter Salanki

>> We're engineering differently.

Dave Vellante

>> Right.

Corey Sanders

>> That's right.

Peter Salanki

>> We say that it's okay if there's an interruption because we can restart you so quickly and we can identify the failing components so quickly that it doesn't matter.

Corey Sanders

>> And to add to that, I think, Peter explained at the beginning this, the liquid cooling design is a good example. To me, that's coming from a hyperscaler and my history that the design and the implementation there is very unique and it creates this level of both quality but also non-redundancy in that quality. And when you think about the hyperscalers, you could say, "Well, yeah, they could always build that," but we've been doing it for years now. And in the age of AI, data becomes the actual currency of being able to deliver great service. And so we have this observability data that is now feeding our mission control service that Peter also mentioned and making customers' experiences better, which anyone who starts today and builds it this way won't have any of that historical data to be able to then continue to make the service better and better.

Dave Vellante

>> All right, guys. We got to leave it there. Thanks so much. I really appreciate your time-

Corey Sanders

>> Thank you.

Dave Vellante

>> ... and sharing with our audience and thank you for watching. We're generating tokens and intelligence here, bringing it out to you. We'll be right back. Right after this short break, theCUBE's NYSE Wired coverage of the next generation of AI infrastructure.