Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud | Direct from the CoreWeave Data Center

Clips
More from Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud

Harsh Banwait

Director of Product

CoreWeave

Jacob Yundt

Senior Director of Compute Architecture

CoreWeave

Zach Merendino

Staff Operations Engineer, Fleet Engineering

CoreWeave

play_circle_outline Introductions of CoreWeave presenters and their roles

Info
Transcript

Direct from the CoreWeave Data Center

Harsh Banwait

Director of Product CoreWeave

Jacob Yundt

Senior Director of Compute Architecture CoreWeave

Zach Merendino

Staff Operations Engineer, Fleet Engineering CoreWeave

In this segment from Scaling the Agentic Era With NVIDIA Vera Rubin NVL72 on CoreWeave Cloud 2026, Harsh Banwait, director of product management at CoreWeave, Jacob Yundt, senior director of compute architecture at CoreWeave, and Zach Merendino, staff operations engineer at CoreWeave, take viewers behind the scenes of the first NVIDIA Vera Rubin NVL72 rack bring-up on CoreWeave Cloud. The team walks through the rack's architecture — 72 Rubin GPUs, 36 Vera CPUs and NVLink 6 fabric — and explains how rack-scale design supports the high-throughput inference and ... Read more

explore Keep Exploring

Who are the speakers and what will they be discussing in this presentation? add

bolt Powered by CUBE AI

Direct from the CoreWeave Data Center

search

Harsh Banwait

>> Hello everyone. Thank you so much for joining us. My name is Harsh Banwait. I'm a director of product care at CoreWeave. Jacob?

Jacob Yundt

>> Hello, my name is Jacob. I'm the senior director of compute architecture, but I generally introduce myself as the Server Guy.

Harsh Banwait

>> Zach?

Zach Merendino

>> My name is Zach Merendino. I'm a staff operations engineer at fleet, and I call myself the Node Guy. What we're here to talk to you about today is our experience behind the scenes, bringing up the first Vera Rubin rack, and talking to you a little bit about some of the innovations that we've built, some of the work that the teams have been doing, not just over the past few weeks but also over the last, I'd say almost two years. So we just want to give you a little bit of behind the scenes look at why we built some of these things, how they operate.

Harsh Banwait

>> Zach, I think our audience might be excited to get better looks at the NVIDIA Vera Rubin NVL72 behind us. Let's go take a look. All right Zach, we're here in front of the first VR NVL72 rack. Do you want to tell me a little about what we're looking at today?

Zach Merendino

>> Absolutely. So first thing we're looking at right up here is you have your first 10 NVIDIA Vera Rubin compute trace. Each one has four Rubin GPUs and two NVIDIA Vera CPUs in it. Right in the middle, you have your NVIDIA NVLink 6, NV link switches. And then, right underneath you have your eight final compute trace, and that makes up all 72 GPUs interact.

Jacob Yundt

>> Yeah. In addition to the GPU servers and the MV link switches, we also have some supporting infrastructure here as well. Up top, we've got the power shelves. So these are much beefier than our previous generation power shelves. Previously each one was 33 kilowatts. One of these is now 110 kilowatts. Beneath that we've got Racky, our custom rack manager. I'll walk you through this a little bit. From the leftmost side, we've got all of our sensor ports. This is for leak grope and some other things like condensation sensors. We've got our two isolation valves here, that's for fast on/off in case something is starting to leak. These quarts here are used to control the energy valve and all of our power shelves. And then some extra I/O here for E1.S storage for the Racky OS. And then the BMC to manage the device and some external ethernet connectivity as well. And then you'll notice at the bottom of the rack, previously this is where we would have the inner rack CPU, but now this is all cooled by Valvey. So Racky is connecting to Valvey. Valvey's not pictured here, but Valvey has the valves that I mentioned here plus some other sensors. And Valvey is what's responsible for modulating the flow, modulating the temperature, making sure that this GPU rack is configured optimally and running cool.

Harsh Banwait

>> Jacob, Racky and Valvey quotes sound great. Would you mind giving us a demo of that?

Jacob Yundt

>> Absolutely. I'm going to show you with Joey, one of the software engineers on my team, how Racky and Valvey can manipulate our valve assembly to stop the flow of the secondary loop in the event of a leak. Today I'm thrilled to announce two of our most recent innovations, Racky, our custom rack manager, and Valvey, our custom valve assembly. Both of these technologies are going to allow us to control and observe our next generation of NVIDIA Vera Rubin GPUs and CPUs. And with that, I'd like to introduce Joey Berkovitz, a software engineer on our Metal Dev team. He's going to run you through Valvey. Joey?

Jacob Yundt

>> Thank you, Jacob. Here we have Valvey, our custom valve assembly. It has two isolation valves and an energy valve. The isolation valves are used to cut flow to the rack in case of catastrophic emergencies, and the energy valve is used to modulate flow to each individual rack to provide optimal cooling to the racks. We can see we have flow right now with the isolation valves open, and now we're going to close one of the isolation valves to cut flow to the rack. And we should see this flow stop as well.

Jacob Yundt

>> Thanks, Joey. In addition to Valvey, I'm very excited about Racky, our custom rack manager. Racky truly is the brains of the operation. It connects all of the power, liquid cooling, sensors, management and observability, into one centralized place that ties into both mission control and CKS. This will allow us to scale to hundreds of thousands of GPUs including NVIDIA's next generation Vera Rubin.

Harsh Banwait

>> Zach and Jacob, this is a single rack behind us, but we all know what we've been through over the course of the last two quarters at least, to help ensure that this comes to life. Could you share a little bit more about why this moment is so exciting for all of us personally?

Zach Merendino

>> Absolutely. And I think the first thing to start with is there might be three of us up here, but it's taken hundreds of people at CoreWeave to get us to this point, from software developers to mechanical engineers and really every bit of the stack. It's just taking so much effort to design and decide what 40 wants to do. So I think it's really exciting.

Harsh Banwait

>> Yeah. How about you, Jacob?

Jacob Yundt

>> I mean, this takes, I don't know, a year plus. We start working on these things so far in advance when it's just like a pie-in-the-sky. What can we do here? What is this product going to look like? It's just so satisfying to be like, "There it is. It's powered on. It's working."

Harsh Banwait

>> Yeah. I think all of us are hardware nerds. So when we learn that there's a new supercomputer coming around, there's just a genuine sense of excitement across the company. And we're just excited to see it in-person for the first time, plug it in, power it on, think really the kind of kid in us that just wants to play with a new computer comes out when one of these goes around every year or so.