Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud | Chen Goldberg Keynote Session

Clips
More from Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud

Chen Goldberg

Executive Vice President of Product & Engineering

CoreWeave

play_circle_outline CoreWeave partnership with Dell and NVIDIA to validate Vera Rubin NVL72

play_circle_outline Vera Rubin: 72 GPUs, 36 CPUs, 260TB/s NVLink6 — 10x Throughput/Watt, 1/10 Token Cost for Continuous Agent AI

Info

Chen Goldberg Keynote Session

Chen Goldberg

Executive Vice President of Product & Engineering CoreWeave

search

Chen Goldberg

>> Welcome, everyone. I'm Chen Goldberg, EVP of product and engineering at CoreWeave. This has been one of those months where I'm reminded why I love being an engineer. CoreWeave, in partnership with Dell and NVIDIA, became the first AI cloud provider to bring up and validate NVIDIA Vera Rubin NVL72. That was a big milestone. Getting here was not easy, but it was also really fun. But before I tell you how we got here, let me start with what a phenomenal technology NVIDIA has actually built here. Vera Rubin is not an incremental upgrade. 72 Rubin GPUs, 36 Vera CPUs, 260 terabytes per second of NVLink 6 bandwidth inside a single rack, which is more data bandwidth than is used by the entire global internet. This is one of the most impressive feats of engineerings I have seen and it's arriving at the right moment. The world is shifting from asking AI questions to having AI actually do things continuously at scale without stopping when you close your laptop. Agents writing code, running experiments, and executing multi-step reasoning loops, that is exactly what Vera Rubin was architected for. It is a disaggregated distributed agent processing system. When your product is an AI agent, every token you generate has a cost and business impact. Vera Rubin delivers 10 times better Inference throughput per watt and one-tenth the cost per million tokens versus Blackwell. That is way more than a spec sheet improvement. That's a technical leap. Now let's talk about customers because the engineering feats only means something when you connect them to actual work. With CoreWeave's new innovations and full stack optimizations for Vera Rubin, customers can train models more efficiently, accelerate Inference, scale autonomous AI, and achieve a lower cost per token. The bottleneck on innovation is in ideas. It's the pace of experimentation and that pace is constrained by capacity. So when an efficiency leap like Vera Rubin compresses the cycles from days to hours, you're not just saving money, you are changing how many experiments a team can run which changes how much their business is impacted. Performance and efficiency are crucial. In the recent MLPerf 6.0, we set new records from benchmarking the largest NVIDIA GP300 cluster with over 8,000 GPUs and we trained DeepSeek V3 with 671 billion parameters in two minutes. With Inference, we also delivered lead-in results with DeepSeek and GPT-OSS. This is why customers such as Anthropic, Cursor, IBM, Jane Street, Meta, Mistral, and OpenAI rely on CoreWeave for performance, reliability, and efficiency. Getting customers to run their AI innovation reliably in production, not just on a benchmark, is where our real engineering work lives. When we look at a technology leap like this, we spend a lot of time on the drawing board, anticipating the bottlenecks and constraint posed by existing solutions. And we get down to innovating around new solutions that will make it easier for the infrastructure to work the way it's supposed to for our customers. With Vera Rubin, we added two new innovations to CoreWeave Mission Control. First, Vera Rubin is 100% liquid-cooled. So we added more sensors and telemetry at the pair rack level. We call the system Valvey, a programmable per-rack valve assembly, which turns cooling from a passive mechanical system into a software-defined rack level control surface. Valvey controls and monitors each rack flow rate, temperature, pressure, et cetera, to keep direct operating at optimal state. We also engineered Recki, our unified rack manager, designed to provide centralized control and observability per rack. Recki works with multiple systems, including Valvey, and aggregates power, cooling, and environmental sensor into a standardized management service, allowing each Vera Rubin rack to be managed as a cloud resource. You'll hear more about Recki and Valvey throughout our event today. We also deployed the liquid-cooled Spectrum-6 Ethernet switches and will provide multi-rail multi-plane networking for Vera Rubin. We will support both NVIDIA Quantum XI100 InfiniBand and NVIDIA Spectrum-X Ethernet with Recki, delivering 1.6 terabit per second of backend bandwidth per GPU. The Spectrum-X Ethernet architecture scales to over 120,000 GPUs. We optimized every layer of our stack and that includes CKS and SUNK. CKS is NVLink-domain-aware. So it places workloads on the NVLink fabric to minimize traffic-crossing scale-out interconnects. SUNK is a topology-aware and schedules workload on the most efficient GPUs while minimizing inter-rack communications for lower latency. I spent time on the floor at our data center where our hardware engineers are working alongside the NVIDIA team. What stood out was how intentional the move to rack-level management was. We thoughtfully leaned into Vera Rubin rack-scale architecture without compromising the granularity we need to operate every component underneath it. I'm very proud of that work. I will wrap up by noting that we are living through one of the most significant transitions in the history of computing. What we build today becomes the foundation that everything else runs on. Thank you, and I hope you'll join me for our deeper dives.

Chen Goldberg Keynote Session

search

Chen Goldberg

>> Welcome, everyone. I'm Chen Goldberg, EVP of product and engineering at CoreWeave. This has been one of those months where I'm reminded why I love being an engineer. CoreWeave, in partnership with Dell and NVIDIA, became the first AI cloud provider to bring up and validate NVIDIA Vera Rubin NVL72. That was a big milestone. Getting here was not easy, but it was also really fun. But before I tell you how we got here, let me start with what a phenomenal technology NVIDIA has actually built here. Vera Rubin is not an incremental upgrade. 72 Rubin GPUs, 36 Vera CPUs, 260 terabytes per second of NVLink 6 bandwidth inside a single rack, which is more data bandwidth than is used by the entire global internet. This is one of the most impressive feats of engineerings I have seen and it's arriving at the right moment. The world is shifting from asking AI questions to having AI actually do things continuously at scale without stopping when you close your laptop. Agents writing code, running experiments, and executing multi-step reasoning loops, that is exactly what Vera Rubin was architected for. It is a disaggregated distributed agent processing system. When your product is an AI agent, every token you generate has a cost and business impact. Vera Rubin delivers 10 times better Inference throughput per watt and one-tenth the cost per million tokens versus Blackwell. That is way more than a spec sheet improvement. That's a technical leap. Now let's talk about customers because the engineering feats only means something when you connect them to actual work. With CoreWeave's new innovations and full stack optimizations for Vera Rubin, customers can train models more efficiently, accelerate Inference, scale autonomous AI, and achieve a lower cost per token. The bottleneck on innovation is in ideas. It's the pace of experimentation and that pace is constrained by capacity. So when an efficiency leap like Vera Rubin compresses the cycles from days to hours, you're not just saving money, you are changing how many experiments a team can run which changes how much their business is impacted. Performance and efficiency are crucial. In the recent MLPerf 6.0, we set new records from benchmarking the largest NVIDIA GP300 cluster with over 8,000 GPUs and we trained DeepSeek V3 with 671 billion parameters in two minutes. With Inference, we also delivered lead-in results with DeepSeek and GPT-OSS. This is why customers such as Anthropic, Cursor, IBM, Jane Street, Meta, Mistral, and OpenAI rely on CoreWeave for performance, reliability, and efficiency. Getting customers to run their AI innovation reliably in production, not just on a benchmark, is where our real engineering work lives. When we look at a technology leap like this, we spend a lot of time on the drawing board, anticipating the bottlenecks and constraint posed by existing solutions. And we get down to innovating around new solutions that will make it easier for the infrastructure to work the way it's supposed to for our customers. With Vera Rubin, we added two new innovations to CoreWeave Mission Control. First, Vera Rubin is 100% liquid-cooled. So we added more sensors and telemetry at the pair rack level. We call the system Valvey, a programmable per-rack valve assembly, which turns cooling from a passive mechanical system into a software-defined rack level control surface. Valvey controls and monitors each rack flow rate, temperature, pressure, et cetera, to keep direct operating at optimal state. We also engineered Recki, our unified rack manager, designed to provide centralized control and observability per rack. Recki works with multiple systems, including Valvey, and aggregates power, cooling, and environmental sensor into a standardized management service, allowing each Vera Rubin rack to be managed as a cloud resource. You'll hear more about Recki and Valvey throughout our event today. We also deployed the liquid-cooled Spectrum-6 Ethernet switches and will provide multi-rail multi-plane networking for Vera Rubin. We will support both NVIDIA Quantum XI100 InfiniBand and NVIDIA Spectrum-X Ethernet with Recki, delivering 1.6 terabit per second of backend bandwidth per GPU. The Spectrum-X Ethernet architecture scales to over 120,000 GPUs. We optimized every layer of our stack and that includes CKS and SUNK. CKS is NVLink-domain-aware. So it places workloads on the NVLink fabric to minimize traffic-crossing scale-out interconnects. SUNK is a topology-aware and schedules workload on the most efficient GPUs while minimizing inter-rack communications for lower latency. I spent time on the floor at our data center where our hardware engineers are working alongside the NVIDIA team. What stood out was how intentional the move to rack-level management was. We thoughtfully leaned into Vera Rubin rack-scale architecture without compromising the granularity we need to operate every component underneath it. I'm very proud of that work. I will wrap up by noting that we are living through one of the most significant transitions in the history of computing. What we build today becomes the foundation that everything else runs on. Thank you, and I hope you'll join me for our deeper dives.