Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud | Opening Keynote, CoreWeave

Clips
More from Scaling the Agentic Era with NVIDIA Vera Rubin NVL72 on CoreWeave Cloud

Chen Goldberg

Executive Vice President of Product & Engineering

CoreWeave

John Furrier

Co-Founder & Co-CEO

SiliconANGLE Media, Inc.

play_circle_outline CoreWeave Vera Rubin rack-scale NVIDIA system deployment and validation

play_circle_outline Mission Control: unified compute, network, and storage orchestration for workload optimization

play_circle_outline Rack-as-Supercomputer: Rack-Scale GPU Architecture for AI with Valvey 7-Second Liquid Cooling and Racky Telemetry

Info

Opening Keynote, CoreWeave

Chen Goldberg

Executive Vice President of Product & Engineering CoreWeave

John Furrier

Co-Founder & Co-CEO SiliconANGLE Media, Inc.

HOST

search

John Furrier

>> Hello, I'm John Furrier here with theCUBE Studios in Palo Alto for a special presentation with CoreWeave's scaling, the agentic era here in the build-out of the NVIDIA Vera Rubin system. The first to stand up and build out and validate the system is CoreWeave, CoreWeave Cloud. Here is Chen Goldberg, EVP of Product and Engineering for CoreWeave. Thanks for coming on theCUBE. Appreciate it.

Chen Goldberg

>> Thank you so much, John, for having me.

John Furrier

>> So huge accomplishment. Your keynote was awesome and you really laid out the situation, but this is a tech marvel and this is not just turning on the power switch. You guys had the significant work involved on this rack scale system, which is proving out to be the model for moving faster and getting the kind of speeds and performance people want.

Chen Goldberg

>> What I think was captured in the photos and the tweets was how excited the entire team was. And it's important to understand that when I say a team, it's not just the CoreWeave team. It's the NVIDIA team, the Dell team, and the CoreWeave team that has been working over a year on setting up this system. What Vera Rubin represents is a leap compared to Blackwell. And to bring all of that together and working on the details and the spec and bring all the systems and then seeing it live, it's extremely exciting. And behind the scene, it's not just like setting that up. We are actually, when we validate it, it means that we have validated that all the GPUs are working well together, that the bandwidth is what we are achieving, the liquid cooling is working. So just the entire system overall is tested together.

John Furrier

>> In your keynote, you said it's the best time to be an engineer. And this seems to echo the sentiment of the entire industry. It's a massive AI infrastructure build up, but it's really difficult. Talk about the scope of the Vera Rubin and validating the workload, because this isn't just loading software and running some tests. This is like a serious systems engineering effort. A lot's involved. What are the key things that you had to do? I mean, you have the keys to the kingdom, product and engineering. Your team made it happen. It's worked. We saw the videos coming out. Take us through that, because this is a marvel of engineering if you look at it. It's a super computer.

Chen Goldberg

>> And if I mention that Vera Rubin started a year ago, our AI-native cloud, this journey started six years ago. So this is not something that happened overnight. And really, the way we approached it is thinking about the workloads, the customers, and what we need to build in order to deliver the best, more resilient, agile performance system that can happen. And that's really what exciting. And I think one of the things that we are seeing in those kind of systems is that once that you are solving one bottleneck and other one appears. So maybe let me take you through a few things that are really unique with how we are building our cloud. The first one, the brain of the system, what we call mission control. And what we do there differently is that instead of thinking about compute network and storage separately, we're actually bringing those systems together and managing them as a whole. And that allows us to be more proactive and really optimize for the specific workloads. So that means that for example, if I'm running a job on hundreds and sometimes thousands of GPUs and something goes wrong along the way, I can automatically detect that and signal to the job how they should perform differently, making sure that every second matters and all the data that you would need is getting to the GPUs in the best performance. So that's kind of work that we've done for a long time. Now, moving to Vera Rubin, that's something that has been really fun for us. Vera Rubin, the same way as Blackwell really changed the way we think about compute. We talked about before we started recording about every rack being a supercomputer, and that's really an interesting way to think about those kind of resources. So we took it to the drawing board and we were thinking about how can we make sure that this super cool computer is the most efficient ever. And we built several innovation, I will talk about two of them. One of them is Valvey and you can see how fun it is for us because we choose our names in a fun way. And what Valvey does, it managed the liquid cooling system in a software-defined way. And to give people and the audience a sense, we can control a single valve at a seven-second time. So if we detect any leak as an example, we take action immediately, and that's unique compared to other industry solutions. In addition to that, we've built Racky that just brings the data from all the systems, all the sensors and allows us to optimize the system in the best way.

John Furrier

>> So you get a lot of telemetry in there on the system so you can move fast so you don't have the criticality of anything happening.

Chen Goldberg

>> Yes, exactly.

John Furrier

>> Talk about the shift. In your keynote, you talked about how customers are moving and the markets shifted with reasoning, multi-step reasoning. The performance levels for tokens are starting to see kind of like almost a portfolio, entry level, mid-range, high end. Vera Rubin's a high end and the rack scale systems have been the preferred method. Talk about that piece of it and why this is so important and why are those workloads requiring Vera Rubin? Or we'll say Blackwell. I mean, you have choices now, but what specific about the AI is making this an always-on resource requirement?

Chen Goldberg

>> One of the things that is really interesting right now in the industry is that the ceiling of the size of the model, for example, has gone away, which means that the accuracy and the latency has changed opening up for new set of use cases. And that's really what we are seeing happening right now. One of the challenges that was before Vera Rubin we're thinking about the economics of that technology and that's what's possible right now. So we have on one hand the models, the size, the correctness of them, but we now also have the infrastructure that makes it possible to create those new use cases. And we are seeing it really changing every industry there is from drug discovery to physical AI to call centers coding and it's really exciting times.

John Furrier

>> Yeah. You guys have all the models except for one and main models in there working with CoreWeave. And it's interesting. You talk about GPU utilization on one hand and model efficiency on the other. What you're getting at is that as you get smarter, the models become smarter, but also the GPU relation with the models. Talk about why the Vera Rubin matters because you have now better customer opportunities to have an always-on agentic system.

Chen Goldberg

>> Exactly.

John Furrier

>> Because now, it's GPU utilization, that's a technical infrastructure thing. But now the models, it's not distilled models, they want to run models. The model relationship to the infrastructure is a key thing, at least from what I'm hearing.

Chen Goldberg

>> Yeah. So I would call it two things here. One, that connection between the model and the infrastructure, it's no longer just the GPUs. We also need to make sure the data is getting there. So we care about latency, we care about, of course, reliability because those are becoming mission-critical systems. The second thing that I will call out that this change in technology is blurring the line between training and inference. If before you had some teams doing research and doing training models and others doing inference, these agentic systems are living systems. So what we want to do is take live results from inference and build it into the model back again. And that's what we really called the AI loop that makes sure that the system continues to learn.

John Furrier

>> I love the AI loop method because it's training and inferring at the same time because more data's coming in the fabrics help there, the networking features. What have you guys done to make that happen with this new rev? Because I see the power and cooling, liquid cooling's great, you got Racky, you got all this mission control, but what features you'll be put in there to make it go faster?

Chen Goldberg

>> So some of the investment that we have made before with our AI cloud is still paying off. For example, the investment that we've made with our object storage solution, making sure that we can deliver the most amount of data into the GPUs. But some of the things that we have now invested in is really working, by the way, with the industry on improving model performance. We talked about like KV cache optimizations as an example, reinforcement learning. So the innovation is no longer just at the infrastructure level, but actually also upper hydrostack.

John Furrier

>> Yeah. And you guys talk about optimizing all levels of the stack, but from a customer standpoint, what does it mean for them? Because where are they at? When you talk to your customers, what does this mean for them from an impact standpoint? Does it change their architecture? How do they roll this out? How do they engage? Does it change the economics? What's the all in? What's the view from the customer?

Chen Goldberg

>> I imagine it feels a little bit overwhelming for folks from a customer perspective because a lot is changing. First, the use cases are now possible. When I talk with customers, they're saying, "Okay, let's just think about how we create more opportunities for ideas to become a reality and impact the business." So creating and giving more resources to researchers and developers to change how things are going on. But I think what's even more interesting that is happening, that it's not just those ideas to reality and how do I train the model and research, it actually changes the entire ecosystem. So in this case, with Vera Rubin, we have solved latency and efficiency of inference. This is amazing. But now, what kind of tools do we give to developers? How do we manage capacity in the best way possible? How do we solve for power, energy, density? There are so many new problems that are big. How do we bring the right data? All of those new bottlenecks. You see me smiling again.

John Furrier

>> It's the best time to be an engineer.

Chen Goldberg

>> It's the best time being an engineer.

John Furrier

>> Engineers want to work on hard problems, but there's real opportunity behind it. What do you see in three years out? Obviously, you're in charge of the roadmap, you work with all the partners. I mean, extreme co-design is an NVIDIA first principle. Dell was involved and you have a lot of partners. What's it look like for three years from a CoreWeave perspective and what it might look like in the market?

Chen Goldberg

>> I'm looking at the past year or 18 month, I expect that this transformation will happen significantly faster than anything we've experienced before. And what we will see customers and the industry working on like with us is really creating new tools for folks. I expect we'll see models continue and evolve with maybe better size and so on. And maybe the most interesting part is making sure how agent systems are becoming a living system.

John Furrier

>> Yeah. Chen, great to see you and congratulations. A lot of exciting things. I'm sure there will be another step function change, more innovation coming. The next GTC will probably have another-

Chen Goldberg

>> And not just GTC. We are going to have a lot of more exciting news in Fully Connected, our first user conference of CoreWeave in September in Moscone.

John Furrier

>> All right, there it is. Another event, and sure it'll be a lot of innovation. Looking forward to covering that. I'm sure it's going to get bigger, faster, more economical for customers. Congratulations on a huge accomplishment.

Chen Goldberg

>> Thank you so much.

John Furrier

>> I'm John Furrier here in theCUBE Studios in Palo Alto. Now, sending it back to Dave Vellante in New York City.

Opening Keynote, CoreWeave

search

John Furrier

>> Hello, I'm John Furrier here with theCUBE Studios in Palo Alto for a special presentation with CoreWeave's scaling, the agentic era here in the build-out of the NVIDIA Vera Rubin system. The first to stand up and build out and validate the system is CoreWeave, CoreWeave Cloud. Here is Chen Goldberg, EVP of Product and Engineering for CoreWeave. Thanks for coming on theCUBE. Appreciate it.

Chen Goldberg

>> Thank you so much, John, for having me.

John Furrier

>> So huge accomplishment. Your keynote was awesome and you really laid out the situation, but this is a tech marvel and this is not just turning on the power switch. You guys had the significant work involved on this rack scale system, which is proving out to be the model for moving faster and getting the kind of speeds and performance people want.

Chen Goldberg

>> What I think was captured in the photos and the tweets was how excited the entire team was. And it's important to understand that when I say a team, it's not just the CoreWeave team. It's the NVIDIA team, the Dell team, and the CoreWeave team that has been working over a year on setting up this system. What Vera Rubin represents is a leap compared to Blackwell. And to bring all of that together and working on the details and the spec and bring all the systems and then seeing it live, it's extremely exciting. And behind the scene, it's not just like setting that up. We are actually, when we validate it, it means that we have validated that all the GPUs are working well together, that the bandwidth is what we are achieving, the liquid cooling is working. So just the entire system overall is tested together.

John Furrier

>> In your keynote, you said it's the best time to be an engineer. And this seems to echo the sentiment of the entire industry. It's a massive AI infrastructure build up, but it's really difficult. Talk about the scope of the Vera Rubin and validating the workload, because this isn't just loading software and running some tests. This is like a serious systems engineering effort. A lot's involved. What are the key things that you had to do? I mean, you have the keys to the kingdom, product and engineering. Your team made it happen. It's worked. We saw the videos coming out. Take us through that, because this is a marvel of engineering if you look at it. It's a super computer.

Chen Goldberg

>> And if I mention that Vera Rubin started a year ago, our AI-native cloud, this journey started six years ago. So this is not something that happened overnight. And really, the way we approached it is thinking about the workloads, the customers, and what we need to build in order to deliver the best, more resilient, agile performance system that can happen. And that's really what exciting. And I think one of the things that we are seeing in those kind of systems is that once that you are solving one bottleneck and other one appears. So maybe let me take you through a few things that are really unique with how we are building our cloud. The first one, the brain of the system, what we call mission control. And what we do there differently is that instead of thinking about compute network and storage separately, we're actually bringing those systems together and managing them as a whole. And that allows us to be more proactive and really optimize for the specific workloads. So that means that for example, if I'm running a job on hundreds and sometimes thousands of GPUs and something goes wrong along the way, I can automatically detect that and signal to the job how they should perform differently, making sure that every second matters and all the data that you would need is getting to the GPUs in the best performance. So that's kind of work that we've done for a long time. Now, moving to Vera Rubin, that's something that has been really fun for us. Vera Rubin, the same way as Blackwell really changed the way we think about compute. We talked about before we started recording about every rack being a supercomputer, and that's really an interesting way to think about those kind of resources. So we took it to the drawing board and we were thinking about how can we make sure that this super cool computer is the most efficient ever. And we built several innovation, I will talk about two of them. One of them is Valvey and you can see how fun it is for us because we choose our names in a fun way. And what Valvey does, it managed the liquid cooling system in a software-defined way. And to give people and the audience a sense, we can control a single valve at a seven-second time. So if we detect any leak as an example, we take action immediately, and that's unique compared to other industry solutions. In addition to that, we've built Racky that just brings the data from all the systems, all the sensors and allows us to optimize the system in the best way.

John Furrier

>> So you get a lot of telemetry in there on the system so you can move fast so you don't have the criticality of anything happening.

Chen Goldberg

>> Yes, exactly.

John Furrier

>> Talk about the shift. In your keynote, you talked about how customers are moving and the markets shifted with reasoning, multi-step reasoning. The performance levels for tokens are starting to see kind of like almost a portfolio, entry level, mid-range, high end. Vera Rubin's a high end and the rack scale systems have been the preferred method. Talk about that piece of it and why this is so important and why are those workloads requiring Vera Rubin? Or we'll say Blackwell. I mean, you have choices now, but what specific about the AI is making this an always-on resource requirement?

Chen Goldberg

>> One of the things that is really interesting right now in the industry is that the ceiling of the size of the model, for example, has gone away, which means that the accuracy and the latency has changed opening up for new set of use cases. And that's really what we are seeing happening right now. One of the challenges that was before Vera Rubin we're thinking about the economics of that technology and that's what's possible right now. So we have on one hand the models, the size, the correctness of them, but we now also have the infrastructure that makes it possible to create those new use cases. And we are seeing it really changing every industry there is from drug discovery to physical AI to call centers coding and it's really exciting times.

John Furrier

>> Yeah. You guys have all the models except for one and main models in there working with CoreWeave. And it's interesting. You talk about GPU utilization on one hand and model efficiency on the other. What you're getting at is that as you get smarter, the models become smarter, but also the GPU relation with the models. Talk about why the Vera Rubin matters because you have now better customer opportunities to have an always-on agentic system.

Chen Goldberg

>> Exactly.

John Furrier

>> Because now, it's GPU utilization, that's a technical infrastructure thing. But now the models, it's not distilled models, they want to run models. The model relationship to the infrastructure is a key thing, at least from what I'm hearing.

Chen Goldberg

>> Yeah. So I would call it two things here. One, that connection between the model and the infrastructure, it's no longer just the GPUs. We also need to make sure the data is getting there. So we care about latency, we care about, of course, reliability because those are becoming mission-critical systems. The second thing that I will call out that this change in technology is blurring the line between training and inference. If before you had some teams doing research and doing training models and others doing inference, these agentic systems are living systems. So what we want to do is take live results from inference and build it into the model back again. And that's what we really called the AI loop that makes sure that the system continues to learn.

John Furrier

>> I love the AI loop method because it's training and inferring at the same time because more data's coming in the fabrics help there, the networking features. What have you guys done to make that happen with this new rev? Because I see the power and cooling, liquid cooling's great, you got Racky, you got all this mission control, but what features you'll be put in there to make it go faster?

Chen Goldberg

>> So some of the investment that we have made before with our AI cloud is still paying off. For example, the investment that we've made with our object storage solution, making sure that we can deliver the most amount of data into the GPUs. But some of the things that we have now invested in is really working, by the way, with the industry on improving model performance. We talked about like KV cache optimizations as an example, reinforcement learning. So the innovation is no longer just at the infrastructure level, but actually also upper hydrostack.

John Furrier

>> Yeah. And you guys talk about optimizing all levels of the stack, but from a customer standpoint, what does it mean for them? Because where are they at? When you talk to your customers, what does this mean for them from an impact standpoint? Does it change their architecture? How do they roll this out? How do they engage? Does it change the economics? What's the all in? What's the view from the customer?

Chen Goldberg

>> I imagine it feels a little bit overwhelming for folks from a customer perspective because a lot is changing. First, the use cases are now possible. When I talk with customers, they're saying, "Okay, let's just think about how we create more opportunities for ideas to become a reality and impact the business." So creating and giving more resources to researchers and developers to change how things are going on. But I think what's even more interesting that is happening, that it's not just those ideas to reality and how do I train the model and research, it actually changes the entire ecosystem. So in this case, with Vera Rubin, we have solved latency and efficiency of inference. This is amazing. But now, what kind of tools do we give to developers? How do we manage capacity in the best way possible? How do we solve for power, energy, density? There are so many new problems that are big. How do we bring the right data? All of those new bottlenecks. You see me smiling again.

John Furrier

>> It's the best time to be an engineer.

Chen Goldberg

>> It's the best time being an engineer.

John Furrier

>> Engineers want to work on hard problems, but there's real opportunity behind it. What do you see in three years out? Obviously, you're in charge of the roadmap, you work with all the partners. I mean, extreme co-design is an NVIDIA first principle. Dell was involved and you have a lot of partners. What's it look like for three years from a CoreWeave perspective and what it might look like in the market?

Chen Goldberg

>> I'm looking at the past year or 18 month, I expect that this transformation will happen significantly faster than anything we've experienced before. And what we will see customers and the industry working on like with us is really creating new tools for folks. I expect we'll see models continue and evolve with maybe better size and so on. And maybe the most interesting part is making sure how agent systems are becoming a living system.

John Furrier

>> Yeah. Chen, great to see you and congratulations. A lot of exciting things. I'm sure there will be another step function change, more innovation coming. The next GTC will probably have another-

Chen Goldberg

>> And not just GTC. We are going to have a lot of more exciting news in Fully Connected, our first user conference of CoreWeave in September in Moscone.

John Furrier

>> All right, there it is. Another event, and sure it'll be a lot of innovation. Looking forward to covering that. I'm sure it's going to get bigger, faster, more economical for customers. Congratulations on a huge accomplishment.

Chen Goldberg

>> Thank you so much.

John Furrier

>> I'm John Furrier here in theCUBE Studios in Palo Alto. Now, sending it back to Dave Vellante in New York City.