CoreWeave Fully Connected 2026 | Ian Buck, NVIDIA

Clips
More from CoreWeave Fully Connected 2026

Ian Buck

VP / GM

NVIDIA

play_circle_outline From GPUs to Systems: Autonomous AI Agents Delivering Enterprise Value via Tool Calls

play_circle_outline CoreWeave Forge Doubles GPU Token Throughput While Addressing Data Sovereignty and National AI Infrastructure

play_circle_outline Inference vs training shift: growing inference demand and continuous training

play_circle_outline Tokens per watt, token economics, profitability and benchmarking

play_circle_outline Disaggregated serving and scale-in/scale-out networking across edge and cloud

Info
Transcript

Ian Buck, NVIDIA

Ian Buck

VP / GM NVIDIA

In this interview from Fully Connected 2026, Ian Buck, vice president and general manager of hyperscale and HPC at Nvidia, joins theCUBE's John Furrier and Dave Vellante to discuss the industry's shift from GPU-centric conversations to full-stack "AI factories" built to produce intelligence at industrial scale. Buck explains how agentic AI pushes infrastructure demands beyond any single model, requiring networking, storage and compute to operate in concert at data-center scale. He details Nvidia's Vera Rubin platform and its CPU-GPU co-design, noting how purp... Read more

explore Keep Exploring

How did AI evolve from running a model on a single GPU to enterprise-grade autonomous agents that require data-center‑scale infrastructure (and companies like CoreWeave)? add

What factors combine to multiply AI-driven productivity—such as access to the best models, reduced cost of inference, platform innovations—and how does data sovereignty play into this? add

What is driving the current inflection point in AI, and how do inference, continuous model serving, and online alignment/training contribute to building and improving next‑generation models? add

How do data-center power limits constrain deployment, how have GPU generations improved tokens-per-watt (for example Blackwell’s reported 30× gain), and how do you benchmark performance across diverse agentic workloads with varying prompt and context lengths? add

Does the emergence of "AI factories" and disaggregated serving represent a template or signal for broader disaggregated infrastructure, and would injecting intelligence at the edge produce the same dynamics even if the footprint differs? add

bolt Powered by CUBE AI

Ian Buck, NVIDIA

search

(INTRO)

John Furrier

>> Welcome back everyone to theCUBE's live coverage here in San Francisco, CoreWeave Fully Connected. I'm John Furrier with Dave Vellante. Two days of live streaming, getting all the action on CoreWeave and all the AI intelligence being produced by the systems that generate it. Of course, NVIDIA is a key partner. Ian Buck's here, the VP and general manager of hyperscale HPC at NVIDIA. Ian, great to see you again.

>> Yeah, hi, thank you for having me.

John Furrier

>> Great keynote up there, was geeking out on the— we saw the Vera. You guys had a great talk. One of the concepts we've been kind of fleshing out this year is the transition from GPU and model conversations to full-blown systems that produce intelligence in the entire system.

>> Yeah.

John Furrier

>> And how that's evolved and it's turning into AI factories being industrial centers producing intelligence.

>> Right.

John Furrier

>> And this is happening. So take us through this transition because this is a systems race.

>> It is, it is. It started, of course, with running an AI model on a GPU. And the original was literally a GeForce GPU, a gaming GPU in a workstation. But that demonstrated the possibility. In 2023, ChatGPT kind of made it come alive. But today we're living in a world of agents. We're actually bringing, and agents are bringing AI to enterprises. To do that, you have to understand the data the enterprise has. You have to understand the context. And they're autonomous. They're thinking, they're operating, they're optimizing the business. The human is there prompting, but the agents are thinking at this speed. To do that, you have to have networking at scale, storage at scale. You have to be optimizing not just your one model, but all the models. You got to be doing all of the tool calling, all the database operations. Everything that an enterprise needs, the agents are operating and interacting with. This is a data center scale problem. That's why CoreWeave as a partner has been such— understands this. They started in the infrastructure space, but as they're bringing it to market with enterprises with Forge, they can bring the entire NVIDIA stack, our CPUs, our GPUs, our networking, our software layers, and activate it for those enterprises.

John Furrier

>> I wrote a post on Substack, LinkedIn, and SiliconANGLE last week that went viral. It was very kind of technical, was out there, but it's basically the premise was the AI factory is turning into the computer.

>> Yeah.

John Furrier

>> And I was saying we might not even have the word data center. It might be an industrial center because of the componentry involved on the systems level. Share your thoughts on that because the system becomes the unit of value, inference becomes the economic engine, and engineering velocity becomes the differentiation. Basically was my premise. What's your reaction to that? And what's your commentary? Because this is kind of where the action is.

>> Yeah, the word data center, we think of data center, we think of historically it's IT, it's infrastructure, it's cost, right? But factories are different. They're producing products. They're cars coming off the line. They're generating revenue with everything a factory produces. Revenue is outputted and a consumer there is to take it to the next level and deploy it and use it. That's why factories is the right way to think about what we're building today. It's instead of cars or devices or PCs, it's tokens. It's units of information, of AI, of intelligence, tokens in, tokens out in order to produce those revenue, the opportunity, both for the infrastructure provider, the services that they're all built upon, and taking the intelligence and applying it to the rest of the market. AI factories is the right way to think about it. It's that these assets are not IT, they're not cost. They're actually appreciating, revenue-generating, fungible, durable, productive parts of an economy.

John Furrier

>> If you have a factory and say cars produce some cars in America, produce some cars in another country, we saw that when I was growing up, American cars were once great, then they had a lag and overseas cars came in. If the intelligence coming off the line, so to speak, isn't good, then that becomes an issue. So this is a lot to consider. That means models are the engine. What are your thoughts on that? Because this is where we start to see sovereignty come in.

>> Yeah.

John Furrier

>> You start to see, what are you optimizing for when you say, I want, everyone wants the best intelligence because injecting intelligence tokens drives revenue. Sure. That's, I buy that 100%.

>> There's actually a multiplicative effect here. It's almost an exponential. First, of course, is having the best intelligence means you have to have access to the best models, either the open models in the frontier or the closed ones. Either way, having the best intelligence can get you the best insights and allows you to apply to the broadest of workloads. On top of that is cost. Using a new model, the next generation of an open model like NVIDIA's Nemotron or the multitude of open models out there, can reduce the cost and as a result, take the intelligence times the cost and times those two together to actually have productivity. The third, of course, is innovation. We can't forget that. As with CoreWeave Forge, Chen on stage talked about using Forge, using their stack. They can actually double the number of tokens with the same GPUs. Literally by using their tools for free, customers get twice as many tokens on those same models. So that is another multiplicative factor on top. All three of these are happening at the same time. Of course, with sovereignty, it's about the data and applying it and having the resources, this critical resource for economies to grow. Every nation wants to have their infrastructure, their capability, not just to grow their economies, but actually have their workforces, their innovations, and their partners all make that possible for their customers.

Dave Vellante

>> So Ian, since this, if I can call it the modern era of AI, there's been a lot of confusion around, inference versus training. People sort of frame it sometimes as a zero-sum game. Michael Intrator communicates in the following way. He says, look, we're not building AI for training and AI for inference. We're building, AI infrastructure.

>> Right.

Dave Vellante

>> Okay. So the survey work that we've done suggests a couple of things. One is that the mix is shifting from 90-10 training to inference to 50/50 and it's headed toward 10/90. Yeah, but both curves are growing quite rapidly, as you well know. But help me square the circle because you have things like LPUs, which is, super low latency. You've got training, you've got inference, you've got, Jensen's Pareto curve.

>> Sure.

Dave Vellante

>> So help us understand what is the difference between AI infrastructure that's— is it optimized for any workload versus sort of optimizing for specific workloads?

>> People see the big super training clusters, right? And in fields in places like Texas building the next frontier models. Absolutely. Being able to do that level of sort of knowledge ingestion and actually build the next frontier model. It's a massive undertaking. But as AI is turning into really making money, AI being profitable, that is inference. And that's what's creating the inflection point that's happening now. And it is serving those models, creating the tokens in and producing like in an AI factory, the tokens out. What's interesting though is that it's a continuous process. It's not just a fire and forget on all these services. As companies are using these models, they're refining them, they're aligning them, they're adding more data to them and to make them more accurate, more capable. Avoiding model drift if stuff happens or if they're— if you're a consumer company and trends shift or new products come online, having them up to date and aware, that actually is a little bit of training. We're seeing the work in reinforcement learning and online alignment at the same time these models are being served continuously add more and more value. So the data centers are fungible. They can be used for straight up serving the models that exist today and existing workloads. Take the existing services and make them continuously smarter by doing continuous training while doing inference and then growing it further. That's why having a platform like Vera Rubin that's fungible, durable, productive, and will continue to add value and actually more value over time is so important right now given where we are in AI, the era of profitable AI.

Dave Vellante

>> And of course, it's not just the frontier labs or frontier companies, I guess, we're being urged to call them now, not just the OpenAI and Anthropic doing the training, it's enterprises. Caterpillar's doing training and so many models.

>> Not everybody needs a Frontier model.

Dave Vellante

>> Are you seeing demand? What kind of demand are you seeing at that far end of the Pareto, the right side of the Pareto for LPX? At one point, I think that was at GTC in March, Jensen was saying, well, it could be as much as 25%. I think he dialed that down to 10%. What are you seeing in terms of demand for that ultra-low latency inference.

>> You know, we share with the world what we're doing with LPX. It's a booster pack that we can put on top of Vera Rubin to really boost up and get the maximum number of tokens for one user. If there's value in those tokens to have the fastest possible thinking, LPX can be boosted on top of an LPX in order— LPX can be boosted on top of Vera Rubin to make that possible. Where is that the most important? It's for those extreme high value, longest thinking but smart reasoning, real-time decision-making. we're seeing a lot of interest in areas like fintech and other areas where things are happening in real time.

Dave Vellante

>> Monetization as well.

>> It's online, it's right now. And the ability to understand and discover and be able to think and have that time to first insight time to first token, or to get that answer right, right now. We can take a model, even a medium-sized model like a Gemma model, and deliver a 10x token rate, 10x faster for that one use case and get that answer immediately.

Dave Vellante

>> I got one more key question for Ian, if I may. There's also a lot of confusion around CPU versus GPU. I heard Cramer this morning kind of fumbling. I'm a big Cramer fan, but he was really fumbling with Michael Intrator. Saying, is that an Arm-based CPU? Of course it's an Arm-based CPU, but I've contended and written extensively that NVIDIA has the best AI CPU. And you hear so much about, well, NVIDIA's behind, whether it's AMD or Intel. Explain, if you would, support my thesis, because it's the integration of Vera and Rubin, that extreme co-design is what always struck me as the ideal optimal model. Give me your point of view.

>> I think people, this is a great topic. People don't know this. we have shipped millions of CPUs. NVIDIA built CPUs. It started actually with Grace Hopper and then Grace Blackwell. We've been shipping, and even companies like Meta haven't talked about, they're deploying Grace by themselves. to do the data analytics, to do the processing as a CPU. Think about it. We needed a CPU that we could pair with our GPU that was good at doing the AI tasks of controlling, of managing the workflow of that GPU to never let that GPU go idle. Turns out, of course, now in the era of agents, they were doing all the things that agents need to do, the tool calling, the processing, the managing, and time and performance and speed was critically important. It was a different kind of workload, different than the cloud CPUs, where cloud CPUs, it's just dollar per core. the use case that brought the cloud to market was I could have my e-commerce business. I could start with 10 customers, 100 customers, 1,000 customers, and scale to millions because the cloud was there. And the CPUs in the cloud delivered that, and you saw that, just scale up, just number of cores I could scale up. It wasn't about performance, it was just about scale. So we never actually saw single-thread performance actually improve. We just saw more and more cores popping in there, but the performance of a single thread, single core, really kind of flattened out.

Dave Vellante

>> Totally.

>> To do that, just to produce more cores, and reduce cost. A good innovation, but it left a gap, a gap where actually perf matters in CPUs, and it did for operating GPUs, and now it matters again for running agents.

Dave Vellante

>> Well, it's interesting. You hear the talking heads say, well, NVIDIA's behind in CPUs. Why do you think they misunderstand this? Because you're just so far ahead in GPUs over these years?

>> Load a sandbox, start an agent sandbox 3 times faster than an x86. We've done benchmarking across all the agent workloads, like running all the open benchmarks from Stanford and others, up to 2x faster. Cognition was seeing a performance of up to 4x faster.

Dave Vellante

>> It's called architecture, John.

John Furrier

>> Yeah, this is why I brought up the system, because I think that's a great example of you had to have your CPU to support the math feeding the math. Exactly. Which is what we now see as AI. And I think what's interesting about Vera Rubin is not just as new silicon, but it's actually a complete system, a unit of value. So I have to ask you on the token per watt benchmarks, this then brings up the revenue piece you mentioned, right? There's AI benchmarks coming in that are much different than the standard benchmarks on the economic side. Okay, so let's take this one step further. You got CPUs that match the GPU. We see why you did that. The rest is history. That's the systems working together. That's the new thing. That's the value, right? Talk about the economic impact as you start to think, okay, now I got to match economics to the architecture.

>> Yeah. Two things that are critically important. Obviously, data centers have a natural cap, and that cap is actually their power. How many megawatts can I deploy? real estate can— square footage, obviously budget matters. But in the end, I can only get provisioned a certain amount of megawatts for my data center. With every generation of GPU, we make sure that our tokens per watt is upwards of 10x more efficient. And in fact, we saw that with Blackwell. We got actually in the end a 30x improvement in token per watt.

Dave Vellante

>> Sandbaggers.

>> Another—yeah, exactly. Sure. It's hard sometimes, but it's there. And that's why benchmarking is so important. We can prove it. We can run those models and not just run a model on one workload. We can run our agentic workload where the prompts are from 10 tokens to 10,000 tokens, where the contexts are a few hundred words to tens of thousands to up to millions of characters. These are really hard benchmarks to run. The workloads are super dynamic and delivering performance across the entire workload.

John Furrier

>> Talk about the multi-generation hardware benefit. It's not a temporary inconvenience when you roll out a new architecture like Vera Rubin. There is leverage. Explain that concept because the old school was, we'll do upgrades, sunset out old platforms. You have a multi-generational strategy. Explain that.

>> So in the beginning, going all the way back to CUDA 1.0, which was 2006, literally 20 years ago, I'm starting to date myself. Yeah. We made a promise that CUDA 1.0 code is gonna work in CUDA 2.0 and 3.0 for as long as we shall live. And we've delivered on that promise. As a result, the workloads that we have been investing in and the developers, the 10 million developers now that are building new and new codes, all the AI frameworks, all the scientific computing codes, all of the enterprise codes, they run on the Keplers, the Voltas, the Amperes, the Pascals, the Hoppers, and the Blackwells. So with every generation, generation over generation, we take the entire stack, we reinvent it, we re-optimize it, we discover there's new workloads. Out of the box, customers see 2x, 3x, 4x more performance per GPU, per watt, per dollar. And then on top of it, they can optimize further through innovations like CoreWeave is doing to get even— because of the software you're sending, because of all the software and the layer of the stack that we operate at a high level, we can innovate down there while all of our partners are innovating.

John Furrier

>> Okay, so that's a good point. I want to build on that. So are we reaching the point where the single GPU is the wrong abstraction? Isn't the AI factory the product?

>> Yeah, that's an excellent point. AI models don't run on a single GPU anymore. They barely even run on, most of them don't run on a single server or a single box. we have HGX, it can still run a majority of the production smaller models. Models today want to run at rack and data center scale. When we do the frontier models, we're running an entire NVL72 or actually multiple NVL72s. We're splitting the pre-fill of the context. We're distributing the decode across all those GPUs, across all those NVLinks. That's why we created NVL72 in the first place. And we're doubling down again with Vera and even the future Vera Rubin Ultra and Feynman to expand that even more.

Dave Vellante

>> But an operator has to still choose or deploy a particular instance, does it not?

>> Customers have choice. They obviously can still get access to all that infrastructure and all the different platforms by going to CoreWeave and asking for different configurations. They know what they want and they can pick and choose, or they can go up level in the services. And in the token, the inference as a service provider like CoreWeave has now become, they can actually optimize your model of a certain size. And you may have a different trade-off between your throughput, how much processing you want to do versus token speed. I want to have the fastest possible tokens. CoreWeave can do that for customers. They don't have to feel overwhelmed by all the choices. That's where our partner ecosystem sits.

John Furrier

>> It's interesting, the magic, it happens when the phrase injecting intelligence into the system. We're seeing AI factories playing out beautifully. You mentioned some of the things also, you guys wrote a great white paper on this in your cloud reference design around disaggregated serving.

>> Right.

John Furrier

>> So would you say that that's a proxy or a telltale sign or a template for disaggregated infrastructure? Because the logic would then say, if you inject intelligence at the edge, right, that might not be the same footprint size, but the dynamics would be the same.

>> That's right.

John Furrier

>> What's your reaction to that? Because it's disaggregated serving, Why do you do it? The cache is full. There's a lot of variety of reasons, but now you got the edge, multiple nodes, factory nodes could be miles apart, closer together.

>> The first thing that determines the configuration is where are we deploying it for what use case? At the edge, you're in a cage, you may only have 100 kilowatts maybe of total power. Yeah. You're not going to be pre-training a Frontier model there. You're going to be serving a real-time interface or maybe it's a cloud-based robotics platform. By bringing it to the edge, your latency is short. That means you must have a real-time use case and you're deploying a, call it a 10 billion parameter model, 50 billion parameter model, maybe even 100. If you're going to the, thinking and agents and interacting with humans, the speed of text can tolerate quite a bit of latency there. So we can go to a more efficient, broader, bigger data center. It's still fungible though. So you typically see people deploying either our HGX 8-ways or the NVL72. Based upon those use cases.

John Furrier

>> You're connecting factories, basically.

>> Exactly.

John Furrier

>> And I love this idea of networking becoming so much more important. It's been covered, we've been covered like a blanket. You're distributing intelligence. It's not networking anymore in the classic sense. you're in the distributing intelligence game with networking.

>> That's right.

Dave Vellante

>> Well, on Monday you added scale-in networking.

>> Scale-in, scale-out, scale-across. scale in every direction.

John Furrier

>> Well, we got you here since you're such an expert and we at theCUBE are sort of a mixture of experts interview show, pun intended. What would you consider a beefy edge? You mentioned 100 kilowatts. That's like a central office. I would say that would be probably maybe 100 kilowatts. What's a good beefy edge for a factory, a good factory that could do training and inference?

>> 10 megawatts? Yeah, yeah. We see people doing, starting in the tens, going up to the 100 megawatt. Once projects start to scale up in the 100 and beyond, we're now starting to think about data center planning and operations. Sometimes we even see the layout of the data center will start to match what the models are going to be run on because they're thinking about both the training, the RL loop at scale and how they want to lay out their network and connectivity, neural pathways across that data center. And whether you shoot long on a spine or across a row and whether you go with L1 or L2 or even an L3, And once you get to 100 megawatts and above, you're thinking about that and you're tailoring it for latency.

John Furrier

>> Because, Ian, great to have you on. We could go another hour. Great to have you on. Appreciate your valuable time and congratulations. Love the innovation. We should call it Fully Connected Intelligence Conference here. This is theCUBE here at CoreWeave's Fully Connected Conference. These factories will be connected, distributing intelligence. The software stacks are fungible. They can adapt to any workload. We're doing our part here on theCUBE to bring you the data. I'm John Furrier with Dave Vellante. Thanks for watching.