This discussion examines artificial intelligence native infrastructure referred to as AI-native infrastructure, document AI and persistent agents. Hosts John Furrier and Dave Vellante speak with Lukas Biewald of CoreWeave and Jerry Liu of LlamaIndex. This interview is recorded live at CoreWeave's Fully Connected 2026 conference.
Biewald discusses CoreWeave's AI-native cloud, hardware and software co-design and capacity strategies for low-latency inference. They emphasize tight hardware and software integration as a means to achieve orders-of-magnitude performance improvements and prioritize latency, burst capacity and observability for enterprise inference.
Liu outlines LlamaIndex's evolution into a specialized AI lab focused on document parsing, context quality and evaluation-driven workflows. They emphasize that high-quality context, rigorous evaluation processes and tailored post-training are essential for persistent agents and document understanding.
Key takeaways include the importance of AI-native systems that prioritize latency, burst capacity and observability, effective governance and privacy controls and inference-driven economics for enterprise adoption. The conversation highlights hardware-software co-design, document-centric modeling, context quality and robust evaluation as factors to consider when deploying persistent agents and large language models in production. The discussion also addresses GPU capacity planning and vendor considerations such as NVIDIA and platform tools such as Weights and Biases.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for Coreweave Fully Connected 2026
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Coreweave Fully Connected 2026.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to CoreWeave Fully Connected 2026
Please sign in with LinkedIn to continue to CoreWeave Fully Connected 2026. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Jerry Liu, LlamaIndex & Lukas Biewald, CoreWeave
This discussion examines artificial intelligence native infrastructure referred to as AI-native infrastructure, document AI and persistent agents. Hosts John Furrier and Dave Vellante speak with Lukas Biewald of CoreWeave and Jerry Liu of LlamaIndex. This interview is recorded live at CoreWeave's Fully Connected 2026 conference.
Biewald discusses CoreWeave's AI-native cloud, hardware and software co-design and capacity strategies for low-latency inference. They emphasize tight hardware and software integration as a means to achieve orders-of-magnitude performance improvements and prioritize latency, burst capacity and observability for enterprise inference.
Liu outlines LlamaIndex's evolution into a specialized AI lab focused on document parsing, context quality and evaluation-driven workflows. They emphasize that high-quality context, rigorous evaluation processes and tailored post-training are essential for persistent agents and document understanding.
Key takeaways include the importance of AI-native systems that prioritize latency, burst capacity and observability, effective governance and privacy controls and inference-driven economics for enterprise adoption. The conversation highlights hardware-software co-design, document-centric modeling, context quality and robust evaluation as factors to consider when deploying persistent agents and large language models in production. The discussion also addresses GPU capacity planning and vendor considerations such as NVIDIA and platform tools such as Weights and Biases.
In this interview from Fully Connected 2026, Jerry Liu, co-founder and chief executive officer of LlamaIndex, joins Lukas Biewald, senior vice president of AI initiatives at CoreWeave, to talk with theCUBE's John Furrier and Dave Vellante about the infrastructure demands of long-running, context-aware AI agents. Liu traces LlamaIndex's evolution from an open-source orchestration framework into a specialized AI lab focused on document parsing and extraction, with workloads now running roughly 75% inference and 25% training on CoreWeave. Biewald explains how en...Read more
exploreKeep Exploring
What was the introduction on theCUBE's live coverage of CoreWeave's Fully Connected event, including who was introduced and the event's focus?add
How should enterprises govern autonomous AI agents—including privacy/access controls, limits on open-ended user experience, and the use of evals and reward/prompt design—to prevent “rogue” behavior while preserving usability?add
How has your company's focus for building AI applications evolved, and what are you currently concentrating on (including why you need GPU compute and inference capacity)?add
Why did you choose CoreWeave—was it solely for GPU availability, or were there other factors influencing that decision?add
How does your company fit into the "infinite loop" Chen showed today—specifically, what role do you play in observability and how do you integrate hardware and software telemetry to tighten that loop?add
How do AI-native cloud platforms like CoreWeave differ from first-generation clouds such as AWS and GCP, and why does that matter for AI companies and entrepreneurs?add
>> Welcome back everyone to theCUBE's live coverage here in San Francisco for CoreWeave's Fully Connected event, their first conference bringing together the leaders, industry participants, and entrepreneurs, and also innovators who are using AI native and also using the great AI infrastructure. Got Jerry Liu here, co-founder and CEO of LlamaIndex, doing some great work. Jerry, great to see you again. Welcome back to
Jerry Liu
>> theCUBE.
Jerry Liu
>> Yeah, it's great to be on,
John Furrier
>> Jerry.>> Lukas Biewald, Senior Vice President AI Initiatives at CoreWeave. Congratulations on all the news and success.
Lukas Biewald
>> Thank you.
John Furrier
>> Business performance, obviously hot market. So congratulations, guys. the topic here is, okay, AI native is happening in the enterprise beyond just like TikTok stuff like video. We're talking about real value being created on these tokens. Data sources are a huge issue topic. Private data is now the moat. Yeah. I need an architecture that's going to be AI native. Not just compute, but like the role of the CPU, the GPU, the system is the unit of measure. So first, how are you guys working together? Talk about the relationship between you guys.
Jerry Liu
>> Yeah, maybe I can trace a little bit of history from LlamaIndex started 3 years ago as a very popular open source framework for developers to actually build applications using the early days of LLMs, right? It was very simple. You build like simple chatbots, you build like simple agentic workflows. Nowadays, obviously a lot of that stuff has totally changed. people are building fully autonomous agents that can run for like hours, if not days, and just proactively listen to everything. But even from the early days, we've always cared about how do you actually get high-quality data into these models? I can talk a little bit about our current focus these days, about what we decided to do with that vision. But even from the early days, we wanted to really provide developers the right tools to actually get data into LLMs that was very high quality. That involved a little bit of orchestration and any sort of agentic workflow that involves orchestration of your data, of course, requires some sort of observability. And so I think I met Lukas, I think 3 years ago or so. I think we chatted a little bit and basically worked on an integration with our framework, with Weights & Biases as it stood. And we basically had a native integration between how do you actually orchestrate over your data with the native observability tools of Weights & Biases. That was from 3 years ago.
Dave Vellante
>> This is prior to the acquisition.
Jerry Liu
>> This is, this is from, from probably 2 years prior or so. So it's been a good, fruitful partnership since then. I'd actually be curious to learn more about the current focus, but especially for us, we've really narrowed our mission from some of the early days because a lot of the patterns, as you might imagine, have changed. People are no longer building very simple primitive chatbots. People are more focused on How do you actually make use of some of these agents, whether powered by frontier models or open weight models, and actually provide the right context and tools and skills to allow it to do various types of tasks? And we really decided to focus on the best-in-class context for document processing.
John Furrier
>> Lukas, talk about the adoption on the enterprise side too, because AI initiatives have opened up with coding, obviously broke down the barriers, search. Glean nailed that use case in the early days. Now they're like, okay, I have initiatives. Initiative, that's what you, on your title. But these initiatives were like, we'll pilot it. Some go into production, some don't. Agents put the enterprise at the center of this because they want them. Yeah, they absolutely do. But if you don't tell them what to do, they go rogue. Now, first of all, I don't think the agents go rogue. They just do what they do. Find the task. Oh, I don't care about that database credential. No one told me.
Dave Vellante
>> Yep.
John Furrier
>> That not to do it. So a lot of things are going on in the enterprise with agents.
Lukas Biewald
>> Yeah, absolutely. And at CoreWeave, our goal is to empower what we call pioneers or the people out in the front adopting real-world applications for enterprises. And at this point, that's everyone, right? There's no enterprise that's not using agents to do something meaningful for their business. So it's been just an absolute explosion of adoption. As you know, it's not just a research topic. It's not like a nice to have at this point. Every enterprise has to have agents really working for them. And of course, a lot of those applications fail, right? A lot of things that you try don't work out. But I think the more surprising thing is that so many of them do work out for really meaningful applications. Like you were just telling me how you use agents to look back at old interviews. That's not surprising at all. A year ago, 2 years ago, That would've been very surprising to see that. Now every interview that I walk into, I expect that it's being transcribed, it's going into data stores all over the place, it's being built on. I expect that most of the questions that I get asked are built by other agents that have researched me and are coming up with examples. So it's just amazing to watch the explosion of use cases.
John Furrier
>> And then you're gonna have agents that fill in for the same repeated questions that you get all the time. One of the things that I like about, first of all, I love that call out because this brings the trajectory In the keynote, what stood out to me was the looping feature and the fact that you started to hear themes like always on, long running, living learning system. Love these computer science concepts because that implies neural networks. It implies kind of the systems approach. Most people think, oh, short form, get a response. It's like a search result. But there's a lot going on with agents specifically and the reasoning steps that might go for a long time. Talk about this long-running, short-running dynamic because you got to have the memory. Now that's a loaded term.
Lukas Biewald
>> Yeah.
Dave Vellante
>> What does that mean?
John Furrier
>> So take us through you guys' perspective on this because this is where it gets interesting because the work could be heavy lifting work or quick hits.
Dave Vellante
>> Yeah.
Lukas Biewald
>> Do you want to take it?
Jerry Liu
>> Sure.
Dave Vellante
>> Yeah.
Jerry Liu
>> I'll give a short answer. I'd love to hear Lukas's thoughts.
John Furrier
>> You give the short answer, you get the long answer. Yeah. Sounds good.
Jerry Liu
>> I would say 3 years ago, people mostly focused on short-horizon AI tasks through like, you ask a question, you got back an answer, it takes a few seconds, it's pretty easy. It became clear over time, general agents backed by these harnesses could start to just handle like more and more tasks. You know, you have Opus 4.5 starting to just automate coding end-to-end. They're starting to automate more and more end-to-end knowledge work tasks across the spectrum from finance to legal to various types of healthcare processes. I think we're starting to see the trend of always-on persistent agents that due to improvements in both model architecture, harnesses, as well as the components that back all these, including the right context, you now have persistent personal assistants. I'm sure you've all seen Instinct and Muse and all
Dave Vellante
>> that.>> use
Jerry Liu
>> it.
Jerry Liu
>> Yeah, and Dots that just came out yesterday.
John Furrier
>> Not gonna give it my banking information yet,
Jerry Liu
>> but—
Jerry Liu
>> And it's a fully autonomous employee or a friend that you can just continue to have it send this email and it will. We're gonna see definitely more of that, whether in the workplace or for personal capacity.
John Furrier
>> The long-running agents, 'cause almost they're agents of agents with infrastructure, plumbing, maintenance to just tackle longer tasks that require different protocols, different tokens.
Lukas Biewald
>> Well, I think it's pretty simple, right? You wouldn't hire an employee for one day no matter how smart they were unless it was an incredibly specialized task, right? The context matters so much. And when agents can keep running day after day and keep learning, there's just so many more things that they could do. And so I think when people first encounter agents, when they first encounter AI, they're amazed by how powerful it is. But quickly they realize that, wait a second, it kind of comes to this task anew every time if it's running without context. And that gets really frustrating. you don't want to keep reminding it who you are, what your bank account information is, where you go to school, etc. And when the agent can access all that, when it can be a single agent that keeps that context in memory, it just opens it up to so many more applications.
John Furrier
>> Talk about the governance piece, because you mentioned context. That brings up in my mind graphs, but also governance. One of the great use cases of AI now is people in the SQL world don't have to do the queries. You can just say, tell me what the sales were in the fourth quarter or whatever, and it just does the query for you. But with agents, you have to be prescriptive to some level, but no one wants to write a long-ass prompt. So they could be directed, but when they're out on their own talking to other agents, multi-step reasoning kicks in. They gotta have some sort of intelligence around that. how do you guys view that? Because this has become the biggest conversation we've been seeing. because everyone says agents are going rogue. I submit that that's not a false statement. They went rogue because they didn't know what rogue is. Let's say they're just doing their job while you're sleeping. So what's your take on this governance piece? And the user experience not being so open-ended?
Lukas Biewald
>> Well, look, I think it's incredibly important. It's one of the things that's most top of mind among CoreWeave's customers. I think this going rogue thing really opened people's eyes to what these agents can do when you're not incredibly explicit about what they can and can't do. And just like you need privacy controls with employees or people that you work with, you need explicit privacy controls with agents. And you can't just give it access to the whole web or your whole intranet or whatever. And I think it's a real frontier right now. I think it's holding back enterprises from adopting them as much as they should. And I think there's a lot of innovation happening at CoreWeave and elsewhere to make governance a more solved problem. Go ahead,
Jerry Liu
>> please.
Jerry Liu
>> I think the other related piece that I think you might like is I think everybody needs to build evals, literally build evaluation benchmarks for every single task because evals are not just about how accurate an agent performs on a given task. It's also whether it's aligned with human goals. and we're starting to see prompt engineering move a little bit upwards from just being extremely prescriptive and detailed about what you can and cannot do towards just being able to make sure you have a broad scaffolding over a task and making sure that you craft the right reward signals to actually achieve the task and actually start to defer more implementation details to the agents.
John Furrier
>> Explain reward system, because I think that's an outcome concept.
Jerry Liu
>> Exactly. If you've ever used any of these coding agents like Codex and Claude Code, if you're a programmer, you're very familiar that there's this like /goal command where basically you just tell it a final end state and it'll just keep working on it until it actually achieves a final outcome. And at the same time, you see every single frontier model is getting run across 1,000 different benchmarks and they're basically advancing across like all the general knowledge work benchmarks over time. At the same time, there's this concept of owning your own intelligence too, right? Within the enterprise, and really tailoring every single model to a very specific task that you care about and optimizing at the Pareto frontier of cost and accuracy. And so I think every— it's basically everybody's responsibility to actually define these right evals and benchmarks to make sure that whatever AI you're using is actually optimized for your needs.
Dave Vellante
>> So I'm curious as to how LlamaIndex has evolved from you were kind of known as the RAG specialist, document parsing, semantic search, things of that nature. How that's evolved. And the relationship with when we first met, it was at a UiPath conference, RPA, which was computer vision and document understanding. These techniques have evolved, these capabilities have evolved. I'm curious as to how you've evolved as the market has evolved and what role CoreWeave played in that evolution.
Jerry Liu
>> Of course. So we used to depend a lot, or we basically focused a lot on orchestration and building the right tools for developers to actually start to learn some of the best practices to build AI applications. RAG, advanced RAG, you know, initial agent loops, we provide the tooling for that. Nowadays, to be totally transparent, you kind of don't really need that anymore because most of these agent harnesses have standardized. But what we saw was really the opportunity at the context layer. And for us, a lot of knowledge was stored within just high volumes of really complicated documents that actually required really advanced AI techniques to help unlock. So we're effectively a specialized AI lab right now that's purely focused on building models for document parsing and extraction. We post-train open-weight models, we gather our own datasets, and we make it really, really good at just analyzing and reading documents to basically extract that data. As a result, we obviously need GPU compute and inference capacity, which basically kind of falls very nicely in line with the themes of this conference. And so, we care a lot about making sure that we can actually tailor everything we're doing at the Pareto frontier of performance, cost, and latency. For our customers.
Dave Vellante
>> So why CoreWeave? Is it anything beyond I can actually get the compute from CoreWeave, it's clearly you guys are able to deliver, you know, faster than the hyperscalers. Is it just that GPU availability or are there other aspects that it—
Jerry Liu
>> Yeah, some of these are ongoing conversations I'd be happy to explore, but latency is a huge consideration for us. Being able to actually run these models at scale and handle extremely bursty traffic is also an extremely interesting aspect. We care a lot about building stuff at the model layer and that we own the cost of compute. And so we want to make sure that we deliver the best experience for our customers.
Dave Vellante
>> And the majority of your workload or all the workload is training or is it a mix?
Jerry Liu
>> It's a mix, probably 75% inference and 25%
Dave Vellante
>> training.>> So mostly inference. Yeah, exactly. And what was it, say, a year ago?
Jerry Liu
>> Basically we didn't really have this a year ago. Okay, so yeah, it was zero. Okay. We process like millions of pages per day in document volume and very bursty traffic and a lot of our customers across finance, legal and insurance have very bursty
Dave Vellante
>> paperwork.>> And training and inference—I presume they're both growing. Yes. I assume inference is on a steeper curve or not necessarily?
Jerry Liu
>> I would say it's probably growing pro rata or kind of in proportion to
Dave Vellante
>> demand.>> So you see that 75/25 is staying consistent as far as you have visibility.
Jerry Liu
>> Inference demand is basically a function of like customer demand. And then the training is kind of a function of our research team. So probably inference demand will grow a little bit quicker, but definitely GPU capacity and compute and optimization are very top of mind topics for us right now. Yeah.
Dave Vellante
>> Excellent.
John Furrier
>> On the NVIDIA side, the relationship you guys have with NVIDIA has come in handy. How is that playing into the initiatives? Because it seems like the software industry and developers moving down the stack, NVIDIA is moving up the stack. You guys are key partners.
Lukas Biewald
>> Yeah.
John Furrier
>> Take us through the impact there to the AI initiatives.
Lukas Biewald
>> Look, I think NVIDIA is always testing different things out, right? So they've, as long as I've watched this industry, NVIDIA has offered software and offered more complete solutions, but it's been an incredibly helpful, productive relationship with NVIDIA. There's been a lot of cross-sharing of ideas. There's been a free conversation about what's the best way to set up these data centers. One of the learnings for me when I came in as a software guy into CoreWeave, I had the same question as you of like, hey, how much difference really could there be in the way that you configure these chips? And I'll tell you, the answer is massive difference. I'm talking orders of magnitude difference depending on how you do the networking for the chips, how you do the power distribution. It's not just privacy and security and reliability. It's literally things orders of magnitude change in latency. And you can see that in the publicly available benchmarks.
Dave Vellante
>> Yeah, you came in through Weights & Biases,
John Furrier
>> right?
Dave Vellante
>> Yeah.>> Yeah. Okay. So I got some questions there, but go
John Furrier
>> ahead.>> ahead. I have a lot of questions on that. So it's huge right
Dave Vellante
>> now.>> So the infinite loop that Chen showed today, where were you guys in that loop? it's obviously the observability piece, but can you help us understand how you fit into that?
Lukas Biewald
>> Well, when we started, we were all about observability and evaluation.
Dave Vellante
>> Yes.
Lukas Biewald
>> And the way we thought about our business actually at Weights & Biases was making that loop tighter. So we are very, very complementary with CoreWeave in the sense that We believe that the faster you make that loop, the faster you can see what's going on with what you're doing and then improve things, the better off you're gonna be. And Weights & Biases was completely focused on that observability piece.
Dave Vellante
>> Right. Right.
Lukas Biewald
>> We didn't help people actually train their models. We just told 'em, hey, are the models working or not? Which is very important, right? It's easy to get that wrong. And so that was my complete focus until a year and a half ago when we did this, acquisition.
Dave Vellante
>> Because I know in speaking to some of your customers, they're, they— you have— CoreWeave has a number of certifications. I think you lead there relative to the neo clouds. Obviously, guys like AWS have been around a lot longer.
Lukas Biewald
>> Right.
Dave Vellante
>> But you have that, BAA, things of that nature. Does this allow you to get more granular, information, logging, telemetry, dashboards?
Lukas Biewald
>> Absolutely, because it's all combined. Right. So One of the first things that we launched together after CoreWeave acquired us is a way for researchers to look at not just is my model working, but where were there hardware failures. And one of the things I love about CoreWeave is they embrace the fact that hardware failures are inevitable, right? Like no one can credibly say I have a data center with 100,000 or 1 million GPUs and there's no failures. rather than say there's no failures, we say, look, we're going to give you complete visibility into the failures. But the failures just on their own are actually not that valuable. It's what happens because of the failures. So LlamaIndex trains a model. They see maybe it's not quite as good as the last model they trained or not quite as good as they were hoping. They can look at the curve of how the model is progressing and they can overlay on top of that what was happening in their data center and see if there were any issues because maybe there's a latency hiccup from a networking issue or maybe something overheated and went down for a few minutes. And so by putting all that together, by blending the hardware and software, we're able to make customers much more successful and actually tighten that loop that Chen Goldberg was showing.
John Furrier
>> And you guys were doing a lot of pioneering work too before you got acquired. I thought that was a great deal by CoreWeave. So smart because we're seeing now, we were kind of riffing on earlier, is if you're going to build a brain, these pathways, weights become huge. You're seeing what's going on with KVCache and disaggregated serving. So take it to the next level. The vision, what's your vision, guys, on if you're going to have this AI-native environment, you want to maximize the compute. Like I said earlier, when I put my hand on the stove, I don't really reason. I just say, take it off. That's almost like a weight. So open weights are coming to the system. How is that going to optimize the retrieval for the intelligence? Because everything's about intelligence production. that's what AI factory is.
Lukas Biewald
>> Yeah.
John Furrier
>> You're producing intelligence. The intelligence comes from, I guess, the brain, my metaphor, but there's costs involved. Retrieval costs, latency, compute. Weights become huge. What's your vision? Because this goes next level.
Lukas Biewald
>> Yeah.
John Furrier
>> Yep. It's got agents who are going to need to reason. You're going to have a compute envelope, energy envelope, rationing energy.
Lukas Biewald
>> Well, look, I think the first generation of clouds like AWS and GCP were amazing for entrepreneurs. I remember when they came out and the fact that you could buy these things more like electricity and not like a huge capital expenditure off the bat was— it was incredibly democratizing. It was incredibly powerful for innovation in our economy. It's one of the most amazing things that happened. And I think what's happening with CoreWeave now is a new kind of cloud that's built from the ground up to be AI native. And that's actually pretty different, right? The AI world is quite different than the past world in that it's ripped up all the assumptions, right? CPUs had the same instruction set for 20, 30 years, right? I mean, I remember studying it when I was in college, but I didn't really think about it much, right? But now, GPUs come along. Each new GPU has different things that it can do, different capabilities. Researchers now have to know what the GPU could do. And there's this whole set of new tools that you need, like observability tools, reinforcement learning tools and things that have never existed that need to help the next generation of companies. And what I'm so excited about, CoreWeave is just enabling folks like LlamaIndex to build their business on top of a really good cloud that's reliable and also AI.
John Furrier
>> So you're— you've been on both sides of the table. You've been on his side. He's there now. He's been innovating on the software side of it with the AI infrastructure. What's different about the system? Because an average person might say, oh, I see Graphics processing unit, I see storage, I see memory. It's a system. It's not just bespoke components. I think that's what I see with CoreWeave.
Lukas Biewald
>> Exactly. Yeah.
John Furrier
>> Explain the importance of this for the software layers that are getting smarter.
Dave Vellante
>> Well, if you're one of my customers, why don't you explain it in the terms of why didn't you just go to like a hyperscaler? Why? What was the difference?
Jerry Liu
>> Yeah, well, so we have conversations with a few different folks, I think. And so, happy to— we're very open to exploring partnerships with CoreWeave and a lot of other providers too. I will say, I think this is just one of those things where I think what we're doing basically encapsulates what pretty much not only every AI company is doing, but also what I foresee every individual doing. Going forward. I have this vision of just like, okay, you have these AI-native clouds, you have a lot of like wide abundance of GPU compute. At the same time, I think the ability to actually fine-tune and post-train intelligence is typically gated to a pretty small subset of people right now. It's not like just your average non-technical user knows how to go and post-train an open-weight 4 billion parameter model to make it really, really good at something. Especially with this, in the spirit of loops, coding agents are getting very good. Everyone is starting to get really good at defining observability and evals and the right metrics to focus on. I think there's going to be a world where we're basically just going to automate this entire loop and make it accessible to everybody. Yeah.
Dave Vellante
>> And so I didn't mean to insult any of your other relationships. I think CoreWeave and the hyperscalers, they're not going away. They have data, they have assets, applications. There's a marketplace there. Of course you want to work with them. So what I'm trying to understand is when does a developer like you think about, okay, I want to go with AI native, that AI layer CoreWeave versus hyperscalers, maybe it is the marketplace, they can sell a lot of business or even do stuff on-prem where you got to worry about GPU utilization and the like. How do you think about those trade-offs?
Jerry Liu
>> I think it's some combination of financial aspects plus really being able to optimize for technical capabilities around latency capacity and performance at cost. we serve a lot of different customers at extremely both persistent and also spiky workloads. And we really, really need to make sure that we have the right capacity to serve our customers without getting throttled. We also need to make sure from just a kind of CapEx perspective that we're able to make the right allocations to be able to predict customer demand over time. And I think the value of a lot of companies like CoreWeave is really being able to provide a lot of this capacity. So that we're actually able to sustain ourselves as a business without actually having to own the operational logistics of figuring out our own cluster of GPUs, for instance.
Dave Vellante
>> So you're not generating your own tokens in a private instance, or are you in certain cases? Maybe some developers have like a Mac mini or something, but—
Jerry Liu
>> Yeah, we don't have a private cloud. That's, we don't have, we don't own our own, for instance, like private cluster of GPUs.
John Furrier
>> Asset-light, as they say.
Jerry Liu
>> Yeah, yeah, exactly. It really does help startups get started because you're able to start owning this research loop of model tuning without actually having to own the kind of upfront cost.
Dave Vellante
>> I said earlier, this is very much like AWS in its early days, enabling startups to do things that they don't want to do on their own.
John Furrier
>> And for AI native though, to his point.
Dave Vellante
>> And it's AI native. But again, you're on a similar trajectory in terms of, okay, you've got compute and now you're adding storage, you add networking, you got a software layer. Right? You're adding certifications, security. I mean, it just seems like
Lukas Biewald
>> a—
Lukas Biewald
>> But I'll tell you, there are some key ways that CoreWeave thinks differently than like an AWS
Dave Vellante
>> might.
Lukas Biewald
>> Sure.
Lukas Biewald
>> For example, CoreWeave knows that they're not the customer's only cloud, right? So I think the cloud vendors are really going to try to lock you in to just use AWS. We want you to live in AWS's world. A lot of things they do have proprietary APIs that are hard to port to other places. They really want to do that. Of course, that's their business model. CoreWeave knows that everyone is coming from a different cloud. Everyone's going to host their web service on AWS or GCP, not on CoreWeave. And so CoreWeave is okay with that, right? So CoreWeave plays much more nicely with the other clouds. It's just one example. Also, I think CoreWeave is much more willing to work with NVIDIA and do standard networking protocols that NVIDIA recommends as opposed to AWS or GCP protocols, which means that your stuff has more open source support. Out there in the ecosystem. So there is a really different mindset here as a newer company that doesn't have the legacy baggage
Dave Vellante
>> of—>> Yeah, fair enough.
John Furrier
>> It's funny, I wrote a post a couple of months, a couple of weeks ago called this AI era is math feeding more math, right? You got to get everything in mathematical terms to feed the GPU. And NVIDIA, Ian Buck was up earlier saying our compute was purpose-built to match our GPU need because the compute wasn't fitting. That's math-based, obviously, at their base. And then floating points, FLOPS per watt. That's no longer a metric. It's token per watt. So you're already starting to see the transition. It's a done deal. The question that we're watching is what gets built on it. So I do think there's a swipe your credit card moment for intelligence infrastructure that you guys have. Hey, you want to do some other stuff? There's plenty of choices to store stuff and host stuff. It's what you're doing that gets the focus because you're now looking at a tailwind of a new kind of creator builder. They're a builder, they're an operator and an investor. They're investing in themselves. They got token budgets. You never saw that before. You were a builder and then you get some money from a VC or capital provider, an angel, and then you hire an operator. Now the individual is all three. You guys are doing it.
Jerry Liu
>> Yes.
John Furrier
>> All right. Let's end with your last word. Explain what you guys are up to now. You have a great history and again, congratulations. Appreciate what you do.
John Furrier
>> Yeah.>> There's a new era in front of you. Put a plug in. What's cool? What's the coolest thing you're working on? What should people do? How do they use your service?
John Furrier
>> Yep.>> What's the coolest thing people are doing? Give a plug.
Jerry Liu
>> Yeah, today, we're a specialized AI lab building frontier models for document OCR and extraction. So if you have really complex PDFs and you want to digitalize it for RAG or for document extraction use cases, come talk to us. There's plenty of use cases across finance, insurance, and legal with a lot of PDFs and PowerPoints and Word docs. And that's exactly what we do with our models.
John Furrier
>> We heard from Caterpillar, who was on stage. They have massive history of work orders. they're sitting on a treasure trove. Good lead.
Jerry Liu
>> Would be a good lead.
John Furrier
>> Lukas will hook you up. Guys, thanks so much.
Lukas Biewald
>> Super cool.
John Furrier
>> Just the beginning. Thanks for coming on and sharing your commentary on TheCUBE.
Lukas Biewald
>> Thank you.
John Furrier
>> Thank you. I'm John Furrier with Dave Vellante. We're here at the Fully Connected— we should call it the Fully Connected Intelligence Conference. AI factories are producing intelligence. That's the unit of measure. Inference is the value. More coverage coming here, live streaming after this short break.
John Furrier
Dave Vellante