Alon Horev of VAST Data, co-founder and chief technology officer, discusses VAST Data's approach to enterprise artificial intelligence, AI infrastructure at CoreWeave Fully Connected 2026 with hosts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research. Horev explains how remote direct memory access, RDMA, quad-level cell flash, QLC, and distributed namespaces enable large graphics processing unit, GPU clusters and support agentic AI use cases. They describe key-value cache, KV cache, management, memory hierarchy design, and disaggregated serving for modern inference workloads, and they draw on VAST Data's experience scaling to exabyte scale.
Horev emphasizes VAST Data's positioning as an AI infrastructure provider and the role of persistent key-value cache, KV cache, in reducing recomputation and enabling higher concurrency. They highlight remote direct memory access, RDMA, BlueField data processing units, DPUs, and disaggregated prefill and decode as enablers for elastic scheduling. Analysts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research note that data movement, traceability and VAST Data's DataSpace caching are critical to performance and cost for agentic AI.
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for Coreweave Fully Connected 2026
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Coreweave Fully Connected 2026.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Almost there!
We just sent you a verification email. Please verify your account to gain access to
CoreWeave Fully Connected 2026. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to CoreWeave Fully Connected 2026
Please sign in with LinkedIn to continue to CoreWeave Fully Connected 2026. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Alon Horev, VAST Data
Alon Horev of VAST Data, co-founder and chief technology officer, discusses VAST Data's approach to enterprise artificial intelligence, AI infrastructure at CoreWeave Fully Connected 2026 with hosts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research. Horev explains how remote direct memory access, RDMA, quad-level cell flash, QLC, and distributed namespaces enable large graphics processing unit, GPU clusters and support agentic AI use cases. They describe key-value cache, KV cache, management, memory hierarchy design, and disaggregated serving for modern inference workloads, and they draw on VAST Data's experience scaling to exabyte scale.
Horev emphasizes VAST Data's positioning as an AI infrastructure provider and the role of persistent key-value cache, KV cache, in reducing recomputation and enabling higher concurrency. They highlight remote direct memory access, RDMA, BlueField data processing units, DPUs, and disaggregated prefill and decode as enablers for elastic scheduling. Analysts John Furrier of theCUBE Research and Dave Vellante of theCUBE Research note that data movement, traceability and VAST Data's DataSpace caching are critical to performance and cost for agentic AI.
In this interview from Fully Connected 2026, Alon Horev, co-founder and chief technology officer of VAST Data, joins theCUBE's John Furrier and Dave Vellante to discuss how data movement has emerged as the critical bottleneck of the agentic AI era. Horev traces VAST's origins back to 2016, before the AI boom, when the company set out to bring HPC-grade performance to enterprise storage. He explains how falling flash prices and faster RDMA networking over commodity Ethernet finally made disaggregated architectures practical, letting compute separate from stora...Read more
exploreKeep Exploring
Why did the industry take so long to adopt disaggregated architectures that separate compute and storage, and what technological enablers made that shift possible?add
How do you think about VAST’s category, role, and product strategy given it overlaps storage, databases, data platforms, and AI infrastructure?add
How does an AI-native philosophy and system architecture align with the emerging storage hierarchy (GPU memory, local CPU memory, and persistent media like VAST), particularly in light of NVIDIA's recent STX/Dynamo announcements?add
How will the rise of agentic AI change infrastructure and storage requirements — including scaling to exabytes, scheduling and migrating thousands of agents and GPUs across regions, handling data locality via caching, supporting multimodal data (tabular, unstructured, KVCache), recording telemetry for traceability and retention, and leveraging agent conversations for fine‑tuning — and how can systems (e.g., DataSpace/VAST) address these challenges?add
>> Welcome back, everyone, to theCUBE's live coverage here for CoreWeave's Fully Connected. I'm John Furrier with Dave Vellante. This is theCUBE, our 17th year, Dave, doing theCUBE. We've seen a lot of movements and a lot of firsts. We saw the rise of Kubernetes, the rise of the cloud. Some things didn't work out like OpenStack, but I got to say the AI, the Neo— Hadoop. Hadoop, that turned into Databricks. We know what happened there, Snowflake. Now with the AI here at CoreWeave, the Neo Cloud era, now the AI infrastructure era is here. Yeah, we've seen the players, CoreWeave, VAST Data, and Alon, who's the co-founder and CTO of VAST, is here with us. Alon, thanks for coming back on theCUBE.
Alon Horev
>> Thanks for having me.
John Furrier
>> Great to see you in person. Brendan's usually on, but we got you now, so it's good to get some reps in.
Alon Horev
>> Second best.
John Furrier
>> They're both good. You're both good. That's co-founders. Going back, I kind of gave the history. When you guys started VAST, we were there before you guys launched. We had you launch in our studio. Dave and I were talking about what could be. We were like, this could be big, like really big. And then we debated, can there ever, can there be another AWS? And we were like almost betting, I don't think so. I don't think Dave said that, but I was kind of leaning into, I was leaning into, I don't think they could compete. Okay, then the demand hit. You guys have been there. Set the context because, unstructured data, a lot of people saw it. But then it became a big trend. Talk about that point in time and where you guys are here today.
Alon Horev
>> Sure. First of all, remember that when we started in 2016, AI wasn't a big deal. OpenAI was founded the same month. It was just starting out, right? But we were looking at the trends in the industry where organizations started looking at data for the value and being able to analyze it and process it, right? And there were a few things that were happening in our favor, like the declining prices of flash, the increased density, increased performance. So we didn't intend to go all in for AI. We intended to combine the best of two worlds. We wanted to bring enterprises the best-in-class performance of a truly distributed system, right? And go to the HPC centers and show them that enterprise features like encryption, protecting data, resilience, and uptime, they can enjoy that too. So we started with the HPC community. We wanted to then over time kind of graduate into the enterprise because enterprises need a lot more features. And on our path getting there, suddenly AI became a big thing, right? And AI is, I think, a culmination of those two things at the same time because we're now participating in building supercomputers, right? They're extremely fast, they're extremely big, but they need all of the enterprise functionality, the uptime, and the support as these environments are very expensive.
John Furrier
>> When we had Ian Buck on first to kick off the show, Dave and I were talking. Dave asked me a very specific question about, people don't know that they actually make a lot of CPUs. And then he had a really good point about how we had to build our CPU for our use case of the GPU. He compared it to AWS and said they're just turning on more cores. oversimplified.
Dave Vellante
>> It's brute force versus optimizing for this.
John Furrier
>> And that really speaks to how this developed, this density. You mentioned density was a big part of what NVIDIA was doing. That changed the game. So compute on the surface, was different with NVIDIA. How has storage with VAST changed in that wave? Because you had all the stuff in the enterprise because they need all those things too, but they weren't enterprise. You have large-scale AI factory providers. What's the difference? What was your version of storage?
Alon Horev
>> So I think, first of all, we built software, but we were privileged to be at the right place in time to choose new hardware components that weren't there before. RDMA networking on commodity Ethernet, as an example. So being able to take those components, QLC flash, which is cheaper flash, very dense, very power efficient, very fast networking, allowed us to imagine a new type of architecture, right? And that architecture then is very different than what previous architectures looked like because it enabled us to kind of reach the scale of exabytes, hundreds of thousands of GPUs, and thousands of workloads running at the same time.
Dave Vellante
>> Do you mean by that architecture, you're talking about disaggregated?
Alon Horev
>> Correct.
Dave Vellante
>> Can you, why did it take the industry, 'cause we've been talking about separating compute and storage since the early days. We mentioned Hadoop before. Why did it take the industry so long to get there? What was the enabler? 'Cause you were kind of the first really to do that. There were other attempts. I think Vertica tried it at one point and it was just really awkward.
John Furrier
>> It was early.
Dave Vellante
>> Early. But what was the enabler to do it?
Alon Horev
>> That's a great question. And I think, you mentioned Hadoop earlier, right? We were living in a world where CPUs were much faster than the network.
Dave Vellante
>> Yeah.
Alon Horev
>> In that world where networks were 10 gigabits or 20 gigabit, 25, the workload would move to the data. This is what Spark and all of the entire kind of Hadoop ecosystem did as an optimization for how to make compute scale.
Dave Vellante
>> Right. It was profound at the time.
Alon Horev
>> And networks became fast enough to kind of liberate the workload, the compute to go anywhere as long as it's on the same fabric. And these very fast fabrics, initially it was, not necessarily easy to consume by the enterprise using InfiniBand. But today with Ethernet, you can see massive, massive networks and we're all the way up to 16 gigabits, right, per second. So it feels to me like the network kind of caught up in terms of the pace and the performance compared to compute.
Dave Vellante
>> So the interesting thing to me about VAST is you've created a category that is still hard to nail down. You're not a storage company, right? But you're not a database company, but you've got a database, file system, data platform. But people think data platform, they think Snowflake, they think Databricks. Okay, you're some of that. You're a little bit of storage.
John Furrier
>> Data operating system.
Dave Vellante
>> You're an AI company, right? So this is sort of this new category. How do you think about it?
Alon Horev
>> So we focus on AI infrastructure. And our perspective is that in 6 months from now, or 12 months from now, there's going to be a new service that we're going to need to build because of the pace that this industry is moving, right? And what's interesting is that once you build a good foundation with a good architecture and you can add more services on top, it made it easy for us to basically come in and after we built a data store for unstructured data, add the database for structured data or an event stream for real-time inference. So the way I see it is that we keep discovering with our customers, the model builders, the neoclouds challenges as they come up. And it's a fascinating world just because it's moving so fast.
Dave Vellante
>> Yeah, it's interesting. Your DNA is notLLMs, you know what I mean by that?
Alon Horev
>> We all come from the software world.
Dave Vellante
>> It's funny, you all do come from that world, but you speak AI very fluently.
John Furrier
>> Dave and I talk about the pod all the time, but one thing we were talking about, I think 5 pods ago, Dave, was you brought up bottlenecks and I wrote a post around the bottleneck clock And it's all about memory. But you have different access ways. Inference is different than training, RAG or whatever. But agent memory is a big discussion. So memory is HBM, whatever. But you're in storage, you're storing data. Yeah. You have the system. Everything's getting closer to the system around the core GPU, XPU. What's that change for you guys? Because now the bottlenecks just move to the next point, right?
Alon Horev
>> Correct.
John Furrier
>> So how are you changing your architecture to handle some of these new requirements for agent memory?
Alon Horev
>> Yeah, yeah.
John Furrier
>> People think, oh, throw more SRAM at it, throw more HBM at it. Not necessarily. You all try to get as much as you can, but memory-first architectures is the number one thing we're hearing in the semi world.
Alon Horev
>> Yeah. So first of all, I think we need to—memory is a loaded term, right? Memory, when we refer to HBM in the context of inference, right, we can talk about how many agentic sessions you can hold and serve at the same time, right? Memory for agents is a bit different. First of all, there are multiple types of memory. There's long-term memory where an agent can see past conversations and past interactions and look back and learn from its past experiences. There's short-term memory, which is I'm actively working right now with a customer, who's the customer, what industry they're in, what other customers are seeing that's perhaps similar. So there's different types of memory. That memory doesn't necessarily need to be in RAM or SRAM. We're talking about kind of the context or the scope of an agentic session of what pieces of information it can stitch together in order to achieve its goal, right? And when we look at that piece of data, we ask ourselves, is it big? Do we have something to contribute there? And I think the answer is yes because when you imagine agents at the enterprise, there needs to be some form of knowledge base, communal knowledge, knowledge that's shared across those agents, right?
John Furrier
>> Alon, talk about the— let's unpack disaggregated serving for a second. One of our hot topics, we love talking about it, but it really solves the KV cache problem. So explain KV cache as a storage problem.
Dave Vellante
>> Sure.
John Furrier
>> Explain why that's—
Alon Horev
>> So let's describe what KV cache is, right? Like the process of inference, right? When we go back to the first interactions we had with models, we would ask a question and get a response, right? This turned into what's called multi-turn conversations where it goes on and on. We provide more questions, follow-up questions, right? The same thing happens today with agents. They go on and on until they achieve their goal or they get tired or they run out of tokens, right? That session, that conversation occupies space in GPU memory, right? The reason that it needs lots of memory is that one of the first things that an LLM does is calculate the relationship between the words. It's called attention, right? And that attention calculation, that process is called prefill. And once I load the prompt, I start with my question. As the LLM tries to understand that question, it creates the relationships between those words. That's a very compute-intensive process and it takes a bunch of RAM, right? So if I take a long session, not necessarily a human conversation, maybe it's an agent that's working on a project, maybe it's coding something, it could have half a million tokens, half a million words sit there in the session memory and that would occupy roughly, a 20th or even 10th of the entire GPU's memory. So now when you think about how do I optimize inference, how can I serve lots of concurrent users and sessions, right? It's also possible the agent would stop talking to the LLM because it's compiling code, it's testing software, or, as a human, I want to have a cup of coffee. Do I abandon this information? If we do, because we don't have enough memory, right? The end user comes back, the agent comes back, and you need to waste CPU, GPU cycles.
Dave Vellante
>> Start all over again.
Alon Horev
>> Do it all over again, right? So what you see is that if you could stretch that memory wall and basically offload those sessions to storage, you can avoid that recalculation.
John Furrier
>> So KVCache handles this persistent memory opportunity.
Alon Horev
>> Yes.
John Furrier
>> But you can't just have all these things hanging out there. Active, not active.
Dave Vellante
>> You've got to flush it because you're running out of space. It's a valuable resource. When NVIDIA announced, sorry, STX.
Alon Horev
>> Yes.
Dave Vellante
>> I think it was at GTC in March, right?
Alon Horev
>> I think it was even before that.
Dave Vellante
>> DC last year.
Alon Horev
>> CES, I think.
Dave Vellante
>> Actually, I wrote about it. It was March. It was March. Well, you guys probably had advanced notice of it, but when they announced it, I said, oh my God, this is so aligned with VAST. This is perfect. Many storage companies I think had to probably scramble and maybe think about re-architecting or align to it. Can you explain how your sort of philosophy and architecture just in your AI nativeness sort of aligned with this new storage hierarchy, which is emerging.
Alon Horev
>> So first of all, the optimal approach to solve for KVCache is to leverage GPU memory as efficiently as possible, then leverage the CPU memory that's local to the same machines, and then offload to some kind of media, some kind of persistent media that can give you petabytes of KVCache, right? And NVIDIA's take, they're contributing a lot to Dynamo, which they started up as the software stack to orchestrate all of that, right? You mentioned disaggregated prefill and decode. Once you look at inference as a distributed problem where you have the opportunity to use GPU memory, CPU memory, and VAST across a fleet of machines, you have more optionality and you have more optimized scheduling, right?
John Furrier
>> Yeah.
Alon Horev
>> You can move a session from one busy GPU to one less busy GPU and move KV cache either over the network or read it from VAST.
John Furrier
>> And that's optimizing the system to have better performance, better latency.
Dave Vellante
>> But the interesting thing there is if you were a supplier of storage and you were tier 1, let's call it, and you were doing transactions, that was tier 1, the high-value layer. NVIDIA now is the tier 1, right? And so it's sort of pushed that— it's expanded that pyramid, if you will. That, I'm sure, didn't bother you. You were like, great, come on in. This is our wheelhouse.
Alon Horev
>> We already have a great partnership on the networking side, right? We have adopted RDMA and BlueField very early, were the first partner to bring BlueField to market, leveraging them for offloads. So being able to use them for more, solutions, and in this case also adopt their DPUs, is an opportunity that we're very aligned with.
Dave Vellante
>> And now BlueField not only offloads, now it's fundamental to what they call scale-in, to sort of agent monitoring security, AI safety, which is obviously a topic that's been a little bit in the news. Yeah.
John Furrier
>> I want to get back to the distributed nature of KVCache because that opens up the direction that I want to talk because we're doing a lot of research on this.
Alon Horev
>> Okay.
John Furrier
>> KVCache, we get how that works. You guys are a nice layer to increase the cache. But everyone was— we've been obsessing over compute with inference and agents, but the big bottleneck shifts to the data movement. Yeah, data movement is the essence because your disaggregated serving is data movement because you're doing it from here to there. KVCache is moving into VAST. Distributed edge is coming. So data movement becomes now the gold standard of value.
Alon Horev
>> That's correct.
John Furrier
>> What's your vision on this? How do you guys look at that as a— how do you solve that bottleneck and cost, quite frankly? But let's talk about bottlenecks first. Cliff?
Alon Horev
>> So I think, we started with a foundation that enabled us to scale a system very big. We're talking about multiple exabytes in a single namespace and, serving the largest GPU clusters in the world. But as you say, agentic AI changes things because you're now talking about deploying thousands and thousands of agents. And those are different. Each and every one of them is an application in itself, right? When you think about optimization globally, you may have GPUs in multiple different places. How can you migrate or schedule an agent at the right place? Today, usually people need to schedule the workload where the data is, right? So our DataSpace functionality allows customers to even run GPU-intensive jobs that are data-intensive by caching data that comes from elsewhere in a different VAST cluster in a different region. When we think about Agentic AI, we put agents at the center. We mentioned memory, we mentioned KVCache. They need to be able to search through enterprise systems for tabular information. They need to perform RAG and search for unstructured data. They need to generate telemetry that needs to be recorded. We need to support all of those modalities of data in order to actually support that kind of workload. When you look at what the incident with OpenAI and Hugging Face, I was really impressed to see they had all the information to be able to explain what happened, right? And this notion of explainability or traceability, we see it being critical for two things. One of them, any enterprise that's going to allow agents to operate on sensitive data, to talk to customers or do actions on their behalf, needs to record everything and they need to retain it for a certain amount of time, right? The other thing is that these conversations that the agent is doing, it's also gold. It's the same information that it can use for fine-tuning or training or creating purpose-built models. That are going to be more optimized and would drive better outcomes.
John Furrier
>> Alon, great to have you on again. Appreciate you spending the time. Say hello to the team over there. The VASTronauts are doing great. That's the word for the employees. Love that. Next time you're in New York, you have to come down to our NYSE Wired studio, our new brand, theCUBE. Next time you're in New York, come down.
Alon Horev
>> Love that.
John Furrier
>> Ren has been on too.
Alon Horev
>> Thank you so much.
John Furrier
>> All right. VAST Data really may have a rocket ship opportunity. Been following from day one. It's been a great journey of being in the market early, riding the wave, but also having the innovation because the scale, the quality of the data, the quality of the intelligence is going to be grounded in how that data moves into the systems. There's new value in it. We're doing our part here on theCUBE to bring you the data. I'm John Furrier, Dave Vellante. Thanks for watching.
John Furrier
Dave Vellante