theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | Feeding the Factory: Storage at the Heart of AI

Feeding the Factory: Storage at the Heart of AI

In this interview from theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event, Glean co-founder and CEO Arvind Jain joins theCUBE’s John Furrier to unpack what’s really working in enterprise AI today and what comes next. Jain explains why knowledge access remains the first successful AI use case at scale and how Glean’s enterprise search brings AI into everyday work. He details the past year’s lessons with AI agents – from the need for guardrails, security, evaluation and monitoring to democratizing agent building so business owners (not just data scientists) can create production-grade agents. The conversation dives into Glean’s vision of the enterprise brain powered by an enterprise graph, highlighting the importance of deep context, human workflows and behavior to reduce “noise” and drive outcomes. Jain outlines core building blocks – hundreds of enterprise integrations and a growing actions library – that let agents securely read company knowledge and take actions across systems (e.g., CRM updates, HR tasks, calendar checks). He discusses how organizations are standing up AI Centers of Excellence, prioritizing “top 10–20” agents across functions like engineering, support and sales, and why a horizontal AI data platform that unifies structured and unstructured data – accessed conversationally and stitched together via standards like MCP – sets the foundation for AI factory-scale operations. Looking ahead, Jain says Glean’s upgraded assistant is evolving from reactive tool to proactive companion that anticipates tasks and accelerates productivity.

Share this session

Clips
More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Alex Bouzari

CEO

DDN

Kam Eshghi

Co-Founder & CTO

Lightbits

Hao Zhong

CEO

ScaleFlux

Avi Shetty

Sr. Director—AI Enablement & Partnerships

Solidigm

play_circle_outline Data is central to AI; it's the raw material creating value

play_circle_outline Lightbits Launches Inference-Focused Software-Defined NVMe-over-TCP Storage to Support Large Context Windows and KV Caches

play_circle_outline Power, thermal, and TCO constraints crucial; storage must be energy-efficient to feed GPUs

play_circle_outline Memory hierarchy evolving: HBM, local NVMe (G3), rack SSD (G3.5), network SSD (G4)

play_circle_outline Edge & On-Prem for Low-Latency Real-Time Multimodal Workloads: Move Metadata, Not Data Streams

play_circle_outline Computational Storage Cuts Time-to-First-Token by 280x: How ScaleFlux and Lightbits Offload Work to Accelerate Inference

play_circle_outline Security, air-gapping, and multi-tenancy are major enterprise adoption considerations

play_circle_outline Neo-clouds, hyperscalers, and enterprises will coexist; neoclouds becoming hyperscaler-like

play_circle_outline Orchestration and intelligent prefetching needed to feed GPUs and maximize utilization

Info
Transcript

Feeding the Factory: Storage at the Heart of AI

Alex Bouzari

CEO DDN

Kam Eshghi

Co-Founder & CTO Lightbits

Hao Zhong

CEO ScaleFlux

Avi Shetty

Sr. Director—AI Enablement & Partnerships Solidigm

In this interview from theCUBE and NYSE Wired's AI Factories — Data Centers of the Future, Alex Bouzari, chief executive officer of DDN, joins Hao Zhong of ScaleFlux, Avi Shetty of Solidigm, and Kam Eshghi, co-founder and chief business officer of Lightbits, to talk with theCUBE's Dave Vellante and John Furrier about why storage has become the critical bottleneck — and breakout opportunity — inside the modern AI factory. The panel examines how the shift from training to inference is creating an entirely new memory and storage hierarchy, with flash tiers emerg... Read more

explore Keep Exploring

Does Jensen Huang’s five-layer AI stack (energy, chips, infrastructure, models, applications) omit an essential layer — namely data? add

What is Lightbits, and what products does it offer? add

What role do compute, network, and storage play in AI data centers, and how can storage and thermal innovations (for example, liquid-cooled SSDs) help maximize GPU utilization and lower total cost of ownership? add

How does the shift from training-focused to inference-focused AI change storage and memory requirements, and what new memory/storage hierarchy is needed? add

How should system architectures be designed to efficiently handle multimodal workloads that require large images and video (for example in autonomous driving, robotics, or factory automation), and why might moving metadata rather than the raw media be preferable? add

What does accelerated storage (computational storage/data acceleration) look like, and how does it help speed up AI workloads (for example improving time‑to‑first‑token and enabling very large context windows)? add

What are the performance, cost, and security implications of reducing data movement and running GPU-based AI workloads close to where the data resides (including considerations like multi-tenancy and air-gapped enterprise deployments)? add

By 2030, where will the majority of computing workloads run — on hyperscalers, on neoclouds, or within enterprise data centers? add

How does computational (in‑storage) computing fit into the overall data pipeline, and what roles do lower‑level coding, edge acceleration, and intelligence (e.g., prefetching and data movement decisions) play in maximizing system efficiency? add

bolt Powered by CUBE AI

Feeding the Factory: Storage at the Heart of AI

search

Dave Vellante

>> Hi everybody. Welcome back to our Palo Alto studio. You're watching theCUBE and the NYSE Wired's GTC Pregame. I'm Dave Vellante. I'm here with John Furrier. We got a great panel feeding the AI factory. We're going to talk about storage that's at the heart of AI. Alex Bouzari is here, he's CEO at DDN. Think AI and storage coming together. HPC meets AI. Alex, good to see you again.

Alex Bouzari

>> Good to see you.

Dave Vellante

>> Hao is here from ScaleFlux. They're doing some really interesting things on computational storage. Good to see you again, Hao. Thanks for coming on. Doing some cool things around offloads that we're going to talk about. Avi Shetty is with Solidigm. Everybody wants a piece of Solidigm these days and exploding with NAND, so great to see you again. Thank you. And Kam is here. He's the co-founder and CBO of Lightbits. They do high performance block storaging. We're going to get into some of the inference products that you got.

Kam Eshghi

>> Thanks for having me.

Dave Vellante

>> You're very welcome. Thanks you guys for coming on. So I want to start with Jensen's five layer cake. The bottom here is energy, chips, infrastructure, models, and applications. It kind of misses data, in my opinion.

Alex Bouzari

>> Absolutely.

Dave Vellante

>> It's kind of in there. I guess it's implied, but data is what it's all about.

Alex Bouzari

>> Well, I think, look, Dave, Jensen has been saying ... I mean, at CES, he mentioned it. It is all about data. Because at the end of the day, how do you create value for enterprises, sovereign nations, consumers? It's all about data. Data is a raw material that feeds the engine that creates value. And so data is the most important thing. As far as I'm concerned, you just can't do AI without data. That's why we always say, DDN is data what Nvidia is to compute. You cannot do it without data.

Dave Vellante

>> And Hao, you guys said it upfront, are doing some really interesting things with computational storage. You're doing some offloads, making things more efficient. Explain what computational storage is and what you guys do.

Hao Zhong

>> So essentially it's just a way to move data more efficiently and fasterly. So as Alex just mentioned, you have to move data to create the value. So from this five layer cake, you will see the data is actually very essential across the layers, because without moving the data, you're not able to do the effective computing and create the value for the customers and the users.

Dave Vellante

>> And Avi, everybody's talking about NAND and memory prices and the like. John just wrote a piece just today actually talking about one of the constraints. I know you guys are working on it and you got some props to share with us, but I mean, in the last four months, just demand has exploded. High bandwidth memory, obviously DRAM, NAND as well. What's going on? Can you make more of this stuff, or what?

Avi Shetty

>> No, I'm very glad we're talking about this in this panel as well as the industry's finally come to talk about storage being a very critical factor in the whole AI data pipeline, as well as AI factories. The first wave was all about GPUs, what's my GPU utilization? Then came about, "Hey, how do I maximize? How much HBM do I put?" But essentially, what happens with every memory hierarchy is get the first few nodes get stressed and the next layer or the next node needs to be created, and that's why we are here, and you'll hear a lot of new innovations happening coming out from GTC about this whole new creation of G3.5 layer, which is essentially saying that HBM is not able to fit my context. So I need a next layer of high performing, fast NVMe storage, which is essentially flash being coming to the rescue and solving it. So nothing new from us from a Solidigm standpoint. We've been talking about this for the last several years, and as a result, investing in the right storage solutions and technologies, and you'll hear more of that going forward.

Dave Vellante

>> Okay. Yesterday was Pi Day. So it was the 20th anniversary of S3, kind of put object storage really on the map. But you guys are doing high performance block storage, which is interesting. And you're doing a lot of it, most of it actually is on-prem, which is kind of interesting because people are bringing AI to their data, but tell us about Lightbits.

Kam Eshghi

>> Yeah, that's right. So Lightbits is a startup in this group. So because we're a startup, we're a mature startup, but we have two products. One is a block storage solution, which is software defined. It's based on NVMe over TCP, a protocol that we invented and has been in production for years. We have financial services and e-commerce customers, as well as neoclouds. All of them are building an elastic block storage service that's based on Lightbits. And much of it is on-prem, but of course the CSPs are providing services to their customers. Then at this show, we are announcing a new product, which is designed for inference. And there, it's not a block storage interface, it's different. It's a way to basically extend the ... We're talking about the KV cache. As context windows get bigger and bigger, you need to be able to augment that memory with external storage and have intelligence on top to be able to feed the GPU all the KV cache data that it needs before it actually needs it, and that's our focus.

John Furrier

>> This is John's favorite subject. Yeah. I mean, well, the title of the panel is Feed the AI Factory or Feed the Factory. I call it Feed the Beast. One of the things we're going to see at GTC is kind of, obviously, publicly been talked about, but the Vera Rubin, which is a really nice system, but it's got all that complexity and it's got storage everywhere, storage and networking. It reminds me of kind of dating myself in the '80s when I was in college, all the memory management, storage swapping. You just wanted to get the stuff into the CPU. But now, if you look at the Vera Rubin, these racks, it's essentially the same game. You got to feed the beast, in this case processors in the old days. Now, it's like a series of other components. This makes storage so sticky, because it also changes the definition. So how has storage changed given that there's a lot of storage in networking in these large clusters, in these supercomputers, in these AI factories?

Alex Bouzari

>> So I think storage, historically, I mean, when you go back to the old days, dating ourselves, both of us, I think it was all about just being fairly static, sitting in containers, sitting in facilities, and then being processed. Now, it's all about data. So it's about data mobility. It's about moving from the edge to the data centers to the clouds. And so I think the center of gravity of all of this is shifting from being storage, same as storage, networking, CPU, compute, all that, into being data. So how does data traverse from the edge to the data center to the multi-cloud in order to deliver value to enterprises, to organizations? It's what value creates.

John Furrier

>> Is the challenge storing the data fast enough, or moving the data fast enough?

Alex Bouzari

>> Oh, absolutely about moving the data. I mean, storing the data is the easy part. I mean, we've been storing the data forever. DDN has been doing it forever, exabytes and exabytes. I mean, it's moving the data, which is complex, because you have to move the data reliably. You have to move it cost effectively and you have to move it at scale. And that is the challenge. So that's a challenge that we solve, I think, with other panelists here. We solve it with KV cache acceleration. We solve it with inference by shrinking latency. We solve it by accelerating the model training and so on, so it's all about moving the data.

John Furrier

>> Dave mentioned block versus object. And the old thinking was, "Oh, you got to fit in a category." Not anymore with multimodal, you have to do a lot of different things in these systems. Any other commentary around what's changed and why it's changing?

Avi Shetty

>> I think fundamentally, the AI data center, from my perspective, is a three-legged stool. You have compute, network, and storage. Just having superfast compute without network and storage, kind of feeding the GPU investment, which you have. Let's be real. The most expensive part of that ecosystem or the data center is the GPU. And you want to maximize your GPU utilization and having inefficient networking or storage just doesn't pair well. So as a result, you need something which is scalable, cost-efficient. I think you talked about the TCO angle. And the other thing which Solidigm wants to bring into this discussion is thermals and power. Storage historically throttles when put under sustained workloads. Those were the old traditional ways of air cooling the storage. And as a result, new innovations have to be done. And Solidigm brought in the first world was liquid cooled SSD, which essentially wants the GPU to be fed consistently and for longer by ensuring that the sustained bandwidth is maintained.

Alex Bouzari

>> The power element is very important, as you point out.

Hao Zhong

>> So one thing I'd really like to highlight is the transition from the training focus to inference focus. We know storage is important, but when the AI, out of the way one, we call it training focused, people rarely talk about much about the storage. But now, moving to inference, that's become a huge problem. And that's why, James, in his keynote, he called out, "We need a new memory and a storage hierarchy." And the SSD, the storage is actually a new memory. And remember, there's a new hierarchy that we are talking about. In the past, people are focusing on the HBM and the system DRAM. Right now with the inference, which demand enormous data, you will need multiple layers of the storage. So we call G3, which is a local GPU SSD, G3.5, that's local rack context memory or SSD and the G4, which is the network SSD. So all those layers will need tremendous collaboration between the industry partners.

Dave Vellante

>> So Kam, I want to follow up on that. John and I were at Barcelona, MWC last week. We had a lot of talk about inference at the edge, mini AI factories and cell towers and the like. You're selling to a lot of financial services organizations.

Kam Eshghi

>> That's right.

Dave Vellante

>> So when I think about block storage, we think about transaction data, things like whether it's security or inference at the edge to do fraud detection. Is that what some of your folks are doing? How does the requirements there change? And we've been doing fraud detection for a long time. The timeframes are compressing. What's the state of that today or what are some of the applications that you guys are working on with your company?

Kam Eshghi

>> Yeah. Basically, with our block storage product, anybody who needs transactional workloads, they need very low consistent latency, that's where we shine and that's where we're in today. And our focus is because we're software defined is on a cloud operating model. Software defined everything, we run on standard servers, we basically pulled out storage and we provide a secure and shareable IOPS and capacity across multiple compute nodes. Now, when it comes to inferences, I completely agree with what was said. Storage is no longer some background service. It's actually driving the performance because it's an extension. It's augmenting the memory that's right next to the GPU to be able to support very large context windows and inference workloads, so it's a whole new ballgame now for storage.

Dave Vellante

>> And Alex, we last saw you in St. Louis.

Alex Bouzari

>> Yes, it is.

Dave Vellante

>> At the Supercomputing show, it's kind of your wheelhouse. You're kind of the other end of the spectrum, but you also talk about inference.

Alex Bouzari

>> Oh, absolutely.

Dave Vellante

>> So explain how you get there.

Alex Bouzari

>> So look, you just cannot do AI unless you cover all of it from the model, pre-training, post-training, to the analytics layer, to inference, which is a value creation. How do you gain insight? How do you make decisions that are beneficial to the organization in business terms or in financial terms? So the inference part of it, we think is absolutely critical. You will see some announcements from NVIDIA in the next day or two, which tie around inference. One of the challenges with inference, as was pointed out, is context memory windows need to get longer and longer in order to gain faster insights and deliver that insight cost effectively. So we've been working with NVIDIA and other organizations in order to shrink the footprint that is required from a power consumption standpoint, which is a major problem, as well as improving the economics of inference. Many organizations are delivering services today that tie into inference. The problem is, as organizations and individuals are putting these queries out, what they're finding is that what is being charged, the service, monthly service, annual service and so on, puts them upside down. The economics don't work out. So it's a money drain. So what you have to do is you have to be able to deliver the services that are being facilitated through inference cost effectively, which means shrink the footprint and shrink the cost per token. So that's what we're doing, that's what we're developing. It's all about the services that sit on top of the infrastructure and the data plane and the orchestration layer.

Dave Vellante

>> So I think I see a couple of trends at play here. One is some of the things that Alex just mentioned, and everybody is sort of driving down the cost. That's what you guys do in this industry. We're a deflationary industry, right? But also a big part of that is what's coming out of NVIDIA. I mean, every 12 months, we saw what happened with Moore's Law when we first saw PCs, it's pretty expensive and then just got really cheap. Seems like the same thing is happening with token generation. You've got some props. What is that behind you and how does this fit into reducing the cost per bit, cost per token?

Avi Shetty

>> Yes. So what I have here, Dave, is our 192 layer floating gate, NAND wafer. And the reason I wanted to bring this in is to show the technology advancements. I think everyone sees a wafer, but I wanted to kind of show you where all the dies actually go into. So that's the wafer of our 192 technology layer, but this is the SSD. It's a foldable 122 layer, 122 terabyte SSD with 48 NAND dies. What you see is these packages, there are 48 of them, both front and back. We use an entire wafer. The die is coming out of the entire wafer for our 122 terabyte SSD. Each NAND die.

Dave Vellante

>> 122 terabytes.

Alex Bouzari

>> I want all of your production for the next few years.

Dave Vellante

>> John, do we still have-

Alex Bouzari

>> I want that.

Dave Vellante

>> Do we still have the DDN system in the back room?

John Furrier

>> We had the first object store, these things that's running, I think.

Dave Vellante

>> What was the capacity of that?

John Furrier

>> It was 2 terabyte Seagate for ops.

Alex Bouzari

>> I know. That's amazing.

John Furrier

>> I think it was so heavy.

Dave Vellante

>> Now, you got 122 terabytes in your pocket.

Avi Shetty

>> And why is that important, right? Because in the end, you need scalability, right? We've talked, I think we are going to hear a lot about agentic AI, multimodal. All of those new AI technologies are going to create increasing context windows. Will require real-time retrieval, data retrieval. Will require more states. All of that means you need more storage and you need fast, efficient storage at each of the layers which Hao talked about, G3, G3.5 and 4. And each of those three tiers have different metrics. G3 is all about latency, performance, local. G3.5 is about a mix of density as well as performance. While G4 is all about just density where you need to have a focus on density. As a result, you can scale. So 122 terabyte SSDs, when you stack up with our OEMs, 24 slots, 1U, you now have four petabytes in 1U. And imagine the scalability which you get by having high density NVMe SSDs.

John Furrier

>> And this also, Dave, when we're at MWC, this hyper-converged edge report we put out, this supports where it's going. You could put that in a 1U box, in a cabinet, in a central office, at a tower.

Alex Bouzari

>> Well, the edge is going to be critical. I mean, that's why you need a small footprint. The small footprint is not necessarily just to serve data centers, but it's also to serve the edge. And the edge is going to explode in terms of capabilities and requirements.

John Furrier

>> So you see distributed AI factories?

Alex Bouzari

>> Absolutely.

John Furrier

>> No-brainer?

Alex Bouzari

>> No question. I mean, look at the geopolitical challenges that we're faced with right now. You hit a data center. Well, that data center goes down if it's monolithic and you have gigawatts in there, that's a major problem. It's a distributed world.

John Furrier

>> I think the AWS example from two weeks ago highlights the need for a decentralized infrastructure.

Alex Bouzari

>> It has to be. It has to be. I mean, we're seeing it. We have many very large customers in the Middle East and they're completely rethinking the way they're architecting data centers. So they're moving away from the multi-gigabyte implementations to smaller footprint distributed just because of geopolitical risks.

John Furrier

>> I want to ask you guys a question, because this came up on the earlier panel with the future of the data center. I was kind of dancing around. I just asked the question, "What's the biggest constraint?" And they all said compute, because everyone wants compute. GPU is the compute.

Kam Eshghi

>> You're going to hear storage on this panel.

Dave Vellante

>> And then they said power.

John Furrier

>> Of course, compute there. But with the models growing exponentially, do we hit a data gravity wall? Because this is potentially a silent constraint today, but you guys are doing your job.

Alex Bouzari

>> Well, especially in multimodal.

John Furrier

>> Is the data gravity going to be a problem, or is it a problem?

Dave Vellante

>> You mean because of what Alex was saying before about moving the data is the hardware?

John Furrier

>> Well, the context wins are moving. More data from the frontier models are coming. The edge is coming, so you get the movement story.

Alex Bouzari

>> I mean, also, if you think in terms of multimodal, look, at the end of the day, many use cases will require images and video. Images and video are very heavy from a gravity standpoint. You cannot move it around. I mean, it's not cost-effective. You cannot do these things in real time. So you need different architectures that can handle the problem efficiently. And so it's about moving the metadata around as opposed to moving the data around. That's something that we've been doing for a decade. Move the metadata around, because if you think that the world will require images and video multimodal, which is clearly, clearly does autonomous driving, robots, factory floor automation, all of that. Well, you have to move the metadata, not the data. You cannot move the data.

John Furrier

>> How you're smirking, you want to chime in there. Go ahead.

Hao Zhong

>> That's right. So one thing I want to echo is about acceleration. So everyone knows about accelerated computing, right? And now, the interconnect, accelerated network. So how about accelerated storage?

John Furrier

>> What did that look like?

Hao Zhong

>> So the original idea of a computational storage about to accelerate the storage by offloading or accelerate the processing in the data pipeline. Now, what we are seeing is it's a more broad concept of move data faster, which means we need more capacity of the fresh memory and SSDs, which storing them is doing an amazing job. And what we are doing is we make silicons to accelerate the data movement, the SSD controller chips, the firmware to move that faster. While DDN and Lightbits, they are orchestrating the data move efficiently to move that faster. So I would say the data acceleration is going to be the key for our future AI.

John Furrier

>> If I get this ... Go ahead.

Kam Eshghi

>> Yeah. I was just going to add, I mean, we're partnering with ScaleFlux and on the show we're actually demonstrating that with Lightbits and ScaleFlux, you can actually accelerate the time to first token for one million token context window by 280x as compared to just using the HBM. And we can also go up to 10 million tokens, which is pretty big context window, and we're headed in that direction for many of the workloads. So I absolutely agree. Acceleration adds a lot of that.

Dave Vellante

>> And that's both a performance and a cost implication?

Kam Eshghi

>> Exactly. Yeah. Cost because the GPU doesn't have to wait, it doesn't have to recompute. So you're more efficiently using the GPU, which is the most expensive part of the system. And you're saving power, you're saving costs on the GPUs, and you're getting better latency. So a lot of other advantages come along with that.

Alex Bouzari

>> I think the other part, which Dave, I mean you've been mentioning is security. Security is extremely important. Air gap solutions, multi-tenancy, isolating users, just air gapping things. I mean, at DDN, we've been working on air gap solutions and multi-tenant solutions that solve that particular problem. So one element of it is data gravity, as John mentioned. The other element of it is cost reduction, both in terms of tokenization, GPU utilization, and so on, but then all of this needs to be done in a highly secure way. And I think the industry is starting to realize the importance of security, which enterprise has known forever.

John Furrier

>> And this is interesting full circle. This panel here, we're riffing in real time, but connecting the dots, this is the future. It's not by storage. It's build a system.

Alex Bouzari

>> Exactly.

John Furrier

>> Because what you're getting at here is that there's a role for a storage component of some sort in the architecture.

Alex Bouzari

>> It's data engines, I think. I mean, at the end of the day, you need the data engine that will power the AI economy. It's as simple as that.

John Furrier

>> So if we doing the stack slide that Jensen said, what do we call this thing? Because it's a little bit of everything because it's designed in.

Avi Shetty

>> And John, it's not something out in the future. It's happening right now, right? Jensen's vision about this whole context memory, which is 100% we align with is not something new. Memory hierarchy, memory wall has existed in every architecture we've all been in. Back in the day, it was all CPU. CPU solved it, L1, L2, L3, different caching levels, pre-fetching, tiering. The same concepts, maybe altered a little bit differently, has to be now applied to the GPU pipeline. And that's where I lead AI ecosystems for Solidigm and that's where the ecosystem is a fundamental is going to be a different game changer, because we can't solve this all alone. We need the storage players working seamlessly with the software ISV ecosystem, as well as the OEMs who are actually providing the end system so that the final customer, the neocloud. The end customers of neocloud is not unique. They don't have one workload. They have training customers, they have inference customers and they have RAG based customers. So you need a storage solution which dynamically meets their wearing requirements.

John Furrier

>> So fast-forward to 2030. Okay? We're doing a preview show. Where will the majority of the workloads run? Hyperscaler, neocloud, or enterprise?

Alex Bouzari

>> Yes.

Kam Eshghi

>> Well, all of the above. Whoever has the power.

John Furrier

>> What's the percentages of that?

Alex Bouzari

>> I think it will be distributed in many ways. The enterprises just want the utilization of it. They don't want to necessarily own the data centers. They just want the utilization of the AI business value and financial value at the lowest cost possible in the most flexible manner possible and in the safest way possible. So it will be a distributed grid of data centers on earth, and then there will be satellites. I think people don't realize how satellites can be beneficial. So where the data is, is not important. It's that data needs to be provided to enterprises, nations, and consumers safely, reliably at any scale.

John Furrier

>> So the hyperscalers win? But by the way, neo clouds are hyperscalers. They have to be to survive.

Alex Bouzari

>> They have to be. Absolutely.

Kam Eshghi

>> Yeah, they're becoming. So basically ... Oh, sorry. Go ahead.

Hao Zhong

>> Interestingly, those things are now interleaved. For example, hyperscalers is helping enterprise to localize their service. And on the other hand, the neocloud is working with both. So I view it's a little bit the interleaving way how them to evolve.

Dave Vellante

>> Well, but I mean, the labs that you work with, the financial services institutions you work with, they're not just putting all their data into the cloud.

Alex Bouzari

>> No, it's distributed. They need flexibility.

Dave Vellante

>> Exactly, the way you describe it.

Alex Bouzari

>> Some of it is in neocloud. Some of it is in hyperscalers. Some of it is on site. They need flexibility. They need secure cost-effective flexibility and that's all there is to it.

Dave Vellante

>> I want to throw out a premise that we've been working on in our research organization. If you look at the macro, about 4% of organization's revenue is spent on technology. We think that number is going to explode and it's going to go 10% plus. And specifically, it's going to happen through accessing token generation through APIs, because they want access to intelligence. That is a major shift.

Alex Bouzari

>> Well, that shift, which you're absolutely 100% correct, is a shift from an OpEx problem. I'm spending 4% in order to cost optimize versus a value creation problem. And when you think in terms of value creation, then you want to invest as much as possible to create value. So that 4% is going to increase significantly, because that is going to be a business enabler, that is going to foster innovation, differentiation, competitiveness, new products, new services. And so it's a complete shift from OpEx to value creation.

Dave Vellante

>> And it will allow organizations to scale without labor, at least not as much labor.

Alex Bouzari

>> Well, that's a horrible thought. We will repurpose the labor, doing better things.

Dave Vellante

>> Yes. You're right. I said that for ... Yes, but you'll be able to scale without ... We do so much more with the existing labor and then find .

Alex Bouzari

>> Superpowers. We all get superpowers. AI is giving all of us superpowers.

Dave Vellante

>> And we believe that we'll create new jobs. There'll be some, for sure.

Alex Bouzari

>> Yes. Sure.

Dave Vellante

>> For sure. But I want to go back to the storage hierarchy, because you're right, Avi, there's always been that pyramid, but you as systems companies, Lightbits and DDN, you have to balance that. There's three things that are always immutable in storage. It's got to be lightning fast, rock solid, and dirt cheap. Okay. And you could sort of optimize that within the box in the day, but what you described as a data engine, that's a new architecture that's emerging to feed the beast, as John says. And it's a critical part of the stack.

Alex Bouzari

>> I mean, it's no longer about storage. It's about data. So think of data as raw material. From that raw material, you need to create value. So what do you need to do as a data engine to create value? Well, you need an orchestration layer that is very efficient that ties into a number of services that are serving enterprises, governments, consumers, and then it needs to tie into a data plane that is delivering safely, securely, cost effectively, as you mentioned. So the paradigm is shifting completely. It's no longer about data, it's tied into storage. Data traverses all of it. Data is the center of the AI economy.

Dave Vellante

>> So power and data and compute go in, intelligence comes out-

Alex Bouzari

>> Intelligence comes out....

Dave Vellante

>> in the form of tokens. And then now, organizations are trying to figure out, "Okay, how do we apply this intelligence safely?" I just ran a panel last week of about 23 executives. They're really leaning heavily into AI. There's no question about it. They're being cautious, because they're deploying agents, they're scared of multiple agents and bad things going on. So they're being very, very careful as you would expect they would be, but they can see the potential. And this is why we really truly believe that-

Alex Bouzari

>> Well, you have to make it easy and safe to deploy. So you have to cater to the needs of business and financial stakeholders while ensuring that the CIO does not see it as a threat. Because a CIO, their job is to keep the lights on. If you're introducing something that potentially has business value and financial value, but it's threatening having the lights on, they're not going to embrace it. I mean, we have some enterprise customers and we would ask them, "Why is it that you're not shifting your workloads into neoclouds faster? Why are you relying on hyperscalers that are far, far more expensive?" And they say, "It's very simple. It starts with security. We have a list of questions we ask. And when we ask these questions from neocloud providers, most of the things we ask, which we take for granted, because hyperscalers have been developing them for more than a decade, neocloud providers don't even know what we're talking about, so it scares us. It scares us." We're like, "If they don't even know what we're talking about, we're going to deploy and then the lights on, well, we'll flicker and we cannot accept that." So it's really that.

John Furrier

>> I have to ask you guys a wild card question because this came up at the telco event in Barcelona when I asked, "If you can inject intelligence at the edge, a lot of their world changes. You can collapse spectrum, you can make access easier. A lot of things can be inferred, reasoned, and solved at the edge." So I guess the question, if we're going to accelerate storage in this GTC, we expect to hear a ton about the role of The architecture around the inner circle processing with storage, what would happen if you inject intelligence into the storage layer? What would change? Or is that even possible if you're down at the firmware level? Because they have room to move around. What do you do? What does that look like? What does intelligence inside storage mean?

Hao Zhong

>> That's why we have been envisioned this for a long time. The computational storage, which means the neo storage computing or in storage computing. So essentially, it's just that you pipeline the data movement into different stage. It's not just about XPUs and HBM, but also the network as well as the storage. Every layer of this data pipeline has to accelerate. So that's our view, including the edge. As you know, you cannot just move everything to the cloud. And there need to be intelligence also means acceleration in the edge as well.

John Furrier

>> So it's just lower level coding?

Hao Zhong

>> Yes.

Kam Eshghi

>> Yeah. And that intelligence, part of it is to accelerate the pipeline. And part of it is a layer that is determining, for example, when it comes to storage, what to move and when. For example, the prefetching we were talking about is important to feed the GPU. Before it actually needs something, it's available for it so it's not waiting. So that intelligence layer is also important to maximize the efficiency across the entire system from storage to memory, to GPU.

John Furrier

>> The orchestration. Alex, you were talking about orchestration. Huge opportunity to innovate.

Alex Bouzari

>> Well, the building blocks, look, the building blocks are no longer racks of compute and racks of networking and racks of ... You have to take all of this and co-locate it. I mean, look at what NVIDIA is doing with GPU, TPU, LPU. You're bringing all of it together in order to drive better economics, better efficiency. And so the data part of it, the storage part of it also logically needs to be integrated into the whole thing. And then you can build it out, but you can build it out with a very, very efficient construct that you can scale.

John Furrier

>> I mean, Dave, so much for storage going away 17 years ago.

Dave Vellante

>> I don't ever put again storage.

John Furrier

>> When we started theCUBE, people call it snorage, because it's so boring. Yeah.

Dave Vellante

>> One of the first questions John asked Joe Tucci in theCUBE is, "Why is storage sexy?"

John Furrier

>> I won't say I forgot, but it's always been there. Okay? Networking's always been there, but it lights up value when it's architected properly.

Dave Vellante

>> It's data.

John Furrier

>> When the data engine kicks in.

Dave Vellante

>> Yeah. Go ahead.

Kam Eshghi

>> So John, compute is going to the store, to the data.

Dave Vellante

>> You bring compute because of the graphic.

Kam Eshghi

>> And over time, that's where AI is going to be, where the data is.

Dave Vellante

>> That debate that. David Flynn of Hammerspace trying to say that we can bring the data to the computer. I still haven't figured that out yet, but he's smarter than I am, so we'll see.

John Furrier

>> Well, let's get their opinion.

Dave Vellante

>> I don't know how close you looked at Hammerspace, but that's his wrap is data gravity is a fallacy. We can bring the data to the compute. Maybe it's a metadata.

Alex Bouzari

>> Well, you cannot bring data to the ... It's just not possible. I mean, think of autonomous driving. Think of robots. Not humanoid robots, but just robots in factories, right? Well, how do you do that? You have to capture at the edge in real time, multimodal content. You have to process it, analyze it, and deliver real time feedback. Well, you cannot move the data. There's too much of it. Imagine a world where we now have hundreds of millions of cars on the road, hundreds of millions of robots. Well, you cannot.

Dave Vellante

>> Speed of light is very inconvenient.

Alex Bouzari

>> Yeah. It's a big problem. Just fold the universe, right? Just fold the universe and then it's all good.

John Furrier

>> All right. I have another wildcard question. I know we're getting tight on time, but I want to get out there. The cloud, some will argue, but I said this in my post. Cloud was a big build out and there was other paradigms. We saw telecom, but let's stay with cloud. That was a software revolution. A lot of great things happened there. This revolution doesn't seem to be about software, because the AI is doing a lot of the software. How would you guys categorize the revolution? Is it physics? Is it industrial? What would be the tagline? This revolution is blank. If it's not software, if the AI build out and CapEx continues, which is going to accelerate-

Alex Bouzari

>> This revolution unleashes the imagination of the human race. That's what it does. It's shifting it from the physical world into what we can imagine, because that's what AI is enabling us to do. The superpower is that. What you can imagine, you can now run into the omniverse. You can do all of your what ifs and choose the best path.

John Furrier

>> AI scales intellect.

Alex Bouzari

>> There you go.

John Furrier

>> Anyone else comment on the physics piece, or any of the industrial side because like-

Avi Shetty

>> I think the industrialization of the data centers is kind of ... We've never seen so many data centers getting built since we've seen. And all going back to your previous question of constraint, power. I think all of us have to operate. A data center has to operate at a power budget given to it by its local municipal. A rack has to be operating under a certain power budget. We, the storage, has to operate. And every time the storage takes a watt away from the allocated power is essentially a watt taken away from the GPU. So the storage has to be very efficient so that it meets the compute needs to work synonymously together to create that AI value. I think that's where I would put it as the industrial revolution.

John Furrier

>> Yeah, keep enable.

Avi Shetty

>> Key centers.

Hao Zhong

>> And the key thing is about data. The data creation is now growing much faster than ever. That's why you see the huge deficit between the available infra versus data creation. And then the other thing is the consumption. So both of the data creation and consumption is going to be ... I would predict this biggest problem of our future AI.

Kam Eshghi

>> I think AI is going to be running everywhere, which is a change from today, which primarily in the cloud. You're going to see physical AI. You're going to see wherever the data is. There's going to be intelligence generated around that data. And the infrastructure that supports it needs to be efficient. It's not just enough to be fast, but energy obviously is an issue and efficiency becomes a big challenge.

Alex Bouzari

>> But I think you'll see two things. One, well, NVIDIA and others continue to innovate in GPU. So a rack of GB300s is 160 kilowatts. A rack of Vera Rubin is 250 kilowatts, so that is increasing. But on the inference front, you're going to see smaller and smaller power footprints. So these two worlds will come together. On the one hand, you will have very, very high power consumptive racks for certain use cases. On the other side, you're going to see very low power footprint racks for other use cases. And these all need to coexist together.

John Furrier

>> Sounds Like distributing computing to me, Dave.

Dave Vellante

>> There you go again.

John Furrier

>> We're back to that again.

Dave Vellante

>> And we're assuming that, that rate of progression continues as did more laws. It's continuing at 5x performance, 1/10 the cost, and that's going to drive Jevons paradox will drive 15x demand in a year. And that is .

John Furrier

>> We had another comment on the other panel I'll share because it's kind of funny, but it's not that funny when you think about it. He said, "We need more investment in infrastructure." I'm like, well, if you read the paper, people would be like, "Wait a minute. I think there's a lot going in there."

Alex Bouzari

>> We need value creation.

John Furrier

>> So the point about the-

Alex Bouzari

>> Trillions of dollars invested in AI infrastructure. It's just like we built the roads. Well, now, we need to put cars on those roads in order to deliver value.

John Furrier

>> Exactly. So in context to like at the hype and the investment cycle, it was a $2 billion CapEx just for 5G rollout. If you put it two trillion.

Alex Bouzari

>> Two trillion. Yeah.

John Furrier

>> Two trillion. Yeah, trillion. That's a lot. And so if you put it in perspective, there's still a lot more to do investment-wise.

Alex Bouzari

>> Absolutely.

Kam Eshghi

>> Yeah. And when you look at the different pieces of the puzzle here, on the compute side, it's already accelerating very rapidly. And talking about inference, which is really at the beginning of taking off, there's a lot of custom chips coming on the market that you don't have to use GPUs anymore. You can use much more efficient custom ASICs. So compute is growing very rapidly in terms of capability and performance, but memory is not. And that's where storage comes in, to kind of close the gap between what you need in terms of storing the memory and keeping up with the compute.

Avi Shetty

>> Well, just to add to your point, Jensen at CES talked about the inference context, TAM expansion. This entire market didn't exist, and it's going to be as big as the storage market of today in five to six years. That's the change we're talking about, right? With inference kicking off, there will be a huge storage demand which needs to be at the edge, at the data center, and everywhere in between. And it'll require different solutions. As I think you rightfully said, the solution designed for a data center might not be the same one for a moving edge low edge device or at your local office right behind here.

Alex Bouzari

>> I mean, the world of telcos will be highly enabling to all of this. And the push to 6G, again, is enabling the edge to be AI enabled, which today is not.

Dave Vellante

>> Right. And whether or not the telcos can monetize that, somebody will.

Alex Bouzari

>> Somebody will. Exactly.

Dave Vellante

>> No question about it. And we're talking about 115, 120 trillion dollar global economy. We can't affect that by 10%. I'd be shocked. I mean, it's going to be probably much more than that.

Alex Bouzari

>> Much more.

Dave Vellante

>> So a $10 to $15 trillion build out is warranted. Guys, great panel. Amazing insights. Thank you so much.

Avi Shetty

>> Thank you.

Alex Bouzari

>> Thank you for having us.

Dave Vellante

>> All right. We're unpacking the AI factories, data center of the future. This is the GTC Pregame, Dave Vellante. For John Furrier, you're watching theCUBE and NYSE Wired. Be right back, right after this short break.