theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | Pioneers of Next Gen Datacenter Infra

Pioneers of Next Gen Datacenter Infra

In this interview from theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event, Glean co-founder and CEO Arvind Jain joins theCUBE’s John Furrier to unpack what’s really working in enterprise AI today and what comes next. Jain explains why knowledge access remains the first successful AI use case at scale and how Glean’s enterprise search brings AI into everyday work. He details the past year’s lessons with AI agents – from the need for guardrails, security, evaluation and monitoring to democratizing agent building so business owners (not just data scientists) can create production-grade agents. The conversation dives into Glean’s vision of the enterprise brain powered by an enterprise graph, highlighting the importance of deep context, human workflows and behavior to reduce “noise” and drive outcomes. Jain outlines core building blocks – hundreds of enterprise integrations and a growing actions library – that let agents securely read company knowledge and take actions across systems (e.g., CRM updates, HR tasks, calendar checks). He discusses how organizations are standing up AI Centers of Excellence, prioritizing “top 10–20” agents across functions like engineering, support and sales, and why a horizontal AI data platform that unifies structured and unstructured data – accessed conversationally and stitched together via standards like MCP – sets the foundation for AI factory-scale operations. Looking ahead, Jain says Glean’s upgraded assistant is evolving from reactive tool to proactive companion that anticipates tasks and accelerates productivity.

Share this session

Clips
More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Jason Hoffman

Chief Strategy Officer

Switch

Alexis Crowell

CMO & President of the Americas

Axelera AI

Tom Traugott

SVP Infrastructure

Edgecore Digital Infrastructure

Niklas Leck

CEO

Penguin Group

Michael Jeter

Head of Sales

Global AI

play_circle_outline AI Factories: How Power and Energy Limits Are Reshaping GW-Scale Data Centers

play_circle_outline Supply chain issues: chip availability, NAND/memory price increases

play_circle_outline The Inference Era: Building Distributed Edge AI Factories for a 31x Post-Training Workload

play_circle_outline Software layer and model mobility: orchestration, secure model migration, heterogeneous stacks

play_circle_outline Purpose-built clusters: training vs inference optimized architectures and optical/memory pooling

play_circle_outline Memory and data movement becoming dominant constraint, not raw compute

play_circle_outline Physics vs software: energy and physics constraints drive architectural innovation and efficiency gains

play_circle_outline Capital intensity: trillions needed; financing constrained by counterparty and credit risk

play_circle_outline Market opportunities across value chain: energy, cooling, hard assets, verticalized regulated solutions

play_circle_outline Enterprise transformation: AI replaces many traditional apps; rearchitecting ERP and workflows

Info
Transcript

Pioneers of Next Gen Datacenter Infra

Jason Hoffman

Chief Strategy Officer Switch

Alexis Crowell

CMO & President of the Americas Axelera AI

Tom Traugott

SVP Infrastructure Edgecore Digital Infrastructure

Niklas Leck

CEO Penguin Group

Michael Jeter

Head of Sales Global AI

In this interview from theCUBE + NYSE Wired: AI Factories — Data Centers of the Future, Alexis Crowell, CMO and president of Americas at Axelera, joins Niklas Leck, chief executive officer of Penguin Group, Tom Traugott, senior vice president of emerging technologies at EdgeCore, Jason Jeter, head of sales at GlobalAI, and Jason Hoffman, chief strategy officer at Switch, to talk with theCUBE's John Furrier about how AI factories are transforming data centers from passive warehouses into high-complexity industrial facilities on par with semiconductor fabs. Hof... Read more

explore Keep Exploring

What are the main topics and trends expected to dominate GTC? add

How should the next-generation data center be defined in light of current supply‑chain constraints and rising memory (NAND) prices, and do you expect the industry to expand or grow? add

Why focus on inference—particularly high-performance, low-power inference deployed at the edge—rather than on training? add

Will AI compute be centralized in the cloud or distributed to devices and the edge (i.e., a hybrid approach), and what infrastructure and challenges—such as heterogeneous architectures, a unifying software layer, secure model migration, data movement, security, and bandwidth—need to be addressed? add

How is AI compute infrastructure evolving with respect to training versus inference, and what technologies are enabling low-latency inference? add

What are the current and near-term hardware and energy bottlenecks for AI systems, and how will those constraints differ between GPU-bound training/chatbot workloads and emerging CPU-bound agent-based AI? add

Will scaling AI compute by many orders of magnitude require a proportional increase in energy (are there physical limits), or can hardware, software, algorithmic, and utilization improvements prevent that? add

Besides technical limits like memory and power, what are the main obstacles to building the physical AI “factories,” and how much capital will that require? add

Where in the AI value chain is value being created and what types of businesses or assets present the best opportunities for entrepreneurs and investors? add

How will much larger AI models disrupt enterprise software and the public markets — which parts of the stack will be replaced, and which companies are likely to dominate the different roles? add

bolt Powered by CUBE AI

Pioneers of Next Gen Datacenter Infra

search

John Furrier

>> Welcome back everyone to theCUBE Studio here for our special Sunday edition pre-GTC NVIDIA event preview content. Experts coming together here in the studio as part of the NYSE Wired and theCUBE programs. Got the pioneers of the next gen data center. We've got an expert panel here going to break it down. But first, I want to give a shout-out to Guidepoint for kicking in for the party tonight and sponsoring theCUBE. Thank you very much. 20 years in the business. Great research. So shout out to the sponsors. We got Alexis back, CMO and President of Americas for Axelera. Great to see you again.

Alexis Crowell

>> Thank you.

John Furrier

>> All right. Niklas, CEO Penguin Group. Tom with EdgeCore, SVP of Emerging Technologies. Jason Jeter, Head of Sales, GlobalAI. Jason Hoffman, Chief Strategy Officer, Switch. Wow, what a panel. The biggest panel I've done here in the studio. But this topic is the hottest topic. The AI infrastructure build out is going crazy. We're seeing the demand curve off the charts. The chips are hard to get. Memory prices are up. Power is in demand. You guys are the center of the action, so thanks for spending the time. I guess I'll start with Jason, you've seen the movie of the big metro areas with Switch. Now, everybody's pumping out data centers. They're trying to sell that capacity to the demand, the Googles, Meta. Everyone wants compute, enterprises using the hyperscalers. What is the state-of-the-art problem that we're going to see at NVIDIA on the front stage with Jensen this week?

Jason Hoffman

>> Oh, that's a great question. I think it's going to be two parts. I mean, so one is when it comes to data centers, they're transitioning from sort of a warehouse type design to much more of a factory type design with factory type outputs and everything else. But the last 12 to 18 months, most of the constraints has been around power. And these factories are relatively straightforward and then electricity comes in and now tokens come out, so that's a much better measurement in many ways than just percent CPU or how many VMs you're running or anything else in there. So I expect a lot of the infrastructure plus energy being a very big topic in GTC.

John Furrier

>> And the AI factories thing in particular is the topic of this event, but that's been two years now since Jensen uttered the phrase AI factories, kind of putting a name to large scale clusters, not just your standard data center rack and stack, but like a system architecture was now two years in. That's center of the action.

Jason Hoffman

>> Yeah. And if you look at, I mean a 2025 training cluster max size was maybe 100 megawatts. If you look at the target training clusters now, they're half a gigawatt to a gigawatt in size. It's a single cluster. And so you have literally a 20 to 100 extensification of what happens on a rack perspective and a 10x extensification of what people want inside of a cluster. And then yeah, the other topic's going to be about inference, because that's the big transition that's happening now as these things go. Vera Rubin in a lot of ways is going to be a very sort of enterprise targeted inference platform, in sort of my opinion. Those are the two big ones. There's on one hand, the sort of constraint model around how electricity gets turned into tokens. And on the other one, it's basically all these things are going into production now. And so, inference .

John Furrier

>> And how would you define the next gen data center given the supply chain, prices on memory? I mean, I was just looking at NAND prices are off the charts. I mean, prices are going to go up a lot of the vendors, but also you got to be on the right side of the supply chain and get the architecture right.

Jason Hoffman

>> Yeah.

John Furrier

>> And you think that's going to be expanding, or growing?

Jason Hoffman

>> I think we'll see a lot of people in 2027, 2028. We'll see who's capable of building a factory and who's still building hotels and warehouses. I mean, so these are now highly technical buildings that have as much complexity in the inside of them as a nuclear submarine or a nuclear reactor or an aircraft carrier or a semiconductor fab. So they're not just a warehouse that you stick stuff in anymore.

John Furrier

>> Alexis, you're making a chip. You've been on theCUBE with your founder on MWC where there's a lot of talk about hyper-converged and starting to see smaller edge devices and also the demand. The chips are the secret sauce too in this. What's your take on the AI factory wave?

Alexis Crowell

>> Well, I think your point on inference is spot on, right? I mean, we talk about training because training is a workload, right? That is a thing people have to go do to build these models. But if you look at the life cycle of that model, it's 31x on the inference side than it is on the training side, because you constantly run it. And the way that we look at that is that is a tool that's going to underpin your actual business process, which for us is, well, then put it where your business process should live, which is at the edge, meaning you have these AI factories that could be closer to those data points. It could be at your telco. It could be Comcast, Telefónica, you pick your favorite company, but if you run the inference right there, then it's actually run as part of a business process to solve a business problem. So that's what we've been focusing on. We think high performance, low power actually is the way to solve for that extra 31x of the model's life cycle.

John Furrier

>> It's interesting when you bring on the edge, it's data centers, plural.

Alexis Crowell

>> It is.

John Furrier

>> Because now, you have multiple connections. Tom, the edge has come up. It's in the name, EdgeCore. You're emerging markets. The distributed computing paradigm makes the data centers a plural, not a singular.

Tom Traugott

>> Yeah. And I would say, I think our focus has been staying proximate to key population centers and hubs, but given the requests and the demand we've seen from customers, we're very focused on bringing scale with proximity to existing hubs. I think there's a place for your proverbial middle of nowhere, ultra low cost sort of scale, but we really think proximity still to key hubs while trying to replicate some of that scale is going to be important. I think it is the case right now. Inference is wanting any capacity it can get. Right now, there's a sprint. So we haven't really seen, I think, at scale demands for near inference yet, but I think collectively, most folks in the room, we know that technology is always liked speed. And by the time we start to get comparing one gigawatt to another gigawatt, that gigawatt that's closer to a population center is going to be more valuable.

John Furrier

>> Yeah, I think inference is pretty much out there. People see it. Michael, on the GlobalAI side, what do you think about the decade long build out now that we're in this front edge of another expansion?

Michael Jeter

>> Sure, sure. So I think that we are kind of going into an era of refactoring kind of the existing landscape of what data centers are. So I'm old enough to understand and represent and remember back when high performance computing was a big deal. And cold and hot aisle isolation was like the new biggest thing to cram as much density as we possibly can into a data center footprint, right? The amount of compute power that you can get inside a kilowatt. I think now, you're seeing this step function change in how data centers are designed and architected to try to eke out as much density as you possibly can, because the demands for low latency inference is tremendous. Customers expect sub-millisecond response times. And the only way that you get that is by packing as much compute power as you possibly can into a floor tile.

John Furrier

>> Niklas, your global position as well, there's a lot of global sovereign AI discussions. That's kicking up more demand, not just spiking capacity issues.

Niklas Leck

>> Yeah, absolutely. So I think it's an interesting question, right? Do all these future data centers need to be in the United States, for example, to be able to serve the US market? There's certain places in the world that might have not as much of a power constrained situation as the United States today has as Europe, for example, today has. We as Penguin, we have always tried to position ourselves energy first. So where energy capacities are available right now and also into a foreseeable future. That's why we run our data centers today in Paraguay. One of the, I think, interesting places, not only for the sheer amount of available electricity, Paraguay has access to one of the largest energy sources in the world. Itaipu, which is a 14 gigawatts hydroelectric dam, but also it's one of the key allies of the United States in South America. So just this week, I think it was at the Paraguayan Senate has approved a certain agreement between a military agreement between the United States and Paraguay, which is called the Status of Forces Agreement, which essentially allows US military personnel, that's one thing to operate on Paraguayan grounds, but it also allows for certain joint deployments of whatever type of action that they want. And I think data centers and compute will be one of these ... Military will span across much more domains than in the past, and compute will be one of these domains, which is why such partnerships with locations that are very much aligned with the United States, but not positioned in difficult geopolitical regions. For example, such as the Middle East, we've seen first attacks on data centers in the last weeks in the Middle East. So we think that Paraguay is an interesting hub to actually operate US serving capacity.

John Furrier

>> I think the Amazon getting attacked really highlights the need for distributed architecture, which brings up the next, I guess, section I want to ask you guys is, what big change will be in the architecture for this next generation infrastructure build out? Because you have technical discussions, you have power, then you got political power, right? So there's all kinds of new things. And then I'll see with the Amazon example, distributed, decentralized, distributed computing, AI. What does that look like? Is it an extension of our current build out? How do you guys see that? What is the architectural shift?

Alexis Crowell

>> I think it's hybrid, honestly. I think that where you can put your compute in the hands of the people that need it, it will happen. And I think where you have to have that backup or you need the bigger compute power, that will happen as well, and those two things have to talk to each other. That software layer that interconnects them really matters. But if we look back 20 years pre-iPhone, we're following that same cycle. It's just happening with AI and it's happening much faster, but it's that same cycle. More and more compute is going towards your hands, whether it's your hands literally in your hands or your home or your office or your near edge AI factory. And I think that's what needs to happen here, which means we have to think through the entire compute stack of not only what is the heterogeneous architecture that's going to connect those pieces, but what's that software layer that's going to allow people to move and migrate their models, right? Because you don't necessarily want to move all the data, because that costs an even bigger issue. And then we got security, bandwidth, et cetera. But how do you move that model in a secure way that allows you to operate irrespective of what might be happening around you?

John Furrier

>> Well, we know from the DoD discussions, you don't want to be relying on one model. So again, what is the architecture? Anyone want to weigh in on this? It's kind of a high level question, but I mean, you're looking at a distributed computing, decentralized, global, distributed.

Jason Hoffman

>> I mean, we have existing paradigms for this, I mean, which is exactly how the internet happened and exactly how mobile happened. And if you sort of reframe the AI part as largely a person communicating with an AI or an AI communicating with an AI, then you can just sort of tack it onto the other communication revolutions or evolutions that have basically gone and happened. And so we see right now is we see a lot of AI infrastructure that looks like a very early ARPANET or a very centralized sort of core routing infrastructure. And then it'll basically start scaling out to resemble much more of the internet, which is a global platform that's in there. And then you'll have another variation of it that looks like mobile networks, which mobile networks are factually throughout the world, sovereign infrastructure. And so very often a country has the ability to turn a mobile network on and off. They have regulatory intercept capabilities. You treat a mobile network like you would treat a strategic petroleum reserve or sort of anything else. And then, of course, mobile devices themselves are going to be edge inference devices. I mean, we already have NPUs on these and everything else in there. And so we can just expect that the stuff that's going on with the AI infrastructure now just looks like early internet and the mobile network sort of equivalent hasn't happened and it'll just follow-

John Furrier

>> It's like dial up.

Jason Hoffman

>> Yeah, yeah. Yeah. I mean, you just go interact with ChatGPT and you sit there and you watch it respond and take 29 minutes to hack something out to you, then it is very much like dial-up. Yeah.

John Furrier

>> Yeah. I like that. I like that tweak.

Michael Jeter

>> No, I was going to say, you're also starting to see this bifurcation of kind of purpose-built clusters around the infrastructure layer, right? Where I think prior to 2024, it's like you talk about 2023, '24, it was all about training, right? How could we train our large language models? How could we do fine-tuning? I think we've entered the inference era clearly. And I think there was a statistic that I recently heard like 80% of all compute power will be kind of earmarked and focused on inferencing going forward. And you're starting to see that with how we construct, build, and operate clusters, where clusters are purpose built for inferencing or purpose built for training. And I think that there's technologies that are coming that are going to assist with that. You have optical switching, you have store memory pools that are going to be focused really on, like I said, getting that sub millisecond latency so that customers get that response time.

Jason Hoffman

>> Well, and it comes down to sort of where the constraint shifts a bit, because on the training side, we've basically maxed out. If you look at the constraint for the next couple of years, it's going to be memory. I mean, if we look at just the basic energy usage of like a subtract inside of your code, it's like 0.5 picojoules. It's sort of the bottom of the physics. Moving into memory is a 1,000 picojoules and then you continue having inefficiencies above it. But we're also moving into this realm where you have a memory constraint that's in there. And then in the case of training, everything happens on a GPU and the CPU that's sitting there is functioning as an orchestrator to do things on the GPU. That's valid for chatbots and reasoning and the way we've been using it, but now that we're sort of making a transition to agent-based things where there's tool hauling and so on, agent-based AI is CPU bound, it's not GPU bound. And so we'll see exactly as the kind of serialization that happens on CPUs and actually having sufficient memory there to hold the model into memory and taking things all the way down to like one bit style inference model so it sort of packs in. But we have exactly this thing where we just-

John Furrier

>> It's like a memory manager. You need to write a new memory swapping out.

Jason Hoffman

>> Yep. Yeah, yeah.

John Furrier

>> So do you think the change then is an architecture is really about more about physics, less about software? How would you ...

Jason Hoffman

>> Yeah, physics is the constraint. I mean, if you look at it ...

John Furrier

>> Because you got energy, cooling.

Jason Hoffman

>> Well, a CPU, a GPU at the end of the day is just doing physics when you think about it and memory there is doing physics. And so there's always going to be physics constraints around how much energy does it take to go do a given function in software. Now, the good news is that a million times more compute doesn't require a million times more energy because when you look at what happens historically over every decade is you have software improvements plus hardware improvements plus algorithmic improvements, plus orchestration improvements, plus utilization improvements. And so you can go and get a million times more compute, but you're only going to need 10 times more power to accomplish that because the other 100,000 sort of improvements that come out is a 50x improvement in compute, a 40x improvement in algorithms, a 20x improvement in software, a 15x improvement in utilization. And so you have this constant discounting in it. We see that, for example, in the reduction in the cost of inference. I mean, if you look at a model like ChatGPT 3.5, if it was being done on a modern TPU today, well, back then it was literally 380 times more expensive to do inference when it was out versus sort of now. And so we see these dramatic reductions in the cost of that. It just is evidence of hardware, software utilization, these things getting better. If we brute forced ChatGPT on circa January 2023 technology, we wouldn't see anything that we see right now. There's been dramatic improvements the last few years in those areas.

Alexis Crowell

>> I think though there is an architectural approach that could change, right? There's kind of three ways to build silicon right now. And the way that a couple of companies, Axelera included, has chosen to do it is we put the memory in with the compute, right? And the more you can minimize that data movement, meaning you do the fundamental compute right next to where you're storing your weights, the faster that is and the less bound you become, right? Now, we're putting 630 tops in a 35 watt chip. So are we solving GPT-4? No, we're not. But if you've got a 7 to a 70 billion parameter model, you can have that running right next to you and you can still do all of those things because the architecture has fundamentally changed. So I think to your point on, yes, it is a physics problem, but it's a physics problem that requires some invention on how you manipulate and where you put those different components.

Jason Hoffman

>> Yeah, it's a memory constraint on the next couple of years. It's not a compute constraint technically.

Tom Traugott

>> If I were to roll up, I think the answer for the foreseeable future next 5 to 10 years, it's more compute. It's of every sort and stripe, the seemingly irrational behavior of hyperscalers renting from neoclouds and the like. I think just shines a bright light that what we're short of is sufficient compute. We're not optimizing right now. There will be evolutions and optimizing of the technology that-

John Furrier

>> Do you the architecture will evolve to manage the constraints, memory, power?

Tom Traugott

>> I think when we catch up with what the demands are seeking perhaps, but really, the other topic we haven't touched on yet is for all the physics and all the physical reality of bringing these AI factories forward. One of the biggest log jams is capital, right? So we're seeking capital sources from all manners of debt and equity sources around the world, because the quantum of capital required to build all of this is staggering. It's bigger than anything else.

John Furrier

>> It's almost funny you say that, because the average person would say that there's so much capital being poured into AI infrastructure. You're suggesting there's even a whole nother level.

Tom Traugott

>> Yeah. Pick your analyst.

John Furrier

>> A reactor, a turbine, I mean building-

Tom Traugott

>> Is it three trillion, five trillion, seven trillion? It's kind of the numbers bandied around by 23rd.

John Furrier

>> Well, I mean, Jason was talking about these paradigms that the reference on like telco, I mean that build out was trillions. So you look at the total CapEx just in the loan.

Jason Hoffman

>> The comparables is, I mean, roughly two and a half trillion dollars was spent globally rolling out 5G. One and a half trillion dollars was spent rolling out LTE. That's on the infrastructure side. If you then go on the other side, you ask yourself a core question of what was spent making iPhones readily available, what was spent making Samsung smartphones readily available, that number doubles again. And so I agree with you, there's sort of constraints, but I think the largest constraint tends to be like around counterparty risk and the credit worthiness of who the capital goes to. And you have a number of people out there, whether it be certain aspects of private sort of neocloud type companies or things that aren't the top sort of three frontier labs or so on, but they're having to have fun things in the most expensive way, which is through equity. If we look at like our business, which happens to be on the other side of it, we find capital readily available, but everything that we do ends up being investment grade things that exit out to that sort of market. And if you're sitting there helping hyperscalers go and deploy infrastructure that fits perfectly fine within their balance sheet and their capital structures, and the result is investment grade infrastructure that you'd basically need anyway, there's readily available with that.

John Furrier

>> Let's talk about the market structure because you mentioned ARPANET and ARPANET didn't really have a lot of competitors. They didn't have a lot of people with intentions and legacy business models. I mean, you look at Amazon and hyperscalers, they behave differently than the folks building out data centers who like look on a longer time horizon. So there seems to be, "Hey, I want to buy a lot of demand from the hyperscalers, but they don't want to write 10-year contracts." So to your point about capital and the economics, like you got a market selection issue.

Tom Traugott

>> The lending community loves investment grade, blue chip tech, and in that neocloud category emerging player, if they can get a parent guarantee or credit enhancement from one of those big tech firms or chip providers.

John Furrier

>> Like NVIDIA.

Tom Traugott

>> Yep. Then that's the magic word and that unblocks a tremendous amount of capital. I think there's some nuances that we see about as big tech as nimble as some of these upstarts in terms of securing capacity, securing power, that's kind of its own topic, but they have surprisingly demonstrated a right to win and leasing. IREN, Nscale, CoreWeave. When I first heard of them leasing to a hyperscaler, I remember scratching my head thinking, "Yeah, that doesn't sound right."

John Furrier

>> Alexis, you worked for a company that had the cadence of long game thinking. NVIDIA has got that long game. What we're talking about here is a cultural shift from like the software days and the tech days of, just say cloud computing. I just wrote a post this morning and said the build out on the cloud was a software revolution. DevOps, that was great, but now AI is more of an industrial. We're talking about like wiring up turbines, building out facilities. This kind of has an industrial vibe to it. So I think that-

Jason Hoffman

>> Maybe you could argue cloud was the same. I mean, there are no cloud providers that started as startups. Every dominant cloud provider today, right now, Amazon, Microsoft, Google had existing balance sheets, existing lines of revenue and so on like that, that weren't cloud. So nobody in the startup space won the cloud either. I mean, that was a very ...

John Furrier

>> OpenStack.

Jason Hoffman

>> What's that?

John Furrier

>> Remember the OpenStack days?

Jason Hoffman

>> Yeah. Yeah.

John Furrier

>> Yeah. They tried, but Amazon was coming in.

Jason Hoffman

>> Yeah.

John Furrier

>> So what's the Amazon equivalent, NVIDIA? I mean, NVIDIA is making the market right now.

Jason Hoffman

>> No, the Amazon equivalent is Amazon, and the Microsoft equivalent is Microsoft, and the Google equivalents Google. I mean, candidly, for me, AI is like a orthogonal workload class to that kind of infrastructure. And so folks that already have an industrialized infrastructure approach is going to continue doing an industrialized infrastructure. When somebody goes to an Amazon and spends a dollar on a GPU, they're spending $3 to $10 on other things. Your data pipeline's larger, your consumption of S3 is larger, your development infrastructure is larger, your consumption of like broader analytic storage services is larger. And so AI just happens to be sort of the dominant workload class that drives that.

John Furrier

>> So then I guess the question on that, if you believe that to be true, which I think you're right on that, is that the demand they're seeing, let's take AWS or Google, the demand that they have, their data centers are full, so they need more. So I guess the question is, if you had to go five years out, would most workloads be running on the hyperscaler, specialty clouds or enterprise owned AI factories? Who picks up the workloads? Or is it spread around, or is it Amazon?

Jason Hoffman

>> Yeah. These things always end up having some sort of logistics function. And so in the early days, it's whoever has the capacity. The neoclouds exist because there's been an infrastructure scarcity arbitrage. If that continues, then there's going to be an infrastructure scarcity arbitrage. If we end up having plenty of infrastructure in 2030, 2032, then people whose business models depend on a scarcity arbitrage model might be challenged in that kind of world in there. But if you look at sort of the steady state, the steady sale will probably map out previous steady states. I mean, the hyperscalers were not 58% of all CPU purchases, but now they're 58% of all CPU purchases. And so they may be a dominant sort of amount of AI infrastructure right now, but as an OpenAI builds more of their own things, or Anthropic builds more of their own things or Vera Rubin gets deployed in the enterprise or sort of everything else, it'll probably, when it plateaus out, will settle into the same type of industry ratios we see. You can probably expect that-

John Furrier

>> The rich get richer?

Jason Hoffman

>> 55% to 65%, 70% of the infrastructure is going to be with hyperscalers and the other part's going to be happy with them.

John Furrier

>> You see a power law. You see a power law.

Jason Hoffman

>> Yeah, it's the same.

John Furrier

>> All right, if given that discussion we had earlier about the chips coming out and the demand and some of the numbers you mentioned about the telco build out there, trillions are coming. Sovereign cloud, the people are going to be building. So the question is, will people still be building and to what scale? Because if you take that forward, then the power law is pretty big. Everyone's going to participate. Yeah, the big guys get bigger because they bring stuff to the table, but I'm trying to figure out where is that piece.

Jason Hoffman

>> I mean, there's a lot of AI demand. And so if you look at now with agents and everything else, there's a lot of demand. And if you look at how those models have to be better, they have to actually be 10 million to 100 million times better. And so if they need to be 10 million times better in the near future, like by 2030, 2032, and that's what everyone's growth is dependent on, they have to be 10 million times better and a million times of it's going to come from hardware, software utilization, algorithmic improvements, then the core side of the infrastructure is we have to grow about 10X. In the data center market, that means that we sort of came into 2023 with about a 16 gigawatt footprint in the United States and about four gigawatts internationally, so 20 total. And most people's plans when you sort of look out into the 2030s is that the data center space will expand 10x because that's our part of the overall 10 million time sort of growth. So if we continue with the last 10 years and the 10 years before and the 10 years before it of hardware, software algorithmic and utilization improvements, then our part still has to go from basically 20 to 200.

Alexis Crowell

>> I don't think it's an either or. I think there's an element that says, at the end of the day, this is going to become a commercial conversation around what makes the best TCO solution candidly. We're in the ramp phase where it's hard to see past that. It's hard to see to the steady state that you're talking about. But I think as you get into that, it's going to become not just a matter of what can you get your hands on, which is kind of what's happening today, right? You put your compute wherever you can because that's what you have access to. People are going to make the decisions based on where do I need to protect my data? Where do I need to run? How fast latency wise? Not just do I need access to it immediately, but how fast do I need that response because I'm doing something that has some sort of human response or human interaction? I need to stop that robot from doing something that it shouldn't be doing. But that is when you get towards, you kind of get past the bubble of what everyone's chasing right at the moment, right? So probably 2030, 2032, 2035, after that kind of 10-year full on dilution, I think people put compute every single place that they can, same way we've been doing for the last 50 years, candidly, right? We just keep packing more and more into it. And I think there's some really great inventions and innovations happening with, whether it's using sparsity, whether it's trying to map how your brain functions, your brain runs on 20 watts, right? I mean, that to me is still super fascinating and we can't get computers to that yet. So there's paths there, right? And I think as more and more of those start to come to play and we get out of the scarcity model that we're in right now that's got people freaked out, you kind of get into that more of the abundance thinking and how do we access it, then it changes. And I think things will start to even out across all of the different nodes of compute that will be capable.

John Furrier

>> So much bigger, almost like the grid computing days, but like for compute.

Jason Hoffman

>> Oh, if we go back to your point around capital availability and everything else, and yours of how long do we basically have? I mean, we were able to respond really well the last couple of years because our CEO was designing gigawatt campuses 14 years ago and we started our first sort of gigawatt campuses in 2014, 2015. And so doing a gigawatt build out today is not that big of a deal, but it takes-

John Furrier

>> Or maybe for you guys, but there's not a lot of gigawatt-

Jason Hoffman

>> No, but it takes 7 to 10 years to build out transmission. It takes about five to eight years to build out distribution and put generation in place. And so we actually have to have 10, 15, 20 year plans, even particularly when we have hyperscale customers that have 20-year contracts. And so if you look and you say from now until say 2036, the next decade, there is sufficient capital in that and sufficient demand and everything else. Then in our space, we can take the data center market from 20 gigawatts to 200 gigawatts. That's what we can do. And we can signal to everybody else based on all the constraints and everything else and building out transmission, distribution, power generation, sites and everything and the capital available and this and that, I can tell people 200 gigawatts can go be built. Based on that, if you want 10 million times more, you got to make yourself a million times better, but you're only going to get 10 times more capacity basically in 10 to 15 years.

John Furrier

>> The realities kick in.

Jason Hoffman

>> Yeah. I mean, we're talking about 200 gigawatts is like the same global ... I mean, it's how much air conditioning and refrigeration is globally today. It's a lot of power. It's nine Californias of power, but that's what can be built with money available and the way supply chains will look over the next decade and so on like that. I know on the data center side, we can 10x the footprint there.

John Furrier

>> Tom, you're smiling. You're smirking.

Tom Traugott

>> I think it's 50-50. I don't think the enterprise side, I think both the capital and frankly, expertise is going to get sucked up by hyperscalers and neoclouds delivering, and I think the reason why-

John Furrier

>> What did you say?

Tom Traugott

>> 50-50 between hyperscalers and neoclouds. I think neoclouds, most projections I see-

John Furrier

>> Well, at the enterprise....

Tom Traugott

>> generally, assume-

John Furrier

>> But enterprise pick up some of their own?

Tom Traugott

>> I think they end up using. I think they sleeve through hyperscalers and neoclouds. And most projections, most analysts say hyperscalers come back, they kind of reclaim the throne on the AI front, but I think the biggest reason why I don't see them doing that is because they're not all playing nicely with Jensen.

Jason Hoffman

>> Yeah, but they have 100% now.

Tom Traugott

>> Yeah.

Jason Hoffman

>> And so-

Tom Traugott

>> Who's they?

Jason Hoffman

>> Hyperscalers and neoclouds. I mean, if you look at real training workloads and inference and everything else, it is 100% on hyperscalers and neoclouds.

Tom Traugott

>> But what I'm saying is between those two, I think most assumptions get hyperscalers get back to 70%, 80% of the market neoclouds. Yeah.

Alexis Crowell

>> But if that were the only option, my company wouldn't have the growth or the visibility into all of the customers that we have, right? Because it's not just a neocloud or a hyperscaler. There's all of these companies that are trying to take control of their own data, whether it's on prem or it's in their cameras or whatever else that I think is driving a different view. Like I said, I don't think it's either or, because if you look at the problem from one side, absolutely, data centers solve everything. If you look at the problem from the edge, okay, maybe the edge solves 80% of it. There is a hybrid solution that means both have to coexist, right? You have to be able to play nicely together. You have to be able to have a software layer that allows you to move, not only your data, but your algorithms. And at the end of the day, all of those things can and will coexist because there is no scenario, none of the analysts are looking at any of the requirements going down, right? Every single one of them has it expanding. It's up and to the right.

John Furrier

>> Change in footprint, like size, but not-

Alexis Crowell

>> For sure.

Jason Hoffman

>> Well, I'll say, I mean, look, you and I started talking about cloud 20 years ago. And 20 years ago, most of what we sort of said then was what 2015 was. I think right now, they have 100% of AI in the hyperscale and neocloud. And I'll say here a decade from now in 2035, and you go and you look at where AI is running, about 60% of it is going to be on hyperscalers and I'll lump neoclouds in there because any neocloud that survives has to become a hyperscaler, but 60% will be on hyperscalers, and 40% is going to be in enterprise and other deployments and everything else. That's how it'll settle out, because for literally the last 500 years, that's how it settled out. I mean, there were only-

John Furrier

>> The 10-year prediction here at theCUBE Studio. .

Jason Hoffman

>> No, if you want to go communications, you can go all the way back to the printing press and the Gutenberg Bible. It started in 50 sites and then everyone got a printing press, but it took 200 years, et cetera, et cetera.

John Furrier

>> Yeah. I think the hybrid makes the distributed computing discussion more relevant, because the superpowers will have the power. We'll get the market share. I would agree with that. But I think that as the footprint of a data center, what we call a data center might change. Remember the old telephone closet days, now you can put a DGX box and they could do stuff. So that kind of changes that piece. But the factor on the cloud and neo scale I want to ask this question. I mean, think about the data centers in the past decade, you had people who were trying to sell to Amazon or turn it into a business. So they're already servicing the hyperscalers. So do you see neoclouds becoming specialized or doing arbitrage? Because right now, they can just open up their doors and saying, "Meta, Oracle, Google, Microsoft, what do you want?"

Jason Hoffman

>> I think by 2028, we'll start seeing some stuff by 2030, we'll see which neoclouds become hyperscalers or not.

John Furrier

>> Well, I think they're kind of already serving the hyperscalers, but the question is, do they become service providers? I might go to-

Jason Hoffman

>> I mean, they're serving hyperscalers, because NVIDIA is limiting their own GPU allocations, and so they're buying a GPU allocation through essentially another company.

John Furrier

>> Do you see specialist clouds like, "Hey, this is the best legal cloud, the best healthcare neocloud?" That's a riskier bet, because they'll take the money off the table with the hyperscalers.

Michael Jeter

>> I think you do see that start to emerge down the road. I think that there's going to be some demands. I think you mentioned it, the latency and being able to stream real-time data and be able to analyze it near to the edge is going to become a requirement. And I don't see the hyperscalers being able to address that market really effectively. Listen, hyperscalers work to the lowest compete denominator, right? They're trying to serve the masses. They don't operate on the edges. Let's just be totally transparent.

Jason Hoffman

>> Well, I mean, I'll challenge you there in that Google operates very well on the edges. I mean, literally having the largest edge infrastructure on the planet when you look at YouTube serving and sort of everything else. And if these models go from text to actually being multimodal and doing video and so on like that, you could argue that Google is a hyperscaler is an exceptionally good spot from sort of an edge inference standpoint when it comes to multimodal workloads.

Niklas Leck

>> But what we see, for example, is that the neoclouds, for example, the ones that we talk to, they seem to be just more agile. So it seems that they're willing to take more risk, be it risk on location, be it risk to go into certain new domains, be it risk to build a certain purpose built data center much more than the hyperscaler. So we think they will for sure continue to capture significant parts of the market, but of course, let's say the hyperscalers that we know, they might absorb them at some point in the future, so they just might become part of them. But whoever, some of them also just might become hyperscalers themselves if they pick the right opportunities today.

John Furrier

>> And NVIDIA is highly motivated to incent speed and agility.

Jason Hoffman

>> Yeah, absolutely.

John Furrier

>> And use their money and products-

Niklas Leck

>> Very much incentivized....

John Furrier

>> to make the market go a little faster.

Jason Hoffman

>> Well, and he said the keyword, which was risk. And so, yeah, neoclouds are using equity because they can't use debt because they're higher risk. They're taking higher risk to go build things and they're selling them to sort of higher risk clients. They're ones that break through like a CoreWeave, the majority of their business actually comes from the literally AAA credit rated Microsoft, and so that-

John Furrier

>> And NVIDIA.

Jason Hoffman

>> And then NVIDIA is basically backstopping some different things like that, so that's sort of exceptionally helpful, but it comes down to exactly the risk profile of it, both technical, credit wise, counterparty risk, everything else. And, yeah, people right now, half the market is sort of what you'd call speculative venture. There's a lot of equity, there's investment going into it. The other half sort of this investment grade, hyper scaly enterprise type sort of build out. And, yeah, that market there, it's a venture market, so people should venture.

John Furrier

>> Okay. I guess the question to you guys is, where's the value being created? Where's the opportunity for value creation for either entrepreneurial activity or business value? We see the mid-market, we've seen some examples where the mid-market is underserved because they can't get stuff. So where's the value get created as these AI factories come out, edge factories. If this happens, it should be computing, heterogeneous, pick your metaphor. I think we'd agree that's probably going to happen. Where's the value get created?

Niklas Leck

>> So I would say generally the value is created across all the value chain. So in the past we had software is eating the world, now we have AI eating the world. So it will probably eat the world, which means that anybody who's entrepreneurially venturing into that space has a chance at least to take a piece of that super large pie which is currently being happening. That starts with people that just build super basic infrastructure. It starts with the transmission lines and with energy generation, but it goes all across the five layers cake as Jensen Huang calls it.

John Furrier

>> It's going to be hard to start a data center business when you know everything's going to go to the hyperscalers.

Niklas Leck

>> Yeah, but you could start to build electricity assets, for example. You don't have to be a hyperscaler yourself, but you can serve the ones that build data center infrastructure for the hyperscaler.

John Furrier

>> A lot of white space you're saying.

Alexis Crowell

>> Well, it goes back to you have to solve one of two problems. You either have to help a company make money or you have to help them save money. I'm really simplistic when it comes to this, right?

John Furrier

>> I love that.

Alexis Crowell

>> If you can find a pocket that you can do one of those two things better than the other guys can. Actually, there is space where a lot of folks are willing to take risks and they're willing to bet on someone that they might not know quite as well if you have a value prop that, again, solves one of those two fundamental requirements in this market.

Niklas Leck

>> Cooling, for example, is interesting I think as well. So back in the day, we had drag densities of 10 to 15 kilowatts per rack. Now we see 200 kilowatts per rack. And that means that there's a need for new cooling technologies, new cooling ways in data centers. There might be new startups solving these issues, so there's opportunities all across the chain.

Tom Traugott

>> I think some of the biggest opportunities end up being the hard assets business, which is data centers and AI factories is a big moat. It's big dollars, big capital investment, lots of debt and equity that you need to raise and the right profile. But what that enables up the stack on the vertical side, a friend who has an AI company and they're applying models, but they laser focus on RIAs and back office. You can do so much more with these hard assets that are in place. I think the moat, it's pretty steep to come up with a brand new data center company right now, unless you're really bringing credible capital.

John Furrier

>> The edge has got a huge opportunity. Any other thoughts?

Michael Jeter

>> Yeah. I think the only other thing that I'll add is that I agree with you that the hyperscalers would definitely suck most of the air out of the room. But I think the one thing that we're kind of all struggling with is the hyperscalers are also struggling with serving the market. They only serve about 10% of the total addressable market for AI infrastructure that's out there. So that means that there's a tremendous amount of white space that lives below that line. And I think that there's a number of, call them neoclouds, call them AI factories, call them edge, AI factories, specialized AI factories that are going to live in that space below the line that are going to extract a tremendous amount of value out of the market.

Jason Hoffman

>> Well, and I think it's the big disruption we're seeing in the public markets right now is, of course, around enterprise software companies. And so the fundamental question I always ask is, if for all the development you're doing or the application that you're using, if the model gets 10 times bigger, does all this still matter? And so when you look at things where companies that are doing things around data, data exposure, all the data stuff is going to matter very much in there. But the part that gets entirely sort of swapped out is traditionally I go and I write an application that has all of my business logic in it, which means it has all of my business reasoning in it, and then it goes and accesses my data. Well, what happens when you sort of sit down and you say, "A model on the fly, given access to my data with MCP, can better reason through my business problems than this static application that gets redone every 18 months or every five years or so on like that." And so the big opportunity is you sort of sit down and say the hyperscalers are going to do what they do from an infrastructure standpoint. In the frontier models, it's pretty clear to me that ChatGPT is going to be a dominant consumer play. Anthropic can be a dominant enterprise play. Google is going to be Google, but they're also leading very much when it comes to the use of scientific and engineering sort of workloads and multimodal style workloads. So they're going to continue doing there. Those three are really going to split the market amongst that. Then when you get on top of it and you say, "Okay, well, now that I have MCP facing my data and now that I have this, now that I have that," then the whole thing that gets redone is the entire enterprise software stack in there. I mean, we've jokingly, because I have parts of the CIO's organization with sort of me as well too. And we sit around there and we've done stuff like taking a traditional Salesforce CRM type implementation that we have a one-year project to go do things on. We put MCP in front of our ERP and then we just let Claude Code go and there's the dashboard, it's done. You can ask questions and sort of here or there and the Salesforce can go in a chat and do these types of things and none of it required this sort of static application writing. And so the big opportunities in it is going to be exactly that sort of complete turnover of the enterprise application stack. And that's why the market is responding the way it is.

Alexis Crowell

>> Yeah, exactly.

John Furrier

>> Obsolescence on one hand and new way to do things on the other.

Alexis Crowell

>> Well, and maybe within that, one piece that we haven't touched on is where is the regulation on some of the more regulated industries, right? Medical, you kind of hit on, is there a verticalized solution, whether it's the neoclouds or someone else, but not only that enterprise stack, but then how are you doing the ... Ensuring that you're abiding by all of those regulatory requirements and financials, medical, whatever else, ends up being, I think, a completely new vector that allows people that find new value based.

John Furrier

>> Like you said, your simple formula hits there. If I can do something better, faster, reduce the step it takes, cheaper, why not? That's obsolescent. And the other method becomes obsolete. I think that's the way to look at it, because it's not the company goes out of business. Now, Salesforce can certainly become something new if they get tooling and do the right things on that scenario.

Jason Hoffman

>> But when you see that, like in the AI space, we see it within all the tooling in the AI space. The hot thing right now is context engineering. A year from now, it's not going to matter. A year ago is prompt engineering, now it doesn't matter. And so there's always these constant things where people go and look at the state of a frontier model right now and then build all this tooling and everything else around it. The frontier model gets better that all that goes away. And then you saw this and then all that goes away. And so you also see a lot of these things where depending on the state of the current frontier model, there's this, "Let's go ahead and compensate for what the frontier model can ... Now, it's better." Okay. So you also have these things where every time-

John Furrier

>> Yeah. It's evolution.

Jason Hoffman

>> Yeah. Every time an Anthropic or a Gemini or ChatGPT does a release, they basically put probably 100 to 1,000 startups out of space that we're trying to compensate for the way the model was a year ago. And then for us on the enterprise, and I'm talking now from an enterprise application standpoint, for us, it's sitting there and saying that anything that has to have a deterministic output where you go and you calculate a number, you continue writing that in software. That software might be executed by an agent that goes and gives you your payroll number on there, but then there's a whole lot of things around our business that we just want to reason around. And so it's this right balance in a company where you're like, "Here's my probabilistic reasoning engine that I can sort of go and work with." And then, "Oh, I need to completely redo my software stack to be purely deterministic around calculating this type of math and I got to redo my data exposure." And it's that right mix of like how your data's architected, how your data's exposed, how you're using a probabilistic model in sort of the right way, and how are you sort of capturing these permanent calculations you have to do as a business, accounting stuff and everything else and sort of that software. And I say this internal at Switch. Now, just when we look at our ERP stack, our software stack, everything we've done internally, we've completely redone it in the last year and continue just getting faster and faster at redoing parts of it and everything else.

Tom Traugott

>> And underpinning each level of that going deeper is token explosion, right?

Jason Hoffman

>> Yeah, yeah, absolutely.

Tom Traugott

>> That outpaces the efficiency that we're generating tokens more efficiently, but we're outpacing that in terms of the-

John Furrier

>> I think we'll broke it down and put it back together. AI factories, the demand for what we just rift on is the future and it's going to need the infrastructure to do it. So thank you guys so much for contributing to that awesome conversation.

Tom Traugott

>> Yeah, thank you.

John Furrier

>> Almost 45, 50 minutes, went by too fast.

Tom Traugott

>> Right. Good. Yeah.

John Furrier

>> All right. Well, thank you so much. I'm John Furrier. I want to thank Guidepoint for supporting theCUBE and the NYSE Wired. Thanks for watching.