theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | Bora Goekbora, BCG

Clips
More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Bora Goekbora

Managing Director & Partner

BCG

play_circle_outline AI infrastructure wave and distributed computing paradigm

play_circle_outline Investor underwriting challenges around high-performance GPU residual value

play_circle_outline Telecoms retrofitting cabinets/central offices for distributed inference workloads

play_circle_outline Observability, orchestration, digital twins for workload inventory and optimization

Info

Bora Goekbora, BCG

Bora Goekbora

Managing Director & Partner BCG

search

John Furrier

>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, host of theCUBE, here with Jay Mollave, my co-host. Hello, I'm John Furrier, host of theCUBE, here at theCUBE's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. It's our NYSE Wired program and the open community of leaders Participating in the revolution of AI infrastructure. This is our AI Factory series. We got a great guest here from Boston Consulting Group, Bora Goekbora, managing director and partner at Boston Consulting Group, or BCG. We're not going to talk about the strategy work that BCG is well known for. He's an expert in the edge and what's going on in the telecom and the infrastructure. Bora, thanks for coming in. Appreciate it.

Bora Goekbora

>> John, it's a pleasure. Thanks for having me.

John Furrier

>> We were talking before we came on about many things that you have insight into. I want to unpack it. But I first want to get your take on the AI infrastructure wave. What's happening? We all know the AI factories are booming. It's our most popular series, but the edge is on the horizon. NVIDIA's telegraphing with 6G. It's just a natural progression. We're in a distributed computing paradigm. We've seen this movie before. Yes, big node, little node, edge node, human nodes. You're going to have this AI network effect.

Bora Goekbora

>> I think

John Furrier

>> so.

>> you're doing a lot of digging into that. What's happening?

Bora Goekbora

>> Right. I think, John, what's been fascinating to see is this is really the evolution of a space that's been in front of us the whole time. I think, traditionally this has been communications infrastructure. This stack has always existed. This is a traditional IT stack, but there's now just so much emphasis on the GPU and the compute layer of that same stack. And so what's happening is we're seeing a ton of attention around high-performance compute. Obviously, investors are seeking how do I underwrite it? What's the longevity of these chips? There are also investors that have been in communications and digital infrastructure in some cases for decades now. And so this is now the natural next asset class that they're looking into. But the characteristics of it are very different than the characteristics of some of what they've invested in the past, things like cell towers, conduit, fiber networks, etc. So the underwriting framework is something that they're deeply, deeply looking into, again, especially about residual value of these high-performance chips.

John Furrier

>> Yeah.

Bora Goekbora

>> On the other end, we have telecoms and the incumbents of this infrastructure layer that are dealing with things like interest rates and balance sheet strain and generally flattening growth because of how competitive things like broadband have become. So now more than ever, there's this tension of how, how do those players express their scarcity and value as investors are looking to continue to enter the space? So that's really what I've been working on lately.

John Furrier

>> I mean, it's fascinating that you— the way you describe it, because you have the physical plant or the physical aspect of things, you have the communication stack you mentioned. That's essentially software now with AI. It's interesting because now that's coming together. It's not like siloed, right? They're integrated. I have to ask this question because one of the things we're seeing on the AI infrastructure side and even in the generational shift is older folks that have been working on hard problems have an awakening because we see NVIDIA and the supercomputing capability, high performance clusters providing massive value to actually solve problems that couldn't be solved before, just by a function of evolution.

Bora Goekbora

>> Right.

John Furrier

>> So now we're in an era where I talk to folks, hey, when I was in grad school in the '80s or '90s, I did that. Now I'm actually doing it because I can do it. Right. So things are more attainable. So the question is, what's the new attainability factor that's coming out of some of these markets? Because they've been around, they've deployed physical plants, telecom, communications. It's been around, right? Is there a new refresh, refactoring kind of mindset where these unsolved problems are being solved?

Bora Goekbora

>> Absolutely. And this is, very much in terms of BCG and my excellent colleagues. we're spending a ton of time with corporations about how to actually embed AI into the work that their companies do, because sure, there's a ton of attention on training and the complexity required for that and the computation and the sophistication of that stack. But it really only makes sense if we see inference come through, which we're seeing. And that inference is getting expressed obviously with prompting of things like ChatGPT, Claude. But really the transformational level for enterprises, how do they embed it into their workflows such that the work is happening, like you said, in ways that it couldn't have happened before or at a much faster pace, at a much higher accuracy rate, etc. So that's really where we're spending a lot of our time with enterprise. The other way to take your question is, what is the new technical capability of things like the edge, and where should the edge sit, and where is this compute really supposed to be? And so we've been spending time there too.

John Furrier

>> What's your vision there? Because the question would be, Bora, what does injecting intelligence at the edge mean? Okay, that's kind of a loose statement, but okay.

Bora Goekbora

>> That's a good question, and for me, because I spend a lot of time with our telecom and cable clients, the edge is not a new word. I'm almost, it's time to really, really, really reinvent it in a way that is probably going to be more built to suit how it's supposed to behave. And so what I mean by that is we've always talked about the edge as really a low latency application or a low latency location where compute should sit such that the end user can perform use cases that are super latency sensitive, like milliseconds is what we're talking about. Autonomous vehicles have been talked about, robotics. What we've really seen though, Waymo, other autonomous vehicle companies, they're doing it all on-prem. So that edge never really worked out, right, in that sense. I think what we're seeing now though is—

John Furrier

>> When you say on-prem, on-prem to a data center or on-prem in the car? On-prem in the vehicle. Yeah, okay, yeah.

Bora Goekbora

>> Yeah, that's where the sensors are. And so the edge is still, it skipped a step. It's actually just going to the device. And so a lot of speculators and observers and folks in innovation are wondering, Is that edge going to converge back into something more in the middle? And so options for that, you've got cell towers, they've got edge businesses at their cabinets, you've got telecoms with their central offices and cable companies with their headends. That's powered real estate. And it's actually some of the most distributed powered real estate we have in the country. There's thousands of them. And so, they're rated up to 1 MW, maybe 2, 3 plus. But there's some retrofitting work that needs to be done. But that seems to me to be an excellent place for distributed inference workloads to be sitting.

John Furrier

>> So if I'm a telco or I'm a carrier or infrastructure provider, my retrofit decision has to factor in economics.

Bora Goekbora

>> Absolutely.

John Furrier

>> Cost.

Bora Goekbora

>> Absolutely.

John Furrier

>> Take us through how you're thinking about that, because it's pretty clear that it's a no-brainer for them to retrofit from a concept standpoint. So you say, hey, you got some dormant power, you got an edge position, What's stopping you? Why not get that intelligence in there? In fact, go faster.

Bora Goekbora

>> Now I understand. And you see them play different ways. Some have said we're just going to sell these assets. And so if you look around the ecosystem, some telecoms have disposed of them and they've given them into hands like private equity investors and they've said this should be shared. We'll do the retrofit and we'll co-locate and we'll actually ring-fence you as a sale-leaseback. It's how they can get structured. The other way we think it may go is The cable companies and telecoms, they may do it themselves. They may inject capital from partners to help with the retrofit, but then they'll have access to compute at that edge. They'll buy those GPUs. The other way they could do it is they could do a powered shell or a powered land, but they definitely have some options.

John Furrier

>> I love that edge story because my story was hyperconverging edge. That was a great example because they are going to the car. That use case, what does a hyperconverged edge look like to you? If it follows the data center pattern, we had storage, network, and compute, they all hyperconverge, now they're unconverging because the systems are looking different, obviously, with the new architecture. What does hyperconverge look like in practice?

Bora Goekbora

>> So for us, when we get questions like this, you really don't see as much replacement right away. You see, like you said, coexistence and then eventual convergence of these different asset types. And so you've got the hyperscale, large facilities, the colo facilities, you'll see an expression of the edge into things like the tower edge, the device edge, and then like what we talked about, more of a regional point of presence edge around the headend. That's going to act as a mesh of different network layers and compute layers that have different properties. Some are closer to the endpoint, but they're also further from the center. So depending on which hops they have to make, they're going to experience latency. They also have different physical form factors. They can only host so much compute. So we're just going to see workload distribution follow suit, right? There's going to be a perfect place for each workload that optimizes cost, location, latency within capacity constraints, and obviously monetary and data sovereignty constraints is another one too.

John Furrier

>> Yeah, you nailed the latency point because if you look at the AI factories, Gen 1, Generation 1 of NVIDIA, InfiniBand, KV cache, NVIDIA Dynamo networking was the key fundamental thing.

Bora Goekbora

>> That's where the bottleneck was.

John Furrier

>> That was the bottleneck, and then KV cache became so important because of the serving of the tokens, pre-fill decode, not to get too technical, but then that cache got fatter. And hey, when inference comes in, now compute comes in, so now you have kind of component subsystems around the massive HPC or cloud system or any AI system. If you take that to the edge, that opens up the idea that, okay, I can disaggregate the infrastructure, and I need networking. So you brought up the service.

Bora Goekbora

>> And fiber is a topic that we've seen actually quite a big surge in the need for not just fiber inside the data center, but also fiber to the data center and the interconnection of data centers. So there's also the concept of clustering, which fiber plays a huge role in because the major constraint has been power and land. We're going to see facilities go deeper and deeper into where costs are appropriate and land is available with power. The thing is, the fiber won't be there. So that mismatch is actually something we're

John Furrier

>> also—

>> So you get the land and power but no fiber.

Bora Goekbora

>> Exactly. And so what we're going to see is a surge. And we're already seeing, we've seen some large deals from Zayo and Verizon. They're connecting these data centers at the cluster. And this is a big business for Lumen as well. That was actually unexpected in B2B fiber.

John Furrier

>> But now there's all these new enterprise companies to scale across, as NVIDIA calls it.

Bora Goekbora

>> And so those data centers are the new enterprise customer. And we're seeing not just new mileage of fiber, but also density of the fiber.

John Furrier

>> Yeah, it's— I want to get to this idea of asset light. It's a term that's been kicked around.

Bora Goekbora

>> Yes.

John Furrier

>> So CapEx build out, trillions of dollars, billions of dollars being spent. And enterprise doesn't mean— JPMorgan's got a $10 billion IT budget. Okay, they're different, but that's IT. Enterprises aren't sitting on a massive financing capability. They want to minimize costs. And so they will work with whatever that data center is. To your point, enterprise is the new data center. Or would you say it's a new enterprise? Okay, because they will use that and create that single tenant experience like we're seeing with tools like Instinct, these new awesome things.

Bora Goekbora

>> Yeah, there's a new part of the stack in managed services, which are basically whether it's on-prem, colo, hyperscale at the edge, there's definitely a growing, opportunity there for investors too in those managed service space because not every enterprise is built to manage the complexity of all this different infrastructure as well. And agentic workloads and high-performing AI workloads are only going to demand more sophistication.

John Furrier

>> And they'll want to plug into that intelligence. Absolutely. And get the tokens like electricity.

Bora Goekbora

>> That's the new commodity.

John Furrier

>> Yeah.

>> Compute. All right. Back to your other example about the edge, because I want to get this out there and help my understanding. I think you've got— we're just kind of talking through and we're riffing on this. If I'm in a metro area, let's just say New York or San Jose, because we had a great demo at GTC around San Jose having a bunch of towers and 6G and all kinds of metro cloud, whatever they want to call it. Okay, I got Homeland Security, I got first responders, critical infrastructure, basic consumer services, and then business. If I'm a carrier, I'm not selling bandwidth in a family plan here. The revenue opportunity, whoever can nail that equation, that's basically quality of service and policy-based management. That's a managed service.

Bora Goekbora

>> Exactly.

John Furrier

>> Why wouldn't I want to be in that business?

Bora Goekbora

>> And Yossi, what's your take on that? Yeah, it's absolutely a required space. The complexity of networking and we didn't even mention satellite, but there's other network layers emerging as well. We're not quite yet at 6G, but still.

John Furrier

>> But is 6G the InfiniBand of wireless?

Bora Goekbora

>> I think there's plenty to accomplish with 5G. And so this is what's called the B2B segment for carriers. And it's certainly a business that, there's complexity in it, but there's huge opportunity to grow. What it requires is they've got to be able to manage the different network layers. In many cases, enterprise doesn't need to know, hey, is it being supported by 5G? Is this a fiberized connection? They just want to know that their use case is going to be supported. And there's also resiliency that they have to manage.

John Furrier

>> Is my latency enabling my workload? Is it secure?

Bora Goekbora

>> Resilient?

John Furrier

>> Diverse? I'm going all day long. I don't care where it comes from.

Bora Goekbora

>> But that complexity is great for infrastructure owners, those that can provide the services layer on top. I'll take it a step further to the workload specifically. The applications in facilities also come with a lot of complexity. Observability is huge right now. Threat detection. we're seeing more than ever the bar is lowering for anything that's AI enabled. So now threats are also multiplying as well. So we're just going to see more and more services like SASE, private networks.

John Furrier

>> You know, I was observing the Nscale investor meeting here a couple of months ago. They bought a company that we've covered since coming out of Cal Berkeley, was Anyscale. Anyscale was acquired by Nscale. They kind of have the same name, but the reason why they bought them, they were vertically integrated NeoCloud or HPC Cloud, whatever you want to call it. But they have, because of this new disaggregation going on, whether it's disaggregated serving or subsystems handling, parts of the workload, orchestration is going on, you need observability. They had to know every piece of resource being touched.

Bora Goekbora

>> It's hard.

John Furrier

>> Up and down the stack. That is extremely difficult. You have to bake that into the stack. That is a really huge, that's a new dynamic we're starting to see.

Bora Goekbora

>> And that's the concept, and a lot of the concepts you and I have talked about, they've been around, but we're seeing a bit more traction than we expected on things like the edge, B2B fiber, but digital twin is another concept you just got into, which is can I understand my inventory and my assets in real time, what's available? What can I sell? Where are there threats? How should I be rerouting or optimizing?

John Furrier

>> Yeah, yeah.

Bora Goekbora

>> Customers are going to be expecting that too, because workloads have to have their cost managed. Every CFO and CTO is battling right now around what's the right answer to get GenAI into the organization, but doing it in an ROI-based way.

John Furrier

>> And so visibility and orchestration are going to have more explainability from a rollback standpoint for Understanding how to handle the resource.

Bora Goekbora

>> Yes.

John Furrier

>> What happened? Auditing.

Bora Goekbora

>> What did I get from it? Am I— is my cost to serve decreasing? My cost per interaction decreasing? I'll tell you a funny story too. I've been experimenting with AI assistants, and what I'm noticing is companies don't necessarily want your agents to call them. They're saying, hey, I'm not ready for that. My handling costs are going to go through the roof. So the agentification in these use cases are moving in that

John Furrier

>> direction.

>> guys must be so busy at BCG because there's so much chaos and it's going to be out there, but it'll be reined in. like you said, there's a lot of concepts. I was just talking with my daughter the other day about, in England they drive on the left side of the road and then you— everywhere else it's right side of the road. That's a governance issue. Okay. Those agents— I can't just have my agent drive on the wrong side of the road. So you're starting to see the sovereignty question become Okay, what are the laws? And if you have agents not liking the laws, then you got the British law and the American law. You don't want to have them make their own rule of law.

Bora Goekbora

>> You have to do it in an AI-responsible way.

John Furrier

>> So you're going to have this, well, agents will just make up their own laws to serve themselves.

Bora Goekbora

>> There'll be guardrails in place. Otherwise we won't be able to adopt them, right?

John Furrier

>> All right. Final question for you. You mentioned earlier a lot of the stuff's been around for a while, computer science, and of course we've seen them. A lot of computer science principles, a lot of networking principles, communications principles, a lot are out there. What do you see as the biggest comeback besides ontologies, which I was just talking about the other day at a graph engineering event. Everyone's like, it's not new. Although Palantir is saying it's new because they want to market it. But there's things like ontologies, a lot of the AI stuff, primitives were nailed in the '80s. Some of the biggest computer science breakthroughs were in the '70s. What is the hottest comeback? Hottest, latest comeback of buzzwords or concepts that are now front and center and making a real impact?

Bora Goekbora

>> Absolutely. I'm going to go back to a topic I touched on a bit, but for a long time, business-to-business fiber and long-haul fiber was seen as a commodity. We're now seeing with how much data center capacity is coming online and also the concept, what you mentioned, of data center interconnect, where you can interconnect facilities to present even larger clusters to your customers. Fiber is arbitraging some of the constraints we've seen in powered land because you're able to interconnect these facilities. And so for me, it's really been about the fiberization of these assets. And for some of our carriers—

John Furrier

>> It's not a commodity when it's a critical path item for turning on a resource that's expensive that produces value.

Bora Goekbora

>> Exactly. And the throughput, the bandwidth required at these sites is more than expected. And so that's been a great growth opportunity.

John Furrier

>> Well, we could go for an hour. Definitely have you back. You're a great, great commentator and also, you know, sharing insights. We haven't even gotten into the whole grid reliability issue.

Bora Goekbora

>> Talk about copper.

John Furrier

>> Copper. We'll come back to copper. I think photonics is going to replace copper in the rack. Maybe it's commoditized back. But for— thank you so much for coming on. Great, great to have you. Great insights. Again, BCG is helping their customers. They're solving problems. This is what they do. This is what everyone's working on. The money and the entrepreneurs and the smart money follow the constraints. And we're seeing a lot of them, once they get taken away, it opens up value creation and extraction. We're doing our part on theCUBE here to share that with you. Thanks for watching.

Bora Goekbora, BCG

search

John Furrier

>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, host of theCUBE, here with Jay Mollave, my co-host. Hello, I'm John Furrier, host of theCUBE, here at theCUBE's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. It's our NYSE Wired program and the open community of leaders Participating in the revolution of AI infrastructure. This is our AI Factory series. We got a great guest here from Boston Consulting Group, Bora Goekbora, managing director and partner at Boston Consulting Group, or BCG. We're not going to talk about the strategy work that BCG is well known for. He's an expert in the edge and what's going on in the telecom and the infrastructure. Bora, thanks for coming in. Appreciate it.

Bora Goekbora

>> John, it's a pleasure. Thanks for having me.

John Furrier

>> We were talking before we came on about many things that you have insight into. I want to unpack it. But I first want to get your take on the AI infrastructure wave. What's happening? We all know the AI factories are booming. It's our most popular series, but the edge is on the horizon. NVIDIA's telegraphing with 6G. It's just a natural progression. We're in a distributed computing paradigm. We've seen this movie before. Yes, big node, little node, edge node, human nodes. You're going to have this AI network effect.

Bora Goekbora

>> I think

John Furrier

>> so.

>> you're doing a lot of digging into that. What's happening?

Bora Goekbora

>> Right. I think, John, what's been fascinating to see is this is really the evolution of a space that's been in front of us the whole time. I think, traditionally this has been communications infrastructure. This stack has always existed. This is a traditional IT stack, but there's now just so much emphasis on the GPU and the compute layer of that same stack. And so what's happening is we're seeing a ton of attention around high-performance compute. Obviously, investors are seeking how do I underwrite it? What's the longevity of these chips? There are also investors that have been in communications and digital infrastructure in some cases for decades now. And so this is now the natural next asset class that they're looking into. But the characteristics of it are very different than the characteristics of some of what they've invested in the past, things like cell towers, conduit, fiber networks, etc. So the underwriting framework is something that they're deeply, deeply looking into, again, especially about residual value of these high-performance chips.

John Furrier

>> Yeah.

Bora Goekbora

>> On the other end, we have telecoms and the incumbents of this infrastructure layer that are dealing with things like interest rates and balance sheet strain and generally flattening growth because of how competitive things like broadband have become. So now more than ever, there's this tension of how, how do those players express their scarcity and value as investors are looking to continue to enter the space? So that's really what I've been working on lately.

John Furrier

>> I mean, it's fascinating that you— the way you describe it, because you have the physical plant or the physical aspect of things, you have the communication stack you mentioned. That's essentially software now with AI. It's interesting because now that's coming together. It's not like siloed, right? They're integrated. I have to ask this question because one of the things we're seeing on the AI infrastructure side and even in the generational shift is older folks that have been working on hard problems have an awakening because we see NVIDIA and the supercomputing capability, high performance clusters providing massive value to actually solve problems that couldn't be solved before, just by a function of evolution.

Bora Goekbora

>> Right.

John Furrier

>> So now we're in an era where I talk to folks, hey, when I was in grad school in the '80s or '90s, I did that. Now I'm actually doing it because I can do it. Right. So things are more attainable. So the question is, what's the new attainability factor that's coming out of some of these markets? Because they've been around, they've deployed physical plants, telecom, communications. It's been around, right? Is there a new refresh, refactoring kind of mindset where these unsolved problems are being solved?

Bora Goekbora

>> Absolutely. And this is, very much in terms of BCG and my excellent colleagues. we're spending a ton of time with corporations about how to actually embed AI into the work that their companies do, because sure, there's a ton of attention on training and the complexity required for that and the computation and the sophistication of that stack. But it really only makes sense if we see inference come through, which we're seeing. And that inference is getting expressed obviously with prompting of things like ChatGPT, Claude. But really the transformational level for enterprises, how do they embed it into their workflows such that the work is happening, like you said, in ways that it couldn't have happened before or at a much faster pace, at a much higher accuracy rate, etc. So that's really where we're spending a lot of our time with enterprise. The other way to take your question is, what is the new technical capability of things like the edge, and where should the edge sit, and where is this compute really supposed to be? And so we've been spending time there too.

John Furrier

>> What's your vision there? Because the question would be, Bora, what does injecting intelligence at the edge mean? Okay, that's kind of a loose statement, but okay.

Bora Goekbora

>> That's a good question, and for me, because I spend a lot of time with our telecom and cable clients, the edge is not a new word. I'm almost, it's time to really, really, really reinvent it in a way that is probably going to be more built to suit how it's supposed to behave. And so what I mean by that is we've always talked about the edge as really a low latency application or a low latency location where compute should sit such that the end user can perform use cases that are super latency sensitive, like milliseconds is what we're talking about. Autonomous vehicles have been talked about, robotics. What we've really seen though, Waymo, other autonomous vehicle companies, they're doing it all on-prem. So that edge never really worked out, right, in that sense. I think what we're seeing now though is—

John Furrier

>> When you say on-prem, on-prem to a data center or on-prem in the car? On-prem in the vehicle. Yeah, okay, yeah.

Bora Goekbora

>> Yeah, that's where the sensors are. And so the edge is still, it skipped a step. It's actually just going to the device. And so a lot of speculators and observers and folks in innovation are wondering, Is that edge going to converge back into something more in the middle? And so options for that, you've got cell towers, they've got edge businesses at their cabinets, you've got telecoms with their central offices and cable companies with their headends. That's powered real estate. And it's actually some of the most distributed powered real estate we have in the country. There's thousands of them. And so, they're rated up to 1 MW, maybe 2, 3 plus. But there's some retrofitting work that needs to be done. But that seems to me to be an excellent place for distributed inference workloads to be sitting.

John Furrier

>> So if I'm a telco or I'm a carrier or infrastructure provider, my retrofit decision has to factor in economics.

Bora Goekbora

>> Absolutely.

John Furrier

>> Cost.

Bora Goekbora

>> Absolutely.

John Furrier

>> Take us through how you're thinking about that, because it's pretty clear that it's a no-brainer for them to retrofit from a concept standpoint. So you say, hey, you got some dormant power, you got an edge position, What's stopping you? Why not get that intelligence in there? In fact, go faster.

Bora Goekbora

>> Now I understand. And you see them play different ways. Some have said we're just going to sell these assets. And so if you look around the ecosystem, some telecoms have disposed of them and they've given them into hands like private equity investors and they've said this should be shared. We'll do the retrofit and we'll co-locate and we'll actually ring-fence you as a sale-leaseback. It's how they can get structured. The other way we think it may go is The cable companies and telecoms, they may do it themselves. They may inject capital from partners to help with the retrofit, but then they'll have access to compute at that edge. They'll buy those GPUs. The other way they could do it is they could do a powered shell or a powered land, but they definitely have some options.

John Furrier

>> I love that edge story because my story was hyperconverging edge. That was a great example because they are going to the car. That use case, what does a hyperconverged edge look like to you? If it follows the data center pattern, we had storage, network, and compute, they all hyperconverge, now they're unconverging because the systems are looking different, obviously, with the new architecture. What does hyperconverge look like in practice?

Bora Goekbora

>> So for us, when we get questions like this, you really don't see as much replacement right away. You see, like you said, coexistence and then eventual convergence of these different asset types. And so you've got the hyperscale, large facilities, the colo facilities, you'll see an expression of the edge into things like the tower edge, the device edge, and then like what we talked about, more of a regional point of presence edge around the headend. That's going to act as a mesh of different network layers and compute layers that have different properties. Some are closer to the endpoint, but they're also further from the center. So depending on which hops they have to make, they're going to experience latency. They also have different physical form factors. They can only host so much compute. So we're just going to see workload distribution follow suit, right? There's going to be a perfect place for each workload that optimizes cost, location, latency within capacity constraints, and obviously monetary and data sovereignty constraints is another one too.

John Furrier

>> Yeah, you nailed the latency point because if you look at the AI factories, Gen 1, Generation 1 of NVIDIA, InfiniBand, KV cache, NVIDIA Dynamo networking was the key fundamental thing.

Bora Goekbora

>> That's where the bottleneck was.

John Furrier

>> That was the bottleneck, and then KV cache became so important because of the serving of the tokens, pre-fill decode, not to get too technical, but then that cache got fatter. And hey, when inference comes in, now compute comes in, so now you have kind of component subsystems around the massive HPC or cloud system or any AI system. If you take that to the edge, that opens up the idea that, okay, I can disaggregate the infrastructure, and I need networking. So you brought up the service.

Bora Goekbora

>> And fiber is a topic that we've seen actually quite a big surge in the need for not just fiber inside the data center, but also fiber to the data center and the interconnection of data centers. So there's also the concept of clustering, which fiber plays a huge role in because the major constraint has been power and land. We're going to see facilities go deeper and deeper into where costs are appropriate and land is available with power. The thing is, the fiber won't be there. So that mismatch is actually something we're

John Furrier

>> also—

>> So you get the land and power but no fiber.

Bora Goekbora

>> Exactly. And so what we're going to see is a surge. And we're already seeing, we've seen some large deals from Zayo and Verizon. They're connecting these data centers at the cluster. And this is a big business for Lumen as well. That was actually unexpected in B2B fiber.

John Furrier

>> But now there's all these new enterprise companies to scale across, as NVIDIA calls it.

Bora Goekbora

>> And so those data centers are the new enterprise customer. And we're seeing not just new mileage of fiber, but also density of the fiber.

John Furrier

>> Yeah, it's— I want to get to this idea of asset light. It's a term that's been kicked around.

Bora Goekbora

>> Yes.

John Furrier

>> So CapEx build out, trillions of dollars, billions of dollars being spent. And enterprise doesn't mean— JPMorgan's got a $10 billion IT budget. Okay, they're different, but that's IT. Enterprises aren't sitting on a massive financing capability. They want to minimize costs. And so they will work with whatever that data center is. To your point, enterprise is the new data center. Or would you say it's a new enterprise? Okay, because they will use that and create that single tenant experience like we're seeing with tools like Instinct, these new awesome things.

Bora Goekbora

>> Yeah, there's a new part of the stack in managed services, which are basically whether it's on-prem, colo, hyperscale at the edge, there's definitely a growing, opportunity there for investors too in those managed service space because not every enterprise is built to manage the complexity of all this different infrastructure as well. And agentic workloads and high-performing AI workloads are only going to demand more sophistication.

John Furrier

>> And they'll want to plug into that intelligence. Absolutely. And get the tokens like electricity.

Bora Goekbora

>> That's the new commodity.

John Furrier

>> Yeah.

>> Compute. All right. Back to your other example about the edge, because I want to get this out there and help my understanding. I think you've got— we're just kind of talking through and we're riffing on this. If I'm in a metro area, let's just say New York or San Jose, because we had a great demo at GTC around San Jose having a bunch of towers and 6G and all kinds of metro cloud, whatever they want to call it. Okay, I got Homeland Security, I got first responders, critical infrastructure, basic consumer services, and then business. If I'm a carrier, I'm not selling bandwidth in a family plan here. The revenue opportunity, whoever can nail that equation, that's basically quality of service and policy-based management. That's a managed service.

Bora Goekbora

>> Exactly.

John Furrier

>> Why wouldn't I want to be in that business?

Bora Goekbora

>> And Yossi, what's your take on that? Yeah, it's absolutely a required space. The complexity of networking and we didn't even mention satellite, but there's other network layers emerging as well. We're not quite yet at 6G, but still.

John Furrier

>> But is 6G the InfiniBand of wireless?

Bora Goekbora

>> I think there's plenty to accomplish with 5G. And so this is what's called the B2B segment for carriers. And it's certainly a business that, there's complexity in it, but there's huge opportunity to grow. What it requires is they've got to be able to manage the different network layers. In many cases, enterprise doesn't need to know, hey, is it being supported by 5G? Is this a fiberized connection? They just want to know that their use case is going to be supported. And there's also resiliency that they have to manage.

John Furrier

>> Is my latency enabling my workload? Is it secure?

Bora Goekbora

>> Resilient?

John Furrier

>> Diverse? I'm going all day long. I don't care where it comes from.

Bora Goekbora

>> But that complexity is great for infrastructure owners, those that can provide the services layer on top. I'll take it a step further to the workload specifically. The applications in facilities also come with a lot of complexity. Observability is huge right now. Threat detection. we're seeing more than ever the bar is lowering for anything that's AI enabled. So now threats are also multiplying as well. So we're just going to see more and more services like SASE, private networks.

John Furrier

>> You know, I was observing the Nscale investor meeting here a couple of months ago. They bought a company that we've covered since coming out of Cal Berkeley, was Anyscale. Anyscale was acquired by Nscale. They kind of have the same name, but the reason why they bought them, they were vertically integrated NeoCloud or HPC Cloud, whatever you want to call it. But they have, because of this new disaggregation going on, whether it's disaggregated serving or subsystems handling, parts of the workload, orchestration is going on, you need observability. They had to know every piece of resource being touched.

Bora Goekbora

>> It's hard.

John Furrier

>> Up and down the stack. That is extremely difficult. You have to bake that into the stack. That is a really huge, that's a new dynamic we're starting to see.

Bora Goekbora

>> And that's the concept, and a lot of the concepts you and I have talked about, they've been around, but we're seeing a bit more traction than we expected on things like the edge, B2B fiber, but digital twin is another concept you just got into, which is can I understand my inventory and my assets in real time, what's available? What can I sell? Where are there threats? How should I be rerouting or optimizing?

John Furrier

>> Yeah, yeah.

Bora Goekbora

>> Customers are going to be expecting that too, because workloads have to have their cost managed. Every CFO and CTO is battling right now around what's the right answer to get GenAI into the organization, but doing it in an ROI-based way.

John Furrier

>> And so visibility and orchestration are going to have more explainability from a rollback standpoint for Understanding how to handle the resource.

Bora Goekbora

>> Yes.

John Furrier

>> What happened? Auditing.

Bora Goekbora

>> What did I get from it? Am I— is my cost to serve decreasing? My cost per interaction decreasing? I'll tell you a funny story too. I've been experimenting with AI assistants, and what I'm noticing is companies don't necessarily want your agents to call them. They're saying, hey, I'm not ready for that. My handling costs are going to go through the roof. So the agentification in these use cases are moving in that

John Furrier

>> direction.

>> guys must be so busy at BCG because there's so much chaos and it's going to be out there, but it'll be reined in. like you said, there's a lot of concepts. I was just talking with my daughter the other day about, in England they drive on the left side of the road and then you— everywhere else it's right side of the road. That's a governance issue. Okay. Those agents— I can't just have my agent drive on the wrong side of the road. So you're starting to see the sovereignty question become Okay, what are the laws? And if you have agents not liking the laws, then you got the British law and the American law. You don't want to have them make their own rule of law.

Bora Goekbora

>> You have to do it in an AI-responsible way.

John Furrier

>> So you're going to have this, well, agents will just make up their own laws to serve themselves.

Bora Goekbora

>> There'll be guardrails in place. Otherwise we won't be able to adopt them, right?

John Furrier

>> All right. Final question for you. You mentioned earlier a lot of the stuff's been around for a while, computer science, and of course we've seen them. A lot of computer science principles, a lot of networking principles, communications principles, a lot are out there. What do you see as the biggest comeback besides ontologies, which I was just talking about the other day at a graph engineering event. Everyone's like, it's not new. Although Palantir is saying it's new because they want to market it. But there's things like ontologies, a lot of the AI stuff, primitives were nailed in the '80s. Some of the biggest computer science breakthroughs were in the '70s. What is the hottest comeback? Hottest, latest comeback of buzzwords or concepts that are now front and center and making a real impact?

Bora Goekbora

>> Absolutely. I'm going to go back to a topic I touched on a bit, but for a long time, business-to-business fiber and long-haul fiber was seen as a commodity. We're now seeing with how much data center capacity is coming online and also the concept, what you mentioned, of data center interconnect, where you can interconnect facilities to present even larger clusters to your customers. Fiber is arbitraging some of the constraints we've seen in powered land because you're able to interconnect these facilities. And so for me, it's really been about the fiberization of these assets. And for some of our carriers—

John Furrier

>> It's not a commodity when it's a critical path item for turning on a resource that's expensive that produces value.

Bora Goekbora

>> Exactly. And the throughput, the bandwidth required at these sites is more than expected. And so that's been a great growth opportunity.

John Furrier

>> Well, we could go for an hour. Definitely have you back. You're a great, great commentator and also, you know, sharing insights. We haven't even gotten into the whole grid reliability issue.

Bora Goekbora

>> Talk about copper.

John Furrier

>> Copper. We'll come back to copper. I think photonics is going to replace copper in the rack. Maybe it's commoditized back. But for— thank you so much for coming on. Great, great to have you. Great insights. Again, BCG is helping their customers. They're solving problems. This is what they do. This is what everyone's working on. The money and the entrepreneurs and the smart money follow the constraints. And we're seeing a lot of them, once they get taken away, it opens up value creation and extraction. We're doing our part on theCUBE here to share that with you. Thanks for watching.