theCUBE + NYSE Wired: Mixture of Experts Series | Tom Traugott, Edgecore Digital Infrastructure

Clips
More from theCUBE + NYSE Wired: Mixture of Experts Series

Tom Traugott

SVP Infrastructure

Edgecore Digital Infrastructure

play_circle_outline Introduction of Tom Traugott, SVP of Emerging Technology at EdgeCore Digital Infrastructure.

play_circle_outline Scaling Data Center Infrastructure: The Role of Power Constraints and the Rise of Liquid Cooling in Hyperscale Environments

play_circle_outline Navigating Data Center Economics: The Power and Cooling Imperative in the Hyperscaler vs. Neocloud Competitive Landscape

Info
Transcript

Tom Traugott, Edgecore Digital Infrastructure

Tom Traugott

SVP Infrastructure Edgecore Digital Infrastructure

In this Mixture of Experts segment from theCUBE + NYSE Wired at the New York Stock Exchange, theCUBE’s John Furrier sits down with Tom Traugott, SVP of Emerging Technology at EdgeCore Digital Infrastructure, to unpack how “bounded by power” has become the defining constraint for AI-era data centers. Traugott traces the market shift since ChatGPT’s launch and EdgeCore’s pivot to single-tenant buildings and mega-campuses purpose-built for AI clusters. He details how EdgeCore remastered its flagship Mesa/Phoenix site from six buildings to three – now approaching... Read more

explore Keep Exploring

What is the scarce resource that is currently in high demand across various industries? add

What are the key considerations for modern data centers regarding power, cooling, and infrastructure? add

What are the current trends and strategies in the cloud computing market, particularly regarding new entrants and their ability to compete with established hyperscalers? add

bolt Powered by CUBE AI

Tom Traugott, Edgecore Digital Infrastructure

search

>> .

>> Welcome back, everyone, to the Cube. I'm John Furrier, your host. We are here at our New York Stock Exchange Cube Studio as part of the NYSE Wired program. Got a great guest here who's going to go into all the ins and outs of the infrastructure, physical side of it that's enabling all the generative AI, and all the demand for the horsepower, as everyone's talking about it. And by the way, I will say crypto as well. I mean data, Bitcoin mining and crypto takes a lot of energy too, of course. This is the scarce resource that everyone's looking for right now. Tom Traugott is here, SVP of Emerging Technology at EdgeCore Digital Infrastructure. He is in the real estate business, he's in the data center business. You guys are in the technology business. Boy, the world has changed over the past couple decades and most recently, the past 10, past five, and the past three have been out of control growth.

Tom Traugott

>> It has.

>> Welcome to the Cube.

Tom Traugott

>> Thank you. Excited to be here. And frankly, I'm a New Yorker, as I was mentioning. I've never been on the floor before, but I've been to the New York Stock Exchange data center in Malwa, so I've been under the hood, but not here before.

>> You were helping those guys with a high frequency trade and get that-

Tom Traugott

>> That's right....

>> fiber edge that every packet, speed of light you can squeeze out of the transit.

Tom Traugott

>> And then they got smart and they stopped the cheating. Everyone had the coiled fiber equivalent sizing and distance. So, I been in and around the data center business for over 20 years here. Started in New York and really in the 2000s, the largest data center market in the world was northern New Jersey. And that baton got passed to Northern Virginia and that's never shifted. EdgeCore Digital Infrastructure, founded in 2018, and we're arguably in kind of a version two of the business. I joined from almost five years at AWS on the infrastructure side, getting to see the... Worked on a new regions team. Got to see how the cloud was being made in the sort of early days of every country gets a cloud. And I think in a lot of ways, EdgeCore's focus was to be a platform to be thinking about the scale that the historical cloud computing, that Big Three plus would require and be a provider to them in core markets. And then we work our ways through COVID and supply chain constraints. And 2022, ran a process recognizing that we had invested with the original investors, invested in large tracts of land, entitlements, and power. What our customers wanted was product and scale, particularly in the face of AI, which was a twinkle in the eye, as we were talking to our now financial sponsor, a partners group. So they bought us in early November, 2022 and a couple of weeks later, ChatGPT launches and everything, all the rumors of AI's takeover become front and center.

>> Interesting. Over the years I've had a chance to interview James Hamilton and just recently I was up in Seattle in June with AWS, with Prasad, who oversees all their deployments globally. And he talked about global scale. Because it's Amazon, of course they got global scale. But what I found fascinating... I also talked to one of his direct reports in the DC Summit. What I found fascinating is it's transitioned to a full design, systems design facilities play. This isn't like the old days of hosting, "Hey, I got a building, I got some power. Okay, let's get some racks. Install the cages." I mean, most people think of cages, of course there's cages and racks. But they squeeze everything they can out of it because of the bounded by power issue. Everything's bounded by power.

Tom Traugott

>> Bounded by power.

>> And so, that is a huge constraint. They got to engineer around. How has that changed the business landscape and how are you guys working with that? Because everyone's talking about the same thing.

Tom Traugott

>> Yeah. So we made a pivot in this post-ChatGPT era of really focusing on single-tenant buildings and campuses. And for instance, we had our flagship campuses in Phoenix, Mesa, Arizona. And what was going to be a six building, 20 to 30 megawatt per building campus, we remastered plan to three. And the first building is the smallest. And it's really, it's now basically two buildings. It's almost a hundred megawatts each, in light of we're really reoriented around the needs of AI. And what's been useful has been... I came into the business as a real estate guy that's become a technologist over the years. But I have learned about semiconductors and chips, more in the last-

>> You have to.

Tom Traugott

>> You have to.

>> You had to. Otherwise, you won't have a happy customer, 'cause they're engineering to the chip level.

Tom Traugott

>> Exactly. And we are still digesting that. We want ultimately know how many megawatts or gigawatts that cluster needs. But what seems to be really different today, there's a scale up and scale out dynamic that's happening at the same time.

>> There's a race to the AGI, right?

Tom Traugott

>> Yeah.

>> So I mean, we're seeing all kinds of deals. The Texas deals, Silicon Sands, they call it in the Phoenix area. Everyone's building FABS. More data centers are everywhere. I mean, this is a complete changeover.

Tom Traugott

>> Yeah. .

>> Can you scope the order of magnitude of the demand curve?

Tom Traugott

>> Sure, yeah.

>> Is this crazy?

Tom Traugott

>> I think a useful way of looking at it, so looking at the recent earnings. just did a roll-up of you have almost $800 billion of whether they call it booked revenue or remaining performance obligations across the hyperscalers plus traditional hyperscalers plus core weave. And assuming that's a couple of years of annualized revenue, they don't say, but that easily is 15 to 25 gigawatts of capacity when you kind of strip out the hardware. Which is going to be, that'll be 50 to 60% of the spend. The data center infrastructure that would be required to support all of those booked deals, there's a shortfall. And I think that's been a pretty consistent commentary, seems to be. We'd be earning more if we had more capacity. But then specifically, I think where we hone in and our lens is focused on is NVIDIA. And NVIDIA's march to-

>> The gigawatt data center....

Tom Traugott

>> blowing past 4 trillion to a next .

>> Well, their vision is the gigawatt rack basically.

Tom Traugott

>> That's right. You're right.

>> They want the gigawatt rack.

Tom Traugott

>> They want the gigawatt, they-

>> But this brings in the economics. And I know NVIDIA's going to make some big announcements this next week at Hot Chips in Stanford, a big event going on for insiders. They're doing more to squeeze more performance out of their gear, which speaks to what you're getting at, which is the AI economics are now kicking in. So there's all kinds of new levers, or are there? I mean, that's, I guess, my question. What are the economic levers? Because they're going to keep grinding, make the algorithms better.

>> .

Tom Traugott

>> What NVIDIA has been supremely good at is staying laser-focused on selling chips, and as many as possible. And I think what we've... And it seems to be-

>> At some point you can't stack up the GPUs and the systems without the power.

Tom Traugott

>> Oh, absolutely.

>> That's like a non-starter.

Tom Traugott

>> Yeah, yeah. And they need folks to buy the GPUs and then place them. But if you don't have the energy, data centers without power are warehouses or empty shells. I think what NVIDIA has really pushed really over the last, with the current generation with Blackwell and beyond, we're seeing that... You asked what's different about data centers? Well, the campuses are bigger than they've ever been. And they're also the densification, I'd say a lot of hyperscale market was pretty resistant to direct-to-chip liquid cooling for many years, right? You've got gigawatt fleets, you don't want to retrofit them. But NVIDIA, with their market share effectively said, "If you want to continue with our roadmap, the DLC, direct-to-chip liquid cooling is the way." So we've, among others, made all of our index around that for all of our campuses and that, data center providers need to get the big things right. We need large blocks of power, so scale, supporting the scale out, but also increasingly large blocks of fluid to be able to run through that capacity. And then each customer has a slightly different way, so we try and build to 80 to 90% of the market and customize .

>> Power and cooling has always been talked about, but now it's front and center. I got to ask you about some of these other new clouds 'cause they have to build out too. So, you have many customers. You have the clouds, AWS, Azure, OpenAI, they got huge demand. But then you got CoreWeave, we'll be talking tonight to Vulture. They're building out these neoclouds.

Tom Traugott

>> Yes. We internally worked through, there's this question on the long term. Is there a right to exist, a right to win, vis-a-vis the hyperscalers? And frankly, what we've seen is in some ways, a nimbleness. And starting without the additional skews and the additional services that still need to be cared for, that may be on traditional X86 architectures, they're able to sometimes more nimbly go after, deliver capacity to the underlying frontier models in a simpler manner. Going back to my Amazon days, there's a demand side, there's a revenue forecast that's architected. You talk to Gardner, you talk to IDC, and you build a forecast and then you solve for it. The simplicity of the AI model is sometimes identify capacity, turn around, ask OpenAI, "If I build it, would you buy it?" They say yes, and they do the deal. And that short-cutting of the process I think, has led to some really big wins that are now public.

>> All right. So, what's on your agenda this year, as the market continues to evolve and change too? There's new technologies coming out, new ways to air cool. I've heard some cool things there. But at the end of the day, this is economics. You got to make money.

Tom Traugott

>> Yes.

>> What's on your plan? What's on your agenda for the second half of the year? And what's the forecast in your mind, for 26?

Tom Traugott

>> Yeah, so as I mentioned, our new partners group really aligned with the vision that we needed to make the most out of the campuses we had, so we've been building. And we've capacity delivering. We have leases that were booked a couple of years ago that we're delivering as we speak, and we'll continue to deliver on. We've got new product.

>> But you guys are looking for more space.

Tom Traugott

>> Correct, yeah.

>> It's like you got to get space to do. It's recruiting.

Tom Traugott

>> We need to stay four to five years ahead of pipeline at any given time. So, we've had some big announcements. Louisa County is a 700 acre property and we're doing everything we can to bring that 1.2 gigawatts forward as quickly as possible.

>> You guys are creating the future. You guys are creating the infrastructure for the future.

Tom Traugott

>> .

>> I got to ask about EdgeCore. First of all, I love the name. It's got edge in it.

Tom Traugott

>> It does.

>> And it's before core. Core is core to edge, but edge to core. But it speaks to another trend. I just want to get your reaction. I'm not sure if you're in this business or not or see it, the whole edge regional, these areas used to be base stations and telecom. We're seeing Amazon go into satellite, they're going into more local and regional, not data centers, but almost substation type data center model. I use the word substation as a kind of old term, but sub data center, ways to kind of disaggregate maybe power.

Tom Traugott

>> Yep. I think that speaking with some folks that are playing in the space, particularly in the AI level, so we generally play on the large cluster side of things. Now because we're located in close proximity to metro, say that we're fungible. We're -

>> You can be a hub and spoke.

Tom Traugott

>> We can be a training environment. And as the latest and greatest gear becomes last year's model, that becomes repurposed as inference. And we're close enough to key population centers that I think that's a big part of our strategy flex.

>> You'll flex the facility to whatever the resource is being powered by.

Tom Traugott

>> Yeah. And with scale. So, we're trying to meet multipurpose. What is interesting too though, and there's a couple of folks I've talked to and NVIDIA, I think is pushing for this, there is also this commoditization of GPUs. So, to make GPUs available anywhere and like a DGX Lepton. So, providing on-demand spot capacity. It's not large clusters. You can't really manage it that way. But you want 10 GPUs, a hundred GPUs on demand, from what I've gathered, GPUs, there's the possibility that they may be able to operate in a decentralized manner, maybe a little more easily than cloud historically. Cloud's got a lot of hypervisor, a lot of management layers. But if you're an edge facility and you've got power and sufficient cooling, you may be able to support your own little cluster of GPUs. .

>> Well, Tom, to keep in touch. I'm really glad you came in. Congratulations. Keep doing what you got to do. Get more power.

Tom Traugott

>> Get more power.

>> It's like Star Trek, Scotty, more power. There's an old school Star Trek reference there. But this is the key. It's not going away. It's going to be a whole other ball game.

Tom Traugott

>> It is, yeah. And one of these days, who knows, our industry kind of a lot of take privates over the last five years because of all the development. But we will see if there's some more bell ringing coming years.

>> There's definitely a demand. Well, thanks for coming on. I appreciate it.

Tom Traugott

>> Thank you so much.

>> All right. Power is the key, cooling is the key. Gen AI needs the horsepower, so does Crypto. We are going to look at a whole nother revolution and evolution in how infrastructure is going to be provisioned and scaled up. Of course, global scales with digital culture, as we're seeing it now with stable coins, digital twins. Our lives are digital and physical coming together, physical AIs, NVIDIA calls it. I'm John Furrier, your host. Thanks for watching.

>> .