NVIDIA GTC 2024 | Jonathan Lin | Equinix

Clips
More from NVIDIA GTC 2024

Jonathan Lin

Executive Vice President and General Manager, Data Center ServicesExecutive Vice President and General Manager, Data Center Services

Equinix

play_circle_outline Equinix and Nvidia Collaborate to Empower AI Workloads and DGX Deployments, Empowering Data Control for Enterprises

play_circle_outline The potential for AI factories and the future of data centers with evolving technology

Info

Jonathan Lin | Equinix

Jonathan Lin

Executive Vice President and General Manager, Data Center ServicesExecutive Vice President and General Manager, Data Center Services Equinix

search

(electronic music intro)

>> Hi everybody, we're back with theCUBE at GTC 2024 here in San Jose, we're at the popup CUBE we're running and gunning. Really excited to have Jon Lin here. He's the EVP and general manager, of the Data Center Services group at Equinix.

Great to see you. >> Absolutely.

>> Thanks so much for coming on and making-

>> Thanks for having me. I love this gorilla interview style, it's fantastic.

>> Love it, too, it's like unconference, the unCUBE.

>> Love it.

>> So you saw the keynote yesterday, which was amazing. We were just in a two-hour meeting with Jensen, and he was just kind of going on and on and he said he was so nervous up there, which was amazing to me.

>> You can't see that, he just- >> He said, "I felt like I was doing crunches for two hours." We were like, "Wow, dude, you're amazing up there."

Well, what do you- >> No rehearsal. No rehearsal, he just goes up there and wings it for two hours.

>> Unreal, I mean, wow, what a talent. Wasn't reading from a teleprompter

>> No. >> And just going for it. What'd you think of the keynote? What were the key takeaways for Equinix for you?

>> Yeah, just incredible. One, the obviously, the announcement about Blackwell and what that means for just the hardware landscape for AI, overall. Just really excited about the work that we've partnered with them already, understanding how to support that. I think the density that they're driving and kind of the unique characteristics from a liquid cooling perspective. Again, we've been working with them for months to make sure we understand that and are ready to support that for our customers. But more importantly, I think, you look at the landscape for AI at large, right? And the economic benefit and impact it can have for the world, and like, the use cases being described and like, this isn't trivial, like, we're making funny cat photos. This is, hey, we're doing drug discovery, we're like automating factories. We're like, creating more productive crop yields. It's really, really powerful stuff.

>> So what exactly are you doing with Nvidia? You mentioned liquid cooling. I mean, I presume you're thinking about packaging and standards and these things run hot.

>> Yeah. >> So you got to figure it out. But give us some details on what the partnership is like.

>> No, absolutely. So we've been working with Nvidia for a number of years, dating back to when they started with the AI Launchpad as a service, kind of introducing that to new users of AI. That was actually powered at Equinix, so giving them infrastructure to be able to support that. Just knowing that coming down the road we could see this workload and this use case was going to be incredibly important for the enterprise. And, we think about ourselves as the world's digital infrastructure company and an obligation to bring to all of our customers, whether it's service providers, the clouds, or the enterprises using that, the best technology in the world. And so a great opportunity to bring NVIDIA to into the bed years ago. And then over the course of the last year, we've been working feverishly to help support their DGX deployments. So really private DGX deployments for the enterprise. What Nvidia had been hearing and what we've been hearing was, the enterprise is really, they're obviously enamored with the use of AI, but they're worried about cross-training of their data, they're worried about leakage potentially into other models. And so the ability to control and own that infrastructure is just incredibly powerful and they get the ownership economics for those workloads.

>> So explain how that's different to our audience from the cloud model. A cloud, multi-tenant, I go in, I rent infrastructure, it simplifies my life. You guys have made a great business sort of colocating,

proximate to clouds- >> Right.

>> So to reduce latency. But you've got a different model. Explain that.

>> Yeah, it's really a kind of a best of both worlds for the enterprise. Obviously everyone is a cloud user. I think the ability to have some of those workloads though, on a private basis for their own compute or making sure they understand the most critical applications or where they want control, they want to have that domain and ownership of that infrastructure themselves. So being able to connect that into the public cloud is incredibly important. And so what we've built then, for the enterprise, is really the nexus of all of the most critical data flows that they have. It's whether they're using the multi-cloud, whether they're using their supplier network, whether they're using APIs with other SaaS providers, all of that comes into their infrastructure that's housed with us at Equinix. And so that's the most logical place then to put these AI workloads, right? You want to put that directly next to the data, right? And so that ability to have that together and colocated in the same facility is incredibly powerful.

>> So people, Jensen in particular, talks about the AI factory.

>> Yeah. >> Do you, when you look at the Equinix infrastructure and the data centers, do you start to envision AI factories? Is that a little hyperbole? Well, how do you think about that?

>> No, I, look, I think the future is heading this way, right? And I think we've obviously been working very closely with Nvidia to understand what will that requirement look like? How do we understand what the latency characteristics of that are going to be for inference purposes? We're in 71 metros in 30 countries around the world. Like that ability to then let a customer have that deployment as close to the end user as they need to, with 90%-plus of the world's population being within 10 milliseconds of our facilities is really, really powerful. And so we are though, the future is evolving so rapidly. I think just again, just from the keynote yesterday about the announcement on Blackwell and what those requirements will look like. It will change the design and and implementation of data centers for the future.

>> So your customers are using AI-powered servers, and other infrastructure as well, storage, networking, et cetera, but they're basically, think Dell, think Super Micro, etcetera, taking servers, putting them, they've got GPUs and they're putting them in your data centers.

>> That's right.

>> My question is how do you think about and how should we think about today's, and they're doing whatever they're doing training, they're doing inference, Jon, is that the same infrastructure or does it require different infrastructure? Is today's training become tomorrow's inference with a depreciated asset? How should we think about the life cycle as we grow from, where we are today to a 1,000-X performance in 10 years?

>> Yeah, I think it's, candidly, I think it's early days, and I think each enterprise's use case of inference or generation as Jensen likes to put it, I think it's going to depend on what they're exactly they're doing with that data, where it needs to live close to the end user and close to the applications that they're delivering. So we're going to see and evolve. I do think though that a lot of the work, when you think about training, training is cost, right? You're ending up devoting energy, time, power, money into that infrastructure. The value is created when you flip the switch and turn it to inference as you said. And then you're going to retrain periodically, but that's not as big a load, I think. And that's what we're seeing as an emerging use.

>> Well, let's talk about energy sustainability. Of course the media's going to go right after that issue.

>> Sure. >> And it's an important issue. But you certainly start it with crypto, and now AI is very energy consumptive. How are you dealing with that problem? You mentioned liquid cooling. Jensen talks about, essentially today he implied, or at least I inferred, that the work that you can do per watt, as long as you can keep the factory loaded-

>> That's right. >> Is going to be more efficient.

>> More efficient, right.

>> But if you don't, then obviously, you're wasting a lot of power. So how, I mean, I know it's an important issue, but where does it stand on the hierarchy of importance and how are you dealing with it?

>> Yeah, for Equinix, I mean, sustainability is one of the core ethoses that we're operating with. We were the first data center provider to talk about going to a hundred percent renewable power. We're actually over 95% coverage right now, globally. And this has been a focus area for us for probably 15 years now, close to. And so when we think about this, and that's across the entire supply chain. It's about where you're sourcing your power, how we're making sure we're doing renewable power, but also on data center design around efficiency to be able to reduce the power utilization and also efficiency around cooling. So the liquid cooling component, liquid cooling is a much more efficient way to actually remove the heat out of the system. And that liquid cooling isn't we're wasting water and splurging, it's actually like a closed loop system. In our case, we're using that and making sure that we're not wasting water, which again is one of the areas that we think about on a sustainability level. But yeah, ultimately we want to make sure that as we're developing this technology and it's being deployed more broadly, we're helping our customers get there. And that's, we can provide them attestable reporting about their own green power utilization. That's incredibly valuable for their commitments around sustainability as well.

>> So you'll use reuse that hot liquid and-

>> We cool it down and yeah, we run it through our central chiller plant. We go ahead and cool that liquid down, and it really is a closed loop system around that.

>> Are there standards emerging around liquid cooling or is it kind of still the Wild West or?

>> I would say it's slightly more mature than the Wild West, but it is still very much an environment where it's moving from fully bespoke everywhere, with relatively like, small amounts of production to kind of factory and standardization. And we've spent the last year productizing, understanding both the engineering for our own sites, how we're going to do this, but also the interface with our customers. And we announced at the end of last year, we have a hundred data centers from our existing fleet that can support liquid cooling, including directed chip liquid cooling around that because we knew that this was going to come.

>> I mean, I've been around a long time, so I remember liquid cooling from the mainframe days. (Jon laughing) And it's really interesting to see when you walk around shows like HPE Discover, or Dell Tech World, the number of liquid cooling companies that are now on the show floor.

>> That's right. >> And obviously, it's going to be a hot space for a while. I want to ask you about, so last year theCUBE research developed the power law of Gen AI, and we used typical power law, which is a hard right angle where you got a few brands really kind of own the day and then you have this long tail. And we said we see something similar forming with, with LLMs, but it's different, in that the dimensions are maybe size of model,

and then domain specificity. >> That's right.

>> And the other difference is the torso we see getting pulled up by open source. When you see things like meta and, and Llama-2

and Llama-3 >> Of course.

>> And other open source and third-party, and (indistinct) here, Sort of an interesting dynamic. But these small language models that are very domain-specific, that will likely be on-prem, that will drive different industries that will allow customers to have their own, sort of leverage their own proprietary data sets, whether they're using RAG, et cetera, what are you seeing there? Is that starting to take shape based on the patterns that you're seeing?

>> It is, and I mean the testament to the 30,000-plus people at this show. I think there's so much interest and energy around this. And this is, we're moving away from the experimentation phase and this is real life deployments now all over the place, across many different industry verticals, and you said it well. That specificity of use case is going to be incredibly important. Not every enterprise is going to end up developing a massive large-language model on their own. They're going to go ahead and purchase models for their use. They're going to go ahead and retrain off of their data and make sure there's also small-language model and very vertical specific use cases that they'll drive. Whether that's video processing or pharmaceutical drug discovery. Like all of that's going to be incredibly powerful.

>> Well and of course let's talk about the macro to the extent that you can share. I mean, Equinix, incredibly successful company, you're probably what, I don't know, 10 billion, with an $80 billion market cap, something like that, which is really unheard of for an infrastructure company to have that kind of revenue multiple. But, but the macro's still there. The macro's still tough. Our data suggests with our partners at ETR that about 45% of the customers that we talk to, say they're funding AI from other budgets.

>> Hmm. >> So it's not just this incremental go ahead and spend. IT spending, maybe it's growing three to 4% this year. So it's not just all systems go, but AI is clearly stealing from other areas. As well, about two-thirds of the customers tell us that they expect an ROI within 12 months.

>> That's right. >> So very, very aggressive. So you said earlier we're moving beyond the experimentation phase. Do you feel like 2024 will be the year of AI ROI?

>> I think we're starting to see that for sure. I mean, again, it's a world where the value generation that the AI use cases are creating is so massive, right? And it's, I don't think about any of this as a zero sum game in terms of like, total amount of spend in GD, because the value creation for the enterprise, when they actually see that being realized, then they can go ahead and fund that even further, right? And that creates more envelope for them as they're driving either higher revenue line, better productivity, better yield, all of that. And so we are seeing that starting to emerge already. And so I think in '24 and beyond, I think certainly '25 is going to be another big year for this entire sector. I think we'll see a continued investment in this.

>> Yeah, I think we are early innings in this super cycle. I was, listen, I heard Bill Gurley the other day say everybody wants to compare it to the .com bust. And he said the big difference here, and he's right I think, is the funding levels are so much higher. The other thing is, .com was funded by a bunch of companies that kind of, like Enron, Global Crossing, that sort of went out of business. Today it's being funded by the CapEx of hyperscalers.

>> That's right. >> And they got, their balance sheets are enormous. So it's a different dynamic. But I hope you're right. I hope that starts showing up and the productivity numbers, or there are 30,000 crazy people here, Jon. Hey listen, I really appreciate you spending

some time with theCUBE. >> Absolutely.

>> It was great to have you. >> Thanks.

>> Alright, keep it right there for more action from GTC 2024 from San Jose. You're watching theCUBE. (electronic music outro)

Jonathan Lin | Equinix

search

(electronic music intro)

>> Hi everybody, we're back with theCUBE at GTC 2024 here in San Jose, we're at the popup CUBE we're running and gunning. Really excited to have Jon Lin here. He's the EVP and general manager, of the Data Center Services group at Equinix.

Great to see you. >> Absolutely.

>> Thanks so much for coming on and making-

>> Thanks for having me. I love this gorilla interview style, it's fantastic.

>> Love it, too, it's like unconference, the unCUBE.

>> Love it.

>> So you saw the keynote yesterday, which was amazing. We were just in a two-hour meeting with Jensen, and he was just kind of going on and on and he said he was so nervous up there, which was amazing to me.

>> You can't see that, he just- >> He said, "I felt like I was doing crunches for two hours." We were like, "Wow, dude, you're amazing up there."

Well, what do you- >> No rehearsal. No rehearsal, he just goes up there and wings it for two hours.

>> Unreal, I mean, wow, what a talent. Wasn't reading from a teleprompter

>> No. >> And just going for it. What'd you think of the keynote? What were the key takeaways for Equinix for you?

>> Yeah, just incredible. One, the obviously, the announcement about Blackwell and what that means for just the hardware landscape for AI, overall. Just really excited about the work that we've partnered with them already, understanding how to support that. I think the density that they're driving and kind of the unique characteristics from a liquid cooling perspective. Again, we've been working with them for months to make sure we understand that and are ready to support that for our customers. But more importantly, I think, you look at the landscape for AI at large, right? And the economic benefit and impact it can have for the world, and like, the use cases being described and like, this isn't trivial, like, we're making funny cat photos. This is, hey, we're doing drug discovery, we're like automating factories. We're like, creating more productive crop yields. It's really, really powerful stuff.

>> So what exactly are you doing with Nvidia? You mentioned liquid cooling. I mean, I presume you're thinking about packaging and standards and these things run hot.

>> Yeah. >> So you got to figure it out. But give us some details on what the partnership is like.

>> No, absolutely. So we've been working with Nvidia for a number of years, dating back to when they started with the AI Launchpad as a service, kind of introducing that to new users of AI. That was actually powered at Equinix, so giving them infrastructure to be able to support that. Just knowing that coming down the road we could see this workload and this use case was going to be incredibly important for the enterprise. And, we think about ourselves as the world's digital infrastructure company and an obligation to bring to all of our customers, whether it's service providers, the clouds, or the enterprises using that, the best technology in the world. And so a great opportunity to bring NVIDIA to into the bed years ago. And then over the course of the last year, we've been working feverishly to help support their DGX deployments. So really private DGX deployments for the enterprise. What Nvidia had been hearing and what we've been hearing was, the enterprise is really, they're obviously enamored with the use of AI, but they're worried about cross-training of their data, they're worried about leakage potentially into other models. And so the ability to control and own that infrastructure is just incredibly powerful and they get the ownership economics for those workloads.

>> So explain how that's different to our audience from the cloud model. A cloud, multi-tenant, I go in, I rent infrastructure, it simplifies my life. You guys have made a great business sort of colocating,

proximate to clouds- >> Right.

>> So to reduce latency. But you've got a different model. Explain that.

>> Yeah, it's really a kind of a best of both worlds for the enterprise. Obviously everyone is a cloud user. I think the ability to have some of those workloads though, on a private basis for their own compute or making sure they understand the most critical applications or where they want control, they want to have that domain and ownership of that infrastructure themselves. So being able to connect that into the public cloud is incredibly important. And so what we've built then, for the enterprise, is really the nexus of all of the most critical data flows that they have. It's whether they're using the multi-cloud, whether they're using their supplier network, whether they're using APIs with other SaaS providers, all of that comes into their infrastructure that's housed with us at Equinix. And so that's the most logical place then to put these AI workloads, right? You want to put that directly next to the data, right? And so that ability to have that together and colocated in the same facility is incredibly powerful.

>> So people, Jensen in particular, talks about the AI factory.

>> Yeah. >> Do you, when you look at the Equinix infrastructure and the data centers, do you start to envision AI factories? Is that a little hyperbole? Well, how do you think about that?

>> No, I, look, I think the future is heading this way, right? And I think we've obviously been working very closely with Nvidia to understand what will that requirement look like? How do we understand what the latency characteristics of that are going to be for inference purposes? We're in 71 metros in 30 countries around the world. Like that ability to then let a customer have that deployment as close to the end user as they need to, with 90%-plus of the world's population being within 10 milliseconds of our facilities is really, really powerful. And so we are though, the future is evolving so rapidly. I think just again, just from the keynote yesterday about the announcement on Blackwell and what those requirements will look like. It will change the design and and implementation of data centers for the future.

>> So your customers are using AI-powered servers, and other infrastructure as well, storage, networking, et cetera, but they're basically, think Dell, think Super Micro, etcetera, taking servers, putting them, they've got GPUs and they're putting them in your data centers.

>> That's right.

>> My question is how do you think about and how should we think about today's, and they're doing whatever they're doing training, they're doing inference, Jon, is that the same infrastructure or does it require different infrastructure? Is today's training become tomorrow's inference with a depreciated asset? How should we think about the life cycle as we grow from, where we are today to a 1,000-X performance in 10 years?

>> Yeah, I think it's, candidly, I think it's early days, and I think each enterprise's use case of inference or generation as Jensen likes to put it, I think it's going to depend on what they're exactly they're doing with that data, where it needs to live close to the end user and close to the applications that they're delivering. So we're going to see and evolve. I do think though that a lot of the work, when you think about training, training is cost, right? You're ending up devoting energy, time, power, money into that infrastructure. The value is created when you flip the switch and turn it to inference as you said. And then you're going to retrain periodically, but that's not as big a load, I think. And that's what we're seeing as an emerging use.

>> Well, let's talk about energy sustainability. Of course the media's going to go right after that issue.

>> Sure. >> And it's an important issue. But you certainly start it with crypto, and now AI is very energy consumptive. How are you dealing with that problem? You mentioned liquid cooling. Jensen talks about, essentially today he implied, or at least I inferred, that the work that you can do per watt, as long as you can keep the factory loaded-

>> That's right. >> Is going to be more efficient.

>> More efficient, right.

>> But if you don't, then obviously, you're wasting a lot of power. So how, I mean, I know it's an important issue, but where does it stand on the hierarchy of importance and how are you dealing with it?

>> Yeah, for Equinix, I mean, sustainability is one of the core ethoses that we're operating with. We were the first data center provider to talk about going to a hundred percent renewable power. We're actually over 95% coverage right now, globally. And this has been a focus area for us for probably 15 years now, close to. And so when we think about this, and that's across the entire supply chain. It's about where you're sourcing your power, how we're making sure we're doing renewable power, but also on data center design around efficiency to be able to reduce the power utilization and also efficiency around cooling. So the liquid cooling component, liquid cooling is a much more efficient way to actually remove the heat out of the system. And that liquid cooling isn't we're wasting water and splurging, it's actually like a closed loop system. In our case, we're using that and making sure that we're not wasting water, which again is one of the areas that we think about on a sustainability level. But yeah, ultimately we want to make sure that as we're developing this technology and it's being deployed more broadly, we're helping our customers get there. And that's, we can provide them attestable reporting about their own green power utilization. That's incredibly valuable for their commitments around sustainability as well.

>> So you'll use reuse that hot liquid and-

>> We cool it down and yeah, we run it through our central chiller plant. We go ahead and cool that liquid down, and it really is a closed loop system around that.

>> Are there standards emerging around liquid cooling or is it kind of still the Wild West or?

>> I would say it's slightly more mature than the Wild West, but it is still very much an environment where it's moving from fully bespoke everywhere, with relatively like, small amounts of production to kind of factory and standardization. And we've spent the last year productizing, understanding both the engineering for our own sites, how we're going to do this, but also the interface with our customers. And we announced at the end of last year, we have a hundred data centers from our existing fleet that can support liquid cooling, including directed chip liquid cooling around that because we knew that this was going to come.

>> I mean, I've been around a long time, so I remember liquid cooling from the mainframe days. (Jon laughing) And it's really interesting to see when you walk around shows like HPE Discover, or Dell Tech World, the number of liquid cooling companies that are now on the show floor.

>> That's right. >> And obviously, it's going to be a hot space for a while. I want to ask you about, so last year theCUBE research developed the power law of Gen AI, and we used typical power law, which is a hard right angle where you got a few brands really kind of own the day and then you have this long tail. And we said we see something similar forming with, with LLMs, but it's different, in that the dimensions are maybe size of model,

and then domain specificity. >> That's right.

>> And the other difference is the torso we see getting pulled up by open source. When you see things like meta and, and Llama-2

and Llama-3 >> Of course.

>> And other open source and third-party, and (indistinct) here, Sort of an interesting dynamic. But these small language models that are very domain-specific, that will likely be on-prem, that will drive different industries that will allow customers to have their own, sort of leverage their own proprietary data sets, whether they're using RAG, et cetera, what are you seeing there? Is that starting to take shape based on the patterns that you're seeing?

>> It is, and I mean the testament to the 30,000-plus people at this show. I think there's so much interest and energy around this. And this is, we're moving away from the experimentation phase and this is real life deployments now all over the place, across many different industry verticals, and you said it well. That specificity of use case is going to be incredibly important. Not every enterprise is going to end up developing a massive large-language model on their own. They're going to go ahead and purchase models for their use. They're going to go ahead and retrain off of their data and make sure there's also small-language model and very vertical specific use cases that they'll drive. Whether that's video processing or pharmaceutical drug discovery. Like all of that's going to be incredibly powerful.

>> Well and of course let's talk about the macro to the extent that you can share. I mean, Equinix, incredibly successful company, you're probably what, I don't know, 10 billion, with an $80 billion market cap, something like that, which is really unheard of for an infrastructure company to have that kind of revenue multiple. But, but the macro's still there. The macro's still tough. Our data suggests with our partners at ETR that about 45% of the customers that we talk to, say they're funding AI from other budgets.

>> Hmm. >> So it's not just this incremental go ahead and spend. IT spending, maybe it's growing three to 4% this year. So it's not just all systems go, but AI is clearly stealing from other areas. As well, about two-thirds of the customers tell us that they expect an ROI within 12 months.

>> That's right. >> So very, very aggressive. So you said earlier we're moving beyond the experimentation phase. Do you feel like 2024 will be the year of AI ROI?

>> I think we're starting to see that for sure. I mean, again, it's a world where the value generation that the AI use cases are creating is so massive, right? And it's, I don't think about any of this as a zero sum game in terms of like, total amount of spend in GD, because the value creation for the enterprise, when they actually see that being realized, then they can go ahead and fund that even further, right? And that creates more envelope for them as they're driving either higher revenue line, better productivity, better yield, all of that. And so we are seeing that starting to emerge already. And so I think in '24 and beyond, I think certainly '25 is going to be another big year for this entire sector. I think we'll see a continued investment in this.

>> Yeah, I think we are early innings in this super cycle. I was, listen, I heard Bill Gurley the other day say everybody wants to compare it to the .com bust. And he said the big difference here, and he's right I think, is the funding levels are so much higher. The other thing is, .com was funded by a bunch of companies that kind of, like Enron, Global Crossing, that sort of went out of business. Today it's being funded by the CapEx of hyperscalers.

>> That's right. >> And they got, their balance sheets are enormous. So it's a different dynamic. But I hope you're right. I hope that starts showing up and the productivity numbers, or there are 30,000 crazy people here, Jon. Hey listen, I really appreciate you spending

some time with theCUBE. >> Absolutely.

>> It was great to have you. >> Thanks.

>> Alright, keep it right there for more action from GTC 2024 from San Jose. You're watching theCUBE. (electronic music outro)