NVIDIA GTC 2024 | John Mao | VAST Data

Clips
More from NVIDIA GTC 2024

John Mao

Global Head of Business Development

VAST Data

play_circle_outline VAST Data's prominence in alternative clouds for AI

play_circle_outline Specialized Cloud Providers: Dedicated Infrastructure and AI Capabilities for Complex Workloads

play_circle_outline The Role of VAST Database in Enabling Structured Semantics and File System Services for AI Workloads

Info

John Mao | VAST Data

John Mao

Global Head of Business Development VAST Data

search

(bright pop music)

>> Hi, everybody, we're back in San Jose at GTC '24, just unpacking the AI era, it's just still pretty packed here, maybe not as so much as it was the other day, but it's good to be here, John Mayo is here. He is the VP of Alliances at VAST Data, thanks so much for taking some time.

>> Thanks Chad. >> Big show for you guys.

>> Huge show, it's good to be back. Actually, I don't think I've been to a GTC since I joined VAST four and a half years ago.

>> Well, yeah 'cause it 2019 was the last one.

>> Right.

>> And they've certainly outgrown 2019.

>> It's been phenomenal. >> Yeah.

>> It's amazing. >> It's pretty good, you guys got a good presence here and we were just at a luncheon that you guys held, thank you for analyst, that was good, and you had one of the leaders from Genesis Cloud there kind of explaining their model. But you guys sort of have quietly, sort of in plain sight, but hidden in plain sight, if you will, have really carved out a really prominent position with some of these alternative clouds. I wonder if you could explain for the audience, like why do people need a different cloud for AI? Why can't the sort of conventional clouds that we all know and love, just kind of support the workloads the way you guys envision it?

>> Yeah, so I mean, last year, 2023 was really a banner breakout year for VAST Data, especially in these kind of specialized AI clouds, like you mentioned, right? Genesis Cloud being one of them, we have great partnerships with CoreWeave, with Lambda Labs, Core42, right, and many, many more. I think there's been an opportunity in the market for these specialized clouds because everyone started in the hyperscaler clouds. But there are some needs where we're talking now thousands and tens of thousands of GPU sometime for some of these very, very large models that are being trained today that just need a lot more horsepower, right? Need a lot more, both on the data side, but also on the compute side, obviously. So there's been a kind of a emergence of these type of vendors and we've been working hand in hand with many of them designing new architectures to enable this type of infrastructure.

>> So Jeff Denworth said something at the launch, I showed up late, so I kind of didn't catch the whole thing, but he said, "Look, the traditional cloud guys, they will thick provision a GPU cluster." And I'm like, "Hmm, wait a minute, that doesn't sound cost effective." What does that mean and how are these alternative clouds different? What does the customer get?

>> So I think a lot of the specialized cloud providers are providing dedicated bare metal, they're not typically like heavily shared infrastructure. Whether they virtualize or not is kind of not the point. There's kind of been lots of technology advancements to give you max peak performance of GPUs in lots of different ways. But I think the biggest difference is that it's really purpose built. I mean, you were at the Keynote I'm sure on Monday.

>> Yeah.

>> Networking's a huge part of it, right? So the considerations that go into building these AI supercomputers, you have to look at it as you're literally building a supercomputer for some of these new workloads, so every single facet of the data center, from physical layouts to how you do network design to how you segment compute and storage networking, all of those things are big factors in making sure these jobs run well and run as fast as possible.

>> So it sounds like these companies are just, they're thinking from the ground up, they're purpose-built data centers for AI.

>> Yes. >> As opposed to kind of-

>> General purpose, yeah. >> Retrofitting.

>> They're definitely not general purpose clouds, but it's been pretty fun 'cause we've been working with CoreWeave, for example, and it's allowed us to really innovate. Like we just made our announcement a couple of weeks ago where we've been partnering with Nvidia on the BlueField DPU technology, so it's their kind of accelerated networking car plus compute all in one technology. And we've been working with partners like CoreWeave to build a system that's more secure, more scalable, more efficient from a power consumption perspective. So when you're running at this scale, some of these AI cloud providers, like, it matters a lot, right? Security matters a lot, multi-tenancy matters a lot, QoS matters a lot for their business, right? And so it's been pretty fun.

>> How are they able to, I mean the hyperscalers have so much cash,

they got cheap cost of money. >> Yeah.

>> They're lining up to buy GPUs. We heard on Monday, they all signed up for Blackwell. They're ready to go, they all got shout outs. How do these guys compete from a CapEx standpoint? Are they just like, give amazing VC funding or they have all kinds of backers? Is it strategic? How are they able to compete?

>> I'm not a finance guy, (both laughing) so I do know that there has been a massive infusion in investments globally. And I think there's also some of it is private money, some of it is government money. A lot of nations now are looking at building sovereign clouds for AI specifically, right? And that's going to stay within their borders and a lot of times it's funded by government money. So I think there's money coming from private sector also from governments and all over. And there's probably more creative ways of financing that I don't quite understand.

>> Well, but it makes sense what you're saying because these governments, they have to protect their own interests,

they have local laws. >> Yeah.

>> They don't want people breaking the laws, but at the same time, if it's really hard to comply, they want to make it easier, so that makes a lot of sense. Okay, let's get to VAST, like what's your role in these alternative data centers? You guys are obviously providing-

>> Yeah. >> File systems. They're doing a lot of like amazing HPC

and HPC and AI are colliding. >> Right, yeah.

>> Give us the rundown there.

>> Yeah, so I mean, historically we've been, people could think of us as a high performance scale out file and object storage system. And we do that very well, we built our amazing business over the last four years on that kind of foundation. But starting last year we started to introduce and roll out and start to ship new capabilities. And I don't know if it's been clear to everybody, we talked about the vision, but we introduced new capabilities like the VAST Database. And people have always wondered why are you getting into that space? And I think as AI's kind of come onto the scene, people are starting to understand that you need those kinds of structured semantics in order to take unstructured data, images, videos, et cetera, be able to put some structure and be able to feed that into these large GPU clusters to unlock a true insight, right? So the reason why we've had so much success recently is that we're starting to introduce new kind of up the stack data services necessary at solving a lot of these large complex AI and data pipelines end to end, right? So it's not just about how quickly can you read and write the IO performance of systems, that's obviously table stakes, but how can you simplify the data management? How do you accelerate the data preparation and the other stages that happen within a very complex AI pipeline?

>> Yeah, we had Renin and Jeff on our breaking analysis program and we were talking about the next data platform.

>> Yeah.

>> And some of the attributes that you think about, you mentioned some, but you're talking about petabyte scale.

>> Right. >> You're also talking about all different data types, different query types. And you know today's data platforms really aren't built for that, they're largely built for analytics.

>> Right. >> Which is great. But they weren't built for AI and now you see the Snowflake's and the Databricks, which are great companies, but they're sort of, it's a pejorative I know, but they're sort of bolting on AI.

>> Right. >> Okay, which will work for them, there's a lot of data in there. But again, I think if I understand it correctly and listening to Renin and Jeff, you guys saw this coming, you didn't know when, so you had a good business just selling big honking object storage, right? And then boom, the market hits and it just comes your way.

>> Yeah, we definitely, if anybody told you this, we definitely didn't predict when this would happen, but you could kind of see it on the horizon. I also think that the time as far as why we're different and why we think we have a unique advantage is we started a little bit late and we built everything from the ground up, like from infrastructure up that was optimized for where we are today, right? In this AI era, so not building and kind of stacking on top of legacy infrastructure approaches. We've custom built literally from the hardware on up from day one, and it's put us in a very good position to win.

>> You know you saw this, it's interesting you say that John, because you saw like pre-cloud companies that were formed in let's say 2006, 2007 timeframe, or even before, even 2009 before the cloud was a thing, before the saasification in business, they had to pivot. And some of 'em did quite successfully, but versus, again, take like a Snowflake that was really born in the cloud, born cloud native, and they were able to excel and reach escape velocity. So the fact that you were a little late, right?

Was actually advantageous- >> Right.

>> In terms of compressing the time to AI, if I can use that concept.

>> No and I think we've not only been able do that for the traditional on-prem data center, but we're also moving the other direction into the hyperscaler clouds to help solve these problems there for customers as well so.

>> Okay, so you're helping those guys move into the AI era as well is what you're saying.

>> Right, right, right. >> And so. So that's going to be an interesting collision down the road. You got the rabbits out front, grabbing some early market share-

>> Right. >> Gaining some traction and probably gaining a lot of innovation. What about the ecosystems that are forming around those sort of AI clouds? What are you seeing there?

>> I think there's obviously a big focus on software partnerships right now. Nvidia's got a very broad range of software solutions, there's a lot of emphasis this week, especially on the inferencing side, right? So inferencing I believe is a big focus for this year. It's where a lot of people right now have bragging rights around training, but inferencing is where the economy's actually going to, you're going to see that explode, right? So it's where the monetization effects are going to happen. So there's a whole entire ecosystem of software, there's a lot of focus on NIM's, the announcements with Nvidia. We're partnering with a lot of different ecosystem players that are helping operationalize a lot of inference workloads as well. So, yeah, I think increasingly it's going to be around how do you help from a full lifecycle management of inferencing train models, getting those deployed through the monetization of those is going to be a big part.

>> What's the partnership with NIM's? Can you explain that and we can unpack that a little bit.

>> I mean, so NIM's is an entire solution where you want to be able to use existing data to be able to fine tune. A lot of these foundational models are becoming really, really, really smart and really good. But if you want to apply kind of domain specific relevance from your business from an organization, you have to feed it with some business specific content, right, information, so those are frameworks that are emerging to help make that simpler, right? To go tap into the enterprise and the strobes of data that they have in their environments make that more tailored for their business. Not even to the organization, but it could be even at a departmental level within a large organization.

>> So you're saying I can basically point that infrastructure, the data that's inside of a VAST object store.

>> Right. >> And then essentially create my own sort of LLM, if you will, but that's very domain specific.

>> Right.

>> So this would be apply whether it's financial services, I could see use cases in healthcare.

>> Healthcare is one.

>> I mean, virtually any industry.

>> I think so, I mean they gave some examples of it, but yeah, absolutely taking, I was talking with a large system integrator the other day, but they are by definite an information environment from how they handle contracts to how they develop knowledge, right? They're like that is their secret sauce. So being able to capture that content that's been stored in archive forever, feeding that in to fine tune and improve these is going to be very, very powerful.

>> So this show, I think this show actually will mark a milestone because there was no GTC really after COVID. This was the first one after COVID, pretty amazing actually. You know most conferences, I'm sure you remember late 2021, even 2022. Ah, isn't it great to be back in person and we all cheer. And now it's 2024, people aren't saying that anymore. But I think this conference, I'd love to get your thoughts as sort of final question, really marks a new milestone in the AI era. It touches virtually every industry, it's not just a bunch of technology companies here-

>> Right. >> Presenting. I mean, I guess the old bromide, every company's a technology company. It's really proving out here. I've never seen a more talked about event. I mean you see things like Dreamforce and some of those other big conferences and CES and others. You know but I mean it's still going on the TV shows and the business channels and it really is, I think, marking a new era for our industry.

>> The cliche, I think it was used by Jensen last year, right, it was like the iPhone moment has arrived and I think it's so true. I mean, the funny, so anecdotally, right? I haven't actually spoken with or met with a university or a research center this entire week, right? It's with like private sector enterprises that are trying to figure out how to infuse their business with AI and possibly either save costs or make more money or increase productivity, whatever it may be. So that is a clear inflection points happening in the industry and it's pretty exciting to see.

>> All right, John, thanks so much for coming on.

>> Thanks for having me,

appreciate it. >> We appreciate it. All right, keep it right there. We'll be back for more action from San Jose, David Vellante and John Furrier from GTC '24, you're watching "theCUBE." (bright pop music)

John Mao | VAST Data

search

(bright pop music)

>> Hi, everybody, we're back in San Jose at GTC '24, just unpacking the AI era, it's just still pretty packed here, maybe not as so much as it was the other day, but it's good to be here, John Mayo is here. He is the VP of Alliances at VAST Data, thanks so much for taking some time.

>> Thanks Chad. >> Big show for you guys.

>> Huge show, it's good to be back. Actually, I don't think I've been to a GTC since I joined VAST four and a half years ago.

>> Well, yeah 'cause it 2019 was the last one.

>> Right.

>> And they've certainly outgrown 2019.

>> It's been phenomenal. >> Yeah.

>> It's amazing. >> It's pretty good, you guys got a good presence here and we were just at a luncheon that you guys held, thank you for analyst, that was good, and you had one of the leaders from Genesis Cloud there kind of explaining their model. But you guys sort of have quietly, sort of in plain sight, but hidden in plain sight, if you will, have really carved out a really prominent position with some of these alternative clouds. I wonder if you could explain for the audience, like why do people need a different cloud for AI? Why can't the sort of conventional clouds that we all know and love, just kind of support the workloads the way you guys envision it?

>> Yeah, so I mean, last year, 2023 was really a banner breakout year for VAST Data, especially in these kind of specialized AI clouds, like you mentioned, right? Genesis Cloud being one of them, we have great partnerships with CoreWeave, with Lambda Labs, Core42, right, and many, many more. I think there's been an opportunity in the market for these specialized clouds because everyone started in the hyperscaler clouds. But there are some needs where we're talking now thousands and tens of thousands of GPU sometime for some of these very, very large models that are being trained today that just need a lot more horsepower, right? Need a lot more, both on the data side, but also on the compute side, obviously. So there's been a kind of a emergence of these type of vendors and we've been working hand in hand with many of them designing new architectures to enable this type of infrastructure.

>> So Jeff Denworth said something at the launch, I showed up late, so I kind of didn't catch the whole thing, but he said, "Look, the traditional cloud guys, they will thick provision a GPU cluster." And I'm like, "Hmm, wait a minute, that doesn't sound cost effective." What does that mean and how are these alternative clouds different? What does the customer get?

>> So I think a lot of the specialized cloud providers are providing dedicated bare metal, they're not typically like heavily shared infrastructure. Whether they virtualize or not is kind of not the point. There's kind of been lots of technology advancements to give you max peak performance of GPUs in lots of different ways. But I think the biggest difference is that it's really purpose built. I mean, you were at the Keynote I'm sure on Monday.

>> Yeah.

>> Networking's a huge part of it, right? So the considerations that go into building these AI supercomputers, you have to look at it as you're literally building a supercomputer for some of these new workloads, so every single facet of the data center, from physical layouts to how you do network design to how you segment compute and storage networking, all of those things are big factors in making sure these jobs run well and run as fast as possible.

>> So it sounds like these companies are just, they're thinking from the ground up, they're purpose-built data centers for AI.

>> Yes. >> As opposed to kind of-

>> General purpose, yeah. >> Retrofitting.

>> They're definitely not general purpose clouds, but it's been pretty fun 'cause we've been working with CoreWeave, for example, and it's allowed us to really innovate. Like we just made our announcement a couple of weeks ago where we've been partnering with Nvidia on the BlueField DPU technology, so it's their kind of accelerated networking car plus compute all in one technology. And we've been working with partners like CoreWeave to build a system that's more secure, more scalable, more efficient from a power consumption perspective. So when you're running at this scale, some of these AI cloud providers, like, it matters a lot, right? Security matters a lot, multi-tenancy matters a lot, QoS matters a lot for their business, right? And so it's been pretty fun.

>> How are they able to, I mean the hyperscalers have so much cash,

they got cheap cost of money. >> Yeah.

>> They're lining up to buy GPUs. We heard on Monday, they all signed up for Blackwell. They're ready to go, they all got shout outs. How do these guys compete from a CapEx standpoint? Are they just like, give amazing VC funding or they have all kinds of backers? Is it strategic? How are they able to compete?

>> I'm not a finance guy, (both laughing) so I do know that there has been a massive infusion in investments globally. And I think there's also some of it is private money, some of it is government money. A lot of nations now are looking at building sovereign clouds for AI specifically, right? And that's going to stay within their borders and a lot of times it's funded by government money. So I think there's money coming from private sector also from governments and all over. And there's probably more creative ways of financing that I don't quite understand.

>> Well, but it makes sense what you're saying because these governments, they have to protect their own interests,

they have local laws. >> Yeah.

>> They don't want people breaking the laws, but at the same time, if it's really hard to comply, they want to make it easier, so that makes a lot of sense. Okay, let's get to VAST, like what's your role in these alternative data centers? You guys are obviously providing-

>> Yeah. >> File systems. They're doing a lot of like amazing HPC

and HPC and AI are colliding. >> Right, yeah.

>> Give us the rundown there.

>> Yeah, so I mean, historically we've been, people could think of us as a high performance scale out file and object storage system. And we do that very well, we built our amazing business over the last four years on that kind of foundation. But starting last year we started to introduce and roll out and start to ship new capabilities. And I don't know if it's been clear to everybody, we talked about the vision, but we introduced new capabilities like the VAST Database. And people have always wondered why are you getting into that space? And I think as AI's kind of come onto the scene, people are starting to understand that you need those kinds of structured semantics in order to take unstructured data, images, videos, et cetera, be able to put some structure and be able to feed that into these large GPU clusters to unlock a true insight, right? So the reason why we've had so much success recently is that we're starting to introduce new kind of up the stack data services necessary at solving a lot of these large complex AI and data pipelines end to end, right? So it's not just about how quickly can you read and write the IO performance of systems, that's obviously table stakes, but how can you simplify the data management? How do you accelerate the data preparation and the other stages that happen within a very complex AI pipeline?

>> Yeah, we had Renin and Jeff on our breaking analysis program and we were talking about the next data platform.

>> Yeah.

>> And some of the attributes that you think about, you mentioned some, but you're talking about petabyte scale.

>> Right. >> You're also talking about all different data types, different query types. And you know today's data platforms really aren't built for that, they're largely built for analytics.

>> Right. >> Which is great. But they weren't built for AI and now you see the Snowflake's and the Databricks, which are great companies, but they're sort of, it's a pejorative I know, but they're sort of bolting on AI.

>> Right. >> Okay, which will work for them, there's a lot of data in there. But again, I think if I understand it correctly and listening to Renin and Jeff, you guys saw this coming, you didn't know when, so you had a good business just selling big honking object storage, right? And then boom, the market hits and it just comes your way.

>> Yeah, we definitely, if anybody told you this, we definitely didn't predict when this would happen, but you could kind of see it on the horizon. I also think that the time as far as why we're different and why we think we have a unique advantage is we started a little bit late and we built everything from the ground up, like from infrastructure up that was optimized for where we are today, right? In this AI era, so not building and kind of stacking on top of legacy infrastructure approaches. We've custom built literally from the hardware on up from day one, and it's put us in a very good position to win.

>> You know you saw this, it's interesting you say that John, because you saw like pre-cloud companies that were formed in let's say 2006, 2007 timeframe, or even before, even 2009 before the cloud was a thing, before the saasification in business, they had to pivot. And some of 'em did quite successfully, but versus, again, take like a Snowflake that was really born in the cloud, born cloud native, and they were able to excel and reach escape velocity. So the fact that you were a little late, right?

Was actually advantageous- >> Right.

>> In terms of compressing the time to AI, if I can use that concept.

>> No and I think we've not only been able do that for the traditional on-prem data center, but we're also moving the other direction into the hyperscaler clouds to help solve these problems there for customers as well so.

>> Okay, so you're helping those guys move into the AI era as well is what you're saying.

>> Right, right, right. >> And so. So that's going to be an interesting collision down the road. You got the rabbits out front, grabbing some early market share-

>> Right. >> Gaining some traction and probably gaining a lot of innovation. What about the ecosystems that are forming around those sort of AI clouds? What are you seeing there?

>> I think there's obviously a big focus on software partnerships right now. Nvidia's got a very broad range of software solutions, there's a lot of emphasis this week, especially on the inferencing side, right? So inferencing I believe is a big focus for this year. It's where a lot of people right now have bragging rights around training, but inferencing is where the economy's actually going to, you're going to see that explode, right? So it's where the monetization effects are going to happen. So there's a whole entire ecosystem of software, there's a lot of focus on NIM's, the announcements with Nvidia. We're partnering with a lot of different ecosystem players that are helping operationalize a lot of inference workloads as well. So, yeah, I think increasingly it's going to be around how do you help from a full lifecycle management of inferencing train models, getting those deployed through the monetization of those is going to be a big part.

>> What's the partnership with NIM's? Can you explain that and we can unpack that a little bit.

>> I mean, so NIM's is an entire solution where you want to be able to use existing data to be able to fine tune. A lot of these foundational models are becoming really, really, really smart and really good. But if you want to apply kind of domain specific relevance from your business from an organization, you have to feed it with some business specific content, right, information, so those are frameworks that are emerging to help make that simpler, right? To go tap into the enterprise and the strobes of data that they have in their environments make that more tailored for their business. Not even to the organization, but it could be even at a departmental level within a large organization.

>> So you're saying I can basically point that infrastructure, the data that's inside of a VAST object store.

>> Right. >> And then essentially create my own sort of LLM, if you will, but that's very domain specific.

>> Right.

>> So this would be apply whether it's financial services, I could see use cases in healthcare.

>> Healthcare is one.

>> I mean, virtually any industry.

>> I think so, I mean they gave some examples of it, but yeah, absolutely taking, I was talking with a large system integrator the other day, but they are by definite an information environment from how they handle contracts to how they develop knowledge, right? They're like that is their secret sauce. So being able to capture that content that's been stored in archive forever, feeding that in to fine tune and improve these is going to be very, very powerful.

>> So this show, I think this show actually will mark a milestone because there was no GTC really after COVID. This was the first one after COVID, pretty amazing actually. You know most conferences, I'm sure you remember late 2021, even 2022. Ah, isn't it great to be back in person and we all cheer. And now it's 2024, people aren't saying that anymore. But I think this conference, I'd love to get your thoughts as sort of final question, really marks a new milestone in the AI era. It touches virtually every industry, it's not just a bunch of technology companies here-

>> Right. >> Presenting. I mean, I guess the old bromide, every company's a technology company. It's really proving out here. I've never seen a more talked about event. I mean you see things like Dreamforce and some of those other big conferences and CES and others. You know but I mean it's still going on the TV shows and the business channels and it really is, I think, marking a new era for our industry.

>> The cliche, I think it was used by Jensen last year, right, it was like the iPhone moment has arrived and I think it's so true. I mean, the funny, so anecdotally, right? I haven't actually spoken with or met with a university or a research center this entire week, right? It's with like private sector enterprises that are trying to figure out how to infuse their business with AI and possibly either save costs or make more money or increase productivity, whatever it may be. So that is a clear inflection points happening in the industry and it's pretty exciting to see.

>> All right, John, thanks so much for coming on.

>> Thanks for having me,

appreciate it. >> We appreciate it. All right, keep it right there. We'll be back for more action from San Jose, David Vellante and John Furrier from GTC '24, you're watching "theCUBE." (bright pop music)