NVIDIA GTC 2024 | Leo Leung | Oracle

Clips
News
More from NVIDIA GTC 2024

Leo Leung

Vice President, Products and Strategy

Oracle

Oracle meets AI demands with cutting-edge database technology

Oracle Corp. is declaring itself a competitor alongside major hyperscalers as companies look for large-scale databases that can meet the demands of artificial intelligence.“The business has progressed from those mission critical types of workloads, whether it’s financials or supply chain or transactions, into the AI space,” said Leo Leung (pictured), vice president of OCI and technology at Oracle. “We’ve had a great few years running these big AI workloads for companies that are training these new models.”Leung spoke with theCUBE’s chief analyst, Dave Vellante, at the Nvidia GTC event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio

play_circle_outline Discussion on Oracle Cloud Infrastructure (OCI)

play_circle_outline Evolution of OCI from mission-critical workloads to AI

play_circle_outline Unlocking the Power of AI: Oracle's Comprehensive Cloud Infrastructure and Data-driven Services for Enterprises

Info

Leo Leung | Oracle

Leo Leung

Vice President, Products and Strategy Oracle

search

(bright music) (air whooshing)

>> Welcome back to San Jose. We're here at GTC '24. People are calling it the Woodstock of AI. It's not really Woodstock, but it's the business of AI. It's all happening here. This is "theCUBE." I'm Dave Vellante. Leo Leung Young is here. He's the vice president of OCI and technology at Oracle. Leo, it's great to see you again. It's been a while.

>> Yeah. It's great to see you, Dave.

>> Yeah, so the journey of OCI has really, has been misunderstood. A lot of people will go back and say, "Well, Oracle poo-pooed the cloud." Like, well, not really. Larry just said, "Look, the cloud is databases and servers and storage and software," which is all true, but, of course, it takes a while to get that right. But judging from the results, you guys got it right. Give us your version of the OCI story.

>> Yeah, I mean, it really started in 2016. So we launched what's now called Oracle Cloud Infrastructure back then. And I still remember the days when it was a few hundred engineers, and we were starting from nothing, and now it's a $6 billion run rate business.

>> Yeah, so, I mean, I get a lot of people, I've been following Oracle for a long time. I have great respect for their security, their mission criticality. As I look at it, you've got a highly differentiated strategy. You're essentially, you know, you're not really doing the commodity cloud thing. It's really, I think of it as the mission critical cloud, the database cloud, so Oracle applications cloud, the ecosystem around that, but is that a fair description in understanding that it's that really those mission critical workloads where you guys differentiate and focus?

>> Yeah, that's definitely where we started, which is would be insane for a startup to start with the most critical workloads that our customer has, but that's how we started. And what's really interesting is the business has progressed from those mission critical types of workloads, whether it's financials or supply chain or transactions, into the AI space. That's kind of why we're here, is that we've had a great few years running these big AI workloads for these companies that are training these new models, whether it's a Cohere as well as other customers that are training these models, as well as customers that are using us for inferencing, including Microsoft themselves is using us for inferencing, for Bing. So from this kind of database, as you say, critical application cloud to now AI cloud, it's been an interesting evolution.

>> Well, the database technology has become really interesting over the last 10 years, right? It was kind of getting boring for a while. You guys have dominated that business. But the database technology has morphed and has included a lot of different data types, a lot of different formats. You guys have introduced MySQL HeatWave, which we've written about and talked about a lot. But talk to me about the database itself and why that is so important in terms of the center of the universe, if you will? 'Cause it supports virtually everything, including the autonomous activities that you're building, the applications that you support, and now AI.

>> Yeah, well, obviously, this is the AI conference, as you say, but if you go to just about any session, one of the big topics is data, right? And where that data is, the control over that data, where it's located, and so much of that critical data is still within databases, right? So we see it as a really interesting opportunity to take that converge approach that you talked about, where, again, a lot of the world's companies store their transactional critical mission, critical data inside of Oracle Database, and now they want to be able to do AI on that data, right? And to be able to, for example, use vectors to go anonymize that data and make it available to models in order to provide that proprietary internal data to these models so that they can serve up the best results. We see that as the next trend, and, again, our position as a database company, as a cloud company for critical workloads puts us, I think, in a really good position to go address that.

>> Well, and vector's a good example of how you're extending, and you're not the only one, but other database suppliers are extending, bringing vector embeddings into their database. We've always said, it feels to us like a feature, not really a market. Would you agree with that?

>> I mean, we think of it as an important technology where, again, we want to make it simpler for customers to be able to access that capability without deploying yet another database in their infrastructure, right? I mean, it's interesting, when I talk to customers, they themselves would admit, "Well, we have about a dozen database technologies or data management technologies in our enterprise, and, honestly, we would love to actually simplify that scope a little bit," right? We think we have good solutions for that, and that's how we think about it as well, is you want access to that data, you want the control over that access, you want to be able to locate it, and that vector capability, something you want to add on, right, to an existing data store versus, again, implementing something else.

>> So how should we think about the Oracle AI stack? You don't make your own chips or you don't design your own silicon, but you pretty much (chuckles) have AI everywhere else. So how would you describe that stack?

>> Yeah, it's very much a complete sort of enterprise stack where we have it in our infrastructure. We have it in our data platform. We have AI services, for example, our new gen AI service, as well as things like vision, computer vision and document understanding, that type of stuff, and it's embedded within our SaaS and our applications, right? So you can think of it that way. Another way to think about it is the different types of business users that are going to need AI and want to interact with AI. We can go everywhere from the builders that are building the foundation models and training them to the developers that are trying to add AI capabilities to their applications, all the way up to the end users that really don't care about any of that stuff. They just want better job descriptions or an easier performance review process, right? Where the AI is just under the covers, powered by the stack. They could care less about GPUs, that type of stuff.

>> So there's a big gold rush for getting GPUs. We've seen it. I mean, all the big three hyperscalers. No offense, I don't put you in the hyperscaler.

>> Oh, we got to change that. (chuckling)

>> Well, we should talk about that. We will in a second because I get a lot of grief from that, from my friends at Oracle. Like, "What? Define a hyperscaler." I'm like, "Ah, (babbling)." But anyway, we'll come back to that. But you're in that mix. We saw your CapEx bouncing up last quarter. Your guidance was more CapEx, so that's clear indicator of demand for GPUs. You, as I started to say, you don't design your own chips, but you guys, you got one of everything. You got Intel, you got AMD, you got ARM with Ampere. You're doing stuff with Nvidia. It's horses for courses, right? So how should we think about your strategy with regard to chips and silicon and all this alternative processing that's been going on?

>> Yeah, I mean, it's very much a best of breed type of strategy, right? I think that certainly other providers are getting into the chip design business, but, in reality, what the customers are looking for is they tend to be looking for Nvidia, right? So one of the announcements we had yesterday was expanding our partnership with Nvidia to adopt their new Grace Blackwell technology, right? And I think they're still the cutting edge, right, when it comes to both the GPU as well as their super chip type of architecture. Some of the stuff I've seen about it is really amazing because, again, having one company really optimize across those different types of chip sets and provide that solution, that's something that really only a company like Nvidia could do, and we just get to take advantage of that technology with our services and our infrastructure on top of it.

>> Yeah, so Jensen in his keynote yesterday basically said, "We need bigger chips." Is that how you see it? Bigger AI chips, that's what your customers are demanding?

>> Some of them, yeah, right? So some of them are using what we call our super cluster, which is tens of thousands of GPUs in a single system, right? And certainly if you look at that, and you look at the architecture that Blackwell is, you're going to get way more power with way more power efficiency, right? And as a CSP, we do care about that kind of stuff, right? There are other customers that don't have those requirements. I think the majority of customers don't really need that. They may need a slice of it, they may need it for inferencing and real-time inferencing. And, again, when you look at whether it's Grace Blackwell or even their older generation of technologies, really, really great stuff to feed that inferencing, to feed the use cases that customers have. So in some cases, yeah, that ridiculously massive machine that he had on stage is the right solution. In other cases, it's a slice of that technology.

>> So because he talks about that. He said, "We either sell systems..." He said, "Basically, we build systems and then we break it apart," and then they sell 'em as different SKUs, piece parts. My question is, how should we think about training an inference? Are they different spaces? I almost infer from listening to Jensen that the same infrastructure can do both. I've also heard him talk on earnings calls, like, "Today's training GPUs will be tomorrow's inference GPUs." And I'm like, "Hmm, does that mean it's a depreciated asset and you're just rolling it down, or is it really the same infrastructure?" And then, of course, I think of my iPhone. It's got inferencing in there. It's got GPUs. And I think about the edge and low power and low cost. So I'm trying to squint through that. Can you help me understand how you see the different markets and the different requirements for those different workloads?

>> Yeah, I mean, I think there's going to be the very, very large scale consumer-facing types of inferencing products, right? Like your ChatGPT, or a customer we have, SoundHound, which takes commands from a Mercedes or a Hyundai and turns it into, "Okay, you're going to play that song," right, or, "You're going to give those directions." There's going to be a different scale of that versus an enterprise, which is going to have a few thousand end customers, 10,000 end customers. They're not going to need this massive cluster behind their inferencing engine, right? So I do see it as a scale where, yes, some of the very biggest consumer-facing products, absolutely, they're still going to require the massive Grace Blackwell type of machine behind the scenes, but there's many, many others that won't. And they'll hit the, as you say, commoditization curve and be very happy with a virtualized A100 or H100 behind the scenes, right, and the end user won't know the difference, right? In some cases, the use case may be more real time, where you literally want that super, super fast response. Again, you're going to go up your scale to the most advanced, most highest horsepower type of platform, and you're going to be willing to pay the price, right? In other the cases, maybe not.

>> Okay, let's have that academic discussion about hyperscalers, so, and again, if you asked me to define it, I'd be stumbling and bumbling, but let's try to... So you would agree, AWS, Google, and Azure, hyperscalers. Why should, and I'm open-minded to this, put Oracle in that camp? Why should we be sort of counting Oracle as a hyperscaler? I mean, you got a legitimate cloud. No question about it. You've done the engineering work, and (chuckles) it's a really good cloud. Same, same, you guys were the first to have cloud customer and the exact same homogeneous infrastructure in the cloud. Huge benefits for customers for consistency, ease of migration, data movement, et cetera, et cetera, et cetera, but why are you a hyperscaler?

>> Well, a few different reasons. One footprint, right? So global footprint, we're roughly equivalent. You could argue we actually have more regions and more presence in more places than anybody else. That's number one. Number two, the types of customers we have now. So again, maybe a few years ago you could say, "Well, not really." Now we've got customers like Uber that are running on us, right? And you would say that Uber is a mega scale type of company that is serving users all around the world, right? You can't serve a customer like that unless you're a hyperscaler. Three, we do offer things like our super cluster, where we have customers today that are using computers that take up an entire data center, right? Over 30,000 GPUs in a single machine to serve a customer. So by those types of attributes, I would say that we're in that mix.

>> So I'm always in the interesting position of defending Oracle because everybody loves to like pound on Oracle, "Ah, prices are too high," et cetera, et cetera, but I recognize the quality of the infrastructure. I talk to a lot of Oracle customers, a lot of Exadata customers. They're actually really happy. (chuckles) They buy the next ones site unseen because it just prints money for them, when you talk to big banks and the like. But my antagonist, Charles Fitzgerald, I don't know if you know Charles.

>> Charles, yes. >> Fitzy, we call him. Says there's no comparison because of the CapEx. So you may have more footprint than anybody, but it's the CapEx men that determines a hyperscaler. So I ask him, "Well, what's the level at which you cross over?" And there's not really a good answer there, but is CapEx an indicator, or is that sort of a false positive?

>> I think at some level, and I think we're there as well, right? When you look at the latest earnings, we've been hovering around the seven, eight billion mark, and based on the latest earnings, it's going to grow from there, right? So you can't say we're not spending money, right?

>> No, listen, when you have a chairman and founder-led company who understands and appreciates technology and insists that you invest in product development, it shows.

>> Yeah, and that's purely for our cloud, right? We don't have warehouses. We don't have a search business. We don't have a gaming business. We don't have an office productivity business.

None of that, right? >> Yeah, yeah. Right.

>> So this is pure cloud infrastructure and services for enterprises.

>> Well, Oracle stocks at close to an all-time high. You guys are pumping 'em all cylinders. You got the AI going. You got the autonomous database. You're still the database king. Leo, so much for coming back on "theCUBE."

>> Yeah, thanks. Great to see you.

>> Oh, great to see you. >> Thanks for having me.

>> You bet. All right, keep it right there. More action from GTC '24. This is Dave Vellante. John Furrier's in the house. You're watching "theCUBE." (bright music)

Leo Leung | Oracle

search

(bright music) (air whooshing)

>> Welcome back to San Jose. We're here at GTC '24. People are calling it the Woodstock of AI. It's not really Woodstock, but it's the business of AI. It's all happening here. This is "theCUBE." I'm Dave Vellante. Leo Leung Young is here. He's the vice president of OCI and technology at Oracle. Leo, it's great to see you again. It's been a while.

>> Yeah. It's great to see you, Dave.

>> Yeah, so the journey of OCI has really, has been misunderstood. A lot of people will go back and say, "Well, Oracle poo-pooed the cloud." Like, well, not really. Larry just said, "Look, the cloud is databases and servers and storage and software," which is all true, but, of course, it takes a while to get that right. But judging from the results, you guys got it right. Give us your version of the OCI story.

>> Yeah, I mean, it really started in 2016. So we launched what's now called Oracle Cloud Infrastructure back then. And I still remember the days when it was a few hundred engineers, and we were starting from nothing, and now it's a $6 billion run rate business.

>> Yeah, so, I mean, I get a lot of people, I've been following Oracle for a long time. I have great respect for their security, their mission criticality. As I look at it, you've got a highly differentiated strategy. You're essentially, you know, you're not really doing the commodity cloud thing. It's really, I think of it as the mission critical cloud, the database cloud, so Oracle applications cloud, the ecosystem around that, but is that a fair description in understanding that it's that really those mission critical workloads where you guys differentiate and focus?

>> Yeah, that's definitely where we started, which is would be insane for a startup to start with the most critical workloads that our customer has, but that's how we started. And what's really interesting is the business has progressed from those mission critical types of workloads, whether it's financials or supply chain or transactions, into the AI space. That's kind of why we're here, is that we've had a great few years running these big AI workloads for these companies that are training these new models, whether it's a Cohere as well as other customers that are training these models, as well as customers that are using us for inferencing, including Microsoft themselves is using us for inferencing, for Bing. So from this kind of database, as you say, critical application cloud to now AI cloud, it's been an interesting evolution.

>> Well, the database technology has become really interesting over the last 10 years, right? It was kind of getting boring for a while. You guys have dominated that business. But the database technology has morphed and has included a lot of different data types, a lot of different formats. You guys have introduced MySQL HeatWave, which we've written about and talked about a lot. But talk to me about the database itself and why that is so important in terms of the center of the universe, if you will? 'Cause it supports virtually everything, including the autonomous activities that you're building, the applications that you support, and now AI.

>> Yeah, well, obviously, this is the AI conference, as you say, but if you go to just about any session, one of the big topics is data, right? And where that data is, the control over that data, where it's located, and so much of that critical data is still within databases, right? So we see it as a really interesting opportunity to take that converge approach that you talked about, where, again, a lot of the world's companies store their transactional critical mission, critical data inside of Oracle Database, and now they want to be able to do AI on that data, right? And to be able to, for example, use vectors to go anonymize that data and make it available to models in order to provide that proprietary internal data to these models so that they can serve up the best results. We see that as the next trend, and, again, our position as a database company, as a cloud company for critical workloads puts us, I think, in a really good position to go address that.

>> Well, and vector's a good example of how you're extending, and you're not the only one, but other database suppliers are extending, bringing vector embeddings into their database. We've always said, it feels to us like a feature, not really a market. Would you agree with that?

>> I mean, we think of it as an important technology where, again, we want to make it simpler for customers to be able to access that capability without deploying yet another database in their infrastructure, right? I mean, it's interesting, when I talk to customers, they themselves would admit, "Well, we have about a dozen database technologies or data management technologies in our enterprise, and, honestly, we would love to actually simplify that scope a little bit," right? We think we have good solutions for that, and that's how we think about it as well, is you want access to that data, you want the control over that access, you want to be able to locate it, and that vector capability, something you want to add on, right, to an existing data store versus, again, implementing something else.

>> So how should we think about the Oracle AI stack? You don't make your own chips or you don't design your own silicon, but you pretty much (chuckles) have AI everywhere else. So how would you describe that stack?

>> Yeah, it's very much a complete sort of enterprise stack where we have it in our infrastructure. We have it in our data platform. We have AI services, for example, our new gen AI service, as well as things like vision, computer vision and document understanding, that type of stuff, and it's embedded within our SaaS and our applications, right? So you can think of it that way. Another way to think about it is the different types of business users that are going to need AI and want to interact with AI. We can go everywhere from the builders that are building the foundation models and training them to the developers that are trying to add AI capabilities to their applications, all the way up to the end users that really don't care about any of that stuff. They just want better job descriptions or an easier performance review process, right? Where the AI is just under the covers, powered by the stack. They could care less about GPUs, that type of stuff.

>> So there's a big gold rush for getting GPUs. We've seen it. I mean, all the big three hyperscalers. No offense, I don't put you in the hyperscaler.

>> Oh, we got to change that. (chuckling)

>> Well, we should talk about that. We will in a second because I get a lot of grief from that, from my friends at Oracle. Like, "What? Define a hyperscaler." I'm like, "Ah, (babbling)." But anyway, we'll come back to that. But you're in that mix. We saw your CapEx bouncing up last quarter. Your guidance was more CapEx, so that's clear indicator of demand for GPUs. You, as I started to say, you don't design your own chips, but you guys, you got one of everything. You got Intel, you got AMD, you got ARM with Ampere. You're doing stuff with Nvidia. It's horses for courses, right? So how should we think about your strategy with regard to chips and silicon and all this alternative processing that's been going on?

>> Yeah, I mean, it's very much a best of breed type of strategy, right? I think that certainly other providers are getting into the chip design business, but, in reality, what the customers are looking for is they tend to be looking for Nvidia, right? So one of the announcements we had yesterday was expanding our partnership with Nvidia to adopt their new Grace Blackwell technology, right? And I think they're still the cutting edge, right, when it comes to both the GPU as well as their super chip type of architecture. Some of the stuff I've seen about it is really amazing because, again, having one company really optimize across those different types of chip sets and provide that solution, that's something that really only a company like Nvidia could do, and we just get to take advantage of that technology with our services and our infrastructure on top of it.

>> Yeah, so Jensen in his keynote yesterday basically said, "We need bigger chips." Is that how you see it? Bigger AI chips, that's what your customers are demanding?

>> Some of them, yeah, right? So some of them are using what we call our super cluster, which is tens of thousands of GPUs in a single system, right? And certainly if you look at that, and you look at the architecture that Blackwell is, you're going to get way more power with way more power efficiency, right? And as a CSP, we do care about that kind of stuff, right? There are other customers that don't have those requirements. I think the majority of customers don't really need that. They may need a slice of it, they may need it for inferencing and real-time inferencing. And, again, when you look at whether it's Grace Blackwell or even their older generation of technologies, really, really great stuff to feed that inferencing, to feed the use cases that customers have. So in some cases, yeah, that ridiculously massive machine that he had on stage is the right solution. In other cases, it's a slice of that technology.

>> So because he talks about that. He said, "We either sell systems..." He said, "Basically, we build systems and then we break it apart," and then they sell 'em as different SKUs, piece parts. My question is, how should we think about training an inference? Are they different spaces? I almost infer from listening to Jensen that the same infrastructure can do both. I've also heard him talk on earnings calls, like, "Today's training GPUs will be tomorrow's inference GPUs." And I'm like, "Hmm, does that mean it's a depreciated asset and you're just rolling it down, or is it really the same infrastructure?" And then, of course, I think of my iPhone. It's got inferencing in there. It's got GPUs. And I think about the edge and low power and low cost. So I'm trying to squint through that. Can you help me understand how you see the different markets and the different requirements for those different workloads?

>> Yeah, I mean, I think there's going to be the very, very large scale consumer-facing types of inferencing products, right? Like your ChatGPT, or a customer we have, SoundHound, which takes commands from a Mercedes or a Hyundai and turns it into, "Okay, you're going to play that song," right, or, "You're going to give those directions." There's going to be a different scale of that versus an enterprise, which is going to have a few thousand end customers, 10,000 end customers. They're not going to need this massive cluster behind their inferencing engine, right? So I do see it as a scale where, yes, some of the very biggest consumer-facing products, absolutely, they're still going to require the massive Grace Blackwell type of machine behind the scenes, but there's many, many others that won't. And they'll hit the, as you say, commoditization curve and be very happy with a virtualized A100 or H100 behind the scenes, right, and the end user won't know the difference, right? In some cases, the use case may be more real time, where you literally want that super, super fast response. Again, you're going to go up your scale to the most advanced, most highest horsepower type of platform, and you're going to be willing to pay the price, right? In other the cases, maybe not.

>> Okay, let's have that academic discussion about hyperscalers, so, and again, if you asked me to define it, I'd be stumbling and bumbling, but let's try to... So you would agree, AWS, Google, and Azure, hyperscalers. Why should, and I'm open-minded to this, put Oracle in that camp? Why should we be sort of counting Oracle as a hyperscaler? I mean, you got a legitimate cloud. No question about it. You've done the engineering work, and (chuckles) it's a really good cloud. Same, same, you guys were the first to have cloud customer and the exact same homogeneous infrastructure in the cloud. Huge benefits for customers for consistency, ease of migration, data movement, et cetera, et cetera, et cetera, but why are you a hyperscaler?

>> Well, a few different reasons. One footprint, right? So global footprint, we're roughly equivalent. You could argue we actually have more regions and more presence in more places than anybody else. That's number one. Number two, the types of customers we have now. So again, maybe a few years ago you could say, "Well, not really." Now we've got customers like Uber that are running on us, right? And you would say that Uber is a mega scale type of company that is serving users all around the world, right? You can't serve a customer like that unless you're a hyperscaler. Three, we do offer things like our super cluster, where we have customers today that are using computers that take up an entire data center, right? Over 30,000 GPUs in a single machine to serve a customer. So by those types of attributes, I would say that we're in that mix.

>> So I'm always in the interesting position of defending Oracle because everybody loves to like pound on Oracle, "Ah, prices are too high," et cetera, et cetera, but I recognize the quality of the infrastructure. I talk to a lot of Oracle customers, a lot of Exadata customers. They're actually really happy. (chuckles) They buy the next ones site unseen because it just prints money for them, when you talk to big banks and the like. But my antagonist, Charles Fitzgerald, I don't know if you know Charles.

>> Charles, yes. >> Fitzy, we call him. Says there's no comparison because of the CapEx. So you may have more footprint than anybody, but it's the CapEx men that determines a hyperscaler. So I ask him, "Well, what's the level at which you cross over?" And there's not really a good answer there, but is CapEx an indicator, or is that sort of a false positive?

>> I think at some level, and I think we're there as well, right? When you look at the latest earnings, we've been hovering around the seven, eight billion mark, and based on the latest earnings, it's going to grow from there, right? So you can't say we're not spending money, right?

>> No, listen, when you have a chairman and founder-led company who understands and appreciates technology and insists that you invest in product development, it shows.

>> Yeah, and that's purely for our cloud, right? We don't have warehouses. We don't have a search business. We don't have a gaming business. We don't have an office productivity business.

None of that, right? >> Yeah, yeah. Right.

>> So this is pure cloud infrastructure and services for enterprises.

>> Well, Oracle stocks at close to an all-time high. You guys are pumping 'em all cylinders. You got the AI going. You got the autonomous database. You're still the database king. Leo, so much for coming back on "theCUBE."

>> Yeah, thanks. Great to see you.

>> Oh, great to see you. >> Thanks for having me.

>> You bet. All right, keep it right there. More action from GTC '24. This is Dave Vellante. John Furrier's in the house. You're watching "theCUBE." (bright music)