AWS Financial Services Symposium | Malcolm deMayo

Clips
News
More from AWS Financial Services Symposium

Malcolm deMayo

Global Vice President, Financial Services Industry

NVIDIA

Three insights you might have missed from theCUBE’s coverage of the AWS Financial Services Symposium

Cloud spending in the financial services industry has been on a growth curve, and Amazon Web Services Inc. has gained momentum in this sector. For financial services companies, broader cloud adoption has led to a heightened focus on data and new ways to organize and manage vast amounts of information to create intelligent solutions using AI.How the financial world is leveraging data to transform operations and increase customer value was a central focus of theCUBE’s coverage of the AWS Financial Services Symposium. TheCUBE, SiliconANGLE Media’s livestreaming studio, provided insights through interviews with company executives, customers and partners. (* Disclosure below

The dawn of AI factories: Nvidia’s full-stack platform tackles AI demands in financial services

Generative artificial intelligence is set to revolutionize the financial services industry, marking a confluence of technology and business operations. While the sector has utilized AI for decades, factors such as the rise of AI factories, accelerated compute platforms and purpose-built infrastructures signal a technical expansion by orders of magnitude.Discussing gen AI for financial services with Nvidia’s Malcolm deMayo.“There is a tsunami-sized wave of innovation, disruptive innovation,” said Malcolm deMayo (pictured), global vice president of the financial services industry at Nvidia Corp

play_circle_outline Impact of AI on improving productivity, operational efficiency, and revenue streams

play_circle_outline Importance of transforming data centers into AI factories for financial services industry

play_circle_outline Need for new hardware, software, and development platforms for AI implementation

play_circle_outline Ability for financial services firms to adapt and integrate AI technology or risk becoming extinct

play_circle_outline Navigating Compliance and Regulation: Tackling AI Implementation in Financial Services while Addressing Job Loss Concerns

Info

Malcolm deMayo | NVIDIA

Malcolm deMayo

Global Vice President, Financial Services Industry NVIDIA

>> The AWS Financial Services Symposium, presented by theCUBE.

>> Welcome back to theCUBE's coverage here in New York City. I'm John Furrier, your host of theCUBE here, and it's Tech Week. Amazon Web Services is having their financial symposium here in Manhattan. We've got a great guest, Malcolm deMayo, Global Vice President Financial Services Industry with NVIDIA. Pleased to have you on, Malcolm. Thanks for coming on. And again, congratulations to everyone at the NVIDIA team and company for just the rise of greatness on a performing stock, company, software, hardware. It's a systems revolution. You guys are leading the way, so props to you guys.

>> Yeah, thank you, John.

>> It's great to be here with you. So we're now living in a world where we're going to look at this moment in time and say, "We have now crossed over an inflection point of massive platform shift."

>> Hardware cycle.

>> Gen one cloud. We saw it happened, SaaS happened. Beautiful things are happening. Now with generative AI, you're starting to see, and with all the success of these new workloads, net new categories emerging. Generative AI. Jensen Huang, your CEO, said on stage, "This is a new category. It's generating things." So you're starting to see an entire application market switch over from pre-existing workloads are getting retrofitted with generative AI to new ones coming online. This is going to be a Cambrian explosion of workloads.

>> Excellent.

>> But under the hood is the systems that run it.

>> That's right.

>> And you guys are part of that, not just GPUs.

>> Correct.

>> It's a collection of things.

>> Absolutely.

>> How do you guys feel right now? What's the impact on the financial services industry as you look at the landscape with now this accelerated disruptive enablement? It's going to disrupt, but it's going to enable. So that's a good thing.

>> So John, financial services, as an industry, has been working with AI for a couple of decades. So AI is not new to financial services. They have put in place multiple lines of defense, very good governance. They've calibrated their control framework in order to make sure that they're using AI responsibly and safely. Generative AI, as you just correctly pointed out, is not another tool in the toolkit. It is a tsunami sized wave of innovation, disruptive innovation. And in financial services, there are, between the front office, middle office, back office, there are literally hundreds of opportunities for firms to leverage generative AI to improve productivity, to improve operational efficiency, and to drive new revenue streams. But they can't do that on their existing systems. They need to transform their data centers, as Jensen likes to say, to AI factories. And an AI factory is a combination of a completely new stack of systems, and that's hardware, software, and development platforms. And it's also leveraging open source models that or APIs, or open APIs. And then finally it's having the stack in place. And so what NVIDIA has done is we've built a complete full stack platform to help financial services to tackle this.

>> It's interesting, it's like before we came on camera, you were saying, "There's now more workloads." AI is just a workload. It's a big workload. It eats a lot of minutes as they say on the GPU clock. But it's not just the GPUs. You're seeing the packaging of systems. AI factories isn't your data center of yesterday. It's different. It's still kind of a data center. It's just hosted cloud, maybe on premise or hybrid. But it's changing how that infrastructure is going to work. The game is still the same, provide as much compute as possible-

>> That's right....

>> to power the software ultimately, and data software are emerging. So in this nexus, what is going away? So disruption means disruption. Enablement means value creation. Okay, so great. Disruption, what gets disrupted? What goes away? What comes in?

>> Yeah. No, it's a great question. The first thing that goes away is yesterday's computers and what's happening under the covers, and this has really been obfuscated by cloud. Cloud has really saved the day. Moore's law, or maybe said differently CPU scaling, has ended. And if you look at how Jensen presents this, we have delivered 1000X speed up in the last eight years in our Accelerated Compute platform. Compare that on a chart with CPU, it's almost a flat line. CPU scaling has almost flat lined, almost not living. And so when you have software as demanding as generative AI is, models have grown 3,600X in the last 10 years, data has exploded to some of our customers have exabyte size data sets, and the transaction volume just continues to grow. So as we build a faster Accelerated Compute platform, the software becomes more demanding. Yesterday's computers are going to go away, and firms have to decide where are they going to do this? Are they going to do this in their data centers or are they going to do this in cloud?

>> Well, Malcolm, I'll tell you, we are totally on that religion as well. One of the things that we are speculating in our thesis on our CUBE research team is that as you guys and the industry gets that acceleration hardware software combination at the infrastructure chip and infrastructure level going faster and getting that up to speed to a steady state, it's going to flip to the next layer of of the stack, which is the data layer. The middleware now becomes data and software, right? Okay, so then you're going to see change there. So financial services has always been data-centric.

>> Oh, yeah.

>> I mean, data, data, data. But it's also a cutthroat market. You got to survive. It's highly competitive. Data will be that piece. As you look at the landscape, how are the companies dealing with this opportunity to use their data not as an ingredient to some app, but natively in the application itself? Because they're in the data business, the products are data. So as the world starts to have a product management view of data, it just changes a little bit the scope of what they do. But the impact is cash in the bank for them, getting more customers, where data becomes the product. What's your reaction to that?

>> Well, data's always been the problem. Has always been one of their challenges. It's not a new challenge. It was a big challenge in the '90s when I was at Teradata. Having a high-quality data set is super important in order to be able to have a high-quality model, AI model. And the leading firms have been working hard on this. This isn't a new challenge. They've been working hard on it. What are they doing? They're building these large AI-ready data sets that they can train with. What's an AI-ready data set look like? Today, one of the big differences is yesterday, yesterday being in the last couple of decades, it was tabular data. This was numeric data for the most part. Now, with generative AI, which understands text, understands video, understands images, is multimodal, you can look at alternative data. For example, in the trading industry, in the trading segment, traders have been using tabular data for a decade or so to look for signals, profit signals, which they call alpha. And then they pursue this. This is how they build... And it is a zero-sum game. So somebody wins, somebody loses. But now they have the opportunity to use a large language model to do the same thing, to identify correlations that none of us would ever notice in large, vast volumes of unstructured -

>> We had a customer on Amazon before you came on, S&P 500 team, global insight, energy, oil and tech. Killer app they built with on Amazon. GPUs were a big part of that to accelerate that platform on the inference side. But that highlights what you're getting at it, which is the AI is the forefront of this content app creation, because data is now content. Again, Jensen, your CEO, talks about content. We're in generative content mode, token economies using tokens.

>> If you think about that, back to your previous question, what's going away? The software that we use today, and Jensen correctly points this out, is instruction-based. It's coded by a half a percent of the world's population. And these rules are-

>> Pre-programmed.

>> They're inflexible. If you want to change something, it doesn't learn on the fly. You have to go in and actually make the changes. Generative AI not only generates content, but learns in an unsupervised way. So learns from feeding it data. And so another thing that you'll see happening is this new way of creating software is going to... It's going to start, we're going to start by paving over or optimizing today's workloads. So we're going to do things the same way with a new technology. But you're going to see startups, we're seeing startups, that are already building AI-native solutions.

>> And that tech stack is looking different than what it was.

>> And it has to because we have a worldwide power crisis. And one of the other things that Jensen will put up is, and we're very proud of this, is that in the last 8 to 10 years, the Accelerated Compute platform has reduced the amount of power needed to drive a workload by 350X. 350X. So let me give you an example of what that looks like. In 2012, we built a supercomputer called, I think it was called Atlas or Titan, maybe Titan. 17,000 GPUs. Took up a tennis court. 10 megawatts-ish of power, a little under 10 megawatts of power and 20 petaflop.

>> Nuclear power plant bolted onto it.

>> 20 petaflops of AI performance today, Blackwell, which is now in production, it's this big, all right? Blackwell consumes several kilowatts of power, takes up this much room. That's how far things have come, 350X reduction in the use of power. And so that's going to enable organizations to do more with less. And as we like to say, the more you buy, the more you save.

>> That's transformational to all potential everyday users. That is just the impact of that scale it brings it. And this really highlights, to me, what's revolutionary about this industry right now, and to your point, it's going to revolutionize the interactions that users have with supercomputers because now we're seeing supercomputing is mainstream. It's in the cloud. It could be a set of clustered systems.

>> Available to everyone on the planet.

>> And so imagine now technology coming on, augmenting human capability, and then new apps coming in and app creation that leverages supercomputing. So now we're all going to have access to the supercomputers. So we believe that this will change the application frameworks in today's FinTech, the ability to integrate, share data. So as we look at this market, financial services-

>> You got it, Mike....

>> there's going to be winners and there's going to be losers. And if you're not leaning into AI... When platform shifts happen, you can't lean in halfway. You go all in or you're extinct.

>> What an analogy that works for New Yorkers, at least, John. We all jump on a train in the morning and come into the city. If you're standing on the platform and you're not on the AI train, the train's moving really fast. And it's just when it goes by, it's incredibly fast. It's hard to keep up with it. But when you're sitting on the train, you don't notice how fast it's going because you're participating. You're learning, you're developing AI literacy. You're experimenting, you're building solutions for your customers, both external and internal. And the leading firms, the ones that come to see us in Santa Clara... And by the way, everyone should come to see us in Santa Clara. They should schedule an executive briefing. When they come to see us, they've decided, the companies that come to see us have decided we are leaning in. We're going to be leaders in this. We're going to be the survivors. Capgemini did a study in 2017. It looked at the Fortune 500 in 2001, and who was left standing in 2017. More than 50% of the companies that were in the Fortune 500 in 2001 were gone. That era, that 17 years, you saw internet, you saw mobile, you saw cloud, but they were moving at a glacier pace compared to how fast generative AI is moving. And in financial services, it's no different.

>> Yeah, I totally think you're on point there. In fact, we use a lot of sports analogies on theCUBE here. It's like the pace of play between little league and pro ball. The pace of play at AI right now is really fast. So companies are adapting and if they don't, they're going to be in the wrong side of history. So as we get into this world, old way, new way, okay, your relationship with AWS is well documented. Also well documented in what you guys are doing separately from Amazon and on premise, which is booming. All these specialty clouds are emerging, which we think can be the future. At the end of the day, it's cloud operations. Amazon might not see it that way, but AWS has a great relationship with NVIDIA. Talk about the relationship with NVIDIA.

>> Oh my God. So Jensen said it best at re:Invent in November. Jensen and Adam said it best on stage during re:Invent back in November. We've had a 13-year relationship of collaboration with AWS. First of all, take a step back. AWS is obsessed with improving customer outcomes, and NVIDIA is obsessed with solving really tough problems. So we're very well aligned from a cultural perspective. And when you think about just our Ampere and Hopper, Ampere AWS calls it Pd4s, and Hopper, they call them Pd5s, and they're available in their EC2 instances and have been for years. Just those two alone, they've invested in over 2 million GPUs in their cloud. That is an enormous investment for their customers. So if I'm a financial services firm and I have to decide, do I build on my prem in my data center, I have to over-provision for peak.

>> And there's power .

>> And there's, well, a lot of data centers are old, so that's an issue. Or do I just fire this up in Amazon? And so they have that opportunity because Amazon's made it available and they've taken the risk. They've taken the approach of if I build it, they will come.

>> That's a good bet, by the way. On the CapEx side, they got a lot of horsepower and firepower there for customers.

>> Exactly. But since then, at re:Invent and then again at GTC, Jensen and Adam kicked up our relationship with several notches. They announced Ceiba. Ceiba will be one of the largest supercomputers based on DGX cloud. And Ceiba, by the way, is the largest tree in the Amazon rainforest. So appropriately named. A massive supercomputer in AWS's cloud. And our R&D team will use it. We'll use it for advancing AI for robotics. We'll use it for our Earth-2 digital twin and climate predictions. And by the way, very relevant in financial services. We'll use it for our Omniverse platform. We'll use it for so many different things. And in addition to that, we're integrating our NIM, our NVIDIA Inference Microservice with SageMaker. NVIDIA Inference Microservice is us taking essentially an open source model, encapsulating it in a Kubernetes container with the inference runtime, the model, and CUDA, and essentially now you have a portable model that's optimized to run on our platform. Maybe think of it this way. If you're a dad, you know this, you've got a home, you've got kids. They leave the windows open, they leave the lights on, drives every dad mad. They run around, turn everything off. What we're doing when we optimize these models is we're turning off parameters that aren't needed. So that saves money in training time, that saves money in inference time, that saves money in the entire pipeline and lifecycle or AI pipeline. And so NVIDIA Inference Microservices with SageMaker allows Amazon's customers to build their own AIs, to own their own IP. And now you're multi-cloud portable.

>> And it's interesting, you used the word AI factory, which is you guys going Dell jumped on the bandwagon there too, which basically I thought was a great move on their part. And in effect, it doesn't have to be a physical location. The cloud is a big part of these AI factories. NIMs are components you connect together so that we're going to be living in this connected new system. I call it this new AI world we're living in. It's a collection of systems working together.

>> Absolutely.

>> That's a NVIDIA also vision.

>> Today in financial services, probably less than 15% of workloads run in cloud. There is still a huge amount of tech debt in financial services. You have these systems, some financial firms, data centers look like the Smithsonian. They have one of everything from every decade. And I've heard people describe their infrastructure as like the layers of the earth. You can go and see exactly what decade was this technology brought in. As generative AI changes the way software is developed and empowers financial services firms to now write new generative software, you're going to see more and more of this probably run in cloud.

>> And I will say, and to agree with you and put an exclamation on that point, is that the winners have to deliver the consumer experience because generative AI is a new categorical experience.

>> That's right.

>> That's obvious as the world in front of us. And if you're not showing that new capability, you're out.

>> You're out.

>> Literally the consumption won't be there.

>> Fortune 500 list, I think 2001, 2017, 50% of the firms are gone.

>> As the fog lifts in this new era, you can start to see the old way, new way. And the new way is that if you're not getting it here, you're going to be out of business. That's what's happened.

>> And financial firms have to be careful, John, because they're heavily regulated and generative AI's still relatively new. So they don't have to rush things into production. But what they do need to be doing is building an AI factory in cloud or on premise, or both. You can build a hybrid AI factory. And they need to be building AI literacy across their organizations. And taking the fear out of their people who think this is going to take their jobs. Jensen made a great observation a year and a half ago, and you hear everyone repeat this now, you're not going to lose your job to AI. You're going to lose your job to somebody who's learned how to use AI.

>> Yeah. And then you're going to lose your job by not leaning into AI right and jumping on the train. Malcolm, great to have you on from NVIDIA. Laying it out, NVIDIA leading the way. Glimpse into the future is here. We're bringing you all the data here in New York City with theCUBE. I'm John Furrier, your host. Of course we're going to tokenize the data, we're going to make it generative. Hope you had a good conversation. Thanks for coming on.

>> Great to be here with you.

>> We'll be right back after the short break.

Malcolm deMayo | NVIDIA

>> The AWS Financial Services Symposium, presented by theCUBE.

>> Yeah, thank you, John.

>> It's great to be here with you. So we're now living in a world where we're going to look at this moment in time and say, "We have now crossed over an inflection point of massive platform shift."

>> Hardware cycle.

>> Excellent.

>> But under the hood is the systems that run it.

>> That's right.

>> And you guys are part of that, not just GPUs.

>> Correct.

>> It's a collection of things.

>> Absolutely.

>> That's right....

>> Oh, yeah.

>> Pre-programmed.

>> And that tech stack is looking different than what it was.

>> Nuclear power plant bolted onto it.

>> Available to everyone on the planet.

>> You got it, Mike....

>> there's going to be winners and there's going to be losers. And if you're not leaning into AI... When platform shifts happen, you can't lean in halfway. You go all in or you're extinct.

>> And there's power .

>> That's a good bet, by the way. On the CapEx side, they got a lot of horsepower and firepower there for customers.

>> Absolutely.

>> That's a NVIDIA also vision.

>> And I will say, and to agree with you and put an exclamation on that point, is that the winners have to deliver the consumer experience because generative AI is a new categorical experience.

>> That's right.

>> That's obvious as the world in front of us. And if you're not showing that new capability, you're out.

>> You're out.

>> Literally the consumption won't be there.

>> Fortune 500 list, I think 2001, 2017, 50% of the firms are gone.

>> As the fog lifts in this new era, you can start to see the old way, new way. And the new way is that if you're not getting it here, you're going to be out of business. That's what's happened.

>> Great to be here with you.

>> We'll be right back after the short break.