AWS Summit NYC 2024 | Matt Wood | AWS

theCUBE.net

info Help
Sign in
Register

Clips
News
More from AWS Summit NYC 2024

Dr. Matt Wood

VP, Artificial Intelligence Products

AWS

Navigating the generative AI revolution: AWS’ Matt Wood weighs in on possible platform shift

The generative artificial intelligence wave is not only revolutionizing the technology landscape; it’s also changing the nature of the resources under the covers that power apps. That poses the question — are we looking at a generative AI platform shift?Generative AI represents a set of technologies with unique capabilities. With that comes different benefits and limitations, according to Matt Wood, vice president of artificial intelligence products at Amazon Web Services Inc.Matt Wood of AWS talks with theCUBE about a possible gen AI platform shift

play_circle_outline The importance of generative AI in changing the nature of resources for app development

play_circle_outline Role of models in generative AI and the need for multiple models from multiple providers

play_circle_outline Building and training models in the cloud for cost, speed, and security benefits

play_circle_outline Navigating Gen AI: The Crucial Role of Mindset and Skillset for Search Teams

Info

Matt Wood | AWS

Dr. Matt Wood

VP, Artificial Intelligence Products AWS

search

>> Welcome back, everyone, to theCUBE's coverage here of AWS Summit in New York City, and we've been coming to these summits ever since they started. New York is one of the biggest ones. It rivals almost like a reinvent in the sense of impact. New York is obviously the center of action. I'm John Fuhr, your host. Matt Wood is here, Dr. Matt Wood. He was on stage giving the keynote alumni, was actually leading the effort. Strong LinkedIn games, song SoCal games, sharing a lot of content from Party Rock to deep Dive, what's going on in Gen AI? You're the leading voice banging the drum-

Dr. Matt Wood

>> Appreciate it.

>> On generative AI.

Dr. Matt Wood

>> It's good. It's good to be there. Thanks for having me.

>> Yeah. I'm following your career before Gen AI you were deeply involved in a lot of the products from IOT data all through the machine learning. We've had many talks on stage and certainly on theCUBE about it. What's different about this gen AI piece first because it's changing the nature of the resources under the covers in order to power the apps. So no matter where you came from the gen AI perspective, the apps I need to write or how do I power them, it's a changeover, it's a platform shift. Is it a platform shift?

Dr. Matt Wood

>> I think that it is a very different set of technologies that have a very different set of capabilities. They have a very different set of benefits. They have a very different set of limitations than anything else. And the way I think about it is that this ability to be able to take large amounts of information, to be able to build a mathematical formula based on the examples that you see inside that information and that inside the math is this emergent property that resembles intelligence, that resembles reasoning. It's a discovery. It's not an invention, it's a discovery. And what we've been doing I think collectively as an industry, as builders, as inventors over the past 18 months is how do you take this discovery? How do you find out what's good about it? How do you find out what's bad about it? How do you find out where it works? How do you find out where it doesn't work? And so that's what I think is just fundamentally different. Nothing like this has really existed before because it is this brand new discovery and as a result we're seeing just a remarkable amount of energy and enthusiasm and investment from customers, whether they are... Every customer I speak to is just investing very deliberately. They're experimenting a great deal. There are some customers that are going all in, they're building out and retasking all of their development engineers overnight to focus on generative AI. And then there are pockets of areas that are moving much more quickly like the regulated industries.

>> Yeah, I mean regulators must be really kind of prime because they've been using the data governance and compliance almost set up for Generative AI 'cause they did all the heavy lifting.

Dr. Matt Wood

>> That's right, exactly. So yeah, the past 20 years they've had to meet all these regulatory compliance guidelines and probably felt like a bit of a headwind at the time, but it's driven the right set of behaviors. It said, hey, you need to understand where your high quality data is. You got to understand what data is useful, who can use it, who can access it, what it can be used for, which tools it can be used for. And they've already invested in solving that problem. And as a result, the generative AI piece is actually a relatively small incremental lift on top of that and they can move much more quickly and .

>> Much more quickly. And they have the system of record data. The question this brings up then just down horizontally scaling across all industries now, taking that perspective is the role of unstructured data and structured data. Obviously structured data is almost system of record, and that's important too. And you see actually some of the use cases we've been hearing on theCUBE, even here, covalent and others are like, "Hey, I can write generative AI to convert unstructured to SQL. So you're going to see the role of unstructured and structured change, more applied use cases with structured and unstructured is just piling up everywhere. So how do you talk about the two dynamics between unstructured? Obviously unstructured gets all the attention right now, but structured is still important.

Dr. Matt Wood

>> For sure. And there's a lot of value locked away in structured data in transactional systems, in accounting systems, all those sorts of things and structured data that lives inside those unstructured documents. Could be CSV files, could be tables inside PDFs, all those sorts of things. And so a lot of what we've been doing at AWS is finding ways to be able to leverage that context where it exists inside unstructured data. So imagine a table of numbers inside a PDF. If you just let the LLM do its thing, it'll just break it down and create tokens out of it. What we do is we say, hey look, this is a table. Whenever you need information from it, we'll provide you that information and the context above and below it and the table headers and all those sorts of things. And that additional context flows into it. Now, in addition to that, the other one thing that we announced today was the ability to be able to build agentic systems. These are systems where you can set an objective and have the AI track towards that objective plan and execute towards that objective. And to be able to-

>> Agentic meaning agent like-

Dr. Matt Wood

>> Agent like, exactly.

>> Agent, agentic, play on words.

Dr. Matt Wood

>> Systems who operate autonomously and can take action on behalf of the user. One of the things we added today was the ability for agents that are running on AWS in a service we have called Amazon Bedrock to be able to not just process all that unstructured data, not just processed all the structured data within that unstructured data, but to be able to actually write code autonomously and then run the code safely in a sandbox, which is precisely designed to allow advanced analytics. So that allows you to take data from all these structured sources and not just ask for the averages and maybe make a few mistakes along the way because LLMs aren't great at math. But now you could say, "Hey, I want to do a linear regression on the data that I got from this transactional system. I'll write a price and function, add the code into the environment, add the data into the environment, build your linear regression model, graph it and expose that so you can inspect the results and use that in the next step of the age agent system." So it's just completely-

>> Well, let's get into the keynote. We were riffing on some industry landscape, which we know it's a game changer. And to you guys say needle moving moment for all companies. First of all, my first question is did you flip a coin with Swami for this one or last time he was given a keynote, he's back at the ranch holding it down, what's-

Dr. Matt Wood

>> He's back of the ranch, holding his hat, and I'm the lucky one win at this time.

>> You were here last year too, so you gave a talk as well. I want to just do a shout-out to Swami because he wasn't here. Just fun to poke at you like that. Okay, let's get into the substance we are in. You obviously laying out the vision of what's happening. It's changing every six months if not sooner, every day. What was the core theme for you this week? What were the highlights on the announcements?

Dr. Matt Wood

>> Yeah, I think the core theme is, look, this technology is probably the easiest technology that I've ever seen to prototype and experiment with. It's got a remarkably low experience bar. The utility is incredibly broad and as a result it's easy to use. You can use it for a lot of things. A lot of people are experimenting with it and that's great. You actually want to encourage that. It's actually one of the biggest indicators of success with generative AI for companies. A lot of our organizations ask me, "Well, what are the use cases?" And you can rattle them off document processing and chat bots and analysis and-

>> No agents.

Dr. Matt Wood

>> All agents, exactly. But honestly, the thing I always recommend customers to look at is deliver a mechanism internally that allows for pervasive and efficient experimentation. And let a thousand platforms bloom, let a thousand flowers bloom. Some of them will be fantastic and you want to double down on them and the others didn't work out, but you've satisfied your curiosity, which is worth something as well. And what we did in the keynote today was say, not only is it easy to be able to experiment with these, but we want to make it really easy to find the ones that are working and then to be able to not just drive into a cul-de-sac where you have something which is interesting, but basically a toy, but be able to take it as far as it can go. And to be able to dramatically accelerate not just the prototyping but the move to production and doing that with agentic systems, doing that with systems like retrieval augmentation, doing it with guardrail systems, introducing entirely new categories of ways to protect your systems, to be able to ground them with your own information and then to be able to drive automation. And that dramatically shortens not just the prototyping phase but the production rollout phase as well.

>> I want to get your thoughts and I want to dive into the models piece you talked about and then guard rails like those, and the app studio was pretty cool too, but the models, I think the big revelation to me and others, well, we already knew that. If you go back to the early days of AWS, and you remember those days, it was a SaaS market. The entrepreneurs were really a target. The benefit was I can get compute, storage and queuing building blocks with a credit card. My alternative was go get a super micro box. Well no, for certain, even the entrepreneur, I got a host of cyber super micro box somewhere or some box, a pizza box and just put it and go do Telenet in, go manage the box, get a router-

Dr. Matt Wood

>> Install it, maintain it.

>> Do a technology tax, get that done. We'll document. iPhone hits the SaaS models built. Amazon at that point was just getting going then good timing, boom, great customers next enterprises. Hey, this is good too. I have to build a data center. Friends don't let friends. And then the great time after that. Now generate AI, you're an existing huge player. So the entrepreneurial market and the developer market is a little bit different. It's not as easy as saying, hey, credit card alternative data center. The alternative is I don't know how all these databases work. So what is the complexity problem alternative for not using AWS for AI, generative AI applications. And I think the model piece speaks to the SaaS category. I think models are a new thing because you pointed out that models now need to talk to each other, which has been obvious for people in the industry for over a year that this is coming. We put a power law out on theCUBE research. So if you believe models will talk to each other and APIs connect to everything, hello, that's data, that's software, that's a whole nother thing. And that could be a new opportunity for developers, entrepreneurs or innovators to say innovate on the model, not just data aggregation. That's old school. What's your thoughts?

Dr. Matt Wood

>> I think the naive approach that a lot of folks started with 18 months ago was take one model, just pick your favorite one, pick the one that was "the best", take a prompt, give it to the model, take the output, give it to the customer. And that's a good place to get started. But it really is insufficient. And what we've seen today is a couple of things. Number one, any sufficiently large organization that is investing in this space isn't using a single model. They're not even using a single model from a single provider. They're using multiple models from multiple providers. And so our approach from day one to your point is with bedrock has been to provide as many models as possible so the customers have the choice and they're able to fit the right model for the right use case. And we see that there was something, research came out of CB Insights where 97% of enterprises that are spending millions of dollars a year on generative AI are using models from multiple providers. And that totally rings true. It's what we do with Q, is what we do with our internal systems. And so that has been a disproportionate advantage for us. As you say, it's been obvious to us for a while. It wasn't always obvious 18 months ago, and I think some competitors maybe took a different advantage.

>> But to take the example of credit card versus provisioning gear, whether it's startup or data center company, what is the equivalent problem alternative that's too complicated versus coming here?

Dr. Matt Wood

>> Yeah, I don't think it's-

>> Or that's not a fair comparison?

Dr. Matt Wood

>> I think it's a fair comparison, but it's not a question of too complicated, too simple. It's a question of incomplete to complete. And so if all you have is a model, then you can only ever do that input to output mapping. But when you talk to customers, they need multiple models. They need guardrails, they need retrieval augmentation. And I was talking to a customer this morning and they also found that they needed databases and they needed queuing systems and they needed other agentic capabilities. And so there is a merging of the generative AI capabilities with the rest of the building capabilities that we've already got on AWS. And if you are building in an environment which has fewer options with models and capabilities, it's just going to be a cul-de-sac and you're never going to be able to build any escape velocity for your product to be able to escape that.

>> And just for the record, you said on stage models will talk to each other and they'll have different characteristics, small, big, music.

Dr. Matt Wood

>> I didn't say that. Yes.

>> Okay. So let's take to the next level. I talked to a bunch of customers in our research and also through just general reporting and digging in. A lot of them have the following perspective. "I'm building my own models, I'm not going to trust anybody. I got my data, screw those guys." I'm not going to say who those guys are. They might be, it's just fear. "I'm not going to let this out. I need to control it." This is brought on resurgence to, "Hey, everything's going back to the on-premise", which has been around for certain compliance things, but most part it's more about their security around their data. So with multiple models coming in, what is the vision for you guys on going to that customer saying, "Hey, you can build and do and you can train, you can infer it and do some reinforced learning build data pipelines, do out all day long, but you can also do it in the cloud." And so what is the answer to that question because they want to have their own models.

Dr. Matt Wood

>> Yeah, I think if you want to have your own models, then you also need an environment in which to train those models and doing that on premises is expensive and slow. It's capacity constrained and it has just a terrible sustainability footprint, just terrible. And so if you believe in that you'll need capacity and low cost and sustainability, you got to do it in the cloud. Now, within that cloud, you also, this is to your point, it's a net new asset that you're creating for your organization. It is something that you're going to pump, if you're building a new foundation model, it could be a couple million dollars that you're going to invest in these model weights. And you're probably over time going to want more of your sensitive data to flow into that model to improve it over time. And so our approach is actually the only place that you can do this is on AWS because we don't have visibility into any of the compute environments that are the models or the data. We prevent. We don't want it number one, where there's technical limitations on having any visibility because of the way that we've architected AWS through the nitro system that I know you're familiar with. So using Nitro, we can't access any of that data, number one. You can't do that anywhere else. No one else has nitro number two, inside your own organization, you're going to want a really strict enclave, a strict perimeter around the weights and around that data. That data could be health data, it could be privacy data, it could be confidential data, even internally. And AWS is the only place that you can bring all of that together and run it in a secure enclave so that even inside your organization, you can't have access it.

>> By the way that Matt Garman told me that a year before he got promoted to CEO, and he's been saying that for over a year, that first it was a VPCs, now it's full enclaves.

Dr. Matt Wood

>> Yeah, enclaves. And then the next step that we're taking with the next generation of Nvidia platform, which is called Blackwell, will have end-to-end encryption.

>> All right, so you brought up cost. I brought up cost in my mind. He's going down the cost side. This is a fear right now, whether it's cloud or on premise, you get over-provision GPUs. That was the power. I run them all. So cost concerns and budgeting. How do you budget against an app that you don't even know it's being built yet? What do you know what you going to put guardrails around that might foreclose new opportunities? So how do you want evaluate a gen AI app? And two, how do you talk to the budget side if I'm a customer because I have to manage one hedge and also keep my options open for more budget or not spend my money and get out on the limb here and have to come all the way back and redo things. So how do you evaluate an app and then what are some of the cost budgeting ?

Dr. Matt Wood

>> I mean app evaluation is, this is actually one thing that I encourage customers to really lean into. It's just one of the things you're going to need to keep doing over and over again. We made some capabilities available to customers through Bedrock, which allow you to automate that evaluation. So you can look at how is it performing-

>> Like from an observability standpoint or more of from a performance-

Dr. Matt Wood

>> You can do observability across AWS that works really well. You can do data observability. You could also do, how's the model performing? What is the toxicity, what's the relevance? Is it robust? What's the coherence of the results? All of those sorts of things. So there's lots of different, just like any application, you're going to want to look at it from lots of different angles. And so we provide the tools to do that. And again, there's a bleeding between generative AI and the cloud because we already have all that observability stuff. If you're building an exclusively gen AI provider environment, you don't get access to queuing and observability.

>> Well, you guys announced the stability AI, multi-year relationship. They're part of this in Bedrock piece too, right?

Dr. Matt Wood

>> Yeah. We all work with stability. We work with anthropic. We've got -

>> I'm sorry, scale AI.

Dr. Matt Wood

>> Scale scale AI we know, yeah.

>> I'm thinking about wrong-

Dr. Matt Wood

>> No worries. No-

>> Scale AI is doing, that was the news today-

Dr. Matt Wood

>> Yeah, that's right.

>> Multi-year deal with them.

Dr. Matt Wood

>> That's right. So multi-year deal with scale AI where we are integrating Bedrock with the scale AI generative AI capabilities. So it's a pretty exciting opportunity, particularly for organizations such as the ones you're talking about that want to be able to create a data which is differentiated, take data that they have today and further differentiate it and then pipe it into the models that are running inside Bedrock. That's what we're working on with that one.

>> If you had to guess, take an educated guess or speculate or say that there's a pattern around how data pipelining, how reinforced learning's doing the big successful players, not perplexity, there're few with them, you guys work with Anthropic and others. So you're seeing a pattern of how the big scale works on not just training and how they management the models, but the data pipeline, the reinforced learning. If that comes to the enterprise, is there a pattern that you see that enterprises should be prepared to so that they don't make decisions around getting too confident in maybe walking into that Swiss cheese graph you put in, which is, wow, I realize that I'm down a road that just is a dead end for me because I didn't factor in these hallucinations or other things, so I didn't monitor my data or I got not enough fresh data, I got stale data. I don't have that image of the person crossing the street from my self-driving application.

Dr. Matt Wood

>> Yeah. That is definitely going to happen. If you're experimenting correctly, you're going to run into situations like that. The trick is just find that out really quickly instead until you get all the way down that road and you find you have to back out. So part of what we're doing with Bedrock is enabling that experimentation without having to write any code so you can figure out what's going to work and not work really, really quickly. But I also think that there is going to be an increasingly focused just on fine-tuning these foundation models. We call them foundation models for a reason. They're designed to be built on top of. So they have a very, very broad knowledge. Some of them have world knowledge, some are domain specific knowledge, but in every case you really want to be able to align their behavior for your specific tasks. And so you can think of that as building on top of the foundations or bending them to your will. But the trick there is that as you start to improve a model's behavior in one element, say you are talking a specific language or medical ontologies or interacting with the system, errors start to spring up in other areas and so they can get less performance. They can end up costing more, there aggregate intelligence decreases in other areas, which might not be what you want. And so part of the magic that we're working on is given that most companies want to do this, how do you do it in a way where you preserve the constitutional AI inside the models where you preserve the cost advantage-

>> 'Cause you got to feed it more data. I think you guys announced a bit of more access to data sources on stage today.

Dr. Matt Wood

>> Yeah, different data source, Scale AI can help with that fine-tuning data as well. So you actually don't need that much data to be effective in fine-tuning. If you're doing the fine-tuning effectively, you meet dozens, maybe hundreds of examples. But what you're really doing is programming the model. You're not programming it with code, with logic and rules, you just programming it by example.

>> Well, only a couple of minutes left. I want to shift gears to your experiences with people. One of the things that's clear coming out of this event in New York is obvious to us now is the word mindset has come up many times. Gen AI mindset. We saw this in the DevOps movement, infrastructure is code. You had the mindset or you didn't and you either failed or succeeded. So we're seeing that similar mindset. I want to ask you, is there a person or a certain profile, persona or orientation, skillset, vibe of a great gen AI person, a thinker? I mean, you don't have to be specific, it doesn't have to be the person, but there's certainly certain skill sets that have that. Is there a certain pattern that you're seeing in someone who's strong and grocking and implementing?

Dr. Matt Wood

>> Yeah, I mean, number one, that mindset, if that's what you want to call it, I think is growing very, very broadly. So I don't think there's just one type of unique personality that gets it or doesn't get it, just like with anything else that you mentioned. There is a group that I've found to be really, really receptive to this general approach. And it's usually search scientists or search teams. Search teams are really, really good at bringing technology, which they either build themselves or they work with others on and combining it with literally the knowledge and the language of their organization. And that is really the magic of generative AI. You want to be able to take the language of your organization with models you've either built yourself or you've gone from somewhere else and bring it as closely together so you increase its understanding. And search teams are remarkably good at doing that. And on the soft skill side, they're very pragmatic. They've got a problem that they want to solve. They have really good relationships inside the organization 'cause they touch so many people as a centralized resource, and so they have a really great perspective, which is both broad and pragmatic, which allows them to be successful.

>> May I understand metadata? How about young person come out of college? You interview a lot of people. What candidates jump off the page for you if you had to look at someone out there coming into the workforce?

Dr. Matt Wood

>> For sure, I think, number one, it's just the attitude really matters. Having an open mind on this stuff and seeing the building opportunity of it is a really big part of it.

>> Matt, thanks for coming on theCUBE. I know you're super busy and great to see you again.

Dr. Matt Wood

>> My pleasure.

>> And great to see you in New York and thanks for your time.

Dr. Matt Wood

>> Thanks, anytime.

>> All right, this is theCUBE coverage here at New York City of AWS Summit. I'm John Fuhr, your host of theCUBE. Thanks for watching.

Matt Wood | AWS

search

>> Welcome back, everyone, to theCUBE's coverage here of AWS Summit in New York City, and we've been coming to these summits ever since they started. New York is one of the biggest ones. It rivals almost like a reinvent in the sense of impact. New York is obviously the center of action. I'm John Fuhr, your host. Matt Wood is here, Dr. Matt Wood. He was on stage giving the keynote alumni, was actually leading the effort. Strong LinkedIn games, song SoCal games, sharing a lot of content from Party Rock to deep Dive, what's going on in Gen AI? You're the leading voice banging the drum-

Dr. Matt Wood

>> Appreciate it.

>> On generative AI.

Dr. Matt Wood

>> It's good. It's good to be there. Thanks for having me.

>> Yeah. I'm following your career before Gen AI you were deeply involved in a lot of the products from IOT data all through the machine learning. We've had many talks on stage and certainly on theCUBE about it. What's different about this gen AI piece first because it's changing the nature of the resources under the covers in order to power the apps. So no matter where you came from the gen AI perspective, the apps I need to write or how do I power them, it's a changeover, it's a platform shift. Is it a platform shift?

Dr. Matt Wood

>> I think that it is a very different set of technologies that have a very different set of capabilities. They have a very different set of benefits. They have a very different set of limitations than anything else. And the way I think about it is that this ability to be able to take large amounts of information, to be able to build a mathematical formula based on the examples that you see inside that information and that inside the math is this emergent property that resembles intelligence, that resembles reasoning. It's a discovery. It's not an invention, it's a discovery. And what we've been doing I think collectively as an industry, as builders, as inventors over the past 18 months is how do you take this discovery? How do you find out what's good about it? How do you find out what's bad about it? How do you find out where it works? How do you find out where it doesn't work? And so that's what I think is just fundamentally different. Nothing like this has really existed before because it is this brand new discovery and as a result we're seeing just a remarkable amount of energy and enthusiasm and investment from customers, whether they are... Every customer I speak to is just investing very deliberately. They're experimenting a great deal. There are some customers that are going all in, they're building out and retasking all of their development engineers overnight to focus on generative AI. And then there are pockets of areas that are moving much more quickly like the regulated industries.

>> Yeah, I mean regulators must be really kind of prime because they've been using the data governance and compliance almost set up for Generative AI 'cause they did all the heavy lifting.

Dr. Matt Wood

>> That's right, exactly. So yeah, the past 20 years they've had to meet all these regulatory compliance guidelines and probably felt like a bit of a headwind at the time, but it's driven the right set of behaviors. It said, hey, you need to understand where your high quality data is. You got to understand what data is useful, who can use it, who can access it, what it can be used for, which tools it can be used for. And they've already invested in solving that problem. And as a result, the generative AI piece is actually a relatively small incremental lift on top of that and they can move much more quickly and .

>> Much more quickly. And they have the system of record data. The question this brings up then just down horizontally scaling across all industries now, taking that perspective is the role of unstructured data and structured data. Obviously structured data is almost system of record, and that's important too. And you see actually some of the use cases we've been hearing on theCUBE, even here, covalent and others are like, "Hey, I can write generative AI to convert unstructured to SQL. So you're going to see the role of unstructured and structured change, more applied use cases with structured and unstructured is just piling up everywhere. So how do you talk about the two dynamics between unstructured? Obviously unstructured gets all the attention right now, but structured is still important.

Dr. Matt Wood

>> For sure. And there's a lot of value locked away in structured data in transactional systems, in accounting systems, all those sorts of things and structured data that lives inside those unstructured documents. Could be CSV files, could be tables inside PDFs, all those sorts of things. And so a lot of what we've been doing at AWS is finding ways to be able to leverage that context where it exists inside unstructured data. So imagine a table of numbers inside a PDF. If you just let the LLM do its thing, it'll just break it down and create tokens out of it. What we do is we say, hey look, this is a table. Whenever you need information from it, we'll provide you that information and the context above and below it and the table headers and all those sorts of things. And that additional context flows into it. Now, in addition to that, the other one thing that we announced today was the ability to be able to build agentic systems. These are systems where you can set an objective and have the AI track towards that objective plan and execute towards that objective. And to be able to-

>> Agentic meaning agent like-

Dr. Matt Wood

>> Agent like, exactly.

>> Agent, agentic, play on words.

Dr. Matt Wood

>> Systems who operate autonomously and can take action on behalf of the user. One of the things we added today was the ability for agents that are running on AWS in a service we have called Amazon Bedrock to be able to not just process all that unstructured data, not just processed all the structured data within that unstructured data, but to be able to actually write code autonomously and then run the code safely in a sandbox, which is precisely designed to allow advanced analytics. So that allows you to take data from all these structured sources and not just ask for the averages and maybe make a few mistakes along the way because LLMs aren't great at math. But now you could say, "Hey, I want to do a linear regression on the data that I got from this transactional system. I'll write a price and function, add the code into the environment, add the data into the environment, build your linear regression model, graph it and expose that so you can inspect the results and use that in the next step of the age agent system." So it's just completely-

>> Well, let's get into the keynote. We were riffing on some industry landscape, which we know it's a game changer. And to you guys say needle moving moment for all companies. First of all, my first question is did you flip a coin with Swami for this one or last time he was given a keynote, he's back at the ranch holding it down, what's-

Dr. Matt Wood

>> He's back of the ranch, holding his hat, and I'm the lucky one win at this time.

>> You were here last year too, so you gave a talk as well. I want to just do a shout-out to Swami because he wasn't here. Just fun to poke at you like that. Okay, let's get into the substance we are in. You obviously laying out the vision of what's happening. It's changing every six months if not sooner, every day. What was the core theme for you this week? What were the highlights on the announcements?

Dr. Matt Wood

>> Yeah, I think the core theme is, look, this technology is probably the easiest technology that I've ever seen to prototype and experiment with. It's got a remarkably low experience bar. The utility is incredibly broad and as a result it's easy to use. You can use it for a lot of things. A lot of people are experimenting with it and that's great. You actually want to encourage that. It's actually one of the biggest indicators of success with generative AI for companies. A lot of our organizations ask me, "Well, what are the use cases?" And you can rattle them off document processing and chat bots and analysis and-

>> No agents.

Dr. Matt Wood

>> All agents, exactly. But honestly, the thing I always recommend customers to look at is deliver a mechanism internally that allows for pervasive and efficient experimentation. And let a thousand platforms bloom, let a thousand flowers bloom. Some of them will be fantastic and you want to double down on them and the others didn't work out, but you've satisfied your curiosity, which is worth something as well. And what we did in the keynote today was say, not only is it easy to be able to experiment with these, but we want to make it really easy to find the ones that are working and then to be able to not just drive into a cul-de-sac where you have something which is interesting, but basically a toy, but be able to take it as far as it can go. And to be able to dramatically accelerate not just the prototyping but the move to production and doing that with agentic systems, doing that with systems like retrieval augmentation, doing it with guardrail systems, introducing entirely new categories of ways to protect your systems, to be able to ground them with your own information and then to be able to drive automation. And that dramatically shortens not just the prototyping phase but the production rollout phase as well.

>> I want to get your thoughts and I want to dive into the models piece you talked about and then guard rails like those, and the app studio was pretty cool too, but the models, I think the big revelation to me and others, well, we already knew that. If you go back to the early days of AWS, and you remember those days, it was a SaaS market. The entrepreneurs were really a target. The benefit was I can get compute, storage and queuing building blocks with a credit card. My alternative was go get a super micro box. Well no, for certain, even the entrepreneur, I got a host of cyber super micro box somewhere or some box, a pizza box and just put it and go do Telenet in, go manage the box, get a router-

Dr. Matt Wood

>> Install it, maintain it.

>> Do a technology tax, get that done. We'll document. iPhone hits the SaaS models built. Amazon at that point was just getting going then good timing, boom, great customers next enterprises. Hey, this is good too. I have to build a data center. Friends don't let friends. And then the great time after that. Now generate AI, you're an existing huge player. So the entrepreneurial market and the developer market is a little bit different. It's not as easy as saying, hey, credit card alternative data center. The alternative is I don't know how all these databases work. So what is the complexity problem alternative for not using AWS for AI, generative AI applications. And I think the model piece speaks to the SaaS category. I think models are a new thing because you pointed out that models now need to talk to each other, which has been obvious for people in the industry for over a year that this is coming. We put a power law out on theCUBE research. So if you believe models will talk to each other and APIs connect to everything, hello, that's data, that's software, that's a whole nother thing. And that could be a new opportunity for developers, entrepreneurs or innovators to say innovate on the model, not just data aggregation. That's old school. What's your thoughts?

Dr. Matt Wood

>> I think the naive approach that a lot of folks started with 18 months ago was take one model, just pick your favorite one, pick the one that was "the best", take a prompt, give it to the model, take the output, give it to the customer. And that's a good place to get started. But it really is insufficient. And what we've seen today is a couple of things. Number one, any sufficiently large organization that is investing in this space isn't using a single model. They're not even using a single model from a single provider. They're using multiple models from multiple providers. And so our approach from day one to your point is with bedrock has been to provide as many models as possible so the customers have the choice and they're able to fit the right model for the right use case. And we see that there was something, research came out of CB Insights where 97% of enterprises that are spending millions of dollars a year on generative AI are using models from multiple providers. And that totally rings true. It's what we do with Q, is what we do with our internal systems. And so that has been a disproportionate advantage for us. As you say, it's been obvious to us for a while. It wasn't always obvious 18 months ago, and I think some competitors maybe took a different advantage.

>> But to take the example of credit card versus provisioning gear, whether it's startup or data center company, what is the equivalent problem alternative that's too complicated versus coming here?

Dr. Matt Wood

>> Yeah, I don't think it's-

>> Or that's not a fair comparison?

Dr. Matt Wood

>> I think it's a fair comparison, but it's not a question of too complicated, too simple. It's a question of incomplete to complete. And so if all you have is a model, then you can only ever do that input to output mapping. But when you talk to customers, they need multiple models. They need guardrails, they need retrieval augmentation. And I was talking to a customer this morning and they also found that they needed databases and they needed queuing systems and they needed other agentic capabilities. And so there is a merging of the generative AI capabilities with the rest of the building capabilities that we've already got on AWS. And if you are building in an environment which has fewer options with models and capabilities, it's just going to be a cul-de-sac and you're never going to be able to build any escape velocity for your product to be able to escape that.

>> And just for the record, you said on stage models will talk to each other and they'll have different characteristics, small, big, music.

Dr. Matt Wood

>> I didn't say that. Yes.

>> Okay. So let's take to the next level. I talked to a bunch of customers in our research and also through just general reporting and digging in. A lot of them have the following perspective. "I'm building my own models, I'm not going to trust anybody. I got my data, screw those guys." I'm not going to say who those guys are. They might be, it's just fear. "I'm not going to let this out. I need to control it." This is brought on resurgence to, "Hey, everything's going back to the on-premise", which has been around for certain compliance things, but most part it's more about their security around their data. So with multiple models coming in, what is the vision for you guys on going to that customer saying, "Hey, you can build and do and you can train, you can infer it and do some reinforced learning build data pipelines, do out all day long, but you can also do it in the cloud." And so what is the answer to that question because they want to have their own models.

Dr. Matt Wood

>> Yeah, I think if you want to have your own models, then you also need an environment in which to train those models and doing that on premises is expensive and slow. It's capacity constrained and it has just a terrible sustainability footprint, just terrible. And so if you believe in that you'll need capacity and low cost and sustainability, you got to do it in the cloud. Now, within that cloud, you also, this is to your point, it's a net new asset that you're creating for your organization. It is something that you're going to pump, if you're building a new foundation model, it could be a couple million dollars that you're going to invest in these model weights. And you're probably over time going to want more of your sensitive data to flow into that model to improve it over time. And so our approach is actually the only place that you can do this is on AWS because we don't have visibility into any of the compute environments that are the models or the data. We prevent. We don't want it number one, where there's technical limitations on having any visibility because of the way that we've architected AWS through the nitro system that I know you're familiar with. So using Nitro, we can't access any of that data, number one. You can't do that anywhere else. No one else has nitro number two, inside your own organization, you're going to want a really strict enclave, a strict perimeter around the weights and around that data. That data could be health data, it could be privacy data, it could be confidential data, even internally. And AWS is the only place that you can bring all of that together and run it in a secure enclave so that even inside your organization, you can't have access it.

>> By the way that Matt Garman told me that a year before he got promoted to CEO, and he's been saying that for over a year, that first it was a VPCs, now it's full enclaves.

Dr. Matt Wood

>> Yeah, enclaves. And then the next step that we're taking with the next generation of Nvidia platform, which is called Blackwell, will have end-to-end encryption.

>> All right, so you brought up cost. I brought up cost in my mind. He's going down the cost side. This is a fear right now, whether it's cloud or on premise, you get over-provision GPUs. That was the power. I run them all. So cost concerns and budgeting. How do you budget against an app that you don't even know it's being built yet? What do you know what you going to put guardrails around that might foreclose new opportunities? So how do you want evaluate a gen AI app? And two, how do you talk to the budget side if I'm a customer because I have to manage one hedge and also keep my options open for more budget or not spend my money and get out on the limb here and have to come all the way back and redo things. So how do you evaluate an app and then what are some of the cost budgeting ?

Dr. Matt Wood

>> I mean app evaluation is, this is actually one thing that I encourage customers to really lean into. It's just one of the things you're going to need to keep doing over and over again. We made some capabilities available to customers through Bedrock, which allow you to automate that evaluation. So you can look at how is it performing-

>> Like from an observability standpoint or more of from a performance-

Dr. Matt Wood

>> You can do observability across AWS that works really well. You can do data observability. You could also do, how's the model performing? What is the toxicity, what's the relevance? Is it robust? What's the coherence of the results? All of those sorts of things. So there's lots of different, just like any application, you're going to want to look at it from lots of different angles. And so we provide the tools to do that. And again, there's a bleeding between generative AI and the cloud because we already have all that observability stuff. If you're building an exclusively gen AI provider environment, you don't get access to queuing and observability.

>> Well, you guys announced the stability AI, multi-year relationship. They're part of this in Bedrock piece too, right?

Dr. Matt Wood

>> Yeah. We all work with stability. We work with anthropic. We've got -

>> I'm sorry, scale AI.

Dr. Matt Wood

>> Scale scale AI we know, yeah.

>> I'm thinking about wrong-

Dr. Matt Wood

>> No worries. No-

>> Scale AI is doing, that was the news today-

Dr. Matt Wood

>> Yeah, that's right.

>> Multi-year deal with them.

Dr. Matt Wood

>> That's right. So multi-year deal with scale AI where we are integrating Bedrock with the scale AI generative AI capabilities. So it's a pretty exciting opportunity, particularly for organizations such as the ones you're talking about that want to be able to create a data which is differentiated, take data that they have today and further differentiate it and then pipe it into the models that are running inside Bedrock. That's what we're working on with that one.

>> If you had to guess, take an educated guess or speculate or say that there's a pattern around how data pipelining, how reinforced learning's doing the big successful players, not perplexity, there're few with them, you guys work with Anthropic and others. So you're seeing a pattern of how the big scale works on not just training and how they management the models, but the data pipeline, the reinforced learning. If that comes to the enterprise, is there a pattern that you see that enterprises should be prepared to so that they don't make decisions around getting too confident in maybe walking into that Swiss cheese graph you put in, which is, wow, I realize that I'm down a road that just is a dead end for me because I didn't factor in these hallucinations or other things, so I didn't monitor my data or I got not enough fresh data, I got stale data. I don't have that image of the person crossing the street from my self-driving application.

Dr. Matt Wood

>> Yeah. That is definitely going to happen. If you're experimenting correctly, you're going to run into situations like that. The trick is just find that out really quickly instead until you get all the way down that road and you find you have to back out. So part of what we're doing with Bedrock is enabling that experimentation without having to write any code so you can figure out what's going to work and not work really, really quickly. But I also think that there is going to be an increasingly focused just on fine-tuning these foundation models. We call them foundation models for a reason. They're designed to be built on top of. So they have a very, very broad knowledge. Some of them have world knowledge, some are domain specific knowledge, but in every case you really want to be able to align their behavior for your specific tasks. And so you can think of that as building on top of the foundations or bending them to your will. But the trick there is that as you start to improve a model's behavior in one element, say you are talking a specific language or medical ontologies or interacting with the system, errors start to spring up in other areas and so they can get less performance. They can end up costing more, there aggregate intelligence decreases in other areas, which might not be what you want. And so part of the magic that we're working on is given that most companies want to do this, how do you do it in a way where you preserve the constitutional AI inside the models where you preserve the cost advantage-

>> 'Cause you got to feed it more data. I think you guys announced a bit of more access to data sources on stage today.

Dr. Matt Wood

>> Yeah, different data source, Scale AI can help with that fine-tuning data as well. So you actually don't need that much data to be effective in fine-tuning. If you're doing the fine-tuning effectively, you meet dozens, maybe hundreds of examples. But what you're really doing is programming the model. You're not programming it with code, with logic and rules, you just programming it by example.

>> Well, only a couple of minutes left. I want to shift gears to your experiences with people. One of the things that's clear coming out of this event in New York is obvious to us now is the word mindset has come up many times. Gen AI mindset. We saw this in the DevOps movement, infrastructure is code. You had the mindset or you didn't and you either failed or succeeded. So we're seeing that similar mindset. I want to ask you, is there a person or a certain profile, persona or orientation, skillset, vibe of a great gen AI person, a thinker? I mean, you don't have to be specific, it doesn't have to be the person, but there's certainly certain skill sets that have that. Is there a certain pattern that you're seeing in someone who's strong and grocking and implementing?

Dr. Matt Wood

>> Yeah, I mean, number one, that mindset, if that's what you want to call it, I think is growing very, very broadly. So I don't think there's just one type of unique personality that gets it or doesn't get it, just like with anything else that you mentioned. There is a group that I've found to be really, really receptive to this general approach. And it's usually search scientists or search teams. Search teams are really, really good at bringing technology, which they either build themselves or they work with others on and combining it with literally the knowledge and the language of their organization. And that is really the magic of generative AI. You want to be able to take the language of your organization with models you've either built yourself or you've gone from somewhere else and bring it as closely together so you increase its understanding. And search teams are remarkably good at doing that. And on the soft skill side, they're very pragmatic. They've got a problem that they want to solve. They have really good relationships inside the organization 'cause they touch so many people as a centralized resource, and so they have a really great perspective, which is both broad and pragmatic, which allows them to be successful.

>> May I understand metadata? How about young person come out of college? You interview a lot of people. What candidates jump off the page for you if you had to look at someone out there coming into the workforce?

Dr. Matt Wood

>> For sure, I think, number one, it's just the attitude really matters. Having an open mind on this stuff and seeing the building opportunity of it is a really big part of it.

>> Matt, thanks for coming on theCUBE. I know you're super busy and great to see you again.

Dr. Matt Wood

>> My pleasure.

>> And great to see you in New York and thanks for your time.

Dr. Matt Wood

>> Thanks, anytime.

>> All right, this is theCUBE coverage here at New York City of AWS Summit. I'm John Fuhr, your host of theCUBE. Thanks for watching.