Cloud AWS re:Invent Coverage | Matt Garman, AWS

Clips
News
More from Cloud AWS re:Invent Coverage

Matt Garman

CEO

AWS

Three insights you might have missed from AWS re:Invent

The AWS re:Invent gathering in Las Vegas last week provided several newsworthy announcements, including the cloud giant’s entry into the world of AI f...

AWS re:Invent 2024: CEO Matt Garman unveils the future of cloud with generative AI and agentic workflows

As AWS re:Invent 2024 approaches this coming week, anticipation is building for what promises to be a defining moment in the evolution of cloud computing. In an exclusive interview at Amazon Web Services Inc.’s Seattle headquarters, I sat down with Chief Executive Matt Garman (pictured) to delve into the themes that will shape re:Invent and AWS’ strategy for the coming years. Video exclusive: A conversation with theCUBEFrom a renewed focus on core infrastructure to advancements in custom silicon, generative artificial intelligence and agentic workflows, Garman’s insights offer a roadmap for the future of cloud computing

play_circle_outline Maximizing Growth and Profitability Through Cloud Innovation in Today's Market

play_circle_outline Focus of re:Invent on infrastructure and AI advancements

play_circle_outline Lowering costs and integrating inference into production environments

play_circle_outline Importance of serverless and agents in AI applications

play_circle_outline Embracing Legacy Modernization: Transitioning Mainframe to Cloud While Prioritizing Value in AI Projects

Info
Transcript

Matt Garman, AWS

Matt Garman

CEO AWS

Matt Garman, AWS CEO, discusses the significance of cloud adoption and the role of generative AI in revenue growth and innovation. He stresses the importance of migrating workloads to the cloud for technological advantages. Matt highlights core infrastructure innovation at re:Invent, along with advancements in AI and serverless computing. He mentions challenges with integrating AI inference into production and reducing costs. Integrating data and applications into the cloud drives enterprise value. Matt discusses automating tasks with agents and managing fram... Read more

explore Keep Exploring

What are some reasons why customers are increasingly interested in moving their workloads to the cloud? add

What areas are some of the focus on in terms of helping customers innovate, specifically mentioning AI and core infrastructure? add

What factors are contributing to the lowering cost of inference in AWS? add

What are the key considerations for customers in ensuring a successful architecture and platform implementation? add

What are some key steps in modernizing and utilizing data effectively in a cloud environment? add

bolt Powered by CUBE AI

Matt Garman, AWS

search

>> Hello, welcome to theCUBE. I'm John Furrier, your host here in Seattle, the AWS re:Invent building with Matt Garman, the CEO of AWS for exclusive interview for re:Invent and all the activity happening. Matt, great to see you. Thanks for come on theCUBE in your studio. Thanks. And home game for you, away game for me.

Matt Garman

>> Absolutely. Happy to do it. Thank you for coming in and chatting.

>> So obviously, re:Invent, the biggest conference you guys have. It's been, I think our 14th year, we've seen the transformation and you as CEO now. We talked in August about your vision. Folks can look at that YouTube video, but we kind of teased out this Gen 2 kind of cloud play with agentic AI and all this gen of AI and agents are coming. The game has changed, but it's still back down to the infrastructure. And so I want to get into that with you, but I also want to talk about your growth. And you guys had great earnings and you had great performance. I think it's a 10-year-high for the stock price. The growth is continuing to go and we're in this post-ZIRP era and there's a real emphasis on profitability. You guys are doing well as well as your customers. What's your view on the current market right now?

Matt Garman

>> Yeah, we're seeing a lot of people that are really leaning into the cloud. I would say as we talk to customers out there, they are increasingly interested in how they drive revenue growth, how they drive innovation, how they get their businesses growing. And a lot of customers are seeing that gen AI is one of the things that's driving that. And as you increasingly talk to customers about how they're really going to get value and how they're going to get enterprise value and drive their businesses forward, they realize that they've got to get those workloads into the cloud so they can have the agility to take advantage of all the latest technologies that are coming out. And that means getting their data to the cloud. That means getting their applications to the cloud so they can apply all these new models, whether they're agentic workflows. All of these new benefits that they see coming down the road, they realize that that cloud piece is there. And so we see that as a real tailwind to the business and something that customers are really excited about moving forward.

>> And what's your focus for re:Invent this year? Because obviously, every year, there's a slew of announcements. It's always the classic stack. You've got infrastructure, you've got the data layer and the higher level services, and obviously the applications are changing radically. What's the focus?

Matt Garman

>> Some of the focus is we're also going to get back to some of the basics on just how do we go and help customers innovate in all the different areas that they get to. And I think look, we'll have a lot of announcements in things about AI and generative AI, but we're also going to have a lot of really cool innovations and announcements around the core of our infrastructure and thinking about compute and storage and databases. That is an area that we have been hard at work innovating on. And I think it's one of those things that we don't want to forget. It's kind of the core of what we do and I think it's something that our customers are hungry for as they look for continued innovation across that whole stack, not just in the shiny new objects, which we'll have a lot of announcements there also.

>> Yeah, I think our last interview, we talked about inference as the killer app, and you said you have good training, you're going to need inference to go with that. What are some of the performance gains we're seeing this year? Because again, the hype curve of generative AI has kind of been gone. Now we're in the reality curve where it's show me the proof, but there's still a lot of work to be done at the infrastructure layer because everyone wants performance and they don't want to pay out the nose for it. They want to have good cost structures and visibility into the unit economics. What's the update there?

Matt Garman

>> Look, I think that there's two things you think about as you think about inference. And this is really is how do people really integrate it in. I think when we saw early on when generative AI hit the scene two years ago, everybody ran and they did a quick proof of concept and they put a chatbot on their website and they were very excited about it and they got some early wins. Now though, I talked to customers and they have 100, 200 proof of concepts out there in the enterprise and they're trying to figure out how they actually now figure out, as you point out, how do you get value from that? How do you actually drive enterprise value, not just have this cool factory, go test? And so the couple things that customers are looking at are, one, what is the cost and how do I bring down the cost of running that inference? And two, how do I actually integrate it into my production environment? And those are two real challenges, which are not there when you do a proof of concept. And so on the cost side, we're driving down a number of areas where we think about new models, we think about new efficiencies, we think about new infrastructure, whether that's new instance types, new networking, et cetera. And so we are pushing really hard to continue to lower the cost of inference. And if you've seen over the last year, the cost of inference has been coming down and down and down. And we think that there's still a long way to go there. And we think that hardware has a lot to do with that, the system or the service that you run there. And we think that Bedrock is a great place for people to go to get that absolute lowest cost as you think about new models and new infrastructure. But just as important is really how do you integrate it into that data? Because customers really are hungry for how do you take those 100 proof of concepts, turn them into the three or four or five real production applications where they're going to drive real enterprise value that's going to drive revenue, lower costs, et cetera. And that means how do you have a framework that it gets tied into your data? How do you have a framework that thinks about security, thinks about permissioning in your enterprise and your organization, thinks about how do you have an actual usable ability to go launch and build agentic workflows? How do you think about that whole piece that you really think about a production? And that's where I think customers see that AWS is quite good at.

>> So you're going to have some IaaS, some infrastructure announcements around performance and silicon, like always, pretty much going to be the same.

Matt Garman

>> Everybody's going to need to tune in for re:Invent.

>> That's a yes.

Matt Garman

>> But we're very excited about many of the announcements we have.

>> Okay, so check the box on that one, and needed by the way. Okay, so that's going to power this next wave. I think we talked about our last interview. You got to pedal as fast as you can on the infrastructure, get the chips going and the speed. And that's going to enable this next layer up in the stack, we talked about the gen stack. So we're seeing growth in the PaaS side. Our research has shown your PaaS business is growing significantly fast. I would say the infrastructure has been the bread and butter core services, but as gen AI hits, we're seeing a lot of activity being enabled in database and other services around workflows end to end. And I talked to Werner about the Lambda 10 year anniversary and what came out of that conversation I want to ask you is Lambda enabled serverless and there was no blueprint. It was solving a problem from core services that I think WeTransfer had and some of your other customers. And that opened up and became a legendary and historic product. And what that enabled was just better everything. Is there a gen AI version of that? Because gen AI has a similar pattern. There's no real blueprint other than you got to get the learnings. And so this is a PaaS layer, it involves database, it involves a lot of these services that used to be run a certain way but might run differently. What's your view on this? And also because agents are going to come out of this.

Matt Garman

>> Yeah, look, almost for sure that the entire way that you go build generative AI applications is going to change and be reinvented. And because we're so early in that space, I think we're all finding how to go build that. And our job is to make that easier for customers. If you take a half a step back, you talk about the core building blocks that we have in compute and storage and databases. I actually have this view that customers ask about generative AI applications. I actually think they're just applications. And so I actually think inference is the next core building block. And it is, if you think about it that way, if you think about inference as part of every application that you go build, it's not just a separate generative AI application and then all my other applications, it's just an integral part just like you would think about... It's not a database application and a non-database application, it's just a part of what you might use. And so if you think about it that way, then you got to think about how does it get integrated into the fabric of what you're doing? And that means you think about data workflows, you think about how these models might interact really well with S3 and EC2 and your database and how do they all kind of work together? And that's the job that we're trying to make easier for customers so that when you're thinking about startups out there or enterprises trying to solve a problem and they're going and building a new application, it's just a tool that they can get really powerful and exciting capabilities deeply embedded into their application.

>> So serverless came out of Lambda, that basically still ran on servers. It really wasn't serverless, it felt like serverless. Is there a gen AI version of that coming that you see? Because if you look at what you guys are doing and you can see as you look at Bedrock and SageMaker under the hood there, you're looking at essentially stitching together the mosaic of services that used to be standalone or configured either manually or the old school way. Old school way, it's funny to say, that's the old cloud way. But the new cloud that you guys have with gen AI, that's interesting because this is what agents need. They need to have the agility. What are you learning from customers that's going to feed into this new direction?

Matt Garman

>> Yeah, I think there's a couple of things that I would say is one of a funny anecdotes. I remember when I was running EC2 and Charlie Bell was super excited about serverless and he was telling all of his directs, "Everybody has to move their services to a serverless offering." And I was like, "What about me?" He's like, "Okay, not you. EC2 is actually servers at the end of the day."

>> It's got to run on something.

Matt Garman

>> You do need to have an actual server at the end of the day, but you fast-forward, and that serverless model is still something that I actually think is super important in AI. And you think about Bedrock, Bedrock is serverless, right? You send it tokens, you get tokens back. And as you think about agentic workflows, again, Lambda is a key part of that. So you can have this bit of compute that helps the models know where they're going to go. So I do think that as we look at those new models and you think about that new programming paradigm, serverless, people don't want to have to go run large clusters of models. They may use EC2 and instances to do things like pre-training and fine-tuning and other things like that. But once you really get to the application, you don't want to have to worry about scaling and managing that infrastructure. And so that vision that we had about serverless 10 years ago absolutely still applies to where we are today. And I think the AI models are easily applicable to that. And we kind of take that same mentality that just because it's a new technology doesn't mean that customers all of a sudden want to manage infrastructure. They actually just want to get the capabilities and actually go think about the logic and the workflows and the-

>> I mean the magic of Amazon was abstracting away the complexities and just the functionality doesn't go away. Talk about agents because agentic is the hot wave, I won't say shiny new toy because there's legitimacy to what's going to happen. You're already seeing MQ doing it for developers, you're seeing it in business. This is going to be the application feature, whether it's retrofitted or net new applications. What's your vision on how you guys are going to create those agents? How do customers have a architecture and platform that's going to be successful?

Matt Garman

>> Yeah, and why people are excited about it, it's not surprising, I mean it's why we launched agents as part of Bedrock probably six months ago. And the key thing you want to think about is the value that you get out of many of these AI capabilities. Number one, there's consumption, creating content, summarizing content. That is super powerful and it is a thing that I think we have seen people get really excited about in Bed and a lot of different things. The next step is how do you get many of these models to actually start executing tasks and doing the next wave? And that is what agents are all about. At the end of the day, they need to take that next step. If you can summarize a bunch of data into a crisp enough thing that you can make a decision on, how do you then automate that step? And it really is just process flow automation. It's thinking about how do you have these things really go? And the cool part, and I think the part of the problem that we want to help customers solve too, is that we're at the part where that's possible now. The interesting thing though is when you really start to get leverage as you have thousands and thousands and thousands of agents doing things, and then that pretty soon gets super complicated and a thing that is really hard to manage and wrangle and figure out how you do. We want to make it possible for people to actually go do agents at scale, think about more complicated processes at scale, and what are the frameworks and areas that we can help them actually drive that so that it's a manageable process, because if you have 1,000 or 10,000 or 100,000 agents out doing things, pretty soon that process gets unmanageable. And so we're trying to think about how do we help customers solve those problems?

>> Well, that's why I brought the Lambda conversations. I think that enables serverless to do a lot of these things of distributed computing. But you're bringing up a great point. I think this is the hot topic that we want to dig into is multi-agent and distributed AI workloads. Because what you're basically saying is it's not an agent, it's this collection of agents, it's multiple agents, it's distributed AI with agents, they have to talk to each other. How do you handle resilience, routing? These are technical but important things. How do you roll back and recover from a security standpoint? How does resilience fit into agents? Because things are happening all the time.

Matt Garman

>> And look, the industry is moving fast and we're trying to help people get their hands around it so that they can move fast with it.

>> Anything in re:Invent you think is going to pop out that's notable, people should pay attention to as they think about distributed computing going agentic, what are the key things that you guys advise and hear from customers that it's on their mind around setting the table for this future?

Matt Garman

>> Yeah, I think one of the things that I think, honestly, there are so many innovations in this space. Much of what I think customers need to do is take a half a step back and really push themselves to think about what's possible. It actually is, it's surprisingly is, is actually this. One of the things that causes people to go the slowest is not realizing how they can think about using these technologies and how. So as customers come to re:Invent, and there's going to be a ton of opportunities for folks in re:Invent to get their hands dirty and actually start playing with these technologies and thinking about them. But come kind of thinking about your business and thinking about the processes and thinking about how could you do things differently because that is the key that we're really trying to unlock here. It's not about making 5% improvements in what you're doing. It really is about making really stepwise changes in the capabilities that whole industries are able to accomplish. And that's really what we're targeting.

>> Compute and data go hand in hand together. You mentioned inference is the killer app, that's key agentics here. What's the customer outlook look like when you guys get feedback, when you have that feedback loop from customers with these end-to-end workloads? Because in the enterprise, it's end to end, and compute's needed. You don't have it centralized, it's distributed computing. What's the customer feedback loops look like when they start to come to AWS and what are you seeing, and from a database standpoint, from a services standpoint?

Matt Garman

>> Look, customers give us a wide range. We have a lot of customers who get a lot of features, so there's not a homogenous customer out there. But I think there's a couple things that if I was to distill a lot of that feedback that we get, number one, they want help. There's a ton of technology that's flying at them. There's a ton of potential they see for the business, and they need our help to figure out how that they corral that. And they also tell us, and you'd be maybe surprised at this, but they also tell us, "Don't forget about the fundamentals of the things that I need to do." And they're like, "It turns out the vast majority of my workloads are still on premise and I actually need help migrating and modernizing those just to get to those, to the cloud. And how can you make that go faster? And that actually will deliver huge amounts of value to my business so I can take advantage of these new capabilities." And so sometimes we get really spun up about, go do agentic workflows or whatever, and the customers are like, "But my data is in a mainframe in my data center. I can't go do an agentic workflow on my mainframe. That doesn't work." And so we also can't forget some of those fundamentals. We've got to get that mainframe moved to the cloud.

>> Well, that brings up legacy modernization as a big trend we're seeing. You got all kinds of existing stuff, you don't need to throw it away. You can put, I won't say a wrapper around it, but conceptually, you can integrate it in with gen AI. What's your vision on that?

Matt Garman

>> Yeah, I think the key piece is getting it to an environment that you can look at it, catalog it, tag it, and use all of that data in a more modern framework. And increasingly, and actually AI helps a lot with this, is that we're making it easier and easier for customers to get their data into the cloud, to get it into a usable format where they can go access it really quickly. And I do think that that is a thing that customers want a lot of help on. And then you can get it into a containerized format where you don't have to completely rewrite things. You can get a lot of benefits there pretty quickly.

>> Last time we talked, the enthusiasm's high, obviously with gen AI and Amazon's position in the cloud right now. But confidence sometimes, when costs are thrown around, resource planning is huge conversation. I know we've had in the past and I think you reinvented what's top of mind as, how do I plan and cost out? Because you can really make an error by getting all this training done. It's like where's the beef? Where is the juice worth the squeeze? Because if the cost overruns happen, you seen a lot of people look at the cost and say, "Well, I got to manage cost. I love doing all this work, but when we put it in production, it's got to work, check." But what's the cost look like? How do you view that, and what are customers saying?

Matt Garman

>> Yeah, I think even as much as important as the costs are, and they're super important and we give customers cost guardrails and we're trying to think about more ways that we help them understand what the costs are going to look like, I think more important though is actually thinking about what is the value that you're going to get from going and doing this work. I think a lot of times, what we saw is people rushed into launching what seemed like a cool AI application and they didn't actually know what the benefit was going to... It kind of like the whole kind of ROI analysis or thinking about the benefit to your business kind of went out the window in the rush of going and getting something out there. So you could tell your board you had a gen AI application. And now, I think this is critical, right? It's the same as it was before. As you look and you want to say, "What is the value I'm delivering to my business? Where do I think the gains I'm going to get, either from reducing my costs somewhere in the organization, ramping revenue faster, delivering value to my customers," whatever that ROI is, and then you can go look at that cost and figure out if you think that that's worth it or not. But until you have that second part of the equation, they're always going to be too high if there's no value coming out the end because any dollar you spend is not worth-

>> But you feel good that you have the cost controls for customers to manage and have visibility?

Matt Garman

>> We do, and customers have to pay attention to it and they have to think about it. But we have lots of capabilities for customers to have guardrails around those, and frankly, we are trying to spend a ton of time lowering the unit costs so that every little bit of work that they do costs less and less every-

>> What's your advice to partners as they navigate through the ecosystem as it evolves? Because you're seeing a lot more connected oriented, it's not just APIs connecting. You have gen AI data layers connecting into the system. What's your message to partners as they try to navigate? Because look, you guys are growing fast and it's getting bigger and bigger and there's probably more value with the cloud, so they need to navigate and be in the good position to monetize their business and get distribution with this distributed computing model.

Matt Garman

>> I think one thing I would say is there is more opportunity for partners than there has ever been. And you know this as well as anybody. Partners has always been a very central part of how we go to market, of how we support our customers. And we're very aware that we can't do everything, that we're not going to be the experts. And many of our partners are the deep experts in the customers and they know deeply their business and they know how those workflows operate. And particularly as you think about, look, as we're talking about how do you go and actually implement an agentic workflow, you've got to pretty deeply know what are the business processes and how do things work and where are the right sets of data and where's the value? And our partners are key to that. Whether it's ISV partners or systems integrator partners, there are many... Customers need help, and they're going to need help to make it easier, they're going to need help to move faster. People aren't going to be able to build all these things themselves. And so I think the opportunity for partners is massive and it's much bigger than it's ever been. And we're not going to get, frankly, as a business for AWS and all of our customers, we're all not going to get to where we need to be if we can't have that partner ecosystem help us get-

>> That's the gift that keeps on giving for partners. Let's talk about developers. I know you got to run, but I want to get into the entrepreneur developer angle in Silicon Valley and New York where we're doing a lot of interviewing with founders, not publicly talking about, but in the hallways we're hearing, "I can't get GPUs." They don't have a lot of cash either, so they're innovating. So this kind of over the top hyperbole, but that's generally the sentiment. They want more value. They're coding. It's the same vibe that was in the early days of AWS. What's your value proposition to those folks that are innovating that need resource?

Matt Garman

>> Yeah, look, we're driving as fast as possible to get capacity and there's been a capacity crunch across the industry for the last two years probably, and it still exists today. And we are adding capacity as fast as humanly possible. We're adding power as fast as possible, and we're thinking about how we offer lower priced offerings. I think both of those are important. I think driving down the costs for startups and enterprises alike, everyone cares about costs, but I think startups, as you know, are near and dear to my heart, and I hear that feedback as well. And so we're both thinking about a lot, about how do we make sure that we just have more capacity for everyone, but also how do we have lower cost options so that people have options out there?

>> And you guys are leading in CapEx. I mean, do you sit down and talk to Andy about, "Hey, did we get a nuclear reactor? Let's spend some more CapEx." How's that conversation go? What happens there on the CapEx side? You guys just are leaders in spending. You got the nuclear option now. Is there more coming?

Matt Garman

>> Look, ours is a capital intensive business and it turns out as you're growing as fast as we are, it means you need to add data centers, you need to have servers. And so that is the business that we're in. I think we've built a pretty good capability of really understanding how do we responsibly spend at the right time, push the spend out so that we have enough capacity for our customers whenever they need it, but also that we are not being irresponsible from a corporate governance point of view. And so yes, spend a ton of time thinking about that and we're very intentional about how we go spend, but-

>> And nuclear's on the table.

Matt Garman

>> And then nuclear, I think is... Look, we're very also focused on carbon zero energy, and I think we spend a ton on adding new carbon-free energy to the grid out there. I think nuclear is a fantastic additional option to that portfolio.

>> Final question. I know you got to go. What's the bumper sticker this year for re:Invent? If you have to put it on the car, what's it going to say?

Matt Garman

>> Do we have bumper stickers? I don't know. I don't know that we have them.

>> What's the electronic billboard say, what's the takeaway? What's the tagline for re:Invent this year if you had to put it into words?

Matt Garman

>> You know what, if you don't mind, I'd steal from Werner. We want people to go out there and build like it is... We build a lot of the technologies and we develop the services that we have so that our customers can go build and they can go invent.

>> Thanks for your time. Appreciate it.

Matt Garman

>> Thank you. Thanks for having me, and thanks for coming out.

>> This is theCUBE coverage here at re:Invent's headquarters. It's a building called re:Invent here in Seattle. I'm John Furrier with theCUBE. Thanks for watching.