AWS Mid-Year Leadership Summit 2025 | Matt Garman, AWS

Clips
News
More from AWS Mid-Year Leadership Summit 2025

Matt Garman

CEO

AWS

Exclusive: Amazon CEO Matt Garman bets on an agentic future

From a glass-lined conference room high above Amazon’s campus, Matt Garman firmly emphasizes one word: “speed.” With a full year under his belt, the new chief executive of Amazon Web Services Inc. (pictured) has just finished what we called a “halftime huddle” — two days of video exclusive meetings and interviews with top AWS lieutenants racing to keep the world’s largest cloud provider ahead of an artificial-intelligence boom that is redrawing the technology industry’s boundaries.The video below is part of our editorial series AWS and Ecosystem Leaders Halftime to Re:Invent Special Report digital event. Look for other articles from the event on SiliconANGLE

Inside AWS’ enterprise infrastructure playbook: CEO Matt Garman on AI, cloud and speed

Enterprise infrastructure is undergoing a seismic transformation as generative artificial intelligence and agentic systems push demands on cloud, data and compute to unprecedented levels. What was once back-end plumbing is now the engine room of competitive advantage.From data-driven decision-making to reasoning-capable agents, the role of infrastructure has shifted from supporting business to actively shaping it. Organizations are no longer just lifting and shifting workloads to the cloud — they’re rebuilding their foundations for a future where AI-native applications, real-time insights and workload autonomy are the norm

play_circle_outline Matt Garman discusses AWS's rapid innovation and the impact of AI on cloud computing.

play_circle_outline Reinventing Infrastructure: How AI Advancements Drive Data Organization and Integration for Enhanced Enterprise Value

play_circle_outline Exploring the Surge of Startups Leveraging AWS for Enterprise Applications Amidst the Growing AI Market

play_circle_outline The European Sovereign Cloud aims to address sovereignty concerns for EU customers.

play_circle_outline AWS invests heavily in infrastructure to meet increasing demand for cloud services.

play_circle_outline Generative AI is expected to yield significant efficiency gains across various industries.

play_circle_outline Empowering Innovation at AWS: How Internal AI Experimentation is Transforming Job Roles and Optimizing Development Processes

play_circle_outline Moving to the cloud enables organizations to remain agile and responsive to market changes.

play_circle_outline AWS's custom silicon, like Graviton, provides competitive advantages in cost and performance.

Info
Transcript

Matt Garman, AWS

Matt Garman

CEO AWS

In this AWS Leadership Summit segment, AWS CEO, Matt Garman, joins theCUBE’s John Furrier in Seattle for a halftime check-in on 2025. Garman explains how generative AI is “turbocharging what’s possible,” driving new agentic services, a relentless silicon roadmap and rapid region growth. He outlines why fewer than 20 percent of global workloads have moved to the cloud, yet enterprise urgency has never been higher, with data-first strategies and Bedrock-powered AI pushing companies to modernize faster than ever.

The conversation drills into cost, scale... Read more

explore Keep Exploring

What is the current pace of innovation regarding AI and how are organizations responding to it? add

What are customers looking for in terms of preparing their enterprises for upcoming technological changes? add

What are the reasons behind startups' growing interest in selling to enterprise companies? add

What are the benefits and features of the European Sovereign Cloud being launched at the end of this year? add

What are the challenges and considerations in scaling GPU resources to meet customer demand in the data center business? add

What are the potential benefits of generative AI for customers? add

What value is being derived from encouraging employees to experiment with new technologies at Amazon? add

What are some challenges that businesses face in adapting to post-COVID changes? add

What competitive advantages does AWS have in terms of its technology offerings? add

bolt Powered by CUBE AI

Matt Garman, AWS

search

>> Hello, I'm John Furrier with theCUBE. We are here on location in Seattle at AWS's headquarters with Matt Garman, the CEO, breaking down the future of cloud, but also looking back at this half year, kind of a halftime report for two days. It's been quite a year. AI Speeds feels like a full year. Matt, great to see you. Thanks for taking the time and taking time out of your busy day. Thanks for coming on.

Matt Garman

>> Yeah, happy to be here. Thanks.

>> I got to ask you, in the 20 years that I've worked with and covered AWS, I've never seen the speed and velocity of this year. It's kind of like a halftime report, mainly because it feels like you could probably do re:Invent right now and have all the news. It's been quite a year. Yeah, how are you feeling?

Matt Garman

>> Great. And like you said, yeah, the innovation pace is at just a breathtaking rate right now, but that's what customers are excited about and I think it's not that often that a technology revolution like AI comes along that really just turbocharges what's possible. And across the stack, up and down for AWS, we're innovating both to support that from the infrastructure layer and the services as well as building brand new AI and agentic services. And customers are excited about it, we're excited about it. And like you said, it's moving like crazy.

>> Last time we talked before re:Invent, we talked about startups and I could see there's some progress in there with startups. Enterprise AI is hot. This year and kind of going into 2026 seems to be the year of cloud, next level cloud with collaboration with customers, partners, and ultimately their end users. So you have this kind of mix of what you guys always talk about working back , but the equation is so fast, but the collaboration with the customers is a key theme. They're all looking at reinventing their infrastructure because they have to usher in the pressure for the agentic wave, which clearly is kind of like this next SaaS kind of cloud play. This is a real factor in the customers. What are some of the things you're seeing in the business right now with the customers and in the collaborations? What are some of the top line things you talk about and what do you see?

Matt Garman

>> Yeah, it's a great point. I think when we talk to customers, they're looking for us to help understand really how can they prepare their enterprise for this coming wave of change and technology. And a lot of them are thinking about and realizing that they either have to get ahead of this technology or they're going to be left behind. And so it starts with some of these customers have already d to the cloud, but many are on that journey. And it all starts with helping them figure out how they get their data in a cloud world and organized in such a way that they can take advantage of it. I think increasingly enterprises and startups and everyone are realizing that their data is going to be their IP. That's the thing that's going to differentiate what they can do versus others. And if that's not in a cloud world, a bunch of these things around agentic workflows, around content summarization, around AI improvements aren't going to really materialize. And so number one is they really are looking at us to get that cloud estate in place as quickly as possible. And so this is actually a huge tailwind to getting more people to move their critical data and workloads into the cloud. And then number two, and this is often in parallel because again, things are moving so fast is they want to figure out how are they going to get real business returns for this? I think as I talked to a broad swath, and this is across almost every single industry, it's fascinating. People are finally realizing that it is their industry is going to change, their business is going to change, the jobs of every employee that they have are going to fundamentally change and they have to change along with that and move quickly. But they're looking for us to see how do they actually make that change? How can we enable them to make that change? And how can we make sure that it actually has a really positive ROI so that when they look at their financials and they look up, they have benefited from this change and they're not behind the eight-ball by spending too much money on things that didn't actually deliver value.

>> I always joke with a customer the other day of yours, and they're also looking at a whole on-prem, basically distributed computing paradigm, which is here, finally it's large scale. They say, "Our agent strategies are about data and tokens out. We need more tokens. Agents eat tokens for breakfast, lunch and dinner." This dynamic of tokens becomes a really big deal in this age of reasoning because it puts more pressure on tokens and they don't have unlimited budgets. So you have these enterprises saying, "Okay, I'm in the cloud, I'm running workloads, but now agents need to feed off these tokens, they need high performance, I need a new software kind of glue layer." They're looking at a kind of a, not reset, but like a re-architecture because the demand for tokens is putting pressure on more large scale systems.

Matt Garman

>> Yeah.

>> What's your reaction to that? What's your opinion?

Matt Garman

>> Yeah. Well look, I think the more these systems are able to think, they obviously require lots of compute capacity. Tokens is really just a proxy for compute time. It's just a proxy really for cycles. And that is what powers these, and these get bigger and bigger models and they think longer, they require more compute time. Now there's a couple of things that we need to do. One is we've got to bring down the cost of that compute. It's still too expensive today. It's a couple of orders magnitude cheaper than it was a couple of years ago and still needs to keep getting cheaper. And it is, and we're on that path and we're driving a bunch of things, some of it with our Silicon roadmap and Trainium too, but also we're building tools and technologies that allow people to cut costs in other ways. And so think about something like Q Developer and our QCLI, which is actually wildly popular with customers right now. When they're developing with that, sometimes I hear developers saying that they get 2, 3, 4 X performance boost gains because we're building agentic workflows across everything that they do. It's not just code completion, but it's writing documentation for them. It's writing unit tests for them, it's actually helping them reason about what the architecture of a system looks like and when you get somebody who can deliver something three X faster, the fact that it uses a little bit more compute time is actually turns out to be a rounding error. And so agentic is one of those things that's changing that math equation, but I think you're absolutely right. We've got to help them and help customers understand where they can get that value. And we're doing a ton to roll out tools and capabilities as well as infrastructure that helps customers make sure that they can get the value out of those things.

>> Yeah, I'll get into that a little bit later because I think that's a huge factor. I mean, EC2 made its bones in the fact that hit databases, people were running stuff on their applications, on their SaaS apps, on the cloud, agents changed the game. Let's talk about the business side. Obviously your growth, still phenomenal numbers. Talking to Andy back in the day, the number's still on a big number. So the growth rates is a function of the number. So talk about what's driving your business now. You can continue to see growth. Every earnings, we see the performance. Talk about the impact of the enterprise and startup ecosystem to the numbers.

Matt Garman

>> Yeah, well look, yeah, the business is going great and I think as we think about how do we help customers move to the cloud, we're still in the early stages of that. And we keep saying this and it keeps being true and I think that the number keeps getting bigger. But it's less than 20% of workloads at this point have moved to the cloud, which it just really ... I mean, stepping back, it's a huge opportunity for both our customers and for us to actually still take advantage of that core capability of getting their data in the cloud and taking advantage of EC2 and S3 and our database services, let alone all of the great new things that we're launching now. And so as we look at that potential, there's just massive potential there. And so I would say that if you look at the adoption curve, it's not like any other industry where there's the early adopters that jump in quickly and then you've got these kind of early enterprises who are kind of leaning forward. I actually think that generative AI and agentic is pushing a bunch of that kind of middle adoption people to realize it is now an imperative. Before, it was a nice to have and maybe even a little bit more nice to have, it was going to give you an advantage, but there was work and you saw lots of customers take advantage over the last 10, 15 years of being in the cloud and they accelerated more rapidly and they were more agile and able to adapt to changes. But there was still some kind of laggards that wanted to go slower or for good reason, sometimes good or bad. I think their customers now are realizing it's a business imperative. They are going to probably cease to exist if they can't take advantage of this technology wave. And so for us, that's a really big tailwind in helping customers of all sizes, frankly, governments the biggest enterprises in the world, banks, healthcare companies, retail companies, all the way down to one person, startups that are building the coolest technology aspects.

>> There's two things out there I want to get your thoughts on and reaction to because I think this is kind of a tail sign of the current wave. On the startup side, I remember back in the SaaS days, well, I'm programming and I was in that boat when I started SiliconANGLE. I didn't want to put a data center out there. I just used AWS. Startups then had SaaS apps, they would build it, they'd host it on AWS, throw the app in the App Store, and then off to the races and they become big brands. Now you got the startups doing agentic things, so it feels like the same vibe. So growth in the ecosystem of say inventors and startups. And the second thing is that in the enterprise side, I hear the term unlock a lot, unlocking value from the data that they have. Talk about those two things. One, the dynamics of the developer because that developer also sits in the enterprise. Now, in the old days, you'd launch on AWS, use higher level services. Now there's all kinds of services, but they're still innovating. So the startup ecosystem looks kind of the same, but it's different. It's not as easy. I won't say easy, but the old days, it seems easy. You throw it in the App Store and you're on the cloud, but now it's like you're enterprise by default because you have all these things to deal with, if you're going to build an agent, for instance, if you were a startup. Now the enterprise has all this data, that's an unlocking opportunity. So talk about that developer mindset of this generation. And then the enterprises are sitting there asking themselves, "How do we unlock the value of our data?"

Matt Garman

>> Okay. Yeah. Yeah, it's interesting. I think more and more we're seeing a bunch of startups be interested in selling to enterprise as you mentioned. And I think they see a lot of this. I think everyone sees this potential, and I think in some ways it goes back to your unlock is an interesting phrase that people say. And I hear that a lot too. I think a lot of this goes back to the companies realizing that their data is the key piece of value that they're sitting on that can really change the game of how they drive more value from their enterprise. And in a world where technology is changing so fast, the barriers of moving from enterprise A to enterprise B or whatever, the value that they really get is that data that they have because that is where they're going to differentiate. And so there's a bunch of startups that are moving from not just building consumer applications where a lot of startups were back in the day, but just really focusing on how do I help enterprises move quickly and get value from that data? And some of that is bringing sources of data together. There today it's been locked up in these kind of vertical data stores that they couldn't combine together. And these companies are really realizing that if you can combine the data from my ERP system with the data from maybe my HR system, which the data from my internal wikis from my sales system, et cetera, there's some super interesting information there. I can make my employees all much more effective and move more quickly and I can go deliver massive value to customers. And so it's a cost reduction play, it's an efficiency play, and it can go faster. And I think that you're right. I think a bunch of startups are building really interesting, fascinating things to try to help that unlock.

>> And props to you. You talked about startups before re:Invent and seeing some momentum. You guys are investing in startups. It's great to see the DNA back, the original Amazonian vision. In terms of the business infrastructure, large scales come up, sovereign cloud is a topic. You have countries who want to have certain protections. What does the footprint look like right now? I know there's news breaking recently about North Carolina expansion, $10 billion. You've had region deployments. I mean, on LinkedIn, almost every other week, you're announcing another region. Where are we on the footprint? What does that mean for customers?

Matt Garman

>> Yeah, there's a couple of things I'll say and I'll touch on the sovereign bit as well. So we're still rapidly expanding both into new regions as well as growing the regions that we have and growing that footprint and the investment that we're making in North Carolina is part of obviously growing our U.S. footprint to support again the massive growth that we're seeing from both AI footprints as well as our core computing environments. And as you said, we're growing internationally as well. We recently launched a region in Thailand. We recently launched a region in Mexico. We announced a new region in Chile. We have many regions coming online and they continue to add more and more of them because customers for a couple of reasons. One is they want to operate locally. And whether you're a bank or a utility or a government workload, they want those workloads to run in-country. And the customers that are actually from a latency perspective get better performance if their applications run and support the local citizens there in that country. There's one other region that we're launching, which we will be launching at the end of this year, which I'm very excited about, and it'll be the first of its kind called the European Sovereign Cloud, and that is a bit of a different angle, which is in Europe we saw a lot of countries and companies in those countries concerned about sovereignty at another level. They're really worried about how do I ensure that this region can really run independently if disruptions happen anywhere? And so for that, we actually launched this year, we're building this European Sovereign Cloud, and the idea there is that everything is fully self-contained. It has its own account structure, it has its own login information and IAM stack and pieces like that. So all of the data in every region, customer data stays in the region, they put it in, but our billing data goes back to a central place and there's a global login so that you can log into all these different places. This is going to be fully self-contained and operated. It's going to be operated fully with EU citizens. We actually have an independent fully-owned subsidiary that's a European entity that actually has a European governing board that's subject to EU laws. And so it really is, we've worked together with many of the governments in Europe to put this together as a way that they can then get comfortable that they can put their critical government data or critical what they consider to be systemically important workloads in this cloud and be confident that they have the right sets of controls that whether disruption happens anywhere and it's the first of its kind and-

>> So built from the ground up?

Matt Garman

>> Built from the ground up and no one has a region like this at all. The other cool thing is that it is a full AWS region. Has all the services, all the capabilities, and so when you go put a workload there, you're not making any sort of trade-offs on having to have a less capable cloud.

>> You've seen that movie before. I have to ask, in covering the CIA cloud and the GovCloud stuff, is there any experience that's learned from that, that applies here? Is it a completely different animal?

Matt Garman

>> No, I mean we learned a lot from some of our isolated government clouds like we do for the intelligence agency, which is just how do you operate something like that. And I think that we learned a lot from building that type of infrastructure. It's different because this has internet access and it's public and people can come use it, and we expect many of our customers, even our global customers, our ISVs are interested in coming and deploying here.

>> It's some of the same concerns, Hey, my stuff's going to run on this.

Matt Garman

>> But the operational controls, how we operate it and things like that, we definitely learned from our decades of running some of the government clouds and GovCloud and other things like that, yeah.

>> So I know you got all kinds of meetings going on, you guys are spending a lot and get the checkbook open. I think last time we talked about nuclear, we riffed on that a little bit, but the CapEx spend that you guys are executing on, it's unprecedented, at least in my lifetime, I've never seen this kind of spend. Talk about the rationale there. Is it competitive forces? Is it demand? Is it something that you're thinking, I hate to say it, but build it, they will come because they want it? There's demand. What is the difference on the demand side versus being proactive and preemptive on the market, on the CapEx spend, on the data center , obviously new regions coming online, tons of more GPUs, XPUs, you guys are spending a lot on this.

Matt Garman

>> Yeah, you're right. And it's demand-driven. We look to where the demand is today and part of our job is to think forward about where demand is going to be from customers a year out, two years out, three years out. Because a lot of times building data centers, acquiring power, things like that are pretty long lead times. And so that's a lot of where we focus our time today is thinking about how we balance that risk. But it's a very capital-intensive business. We knew that when we got into it and as you scale, the numbers get bigger and bigger, but as I said, we're also in the very early stages of adoption and so the potential is massive. Part of the promise we make our customers is that we'll scale for them and when we run out of capacity or we run short of capacity, it's painful for our customers. And so that is part of the promise that we make is being able to scale ahead of that. And as we know, much of the AI world has been short of capacity probably for the last two years and still is today, and we're still ramping as fast as possible.

>> So there's obviously demand.

Matt Garman

>> There's lots of demand. That's right. And we're careful about how we invest long term and we're thoughtful about how we manage risk in that spend, but it's to support demand.

>> It's not an overbill, it's simply you guys have an intentional plan.

Matt Garman

>> That's right.

>> All right. Talk about the agent. This tees up the billion dollars that are available. I mean some say we even ... We're riffing on our CUBE podcast that the agentic will be a multiple bigger than SaaS. If you look at kind of agentic as a size, and that's a little bit over the top, but what's your view on agentic? It's SaaS-like in the sense that it's a whole nother wave of applications, but it's up and down the stack. Is that the infrastructure layer and all the way up to the business logic, do you see that as a multi-billion dollar opportunity for AWS? How do you scope that? How do you thinking about agentic as a core AWS opportunity?

Matt Garman

>> Yeah, look, I think it's an enormous potential for many of our customers and I think if you think about all of the value that we've already seen from generative AI today, from generating of content to summarizations to things like that, I think that that is ... It's led to 10%, 20%, 30% efficiency gains and benefits for customers. I think the promise of agentic is when these models can go and actually accomplish tasks for you and long-running tasks where they can go do stuff, where they can use tools, they can use browsers, they can use capabilities out there, they can use text editors, et cetera. That's when you start to see not 20% gains but 200% gains. And so that's where I think that the potential is massive. It's a huge investment for us. I pulled a dedicated team together that's very focused on just going and building agentic capabilities for our customers so that they can build on us and take advantage of that. I think it's that important for us and for our business and for our customers. I think when I compare it to SaaS, I'm not sure if that's exactly how I would put it because I actually think that it's evolution. It's not like someone's going to take their application and throw it away. I think that they will still want to have those applications. Humans are still going to want to have a UI, we're still going to want to have a system of record, we're still going to want to have a bunch of these applications. But the ways in which you interact with them, the capabilities that they'll have will change. And so just how at my talk at re:Invent this last year, I said that inference is just the new building block and really think about it like that. In some ways, agentic workflows are just an extension of inference. They're just going and running models, taking data, but they can iterate more and do a bunch of these kind of reasoning loops and think for longer and do more capabilities, but at the end of the day, it's a building block inside of the thing that you're trying to go do. And so I think it's an extension of that. It's an adder on to that. I think it actually explodes the size of what the overall market can be, but it's not really a replacement.

>> It's still an application.

Matt Garman

>> It's still an application at the end of the day, and so it doesn't really replace it. It's just augmenting and changing.

>> It actually makes it bigger actually. Expands the aperture.

Matt Garman

>> That's right.

>> The business logic is at the app layer you get the horizontal scale, you got the global footprint, all the actions and the domain experts and the domain expertise and the domain data.

Matt Garman

>> That's right.

>> That's kind of in the vertical. So we started to see healthcare booming. You're starting to see life sciences folks that are kind of, I call first generation democratized super-computing. They're getting the benefits. What are you seeing in these markets? Obviously financial service is always in there, early adopter side, but healthcare specifically, you wouldn't have guessed. Well, you guys have guessed. But they really weren't IT-heavy mainly for all their other reasons, but now they seem to be booming. Healthcare seems to be hot. Life sciences, oh my god, a miracle. I got super-computing. More competition.

Matt Garman

>> This is actually one of the super interesting things, which I think if you would've talked to a healthcare life sciences customer six or seven years ago, they would've talked about this as kind of the bane of their existence. But a number of the regulatory agencies have forced a number of these customers, whether they're healthcare or life sciences customers, to organize and catalog and tag their data for a really long time. They have to keep long-term records, they have to know where it is, HIPAA requirements, et cetera. And because they had this repository of data that was actually really well labeled and organized, they were actually ready to move quickly and take advantage of AI-

>> AI, before they knew AI-

Matt Garman

>> Exactly. It was kind of a happy accident for some of these folks, but I agree with you. And by the way, it was probably an industry that, particularly on the healthcare side, not as much on the life sciences, but on the healthcare side, it was an industry that probably hadn't taken advantage of some of the earlier technology innovations where they've been a little bit slower to adopt.

>> It's just regulated industries almost are well-prepared because of the data hygiene they-

Matt Garman

>> Now they're going to have to change their mentality and move fast. And so we're still working with some of them to do that because I think that's the thing that could slow them down. And that's where you've seen a bunch of the life sciences customers and pharma folks move fast. They're more apt to move fast, they recognize the value here, and so they're building large language models based on finding new proteins or discovering new drug designs or frankly innovating on the whole pipeline of how do you think about getting drugs through the pipeline faster to solve different diseases. They've been really grabbing onto that and I think there's an enormous opportunity in healthcare where we see the opportunity to really reimagine that entire value chain. There's just so much waste in our healthcare system today where there's a lot of opportunity for efficiency gains, and that's an area that we've actually spent a bunch of time together with many of our partners working on that problem.

>> Yeah, I know you guys bought One Medical and that could be the Amazon Prime for healthcare if you think about it. That's a bad analogy, but you're saying it's like you need to get the user experience, which is the care side, then there's administrative.

Matt Garman

>> That's right.

>> Compliance-

Matt Garman

>> down the stack. It's providers, it's insurance companies, it's hospitals, it's up and down the stack, I think there's opportunity for efficiency gains where we can provide better care for citizens at a lower cost.

>> Okay. You're one year in as CEO. I remember when you came on board, we were commenting on, wow, product-led growth will be a big driver for AWS. You have product background, you're at the helm and the AI thing didn't hit the scene, but you guys saw it early. But now that you're in, you come in, you got the organization, there's been some changes, agents come in. How are you guys using it? Because I know you're using a lot of agentic internally from sources and folks that I talk to, some of it's public domain, so you're kind of and your client's zero, whatever the name is. Tell me some of the stories that you guys are doing one year in. What did you do to make the adjustment? How are you using AI to one be a better organization? Have you changed how you organize, how you budget, how you do the two pizza teams? What has been the impact internally with gen AI for you as a leader?

Matt Garman

>> Yeah, it's been massive and I'll tell you, I think we're still early in what we can accomplish, but I'll tell you that the initial thing where we see a lot of the value is frankly, it's encouraging our employees to experiment, learn what those technologies are out there. We have this leadership principle at Amazon called Learn and Be Curious. We want people to learn new technologies, not say, "Ooh, that's scary," or "I don't want to look at it." But we want them to learn what's possible and spark that creativity of figuring out what might be possible and how could we do things differently. And so we've seen a lot of interesting innovation there everywhere from fulfillment centers to how we think about organizing work, to how we do recruiting, to how we do development, and everything in between. And I was recently talking to my team and told them that I think every single job inside of Amazon is going to change and is going to use AI. And if you're not using AI, you're not doing your job to the best of your possible abilities because you're not being as efficient. And our job is to figure out where we're seeing the biggest sources of gain and then double down on that to help with tools and capabilities and frameworks. And so just like many other places, customer service was an early place where we innovated and we've seen massive gains there. I think where we see kind of scale manual work I think has been a big area for us and I'm quite excited. I think we are making a lot of strides in agentic software development, and I think that that is a huge opportunity for us to just ... For the entire history of software development, we've been limited by humans who are good at writing software. That has been the limiter. We've had loads of things that we wanted to get to and just not enough people who are great at writing software. I think that that equation is going to pretty quickly shift here and probably over the next 6 to 12 months, I think we're going to be limited by great ideas and not necessarily people who can write software because you're going to be able to write software multiple orders of magnitude faster, better. You're going to have security built in, you're going to have automatic best practices built in, you're going to be able to deploy and AI is going to help you with all those things. But you're still going to have to be thinking about what is that great customer experience I want to deliver? How do I want to change how that works? How do I want to think about how to use this data better? And that I think is going to, which is by the way, a pretty exciting world to live in when we can be limited by the scope of our ideas as opposed to the manual work that we have to do to implement.

>> Yeah, let the best idea win. I mean, you guys have already been a productive company. We've been following it from experimentation, the two pizza teams, how you guys make decisions, how you look at momentum and either go, no go. We've been tracking that, but now it's even faster. So one of your other core principles is bias for action. So I have to ask, has there been a lot of people taking a lot of action? There must be a lot of people coming out of the woodwork. Is there innovation increase internally? Are you seeing people in your jobs, "Hey, I just solved the problem, I don't need to submit a request. I built an app." What's been the impact of the organization there as a tsunami of ideas?

Matt Garman

>> Well look, and this is one of the things that we're also trying to help our customers get their heads around is how do they make sure that they actually still have the right controls and compliance around the things that they go build? Which is an interesting problem because when you see that rate of change and that much more code being created, how do you really think about making sure that you can still have the right security controls and compliance controls? I think we're thinking about things to build to help them with that as well. But I'll use one example is a bunch of teams internally were building a bunch of agent workflows and they found that it was hard. They were found that they were having to do build a bunch of scaffolding and whatever. And so they built this framework internally just to help them go and build the applications. And they found out that ... They spent a couple of days building this framework, maybe a couple of weeks. And once they got it, the velocity of their development really accelerated. They found out, okay, this really helps. Now I can go just think about building agents and thinking about how I set up these systems and this framework really works. And the team said, "Well, if this is useful for us, maybe we should give this to our customers."

>> Sounds familiar.

Matt Garman

>> Exactly. And they're like, "Well, we should just open source this. It's not really service. We'll just open source this as software." And historically they're like, "Well, there's an open source process we could go to." And they're like, "You know what? The world is moving so fast we got to do this quickly." So they wrote up a three-page document of why they thought it was a good idea. They said, "Tomorrow morning, let's get all of the deciders in the room and look at it." And within eight hours we decided, "Okay, great, we're going to go launch this." They packaged it up a couple of days later, we launched the Strands SDK, which has been hugely popular and has really unlocked both for us and our customers. It's called Amazon Strands SDK, and it's this framework that's an open source. Other people have already integrated it. I think we've seen Google integrate it and others have integrated their services into it. But it's a fantastic and really easy to use framework that helps people build agents more quickly. And it's a good example of just a couple of people were building this on the side and they said, "Hey, this is something that can help our customers." And it's taken off like wildfire out there. And is-

>> This is my point. I mean, one of the things that's unique in this time, this is a good illustration, is that within the new models, new models being whether it's foundation models or just new business, new technical models creates new usage patterns. This is a great example of a discovery that was intended to be just a test bed or something, become something.

Matt Garman

>> That's right.

>> This is new. We've seen this kind of slower roll before. That's where the two pizza teams kind of came in, but now this is happening at such a high velocity. What does that mean for customers? Because their workflows are changing, their cultures are changing. As a leader, have you changed how you reward people, how you greenlight projects? In this case you proved it, but what else are you missing? I mean, productivity is the key.

Matt Garman

>> Yeah, look, I think you both still have to have a long-term vision and an important strategy of where you're going. You can't just have people going off, and I don't need AWS to build a taco stand. I guess we have a banana stand out here outside of Amazon, but that's not part of the strategy of what we want to go build as a business. And so it's got to still be within the strategy. And so I think it's important for customers to think about what is the core thing that they focus on, how do they direct their teams to run fast, but in a direction and not just kind of randomly scattershot? I think it's important. And what I'll tell you is I think it's a critically important thing. It's another reason why customers need to be in the cloud. We can help them be agile, we can help them move quickly. When they're in the cloud, you can be more reactive to change. I think one of the things that I've heard a lot of people say is I remember coming out of COVID actually, I heard a lot of customers say, "Oh man, I can't wait for things to just get back to more normal and settle down." Well, that hasn't happened and I don't think it's going to happen. And whether there's geopolitical change, whether there's technology change, whether there's a global pandemic, whatever the thing is that happens, customers and companies need to be ready to deal with that change. And being in a cloud world allows you to be agile, scale up and down, move to different geographies, change technology stacks, whatever you need to do much more quickly. And I think we can be a big strategic lever for-

>> It's interesting in COVID, one of the things, that term was used, the new normal, it never went back. It's just the new normal. And then things changed from there. AI's got a similar kind of philosophy where the innovation equation is different. How you organize your teams are different. This is a huge deal that customers are struggling with. And also they're wondering, how do I architect it? So coming back to some of the application side, you got to run on something, you got to run your apps, your workloads on something. So the question comes up, how do I decide my 10-year future, next 10 years, next 20 year innovation cycle? We seem to be seeing that certainly in the enterprises and as they think about their cloud. What's your vision on that? Because there's a lot of things being peddled to customers right now. Buy my system, and we're the only answer. Power my system. When people just want a heterogeneous, cost-effective environment, how should they be thinking about the horsepower under the hood, XPUs, the GPUs, the configuration? I know you're close to this.

Matt Garman

>> Well, look, one of the benefits of, and what we see with customers and we tell them is like, "Look at AWS, we want to be your decades-long partner. We think we are the right technology solution, we have the broadest set of capabilities, we innovate the fastest and we operate in a way that we are true partners to our customers." And I think those are the things that you're not going to want to change. You're going to want your technology partner to be a great partner of yours and not somebody that doesn't necessarily have a history of doing that. You're going to want somebody who has a history of innovating really fast and scaling with you. And then when you're on a platform like that, you don't necessarily have to lock yourself into that. When you buy a server and you stick it in your data center, you've got it for five or six years. When you build a data center, you've got it for 10 years. And if all of a sudden you're like, oops, this needs to be in a different country, you can't move your data center, you can't move your server. And AWS, you can do that. And so in the cloud you can do that. If a new technology comes out, or a new model comes out, if you're running software on-prem that's the software you get. And so I do think that that is increasingly where you see a lot of that value. It's just assume that you don't have to make those decisions. If you can push those decisions off and just say, "Look, I can trust AWS because of their track record and that partnership and promise and investment and rate of change and innovation that I'm seeing out of them, I can trust them as a technology partner," a lot of those other things you can trust that you can adapt as you go.

>> Yeah, there's a lot of engineering going on around solving the distributed computing scale problems which you guys nail and Sovereign Cloud is going to give that benefit. When you look at the innovations on the software layer with the infrastructure not at the agentic layer, that interoperability has to work. Are you seeing any things on your end where you say, okay, this is the kind of configurations to enable the next DeepSeek-like innovation? DeepSeek was an innovation in the sense of how they got there. And in fact, two years ago I asked you about models. You said, "Yeah, we want to have all the models." Bedrock was born, but you also write about the models integrating. The models themselves are innovative opportunities. Anthropic's performing well. So you got to get the models with the chips with the software. So how do I look at that when I'm thinking about running and optimizing that software layer for my company? What's your vision there? I know training and inference have been innovating on top of these chips.

Matt Garman

>> Yeah, well, most customers are not going to have to worry about a bunch of those pieces. Most customers are not going to have to think about how do you go build a foundational model. There's a few. There's the Anthropics of the world and others that are going to go build these really large models. Most of the others are either going to do some fine-tuning on top of those and create their custom data. And they're mostly going to focus on inference and you're working through Bedrock. You don't have to think about necessarily, do I use GPUs or do you training or do I use whatever? You're kind of operating at that API layer, which is a good abstraction layer because it allows you to move quickly and just think about your application. And so that is kind of where we are trying to help customers not have to deal with all of that changing complexity at the core infrastructure layer and the model layer and just give them a rich set of capabilities so that they can think about how do they construct their agents or their applications.

>> But you guys have differentiation. I mean, again, if I was a model and I had a model, I want to work with Amazon, the most hardware gear and customer base.

Matt Garman

>> Well, we definitely try to differentiate and we do differentiate at that layer. I think look, the custom silicon is a huge competitive advantage for what AWS does. I think when you go look at, of course Nitro, but also you look at Graviton, it's been wildly successful and many customers save 30, 40% off of their compute bill by using Graviton, which gives them better performance at an absolute lower cost. And that's a huge win, and it is a differentiating capability that is not available anywhere else. And so it is a reason to use AWS and it is a reason that we see people use AWS and save money.

>> And what are some of the stats on optimization? Because again, we've always said, I think you've said it too, but what's around the chips that matter, you can optimize for certain use cases. Again, you have a large scale infrastructure, very broad, a lot of customers. It's almost a sandbox for the models to be in there. What are some of the gains you're seeing? Can you share whether it's Anthropic or maybe custom configurations from a customer workflow?

Matt Garman

>> Yeah. Well, there's a whole bunch of things. And I don't know if we have hours to go through all of the components.

>> Just give one example.

Matt Garman

>> But I think, look, I do think that at the infrastructure layer, you look at something like Trainium. It's enabling Anthropic to go build their next generation of model at an absolute lower cost with better performance. And so we've talked about building this massive cluster that's five times bigger than the last one that they built. And that's why they're able to stay ahead and still have industry-leading models in that sense. But it's not just about models too. I think when you think about models, they need databases. And actually Amazon DSQL is a fantastic database for people who are building agentic workflows because it can scale down to zero, it can instantly scale up, it can scale globally, it's durable, you don't have to think about it, you don't have to manage the infrastructure. And so we're actually finding that as a very popular database for AI and agentic workflows where people can just call this database and create and scale from there. And so all of these components really go to building a unique and differentiated set of infrastructure components. Even something like S3 where we've really reinvented what S3 is capable of doing.

>> I think the diversity is going to be an enablement. I think next time this year we'll probably be talking about what that means because I think I was talking to an engineer friend and we were debating about small language models and most people think I'm just going to build my own model. He said, "Just distill it off, Anthropic." So just the idea of distilling from something to get something smaller. So that's a different thinking, that's a whole different enablement. So are we kind of in this new age of enablement from the integration of Amazon's infrastructure with these new models and the new capabilities?

Matt Garman

>> I think that's how most people are going to over time build custom models is they're either going to do fine-tuning or post-training or distilling from these large called teacher models. And what the idea is, is you have this really large model that knows everything about everything, but it's slow and expensive because it's huge. But what you do is you take that and you say, I don't need you to know everything about everything. I need you to know about this one thing. And so what you do is you use this teacher model to train a model to be an expert at the one thing need it to be. And so whether it's legal documents or coding or how to process an insurance claim form or whatever it is. And so it doesn't need to know about ... If it's processing an insurance claim form, it doesn't need to know about how to write haikus or about physics or anything like that. And then you get that much smaller model that can be faster, better responsive and cheaper. And it's a great model then to go run and do that. And so I think that is how, and by the way, when you do that, you can add your own enterprise data to that too. And you create this custom model that's distilled from the large teacher model added with your own proprietary data. And that's when you can have a really powerful model that's great at processing insurance claim forms because there's also some data you added, that I'm assuming you're an insurance company, that only you know about.

>> Yeah, it's an exciting time. I love talking with you, man. You're like a helicopter. You can go high level and then go deep into the weeds on the tech. I have to ask you if you can go back and let's just say we were in college together and we were roommates, and we wanted to do some innovation, what would you work on now? Knowing what you know now, obviously you got the keys to the kingdom, you're seeing everything, what are some of the opportunities? A lot of young people coming in haven't seen some of the waves. They were born when the web hit the scene. So this is a major wave that's coming in here. What would we build? What would you work on?

Matt Garman

>> Wow, that's a good question. I think there's never been a better time to innovate, honestly. It is an incredible opportunity where you don't have to be an expert at coding. You don't have to be an expert at architecture in order to go build a new system. I think we're not that far away from when an individual person or two or three people can build an entire company just with them and not have to actually go hire tens and hundreds of people. Eventually they'll want scale, but they can build the core of a product just in their dorm room. And I think that is a fascinating opportunity. And I will say there's some fundamental things that don't change. At the end of the day, you still have to solve problems for customers and really think about keeping customers first. And so sometimes we get blinded by the cool technology pieces and whiz bang, whatever. I would still think about how do I find a customer need that is unmet or that I think I can meet better and the tools are there for me to actually go execute on that vision. But like I said, I think we're going to be limited by ideas. And so really coming up with what is that customer problem and how can I solve it in a very different way than someone else has done before? I think that doesn't change and that's still how I'd approach the problem.

>> I love that mindset. In fact, you go back in the past year, I would've been like, "Yeah, enterprise is where the action ... That's where the action is today." But I still think we're going to see a wave of massive consumer apps because the consumer market seems to be kind of, I won't say shut down in terms of there's not a lot of startups saying, "I'm the next consumer app." But I think to your point, if someone could nail a user experience, there could be a big consumer-

Matt Garman

>> There's a lot of consumers out there. So yeah, I think it continues to be a big opportunity and I am sure that people will come up with new experiences and new solutions for consumers out there, for sure.

>> My final question for you is, Andy wrote the Next Generation about the why culture. What does that mean to you at AWS? And for folks out there either internally or externally in their own companies who have want to take a bias for action, anyone can change the game. It could be any position in any company. You don't have to just sell the CIO or anyone else. There's a lot of innovation, a lot of ideas out there. What's the why culture mean for you?

Matt Garman

>> It really is just ... It's questioning how things work and why they have to be how they are today. And I think that the core of that is it is that we want to innovate. You ask why does something operate like it does now and why did something happen and how do we go fix it? And it's being willing to introspect and think about what you can do different. It's willing to look at how the world is today and how does it have to be that way and how can we change it? And some of that is our organization, some of that is our technology, some of that is our products. And that's why we go in there and innovate in new spaces and continue to learn and develop for-

>> Like that example, Strands, that's a why question.

Matt Garman

>> That's right. Why do we have to be doing this work every over and over again? I can go build this framework and then why should I keep this framework for myself when customers can get that benefit and build more quickly? And now we've got a really popular open source project out there that's enabling tens of thousands of AI developers to build agents much more quickly.

>> All right. Since the last time we talked at re:Invent, now it's halftime, in the midpoint of the year. Looking back, what's your view of what's happened? I mean, this pace of play has been high, very fast. Six months we'll be at re:Invent. What's happened? In your mind, what is the key takeaway for the past six, seven months?

Matt Garman

>> Yeah, look, I think a couple of the key takeaways ... Look, it all comes down to the world is not slowing down and speed really matters. And for us, innovation matters, speed matters, the pace at which our customers need to move matters. And I do feel like it's our job to keep up with that pace and to move faster than everyone else so that we can make sure that we make things easier, more approachable, and more focused on delivering value for those end customers. And it really comes down to that. And I think our teams are energized by that speed, they feel that urgency of the speed of how fast we've got to move because the world is out there moving fast too. And frankly, pretty excited about the opportunity.

>> Large scale game. Great to see you.

Matt Garman

>> Awesome.

>> Thank you for your time. Matt Garman here taking the time out of his super busy schedule to meet with theCUBE for our kind of halftime look at the midpoint of the year. Thanks for watching.