AWS Mid-Year Leadership Summit 2025 | Atul Deo, AWS

Clips
News
More from AWS Mid-Year Leadership Summit 2025

Atul Deo

Director of Product, AWS Bedrock

AWS

Exclusive with Amazon Bedrock chief Atul Deo: AWS’ big bet on cheaper AI — and a new breed of software agent

Atul Deo’s goal is to make artificial-intelligence software both cheaper and smarter at the same time. The up-and-coming executive, head of Amazon Bedrock and generative artificial intelligence for Amazon Web Services Inc., has only six months to show it can be done before the company’s blockbuster annual re:Invent conference in December.“The AI space is moving faster than anything I’ve seen,” said Deo (pictured). “Models get better every few weeks — but customers won’t deploy them unless the economics pencil out.”That tension between technological leap and production cost is now redefining the cloud industry’s next phase

Model mashups and AI agents: How AWS is preparing for the next workforce

Amazon Web Services Inc. is ready for the next generation of workers — and by that, the company means a host of different models and artificial intelligence agents.The cloud giant is preparing for the upcoming AWS re:Invent later this year, with a series of product updates based around intelligent automation and agentic AI.TheCUBE’s John Furrier talks with AWS’ Atul Deo about the development of AI agents.“We talk about agentic capabilities, but particularly models being able to follow instructions and being good at it … make a huge difference in the entire workflow,” said Atul Deo (pictured), director of product, AWS Bedrock, at AWS

play_circle_outline New models like Sonnet Four and Opus Four enhance reasoning and instruction-following capabilities.

play_circle_outline Prompt caching enables significant cost savings by reducing repetitive token usage for agent workflows.

play_circle_outline Intelligent prompt routing dynamically selects models to optimize cost and performance for applications.

play_circle_outline Embracing Asynchronous Workflows: The Shift Towards Generative AI and Real-World Communication Dynamics

play_circle_outline Customers must balance accuracy, latency, and cost when selecting models and infrastructure.

play_circle_outline Automated reasoning gives customers trust in model outputs by ensuring high accuracy.

play_circle_outline Model choice is crucial for different use cases, leading to increased diversity in applications.

play_circle_outline Multi-agent collaboration allows diverse models to tackle complex workflows efficiently.

play_circle_outline Observability is critical for monitoring interactions and performance in agentic environments.

Info
Transcript

Atul Deo, AWS

Atul Deo

Director of Product, AWS Bedrock AWS

In this exclusive AWS Mid-Year Leadership Summit conversation, Atul Deo, director of product for Amazon Bedrock, sits down with theCUBE’s John Furrier to unpack the rapid evolution of generative AI across AWS’ ecosystem. From infrastructure and model innovation to agent-based tooling and cost optimization, Deo offers a comprehensive look at how AWS is shaping the next wave of enterprise AI adoption.

Deo dives deep into key updates from the past six months, including the launch of powerful new models like Nova Premier, Claude Sonnet 4 and Opus 4, all ... Read more

explore Keep Exploring

What recent developments have occurred in the field of AI models, particularly regarding their capabilities and performance in following instructions? add

What is the significance of prompt caching in agent use cases, and how does it affect cost and performance? add

What new feature related to prompt routing has been made generally available, and how does it benefit customers? add

What are the trends in how customers are applying generative AI, particularly in terms of real-time versus asynchronous use cases? add

What capabilities were launched at reInvent last year to improve the balance of accuracy, latency, and cost in model performance? add

What is Bedrock's approach to ensuring the accuracy and reliability of its automated reasoning capabilities? add

What are the benefits of offering multiple database options to customers, and how does that relate to the development of foundation models? add

What are the advantages of using multiple agents powered by different models in problem-solving? add

What are the current developments and considerations in the application layer of technology, particularly regarding customer understanding and security in the context of generative AI? add

bolt Powered by CUBE AI

Atul Deo, AWS

search

>> Hello, I'm John Furrier with The Cube. We are here in Seattle at AWS headquarters for a half-time report with The Cube and AWS getting the updates. What's happened, a lot's happened in the past six months. Foundation models, the AI infrastructure's advancing very, very fast. The model efficiency, the new usage patterns are bringing in a new wave of innovation, setting the table for the agentic wave, but ultimately the business logic and the applications, powering all that are these models and how they work with each other. We've got a great expert here. Atul Deo, the director of product at AWS Bedrock. He's got the keys to the kingdom. We're going to get the scoop here, so great to have you. Thanks for coming on. Good to see you.

Atul Deo

>> Thanks John. Excited to here.

>> You've been building at Amazon multiple projects engineering side. Now, the hottest area of these models because everyone can see the benefit. We're starting to see the first wave of agents coming in on workflows that are well-defined, differentiated, heavy lifting, but it's exciting because it's becoming a programming layer of value on the data where these models are shining in the enterprise and these new consumer apps that are coming out. You're starting to see new entrepreneurial activity and the startup ecosystem your partners are integrating in. There's all kinds of integration opportunities. There's new kinds of opportunities around using data in new ways. There's new usage patterns that are spawning more creativity. A lot of people are looking at this and saying, "Wow, I could actually refactor my business model," so just up and down the stack, there's massive amounts of innovation. Give us the updates. So much has happened. It's like a full year has happened in six months on your watch alone, I can't even count how many updates on the models. There have been Nova Claude Anthropics kicking butt with you guys, so we're seeing tons of stuff. What's the update in the past? What's happened in the past six months?

Atul Deo

>> Obviously, John, a lot's happened in this space. This is one of the fastest growing spaces that I've at least seen in my time. Obviously, we've seen the emergence of a lot of capable models. More recently we've seen Sonnet Four and Opus Four. These are really powerful models. They're particularly well on agent use cases. They have improved reasoning abilities. They're also great at following instructions. Now, I think this is one of the underrated kind of things that we don't talk about enough. We talk about capabilities, but particularly model, being able to follow instructions and being good at it, I think, makes a huge difference in the entire workflow. We are seeing kind of improved ability for of these Claude models. We also have been busy launching some of our first party models with Nova. We launched Nova Premiere, which is an incredibly powerful model, which also comes with remarkably good price performance. On the model, in addition to the model layer, we have also been busy launching, so we had reInvent last year, we launched a number of capabilities based on customer feedback. One of the most popular capabilities in which personal favorite of mine is prompt caching. Now, the thing with prompt caching, particularly as you think about agent use cases, getting the agent to work is one thing, but also making it cost effective and fast is an important thing. In agent use cases, there's typically a lot of instructions that are part of the prompt for the agent, and now these instructions have to be fed every single time and the cost can add up. Also, the time to first token increases if you don't cache these tokens. With prompt caching, in Amazon Bedrock, which we just made generally available in the last few months after reInvent now allows customers to use this capability for Claude models for Nova models and get remarkable price performance so they can save up to 90% of your costs by caching these tokens. Now, that's just one example. In addition to that, we also made intelligent prompt routing generally available. Now, this is combination, allowing customers to use a combination of a cheaper model and more capable model, I should say faster model and a more capable model for the same application. Now, dynamically at runtime, customers can send the same query to a cheaper model or a faster model if it doesn't have to necessarily invoke the larger model. Now that, again, can lead to significant cost savings and practically no difference in terms of the customer experience. Now, obviously, in addition to prompt caching, intelligent prompt routing, we are seeing customers use, I think, the paradigm of how the applications around generative AI, they're changing from just being pure real-time in nature to this more agentic kind of nature, where not every request has to be served immediately. The way most customers have applied generative AI, they almost had the existing use cases. Again, if you think about pre-generative AI, most popular use cases for AI, were in customer care where usually there is an end kind of user and they need a response right away. Now that's the very typical paradigm where customers generally does those things really well and I want a response right away. Where the puck is now shifting and we are increasingly seeing a focus is more on the not so real-time, more asynchronous use cases. Typically, it's kind of very similar to how real world happens. Like one of our execs or Matt sends me an email and asks for something complex, he's not expecting a response in near real time. He would rather have me go spend some time.

>> Yeah, think.

Atul Deo

>> Think, do the work, collaborate with other people and get back with some comprehensive thing and not just tell me what the answer is, but tell actually that I have done what you asked for. I think that kind of paradigm does not ... That requires a different set of tooling, different set of infrastructure capabilities in the background because you want to be able to do that in a cost effective manner as well, because doing everything in a near real time is not going to be scalable in the long run. I think we are starting to see that's where I think particularly now these newer models, as we were talking before we started streaming, there are two things that are required here. One is a set of tools to make agents. The second is the models themselves, and I think the capabilities of both have to go lock in step. If you have a very awesome set of tools but you don't have the capable models, you're not necessarily going to have the desired result that you're expecting.

>> Yeah, I love that. I love that progression because Mary Meeker just put out a report recently. It's like a hundred zillion pages, but why that's important is that she pioneered the internet online population report back in the web wave and that was online population. More people use the web, therefore, advances happen. We're seeing that progression of some maturing fast. First token out is a concept. You mentioned the one shot answer fast.

Atul Deo

>> Yes.

>> Okay, I got a response, great answer, and I put the LinkedIn post or whatever I'm doing. Now, you have multi-step thinking, which basically is a data problem, data architecture problem, but also, a relationship to the infrastructure because now you have resource management. We're starting to see the advanced aperture of use cases scale out to be broader. I mean, you got more broad use cases. I want to think and have a better answer. I got more time, I want a fast answer for a token out or now I got reasoning, more tokens are needed. All that puts pressure on the scale of the system, both the hardware and the clusters as well as the data layer. What's available for data. Again, the complexity is increasing as the functionality demand increases, more tokens requires bigger systems, compute. That seems to be the current state. Now, we didn't even get to the application layer, so that's just looking down, looking up the business logics is being integrated in, so that's kind of like the current problem. What's your reaction to that and how do you guys view that? Because this seems to be the next hurdle of innovation where, okay, I got more complexity, I need more data sources, I got to manage the hardware from an efficiency, performance and cost standpoint while providing the app layer of support. That seems to be, what's your reaction to that? What's your frame?

Atul Deo

>> I think let's go from different layers of stack. I think first and foremost, you got to have incredibly compelling hardware and I think we have been investing a long time in infrastructure and specifically, custom chips and we are heavily using Trinium two for Amazon Bedrock and for powering these models, and it allows us to get good performance for our customers. The next layer of abstract kind of in terms of the stack, we think about Bedrock infrastructure and I think they're obviously the pure on-demand, real time, almost real time responses. I think that has been the existing paradigm. We also introduced an offering called Batch. Basically, allows you to send large volume requests and get response. Sometime in 24 hours it is 50% cheaper, but I think we do expect not every use case there is something in middle as well and you're going to start seeing where , it's like usually I don't respond to the email in 24 hours. That's too long of an SLA, but I usually have to do hustle and respond in a few hours. We are going to see some improvements in how fast we can do that. Again, that allows us to help customers move faster but also, be efficient on the infrastructure side. We are going to do make a lot of improvements on our end so that customers can get responses that are not, as I said, not real time but can get them faster. The next layer of iteration would be on the tooling side. You go to see, you obviously have seen MCP has become a big protocol. We expose a lot of our AWS services. We have MCP server implementations that we have publicly released. Customers can use that and start interacting with our services.

>> Why is MCP important right now? Why is that relevant? I mean, it's certainly cool, the alpha builders are all over this. What is it and what's the benefit?

Atul Deo

>> I think there are two things that come to mind with MCP, for me, I think the first is the analogy, I think this is not my original analogy, but MCP is kind of like a good standard interface for connectivity. It's like the USB-C for AI. I really like that analogy.

>> Like one interface.

Atul Deo

>> It's a great connector and I think the different-

>> Model connector?

Atul Deo

>> Correct, correct model and your data. I think it's a good interface for connecting models to data and systems in general. I think the two things that I like about MCP, number one is things that dynamic in an application can discover different sources and it doesn't have to be hardwired beforehand. Second one is unlike API, which is typically stateless, this one can maintain state and can be secure and things like that, so those are kind of interesting.

>> Yeah, I mean, it's just a phenomenal advancement. Then the tooling side's moving fast too because a lot of the integration in the old way, the way of APIs, okay, connect APIs, but now you have agents that have state, there's delegation, there's trust requirements, they're doing things, they're taking action, they have agency. This is now a new factor.

Atul Deo

>> Yeah, agency has a new meaning now.

>> Agency, authority, but they're acting on behalf of something a little bit more intelligent and more intelligence will come in. That's a whole another thread, but that's a super new thing. I would imagine with the tooling advances, with the coding assistance and with Q, Q for developer, you're going to see massive, "Hey, I got to build the tool to integrate. I don't need to spend weeks building a tool, I can do it in 48 hours."

Atul Deo

>> Absolutely, so right now, agents are still the models. As I said, models are getting better and better and so far, I think the biggest area where we have seen agents deployed successfully at scale have been primarily in their realm of code generation and software development broadly. We are starting to see examples where customers are using agents for complex use cases, but so far they have been mostly in the software development space and I think as models like Claude, Sonnet Four and Opus Four, as they become ... Right now, customers are probably starting to use the models and then the actual deployments at scale for new use cases other than existing code generation use cases that are starting to happen, we would probably see some of these effects in the next few months.

>> It's interesting at reInvent, everyone's always trying to read the T leads and connect the dots. I think you guys were on the right side of this, but what's clear in the past first six months of the year is looking back at the first half of the year, it's clear that the two hottest areas are sales and marketing and coding. If you look at it from a business standpoint, that's where the value is, right? The value is right in those areas. Every business has to code stuff. There's applications, there's interfaces, and then obviously, sales and marketing is you sell stuff and sell value. That's going to expand and now agents are going to work in both those areas, so now multi-step reasoning is a hot topic. This is the multi-step it's thinking, some people call it thinking, but there's reasoning. There's multiple steps to take. Swami talked at length about this both at reInvent and HumanX. I saw his speech there. This is kind of where the business logic meets kind of agent Bedrock layer. This is now the fun kind of the innovative, creative, builder piece. What's your view on that?

Atul Deo

>> Yeah, so I think one customer example, literally as you were mentioning that that started coming to my mind, which was I won't name the specific company, but it's a mortgage startup and basically, they have been able to implement agents to completely automate some of their end customer workflows. I don't know when it was the last time you got a mortgage, I got one few years ago, and I think the whole process is so painful and I think this company is making use of agents to dramatically improve the customer experience and they're seeing improved a significant multi-fold improvement in their conversion rates with customers based on this agentic experience. Because they're no longer tied to, hey, the human is not being responsive or they're not, and even on their internal workforce, the efficiency and the effectiveness is tremendous because the customer's already done this, here's the next step, here's everything's attached, and you're like, "Oh wow." The internal employees are seeing a massive boost in productivity.

>> Yeah, and I know that Amazon, you guys have a mandate that everyone should be using AI and build their muscle teams have been directed by Andy and Matt. Hey, leverage AI because you got to perfect it. If you're going to be the master of AI, you better use it yourself. To your point about this example, there are all these, I would say, boring areas out there that are being innovated by non-techies. I interviewed a startup, they're working on agents in real estate transactions. You got a buyer and a seller, the seller of the building and you going to buyer of the building. Now, that process takes months. They've shrunk it down to days because they're using AI. Now, this is a creative person who knows the process, not a coder. The old way was like, "Okay, great idea. Hey I need an engineer, where's the team to build it?: She on her own got the prototype up and running, proved the use case, gets a coder, has code assistance, they're in market in weeks, they now have a business. This is the innovation we're seeing in the market.

Atul Deo

>> Totally.

>> This is real, so I've seen great examples. Now, let's go to the enterprise. They're going, "Wow, we used to do things the old way. Now we're in the cloud, we have now secure data on prem. I got the edge exploding, I got computer vision. That's a data problem, so I'm going to need to up my cloud game, deal with my data, I got my databases everywhere." That challenge is a little bit different. How do you view that when you talk to customers that have enterprise AI needs? They're kind of reverting back to fourth grade and go, "Wow, we should put it on prem." There's kind of a debate right now between, "Okay, what is the ideal solution to leverage my cloud presence and my on prem needs?" Because sometimes compliance requires an on prem if you're a bank or whatever. Sometimes they're in the cloud, sometimes there's all on the cloud. There's a data engineering discussion around, "Okay, what do I do?"

Atul Deo

>> Yeah, so I'll just be candid. I think most of the customer examples that I have come across, my conversations I have not usually seen the on prem conversation come up. What typically comes up is, "Hey, I want a better balance of accuracy, latency, and cost." One of the things that we also launched at reInvent last year was this model distillation capability. I know that's one of the topics you are passionate about is really taking the outputs of a larger model and using that to pass on the intelligence to a smaller model, which is obviously, going to be more cost effective, but more importantly it's also going to be fast in terms of the end user responses. We made model distillation generally available. It is available for cloud models, it's available for Nova models, for LAMA models and you obviously can use, for example, you can distill intelligence from a Nova Pro or a Nova Premier into a Nova Lite or a Nova Micro and that obviously, you can use some of the larger intelligence models, intelligence into the smaller ones, and I think having that workflow be seamless, I think, is kind of where the magic is.

>> Talk about that. Let's unpack that. I think this is a hot area because there was a big rage about small language models. Of course, we were promoting that because we think small language models have domain expertise, but you don't have to build a separate model. You might have a separate small data model. It could have language in it, could have vision or computer graphics and multimodal capabilities, but that doesn't have to be a model, like a whole language model. Distilling off the big models is the technique because it's faster, saves time.

Atul Deo

>> Essentially-

>> Or it does it, I don't know. How are people distilling off the main models and why?

Atul Deo

>> Yeah, so essentially as I said, typically customers for their end applications, they want that. They want to be a specialist in that particular area and usually these large models are, they are experts in almost everything but they're not specialized enough on one thing and for when you are actually deploying a model in production, you don't care about probably everything kind of brilliance of the model. You don't care about, "Can this model be really good on agent use case and chain of thought?" You probably don't need that use because it's in your application, it's targeted for a particular use case. What you really want is, "Hey, is the model going to be incredibly good on this mortgage workflow?" For example, is it going to really understand what the customer is kind of questions the customer is going to ask kind of things that you are going to organize and do? Essentially, what happens in this case is as a customer when I am going through a distillation workflow, I'm going to give a bunch of examples for bunch of prompts for the larger model and I would extract responses, and I'm then basically behind the scenes what Bedrock would do as part of the distillation workflow is we'll generate additional data for the distillation process based on the prompts that a customer provide. Because the customer does not have to do the work to be comprehensive. It can give a few indicative of a 30, 40 prompts of what it wants generally for the distillation purpose. Then behind the scenes, we can generate 10 x more data, which is basically synthetic data to then that synthetic data response of that larger model then gets used to essentially make the smaller model more targeted and focused. Generally, as you know, when you fine tune any model, you reduce some of its generic expertise and you make it more specialized and that ultimately, that distill model can then be deployed in production and you get your better combination of accuracy, latency and cost for that particular .

>> I love that targeted approach and again, I think one of the things we're seeing, I'd love to get your reaction to it, is it reminds me of the scene in the Matrix, the movie, everyone's seen the Matrix, the scene where he says how to fly a helicopter and then they upload to Nero's head the helicopter, now he's a pro helicopter driver or judo expert. That's what models are, and so I see verticals like healthcare, life sciences, we're seeing robotics plugging in models from open source. There's a trend towards a long tail and specialized models that have to be accurate, not stuff in hugging face just like we're talking this is approved. I see an era where models will be SLA-based, accurate, targeted. How does that work in your mind? How does that going to progress?

Atul Deo

>> I think the robotics use case is an interesting one that's probably going to progress, but particularly in areas like healthcare and financial services where the accuracy of the model like LLMs are going to hallucinate like no matter what. One of the areas where we have been actively investing with is an area of differentiation for Bedrock is automated reasoning. We also launched that one of the capabilities last year. Where we really want to make sure that the responses of the model can are not just based on LLM but we have independent verifiers of the model that basically use mathematical proof and basically say based on the company's kind of rules or knowledge base, we have verified this and the answer is like 99.9% correct. If it is not certain, at least it'll tell you and when it says that it's certain of certain answers, it can actually show the various rules that it knows of in your corpus in the customer's corpus and that it used to ensure it's-

>> Humans can get in the loop and then look through that and say, "Okay, I verify that," and be confident.

Atul Deo

>> I think that is an area where particularly as the use cases that is model start getting and when they're getting into mission-critical areas, you are going to need more certainty. They're good enough. The developer use case is a good one because this is an expert. Developers are experts, they are in the loop. We are not in the point where click, click, click and you're like you have a production-ready app ready to go. The developer is the ultimate expert.

>> They're doing code review.

Atul Deo

>> Exactly.

>> They didn't write the code.

Atul Deo

>> Now, they have not such a interesting thing to do, but-

>> They can literally eyeball. I mean, developers are smart. Okay, so syntax error or whatever, bad reference there. Okay, no problem. Check.

Atul Deo

>> Absolutely.

>> Business logic users .

Atul Deo

>> Right, and also, there is an easy way in quote to actually verify the output whether it's working or not. You can also do test code generation like unit tests and stuff to actually ensure you have coverage, but in the real world, not everything can be verified that way and sometimes based on your company's heuristics and various domain knowledge that you may have. This is where automated reasoning comes in and it's also a thing that we have used in AWS for years for things like IAM, Access Analyzer and so on. I think we are really doubling down on our expertise here to give customers something they can stand behind.

>> I think you guys are really onto something. I think the efficiency piece you were mentioning about distillation points to the era of cost, like prompt caching, intelligent routing, these point to this era of evolution around, "Okay, I don't need to do all the big stuff to get that answer. I can understand the scope of an agent or query or prompt and get it down." I think DeepSeek I saw as a great example of how they were so efficient algorithmically and how they managed around that piece. Everyone was going crazy about DeepSeek but I love what they did about but I think the real innovation is how they got there. Explain that, because I think this is one of those areas where model choice matters because if you have a lot of model choices, you can actually engineer the efficiency side, so the cost is a huge issue.

Atul Deo

>> DeepSeek is remarkable for a number of reasons. It's one of the first thinking models where it can actually go take an output and go back, feed it to itself and so on. One of the interesting things about DeepSeek was the actual training costs itself was significantly lower. There's a lot of debate and on the specifics, but one of the things that they did was unique was actually using reinforcement learning as opposed to reinforcement learning with human feedback, RLHF. In the past, basically you would have humans give the preference data and then if we feed to a reward model and then the reward model would use to do adaptation here as part of during the training process for DeepSeek, basically the model was able to introspect and basically say is it in line with human feedback or not and quickly adapt and that, obviously, significantly helped in the innovation and how fast they could launch.

>> It's a tell sign too, I think, of some of where the optimization will come in and what you optimize for. I want to come back to the original premise that Matt Garman made two years ago here on The Cube. I asked him about could we put out the power law. We were the first ones to put out that power law. We were like what's this power law, a little bit popular model but then it's going to be a long tail and a fat belly of specialty models size and performance and he was adamant, model choice was huge.

Atul Deo

>> Yes.

>> Continue forward today, that was a good call. Why is model choice important? You guys were there with that from day one, that was Bedrock's foundation first principle. Why did you guys go that way and then how did it turn out? What's happening now?

Atul Deo

>> I think that is some history there. I think, so again, I was there on Bedrock since day one, so if you look at some of our successful businesses, think about RDS and an example with Aurora and what we have done with other database engines in RDS. I think we benefit a lot by giving customers the choice. Customers have different use cases. We see incredible business on Aurora and we see incredible business on SQL Server and on Oracle and everything, I think on RDS, and I think the same kind of mental model is very relevant for foundation models where we know we have to build our first party models and we are making progress with Nova quite a bit in the past few months, but we also know at the same time that there is going to be incredible set of new models coming out. Claude obviously is awesome, but we are seeing a rise of DeepSeek excited about whatever is the next version of DeepSeek that comes out. There's LAMA and I think it's hard to predict who's going to be the next big thing here. I think this is a space where things are changing very rapidly and, of course, we would like to know what to be the best, but customers will have different kind of requirements on this curve of accuracy, latency and cost, and we want to give customers all sorts of options. I think in addition to just the generic aspects of it, there's also a specialized domain or industry vertical aspect to large language models and that's why we launched Bedrock Marketplace last year as well and we are going to have plans to make that even more better over next few months, but basically in addition to the-

>> It's like an app store for models pretty much, right?

Atul Deo

>> Yeah, exactly, but in addition to the main kind of models that we have in Bedrock today, you can also get, if customer has a specific need for example, evolutionary scale as a model, which is focused on life sciences and if a customer is in that space and they care about that particular model they have access to and they can use that same model in Bedrock along with the rest of the tooling and go about the use case, they're not blocked. I think that's basically our thing. Even for that's for the long tail doesn't matter because we don't want to leave any customer hanging. We want to make sure their needs are met on AWS

>> Then you have all the data and the bigger models are doing that work. I want to ask you about the breakthroughs. We talked about sales and marketing and coding being the value areas where people first go at because it's natural instinct to go where the value can be extracted and created and extracted. When you look at the two areas that we're looking at with The Cube is there's the new thing that's going to emerge out of the models, a new consumer app, who knows what's going to come next in all these waves. It's a Petri dish right now, a lot of things' going on, but what will happen is a new brand will emerge that's just natural. We predict that. We don't know yet what it is, but call that a net new thing. Then the existing environment that we live in is going to be focused on. If you've got coders and you've got sales and marketing, you're selling something's impacted, that's workflows. There's a real era towards applying agentic to workflows because they're already there. You mentioned healthcare, I mean, they've been tagging data, they're actually been AI ready before AI readiness was a term because they had to because of the regulation and so that's kind of a gift. Thank you very much. IT budget has not always been as big as JP Morgan, but they're kind of in a good spot. You have workloads that are well-scoped with domains, with domain experts, kind of lines up to all, it all rhymes now, but now they have the cost problem. Is there an AI agent to say, "Hey, I know your workload, you're better off running a little bit of DeepSeek, a little bit of that." I can see an era where the intelligence gets down to understanding the infrastructure and then matches the workload and the agents are kind of managing that.

Atul Deo

>> I think that is kind of, we launched this thing called multi-agent collaboration at reInvent last year and it became generally available within the last few six months that we are covering in this interview. One of the things there is you can have different agents powered by different models and when you have this multi-agent collaboration, you basically break down your bigger problem into smaller chunks. Just like in a company, there's an HR person, there is a person that focuses on finance. You can have basically different, and again, each workflow will have different set of complexities and you probably want not just complexities, also specializations. For a specific kind of workflow in the healthcare example that you took, there may be an agent that makes use of evolutionary scales model, other agent needs more thinking reasoning, it makes use of cloud the Sonnet Four and then there's another agent that makes use of Nova Lite because it's kind like a very high volume use case that you want to do in an absolute kind of economic way. You're probably going to see rise of different models even being used in the same workflow more and more often. We already had some of that with RAG where everybody had to use an embeddings model and everybody had to use at least one kind of generative model. That kind of combination existed in the past, but you're going to see increasing combination of multiple models being used by customers even in the same workflow.

>> I was talking to Matt Garman and I was commenting because, obviously, been following Amazon for the past 20 years as a customer and 15 with 16 with The Cube and I've watched the progression of DevOps and DevSecOps. The cloud has been great, been great for everybody and great for the industry, but now we're at a time where you guys, I say Petri dish, it's been really more of a test lab because if you look at what Amazon has, you have DevOps, you have platform engineering, you have all the building going on, you have the models and that's why I love the memo of getting used to the muscle of AI from a workflow stand because that's your building culture. I get that check little commercial for Amazon there, but what it means is that you guys are in the best position to understand the intersection of DevOps and how to build models for value. To create the value and then extract the value. I have to ask, when we look at the intersection, let's say platform engineering, Kubernetes has been a great rallying point around that part of the stack for orchestration. Okay, great, microservice all that stuff, containers, whatever. You guys are great at that. Now, the question is what are startups and companies rallying around the Bedrock and above infrastructure? Is there a solidarity? Is there a de facto? Is it MCPs? What is the rallying point? Because as the startups get in, they're going to be selling to the enterprise, they're going to be creating the next big thing. What is the rallying point or what is the gravity around people to get involved in to get some sort of facto standard or vibe like, "Hey, we can do this and then articulate that to say customers?"

Atul Deo

>> I think there is, we obviously have some building blocks for those things about the model that are going to be required, but the key kind of high-level paradigms that are, I think, of security is going to be a critical part for when customers build these applications and they're agently-powered, like, "How do these agents communicate with each other securely? There is a lot of data that's going to probably go across boundaries. How do you make sure that data is secure?" I think that's number one. Number two is observability. I think you want to understand a particular interaction with the customer happened. What was the workflow and what data went where, and if things failed, why did they fail, and you would want a better version of evaluation where you're not evaluating models but you're evaluating the entire pipeline. I think it's almost like, I mean, applications-

>> It's blood flow. It's like the bloodstream brings travels through the body. I mean, the agents can be running around everywhere doing things.

Atul Deo

>> I think with analogies we've had application performance monitoring and things like Amazon X-Ray, but you need that for the agentic world where you're going to have a complete visibility into a request coming in and the final kind of thing when the application workflow is completed. How do you ensure all that is happening seamlessly in a secure manner and you are not seeing deviations? I think that's going to be the kind of next big thing. I think just having better control on some of this is what customers are going to care most about.

>> How do you integrate the observability? I think that's a great point because observability, when you look at kind of this origin, go back to the old days of Splunk, log files, and then the conversation there was data exhaust is now data gold and that was very valuable for understanding what's going on, where the trouble spots are, how to automate, so early days automation. You take that to agents that is basically monitoring, so whether it's application monitoring or agent monitoring. There's a surveillance kind of active passive kind of environment of what's going on like a doctor. What's the health of the system? Is it accurate? How do you view that and how do that integrate the Bedrock and how should customers be thinking about framing that kind of part of the architecture?

Atul Deo

>> I think, again, we do have some kind of building blocks today you're going to see improvements on that and some to varying degrees, some improvements are going to be small, some improvements are going to be big and you'll see in the next coming months. As I said, Bedrock today already kind of customers can use Bedrock in the context of the VPC. A lot of customers do that because security is important. We already have a model evaluation, we have RAG evaluation. We don't necessarily have good agent evaluation and that's probably an area of growth where you would see the next extra few weeks and months. Obviously, we have integration with CloudWatch CloudTrail. Today, customers can log basically all their interactions with Bedrock models. We have called model invocation logging where every prompt and the response can be logged and customers, this is not accessible to Bedrock. This is in customer's account and they can go and access that. I think it is start of a solution. It is not the solution. Just to be candid. I think the way things are evolving in this space right now, we will have to do a lot more to help customers truly understand going beyond the models and going to their application.

>> I think the application layer is going to be the hottest. We were at a bunch of tech guys getting together kite surfing last weekend in Hawaii and we're talking and debate, not debate, conversation was at the business logic, the application layer. You're enabling that with Bedrock. It's very clear and it's clear from all the presentations. The question is, okay, how do you do an AppSec review? He asked the CIO of JP Morgan Chase at reInvent and The Cube, to Lori Beer, I said, "So, oh, gen AI." She's like, "Look, we know machine learning, we've been doing that fraud detection list goes on and on. Gen AI, they have a high bar for the cyber resilience for security." I go, "How do you deal with?" "Well, it's just another app to us." I mean, a little bit off the cuff, a little bit over the top, but not wrong. If Gen.AI is an app, it's app review, so what do you guys look at in the roadmap for customers to think about, okay, what's the sequence of events, what's the requirements? How do I get stuff into production faster? That's a big trend we're seeing. A lot of sandboxes are overloaded right now with POCs, startups kind of in purgatory like waiting. They're trying to figure out how to integrate in to the enterprise on the enterprise side. There's kind of that going on. What's your view of that and how are you helping unblock? Are you like the TSA pre-clear for enterprise?

Atul Deo

>> You are asking definitely the right questions, I think as I said, we have certain offerings today and we are going to continue to help customers on this front and I think it comes down to helping customers debug because better, I think at the end of the day this is an application and a workflow. There could be many things. It is like how do you chunk your data? You have the right kind of re-ranker, what model do you use? Was the latency good enough? Did your agent synchronize properly? Did you not synchronize properly? Did you regress in performance for various reasons? Now, this is going to be the workflow that is going to be there. I think debugging applications itself has evolved. It's not like a static thing. You're going to have new kind of set of problems in the agentic world that will have to be tackled and I think you're absolutely, I think that's definitely a customer need that is going to get more and more kind of intense over the next year.

>> Then you guys, your mission is to make the customers distract away the complexities, so hopefully we'll see that. The demand for is high obviously. If you look back at Bedrock, congratulations on all the success. I remember when it first came out it's like what's Sagemaker making of all this? Good clarity on the positioning. I like the infrastructure vibe there, the enablements there, the markets evolve and so you just kind of got to roll with the wave you're on. If I'm a customer, how do I figure this out? As look, knowing what are you guys doing deep research into making the native AI capabilities better? Could you share what you're doing on research things that you do and what customers can do? Do they have to have a little new research competency team? We're seeing the role of the chief AI officer, which is not becoming like a political position, becoming more of a CTO who's just got another job responsibility. I talked to one the other day, they're like, "Yeah, it's my fourth job. I want to get out of that as fast as possible," which is counterintuitive because they want to infuse AI into everything and not have to be the chief AI officer.

Atul Deo

>> Well, I think if I think about Bedrock, there are two things. Number one is we want to provide customers the best choice of models. Number two is they should have the best combination of accuracy, latency, and cost for those models on Bedrock, which includes also, all sorts of capabilities that help you manage the combination of accuracy, latency and cost example, prompt caching and intelligent prompt routing. The third one is additional set of tools that you would require things to build agents to build rack pipelines, things like all related, evaluation and then all the observability capabilities that we kind touched upon. You're going to require all those tooling and we want to be comprehensive on those capabilities and we want to be good at those capabilities. That's kind of how I would think about it. In terms of your question on the research arm, I think one of the things that Gen.AI does is that it offloads a lot of the research capabilities to the best labs in the world. You have Anthropic doing cutting edge research, you have SmartFolks and DeepSeek doing things. We are investing in our research and models with the Nova team. I think most companies, and for most customers, they don't have to do cutting edge research anymore. I think it's more about how do you use this models and the tools to do your job effectively? I think ensuring that someone's thinking about the end-to-end apps for this and requires some thinking, but it is definitely not a research job necessarily. Of course, for some companies they may have to do complex fine-tuning, complex distillation, but those I expect to be more of the exceptions than the normal.

>> You guys are taking that on a Bedrock. You're doing a lot of that on behalf of customers.

Atul Deo

>> Right, but I think the important is you mentioned SageMaker. I think there are going to be because SageMaker also allows you to do fine-tuning and distillation. It is just that. It's the differences number of knobs and levers. For someone who's a data scientist, someone who's a machine learning expert who wants absolute control, they want to do complex things and add new neural layers or do something kind of unique on the model front, they can go use SageMaker. Bedrock, the number of knobs and levers is limited. It's more designed for a developer who wants to get to the outcome fast. It is designed for that. Of course, we have good integration between Bedrock and SageMaker with the custom model import. If a customer ever wants to do complex things with the model, they can do all sorts of surgery on the model and then they can basically export that model into Bedrock and then that's available for them in the Bedrock.

>> The SageMaker is where the action is if you want to get under the hood basically.

Atul Deo

>> Exactly, yes.

>> The model on the dashboard is Bedrock. I want to mix and match and interface it at QuickSight and things of that nature, that you can come in and get great value. Great innovations on Q. I want to get your thoughts because I think the real upside to me is what Bedrock and SageMaker do with all the, obviously, integrated and the chips. We didn't even get there, but maybe do that another interview, but that's happening. We know that is that once this is set up, QuickSight comes in Q for developer takes in, this is where the action is.

Atul Deo

>> Yes.

>> Share your thoughts 'cause you have history with Q, and now that tooling is almost going to be seamless under the hood for the developer and the business user.

Atul Deo

>> Yeah, so I think, again, I think to touch upon our strategy on Gen.AI, which is we have three layers of the Stack, the bottom most layer is the infrastructure layer where we have custom chips, we have SageMaker for people who want to build their models and do complex surgery. Middle layer, which is Bedrock, which is the classic developer destination for platform, kind of Gen.AI Platform tools, capabilities, models, and then the top layer, we have applications where customers are ready to go use these directly. They don't have tinker with any of the platform layer capabilities or the infrastructure capabilities. Q Developer, I am passionate about Q Developer because I was one of the founding launch PMs for Q Developer back in the day or back then known as Code Whisperer and I think I'm-

>> Must be a proud papa then.

Atul Deo

>> Well, it is very awesome to see the evolution of Q, because Q Developer because the models back then were not as powerful and compelling as they are. We didn't have the notion of agents kind was vaguely present, but we didn't have models that could reason, and now the level of capabilities that it could do with Q Developer and the kind of experience within Amazon, we see people using Q CLI like it has really taken off in a big way and we see developers using that nonstop now.

>> It's a classic value creation, value extraction, the first two layers, a lot of value creation on the setup, get everything teed up, Q, QuickSight value extraction for the customer.

Atul Deo

>> Yeah. Correct.

>> All right, I got to ask you, so a lot's happened, summarize the first six months if you could. I know it was a lot. Then looking forward, I know we've got the summit in New York, we'll be there with Q all week in our NYSE studio as well as on-site on Wednesday the 14th I believe of July. Then we've got reInvent almost six months away. I'm sure you're getting ready for that too. It feels like reInvent's already happened, again, this feels like halfway points is a reInvent moment. Much has gone on. What's happened in the past six months? How would you articulate the past six months of your life?

Atul Deo

>> I think past six months we've seen a lot of model launches and amazing model launches. Sonnet 3.7 came in, it was one of the best coding models and then now recently it was appended by Sonnet Four and an Opus Four. We are seeing kind of tremendous traction on the coding front. We saw improved models from Nova. We saw Nova Premier, which is our state-of-the-art model. We also had Nova Sonic, which I'm personally Eric excited about because I do like interacting with these models on a speech-to-speech basis. The responsiveness for someone who has a background in speech recognition and ASR, automatic speech, speech-to-text these models just completely bypass this whole speech-to-text part and just get it. It is amazing to interact with these models. That's the part and obviously DeepSeek happened and then we are going to see more of DeepSeek in the next few months LAMA IV happened. I think there's just a lot of innovation on the model front. Late last year, MCP came out and I think it was people who were in the space were wowed, but I think the level of excitement and interest in MCP is obviously interesting.

>> That's a rallying point for sure.

Atul Deo

>> Yeah, and I think you are going to see us leaning more into MCP over the next few weeks and months. We have done already, support, as I said, AWS services. Number of AWS services support MCP have an MCP server and you can interact with them that way. We've also made a number of our capabilities that we launched at reInvent, generally available, prompt caching, model distillation, intelligent prompt routing, and some of the other capabilities that are still in kind of preview will be generally available over the next few months. One of the things, as I said, I'm very excited about automated reasoning. It really changes the game on making sure you can rely on the model. I think that part is going to be awesome in the next few months. I think the last I would say is just the rise of agents, but we launched Bedrock agents in New York Summit of '23 and back then it was-

>> BI basically was the pitch there, right?

Atul Deo

>> '24, I'm losing track of time. New York Summit of 20 ...

>> 24.

Atul Deo

>> 24. Oh actually no, I take it back. It was '23. We launched New York Bedrock agents.

>> Yes, Swami was on stage year.

Atul Deo

>> Correct, exactly. It was like agents was a very foreign topic at that point. The React paper had literally just come out in earlier that year, the reasoning action paper, not the other React. We launched Bedrock agents, but again, the models at that time did not have this incredible reasoning. That's why I keep saying that the tools in the models have to be in lockstep and now I think-

>> They are in lockstep right now. Would you agree?

Atul Deo

>> Yeah, I think they are.

>> Things are moving super-

Atul Deo

>> Way more in lockstep than ever before.

>> Well, I think the activity on the open source on the developer side, say another tell sign lockstep is seeing action. I have to ask you as someone who has a CS degree from over 30 years ago, I'm a founder so I can relate to the younger generation, but the Gen Alpha and the Gen Zs, they take to AI like it's a fish to water. What are you seeing from the young guns coming up? When I was 15 years ago, I was like, "Oh man, EC-II, I'll have to get a data server and then put it in the data center. Oh this is great. Three building blocks, okay, got some primitives." Now, you got a whole nother stack on top of Amazon from services, you still got S-three, still got all that goodness got serverless. You have still all that stuff going down over there. What are the young people gravitating to? What's the usage patterns? Can you share any anecdotal view of as you see Bedrock kind of rising? What is the action for the young talented or old systems thinkers who see this as a dream scenario?

Atul Deo

>> You're asking about the younger position or the-

>> Well both. I mean I've seen some older system folks that have systems background saying, "Oh my God, this is an operating system, I can just code it in a day." I mean, it's kind of that almost magical on the computer science side, so start with the younger generation.

Atul Deo

>> I think on the younger generation part, I think the expectations are just high. I think they expect Gen.AI to be natively integrated, particularly in terms of applications, why my kids are seven and nine and they almost expect, they expect an intelligent response from any of these tools now, it's just like shock.

>> It's your own QA department at home.

Atul Deo

>> I'm like, "Wow," and they're like, "If you don't know, go ask Chat, like ChatGPT," or any of the consumer applications to get a response or perplexity. The responses are pretty down good because they use obviously Gen.AI behind the scenes. Then on the other end of the generation, I think you are seeing productivity boost, particularly people ... I think there was an initial reluctance to fully trust and believe in some of these things, particularly two years back or three years back when we first started seeing this. Quality has improved now where it's now starting to become the norm. I think the level of adoption, I think it's the new norm on-

>> There's classic enthusiasm now confidence because, I mean, it's almost natural be skeptical.

Atul Deo

>> I mean, I personally use Amazon's a very company, very heavy on docs and thinking, and I literally feed my entire strategic doc. I'm presenting to Matt or Andy and I basically say, "Here's my doc, kind of critique it. I'm presenting this to the CEO of Amazon, CEO of AWS, and what do you think? Am I missing? Is this coherent?" Sometimes the more recent example was I had a very strategic doc and I had a follow up doc and I basically pasted the entire-

>> You can train your AI to know how Andy and Matt review docs.

Atul Deo

>> Well, this is all in context learning, so it's not trained, this is the models, but if you pass the right context and you basically, you can ask it to criticize, you can ask it to find thinking flaws and it is like given this context, what else would you expect? It actually has pretty good kind of responses.

>> I mean, I've used Anthropic to check some of my, right, does this hang together?

Atul Deo

>> Oh yeah.

>> Think about, it's really a great recommendation engine, feeds my learning, also my output.

Atul Deo

>> Yeah, also, for brainstorming use it. I think people are not using the full power of it in the way that you should, and I think this model is an incredible tools as well.

>> I was talking to a healthcare worker and she asked me advice on career. She's like, "Should I start coding Python?" I'm like, "No, no. That was three years ago. Get into AI." She says, "What do I do?" I'm like, "What do you know?" "I know all the workflows, I know all the business logic." I go, "Write that down." How should people like that person who have opportunities to be a builder and a needle mover to go in and make effect change because they have all the requisites in place, they just don't have the coding. How does that person connect into Bedrock? How do they engage and how do they get instant gratification that their knowledge could be translated into something that moves the needle?

Atul Deo

>> I think, again, I think this is where the level of abstraction comes into picture. For Bedrock, we wish we could make it that easy for everyone, but that's not our target kind of customer base. We are targeting developers. It's an API-based service. I think for customers like that who want to make use of their skills, just be on a prompting basis, Q Business would be the way to go about it. Q Business, QuickSight, those would be the tools for them where they can create applications. Q Business also allows you to create apps and you can create those apps just based on prompts and go about your work.

>> Yeah, the old days was you get a PowerPoint, get funding, then you do a prototype, put it on AWS, get funding. Now it's like build an app and have a bunch of users on a prototype in a week and now you get funding, so the whole entrepreneurial equation's changed.

Atul Deo

>> I think the statement that you're going to have multiple billion dollar companies powered by a single individual. I think that is really going to, I think, it's a matter of when and not a question of if at this point.

>> It's so great to have you on and, again, congratulations on all the success on Bedrock and following from day one. The clarity's there, love the developer positioning, a lot of needs there. Again, sales and marketing is product app and engineering and software is where the value's created. Value creation market we're in and now let's get that value out.

Atul Deo

>> Thanks, John.

>> Thanks for coming on. Okay. I'm John Furrier here for the AWS Halftime report. Really the most important story is where these models fit into the scheme of creating value and extracting value and also putting together the stack to scale them at large scale. Of course, tokens will be feeding these agents and the agents are coming, they're already here. I'm John Furrier, thanks for watching.