AWS Mid-Year Leadership Summit 2025 | Nandini Ramani, AWS

Clips
News
More from AWS Mid-Year Leadership Summit 2025

Nandini Ramani

VP, Search, Observability & Cloud Ops

AWS

AWS retools for an agentic future with observability and OpenSearch

The first half of 2025 has seen an unprecedented acceleration in artificial intelligence-driven innovation as enterprise technology undergoes a major shift. Trends like AWS AI agents, real-time observability and smarter data architectures are redefining how organizations operate and scale.For Amazon Web Services Inc., the first six half of the year has included a focus on Amazon OpenSearch. OpenSearch is a multifaceted service that functions as a search engine, a vector database and a log analytics platform all in one, according to Nandini Ramani (pictured), vice president of search, observability and cloud ops at AWS

pause_circle_outline Introduction to theCUBE's Halftime Report at AWS headquarters in Seattle.

play_circle_outline Nandini Ramani discusses her portfolio, including OpenSearch and CloudWatch.

play_circle_outline Importance of observability and search in the modern enterprise environment.

play_circle_outline Customers shifting focus to cloud architecture and data usage for AI applications.

play_circle_outline AWS's commitment to meeting customers at various technological stages of development.

play_circle_outline Trend towards using ephemeral workloads and automated processes for efficiency.

play_circle_outline Emphasis on leveraging data for proactive management and security in operations.

play_circle_outline Insights into the future of open-source and its role in cloud innovation.

play_circle_outline Discussion on open-source contributions and collaboration within the development community.

play_circle_outline Anticipation of new tools and agents to further enhance AWS customer capabilities.

Info
Transcript

Nandini Ramani, AWS

Nandini Ramani

VP, Search, Observability & Cloud Ops AWS

In this interview from the AWS Mid-Year Leadership Summit, Nandini Ramani, VP of Search, Observability and Cloud Operability at AWS, joins theCUBE’s John Furrier to unpack how AWS is shaping the future of cloud operations in the age of generative AI. Ramani offers a detailed look at the evolving role of observability, search and data infrastructure as enterprises shift into the agentic era. She shares how services like OpenSearch, CloudWatch and CloudTrail are becoming critical for building intelligent systems that can operate at scale, support autonomous age... Read more

explore Keep Exploring

What event is being covered in the provided text? add

What services are included in the portfolio mentioned? add

What is the significance of enterprise search in the context of data utilization and AI? add

What challenges are enterprise customers facing when considering cloud solutions and system architecture in the context of distributed computing? add

What is the perspective on the transition to utilizing large-scale AWS cloud services for customers? add

What are the challenges and considerations involved in pricing for computing resources and workloads? add

What are the experiences and changes observed in customer behavior and support efficiency in various companies over time? add

What is the stance on open source innovation and partnerships within the organization mentioned? add

What is the role of open source in innovation and collaboration among developers, and how does it relate to governance and intellectual property? add

What are the goals and focus areas for the upcoming re:Invent event? add

bolt Powered by CUBE AI

Nandini Ramani, AWS

search

>> Hello, welcome to theCUBE here in Seattle at AWS' headquarters. I'm John Furrier, host of theCUBE. We are here for our Halftime Report. We're breaking down all the action that's happened. So much has happened in six months. It's almost like a full year happened in a half a year. This is the new AI cycle. We had a great lineup of two days of guests. Nandini Ramani, who's the VP of Search, Observability, and Cloud Operates for AWS, voted one of the most powerful female engineers in the industry. Great to see you. Thanks for coming on theCUBE. I love the AWS bling. Thanks for coming on.

Nandini Ramani

>> Thanks for having me, John.

>> So this is a Halftime Report, mainly because so much has happened since re.Invent. It's almost like re.Invent could happen today and it would be a slew of announcements, slew of activity. But it's only been compressed into a half a year. It's been a wild ride.

Nandini Ramani

>> It's been a lot.

>> What's the biggest thing going on right now for you?

Nandini Ramani

>> So in my portfolio, if you look at what I own, it's OpenSearch for search. And OpenSearch is an interesting multifaceted service, because it's a search engine. It's also a vector database. It's also a log analytics platform. So it's got multifaceted things, each of them very applicable to this agentic world. And then, I have the CloudWatch, which is my flagship product for both Amazon and AWS, for troubleshooting and observability. Also have CloudTrail for governance and things like that. And I have other services as well, but this portfolio of things in an agentic world, I think, are disproportionately important, if you think about it, like everything. And I want to tell you a funny little anecdote. I started my career in hardware, so I'm not a traditional CS engineering. So I think about the GPUs I used to build, and now, we've come full circle with the NVIDIA and like the things we are building with Inferentia and Trainium. So I always feel like, oh my God, I've come full circle back to the passion that I've always had.

>> It's interesting, I went the other way around. I was a software engineer, ended up working for a hardware company that got out of the software business and then, learned hardware. But if you think about that comment you made, because I think it's really ties to the mega trend, which is we are living in a systems thinking world right now. And hardware, back in the day, you had to think about the system, the motherboard, the connections...

Nandini Ramani

>> Yes, exactly....

>> the processors, the circuits, the bus. And if you think about what you're working on at AWS, you're kind of like the data glue layer.

Nandini Ramani

>> Yes.

>> You're the lifeblood. The data is the key to understanding everything, finding things, connecting things, moving things from point A to point B, kind of like a backplane.

Nandini Ramani

>> It is the backplane. And it's also, one is the data gathering. How do you do it efficiently? Do you do it in tiered? How do you deal with structured versus unstructured data? How do you play well, because we have S3 Tables? And how do we back into that neatly? We can go all the way down to the silicon, because AWS has Inferentia Trainium chips that are very suited for GenAI. So anyway, I'm very excited. I've forgotten your question, so let's...

>> No, no, this is good. We're kicking off with a good riff here, because I think that sets the table for the importance of observability, the importance of search. And if you go back a decade or two, search was, oh yeah, search engine, structured data. It's hard in the enterprise. Easier. Google did it. Okay, now, search is a key piece, because getting data, not from an end user, at process might need data.

Nandini Ramani

>> Yep.

>> Enterprise search is the hottest category in enterprise right now, because with AI, you can abstract away all the complexities and find what you're looking for. And then, the observability between systems becomes what was once data exhaust to data value again. So in the world where that data can be used as input and to train...

Nandini Ramani

>> To give you insights and feed insights, it can be used to train agents in this agentic world. So there's so much value in all of these various things.

>> Okay, so customers right now that I talk to on the enterprise side that have cloud or looking at them going, "I don't have a lot of CapEx, I don't have a hundred million dollars laying around, unless I'm JP Morgan." They have a ton of money, but they're like, "Okay, I need to build the new system. I got the cloud, I got some data on-prem, I got an edge." It's essentially distributed computing. Okay, we know what that looks like. They're thinking, "How do I architect? How do I use my data, observability? How do I understand how to find what I'm looking for," for maybe a neural network that they're building with vector embeds? All these things are factoring into the pre-dawn era we're in for agents.

Nandini Ramani

>> Yes.

>> And they're kind of in setup mode. How do you view that? How do you guys look at that? Because you now have the portfolio, you've got the large-scale AWS cloud. Is that like a mind shift for the customer? Is it just a natural extension?

Nandini Ramani

>> It's going to be a natural extension is how we see it. So you got to meet the customer where they are. As much as we are, on the one hand, talking about agents and all the advances, don't forget, we still have monoliths and mainframes and the spectrum is vast. And of course, GenAI can help you in the migration and it can make plans for you. And it can even refactor COBOL into Java. So we have a wide spectrum here, where we have to meet the customer where they are. And in terms of agents, I think we have to see where the puck is going, and the puck is going very fast. To your point, everything we thought about at re:Invent, we went from prompt engineering to vibe coding to now agents in like two quarters. So it's evolving. But on the good news, at our foundation, AWS is, by default, we are secure. By default, it's in our... Operational excellence is in our DNA, we're known for it. And end of the day, the differentiation is going to be, how do you consume all this data? We have vast amounts of it. What is the easiest way? What is the user experience going to be? Do you want agents for every small little task? That's going to be like... I remember when we went from monoliths to microservices. We've shifted too far on one end. So that's the thing with agents to see. It's easy. Now, we have MCP servers, not just to talk to the tools, but from agent to agent. So we got to watch where it's going and be thoughtful about, what is that user experience? How would an operator, human operator, work with this array of agents?

>> The MCP is a good highlight. I think that's one of the hottest areas I see right now around models talking to each other. That's going to be a big component.

Nandini Ramani

>> We are really pushing. I think it's a very valuable...

>> Yeah, it's a huge trend. Folks watching, check out MCP. It's most cutting-edge thing. What that's pointing to, in my opinion, I want to get your reaction, is, on the operational side, there's all kinds of built-in stuff, you just mentioned some legacy, on the build side, where the breakthroughs are happening, the tooling's getting better. We're seeing a rapid advancement in tooling. So you got, on one side, massive innovation surge, open source models, foundation models, computer visions rocking, but the tooling is getting so good that you're starting to see the apps being built faster. Meanwhile, you got the whole platform engineering, DevOps culture going, "Okay, we got Kubernetes, it's stable, we're ready to go." So you have this collision, and it's a good collision, it's an intersection of platform engineering stability to kind of Wild West.

Nandini Ramani

>> It's a little bit of both. So on the one hand, these agents are proliferating, if you will, right? So you need a way to observe the agents. They're complex distributed systems that are going to go all over the place. You need some way to observe those agents. Otherwise, you wake up in the morning and the agent's done a bunch of things. And in fact, I've had customers ask me, "With my developers, we have performance cycles. We know how they're doing." We're going to need things like that for agents. We need proactive detection of vulnerabilities and things like that in code before it goes into production. There's so many layers and aspects to this.

>> All right, let's get into the operations side, because now, the operating teams, whether it's cybersecurity or platform engineering, they're trying to set up a stable environment. They even use the word, "guardrails." Has been overused many times, but I get that. The ball's bouncing around the guardrails around the innovation. And the innovation is I'm using tokens. The tokens are in massive demand for these systems. And the more tokens that are needed, the more large scale I need. So you have a kind of an auto-scaling kind of vibe going on, where it's like, "What do I need?" Now, perfect for cloud, because it's elastic. You can say, "Hey..."

Nandini Ramani

>> You'd use as much as you want in ephemeral.

>> How do you view that? Because there's been a big discussion of price per token. That's more of a benchmark on who's got the better model. New models bring new usage patterns. New usage patterns bring more exhaust. So everything's kind of like in your area right now.

Nandini Ramani

>> Yeah.

>> Unpack that. What is going on?

Nandini Ramani

>> So pricing is a complicated, challenging thing. We are all trying to figure that one out. But I agree with you. The advantage we have is we do have the storage layer, we have the compute layer. We can play around with what is the best experience. And if you think about compute and how it's evolved, we went from buying these boxes that would just sit there to, now, ephemeral workloads. And I feel agents, this is my opinion, of course, nobody has a crystal ball, but we believe agents are going to be those ephemeral things that do a job, job to be done, and disappear, like spinning up Lambda functions or things like that. That's one model of work. The other is you could have a super agent that orchestrates all of these, because the tokens, the models, that's going to evolve at its pace. All models are getting better things. There, it's more about, how do you make it cost effective? Because if every new thing you do is a brand new search, it's going to swallow up a bunch of compute. So we are working on memory and how do we...

>> Systems.

Nandini Ramani

>> Exactly, systems, which we are really good at. So that's where I think we can add value for our customers, because we know our customers, we know how to meet them, and we have the cloud.

>> Yeah. So one of the things I asked Matt Garman when he was in earlier was, you have Amazon Web Services as a huge horizontal scalable platform. We've been covering that for 15 plus years. Now, you have this Petri dish. We're almost kind of in a mode where we don't know what we don't know. The enablement that's going to come from how you guys understand your resource and then, offer that up to customers and partners and their customers, I think this next year will be very telling. I think new things will bloom from this. What's your perspective on that enablement? Because you guys have a unique perspective things like LLM, routing is a topic people talk about. Routing and you talk about memory. Memory, these are model layer memory, so not computer memory.

Nandini Ramani

>> No, it's different memory, yes.

>> HBM and solid state. There's some stuff going down there too. We can...

Nandini Ramani

>> We could go as deep as we want.

>> We don't want to go there. But data's having new characteristics.

Nandini Ramani

>> Yes.

>> It looks like it's a large scale system to support the demand for agents. How do you view all that?

Nandini Ramani

>> Yeah, so I'm sure Matt covered this way better than I can possibly, but I think, while this Petri dish is evolving, we have some core tenants. Like I said, we are secure by design. We are really known for responsible AI. We take very good care with guardrails and give you the tools, because these agents can go rogue as well. And we have all of this data. We can curtail that within the AWS. And one advantage, in fact, I want to bring you to a launch that we did at re:Invent, which is still relevant in an ever-changing world, AWS has over like 17, 19, whatever, since S3 was born. We have data on which we call Correction of Error. We have a whole knowledge base of failure modes for our infrastructure. That is something we know, how it fails. It's our data. We've built a knowledge base. We released an investigations assistant in CloudWatch, and you can see, it has cut down time for so many customers. The nice thing is this is applicable to everybody. You can take Cedar Gate, which is a healthcare company, they have seen their troubleshooting times go down from two hours to 30 minutes. Amazon Music, using vector databases. So what they're doing is they have a hundred million songs of library of songs, and they're using OpenSearch to vectorize all of these, to create vectors and decorate that data. So when I do a search, it knows, oh, I like EDM, so it curates and feeds it. So GenAI is not a one thing, I think. And we have many of the components and ingredients from the ground up, like I said, all the way from the silicon to the infrastructure layer. And next, we are also meeting the customers where they are, because not all customers... Some of them have, like I said, mainframes running on their data center. We want to give them the ability to point at their data and get just as much value as they do when they are in our infrastructure vended side of that.

>> And you guys are also a customer zero for all the technologies. Customers want to do the same exact thing for their data.

Nandini Ramani

>> Exactly.

>> They have all that exhaust too...

Nandini Ramani

>> Exactly....

>> on their workloads.

Nandini Ramani

>> Exactly.

>> And they want agents to do stand-up services, pull them down at will.

Nandini Ramani

>> Exactly. And we want to meet them where they are to do that. And one of the other great things about AWS, which we have always done, we drink our own champagne, if you will. Amazon is a big client, and they use all of our services. And then, we expose it externally. So we continue to do that. So until, we have confidence.

>> Okay, so have you given the memo out to your team that says, "We all have to use AI and build the AI muscle?

Nandini Ramani

>> 100%.

>> What's been the reaction?

Nandini Ramani

>> Yeah, so I will tell you, it's been an interesting journey over the last, what is it, 18 months or whatever it's been since we took that giant leap into GenAI. And the models have become incredibly better these days. So initially, we had skeptics, even our customers weren't quite there, and my engineers thinks it's clunky initially.

>> Change management.

Nandini Ramani

>> But now, they're all over it. There are certain things that they do really well. And so, we are seeing productivity gains across my... And we almost gamify it. So I've challenged my team to see who gets the most done with the smallest and how are they using it. And they do hackathons, John, and it's so fun to watch the creativity that's coming. While it's strange, isn't it?

>> Has any engineer actually automated the PRFAQ process? My question I ask all the leaders.

Nandini Ramani

>> It's pretty close. We are getting there. So even I use it extensively, I use Q, we use our own tools, and it spits out a lot of things, but you have to have an eye for it. Because I can tell when it's fully generated versus overlaid. So that art of taking what it gives you and overlaying, this is true not just in docs and things like that. It's going to be very true in the operational stuff too. It'll tell you a bunch of things, but you still need...

>> The human in the loop is critical.

Nandini Ramani

>> It's critical.

>> Yeah. I love Matt Garman's comment, and he said, "We're going to need ideas." Ideas are going to be the new resource, because a lot of the coding and a lot of the augmentation assistance will come in from AI. And so, love that perspective, and I totally agree with him. And I have to ask you, being an engineer, if you look at today's market, what has been the reaction? Because obviously, the use cases right out of the gate are sales and marketing and engineering, coding. What's been the reaction? I would say the hackathons, obviously. Has any innovation come out of that? How did they respond? And how do the non-engineers respond? Because now, you're seeing people in the business side going, "Hey, I don't need to have a CTO for my startup. I don't need to have an engineer assigned to my product, or maybe not as many. I could code."

Nandini Ramani

>> So you're right. I think we're in a transitionary phase. I firmly believe, yes, we will need fewer people possibly, but in the fullness, once we adapt to this new world, I think we'll need more people. Because we'll be churning out so much more stuff. And ultimately, humans are the ideators and collaborators and all of that and we live in this three-dimensional world till the robots come. So I firmly believe we will just move up stack. If you look at,, historically technological advancements, and this is just one more of them, that's pivoting us, we've adapted to the new thing. Not all jobs went away. We just up-leveled, upskilled. And so, the new type of people we are going to need are not the coders that we used to hire, because that can be done. If you think about what we launched in OpenSearch for natural language querying capability, that world of, do you know SQL or PromQL, that world has disappeared already. We are going to have voice, we're going to have natural. So the type of people we need, that adapt to this, is going to change. At least that's what I believe.

>> A lot more creativity, a lot more problem-solving.

Nandini Ramani

>> Exactly. A lot more problem-solving. And we're in that transition phase where everybody thinks we're going to need fewer, because agents will do. But I think we'll need people on top to create more.

>> You know what's interesting is I'm seeing a bifurcation where back when I went to college, we were called software engineering. They actually had the word, "engineer." They went to coding. So it's all computer science now, but you're seeing some of the, I call them, alpha coders or the top-notch software engineers. They're going lower in the stack to squeeze value, almost old-school, nineties days, like memory management. I want the HPMs...

Nandini Ramani

>> That's going to become so important. That's why I gave you the example of NVIDIA. So when I was doing hardware, we got distributed, I went to software, but the detail it takes to make compute viable and work well for these cases, there's so much innovation to be had in the storage layer. We have Iceberg and Parquet file. There's so much of innovation in other places that...

>> There's engineering to be done.

Nandini Ramani

>> Exactly. There's lots of engineering to be done.

>> And then, at the top of the stack, where the application layer is, that's where the business logic is, that's where we're starting to see still kind of low code, no code, but still coding, voice prompting.

Nandini Ramani

>> There's going to be different kinds of coding. So the traditional way of doing things will be more low level and up stack. But if you look at the evolution, we went from, what, assembly to C to C++, Java, Rust, Go. I just look at this as the next...

>> Okay, so back up. Let's go to the mainstream enterprises, pick middle of the enterprise market, like Main Street USA kind of enterprise, not like the big banks. They have operations, and they're like, "Okay, we've been doing this rack and stack and stuff. We now moved to the cloud." Their IT is working. But now, they're like, "Okay, what do we do now?" But you have discipline in operations process. You have some platform engineering advancements. Those enterprises are going to be disrupted, and they're going to be enabled.

Nandini Ramani

>> They're going to be enabled.

>> How do you see that disruptive enablement happening? What's the key for an ops team? Yeah, we've been doing DevOps, DevSecOps, that's all in play. Now, we have all these ideas coming in from the business side. Who's going to code them? What's...

Nandini Ramani

>> The world is going to converge on all of those above. You are already seeing security observability coming together, because it's all the same data end of the day. And this data is going to be generated in real time now, the amount of logs. And everything's emitting. Everything's an endpoint with data coming. So this disruption for those ops teams, in some ways, it's only going to get more complicated, I think, because now you're going to have to manage all these agents. You have to incident... There's two types of things. One is you have to manage the agents and understand what these agents are doing. The second is the human and agent interactions for troubleshooting. How do you do that as well? So it's going to be a multi-prong thing. Same thing is true for security. How do you proactively do pen testing and vulnerability, all of that, in code? But then, nobody can 100% catch it. There's going to be attack surface as well. So you also need to do it post, but you'll need fewer pen testers. Because you'll...

>> I've heard the term, minus zero days.

Nandini Ramani

>> Minus zero days, exactly.

>> Meaning there's already telemetry coming in where, by scouring the dark web, you can see mobilization things being constituted for attacks.

Nandini Ramani

>> The heat map of it, exactly.

>> And you know, this is a ransomware coming, "Hey," that sends a SOC an alert. That's happening. So you're in the position in AWS where you're sitting in a really key area. You have not only multiple pucks coming at you. You got to skate to like three pucks.

Nandini Ramani

>> Exactly. Because it could come from anywhere. And that's something we do well. Knock wood, never want to say that, but you know what I mean. So those are some of the advantages, I think, and it'll enable a lot of these companies, not just SMBs, even the large ones, to move faster to this... It's almost like they're still stuck in... Many of them are struggling with that. This is going to force and give them momentum to move into the...

>> The other trend I want to get your reaction to, whether this is true or not, from researching it, but on the surface, it seems like it's working. The bigger getting bigger, because they have a lot of data. So you're starting to see leverage from the operations that they're getting. The smaller companies are getting powerful, because they can punch above their pay grade with cloud. The mid-range seems the ones stuck in the middle between, do I go big? Or do I go small? It's almost like... Do you agree with that, generally, that general oversimplification? Because AI gives more superpowers to the people who are agile and who lean into the new technologies. And the big players have a lot of data.

Nandini Ramani

>> And they have a lot of data, they also have more resources. And...

>> And the mid people......

Nandini Ramani

>> it's harder on them to...

>> You agree?...

Nandini Ramani

>> pivot on a dime. It is true, which is why we have to enable. Cloud still helps them, the sooner they can come to it. Startups have this unique ability to just kickstart and go. I remember not having to... None of that. You're just liberated to go build what you want.

>> You are the IT program.

Nandini Ramani

>> Yeah, you are everything.

>> DevOps is IT.

Nandini Ramani

>> But I do think we are really on a journey to help SMBs. We have a startup ecosystem that we help our customers. And so, Amazon does have many programs, because it is harder. Because they have limited funds. It's harder, they're bigger than the nimble startups, but not as big as the giants. But several of them are going to become the next big thing, so you want to foster that work with them as well.

>> Would you say that you're in the data business, data ops business?

Nandini Ramani

>> Definitely in the data ops business.

>> All right, so how does GenAI impact data operations? Is it looking like an SRE type model? Data engineering? Is it going to be abstracted away? Because if you don't have the data telemetry, the data observability, you might not know what's going on. That could create hallucinations in the model. So not knowing everything, you got to know everything.

Nandini Ramani

>> You got to know everything, which is why I said we not only do everything for the AWS ecosystems, which is the first step. We have to do that for AWS. In terms of your question about SRE versus DevOps persona versus SOC, and there's so many of these different... I think they're all going to start collapsing and converging. It's not clear whether the industry is going to create agents that are dedicated, ephemeral, come do that one thing and disappear. Are we going to hire an agent that can do anything, it can be an SRE and a engineer? And so, I think we'll have to see how this evolves, to be honest. Yeah.

>> I was talking to Colleen, and we talk about the notion of digital workforce, digital worker.

Nandini Ramani

>> Yeah.

>> How does the roles change in ops and IT and cloud, where the jobs are changing... Again, they're moving to different jobs. It's not like they're going away. Some stuff goes away, that's redundant, mundane, undifferentiated heavy lifting, but there's new opportunities. What does the role, what does the workplace look like for operations? Are they wrangling data? Are they just managing agents, managing SLAs?

Nandini Ramani

>> Many of them are going to be managing... We are never getting away from managing to SLAs. I firmly believe that. Even in a real time world of agents doing things, we still have to be like SREs nine, nine, whatever... You need the four nines, whatever. You have to have SLAs on commits to custom. That's not going away. However, I do believe, in this new world, how do you manage agent identities? How do they differ from human identities? The HR workforce is a real thing. Some customer was asking me, "How do I know how many vulnerabilities were created by this agent? How do I know what good looks like and bad looks like?" It's going to adapt and learn. And so, we are going to have to support them where they are, because there's one aspect is we are very infrastructure bound, right? So we have resources and tags and accounts and roles and all of that, but end of the day, an agent's going to be given permission similar to a human. Now, the logs on.

>> A little delegation...

Nandini Ramani

>> It'll delegate and you have to give it some trust, and then, you need to be able to provide feedback for it. And so, there's a whole other thing that we're working on to see, how do we manage these agents? Because you need...

>> It's like putting your kid through school. You got to take them through a progression.

Nandini Ramani

>> Yes, you do have to.

>> The thing about ops that's interesting is that the efficiency and the productivity has been the big top line story. Where's the productivity gains coming? And what do you see customers doing? Where are they? You're meeting them where they are. Where's the productivity gains? Where are the people starting? And where are the customers that you talk to? What's going on in their mind? Take me through the day in life of a typical customer.

Nandini Ramani

>> So, because these are all still evolving, the customers who, last year, were saying, "Yeah, I don't know. We're waiting to watch," everybody's on it now. That's why I said Cedar Gate is one in the healthcare space. SmugMug, which is a very different one, they're also using all these troubleshooting capabilities and agentic reduce their troubleshooting by 50%, down to 20 minutes or something. So every second counts in business. The times you're not spending when an alarm goes off, et cetera, you can be innovating for your customers. So we are seeing this in a wide... And like I said, we start with us. Amazon Music is like having an on-call assistant. Kindle has reduced... So Kindle has a big support team. They've reduced their troubleshooting costs by 65 to 80%. They were one of the early on-boarders. So across the board, depending on how much adoption it has, the gains come from it. Coding is going to get much easier as... It reminds me of, dating myself, but when IDEs came and code completion, we were like, "Oh, it's magic." Now, we're in a whole other world, where you can feed it a documentation and it spits out everything you need. But it's also like code is still code, no matter who writes it. And somebody was telling me it's fun and games when you do the tip of the iceberg, there's a whole process. We have a whole ORR process, which is like operational readiness. You have to stand behind what you ship. And so, I still think there's...

>> The resilience bars are high still.

Nandini Ramani

>> Yeah, exactly.

>> You still got to meet the hurdle. Yeah, I wrote a game over the weekend. Great, it's not going to production.

Nandini Ramani

>> It's not going to production. That step from going from something cool to production, there are so many layers to it, which is where I think my teams can help.

>> Yeah, you guys really are again......

Nandini Ramani

>> at the heart of it, yeah.

>> I think you're the lifeblood, you're the bloodstream, because data's flowing everywhere. Production is a big conversation that comes up a lot. What's your experience with customers that have all these POCs, all these experimentations going on, what's the process in production? Seems to be a backlog. I was joking with the customer like, "You need TSA pre-clear to get stuff into production." Is there a path? Are we waiting... They're all waiting, "Can I get my turn"?

Nandini Ramani

>> Yeah. And the other thing is, with hallucinate, we use automated reasoning for hallucination, so we can check it before it hits, et cetera. But I think we are seeing more in production, but the most we are seeing is actually still in training, leveraging it for developer productivity. Those are the use cases mostly. But there is a backlog. And when the floodgates open, we're going to see an uptick in production.

>> I want to get your thoughts on this, because now, let's get back to some of the nerd stuff on the infrastructure. Because I think what we're seeing is is that, as Seattle Seahawks, used a football metaphor, it's like in the red zone, where the ball is close to the end zone for unleashing the agent way. Because the infrastructure is getting smarter faster. And so, you're seeing models tuned for the hardware and the chips.

Nandini Ramani

>> Yeah.

>> So you're starting to see workloads pick certain configurations for the workload. So you got not only the hardware or the configurations, the hardware and configuration and the workload is three things.

Nandini Ramani

>> Yep.

>> Those can be mixed and matched, not one purpose fits them all. That's going to drive a big push into performance. So now, you've got performance. So how has the Amazon investment in the CapEx, in Trainium, Inferentia... You see Anthropic tuning their stuff beautifully into the chips that you guys have. You got NVIDIA out there, you got a multitude of stuff. What have you seen in your group that's been a benefit of those performance gains?

Nandini Ramani

>> So massive gains two-fold. One is the models evolving on their own. Second is, when we get the next generation of the chips, because we are also in line, by the way, we are customer obsessed. So customers get first pick and then, my teams. And so, it's incredible to watch one generation to another one. Even in CPU, like Graviton, the evolution that we've seen, every time, it's like a nice big bump just from doing nothing. Having said that though, the complexity is still... This is why Bedrock supports every model there is, because we believe in customer choice. And there is no one model to rule them all. We do firmly believe that. You have to fine-tune it, you got to pick the use case and build one for that. And then, like you said, we partner with NVIDIA and their latest, greatest chips. We're always optimizing on all of them. And in addition, Dave Brown has a whole team dedicated to make all of these better.

>> You have CloudWatch. Again, these are mission critical, important features. As the performance comes in, what are you seeing internally that you're doing with AI for the products that the customers can leverage? Is there anything that's come out in the past six months that you see and maybe what's coming at the re:Invent that you're focused on, that you're doing, tapping into the research and knowledge of AWS? Yeah, you get the lobbying for the best chips and everything, okay, but there's a trend towards internal native AI usage, for product advancements. Can you share some thoughts on what you're working on there?

Nandini Ramani

>> The product itself, this is the one thing I was saying about, if you look at investigations in CloudWatch, the ability to understand failure modes and the topology of which services calling which in the AWS infrastructure, we have a whole knowledge base. So we can provide that into the first party experience and our native tool, not just in CloudWatch, but also, in OpenSearch. And we're leveraging our expertise to do that. That is thing one. Second, big advancements, vector database, OpenSearch is the vector database for Bedrock, the opinionated one. Obviously, we support all the others as well. So we are making a lot of investments in the KNN algorithms. A lot of it is in open source, so many companies are joining us on this journey. We also work very closely with the S3 team on their S3 Tables, and they have Iceberg support. We're also working with the SageMaker team, which is also they have a lake house. So you talk about us being a data store, we feed that to the aspect of you can do your machine learning and your notebooks and everything in one place, bringing Zero-ETL, basically. Just meet the customer where they are. So these are all things we are ourselves doing and...

>> Yeah, connective tissue between the services.

Nandini Ramani

>> Yeah. The other thing is we have systems manager with runbooks. We are embedding all of those, so that we can provide runbooks. And you can debate, in an agentic world, what will the runbooks do, but it will still use the runbooks in a different way.

>> Yeah, I could see a scenario where GenAI would benefit a customer that says, "Hey, I want the GenAI to pick the infrastructure configuration based...

Nandini Ramani

>> That's best suited....

>> upon for the workload at any given context.

Nandini Ramani

>> Which already, in fact, in Bedrock, we already have that. We support a feature for that routing. You can tell it, "Here are the choices, this is what I want. There are the constraints. And if you go pick..." Cost is the biggest one.

>> Remember the old policy days on routing, cost, routing, best performance? We're kind of seeing that at a large scale for workloads.

Nandini Ramani

>> Yeah. And currently, it's still expensive, but I think we've seen the thing come down significantly already. And we're just at the beginning of this journey.

>> All right, what's the most exciting thing you're working on right now?

Nandini Ramani

>> Oh my God, like...

>> Pick a favorite child or two.

Nandini Ramani

>> So one thing I will leave you with, right, we are on a journey. We have so many services. We're trying to, not trying, we are heavily working on building solutions, bringing things together, in the spirit of we now run OpenSearch analytics on top of CloudWatch. We support natural language across the board. We don't want to confuse customers. We want to make it as simple as possible, so they land in one place and they can see everything they want. So I'm super excited about that. Very excited for this agentic journey. We're using MCP servers to... And tooling is going to be supremely important, because we are moving from an API land to tools. The model will only be as good as the tools it can invoke.

>> I saw the news about Strands that came out...

Nandini Ramani

>> Yes, Strands came out in May.

>> That came out of some scaffolding that was being built for something else.

Nandini Ramani

>> And then, we open sourced it in May.

>> Huge.

Nandini Ramani

>> And by the way, we are heavy into standards. I mentioned Iceberg and Parquet files. We are also supporting OpenTelemetry, because it's a great way, it democratizes. Otherwise, everybody's in their wall garden of my formats and my schema. So Strands supports OTel, and it'll spit out distributed tracing and everything. So your audience should try that out as well. And more to come. Watch this space.

>> All right, so now, I have to ask the open source question. Because one of the most exciting areas right now is the, I call, the AI native developers. There are folks writing code, startups coming, the large enterprises are cloud native already. They're becoming more AI native. They're looking at how to integrate into the cloud faster and having all those great models to work from. The innovation and the starvation that's going on in open source, they're hungry for more stuff. And yeah, there's some good coding in there. Go to Hugging Face, play with some models, all the Linux Foundation, Apache, all got great stuff, that's going to bring in a whole nother wave of builders. What do you offer that community, because that's going to be a bridge for commercialization? What's your vision on that open source transition? Because that's coming fast.

Nandini Ramani

>> It's coming fast, and we are very supportive. Look at what I run, right? OpenSearch is entirely... In fact, we gave it to the Linux Foundation. And then, my background is from Java, which I open sourced and ran it in open source. And so, I am a firm believer that innovation starts there and we should foster a culture of it. So we are working, obviously, that's true for all of AWS, we're working with Hugging Face. We support DeepSeek as soon as it was launched. And so, we take a very open approach to things. And in my team in particular, we want to partner with everybody. They're all coming now to Linux Foundation, helping us innovate in open source with...

>> You brought up Sun. A little triggering moment, opens part of my brain. I think John Gage was the one who said, "Network is the computer." And then, Scott McNealy picked up on it. I love that phrase. A little early, then cloud happened. But networking is hot right now. If you look at the network advancements among the systems themselves and also at the edge, you're seeing a huge network fabric emerge. How does that factor into your group? Because now, data is coming in across boundaries in the network...

Nandini Ramani

>> Yes, in the network....

>> for the network.

Nandini Ramani

>> So we do, in CloudWatch, we launched network monitoring. And if you look at troubleshooting, I talked a lot about the AWS infrastructure and then, the applications that our customers build. But if you think about it, there's a whole internet layer. So when something goes down, you don't know where the problem is. It is in your firewall. Is it outside? So troubleshooting is going to extend far beyond... The edge is now...

>> There's no edge. It's the world.

Nandini Ramani

>> There's just the world.

>> It's really competing everywhere.

Nandini Ramani

>> So there will be a whole lot more. And we have teams within AWS doing parameter protection. We have teams... There's lots of innovation happening.

>> And I have to ask you, on a personal note, you mentioned, at the beginning, you had an experience in systems before software. There's a big trend going on right now around systems thinking. Every wave has an agile thinking, creative thinking. There's a real systems thinking mindset right now for the people putting all these systems together, because we are living in a system world. There are consequences, 1, 2, 3 orders of consequences to decisions. It's not just, "Oh, the code looks good, ship it." Whoa, it broke five other things.

Nandini Ramani

>> Exactly.

>> So there's a, even in the younger generation, a huge mindset. What would be your advice for folks out there who are trying to disrupt in a positive way innovation? How should they think like a systems person?

Nandini Ramani

>> Yeah, systems thinking is really hard. You have to throw yourself in it and learn, or at least that's how I learned. Meaning I don't think you can teach it in school necessarily. You can pick it up watching... Nowadays, you can watch YouTube videos or whatever to learn it. But this is what's going to separate the winners, if you will, is the system thinking. Like I mentioned earlier, do you have ephemeral little agents? How do you put these agents together? That's going to become our jobs. And the ones who are successful, we'll figure out a way to orchestrate this, because think about the complexity, right? Different models are good for different use cases. Everything is multimodal. And the cool stuff is only the tip of the iceberg. There's a whole, how do you reach SLAs? So the humans that can actually think through this in a better fashion and not just rushing into, "Let me keep launching stuff," I think, will do well. So we take a more... That's the approach we take.

>> Of course, the put new stuff in the networks, so throw them up, all kinds of problems. Great mindset. Since you broke up the iceberg, I think of the shark fin, the iceberg, we used to put those on slides, like cost of ownership. Cost of ownership now is calculated differently. How do you value a success of a project? This has come up a lot in a lot of the business conversations we have is whether you're a CFO approving a CapEx or OPEX expenditure for more GPUs or a business person saying, "Hey, I want to implement this end-to-end project that's complex, but automatable, it's gettable. We can do it. We can attain the goal," how do you look at the ROI now? The old days was PCO calculators. How much do we pay? What's the payback? Now, you're in a world now where there's all this new data. You don't need to apply mechanisms in pre-old forecasts of what we think it might be. You can actually get real data. How do you look at this whole PCO valuation?

Nandini Ramani

>> So this is a complicated one. There's so much data. There are, in fact, startups in San Francisco that do exactly this. They can tell you, that agent can tell you, the odds of success for each of these companies and where you should invest, et cetera, which is what we used to hire a bunch of people to go data mine and do things. So I think...

>> The cost optimization, like the do ITs, they're turning into much more financial engineering.

Nandini Ramani

>> Exactly.

>> And firms.

Nandini Ramani

>> And so, valuations are going to be very different coming. You hear people making claims that a one person team will be a billion dollar company, but you need people to buy it though. So we are going to see the... I don't have an easy answer for you on...

>> All right, so open source, you mentioned Linux Foundation, open source, love the work you do. We cover Kube constants at the beginning. We'll be at OSS as well covering it. Love that community. Where's that go? Because you're seeing that open source evolve. The contours around HashiCorp, others. I don't think it's a bad thing. Personally, I think it's a good thing. I think still the game is still the same. Are there any insights that you see around how open source can continue to mature? Because obviously, it's a steady state now. It's the key area for innovation for developers and bringing teams together, commercialization paths, the future of open source. What's your vision on that?

Nandini Ramani

>> It's an interesting thought. I think it also ties into governance, because it's IP you're putting out there. And everything you put out there, something's going to start learning from it. It's no longer... So I think this is a complex question about how open source evolves. Personally, I'm with you. I'm a big advocate and proponent for open source, and I think we should all collaborate together. There's so much goodness that comes out of it, because you get to see different types of use cases. We've seen the momentum behind moving this to Linux Foundation and the innovation that's coming from it. But I read...

>> You mentioned Java, because you were involved in Java. If you think about Java, why that was built was built for a whole nother use case, it became a standard and it's got a renaissance. Because there's a lot of great chatter going on. People love Java.

Nandini Ramani

>> And I can tell you, John, when I was running it, we stagnated for a little bit when we got to Oracle, and I think it was JDK8 or something, we got stuck. Then we picked up the velocity, and I remember people telling me, "This is like COBOL." No, look at it now. Sure that we have Rust and Go and other stuff. We still have Erlang for all that.

>> Agents can learn Java.

Nandini Ramani

>> They can learn Java, they can transcode, compile from COBOL to Java. These things already exist.

>> I think Swami told me about a year ago, two years ago maybe, that you guys rewrote, upgraded the Java staff...

Nandini Ramani

>> Oh my God, it was...

>> Talk about that.

Nandini Ramani

>> Yeah, the Log4j, when it came, I've forgotten the stats of exactly how much.

>> It's pretty impressive.

Nandini Ramani

>> It's very impressive. My teams benefited from that. We were just automating everything. And this was before models got to where they are today, and so, the benefit is just unbelievable.

>> I bring that up, because we kind of riffed it down that road. But that is a highlight of the operational dynamics going on. You don't know what's going to happen next. There could be a threat. It could be another zero day. Something could happen. You got the ability to...

Nandini Ramani

>> The ability to mitigate it as quick......

>> rewrite code, upgrade code basis.

Nandini Ramani

>> Yes.

>> That's what's happening now.

Nandini Ramani

>> That is one of the classic use cases. It lend itself really well to automation from GenAI.

>> Okay, great conversation. Since it's halftime in the year, Halftime Report on theCUBE for AWS, look back six months since the last re:Invent and now, look forward. What's happened in the past six months in your world? And what's your focus going into re:Invent?

Nandini Ramani

>> Yeah, so in the past six months, like I said, we've launched a lot of these capabilities that are already helping customers. And we are loving everything that's going on. The next phase, I think, is going to be, how do we make all this usable? It's all the things you and I discussed. In six months, next time I see you at re:Invent, we're going to have so many more agents. We are going to have so many more tools. We're going to be able to provide not just AWS investigations, but go beyond that, to meet the customer at whatever lake they are using, et cetera. So we are working on a bunch of these things, as I speak, and so, super exciting times is what I would say.

>> So two words I hear a lot these days. One is unlock, okay, and automate the agents. What's your reaction to the unlocking term? Because it's unlocking value. Where do you see the unlock coming, going into the benefits of...

Nandini Ramani

>> I think it'll be both. Both will be true. It unlocks a whole lot of value in new use cases every day. Like we mentioned, the Log4j, that Swami mentioned to you, but we're seeing further use cases now. You can just sit there and say, "Show me." There's so many benefits to it. You can listen to it in a car drive, all of that. So you'll unlock a whole lot of potential for security use cases, observability use cases, et cetera. But at the same time, the velocity is going to be incredible.

>> Well, you're in the velocity stream. You got all the data. Congratulations on all the work. Thanks for the update.

Nandini Ramani

>> You're welcome. Thanks for having me on.

>> Thanks for coming on theCUBE. It's the halftime report. The data is flowing. If you don't have access to the data, know what's going on, then you can't manage the agents, you can't manage the infrastructure, you can't manage the applications. That's the takeaway. I'm John Furrier, host of theCUBE. Thanks for watching.