AWS Mid-Year Leadership Summit 2025 | Prasad Kalyanaraman, AWS

Clips
News
More from AWS Mid-Year Leadership Summit 2025

Prasad Kalyanaraman

VP, AWS Infrastructure Services

AWS

AWS doubles down on AI-first infrastructure to meet soaring global demand

Artificial intelligence is no longer just a use case layered onto cloud platforms — it’s the driving force behind a global shift to AI-first infrastructure, and Amazon Web Services Inc. is right in the middle of it. As businesses scramble to deploy generative models and agentic systems, AWS is retooling its stack to meet massive new demands.Just a decade ago, cloud adoption was still a cautious conversation. Now, the conversation has flipped — and the pressure is on. AWS must deliver not just scale, but performance, resilience and data architecture fit for a world where AI is baked into every business process

AWS DC Summit 2025: The cloud becomes strategic national infrastructure for the AI era

At this year’s AWS DC Summit, one thing was clear: AI, cloud infrastructure and public-private partnerships are no longer future ambitions — they are national priorities happening now. TheCUBE Research’s Scott Hebner and I had a chance to interview the top leaders and ecosystem partners to extract the signal. The game is about capital spending scale and generative AI transformation. With $30 billion in new data center investments across Pennsylvania and North Carolina — and billions more nationwide — Amazon Web Services Inc. is not just building cloud regions. It is constructing the digital backbone of an AI-driven, resilient U.S. economy

play_circle_outline Transition from explaining cloud benefits to meeting increasing customer demand

play_circle_outline 15 Years of AWS: Global Scale, Foundational Services, and Customer-Centric Growth and Challenges

play_circle_outline Evolution of infrastructure and AI capabilities

play_circle_outline Seminal moments in Amazon's history, including Nitro and Annapurna

play_circle_outline Importance of choice and operational excellence in customer trust

play_circle_outline Use of gen AI for optimizing rack placement in data centers

play_circle_outline Control and optimization at chip, network, and service layers

play_circle_outline Education and awareness of sustainability in geopolitics

play_circle_outline Customer focus on external, internal, and community stakeholders

play_circle_outline Future potential of AI-driven automation and assistance

Info
Transcript

Prasad Kalyanaraman, AWS

Prasad Kalyanaraman

VP, AWS Infrastructure Services AWS

In this exclusive AWS Mid-Year Leadership Summit interview, Prasad Kalyanaraman, VP of AWS Infrastructure Services, joins theCUBE’s John Furrier to unpack the unprecedented scale and complexity of today’s cloud infrastructure demands. Kalyanaraman outlines how AWS is rethinking data center design to keep pace with the explosion in generative AI workloads, revealing how innovations in chips, optics, cooling, and software-defined networks are reshaping what he calls “compute factories.”

The conversation traces AWS’s evolution from its early days to its... Read more

explore Keep Exploring

What is the comparison of the complexity and scope of challenges in meeting demand for data centers from 15 years ago to the present, particularly in relation to AWS and its growth? add

What has been the focus of AWS throughout its journey and how has that influenced its success over the years? add

What opportunities does generative AI present in the context of cloud computing? add

What are some seminal moments or foundational decisions made by the company regarding their investments and services? add

What are some seminal moments or key focuses for the company mentioned in the text? add

What challenges are involved in optimizing the placement of racks inside a data center? add

What are the advantages of using intelligent networks and software in improving performance and managing trade-offs in technology? add

What are the benefits and challenges associated with partnerships in renewable energy projects, particularly in relation to Amazon's influence on energy resources? add

What is the significance of considering various groups as customers beyond just external clients? add

What is the timeline for having an agent manage these tasks? add

bolt Powered by CUBE AI

Prasad Kalyanaraman, AWS

search

>> Hello, I'm John Furrier with theCUBE. We are here at Seattle at AWS's headquarters for a halftime report, we're midpoint in the year, we got six months to re:Invent, but so much has happened in the first six months that we want to have an action-packed set of executives and leaders come in from AWS and the ecosystem to talk about all the change. Prasad's here, he's the Vice President of AWS Infrastructure Services. He is responsible for the build out and the pressure to get the demand filled for all the compute, all the energy, the data centers, and all the news around the demand and the build out for more and more capability to power AI and all the services that are coming. Thanks for coming on and great to see you. Thanks for taking time out of your busy day.

Prasad Kalyanaraman

>> Thanks, John.

>> If you're not in here-

Prasad Kalyanaraman

>> Nice to see you....

>> you're out building data centers for AWS. I love hearing the earnings calls both on Amazon and AWS, the CapEx numbers, everyone talks about that. There is so much demand for power and computing power for data centers to power all the AI, and just the past years, I've never seen it this frothy in terms of demand, but there's actually real money being spent. You're in charge of all that. From the beginning, you've been around for a long time with AWS. Just put it in perspective, go back 15 years. Then and now, what's the level of complexity and scope of the challenges to deliver from a demand standpoint and from a deployment standpoint?

Prasad Kalyanaraman

>> Yeah, well, the first thing I'd say is that every single era throughout our journey at AWS has had its own challenges. But one thing that we've been relentless about is that we've been relentlessly focused on our customers, and that has actually paid out very well for us. If you asked me the question 15 years back, we were some of the pioneers on cloud computing and at that time we were just fortunate that many other providers weren't there, but we knew that this was a big business for us. And at that time, we actually said that we were building a multi-billion dollar business for us. It's true today. I'm sure there were lots of people who were doubting that, but there were other challenges where at that time we always believed that we were building it at global scale and internet scale. We started building our AWS regions, and we had very clear concepts about how we think about regions with clear concepts about how we think about foundational services. If you look at all our services across compute and across databases, across networking, across storage and so on, we're very clear that we had to build those building blocks, and we knew that we needed to give customers choice. That's the reason why you have all these instances and all our services. We have over 250 services right now, and the challenges those days were about how do we explain to customers why this was better from the way they build their applications? The conversations were like, "Why cloud?" And that's no longer the case right now, but we had to actually explain to customers and we were hearing from customers that they wanted immediacy on computer infrastructure, but at the same time we were much smaller than what we were right now and still growing at a fast clip, but we were smaller. Fast-forward that today, I think gen AI has created yet another opportunity for us where I think we have the opportunity to build the equivalent of another AWS or probably even bigger. Every single service, and every single foundational service is getting reinvented in terms of how they actually use gen AI. Obviously, the scale and power requirements and the network requirements and the computer requirements from this are exponentially larger as well. I would say that if I had to contrast 15, 20 years back versus right now, no one's actually questioning the value of cloud computing and clearly, everyone's recognizing how we've had a leadership position in that space for a long period of time. Everyone's looking at us to actually help with their compute needs and their gen AI needs as well. We're at the same inflection point where we're starting to see a pretty significant increase in the amount of customer demand for us and we're trying to actually keep up with that demand.

>> You got capacity growth, you've got sustainability challenges, you have community engagement because you're putting data centers in these communities. This is obviously a global footprint. Very, very hard to do. I want to get into that a little bit later, but I want to just as context, the old AWS, one of the things I've noticed was is that you got better the efficiencies of, "Hey, we got a bunch of servers." We had customer zero, Amazon retail, you have a lot of internal customers. We heard from Matt and all the rest here, and the update is that there's a lot of internal use of AI funding things. Now you've got the pressure from the customers. Where is the innovation? Because energy-bounded problems like this change the problem statement. So it's not just data centers, not just get a building, no. So what is the problem statement that you're solving? Because it is energy bounded, get sustainability issues, some regions, some grids are better than others. You got to figure this out. How do you scope that problem statement? When you look at, okay, we got to get another data center, we have a new region. Why North Carolina, you had news yesterday, $10 billion investment, you have all kinds of new regions popping up all around the world. What's the problem statement?

Prasad Kalyanaraman

>> Well, at the highest level, the problem statement is very simple, which is how do we make sure that our customers are satisfied and they have a voracious appetite and they're going to be pushing us in new dimensions? And so we have a huge opportunity, our problem statement is making sure that our customers are extremely happy with what we actually provide. That starts off all the way from security, the best set of performance, the best set of capabilities, as well as making sure that the cloud needs to be seemingly infinite. That is what it needs to actually provide. At a high level, that's a problem statement. Now, in the details and the specific, this actually translates to many, many things. I can't actually point at one thing and say, "Oh, power's the only problem that we actually have right now." It's not the case. Power's one of the things.

>> Supply chain.

Prasad Kalyanaraman

>> Supply chain. How do you actually get the right set of chip supply chain, the server supply chain, the network supply chain to be ready for it? How do you do it securely? How do you build the right foundational services for customers? Very early on, our approach to gen AI was we always thought about as three layers, which is we said it's the underlying infrastructure layer, which we know that some customers like Anthropic and others and internally as well as our Nova models will require that underlying infrastructure to build a lot of training clusters. Then we knew that there was a ML infrastructure where we had to build with SageMaker and Bedrock and so on. Similar to what we started off with, we were misunderstood before where we built Bedrock as this playground for multiple models to be there. And then when you saw models like DeepSeek come along, everyone was panicking, but how do we support DeepSeek? And for us, it was a week of worth of work or something and we could actually enable DeepSeek very quickly on Bedrock. Our bets turned out to be right because the fundamental thing we focused on is we focused on what customers wanted and customers wanted choice, and there was never going to be a single model that rules everything that customers want to do so we needed to give that choice. And so today, we support a vast majority of models on Bedrock and so for us to be able to enable the next model was not rocket science.

>> Well, you had the foundational enablement, you didn't have to change a lot in the stack.

Prasad Kalyanaraman

>> That's right.

>> To fix that. Bedrock and SageMaker, SageMaker under the hood to turn more knobs, Bedrock nice for developers, okay. Embedded in, adjust because it's decoupled, if you will.

Prasad Kalyanaraman

>> That's right.

>> From the system, and you're building those systems. I mean, this is where I find it fascinating because you're essentially doing system design in a real estate transaction.

Prasad Kalyanaraman

>> Yes.

>> You have to figure out, okay, where do I put this thing? Don't say thing, but data center, large scale, and it's essentially a computer factory.

Prasad Kalyanaraman

>> Yes.

>> You are building a mega center, Amazon read cloud.

Prasad Kalyanaraman

>> Yes. That is what it is.

>> It's not like, "Okay, roll in and bring the racks in." It's all that plus all the engineering, all the foundational services. Okay, so how long does that take? Does Matt and Andy walk and he goes, "Hey, let's just launch a data center in North Carolina." "Yeah, that's great. I got to find a spot near to the beach." No, it's not that easy.

Prasad Kalyanaraman

>> It is not that easy.

>> Take us through.

Prasad Kalyanaraman

>> As you called out, we call it internally as a compute factory. And if you think about a factory, you have to actually, we have an advantage that we own every portion of the factory, all the way from the silicon to the supply chain associated with it, to the networks, to the manufacturing, to the server supply chain into our data centers, to the network that connects these servers into an UltraCluster network to all the higher level services. And this is something that is unique about us, which is we own top to bottom all the way from the lowest infrastructure layer to the services layer. You ask how long does it actually take? It is not about just the length of time to go and build up a data center, it's also the length of time it takes to build all the other services around it, the network around it, the database services, the storage needed for it and so on. And that is, if you had to start from scratch right now, it'll take anyone years. If you actually invested the time on this and built that experience that we've done over many years, then it's not, then we can actually turn those up fairly quickly. You've seen just the amount of capabilities we've turned on just in the last, since even re:Invent, we've talked about Project Rainier where we're working with Anthropic as partners to build the entire ecosystem for them.

>> And you have the project in the Middle East too, I saw you guys are involved in.

Prasad Kalyanaraman

>> That's right, the Humain zone.

>> Humain, that's called?

Prasad Kalyanaraman

>> Humain is the entity that is associated in Saudi Arabia that is responsible for AI, and so we're working with them and we're partnering with them on that as well.

>> And you're overseeing that too?

Prasad Kalyanaraman

>> That's right, yes.

>> Okay, so that's happening very fast. By the way, the time, just scope the time. Go back the old days, old days, a couple, five years ago, how fast would you turn on a region data center to today? What's the ballpark? Is it prefab now? Is it more of a crew swing?

Prasad Kalyanaraman

>> Lots of things. We've been doing this on a, it's continuous improvements that we've been doing. It used to take us over a couple of years to actually build up a brand new region like that. Now, we're able to do it in multiple months like that, so we've been continuously reducing the time. A lot of that is coming from a ton of innovation that's happened across the teams. One of the important ingredients of that innovation is our ability to control the stack all the way top to bottom, right?

>> Prasad, I got to ask you because a lot of people who know the Amazon story, we've done our best to tell, it's a complex story, but it's historic. The early days, EC2, S3, SQS, basic building blocks that got cloud off the ground, but you talked to the insiders. Oh, it was a bunch of servers, okay, rack and stack, and then on the other side you have higher level services come along. Okay, that's created the cloud goodness we see today. But to me, I think that there's two moments in time that I point to and I want to get your reaction to this is that when I was talking to James Hamilton around Nitro, Nitro was a moment where things clicked, and then I was at a replay party and some guy says, "Hey, you're theCUBE guy." And he opens up his hand and he shows me the first Annapurna chip. So if you look at Nitro and Annapurna, those were two moments in Amazon history that changed the game. I mean, of course the cloud original formula was home run, but a lot of engineering happened around these moments. Those are just two, there's probably more, but to me those jump out as inflection points that changes the stack of how servers are configuration. All the networking that went in, all the advancements around virtualization, I mean all these things were moving along and then you got the chips. How is those two things and others, what would you say are the keys? Because you guys have a unique factory model that's, I won't say homegrown, home innovated, engineered. That's not just a bunch of servers. Explain what your view is on this one. Am I getting it right? Do you agree or disagree or maybe that's-

Prasad Kalyanaraman

>> Well, for certain our investments in Nitro and our investments in Graviton after that and Trainium are seminal moments from our, but I would actually not say that those are the only ones.

>> Yeah, share some more.

Prasad Kalyanaraman

>> We have a lot of things there that I would actually point out as were seminal moments for us. Obviously, the fact that we had thought about compute and storage and database services and networking as foundational building blocks was one of them. The second thing that I think was foundational for us is that we were very clear early on that we needed to have choice for our customers, and that was another foundational moment for us, which is if you just look at the number of database offerings that we have or the number of computer offerings we have, it's a pretty staggering number. And so a single provider being able to provide this vast array of options is pretty unusual. That's one. That's another one that I would actually say is that, and this is something that we have to earn the trust of our customers every single day, is our operational excellence. I would say that this is in some ways the unsung hero here, which is that customers will, sometimes we think about it as, "Oh, here's security, here's operational excellence, here's a set of features that we have." And customers will be annoyed if we actually get delayed on a set of features. But if you have a security event or an operational event for them, not only will they be annoyed, but they will actually remember it for many, many years, and gaining that trust back takes a long period of time. I would say one of these seminal moments for us is we were very clear early on that security was job number zero for us. We were very clear on that without operational excellence, you just can't build the cloud so that's another one for us. I would also say that our investments in the way we've thought about SageMaker and Bedrock and the agentic workflows, I think it's another seminal movement for us, which is that we were very unique in the way we thought about Bedrock. You can see that the underlying thing is still the concept of choice for customers. It's there across our security services, across our storage services, across our database services, across our compute services. And that has been really-

>> Actually, I remember talking to Stephen Schmidt in 2015, and at that time the big debate on AWS was the cloud's not secure. Remember those days? And actually, it's more secure.

Prasad Kalyanaraman

>> It is more secure.

>> And it was more secure, but no one understood that.

Prasad Kalyanaraman

>> Right.

>> Now it's actually more secure because there's so much stuff and you guys have such operational skills, you're seeing that so good call out there. I do want to say one thing. This database thing is a great add because we used to say the holy trinity, if you will, was compute storage and networking, but I'd add databases because you got to have the data with the compute.

Prasad Kalyanaraman

>> That's right.

>> And if you look at the database diversity you guys have had from day one, the choice, that was a key factor that enabled serverless, it enabled a bunch of other services that added on top of it. Okay, so now all that's the Amazon innovation. How does that translate into some of the stuff you're doing now? Now, you're taking that, I won't say putting it in a box that's a little trivial, but you're taking it and deploying these data centers and sovereign cloud's going to require to do the same thing by country. So I'm sure there's a lot of pressure on that too, to have country clouds, data centers in the country. You're basically abstracting all the Amazon goodness and deploying it.

Prasad Kalyanaraman

>> Yeah, I would say that very early on we knew that sovereignty was going to be a thing. It's not a new thing from gen AI quite honestly, and so we'd always built our cloud with sovereign by design concepts. We have 36 regions across the world and five more that we've announced and that'll be coming in the next few years as well. I would say that data sovereignty and the ability for some of the critical workloads to actually run in country, both from a latency perspective and from a control perspective is going to be there for a long period of time. In many ways, that would be a hard one to actually make a significant change on. You're not going to have the EU suddenly say, "I'm going to actually put this data in APAC," it's not going to happen. Data sovereignty is core to how our customers will think about it, and it is true about the non-ML workloads. It's going to be more true if anything else on the ML workloads. I would say that trying to actually build this capacity, and one of the reasons why we've been thinking about the AI zones in Saudi Arabia, and I expect that this will be more so than less so. I think we have the right foundations in our AWS regions already. In fact, many of our AI workloads and our gen AI workloads are running in country today, and so I just expect that journey to continue.

>> How is the AI demand and also the requirements impacting some of your build outs and capacity and your CapEx plans? Because if you look at some of the latest trends, I mean we just covered Broadcom's Tomahawk 6 launch, which a little gets nerdy in the weeds, but in the scale up pot side of the business, they reduce a hop on the network so you can have more XPUs in the system, this will impact your supply chain. But some customers have Tomahawk, I think 2, up there still so these are network fabrics, but this is what's connecting us. The configurations of the systems are being tweaked a little bit differently because with AI, there's a big demand for tokens which drive all the reasoning, and so tokens is the currency for feeding the AI. More data comes in, tokens go out, and as it gets smarter, more tokens are generated, which creates the flywheel for more demand and for bigger systems. The systems requirement for the GPUs and XPUs token usage, not to get in the weeds but to make the point is the usage patterns of AI have changed the system architecture and the configurations. I've been with memory, solid state as a storage tier. These are new things that are being identified as a system construction element for building into these factories.

Prasad Kalyanaraman

>> It's a good point, which is that if you look at the research on the size of these models, they're all growing in order of magnitude at the same time, both the compute capabilities and the number of T FLOPS from our hardware and our chip designs are also growing and then the networks are getting faster and higher bandwidth as well. The way these things both grow, you get into a situation each one is actually trying to get ahead of the other as well. I would say that few things are happening on the network side as well as the chip side. One, the chips are getting faster. If you look at Trainium2, Trainium2 is actually almost double the performance of Trainium1 as well, and 30% more performance than the P5s and the H200 based platforms. If you look at our ultra server that we actually reviewed at re:Invent, you're actually able to get 64 chips together acting like one big computer, and so that's another one. HBM, you talked about memory. Memory is actually continuously increasing as well over time. And our UltraCluster networks as well, they have, we call the 10 p, 10 year network. It's tens of petabits of second and less than 10 microseconds of latency. So you're reducing latency, you're increasing bandwidth for the collectives and so on. And at the same time, your chip capacity is improving as well as your network capacity increasing while the models are getting more and more powerful. Now, that's on the training side, but you also have the inference side of the equation and the inference side of the equation. As the models start getting bigger, you need to start actually building UltraClusters to actually store the models because the amount of memory is not sufficient for a single chip. You need to actually use the ultra server to be able to support that inference, but not all inference is like that. So customers are going to start using some of the smaller models for simpler tasks and then the bigger models for other tasks as well and so you need a combination of different kinds of prompt-

>> And you guys have caching and Bedrock.

Prasad Kalyanaraman

>> That's right.

>> And your infrastructure layer, which helps save on any kind of cycles on spinning up more compute or FPUs.

Prasad Kalyanaraman

>> Or in fact, the way inference works is that where you do your caching, where you actually do you take your prompt and collect and convert them to the actual weights and so on. That's more compute intensive versus memory intensive so you start splitting the way inference works, so lots of different technologies.

>> I love this conversation, I could spend an hour in fact that you bring up so much there. If you think about the old class of computer science, it's take an operating systems concept, it was pretty much software and the hardware. There's some machine coding, but all the access and networking because networking is controlling all the resources. It seems to be that's linking everything together and connecting and connectors are hot right now. You hear all the big things up at Snowflake and Databricks Summit, the connectors are not just like connectors, it's like MCP's hot.

Prasad Kalyanaraman

>> MCP, yep.

>> So you're starting to see new interconnects if you will, around this concept. It seems like networking is becoming the key software. Last year when I talked to Dave Brown before re:Invent, we were riffing that there's a lot of software innovation going on at this layer because of the new systems because it feels like the '90s back in memory management, about HPN, remember those days, a lot of engineering has to be done. What do you see impacting the data center? Because it is power-driven workloads might want to say, "Hey, here's a policy. I'm not in any hurry. I'm doing some thinking, but I also want to break the bank. Can you configure the resource to serve me up in a way that's acceptable, but I don't need to blow out the compute budget?"

Prasad Kalyanaraman

>> No.

>> These are things, I'm just making this up, but that's a scenario I can see happening.

Prasad Kalyanaraman

>> Also, evolution on the network has actually also grown at a fast pace right now. Some of it driven by gen AI, some of it was something that we had to innovate ourselves to keep our costs low for our customers.

>> And your costs and profits up, because no, economics are involved.

Prasad Kalyanaraman

>> That is totally right, and so very early on we started investing in building our own switches because we knew at our scale we needed to actually have our own commodity-based switches as well. We've actually gone deeper into our supply chain on optics, believe it or not. Optics is a big thing-

>> Co-packaged optics are hot.

Prasad Kalyanaraman

>> Right.

>> Well, talk about that benefit because not only just yields, it's failure rates, right? I mean or-

Prasad Kalyanaraman

>> It's true, but....

>> size?

Prasad Kalyanaraman

>> The thing with co-packaged optics is that we're not sure that's the right path to go to. And the reason for that is that when we look at diversity of the supply chain, you want to have enough diversity in the supply chain, will there be a place for co-packaged optics? Probably, but is that the first place you want to go? We're not convinced that that's the only place to go to.

>> In terms of trade-offs or optionality?

Prasad Kalyanaraman

>> Optionality, and trade-offs as well because you want to build a good set of concepts and distributed systems where it is disconnected so you can innovate independently on them. When you start actually merging them together, then you start stalling and you start having dependencies one from the other. As a networking layer for sure, there's a ton of innovation that has actually happened. We've had to go and rethink how we connect our networks together as well for ML, given the latency requirements that were there. And plus, when you start looking at larger and larger models and larger and larger clusters, you have to start thinking about how the collectives are going to work. And for that, you need to build a network that's fast enough to be able to not stall the model progression and the network should not be in the way for model progression, so we have to go and innovate ourselves out of that as well.

>> Let's talk about footprint because one of the things I've been riffing on our CUBE Pod is I've been saying the data center is the computer, a little goof on John Gage's network is the computer back in the days when Sun was around, if you remember those days. He was actually right, but a little bit too early.

Prasad Kalyanaraman

>> That's right.

>> If you look at the totality of the footprint, the building and everything that goes with it, you can use all elements of it to design something for the factory versus that's where the power plug goes. I'm oversimplifying, but as you rethink that footprint, things like the optics, these decisions you make, where are you guys now on that evolution? Would you say that the progress bar is very good, complete, more work should be done? Are you maximizing every aspect of the facility and the environmental conditions?

Prasad Kalyanaraman

>> Oh, I would say that this is something that I don't think our jobs are going to be done in that area. And the reason for that is that, so take from the top, if you look at the most expensive resource for our data center, it's power. But if you look at it as most expensive resource for the overall computer factory, it is actually the server and inside the chip and the memory that becomes, start getting really expensive. You always want to optimize for the most expensive resource and make sure that the most expensive resource is not constrained. If you talk about power, you want to make sure that none of the other parts of building a computer factory is actually constrained by power. What that means is that you want to give the maximum amount of available power, utility power to the compute and you want to spend the least amount of power for your mechanical loads or for networking or for storage and so on because by far, that's the most important thing that needs to be driven. We've spent a lot of time trying to optimize a few things there. One is how do we land racks inside a data center? And if you think about it, it's actually a bin packing problem. You have a certain amount of stuff that you need to actually land in a lineup, and you want to make sure that the right set of servers are there on that lineup, otherwise you start stranding power. That is a concept by in and of itself, and in fact it's wonderful places where we use gen AI to go and figure out how do we land these powers? Because if you think about it, you have a set of servers-

>> Power bound problem.

Prasad Kalyanaraman

>> It's a power bound problem, and you have a set of servers that are coming in through the supply chain that certain randomization associated with that in terms which set of server rack is going to come. Not all server racks consume the same amount of power. Then once you land a server on a particular lineup, it's not going to constantly use the same amount of power, there are variations there. And how do you use that variation to your advantage? That's another one that we think about. And third is that when you actually have correlated workloads, how do you make sure you don't exceed breaker settings and so on? So how do we load balance that particular power within a data center? So that's on the power side. Then on the mechanical cooling side, you have to think about, that's actually a PID control system which looks at the temperature in a cold aisle and it decides whether you want to actually mix outside air versus you want to mix it with the hot air, which is already there in the data hall. How do you keep that temperature within certain bounds? The reality of it is that the best way to cool a data center is actually just use outside air, not to suddenly start doing process-based cooling and so on. We've pioneered that for many years where we've tried to use outside air for a large majority of the time of the year because outside air is good enough for you to use it. You don't need to use power to actually start-

>> If designed properly.

Prasad Kalyanaraman

>> That's right, if designed properly. And so we do that. One of the concepts is how do you get hot air outside the data center? Well, the traditional way of doing it is to actually use an exhaust fan to exhaust the air. Well, that's consuming power, but if you think about it, hot air actually has lower pressure, so it's going to raise, right? And so how do you use that pressure differential to automatically exhaust the hot air? We've innovated on what we call as passive exhaust, don't use electricity to exit out hot air. Let it actually-

>> for the GPUs.

Prasad Kalyanaraman

>> It's a very simple concept if you think about it, but to make it a reality in a data center of this scale, it takes a lot of engineering.

>> This is why I like this conversation, because you're essentially looking at the footprint as a holistic design, almost like a motherboard.

Prasad Kalyanaraman

>> Yes.

>> It's like there's components we're dealing with, and knowing the characteristics and having a management layer over top based upon the premise from the data you have or gen AI, you normalize the environment.

Prasad Kalyanaraman

>> That's right. So this is why having control of all the way down to the chip level, to the network layer, to the data center layer, to the services layer gives us a huge advantage where we can start tweaking these parameters at any layer and see the impact on the other layers. And that way we can actually build balance between our trade-offs.

>> I mean, if you go back in history, look at the telephone and internet when you had packet switching networks. Oh my God, that's a revolution. You're essentially putting intelligence into the data center to maximize the power, because your power bound problem so your working problem at the same time you got to squeeze the most performance out for the workloads, right? So that's why I got this, I'm riffing on this network concept because the network has to be intelligent, smart, and know what is connected to and when and why.

Prasad Kalyanaraman

>> And where you place the word order.

>> And that's a software opportunity.

Prasad Kalyanaraman

>> That's right, yeah. Because many of these things that we build, we've actually built it in software because it gives us the advantage to move very fast on it and then in some cases we will actually build it into the hardware. For instance, when you think about the network switches and so on, some of those functions, we've moved it to the hardware or you take it Nitro for instance. We've done that because faster to do that.

>> That's easier.

Prasad Kalyanaraman

>> But you do that only after you've gained that experience to decide what is the balance between what you want to control and solve.

>> Well, you want to lock it in and maximize performance, get in the chip.

Prasad Kalyanaraman

>> That's right.

>> It's fastest. All right, I want to ask you about a concept that's going around this year. In the past six months, I've never seen this before, so it's new to me. Maybe I'm hallucinating, but I've heard other people, other vendors say words like homogeneous actually say that on stage at a telco, "Our system's homogeneous." I don't think I've heard the word homogeneous describing a market that is clearly choice driven. Heterogeneous has been standard. Open source is never going away.

Prasad Kalyanaraman

>> It's not. In fact-

>> Is there a hard top where it's okay to be homogeneous? To some extent you guys are abstracting away and optimizing, so I don't care if it's proprietary Amazon in an intellectual property if it's making things better for my choice.

Prasad Kalyanaraman

>> Yeah.

>> So where's the line in your mind? First of all, I think you agree homogeneous is not a winning strategy in a heterogeneous world, but is there an AI solution where vertical integration can creep up and is there a hard top where it doesn't really matter, everyone can compete on the value of what they do?

Prasad Kalyanaraman

>> Here's what I would say. I would say that first, I don't know in what reference the homogeneous terms used, but I'll say-

>> It was a telecom example. "Here's an AI factory, you should buy it from us." It's heterogeneous, I mean homogeneous runs everything you need. It has NIMs. I'm kind of revealing the vendor, but yeah.

Prasad Kalyanaraman

>> The reality of it is that I find it hard to believe that that concept will actually be there for a long period of time because predicting how our customers will use the underlying capabilities is a hard problem. And the reason for that is that take for instance, DeepSeek, right? They innovated on an area where they said, "Well, look at the cost of inference per token." And they said, "This is untenable." If you want to actually have this capability appear in many, many use cases, you've got to go innovate on it. And what are they going to do? They're going to actually go and hyper-focus on making that really, really, really fast. There are lots of companies that will actually do that. When you have an area that is growing so fast and so much of opportunity, you're going to find people who want to optimize portions of this thing and trying to provide a good homogeneous system for all of that is, I didn't find it to be untenable.

>> I agree with you. The reason why I bring it up because I was talking with, again back on our podcast, if you remember the '90s, the OSI stack was a seven-layer model that brought in ushered in the era that we live in interoperability standards, but a portion of that stack got standardized up to IP. So TCP/IP stops there and the rest just, everyone was open and they played on top of everything down to the physical layer so you had some de facto standards-

Prasad Kalyanaraman

>> Oh, yeah. For sure there are going to be standards like MCP is one of them, right? The OSI model as you call out. Yes, I totally expect that there'll be standards for it, but the implementations will be very unique to different use cases. Just because the OSI model came out, it doesn't mean that every network in the world is the same, right? Someone could actually claim that OSI is actually the standard. Yes, it's a standard, it's not a homogeneous system.

>> It's an open standard.

Prasad Kalyanaraman

>> It's an open standard. I expect a lot of that to happen for sure. We are already starting to see that in some of these areas, so I do expect that to happen. But would the implementations and how the value provided for every single customer going to be the universal? I highly doubt it. Even in the OSI stack, even though IP is fairly standard, just look at it. How networks and optical connectivity has changed over a period of time has been a vast amount of improvements. The OSI model has been there for decades. It doesn't mean that what we had as DAC cables are still the case right now, right?

>> As you look at the supply chain, as advancements come out, you're going to start to see the winners get separated from the field a little bit, whether it's a Broadcom or an NVIDIA in certain areas. But as you said, there's a lot of other things going on when you look at things like where you get efficiencies like co-packaged optics, you can make an argument that the connections can reduce what's going on in a rack but might not give optionality for you. But when do you decide it's a go, to go with the next big thing or the next big technology?

Prasad Kalyanaraman

>> Yeah, generally the way we've looked at that is we've said, does it actually provide a pretty meaningful step-up function for our customers and from a security or performance or capability perspective? And then we look at it to say, is this something that's a long-term play for us that will remain for a long period of time? And so you talked about optics. One of the things that we've been working on for many years is making sure that the DSP, the lasers, the packaging and how we build the optics, we want to make sure that there is enough diversity in each one of them. We could have actually gone and said, "Okay, it's this DSP and this lasers and this transformers and that's it, that's the only way our optics will be." That would be extremely limiting for us at our scale. Instead, we took the approach that there's going to be innovation on DSPs, independent from lasers, independent from how we actually do this package as well. And that has given us a lot of advantage, but we have a lot of diversity now. And now, is that a standard for how we think about optics in terms of what the signal of integrity needs to be on the optics? Yes, absolutely. We want interoperability among them. That would imply that that's harder work for us because one end of the optic could be from one set of DSPs and lasers, the other end is the other kind. If you draw homogeneity across them, they're going to be beholden to just one kind and that'll be very hard.

>> I know you've been in Amazon for two decades and seen the movie and seen the growth has been a great run, congratulations. I'm not sure if this is an Amazon principle, like a bias for action, all the things you guys have, but I hear people talk Amazonian and talk, they say, "What are you optimizing for?" Because if you optimize for optionality but don't deliver.

Prasad Kalyanaraman

>> Yes.

>> Okay, you optimize for the wrong thing, so you got to balance it. As you look at the deployments, you're going to be rolling out more regions. You've got the Middle East project, you've got all kinds of new regions, one that have been announced or delivering on. I'm sure there'll be smaller regions. I mean, I'm just imagining the footprint being more distributed because of energy. How do you look at that demand when you roll out into these regions because you're energy bounded? I think if you look at the number of data centers, if you look at the stuff that's being announced in Texas and in Arizona, the already committed data centers, I think the estimate was 40 million homes in the United States power. The power requirement's huge, so there has to be an innovation in the power bounding element. Power's a scarce resource. You're bounded by power, so there's got to be some innovation on your mind as you roll out these regions. One, can I turn it in a box kind of thing? You read it in a box, that's my word, not yours. That's the way I think about it. Maybe that works. But does there have to be an innovation and energy?

Prasad Kalyanaraman

>> There absolutely has to be, and you're starting to see that in some ways. So I talked about some of the innovation we've done on efficiently using power, but on the other side of it, I don't believe that there's going to be a single solution for all of the power needs, right? Because if you look at our portfolio of how we've thought about power, beyond all the make sure that you're using the watt the most efficient way possible is that it's going to be a combination of wind, solar, hydro, batteries, nuclear, and some sort of bridging capabilities as well while we get there, right? Many years back, we started investing in solar and wind and hydro. Why? Because one, we had a goal to actually get to net-zero and 100% renewable globally as well. And two, because we knew that longer term, all these will play a part in our solution to actually solve this problem. The way I think about it is that there's going to be a lot of innovation on new energy sources. There's going to be a lot of innovation on carbon capture and so on. There's going to be innovation on battery technology because storage will be an important ingredient to how we solve for power. It's not there yet, but that doesn't mean that we're not investing in it. I'll give you a very concrete example. If you actually look at the power profile for a regular household, you'll see that there are peaks and valleys. You look at the power profile for a data center, it's also going to have peaks and valleys, but not as peaky as a regular household would be. The obvious question is what happens to all those troughs and what do you do at that time?

>> Sell it to somebody else, or use it.

Prasad Kalyanaraman

>> Or use it, and who's there to actually use it? If the world is sleeping or if a particular region is sleeping at a certain period of time, you're going to use a certain amount of power. The sun does not actually shine 24 hours a day, it shines a certain time of day. The wind doesn't blow all the time as well. So what do you do when you actually have abundance of a certain amount of energy source? Obviously storage comes into play. Now, when you have storage, what is the scale of the storage? What is the cost of that storage? And so that will be innovation on trying to actually use those troughs. And so we've looked at it and we've said if we looked at those peaks and if we actually worked with the utilities to say we can shave off and we call it peak shaving and we're able to shave off the peaks, how much more power can you enable? And so if we look at our transmission capacity, they're all built for that P100, right? But if we went to our providers and we told them, "Look, we can reduce the P100," suddenly the aperture opens pretty significantly. Now you're actually having a conversation, "Oh, we can actually raise the watermark on the minimum a lot more." And so utilities love that because what they don't want to be doing is to suddenly have these huge peaks and valleys and pikes. That's what they cannot handle. But if you told them, we can give you a lot more stable usage, they love it. And so enabling that and working with our partners to be able to enable that-

>> This is the collaboration piece I like, I think this is a big story because communities, it's not just, "Hey, I'll build a school and donate to the tech robotics club," or whatever people do. It's actually real value because, one, you're not putting pressure on the grid, get solar. If you're a self-contained data center, that would be the ideal scenario. Are most of your data centers self-contained for power yet? Or is it-

Prasad Kalyanaraman

>> No, we do actually work with the utilities, in fact, quite the other way around. The reason is because you have a lot of advantages when you're utility connected because you can sell that extra power if you need it to when you're not using it that you can load balance across the thing. But of course, as I said, that's another one-

>> So you're a participant in the grid.

Prasad Kalyanaraman

>> Yeah, we do both ways.

>> You're not just the consumer.

Prasad Kalyanaraman

>> That's right, that's exactly right. The advantage of doing this is because we bring a load which is a lot more stable, they love it because they would rather actually consume all of that excess power-

>> Well, you're reliable. It's like a credit rating.

Prasad Kalyanaraman

>> Exactly.

>> They're going to maintain their budget. Talk about the impact of the geopolitical because I can imagine that it's not always as savvy as you roll out these data centers. Obviously, these highly motivated areas, we see those, but whether it's the country, how educated are they on some of these innovations? Is that an opportunity for you?

Prasad Kalyanaraman

>> I think world is getting a lot more educated on this thing. I would say that especially with, I think we started doing that education many years back when we started cloud computing, like the research that came out from Accenture, we talked about how you'd be 99% more efficient running on the cloud than running on your own data center. I think it opened up a lot of conversations about how this is a better way for us to actually support the world. I think lots of countries are now, it's a fairly obvious thing that gen AI has huge potential, no one's actually doubting that. I think every single geography is trying to figure out what it means for them, and they're obviously welcoming us to try and actually help them. You saw that in Saudi Arabia, we were seeing it in many other countries as well. I do think that, I think geopolitically, many countries are realizing the value of what this can bring to them, and they're seeing this as an opportunity to grow as well. In many ways, this is turning out to be a good partnership with many of the geographies we work with.

>> You got to have an advantage because you can come to the table and interest the elephant in the room, which is Amazon's going to suck all the power out of our grid because that's probably a knee-jerk reaction on some folks.

Prasad Kalyanaraman

>> Most people actually have that knee-jerk reaction when in reality, if you just look at the number of projects that we've actually enabled in terms of we have over 600 projects of wind and solar in the world right now. And in fact, many of these geographies, we have gone there and put wind and solar projects well before we even thought about building our infrastructure. And so that's actually a really valuable thing for whether it is a jurisdiction or a country and so on, which is they're getting access to a certain technology, which is well beyond.

>> Prasad, talk about for the folks watching, what are you most proud of? I mean, you have quite the job. You have really important job for Amazon Web Services and Amazon proper. You have to deploy one of the most demanded areas right now, which is the CapEx and build the facilities and all the systems that power the future of society and the sustainability so you're highly threaded in the world these days. What are you most proud of work you've done in your team?

Prasad Kalyanaraman

>> Yeah, I'd say that one of the things, so first I'm very appreciative that I have a team and a set of team leaders who I get inspired by every day. If you look at my journey at Amazon, I've been always a software leader before, and for me to run infrastructure, it's quite a change. I'm very fortunate to actually have the opportunity. I would say that the things that I'm most proud of at Amazon is that sometimes people ask me, "What's the recipe to be successful at Amazon?" And I think we've figured out a way of taking that question and translating it into something concrete with our leadership principles. The one I'm most proud of on the leadership principles is what we say is customer obsession and our ability to deliver results. And internally we sometimes talk about as the tyranny of the ors. I'll give you an example where people can come and say, "Oh, gen AI is actually growing so very fast and that's going to consume a lot of power. How are you going to actually deal with all the sustainability challenges that comes with it?" And my answer is, we don't accept the tyranny of the ors. For us, it's not an or. For us, it's an and. What that means is we have to support this workload, we have to grow, and it's a huge opportunity, at the same time still be sustainable.

>> And.

Prasad Kalyanaraman

>> The and part of it-

>> Not maybe or or.

Prasad Kalyanaraman

>> That's right. The thing I'm most proud of is that every Amazonian never thinks about it as I can do this or I can do that. We always think of it as we have to do this and we have to do that. The advantage of pushing ourselves that way is that you actually accomplish new things that most people would look at and say that is quite, one, industry leading; two, it is so focused on what our customers want. And it is also very invigorating, right? Because when you're able to accomplish something personally that one, you learn new things. I believe that every leader at Amazon is a lifelong learner. And when you're learning something at the same time, when you thought a few months back, this is impossible to do and you still made it possible, that is very invigorating. And so the power of many things-

>> That's a mindset because if you say, "I want to support this Amazon goal and the customer," or the customer and our goal, if that's in the problem statement, you work the problem.

Prasad Kalyanaraman

>> That's right.

>> And so not I'm working the problem and then, okay, how do we retrofit that in? So most companies do it the other way around, how do I sell this to the customer?

Prasad Kalyanaraman

>> Yes.

>> Are you happy?

Prasad Kalyanaraman

>> Right. We are able to do that in a way. In fact, that is a mindset and as you rightly called out, it's ingrained in our leadership principles. If you look at our leadership principles, one of my favorite ones is we have a leadership principle called on frugality. And at the same time, you have a leadership principles on deliver results, right? Now, someone could naively look at it and say, "Oh, I could deliver a lot of results by spending a lot of money." But we don't accept that. We're actually asking our leaders to be both frugal and deliver results. There is a good balance, they have healthy tension in our leadership principles.

>> That's the balance and the equilibrium that if you're too frugal, you don't deliver results, but if you get that right.

Prasad Kalyanaraman

>> Getting that balance is always going to be very key. But this is, in my opinion, one of the things that I am most proud of is this.

>> What is your goal then? Who is your customer for your group? Is it the world? Is it the?

Prasad Kalyanaraman

>> It's the universe.

>> Is it the community? I mean, you go to North Carolina and you have other outside the country goals. Is the customer the end user or when you have the end, you have multiple end statements actually going on, for you, but what is the customer?

Prasad Kalyanaraman

>> It's a good question. The reason why it's an important question to answer is because we don't singularly think of customers as just our external customers who are actually going and using ourselves and paying us for the use of the services and building applications. When we go to any location, we think about the community as our customers. We think about our internal employees as our customers as well. We run some of our infrastructure services like look at our data center communities. Our data center communities. Think about the communities that they operate in as our customers because we are in the location where we're building, and so we need to become part of the ecosystem that we are operating in. And so that's the reason why we invest in STEM education and local upscaling and so on, because why? We think about them as customers.

>> That comes full circle back around.

Prasad Kalyanaraman

>> That's exactly right. When you change that mindset to say our customers are broad and it's not just the customers are buying something at amazon.com or using some of our services, I think you have a different perspective, and this is where the concept of tyranny of ors goes away because we have to build this and we have to actually keep these set of customers very happy.

>> When are you going to have an agent doing all this for you?

Prasad Kalyanaraman

>> Oh, I think that is, stay tuned. It's going to happen very soon in my opinion.

>> Great to chat with you, I really appreciate the time. I know you're super busy, and again, congratulations on having probably one of the most exciting, provocative jobs, probably intellectually challenging. Got a lot of ands in there, thanks for coming on theCUBE.

Prasad Kalyanaraman

>> Thank you, John. Thank you for the conversation, really enjoyed it.

>> The complexity involved in the energy bounded problem of delivering the value of AI is all driven by the engineering the system. Amazon is leading the front there. I'm John Furrier with theCUBE, bringing all the action for a halftime report, midpoint between re:Invent, all the action here in Seattle at the headquarters of AWS. Thanks for watching.