This YouTube interview features Andy Warfield, vice president and distinguished
engineer at Amazon Web Services, during the AWS re:Invent 2025 conference.
Warfield delves into the transformative shifts within AWS storage, particularly
focusing on the role of Amazon Simple Storage Service (S3) in the evolving
landscape of agent infrastructure and generative artificial intelligence, as
discussed in TheCUBE Research. In this insightful interview, Warfield explores
the changing dynamics of S3 and storage within AWS's architecture. The
discussion highlights the emerging importance of AI agents that operate as
integral components of workflows rather than simplistic tools. Warfield also
addresses advancements in S3 storage functionality, emphasizing the role of
vectors and metadata in enhancing AI workloads and overall performance. Key
takeaways from the video, according to Warfield and TheCUBE analysts, include
the significant cost benefits and flexibility offered by S3 vectors in contrast
to traditional memory-based vector databases. Warfield illustrates how AWS's
storage solutions continue to innovate, serving a broad array of use cases such
as life sciences and retrieval-augmented generation. The discussions also reveal
the significance of multimodal embeddings in advancing AI capabilities.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
AWS Marketplace Newsroom . If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for AWS Marketplace Newsroom
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for AWS Marketplace Newsroom .
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
AWS Marketplace Newsroom . If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to AWS Marketplace Newsroom
Please sign in with LinkedIn to continue to AWS Marketplace Newsroom . Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Andy Warfield, AWS
This YouTube interview features Andy Warfield, vice president and distinguished
engineer at Amazon Web Services, during the AWS re:Invent 2025 conference.
Warfield delves into the transformative shifts within AWS storage, particularly
focusing on the role of Amazon Simple Storage Service (S3) in the evolving
landscape of agent infrastructure and generative artificial intelligence, as
discussed in TheCUBE Research. In this insightful interview, Warfield explores
the changing dynamics of S3 and storage within AWS's architecture. The
discussion highlights the emerging importance of AI agents that operate as
integral components of workflows rather than simplistic tools. Warfield also
addresses advancements in S3 storage functionality, emphasizing the role of
vectors and metadata in enhancing AI workloads and overall performance. Key
takeaways from the video, according to Warfield and TheCUBE analysts, include
the significant cost benefits and flexibility offered by S3 vectors in contrast
to traditional memory-based vector databases. Warfield illustrates how AWS's
storage solutions continue to innovate, serving a broad array of use cases such
as life sciences and retrieval-augmented generation. The discussions also reveal
the significance of multimodal embeddings in advancing AI capabilities.
In this interview from AWS re:Invent 2025, Andy Warfield, vice president and distinguished engineer at AWS, joins theCUBE’s John Furrier to discuss the pivotal shift toward agentic infrastructure and the evolving role of storage in the AI era. Warfield explains how Amazon S3 is transforming from a simple object store into a critical component for AI factories, specifically highlighting the launch of S3 Vectors and improvements to S3 Tables. He details how these advancements bridge the gap between generative AI capabilities and enterprise data, offering a unif...Read more
exploreKeep Exploring
What is the significance of storage in the context of this year's developments in AI and agents?add
What are the current advancements and challenges in integrating generative AI capabilities with data storage solutions?add
What are the considerations and challenges involved in implementing vector databases for storage systems?add
What are the use cases for S3 Vectors and how are they being applied across different industries?add
>> Welcome back around to theCUBE's live coverage of AWS re:Invent 2025. I'm John Furrier, host of theCUBE, with the whole CUBE team here on the ground, 60,000 people always are packed as usual. Again, another great performance from AWS. My 13th year covering it and this is one of those years where you start to see the shift. Every five or six years, something game changing happens and it's around agent and agent infrastructure. Abstracting away the complexity around the work, first you abstract away the infrastructure, that was cloud. Now you've got agent infrastructure, abstracting away all the logic and the work, creating more value. That's the key focus of re:Invent. Andy Warfield's here. Vice president, distinguished engineer at AWS. Talking about storage, S3, all the goodness last year. Andy, great to see you.
Andy Warfield
>> Good to see you.
John Furrier
>> Thanks for coming on. I know you're super busy.
Andy Warfield
>> Thanks for having me back.
John Furrier
>> Thanks for making the time.
Andy Warfield
>> 13 years.
John Furrier
>> 13 years. We've seen it all. 14 re:Invents, has only been 14. We've been to 13 of them. A lot of change.
Andy Warfield
>> Yeah.
John Furrier
>> So tell me, this year, a lot of agent and AI stuff, obviously, that's the market hype and the reality is they're not toys anymore. Agents, chatbots aren't just toys, they're teammates, they're systems. A lot of under the covers things happening in storage. All the AI factory conversations that I've been in, storage, memory, compute are all discussed all the time. So the role of storage specifically has changed a lot. Give us your take on how that relates to this year's theme. We heard AI factories, we hear agents, storage is a key part of that. What's changed? What's happening in the S3 realm?
Andy Warfield
>> So exciting.
John Furrier
>> It is. It's pretty fun.
Andy Warfield
>> I think one of the things that we've seen, we're still so early with the agent stuff, I think this is the year where we've started to see productive work happening. I'm doing a lot of actual hands-on development more than I have in years directly through agents and stuff. It fits into the pretty busy schedule that I already have and it lets me be more into details on things. So it's just been really personally cool to be back into coding on things. I think that what the storage teams are really seeing is on the generative AI side, we've got models that can write code. We've got models that can write docs. And at the end of the day, the big bridge that is on us to solve is the bridge between all of that kind of capability and the data that people have. And so a lot of what's happened in, especially S3 and the file services, is providing that metadata layer and that integration layer to find and work with right data.
John Furrier
>> Yeah. And they put the chart up up on stage in the keynote, three major boxes, instances, S3 and databases. So database is the fourth pillar of compute storage networking and database that runs the infrastructure. S3's got six items in here, object store, tiering with tables, data channels, you got 10X batch operations, replication, automatic replication, and then all kinds of other things, access points. More things are happening. What's the hottest area that you're pumped about this year? Is it the vectors? Is it the...
Andy Warfield
>> The vectors launch is a big deal that I think everybody's really excited about. I think, I mean, with so many mature customers building on top of S3 and so many emerging customers, like just starting with objects and building and filling out these other data types in tables for structured data and vectors as a way to search, is letting them kind of build apps that are kind of soup to nuts S3 based, which is super cool to see.
John Furrier
>> People are scratching their heads, I know I am a lot of times and others that are in the know on how to architect all this. They either have S3 or they're putting stuff on S3 or they have other vector embeds and there's all kinds of confusion. Everyone seems to have a vector database or vector capability. Talk about what you see as important on S3. Why switch? Why do vectors there?
Andy Warfield
>> Why do vectors on S3?
John Furrier
>> And when I already have it over here. I mean, just search or is there a headroom there? What's the reason?
Andy Warfield
>> Well, first of all, John, we've talked before and you know how conservative the storage teams are about big changes like this. We argue about them, there's passionate discussion. And so the decision to go and do tables in the first place and vectors were not light decisions on the team. We really, really spent time thinking about that. The thing with vectors, so just to make sure that everybody understands this is a way of embedding like a vector is a description of a piece of media or a text that you can search for. Really, really important for AI workloads because AI needs to find the data that it's going to work with. And for a lot of S3 customers-
John Furrier
>> And it's math.
Andy Warfield
>> And it's math, yeah.
John Furrier
>> Math is good.
Andy Warfield
>> And the bit of the math that's been kind of a tricky and challenging bit is the vector database implementations tend to be built in memory. The data structures do a lot of back and forth from memory, and so they need fast storage. The vectors themselves are kind of large. In other cases where you're indexing storage, you generally have like a really, really small index that you don't worry about relative to the size of storage. You worry about the computer maintaining it, maybe. But with vectors, in the case of text, especially text, text is so rich with meaning and so small to store you often generate a vector like a paragraph basis in text. And so your index can be 200 times, like the size of your data. As a result, if you're storing that in SSD or DRAM, you can end up in a spot where like the index is actually quite expensive relative to the PDFs or the text that you're storing. And so that was a thing that we were seeing customers that were building on top of document databases like text repositories, code, call center logs, things like that on top of S3, really wanted a better tool that gave them the scale of S3, the smallness of starting with S3 and the performance.
John Furrier
>> Yeah. And so it's really a combination of both. What's the current state of the art right now? What are you seeing? Is there a certain use case pattern people are using it for? What are some of the use cases?
Andy Warfield
>> It's been our most popular preview. It's actually like we launched the preview in July, and I can't think of another thing that we've launched that has grown so fast on S3. Which was kind of surprising because we weren't sure how broad vectors were. We knew that there were some customers that really wanted them, but we weren't sure how broad the uptake would be. Obviously a lot of it is direct retrieval augmented generation, RAG, interface with ML pipelines. However, we're seeing stuff in life sciences. We're seeing people use S3 Vectors to do drug development, like radiology applications. It's actually like really, really wild to see the kinds of things that people are using vectors for.
John Furrier
>> It's funny how good metadata can be. And you mentioned context is a huge part of it. AI is all about context. Agents need contextually relevant data.
Andy Warfield
>> Right.
John Furrier
>> Talk about the relationship between the cost envelope and performance, because you mentioned if you have a big fat vector, how do you talk about the cost performance ratio? Are there any benchmarks out there? What are some of the stats? Can you share the value there?
Andy Warfield
>> I mean, the stat that we came up with when we first worked through the vectors preview and over the past bunch of months is we're seeing about a 90% cost reduction for storing a vector index in S3 relative to storing it in like a provision managed in SSD instance. The other cost factor that's I think really important with vectors is that a thing that we just keep hearing is like vectors is four APIs. There's no provisioning. It's S3. And so you can sit there-
John Furrier
>> ....
Andy Warfield
>> or sit with a coding agent and just like create the index, go do an experiment, and it's really playful as a tool. You're not provisioning up a database and making a bunch of decisions and stuff like that. And that property of being able to just get going and the property of being able to like leave the vectors because you're paying for generating the embeddings to leave them as storage and only really like pay for requests as you access them is like a really attractive property. A lot of these vector stores tend to be idle a lot of the time and then someone will log in, they'll start using them and then they'll go away. And so it seems to be a good shape for the way that vectors are used.
John Furrier
>> I'm going to ask a dumb question so you get to correct me if it's too dumb. But if someone has a vector database, got in early, and wants to move it over to S3, can they do it? How do they do it? How does someone say, "Hey, I really want to take a look at this."? I mean, you can certainly try things with a clean sheet of paper. But say I have my whole corpus and multiple data, I'm running a RAG engine, retrieval augmentation generation, some retriever product out there.
Andy Warfield
>> I think you're off to the races. So obviously if you're starting with a clean slate, you just go at it. The APIs are simple enough and the vector store is un-opinionated about embedding model. And so we've built it to support a wide range of vectors in terms of the-
John Furrier
>> So it just takes all the vectors already free done. No need to generate it.
Andy Warfield
>> Yep. Move them in there and off you go.
John Furrier
>> So no need to generate.
Andy Warfield
>> That's right.
John Furrier
>> Because you already have.
Andy Warfield
>> That's right. You would have to do a migration.
John Furrier
>> All right. So what's the coolest thing besides vectors that's going on? Could be a renaissance of another product, the role of instances now, play closer. What are you excited about on this re:Invent?
Andy Warfield
>> So just staying on vectors for a bit, just in that there's some exciting stuff.
John Furrier
>> stay.
Andy Warfield
>> There's one thing that I want to say. The Nova embeddings model, the team's been playing with it and this idea of like multimodal embeddings and being able to like work across lots of data. I mean, you guys use all this stuff for your own library. It's just been fun to play with. So I would just like mention that, go with play with-
John Furrier
>> I can talk about vectors all day long because we use them. All of them.
Andy Warfield
>> Awesome.
John Furrier
>> And of course we do video and we have text. We don't really have an audio speech thing yet. We're working on that. But multimodal is the requirement. Expand more on that concept.
Andy Warfield
>> I mean, Matt even talked about it in his talk this morning. It's been difficult up until like very recently to work across data types. There have been limited multimodal models, but generally if you really wanted to do great with video, you went and used a video model. If you wanted to do great with text, you went and used a text model. And so that was fine, but now you're like working across different searches and different types of data. And so being able to do what the Nova embedding folks have done, combine those into a single model and code individual vectors and do well with them is a big deal.
John Furrier
>> Yeah. I mean, pretty cool announcements this re:Invent. I was saying all the 13 years, you can feel the ground shaking in a good way because you have now the AI infrastructure, the things they're talking about with frontier agents, I love that concept of having that kind of intelligence longer.
Andy Warfield
>> This idea like in Lambda of longer running sessions and agents that run longer is really-
John Furrier
>> That's a really game changer. And then the Nova Forge, I mean, that's like philanthropy for Amazon to give that away. I mean, they're going to charge for it, but they're basically going to give-
Andy Warfield
>> It's going to be really cool to see what-...
John Furrier
>> frontier capabilities to a data set. So we could literally take CUBE Corpus and take Nova and make that like a frontier model.
Andy Warfield
>> CUBE-Nova.
John Furrier
>> Yeah. CUBE-Nova. I mean, I got to come up with a name. Everyone's got names. Launch monitors called their Edwin. We have to call our model something the name.
Andy Warfield
>> So the broader S3 stuff, I think the year in tables has been incredible. I remember being here last year and talking to you about it. It was like a pretty hot off the press launch and the experience that we had with that launch was, I think it was like a wonderful thing for the team. We launched that thing and customers assumed that tables had every single property of S3 and they did have all of the fundamental properties in terms of availability and durability, but customers wanted every feature and just assumed it was there. And the team has just been like every month cranking. And so this week we've launched cross region replication for tables. We've launched intelligent tiering to get you to automatic storage class for them. Moving forward to Iceberg V3. The uptake on tables, and the thing that I am like so kind of inspired by in terms of adoption is we launched it with Iceberg being a format that analytics services use, but what we're seeing is application developers just building directly to tables. So we're seeing people use tables as a way to store state for their applications.
John Furrier
>> Yeah. And that's when you start getting this low level coding theme going on. You mentioned tables. It made me think of the announcement that Matt talked about with evaluations. I know Databricks talked about that at their event. You guys, we hinted about that at tables. So things like state are happening. What is the state of the art right now thinking around data lakes? Because data lakes seem to be evolving into, certainly with AI factories, almost unbundling of things. We started to see more modular thinking around storage because like if you put storage next to the compute, we see this obviously with the GPUs and kind of like solid state and hype bandwidth memory, you get better performance. And with all these new ultra servers, is there kind of a developer trend or pattern around how to think about that storage layer?
Andy Warfield
>> I think so. I think broadly with data lakes we're really seeing customers lean into like componentized open architectures. And so with tables, with us building tables on top of Iceberg, a thing that we hear customers say that they love is they know that data is in a state that it's going to be accessible by any engine. So they're not locked in and they're free to work with it however they want. I think a thing that you're seeing up and down our stack is us continuing to focus on decomposing things that way. And so there's a really cool launch actually in the analytics organization this week, which is materialized views effectively as an externalized surface. So materialized views, which is a function that historically has been inside the engine, we've taken built on top of tables and on top of normal S3 and you can register a Spark query that describes your view. You can say, generate me a table that looks like this with these input tables and we will maintain that table in an S3 table and you can use it from any analytics service that interacts with S3 Tables.
John Furrier
>> Sounds like an engine compatibility feature.
Andy Warfield
>> It's almost just like taking the engine apart and making sure that if you're going to go and do a bunch of compute to generate a view, that that compute in the same way that having the data in tables serves whatever engine you want to bring to bear on it, that view generation also is there.
John Furrier
>> For the folks watching that know and dealt with S3 and is working with storage and compute and all this instance, with all this AI discussion with agents, with agents going, frontier agents, they're going to be smarter and require and they're going to be more autonomous. Is there a storage paradigm that you like or you recommend people think through from an architecture or deployment standpoint? Is there a best practice or a vision around how this will evolve because the parts are coming together with tables, got vectors. Is there a path, I guess?
Andy Warfield
>> So I think that there's a really interesting observation to the thing you're asking, which is that the opposite is actually true in some senses. One thing that we've found with writing code using agents against all of our storage services is the agent is very fast to understand our docs and to play with our APIs and to learn how to build stuff. And so what historically has been a bit of friction in terms of like growing API with things like that is actually like an accelerator for like agentic-
John Furrier
>> It's almost the more chaotic it is, the better the agents are.
Andy Warfield
>> Yeah, totally true.
John Furrier
>> It's like more functional decomposition, more breakdown, if you will, so they can then stitch their own because the humans aren't going to do it.
Andy Warfield
>> I think the thing you're calling out though around guardrails or facilities to let agents play and experiment and for you to decide what to keep, I think that's the thing that the team's thinking a lot about and that we're going to continue to work on.
John Furrier
>> Andy, great to have you on. Always a pleasure. Again, S3 always been a key success, obviously still on the big chart there, but as storage, you don't want to have that in the compute, you want to make it available as fast as possible. And with vector, it's super exciting that AI format is all math. I mean, GPUs and the high performance chips all love matrix multiplication. They love that.
Andy Warfield
>> That's totally true.
John Furrier
>> So the world's gone math. So congratulations.
Andy Warfield
>> Awesome.
John Furrier
>> Thanks for coming on.
Andy Warfield
>> Thanks for having me.
John Furrier
>> All right. I'm John Furrier with theCUBE here at re:Invent, 13th year covering, again, the building blocks keep on getting bigger or smaller in this case and broken down and now agents are going to stitch it all together. Exciting time as we look at the infrastructure expanding into the AI era, more coverage after this short break.
John Furrier
Andy Warfield