Name: Maximize the value of your data: Unifying Data, Analytics, and AI on AWS
Uploaded: 2025-03-14T15:00:00.000Z
Duration: 16 min 47 s

Clips
More from Digging Into AWS Pi Day

Sirish Chandrasekaran

VP, Analytics

AWS

play_circle_outline Next-generation Amazon SageMaker features and announcements

play_circle_outline Shift towards open-table formats and lakehouse strategy with SageMaker

play_circle_outline Real-world applications of agentic AI in industries like automotive and healthcare

21:49

Info
Transcript

Maximize the value of your data: Unifying Data, Analytics, and AI on AWS

Sirish Chandrasekaran

VP, Analytics AWS

Maximize the Value of Your Data: Unifying Data, Analytics, and AI on AWS

Join theCUBE Research for an insightful discussion with Sirish Chandrasekaran, Vice President of Analytics at Amazon Web Services, as we explore the intricacies of advancing AI innovations within enterprises. Filmed during the AWS Pi Day event, this session reveals key strategies for overcoming common data challenges and enhancing AI-driven decision-making.

This video features a conversation between Chandrasekaran and Rob Strechay of theCUBE Research, focusing on the conver... Read more

explore Keep Exploring

What are the three new components introduced in SageMaker AI at re:Invent? add

What are customers looking to maximize when it comes to their investments in data and what is the strategy for meeting customers where they are in terms of data usage? add

What are some real examples of companies using agents in innovative ways to improve data analysis and problem-solving? add

bolt Powered by CUBE AI

Maximize the value of your data: Unifying Data, Analytics, and AI on AWS

Rob Strechay

>> Hello, and welcome back to this special CUBE event, where we are digging into AWS Pi Day. I'm Rob Strechay, managing director with theCUBE Research. As businesses push the boundaries of AI innovation, they face real challenges, siloed data, complex pipelines, and inconsistent access controls that make collaboration very difficult for their data teams. And with ever-growing need for security, reliability and compliance, organizations need not just accessible data, but trusted, well-governed data to fuel AI-powered decision-making. To dig into this, I'm joined by Sirish Chandrasekaran, who's the VP of analytics at AWS. Welcome onboard, Sirish.

Sirish Chandrasekaran

>> Thanks for having me, Rob.

Rob Strechay

>> I think that this is one of those things that we look at. Analytics is on fire and fueling so many things, especially with all the stuff you're working on at AWS and you're in the trenches day-to-day. And I think one of the things when we're talking to organizations is that there's a lot of things that we hear that are keeping up chief data officers and those folks working on AI innovation. What are you hearing from customers?

Sirish Chandrasekaran

>> Such a good question. Every leader I talk to who's responsible for the company's data or AI strategy has some version of the same question, which is, "How do I differentiate, in this fast-moving environment, where almost every week you have a new technical advance, whether it's a new gen AI model or a new use case?" And the answer I give them, and the answer, many of them are coming to by themselves, is rooted in something that has been true for quite a while now, which is that every modern business is a data business. And so, I tell them, "You're a modern business and your data is your differentiator." Because when you think about transformations of the past, whether it's ERP transformations, cloud transformations, same thing here. When you get down to the basics, data is the make-or-break factor for your AI success.

Rob Strechay

>> Yeah, I think that is such... Again, being a data person and a storage person, we talked about this with Andy and everything, customers are just dealing with the same data issues as they have before, but the scale is just so much larger with AI.

Sirish Chandrasekaran

>> It is.

Rob Strechay

>> And what are some of the other things that are really pushing them in this new AI era?

Sirish Chandrasekaran

>> Yeah. When I talk to these customers about what they're doing to strengthen their data foundations for AI, really many of them come up with similar challenges. Some of them old, some of them new, but it's around three themes. And these themes are convergence, governance, and agents. And so, when you think about convergence, it's about convergence of workloads, convergence of user personas, of the data in AI stack. Previously, you would think of different silos, right? Structured versus unstructured data, warehouses versus lakes, data scientists versus data analysts, BI dashboarding versus model development, but those lines are blurring. And as these lines are blurring, what we are hearing from customers is they want converged and unified solutions, where you can run faster and maximize your value out of the data investments you've made. And the second thing is governance. Governance has always been critical in data, right? It goes without saying. But what's different about governance in the AI world is it's not just about compliance, it's actually about confidence. You want the confidence that your AI is trained on trusted data and that your AI is adhering to responsible AI use policies. So, it's not a checkbox, it's something that has to be woven into every part of your data and AI workflow. And then, agents, everyone's talking about agents now. And really hear the questions that I hear from leaders who are working with data and AI is, "How do you build intelligent apps that are taking advantage of all of your data, all of your structured data and unstructured data? And with this new concept of multi-agent collaboration, how do you build sophisticated applications that are mirroring human teams where a bunch of agents come together to solve complex tasks?"

Rob Strechay

>> Yeah. No, I think again, we can't go a day without hearing around convergence and how things are going. And I think that really fits those themes, especially with the multi-agent types of aspects of it. Speaking of convergence, I loved all of the next-gen Amazon SageMaker announcements that you had at re:Invent. It was such a great set of announcements. What are, really, organizations seeing when they look at SageMaker as... It used to be, in their perception, all about machine learning. Now, it's really transforming. How are you getting past that and where do you see that going in the future here?

Sirish Chandrasekaran

>> Yeah, you're right. So, we launched SageMaker in November of 2017 as an ML service that enables customers to create, deploy, train models. And hundreds of thousands of customers do exactly that, right? It's a very beloved service. And as we're talking to customers over the last year and two about this convergence trend that I mentioned, what became apparent is that they use a wide range of AWS services across their entire data and AI workflow. Services that are just as beloved as SageMaker. Now, for example, Amazon S3, which is the most popular place to build data lakes. We have customers operating over a million data lakes on S3. Services like Redshift, which offers up to 7X price performance versus alternatives to BI workloads, or services like Glue, where customers run hundreds of millions of ETL jobs every single month. And so, we got to thinking customers love these services, how do we bring them together into a seamless experience to enable this convergence? One place where you get access to all of your data, one place where you can discover and govern all of this data and one place to do all of your jobs to be done, and that is the next generation of Amazon SageMaker.

Rob Strechay

>> Yeah, I think that to me is such a key because when you start to look at redefining the product, it's more of a solution than a service. And I walked away from re:Invent just blown away by that and I think coming into this Pi Day was really excited about that. So, tell us more about what's new and different about it right now.

Sirish Chandrasekaran

>> So, the first thing, what used to be called SageMaker, we now call SageMaker AI. So, all the machine learning capabilities that you love, you still have it. The three new components that we introduced at re:Invent SageMaker, first is the Lakehouse. So, the SageMaker Lakehouse gives you a complete picture on all of your data, whether it's in warehouses, lakes, transactional systems, like Aurora, DynamoDB, SaaS applications on-premises or multiple clouds, and it's all exposed through a standard Apache Iceberg API. So, that's the first, the SageMaker Lakehouse. The second is the SageMaker catalog, which is your nerve center for all your governance needs across data and AI. And the third one is the Unified Studio, which we're making generally available as part of Pi Day, which is the one place, it's a modern development environment for both data and AI for everything you want to do around SQL analytics, data prep, model development, gen AI development, all in one place. So, those are the three components.

Rob Strechay

>> That's great. And I think, again, coming into Pi Day and you made some announcements. What were the announcements around SageMaker that you made? We got into a little bit with Andy with some of the S3 ones. What do you have cooking?

Sirish Chandrasekaran

>> We have two announcements that we're so excited about that we think customers are going to love. The first one is the general availability of SageMaker Unified Studio. So, we previewed it at re:Invent, and we've been hard at work since. It's now available in seven new AWS regions, and we're making it generally available. And I'd say the two things that I'd put a spotlight on as part of this GA announcement, the first is Amazon Q Developer, which is our capable AI assistant, it is infused throughout the experience. So, everything from discovering data, writing SQL queries, creating ETL jobs, building gen AI apps, you can just use natural language and it helps you through every part of the journey. The other one is Bedrock. So, Bedrock was in preview, and what we've done as part of GA is now it's built in to the SageMaker Unified Studio. And so, since preview at re:Invent, we've again been hard at work. So, we now support powerful new models like the Cloud 3.7 Sonnet, DeepSeek R1. We have latency-sensitive inferencing for specific models from Amazon, Meta and Anthropic. And so, we're really excited to bring all this together as part of the studio. So, that's the first big announcement. The second big announcement is that we now have integration between Amazon SageMaker Lakehouse and S3 tables, which Andy talked to you about. So, S3 tables, the first cloud object store, which has native Iceberg support, we think this is super powerful for our customers, where now they can combine all the data on S3 tables, get access through it through the Lakehouse, and combine it with the rest of the enterprise data.

Rob Strechay

>> Yeah, I mean, I think again, looking at it, the momentum has been incredible for Iceberg and lakehouses are all the rage. And why don't we kind of dive a little bit further into that. What's driving the shift towards open-table formats and how does that really affect the lakehouse strategy with SageMaker?

Sirish Chandrasekaran

>> So, two different questions. The shift towards open-table formats and then our strategy. So, to answer the first one, customers are looking to maximize the value that they get out of their investments in data. We talked about how data is your differentiator. And to get the most differentiation, you want all of your data working across all of your tools, no more silos. And open standards offer the way forward for that and that explains open-table formats and Iceberg's emerged as a leader there. In terms of our strategy, our strategy is to meet customers where they are. Data is everywhere. Customers use warehouses like Redshift. Like I said, they have more than a million data lakes on S3. They have data locked into transactional systems, like Aurora, RDS, DynamoDB, Salesforce, on-premises and in clouds. And historically, the approach would have been to build and maintain, extract, transform and load ETL pipelines, which as anyone who's done this, can be very cumbersome. And so what we've done is we have reimagined the lakehouse concept for this AI era. The reimagining really has three characteristics, unified, open, and secure. So, now you do not have to move data from Redshift to S3 or S3 to Redshift. You can expose this data in place and query them in place from any tool you like, any Iceberg-compatible tool, EMR, Redshift, Pi, Iceberg, you name it, in place. If you have data on-premises or in the cloud, again, if you have federated connectors, you can query them in place without having to bring them into the AWS cloud. And third, where you choose to replicate, let's say from an Aurora or Dynamo or Salesforce, we offer something called zero ETL integrations, which think of them as push button workflows where we do all of the work around creating the pipelines for ETL and managing it, and it's just a push-button workflow for you. We have 15 such integrations and more coming.

Rob Strechay

>> Yeah. As somebody who's built some of those integrations and done ETL in the past, I think again, getting back to that and how difficult that is to do and making a push button and seamless and getting it to all those regions is just quite an accomplishment. But let's dive into another one that's a topic that comes up all the time and it's a trend that you mentioned in passing there really around data catalog as the control layer for the data stack. It is something that we're seeing. I'm sure it's something you're seeing. How do you see that trend going?

Sirish Chandrasekaran

>> First, I have to say, catalog is such an overloaded term. And when we talk to customers about what they really mean, a lot of folks think of catalog as just being a technical data catalog. But when you dig into it, what you call the control layer, customers' needs go much beyond just knowing what data is there, what schemas are there, making sure that the compute engines can connect to the data. It's really broader about governance, right? And when you think about governance in this AI era, it's not just a technical catalog. You need gen AI powered creation of rich business metadata that your teams can easily discover, make sense of the data and use it. You need publish-and-subscribe workflows, so that you can easily share. You need to have a more expansive view of what your assets are. It's not just structured and unstructured data. It's also your machine learning models and your gen AI apps. And then, you go one step further, because it's no longer just about auditing, real-time monitoring compliance. It's about how is governance really built into every step of your workflow. So, toxicity detection, bias detection, things like policies for blocking harmful content, preventing hallucination. And that's where capabilities like Bedrock guardrails come into play. You can just specify in natural language what policy you want. And we use a technique called automated reasoning, this is actually very cool, it allows you to have sound mathematical proofs, verifiable proofs that your policies work as intended. And this is a technique we've been using in IAM and S3 for years now, where we use it to make sure that you have a high security and durability bar through millions of scenarios. And now, it's all part of Bedrock, it's part of SageMaker.

Rob Strechay

>> Not only has it been there, as you guys have been saying, but it's all the buzz as well, which is I think great to hear. And there's a lot of buzz around agentic AI. I mean, you can't go anywhere without seeing that. What are you seeing in terms of real-world applications where the rubber meets the road?

Sirish Chandrasekaran

>> You're right. Agents are everywhere. It really promises to change the way we build apps and solve complex problems. And in terms of real examples, I'll give you one, BMW. So, they have used AWS to create something called the Cloud Data Hub. And this is a way for them to cut through all the data silos where you can get terabytes of telemetry data into one place. And now, with agents, they're able to do root cause analysis of issues with up to 85% accuracy. So, engineers don't have to spend so much time digging through the data, trying to figure out what's going on, but agents are able to enable it. And then, you have other examples like Hippocratic AI. Now, they use SageMaker HyperPod to go create agents. This is in assisted-living scenarios. Agents like Judy and Sarah. Judy is an agent that builds relationships with patients, figures out when they need to take their medications. Sarah helps with pre-op and procedures. So, these are real, these are in the field now. Customers are using it and I'm very excited for what the future is going to bring here.

Rob Strechay

>> Yeah, I mean the ROI of AI right there.

Sirish Chandrasekaran

>> Absolutely.

Rob Strechay

>> That's crazy, great... So, your last thought, what are you really excited, as you look out past Pi Day here, what are you excited about with SageMaker? Because you guys are working backwards all the time from customers.

Sirish Chandrasekaran

>> Yeah, we have a saying at Amazon, "It's always day one," right? So, we're just getting started and we are always looking for what customers want and working back from it. Over the rest of the year, we expect to expand the capabilities of SageMaker Unified Studio and SageMaker with capabilities for streaming, BI and search. And then, we just keep going from there.

Rob Strechay

>> Yeah, I love it. I mean, the innovation, I mean, again, having been part of it at one point in time, but when you start to look at it, you guys just keep going and keep rolling and really impressive to see what has happened since re:Invent to this Pi Day, I'm really excited about it. Thanks for coming onboard and sharing with us. And I think a lot of people will get a lot out of this Pi Day presentation, so thank you.

Sirish Chandrasekaran

>> I think so. And thanks for having me.

Rob Strechay

>> Yep. And thank you for watching this special event on digging into AWS Pi Day. Stay tuned for the main show, for which you can find the link below on this screen here. And you can find out more about Amazon S3 and Amazon SageMaker and hear all the announcements and get all the details. Stay tuned and go enjoy.