Name: Igor Sedukhin, AWS
Uploaded: 2024-10-16T18:16:00.000Z
Duration: 20 min 32 s

Clips
More from AWS Generative AI Analyst Summit

Igor Sedukhin

General Manager of Application Observability

AWS

play_circle_outline AWS observability solutions: CloudWatch and managed open source suite for observability.

play_circle_outline AI playing a role in summarizing and extracting insights from observability data.

play_circle_outline Providing observability tools for different personas in the application stack.

play_circle_outline Integration and collaboration of different personas in addressing and resolving problems.

play_circle_outline AWS's strategy around open source observability projects like Cortex and OpenSearch.

Info
Transcript

Igor Sedukhin, AWS

Igor Sedukhin

General Manager of Application Observability AWS

AWS offers observability solutions through CloudWatch and a managed open source suite using OpenSearch, Prometheus, and Grafana with OpenTelemetry. The goal is to improve application reliability and manageability. AWS provides tools for different user roles and a unified view of application performance. Collaboration and cost analysis are facilitated. AWS contributes to open source projects like Cortex and drives OpenSearch development. The focus is on meeting customer needs and providing a platform for running open source tools effectively.

explore Keep Exploring

What observability solutions does AWS offer for customers with different needs? add

What is the benefit of standardization in LLMs for extracting insights and summarizing data? add

What is the importance of observability in the integrated life cycle of using AWS or any cloud platform? add

What kind of tools are built for different personas in order to address and collaborate on problems, and is there a point where recommendations are made or AI will play a role in that process? add

What is the importance of open source in our business strategy and how do we prioritize customer satisfaction when it comes to collaborating on open source projects? add

bolt Powered by CUBE AI

Igor Sedukhin, AWS

Dave Vellante

>> Hi everybody. Welcome back to our special CUBE presentation here on the ground in Seattle. My name is Dave Vellante. I'm here with my co-host, Rob Strechay just digging in into some of the innovations at AWS and we're really excited to have Igor Sadukin here. He's the general manager of application observability at AWS, hot topic. Igor, welcome. Good to have you.

Igor Sedukhin

>> Thank you Dave, Rob, excited to be here.

Dave Vellante

>> So observability has sort of come out and mainstreamed the last couple of years. People could know it as the old days application performance management, but we're talking now about a much wider scope of capabilities. So help us understand how AWS thinks about observability. What's the sphere that you operate in?

Igor Sedukhin

>> Great. At AWS, we care a lot about what our customers need. And as a global provider for cloud, we have a lot of different customers and their needs are very diverse. So what we do, what we provide as observability solutions is twofold. One is for customers that need native software as a service, pre-built observability solution, we have CloudWatch. And so that provides everything from logs to metrics to traces to synthetics, to real user monitoring, to many other things including integrated suite for application observability. So that's out of the box, it supports AWS services, supports any environment that is outside of AWS as well. And so that's pre-built solution. And for customers, for other set of customers that actually want a full on open source observability, we offer managed open source suite for observability. That consists of OpenSearch for logs and Prometheus for metrics and then Grafana, mainly Grafana for the interface. All of these solutions, either of them are actually rooted and enabled through OpenTemetry and that's what makes it perfect because from a customer perspective, you instrument ones, it can benefit from either of.

Rob Strechay

>> Do you see OTEL as being kind of that getting a center of gravity because there's been a lot of talk in the open community about various different things like we were talking about with Prometheus and others and there's a whole litany of ones that are going to be talked about at KubeCon in a couple of weeks. Is OTEL really one where the community has really focused on because developers like it and can instrument with it pretty easily?

Igor Sedukhin

>> That's a great and exciting topic. I'm super excited about OpenTemetry. I'll give you maybe an example from a long past. Probably 30 years ago, SMMP was introduced, simple network management protocol and it really made our networks reliable because power of standardization is hard to underestimate. It was really hard to make boundary conditions between variety of network vendors to work. And so the networks were just inherently unreliable and unmanageable. And so with this NMP, we're able to make our global networks very reliable. And I think that's the same transformation that's going to happen through OpenTemetry that now we're going to make applications reliable at the same pace as we did with networks. It took a little while. It took a little while for all the devices to start to support SNMP and actually play. And so I think we're going through that same transformation.

Dave Vellante

>> So can you explain that a little bit because you're right. I remember the days of DECnet and IBM SNA and everything was sort of incompatible and so are you saying that there's an analogy with applications, why? Because the applications have locked inside of them, all the data and the metadata and it's a bunch of automation stovepipes?

Igor Sedukhin

>> Yeah.

Dave Vellante

>> Carry that through and explain how you address.

Igor Sedukhin

>> I think there's that power of standardization where there's a common way to observe an application for example. There's a common way to record when there's an order is being placed and what do I need to know? A customer ID. I need to know where it came from. Maybe I need to know the API name, maybe I need to know what service it came from. And so the moment you can standardize a lot of those practices, then engineers who build applications can benefit from that and they can do their operational tasks in a very common way and not having to worry about customization, which is a killer. The diversity of these customizations is what makes people unproductive. And so in this case, if you just solve that part the same way as this NMP solve a simplest way to observe the devices up or down, we can do this for applications.

Dave Vellante

>> So that goes to the source though, the main spring of the problem. It's not like you're harmonizing all that disparate data at the tail end. It's what guidelines for engineers essentially to write to the standards and that fundamentally solves the problem.

Igor Sedukhin

>> Correct. And a lot of it is also through the OpenTemetry community effort. A lot of it is enabled out of the box and the engineers don't even have to know in many cases, they just embed libraries that auto instrument and then adhere to the standard and then provide the common pipe and the common way that this data is sent and so can be sent to anywhere. And so even in those networking days, it didn't matter what vendor you're using to monitor your device, as long as you can establish common grounds of interoperability and making it all reliable. Here at AWS, we're all about making applications reliable for our customers. And so that's why it's such a key strategy for us.

Dave Vellante

>> What does that mean for customers? It means the performance is more predictable, the recovery time is better, doesn't go down as much. What does that mean for the customer?

Igor Sedukhin

>> So simple things like knowing what your latency is of placing an order. Something as simple can now be much more automated and much more reliable and much more common practice that any engineer that runs an application or a service can simply leverage instead of inventing their own. And then moreover, as engineers change teams and they go work on different services, the more common it is, the easier it is to manage elsewhere. And so this is leverageable skill, so you're not stuck in any particular dynamic of certain solution. You can go elsewhere and be as productive.

Rob Strechay

>> And I think one of the interesting things, and you kind of hit on it, OTEL is a good place to focus having been there with SNMP, but then you had Armand and other things that came along to give you different levels of detail similar to logs, metrics and traces. How do you see GenAI playing in this game? Because again, a lot of people are saying, yeah, great, I go and instrument these applications, these massive scale out, scale up, then scale back down type applications. But I'm overwhelmed with the amount of data that I'm actually bringing in. Where is GenAI or AI playing in this space for you guys?

Igor Sedukhin

>> So where we see a lot of benefit is summarization and just extracting interesting insights of all this data. And that's where by the way, a lot of the standardization also helps because it's an open standard so LLMs understand and know about it. And so it's much easier to make certain conclusions and interesting summarizations. So that's where we see a lot of it playing the role in just exposing this information in a much easier to consume form without you having as a customer, dig through the data manually yourself. Those insights can be very easily extracted and presented.

Dave Vellante

>> But as you think about how AI is going to affect the application stack of now the big buzzword is agents and not just single agents or copilot, multiple agents, swarms of agents that I can only imagine what's that's going to do to your challenges. How are you guys thinking about that and is the plumbing there to handle that, how will that evolve or will it evolve?

Igor Sedukhin

>> So we see a lot of customers that come to AWS to build GenAI-enabled applications. So you're leveraging Bedrock, SageMaker and so on. And we offer a lot of observability solutions. Why they're important is if a customer builds an application that leverages particular set of models, they probably want to know how well they respond with what latency because it will affect their end users. And so we offer solutions within CloudWatch for example, to observe behavior of different models. If you use them in your application using AWS SDK, you're making call to say Bedrock, to leverage a particular model to evaluate an interesting result you want to return to a customer. We also offer a set of observability solutions for people who use their own models at the bottom layer where a lot of the GPU visibility is very important, especially into cost and performance of those aspects when you're running your models inside of, for example EKS cluster. So we offer visibility solutions for that.

Rob Strechay

>> So it's helping unpack the entire stack and also looking at how it plugs into the application. Most of the time an agent is not going to be standalone. Most people are not putting out ChatGPT competitor or something like that. They're building into customer support application that's on a mobile or something like that. So are you providing solutions that actually help understand, because there can be drift, there can be hallucinations. How far into that stack do you currently go today?

Igor Sedukhin

>> So we generally provide solutions for observing the way these models are used inside of the application. So for example, in CloudWatch application signals we released support for analyzing how well the customer that wrote some code that leverages a particular model, how well that model is responding. And a customer can get visibility down to what sort of responses are getting generated and what is their temperature, what is their latency. As you said, it's all going to come down to is my customer at the end happy? And if the customer is receiving slow answers, then they're probably not going to be as happy. And so this is a tool that many customers can use to evaluate and build better GenAI-enabled application.

Rob Strechay

>> Because I think you just hit on one thing that I find very interesting is how hot is it? Or you're talking about how really accurate it's being or is it coming back for retries or somebody retrying the same prompt so you can actually understand and you're looking at prompts and being able to evaluate that.

Igor Sedukhin

>> Exactly. And so that's how you gain better visibility into how you're using your LLM models or underlying models and whether they're doing the job.

Rob Strechay

>> How does this, because that's super exciting stuff because it actually talks to different personas. You're not talking to the app developer necessarily that is the full stack engineer or something like that. You're talking maybe even to data scientists who are having to give out. But how do you handle providing observability tools to people of different personas in that way?

Igor Sedukhin

>> Yeah, that's a very interesting aspect in that generally observability is part of the integrated life cycle. So when you come to AWS or any cloud, you are going to create an environment, you're going to do some amount of governance to set up security policies and they're going to develop applications and then you're going to put them in production and then you're going to observe them and you're going to secure them. So we've spent a lot of time listening to customers on that aspect. So we understand that ultimately AWS is the product, not individual services. This is the product that you use and it has to be an integrated life cycle. So couple of examples where we heard customers and actually launched features to help with that. So number one, last reinvent, we launched a feature called myApplication. So this is a common AWS context to unify understanding of what application is. Here's my payment application that I care about. And in that integrated experience you can see how much it costs to run that application, how secure it is, is it healthy, what is the current latency of responses of that application? How many recent deployments happened for that application. So you can see a unified view of an integrated set of concerns that you would have and that normal life cycle with an application.

Rob Strechay

>> Oh, go ahead. So yeah, that's fascinating because I think one of the FitOps aspect of it has become a very hot topic for a lot of organizations. And knowing a little bit about the AWS services, does that plug back into something like Cost Explorer so that the people in the finance department who are probably only going into Cost Explorer, they're not hopefully going through all of concept. And how does that really work for that?

Igor Sedukhin

>> That's the beauty of it because in this integrated it does. So all the information comes from Cost Explorer, it just gets sliced so that it gets presented with respect to that application, not your entire account as an example. And so as a customer you can sort of finesse that understanding of what that application is and then present it in the right manner. However, a different persona that for example may be worried about your account costs or your governance can go to an individual service and get the same perspective from a different angle. And so this allows variety of personas to interoperate and collaborate in AWS to achieve sort of a better business outcome.

Rob Strechay

>> When you take that other approach and you take the service down approach, can I go in there and say here's my hottest from a cost perspective application solution, what do you call it? myApp?

Igor Sedukhin

>> MyApplication.

Rob Strechay

>> Your myApplication, here's the finance, myApplication, GenAI doing my 10K work at the end of the quarter and it spikes and I see it spike every last month or quarter and then it comes down and it spikes and comes down. Is that how somebody could actually go in and see it from in the EKS? I want to see what are the hottest myApplications on EKS?

Igor Sedukhin

>> Yep. And so that's the idea. I mean we're still on the journey to integrate it more deeply in AWS, but for the services that we've been able to integrate, it provides both of those perspectives. So one is when you log into AWS console, right in the home page you can see here's my applications, and click on each one and get that integrated perspective. But you can also go to individual services like security or cost and understand the items that relate to that application in reverse.

Dave Vellante

>> So when there is a problem, you're obviously providing the data to observe that, visualize it, et cetera. What happens when there's a problem? Who addresses it? Maybe it's different personas, they maybe collaborate. You're feeding that information. Do you see a point where or are you making recommendations as to what to do? Will AI at some point do that?

Igor Sedukhin

>> So there's the variety of tools are built for different personas to do their job. Security engineer will go look at Security Hub and understand whether there's any vulnerability. Ultimately they're built for individual personas, but they all have to collaborate. And then there has to be some rendezvous points where they can meet and share information that would be relevant to each other. So my applications is one, typically the aspects of observability is another. So what metrics do I have? Are they telling what do we have in common? We have a problem where our orders are not getting processed and that starts the investigation and it can go in variety of aspects of your lifecycle to be able to resolve it. So we try to provide those journeys and connect them as much as possible. A customer who sees an alarm somewhere relative to the orders may actually go and have to increase capacity and therefore be concerned about the cost as an example. Another good example of an integrated lifecycle of that sort is it's not all about firefighting. A lot of the engineering that the customers do in AWS is about kind of fireproofing their applications. So here's one example. So we've published a well-architected framework that goes ad-dubs in variety of aspects of how to be more resilient. Here's one of them that we do internally as well. You can do a lot of injected faults, for example, simulate a region value and then use observability to understand the impact of that on your application. And so that's where a lot of the tools that we build, they kind of interplay and they complete the full lifecycle that way.

Dave Vellante

>> So a bit of a what-is, and you can sort of simulate that. Are you-

Rob Strechay

>> Yeah, last question is that, again, we talked about OTEL, but I have a feeling that's not your only open source strategy from that and open source is very big with this, especially in that community and the Dev community and Kubernetes and things of that nature. What really is your strategy from an open source perspective around observability?

Igor Sedukhin

>> So we love open source, right? So as you said, it's not just opened to one. It's a key element that we bet on across the board, but it's not just one. For example, last year we were number one contributor to the Cortex project, which is a multi-tenant engine behind the Prometheus. We are driving Open Search. Open Search is a big project. We're investing a lot of resources to make it awesome. And so to us it all comes down to customers. If a customer wants to have open source and wants to collaborate with us on it, we want to make sure those customers are provided for and they're happy. And AWS is the best place to run any open source. That's ultimately where it comes down to is what customer choice and whether AWS is the best place for it.

Rob Strechay

>> Igor, thanks so much for coming on. I'd love to have you back and dig into this further.

Igor Sedukhin

>> Absolutely. Thank you.

Rob Strechay

>> You're very welcome.

Dave Vellante

>> Okay, this is Dave Vellante for Rob Strechay. We're here on the ground of beautiful Seattle. The weather is awesome. We're going to go to the football game tomorrow night. Keep it right there. We got more action from The CUBE right back.