AWS Generative AI Analyst Summit | Karthik Bharathy, Amazon & Chris Wegmann, Accenture

Clips
More from AWS Generative AI Analyst Summit

Chris Wegmann

Managing Director, Accenture AWS Business Group

Accenture

Karthik Bharathy

Director of Amazon SageMaker

Amazon

Rob Strechay

Dir./Principal Analyst & Host

theCUBE Research

play_circle_outline Exploring Gen AI's Influence on Customer Buying Decisions and Overcoming Responsible AI Challenges in Transition to Production

play_circle_outline Prioritizing Responsible AI Across Industries: Key Role in Public Sector and Life Sciences Compliance

play_circle_outline Gen AI Competency Center and partnership between AWS and Accenture

play_circle_outline Revolutionizing AI Adoption: Accenture and AWS Partnership Delivers Value to Customers with SageMaker and Bedrock

Info
Transcript

Karthik Bharathy, Amazon & Chris Wegmann, Accenture

Chris Wegmann

Managing Director, Accenture AWS Business Group Accenture

Karthik Bharathy

Director of Amazon SageMaker Amazon

Rob Strechay

Dir./Principal Analyst & Host theCUBE Research

HOST

The discussion in Seattle is on responsible AI and its impact on customers. Gen AI, powered by machine learning, allows customers to generate new content and user stories. Challenges include selecting the best foundation model, minimizing costs, and customizing models with their own data. Accenture and AWS collaborate to help customers adopt AI responsibly, with a focus on highly-regulated sectors like the public sector and life sciences. The responsible AI platform powered by AWS assesses readiness, monitors AI models for bias, standards compliance, and ongo... Read more

explore Keep Exploring

What are some challenges customers face when trying to integrate gen AI into their applications? add

What industries do you think will see the widespread use of responsible AI in the future? add

What is the Gen AI Competency Center at Amazon and what services does it provide to customers? add

What are some of the challenges that can be solved by Bedrock and SageMaker and how do they work together? add

bolt Powered by CUBE AI

Karthik Bharathy, Amazon & Chris Wegmann, Accenture

search

Rob Strechay

>> Hello and welcome to theCUBE, live from Seattle, where we're going to be talking about what's going on in AI and customers and how they're thinking about AI. And really, one of the big topics is responsible AI. And today, I'm joined by Karthik Bharathy, who is the director of Amazon SageMaker, and Chris Wegmann, who's the managing director of Accenture AWS Business Group. I've been talking to a lot of different organizations over the past year, and one of the things that has really stopped them from getting from POC to production is really understanding how they can do it responsibly and understand how to get from here to there and make sure that they understand lineage and other things that go along with it. I think one of the things and, Karthik, is really what has been the impact of gen AI on the customers and kind of their buying decisions? And really, how have you been seeing traction within those gen AI products that you guys have?

Karthik Bharathy

>> Yeah, that's a great question, Rob. So, at the outset, gen AI is a type of AI that lets customers generate new content or user stories. And fundamentally, what's driving or powering these gen AI is a model just like a machine learning model, but one that is trained on large number of parameters, on a vast amount of data, and it's known as the foundation model. Now, when customers look at gen AI, they see the potential to integrate that with their applications to offer new services, but they're running onto several challenges. One, they want to be able to leverage the best foundation model for their use case. They also want to do that at the lowest cost without spending a lot of time and setting up the infrastructure and spending a lot of cost. And three, they want to use their data, so that they can customize these foundation models for their use case. So, with machine learning, what we did at AWS was to democratize it and meet customers where they are, and we have over 100,000 customers using machine learning on AWS today. We are taking the same principles, making them available to gen AI, so customers can embrace these gen AI tools and services that AWS has to offer. One of those use cases that we're hearing today is on how customers can build AI applications more responsibly.

Rob Strechay

>> Yeah, and I think that to me is one of the keys that we see is especially when they're using the crown jewels of their data in there. Chris, help us understand where Accenture plays with responsible AI and really your view on responsible AI?

Chris Wegmann

>> Yeah, let's start by talking about what is responsible AI? So, in cloud and in IT all the time we talk about security. Is my data secure and things like that? You got to look at responsible AI much broader than that. It is actually looking at it from an end-to-end perspective. It's looking at it from ensuring that I am not creating bias, I'm meeting all the standards that are in the market, which are changing very, very rapidly, and I can continue to monitor it and see how it goes throughout the process. We see a lot of customers who are going to use multiple models, maybe in the enterprise space, they could have 100 different models. They might be small, they might be large, but you can't just rely on just having a governance board to look at those AI models when they choose them. You've got to look at it throughout the process and how those models change and mature and just go through the learning process. You got to keep looking at that. So, from a responsibility perspective, we look at it end-to-end, we look at it making sure that the developers look at it as they're doing it, as well as all the way through the risk teams and ensuring that there is no bias, there is no impact because AI it's going to impact everything. It's going to impact people, it's going to impact government, it's going to impact everybody. It's going to have an impact and it has to be responsible.

Rob Strechay

>> Yeah, I think that you hit on a hot topic for a lot of people is the whole bias and how things are really challenging from that perspective, based on how the models are trained and things. And like you said, size can vary, from small language models to large language models, but also, the fine-tuning and fine-tuning on the data and what data it gets fine-tuned on. What are some of the key industry verticals you're seeing responsible AI really be a driving force within there that it's critical for them to get from that POC to production?

Chris Wegmann

>> Yeah, I think over time, we're going to see responsible AI in every industry. I don't think there's going to be an industry that it's not going to be applicable for, but as you can imagine, right now in highly-regulatory environments like public sector, like life sciences, that's where it's really starting off. And they are setting a lot of the examples, and they're the first ones that are looking at this stuff end-to-end because they have to. Because whether it's in financial services and there's regulatory requirements, whether it's country requirements. We see the EU having its own standards, we see NIST, we see other standards that are coming out, all that has to be applied. And mainly in the regulatory areas where we're seeing it start.

Rob Strechay

>> Yeah. No, I think that makes sense. In fact, those are some of the, I guess you could say organizations, the public sector in particular where they kind of got passed by digital transformation a little bit, but have really taken a hard look at AI, and so it makes sense that responsible AI. So, Karthik, help us understand partnership between AWS and Accenture and really how this is driving value for the customer.

Karthik Bharathy

>> Yeah, absolutely. So, Accenture and AWS have been collaborating for several years now. And over the last several months, we've been looking at how customers can scale their gen AI adoption more responsibly. So, in August of this year, we announced the launch of the Accenture Responsible AI platform powered by AWS services. What this means is customers can use our AI platform with services like Amazon Bedrock and Amazon SageMaker in order to assess their readiness for responsible AI. You have the expertise from Accenture, along with the robustness and reliability of AWS services coming together to help customers adopt AI.

Rob Strechay

>> Yeah, I think that's key because when you look at it, there's only so many people who understand the full end-to-end nature of that because there are so many personas that are actually play within the AI from data engineering, platform engineering, all the way up to data science. So, Chris, expand on that relationship and why Accenture really built its responsible AI platform powered by AWS.

Chris Wegmann

>> Yeah. So, of course, as Karthik said, we had a long relationship between AWS and Accenture, well over 14 years. Accenture was actually one of AWS's first customers. We started leveraging S3 to store things like our expense reports. So, we've had a long history, same thing for Accenture. We've been using AI in different parts of our business for a while. And the concept of this responsible AI platform came from our own experiences. We obviously have 750,000 people. We operate in almost every major country. We had to find a way to make sure that all these models that are getting created are being created and run in a responsible way. So, the concept of this platform came from that, but it also came from our customers. As we started working and we've implemented thousands of AI use cases, we saw a couple things. We saw, number one, they get stuck. They get stuck going through reviews, getting the risk reviews, the legal reviews, the security reviews, and that's a very manual process in a lot of cases today. So, we went back and looked at the partnership and said, "Okay, we're both doing these POCs, we're doing these AI. Where are we getting stuck and how can we work together to unstick them?" Obviously, Amazon's got the technology leveraging SageMaker and leveraging Bedrock. So, we were able to take that technology, take our approach to responsible AI and put them together, and then take that to market together. So, we both are in thousands of customers working together. We're able to talk to them about how this platform will help them accelerate and get these POCs into production where the value's really going to be driven. And I think it's a great testament to our partnership. We were one of the first AI competency partners for AWS. We've been the launch partner for many, many competencies in AWS, and it allows us to accelerate building these type of platforms.

Rob Strechay

>> Karthik, why don't you touch on the competency and what it's about a little bit?

Karthik Bharathy

>> Oh, yeah. So, Amazon has this Gen AI Competency Center. We started off with 40 partners back in March. Accenture is one of the key partners in that Competency Center. What we do is we help customers be successful with building gen AI and adopting those gen AI solutions across different verticals and use cases, how they can quickly build a minimal-lovable product and embrace it as part of their workflows. That's something we've been doing. We have over 156 public customer references of customers who benefited from the gen AI Competency Center.

Rob Strechay

>> Yeah, and I love MLP. I am a big fan of minimally-lovable versus viable, because you want the customers to love it out of the gate, and maybe that's my background having been here as well. But I think one of the things that's really interesting is, and I think you kind of touched on this, is that there's some key governance, risk, compliance and security considerations that people really have to think about when they're embracing gen AI in particular. And usually, it's not just gen AI, it's gen AI with what we call traditional ML in the back, and there's some interactions there. How does this really get taken into account, that the risk, security, compliance aspects of it from responsible AI platform?

Chris Wegmann

>> So, the platform is very built around the kind of end-to-end steps. So, through design, through development, through execution and reporting. So, I always think about from a coding perspective. I come from a coding background. Once you wrote the logic, the logic was the logic. It didn't change. So, you knew that first time you went through and you looked at it, ran the security checks, did the testing, you knew the answer that was going to come out of the code. Well, with AI, that answer could change over time, and we create things like guardrails. We get really good at doing prompts, so they don't get hallucinations and things like that. But when these things start running through hundreds, if not thousands of transactions, they can change as they get more data and things like that. So, the platform really looks at that ongoing review, making sure that there isn't bias, making sure that it's hitting standards and things like that, and giving the users, whether you're a data scientist, whether you're a chief compliance officer, could be a legal person, it gives them that view they need to take action. And in some cases, they can take the action in that tool and it will go and take the action as needed to get it back into compliance. So, you set the compliances, you leverage the standards that are out there, so you can go to the regulatory agencies and say, "I'm meeting these standards and here's the reporting to do it." And again, Accenture is using this internally. This is how we report to our board and show what we're doing and that we're using AI responsibly.

Rob Strechay

>> Yeah, I think that speaks volumes when you can say, "Hey, we're not only the of this, but we are actually using this," and you're going back to your board. So, Karthik, help the folks at home here to understand where some of the challenges that can be solved by Bedrock and by SageMaker and kind of how that works.

Karthik Bharathy

>> So, if you look at what customers look for, and we work backwards from their needs. They want a choice when they want to select different foundation models. They also want to make sure that they have the infrastructure at lowest costs, like I mentioned. So, if you look at the three-layer gen AI stack, at the lowest layer is the infrastructure innovation, and that's where Amazon SageMaker comes in. It's a fully-managed service for machine learning where customers can build, train, and deploy ML models at scale. You have capabilities like SageMaker HyperPod, which lowers the training time by up to 40%. And you also have capabilities around MLOps where customers can use SageMaker pipelines to have repeatability in their workflows or use managed MLflow when they're running experiments to lower the cost of infrastructure. At the same time, we also offer the latest hardware with EC2 P5 instances. We have in-house accelerators, like AWS Trainium and AWS Inferentia. The next layer of the stack is where customers can offer the choice of foundation models if they don't want to build their own models, but start with what's out there with some of the industry-leading model providers, like Anthropic, or Meta's Llama models, or AI21, or they want to use Cohere models or Mistral models, the whole suite of models that they can choose from. And within a few clicks, directly deploy it in their environment, or they can fine-tune those models and integrate it with their application. Bedrock also offers a suite of tools, like we have agents where customers can compose their orchestration across different services. We also have guardrails, which is very important from the context of responsible AI, where guardrails let customers safeguard their ML applications, look for topics which are denied. How do you handle PII data or how do you handle harmful content or toxic language and so on? There's enough controls built in so customers can use these capabilities with their gen AI apps.

Rob Strechay

>> Yeah, I think to me, it's bringing it all together. Is that where you see the responsible AI platform coming from is that, hey, you have the parts and the pieces. Most people hopefully aren't going out and building their own large language model, unless they're one of those startups. But you would see that they want to fine-tune things, they want to ground it and then fine tune it in that. And is that really where power of this platform is coming in?

Chris Wegmann

>> Yeah, you're exactly right. I mean, it is bringing and leveraging all those foundational services and bringing those and exposing them in a way that users, again, from developers all the way through the C-suite, can understand what's happening in those models as they go, versus just relying on, "Okay, I've got the guardrails in place. I'm good. I look at it once and I'm done." And I think when you do that, it puts you in a box, so to speak. And you don't let that model do and you don't let the AI do what it's good at, is learn and continue to mature and continue to bring new insights with each transaction. So, yeah, that's exactly right. That's the way the platform comes in and brings that all together.

Rob Strechay

>> Yeah, I think that is really key to a lot of organizations who are out there that have struggled with these pieces and how they pull them together and how they do it responsibly, because, like you said, they have to go to their boards where they have even more. So, they have to be able to explain it to their customers and be able to be, "Hey, this is how we do this in a safe and responsible way," and really do that. Well, I want to thank you both for coming on board today and helping... From our AWS studios here in Seattle, I really want to thank you for being on board.

Chris Wegmann

>> Thanks for having us.

Karthik Bharathy

>> Sure.

Rob Strechay

>> And thank you for watching this episode of theCUBE, live from Seattle. We'll be back with more, so stay tuned.