SAS Innovate 2024 | Bryan Harris & Marinela Profi | SAS

Clips
News
More from SAS Innovate 2024

Marinela Profi

Product Marketing Manager for AI

SAS

Beyond the hype: Navigating the generative AI ecosystem

As the world eagerly anticipates the next frontier in technology, SAS Institute Inc. unveiled its vision for generative AI and its transformative potential at this week’s SAS Innovate event.The journey toward AI maturity is fraught with complexities, but the evolving landscape of artificial intelligence is set to have a profound impact on businesses worldwide.“Generative AI is this new thing,” said Bryan Harris (pictured, left), chief technology officer of SAS. “It’s actually a subset of the larger AI umbrella. The problems being solved by the rest of the AI space are much larger than the problems being solved by the LLM and generative AI space

play_circle_outline Harnessing Generative AI for Business Growth: Understanding Real-World Problems and Delivering Value

play_circle_outline Georgia Pacific case study and use of SAS Viya in generative AI workflow

play_circle_outline Empowering Enterprises with Viya as an Orchestrator: Harnessing SAS API for Quantitative Decision Making

play_circle_outline Advancing AI: Explorable Models, Secured Systems, and Purposeful Solutions for Productivity and Real-World Value

Info

Bryan Harris & Marinela Profi | SAS

Marinela Profi

Product Marketing Manager for AI SAS

search

>> Welcome back, everyone, to theCUBE's live coverage in Las Vegas. I'm John Furrier, host of theCUBE with Dave Vellante. We're here with SAS at SAS Innovate 2024. We're back at two great guests, Bryan Harris, chief technology officer at SAS. Marinela Profi, global product marketing strategy for AI and GenAI, both stage performers today on the keynote. Bryan, Marinela, thanks for coming on theCUBE again. Good to

>> see you.Hey, guys.Thanks for having us.How you doing? Great to be here.

>> So we were just at your event last year. You said some things, put some things out there, "This is what we're going to do with AI, generative AI." It was all the hot rage. ChatGPT really innovates on the user interface check, we see that. Getting things done is a whole nother ballgame. You guys got some huge announcements. The models, I love it. Let's just get into it. What's going on right now? What's the key things for your transformative technology around GenAI?

>> Well, let's just get to generative AI out of the gate 'cause everyone's talking about it, huffing and puffing, which is, again, it's a really exciting technology, but it has to be put in context to actual business outcomes. And so I think, for me, I'd like to actually give it over to Marinela a little bit because a day-to-day data scientist out there, and on the ground, meeting with customers, hearing the feedback. So Marinela, why don't you talk about about what you're seeing on the ground.Absolutely.

>> Thanks, Bryan. Yeah, so from my perspective, I'm a data scientist, and I think that the number one lesson that I've learned, if there is a lesson that I've learned, well the past three and a half, has been LLMs, large language models, alone do not solve business problems. The technology is so cool. And no matter how cool it is, it's still imperfect, and it's all going to come down to solving real world business problems for specific industries and specific people, right? We're seeing a lot of companies just saying, "We have this technology. Let's go and build it. Let's . Here's how we're going to do it." So a lot it's around how do we build something, but that there is very little around why. Why are we even building that? Who's going to serve? What's the value that's going to deliver? Right? And so for me, from my perspective, what I've been seeing is that generative AI and large language models in this case are like the 20% of the entire value chain if you consider a real world use case. And I think what Bryan shared today with Georgia Pacific was an incredible example of that. I don't know if you want

>> to add more.Yeah, definitely. Case and point that Georgia Pacific is on stage talking about their use of SAS Viya as part of our generative AI workflow. There is a lot of moving parts. There is no more kind validated feedback on a production use case of GenAI when they're actually trying to run their plant. Their plant operations is leveraging generative AI. And as part of that, they're getting corrective actions through this generative AI experience that is leveraging SAS Viya to orchestrate across LLMs, orchestrate across quantitative reasoning needs for... We're interacting with enterprise systems. So all of this orchestration is happening, ultimately to give a human language response back to an operator who is three, five years experienced their job, and they're trying to convert them and upscale them to a five to seven year employee.

>> There were a lot of components to that stack.

>> Yes.

>> And you think you got a picture of it here. You had Bedrock on there, you had SageMaker, you had SAS, you had another couple other pieces. And so you see GenAI as the orchestrator

>> for all that?Yeah. For us, at some point, you got to integrate all of the tech, and what you're really getting to is you're making a final decision to give a response back to a user. And as we know, a lot of these cases right now are very shallow. They're just prompt engineering to an LLM and come back. And yes, RAG architectures are there, but we got to get things done in sub-second response time here. So there's very, very challenging integration and expectations of responses from GenAI workflows that we have to get done. So for us, Viya sits at the center of that. And even Sam Coyne mentioned it today that we really bring all of these things together. We can augment prompts, we can orchestrate against other traditional models and things like that, and bring that data back in so that the LLM can reason over that and send back the response back to the user.

>> And answering to your question, I think Viya is orchestrated here, not generative AI. Generative AI is a piece that can augment existing processes, and Viya, our data AI platform, functions as that orchestration of the entire decisioning flow. And that's exactly what we are announcing today. We've extended Viya to allow developers to use our API to govern, orchestrate, and build real-world use cases with GenAI.And

>> I think the point about GenAI is an element of the stack, is at the higher end the stack. I have a picture. I want to just get your reaction. Okay, you built on AWS, so got some hardware, got some cloud.

>> There you go.SAS

>> intelligent decisioning, that's your layer, right?

>> Yeah.

>> That's the-Decisioning system.... decisioning systems, and then multiple real-time data knowledge, bank process model. Okay, that's the meat and potatoes. That's the stuff that's the key jewels to make that thing work. And then you've got the GenAI on top managing it, and then you got the outcomes.

>> Yeah, so you think of it as that something has to bring all these things together, because right now, a lot of the reasoning capabilities in text for LLM is around textual fact-based reason responses, but a lot of the decisions in the enterprise are quantitative. So that's why we pride ourselves on being the quantitative reasoning engine inside of GenAI workflows. And intelligent decisioning is such a flagship product for us because it allows us to... We can bring four or five different models to actually integrate across text across the LLM itself, across the actual enterprise systems we're integrating with.

>> I want to connect it to something you said in the media briefing, if I could, which is about ROI. If you look at how people are using GenAI today, it's very ChatGPT-ish.

>> Yes.

>> So it's not literally blowing away the cash flows. It's not really paying for itself. And so when you start talking about these industry-specific models, that's where you're going to get return. And people are pushing out their ROI expectations based on the surveys that we have. And so they're being a little bit more conservative because saying, "Okay, this is fun, this is cool, but we need real dollars."When

>> you open up a prompt to thousands of people in your organization, the quality of your questions is now your biggest concern, right? Now, people are paying for this. So now, okay, well, that's why you saw us talking about stored prompts because our ability to say, "All right, look..." There's not an infinite set of questions like in the public domain with ChatGPT, right? In an enterprise, there's a finite set of questions. So why don't we understand those questions you want to ask, make an inventory of those, use Viya to do the semantic analysis on this to then say, when three, four people ask a question, we're going to get the question that the right question you're asking, and then let that then reduce the error to the LLM, reduce the error to the vector database, reduce the error to the quantitative reasoning with Viya

>> as well?I thought that was a very nuanced point. I thought one of the things that jumped out at me was the stored prompts because Dr. Goodrich was actually bragging about the compiler versus the interpreter on stage. If you think about that, as you get smarter with these stored prompts, you can learn, and that's like compiling away the LLM risk.

>> What's funny about it is if you remember, what do database providers want to do? They wanted stored procedures, and stored procedures was their way to govern the way to build optimized queries for databases. So stored prompts are really just an extension of that entire model.

>> People are doing this in all kinds of use cases to get smarter. You make a prompt, you get an answer, you come back. It spins back down to the architecture. Why I liked that graph and why I called it out was that's a production workload.

>> Yes.

>> Correct.

>> The only thing that's outside there is the customer's environment and Amazon and your SAS stuff. So that brings up the big picture here. You got training and inference is the big discussion on how to deal with data, and then you got prompt response, and then you've got reasoning in the AI paradigm. Okay? Architectural and behavioral. So you're essentially rolling all that up. So okay, prompt response, that could be real, like IO. Reasoning could take a little longer maybe, or... So I got to run that somewhere. How do you see that applying to known end-to-end workloads. Because this is the theme that's popping into this AI world and the enterprises. I got known workloads. I can scope them. I can qualify them.

>> I think the way it rolls up is that we're seeing... Definitely the agent design pattern is emerging. In fact, in our Viya Copilot demo that Marinela was showing today, we're actually doing some iterative work where we have one generating the code, right? One part of the orchestration is generate the code underneath it, execute the code. If it fails, tell the LLM failed. It'll make the corrective action on it and execute it again. And then we'll get the actual-

>> No human.

>> No human in that one. And then it's why you saw a map come on the screen, or you saw some of the extra follow-on visuals or suggestions. So that's why this orchestration is so important, because it really is an iterative process. It's not a single request, single response experience. And so for us, this reasoning thing, if you think of reasoning, reasoning has to span text-based, fact-based things, media like images and video. But most importantly, in the enterprise, quantitative reasoning is really the majority of the workload challenge. It's not about just... Customer service, yes. I got a knowledge base. I can give you answers on it.

>> Right.But

>> a lot of the enterprise class problems are, how do I look at numbers, analyze numbers, and make a recommendation on our next best actions where-And

>> that's why I asked that dumb question, in the media thing.

>> That was not

>> a dumb question, by the way.Not a dumb question.That wasn't a dumb question.

>> Well, I was trying to get... Came out

>> wrong, but-I've seen a lot dumber questions than that. I'm usually the for dumb questions.

>> I got a whole bag of them over

>> here.But the point is customers today have to go to different models for different modals. So they want one place, they want to have the reasoning and all this, it sounds complicated, but put it in one place. You can do vector embeds for unstructured data. I can bring a table in. I can bring in a DNA sequence. That's a mode, right? All kinds of data can be structured that way if it's in one place, I don't want to go five different places.

>> And then do you have the technology? I think that you can talk about... Why don't you give the example of the banking one we talked about today,-

>> Correct. Yeah....

>> where you can really talk about all the tasks that were required to really make it work?

>> Yeah. So this is one example that we were working with a global bank where they were doing complaints management, right? It's a simple use case. It's not revolutionary, but the question was, how can I augment what I'm doing with generative AI, with large-language models? And the integration of the LLM is not just easy as, oh, give ChatGPT a complaint and generate a summary. And then based on that, it'll ask to generate a reply to the customer and send that reply to the customer. That's not how it actually would work because you have so many other pieces in the middle. And in this case, the flow that the developers were able to build to actually productionalize that use case involved other things like, okay, the summary is generated by the LLM, but how do you explain if that's accurate? How do you explain how you generate that summary? And so that's where natural language processing techniques, which is actually a that has been proving to be really critical and fundamental in explaining some of these, the output generated by large-language models, that. And then how do you integrate your customer data within that summary? The LLM doesn't know if your customer has a high credit score, is a good payer, is not a good payer. Those are your enterprise data. The LLM... Unless you want to give that to the LLM and expose them to the LLM, which I don't think that's what our customers are asking to, you need that quantitative decision engine that augments that information and sends back that information for the LLM to actually generate that summary and generate that email in a more reasoned and a way that makes sense.

>> If you think about it, we compress all the complexity of the quantitative side into a much more consumable fact that could be presented back to the LLM, so we can use strength, which is to generate natural language

>> response.Which that might be a score, for example?

>> Yeah.

>> Okay. And then the

>> LLM-Think of it as a score.... will

>> take that score as opposed to trying to generate the score, which it's not good at.

>> Right.

>> It's actually horrible at that.

>> Exactly.

>> That's actually horrible.You

>> said it well.

>> Right.I

>> would say it exactly that way. It's Terrible.

>> Dave is He's a SAS user in our actually used to program in SAS

>> back in that day.We appreciate it.Back

>> in the day.

>> born Those

>> about the proc, we salute you.

>> And if your customer is complaining about, your credit service, the LLM is not going to be able to generate the next back action for your specific customer because it doesn't know your customer. It doesn't know what a next back action is.

>> Since you up customer, I want to get more use case information, what you guys see in the marketplace. As Dave and I always talk when Dave does some research on the monetization, because that's really going to be scoreboard, is it adopted in production, which you guys have shown some examples of great others and the monetization of AIC, the early adopters, some are monitoring really well, so that's also a tell sign. Monetization, uptake, and then customer success. So you mentioned you're doing well with the business performance, and you spent a billion dollars of investment and spending more on the tech. So you've got tech investment and monetization. Good job. What are the unit economics look like? What's the use cases that are emergent you can point to and say we're starting to see the things line up in the market. The fog's lifting in the enterprise. What is it out there, and how is it being Beyond

>> the chit-chat, right?

>> Yeah, I'll get there. I would say that we hear a lot of customers worried about yet another metered challenge to the business. There are CFOs saying, "Look, we're trying to run a profitable business." Look, if you can tolerate metered billing when you know you've got enough room on the financials to tolerate it. But when you start to... Remember, we're lowering the barrier to interaction for data and AI across the entire enterprise. When you do that, you're actually encouraging usage of these metered services, which is fine, but you have to make sure that the use cases are driving an ROI that can support those metered interactions. And then we have other customers saying, "Can you take me off the meters? Can we just run..."It's

>> my data. I want to bulk upload my... It's my own data.

>> Yeah, exactly. So can I... And we want to respect all of those, right? Many customers who... We're partnered with Microsoft. We want them to use, leverage their OpenAI services where possible we're working with Bedrock and Amazon, and AWS will do that. But there are customers who are like, "Hey, I want to run with minstrel. I want to go and I want to run that in a GPU that's hosted in their environment." So the same cloud spend conversation that was really-

>> So they want optionality, but they want to have confidence on what the spend is going to look like. They need to-

>> Yeah.

>> Absolutely.... have some scope.

>> Yes, because what's happening is everyone... Again, the FOMO that was happening around just go run to this thing is now being realized in the financial outcomes of these businesses.

>> These are big customers? Can they get GPUs? You're not spending 10 million bucks on video.

>> No, I think when you're training models, it's one thing. If you're just looking to go and do the hosting them from scoring these models, I think there's options there. There's a lot of things coming down in size on it. Now, we're just going to make sure that our stuff can react to any one of these scenarios. If we want to use a cloud service provider, we're happy to do it.Right.Some of the customers just want... That's what they want to do, so we'll leverage it. But we're seeing others now starting to say, "Let's bring it locally.

>> "Is it tracking like DevOps? Did you get a use case and you experiment with something, or is it end-to-end workflows that are emerging? What are some of the

>> Or is it the data is on-

>> I think first and , I think there's a little bit of a hammer and nail problem where everyone's saying, "Let's start with generative AI" when really-

>> Correct.... generative

>> AI is a subset of AI. It's not a superset of AI, which is a big problem because it's been presented in the market as if generative AI is this new thing. It's actually a subset of the larger AI umbrella. In fact, the problems being solved by the rest of the AI space are much larger than the problems being solved by the LLM and generative AI space.

>> Some of the best models are small.

>> Yeah, yeah, exactly.

>> I think that's going to be the future where the smaller

>> Yeah, small language models. Yeah.

>> Yeah, smaller language models that would say almost auto-fine tune themselves for specific industries

>> and specific use cases are going to be-And that's because they more accurate and faster or-

>> I think it's like anything else. As you go to the public... The large language models that are out in the public are just, they're generalists in these things, and they're powerful for many reasons.

>> But they get vanilla.

>> Right. Exactly. If you want to use an example here, and I... Google was trying to... Remember when Google was so huge, they wanted to get into the enterprise space, right? How can we put that Google engine? People were going home and they were having a Google experience for search, and they would go into the business and say, "Why can't I have that same search experience in my business?" The use case was different, right? Google's indexing engine was about finding information authorities. Meanwhile, in the business, it was about quantitative analysis. So they never could really penetrate that market. We're having the same challenge now with generative AI, which is that you can have a very public domain question and answer experience with LLMs that are very powerful, and you want that capability on the enterprise, but there's more work to be done. And what people are realizing now is how much more work there is to be done. Especially for us, we're in regulated industries, so we can't be wrong. So I can't just go and run-

>> Can't wing it.

>> Yeah, can't wing it. We've got to go and work meticulously with them to understand here's what happens when you do this response. And if that response gives an answer and you need proof of it-

>> Yeah, there's

>> no space for hype.... we need lineage. You need all the things that go behind it.

>> No space for hype for us. What I would add is also that... My question is, until what point this gigantic large language models providers are actually realistically going to have data to keep training these huge models?

>> Yeah, meaning they're going to run out of data.

>> They are.I think so.It's call strip-mining-I think so.And either it's because it's going to become a legal issue at the point where you can't take my data anymore to try and whatever you want to train, or there's just not more data so we have to go back to synthetic data

>> Can you believe you were

>> saying that? I know.Adobe's charging $7, I think a minute for the video, right?

>> And that's why data Maker for us is the big announcement.

>> Correct.

>> Because SAS Data Maker for us allows us to attack the gap in structured data for synthetic data. And by being able to do that, we can go into regulated spaces and give proof to the statistical congruence of synthetic data that can bootstrap models that will allow them to run effectively on real-world data.

>> You got to have explainable AI, table stakes, and security built in. Great to have you guys on. We're out of time.

>> Got

>> to go?Final question. They got the hard stop.

>> I was enjoying this. Wow.

>> I'll give you one more question then. We got one more.

>> Maybe two.

>> So next, great step up from last event... Thank you for bringing all the goods to the table. Next year when we're sitting here, what are we going to be talking about? What's going to be the big discussion? Bryan, we'll need share your thoughts. We'll start with Bryan, and we'll

>> close this out.I think that what we're about to achieve with App Factory, still we're building our own solutions with it, but the productivity of that thing is incredible. And we started wrapping some generative AI experiences on building applications that are... The treatment on that's going to be credible. I think what you're going to see on our Viya Copilot's work that we've just done and demonstrated today, the ability for our Viya Copilot's to be a very practical approach to the AI life cycle is... I think it's going to be world-class and second to none. So that's where I think we're at.

>> What I am really excited about is to continue... So the strategy that we have taken over the past two year and a half, two years, is building something with a purpose. And I'm really excited to come and see that come in action. We're not just a startup that is going around looking for funds from a VC. We're building these things for actually being used in the world and creating real world value. And I can't wait for that to continue to be and have even more... I want to come here next year and talk about other 10 customers that we've done, and all the models that we're going to sell. So that, to me, what I'm really excited about.

>> That billion dollars in investment, you're monetizing well, which is proof. And you got production workloads. Just keep it coming. Yes.

>> Yes. And I would say too, we will have a significant amount of progress on penetrating the market with selling our models, which we're really excited about.

>> I got to ask one more question. We got time.

>> Ask for forgiveness. We're good.

>> You said in the media briefing, nobody's looking at the impact of AI the way SAS is. And today, it's very clear. You're talking about very focused on solving a lot of business problems. How far out are you looking?

>> What do you mean by that?

>> In terms of the impact of AI. AGI, if we have AGI by the end of the decade, are you thinking-

>> Robots....

>> that far out? And then if you think about exponential growth of computational power, mid-century, when you make that statement, I have to ask you literally how far out are you thinking? Is it more-I

>> think we'll have tools that we'll be able to evaluate... We'll have capabilities that can evaluate the reality of these models. Some of these models are self-fulfilling validations where it's like there's no real... They're not deterministic, right? So there's certain kind of workloads we put against them. We try to measure them on grading these things, but they're not measured on the enterprise's actual workload. So how do we think about what does the answer look like in an enterprise, and how did that model react to it? And then being able to go and say, "That worked, that didn't work," it starts bringing some true in-customer validation endpoints to, what are we doing with generative AI? So I think that's the area where you'll start seeing these things will have to be validated independently of the model creators.

>> Well, the stars of the show, you're at high demand. You got to go. We're getting the hook. You got to leave it there. Okay, we're going to continue the conversation here on theCUBE. We'll be right back after the short break. I'm John Furrier with Dave Vellante. We'll be right back.

Bryan Harris & Marinela Profi | SAS

search

>> Welcome back, everyone, to theCUBE's live coverage in Las Vegas. I'm John Furrier, host of theCUBE with Dave Vellante. We're here with SAS at SAS Innovate 2024. We're back at two great guests, Bryan Harris, chief technology officer at SAS. Marinela Profi, global product marketing strategy for AI and GenAI, both stage performers today on the keynote. Bryan, Marinela, thanks for coming on theCUBE again. Good to

>> see you.Hey, guys.Thanks for having us.How you doing? Great to be here.

>> So we were just at your event last year. You said some things, put some things out there, "This is what we're going to do with AI, generative AI." It was all the hot rage. ChatGPT really innovates on the user interface check, we see that. Getting things done is a whole nother ballgame. You guys got some huge announcements. The models, I love it. Let's just get into it. What's going on right now? What's the key things for your transformative technology around GenAI?

>> Well, let's just get to generative AI out of the gate 'cause everyone's talking about it, huffing and puffing, which is, again, it's a really exciting technology, but it has to be put in context to actual business outcomes. And so I think, for me, I'd like to actually give it over to Marinela a little bit because a day-to-day data scientist out there, and on the ground, meeting with customers, hearing the feedback. So Marinela, why don't you talk about about what you're seeing on the ground.Absolutely.

>> Thanks, Bryan. Yeah, so from my perspective, I'm a data scientist, and I think that the number one lesson that I've learned, if there is a lesson that I've learned, well the past three and a half, has been LLMs, large language models, alone do not solve business problems. The technology is so cool. And no matter how cool it is, it's still imperfect, and it's all going to come down to solving real world business problems for specific industries and specific people, right? We're seeing a lot of companies just saying, "We have this technology. Let's go and build it. Let's . Here's how we're going to do it." So a lot it's around how do we build something, but that there is very little around why. Why are we even building that? Who's going to serve? What's the value that's going to deliver? Right? And so for me, from my perspective, what I've been seeing is that generative AI and large language models in this case are like the 20% of the entire value chain if you consider a real world use case. And I think what Bryan shared today with Georgia Pacific was an incredible example of that. I don't know if you want

>> to add more.Yeah, definitely. Case and point that Georgia Pacific is on stage talking about their use of SAS Viya as part of our generative AI workflow. There is a lot of moving parts. There is no more kind validated feedback on a production use case of GenAI when they're actually trying to run their plant. Their plant operations is leveraging generative AI. And as part of that, they're getting corrective actions through this generative AI experience that is leveraging SAS Viya to orchestrate across LLMs, orchestrate across quantitative reasoning needs for... We're interacting with enterprise systems. So all of this orchestration is happening, ultimately to give a human language response back to an operator who is three, five years experienced their job, and they're trying to convert them and upscale them to a five to seven year employee.

>> There were a lot of components to that stack.

>> Yes.

>> And you think you got a picture of it here. You had Bedrock on there, you had SageMaker, you had SAS, you had another couple other pieces. And so you see GenAI as the orchestrator

>> for all that?Yeah. For us, at some point, you got to integrate all of the tech, and what you're really getting to is you're making a final decision to give a response back to a user. And as we know, a lot of these cases right now are very shallow. They're just prompt engineering to an LLM and come back. And yes, RAG architectures are there, but we got to get things done in sub-second response time here. So there's very, very challenging integration and expectations of responses from GenAI workflows that we have to get done. So for us, Viya sits at the center of that. And even Sam Coyne mentioned it today that we really bring all of these things together. We can augment prompts, we can orchestrate against other traditional models and things like that, and bring that data back in so that the LLM can reason over that and send back the response back to the user.

>> And answering to your question, I think Viya is orchestrated here, not generative AI. Generative AI is a piece that can augment existing processes, and Viya, our data AI platform, functions as that orchestration of the entire decisioning flow. And that's exactly what we are announcing today. We've extended Viya to allow developers to use our API to govern, orchestrate, and build real-world use cases with GenAI.And

>> I think the point about GenAI is an element of the stack, is at the higher end the stack. I have a picture. I want to just get your reaction. Okay, you built on AWS, so got some hardware, got some cloud.

>> There you go.SAS

>> intelligent decisioning, that's your layer, right?

>> Yeah.

>> That's the-Decisioning system.... decisioning systems, and then multiple real-time data knowledge, bank process model. Okay, that's the meat and potatoes. That's the stuff that's the key jewels to make that thing work. And then you've got the GenAI on top managing it, and then you got the outcomes.

>> Yeah, so you think of it as that something has to bring all these things together, because right now, a lot of the reasoning capabilities in text for LLM is around textual fact-based reason responses, but a lot of the decisions in the enterprise are quantitative. So that's why we pride ourselves on being the quantitative reasoning engine inside of GenAI workflows. And intelligent decisioning is such a flagship product for us because it allows us to... We can bring four or five different models to actually integrate across text across the LLM itself, across the actual enterprise systems we're integrating with.

>> I want to connect it to something you said in the media briefing, if I could, which is about ROI. If you look at how people are using GenAI today, it's very ChatGPT-ish.

>> Yes.

>> So it's not literally blowing away the cash flows. It's not really paying for itself. And so when you start talking about these industry-specific models, that's where you're going to get return. And people are pushing out their ROI expectations based on the surveys that we have. And so they're being a little bit more conservative because saying, "Okay, this is fun, this is cool, but we need real dollars."When

>> you open up a prompt to thousands of people in your organization, the quality of your questions is now your biggest concern, right? Now, people are paying for this. So now, okay, well, that's why you saw us talking about stored prompts because our ability to say, "All right, look..." There's not an infinite set of questions like in the public domain with ChatGPT, right? In an enterprise, there's a finite set of questions. So why don't we understand those questions you want to ask, make an inventory of those, use Viya to do the semantic analysis on this to then say, when three, four people ask a question, we're going to get the question that the right question you're asking, and then let that then reduce the error to the LLM, reduce the error to the vector database, reduce the error to the quantitative reasoning with Viya

>> as well?I thought that was a very nuanced point. I thought one of the things that jumped out at me was the stored prompts because Dr. Goodrich was actually bragging about the compiler versus the interpreter on stage. If you think about that, as you get smarter with these stored prompts, you can learn, and that's like compiling away the LLM risk.

>> What's funny about it is if you remember, what do database providers want to do? They wanted stored procedures, and stored procedures was their way to govern the way to build optimized queries for databases. So stored prompts are really just an extension of that entire model.

>> People are doing this in all kinds of use cases to get smarter. You make a prompt, you get an answer, you come back. It spins back down to the architecture. Why I liked that graph and why I called it out was that's a production workload.

>> Yes.

>> Correct.

>> The only thing that's outside there is the customer's environment and Amazon and your SAS stuff. So that brings up the big picture here. You got training and inference is the big discussion on how to deal with data, and then you got prompt response, and then you've got reasoning in the AI paradigm. Okay? Architectural and behavioral. So you're essentially rolling all that up. So okay, prompt response, that could be real, like IO. Reasoning could take a little longer maybe, or... So I got to run that somewhere. How do you see that applying to known end-to-end workloads. Because this is the theme that's popping into this AI world and the enterprises. I got known workloads. I can scope them. I can qualify them.

>> I think the way it rolls up is that we're seeing... Definitely the agent design pattern is emerging. In fact, in our Viya Copilot demo that Marinela was showing today, we're actually doing some iterative work where we have one generating the code, right? One part of the orchestration is generate the code underneath it, execute the code. If it fails, tell the LLM failed. It'll make the corrective action on it and execute it again. And then we'll get the actual-

>> No human.

>> No human in that one. And then it's why you saw a map come on the screen, or you saw some of the extra follow-on visuals or suggestions. So that's why this orchestration is so important, because it really is an iterative process. It's not a single request, single response experience. And so for us, this reasoning thing, if you think of reasoning, reasoning has to span text-based, fact-based things, media like images and video. But most importantly, in the enterprise, quantitative reasoning is really the majority of the workload challenge. It's not about just... Customer service, yes. I got a knowledge base. I can give you answers on it.

>> Right.But

>> a lot of the enterprise class problems are, how do I look at numbers, analyze numbers, and make a recommendation on our next best actions where-And

>> that's why I asked that dumb question, in the media thing.

>> That was not

>> a dumb question, by the way.Not a dumb question.That wasn't a dumb question.

>> Well, I was trying to get... Came out

>> wrong, but-I've seen a lot dumber questions than that. I'm usually the for dumb questions.

>> I got a whole bag of them over

>> here.But the point is customers today have to go to different models for different modals. So they want one place, they want to have the reasoning and all this, it sounds complicated, but put it in one place. You can do vector embeds for unstructured data. I can bring a table in. I can bring in a DNA sequence. That's a mode, right? All kinds of data can be structured that way if it's in one place, I don't want to go five different places.

>> And then do you have the technology? I think that you can talk about... Why don't you give the example of the banking one we talked about today,-

>> Correct. Yeah....

>> where you can really talk about all the tasks that were required to really make it work?

>> Yeah. So this is one example that we were working with a global bank where they were doing complaints management, right? It's a simple use case. It's not revolutionary, but the question was, how can I augment what I'm doing with generative AI, with large-language models? And the integration of the LLM is not just easy as, oh, give ChatGPT a complaint and generate a summary. And then based on that, it'll ask to generate a reply to the customer and send that reply to the customer. That's not how it actually would work because you have so many other pieces in the middle. And in this case, the flow that the developers were able to build to actually productionalize that use case involved other things like, okay, the summary is generated by the LLM, but how do you explain if that's accurate? How do you explain how you generate that summary? And so that's where natural language processing techniques, which is actually a that has been proving to be really critical and fundamental in explaining some of these, the output generated by large-language models, that. And then how do you integrate your customer data within that summary? The LLM doesn't know if your customer has a high credit score, is a good payer, is not a good payer. Those are your enterprise data. The LLM... Unless you want to give that to the LLM and expose them to the LLM, which I don't think that's what our customers are asking to, you need that quantitative decision engine that augments that information and sends back that information for the LLM to actually generate that summary and generate that email in a more reasoned and a way that makes sense.

>> If you think about it, we compress all the complexity of the quantitative side into a much more consumable fact that could be presented back to the LLM, so we can use strength, which is to generate natural language

>> response.Which that might be a score, for example?

>> Yeah.

>> Okay. And then the

>> LLM-Think of it as a score.... will

>> take that score as opposed to trying to generate the score, which it's not good at.

>> Right.

>> It's actually horrible at that.

>> Exactly.

>> That's actually horrible.You

>> said it well.

>> Right.I

>> would say it exactly that way. It's Terrible.

>> Dave is He's a SAS user in our actually used to program in SAS

>> back in that day.We appreciate it.Back

>> in the day.

>> born Those

>> about the proc, we salute you.

>> And if your customer is complaining about, your credit service, the LLM is not going to be able to generate the next back action for your specific customer because it doesn't know your customer. It doesn't know what a next back action is.

>> Since you up customer, I want to get more use case information, what you guys see in the marketplace. As Dave and I always talk when Dave does some research on the monetization, because that's really going to be scoreboard, is it adopted in production, which you guys have shown some examples of great others and the monetization of AIC, the early adopters, some are monitoring really well, so that's also a tell sign. Monetization, uptake, and then customer success. So you mentioned you're doing well with the business performance, and you spent a billion dollars of investment and spending more on the tech. So you've got tech investment and monetization. Good job. What are the unit economics look like? What's the use cases that are emergent you can point to and say we're starting to see the things line up in the market. The fog's lifting in the enterprise. What is it out there, and how is it being Beyond

>> the chit-chat, right?

>> Yeah, I'll get there. I would say that we hear a lot of customers worried about yet another metered challenge to the business. There are CFOs saying, "Look, we're trying to run a profitable business." Look, if you can tolerate metered billing when you know you've got enough room on the financials to tolerate it. But when you start to... Remember, we're lowering the barrier to interaction for data and AI across the entire enterprise. When you do that, you're actually encouraging usage of these metered services, which is fine, but you have to make sure that the use cases are driving an ROI that can support those metered interactions. And then we have other customers saying, "Can you take me off the meters? Can we just run..."It's

>> my data. I want to bulk upload my... It's my own data.

>> Yeah, exactly. So can I... And we want to respect all of those, right? Many customers who... We're partnered with Microsoft. We want them to use, leverage their OpenAI services where possible we're working with Bedrock and Amazon, and AWS will do that. But there are customers who are like, "Hey, I want to run with minstrel. I want to go and I want to run that in a GPU that's hosted in their environment." So the same cloud spend conversation that was really-

>> So they want optionality, but they want to have confidence on what the spend is going to look like. They need to-

>> Yeah.

>> Absolutely.... have some scope.

>> Yes, because what's happening is everyone... Again, the FOMO that was happening around just go run to this thing is now being realized in the financial outcomes of these businesses.

>> These are big customers? Can they get GPUs? You're not spending 10 million bucks on video.

>> No, I think when you're training models, it's one thing. If you're just looking to go and do the hosting them from scoring these models, I think there's options there. There's a lot of things coming down in size on it. Now, we're just going to make sure that our stuff can react to any one of these scenarios. If we want to use a cloud service provider, we're happy to do it.Right.Some of the customers just want... That's what they want to do, so we'll leverage it. But we're seeing others now starting to say, "Let's bring it locally.

>> "Is it tracking like DevOps? Did you get a use case and you experiment with something, or is it end-to-end workflows that are emerging? What are some of the

>> Or is it the data is on-

>> I think first and , I think there's a little bit of a hammer and nail problem where everyone's saying, "Let's start with generative AI" when really-

>> Correct.... generative

>> AI is a subset of AI. It's not a superset of AI, which is a big problem because it's been presented in the market as if generative AI is this new thing. It's actually a subset of the larger AI umbrella. In fact, the problems being solved by the rest of the AI space are much larger than the problems being solved by the LLM and generative AI space.

>> Some of the best models are small.

>> Yeah, yeah, exactly.

>> I think that's going to be the future where the smaller

>> Yeah, small language models. Yeah.

>> Yeah, smaller language models that would say almost auto-fine tune themselves for specific industries

>> and specific use cases are going to be-And that's because they more accurate and faster or-

>> I think it's like anything else. As you go to the public... The large language models that are out in the public are just, they're generalists in these things, and they're powerful for many reasons.

>> But they get vanilla.

>> Right. Exactly. If you want to use an example here, and I... Google was trying to... Remember when Google was so huge, they wanted to get into the enterprise space, right? How can we put that Google engine? People were going home and they were having a Google experience for search, and they would go into the business and say, "Why can't I have that same search experience in my business?" The use case was different, right? Google's indexing engine was about finding information authorities. Meanwhile, in the business, it was about quantitative analysis. So they never could really penetrate that market. We're having the same challenge now with generative AI, which is that you can have a very public domain question and answer experience with LLMs that are very powerful, and you want that capability on the enterprise, but there's more work to be done. And what people are realizing now is how much more work there is to be done. Especially for us, we're in regulated industries, so we can't be wrong. So I can't just go and run-

>> Can't wing it.

>> Yeah, can't wing it. We've got to go and work meticulously with them to understand here's what happens when you do this response. And if that response gives an answer and you need proof of it-

>> Yeah, there's

>> no space for hype.... we need lineage. You need all the things that go behind it.

>> No space for hype for us. What I would add is also that... My question is, until what point this gigantic large language models providers are actually realistically going to have data to keep training these huge models?

>> Yeah, meaning they're going to run out of data.

>> They are.I think so.It's call strip-mining-I think so.And either it's because it's going to become a legal issue at the point where you can't take my data anymore to try and whatever you want to train, or there's just not more data so we have to go back to synthetic data

>> Can you believe you were

>> saying that? I know.Adobe's charging $7, I think a minute for the video, right?

>> And that's why data Maker for us is the big announcement.

>> Correct.

>> Because SAS Data Maker for us allows us to attack the gap in structured data for synthetic data. And by being able to do that, we can go into regulated spaces and give proof to the statistical congruence of synthetic data that can bootstrap models that will allow them to run effectively on real-world data.

>> You got to have explainable AI, table stakes, and security built in. Great to have you guys on. We're out of time.

>> Got

>> to go?Final question. They got the hard stop.

>> I was enjoying this. Wow.

>> I'll give you one more question then. We got one more.

>> Maybe two.

>> So next, great step up from last event... Thank you for bringing all the goods to the table. Next year when we're sitting here, what are we going to be talking about? What's going to be the big discussion? Bryan, we'll need share your thoughts. We'll start with Bryan, and we'll

>> close this out.I think that what we're about to achieve with App Factory, still we're building our own solutions with it, but the productivity of that thing is incredible. And we started wrapping some generative AI experiences on building applications that are... The treatment on that's going to be credible. I think what you're going to see on our Viya Copilot's work that we've just done and demonstrated today, the ability for our Viya Copilot's to be a very practical approach to the AI life cycle is... I think it's going to be world-class and second to none. So that's where I think we're at.

>> What I am really excited about is to continue... So the strategy that we have taken over the past two year and a half, two years, is building something with a purpose. And I'm really excited to come and see that come in action. We're not just a startup that is going around looking for funds from a VC. We're building these things for actually being used in the world and creating real world value. And I can't wait for that to continue to be and have even more... I want to come here next year and talk about other 10 customers that we've done, and all the models that we're going to sell. So that, to me, what I'm really excited about.

>> That billion dollars in investment, you're monetizing well, which is proof. And you got production workloads. Just keep it coming. Yes.

>> Yes. And I would say too, we will have a significant amount of progress on penetrating the market with selling our models, which we're really excited about.

>> I got to ask one more question. We got time.

>> Ask for forgiveness. We're good.

>> You said in the media briefing, nobody's looking at the impact of AI the way SAS is. And today, it's very clear. You're talking about very focused on solving a lot of business problems. How far out are you looking?

>> What do you mean by that?

>> In terms of the impact of AI. AGI, if we have AGI by the end of the decade, are you thinking-

>> Robots....

>> that far out? And then if you think about exponential growth of computational power, mid-century, when you make that statement, I have to ask you literally how far out are you thinking? Is it more-I

>> think we'll have tools that we'll be able to evaluate... We'll have capabilities that can evaluate the reality of these models. Some of these models are self-fulfilling validations where it's like there's no real... They're not deterministic, right? So there's certain kind of workloads we put against them. We try to measure them on grading these things, but they're not measured on the enterprise's actual workload. So how do we think about what does the answer look like in an enterprise, and how did that model react to it? And then being able to go and say, "That worked, that didn't work," it starts bringing some true in-customer validation endpoints to, what are we doing with generative AI? So I think that's the area where you'll start seeing these things will have to be validated independently of the model creators.

>> Well, the stars of the show, you're at high demand. You got to go. We're getting the hook. You got to leave it there. Okay, we're going to continue the conversation here on theCUBE. We'll be right back after the short break. I'm John Furrier with Dave Vellante. We'll be right back.