theCUBE + NYSE Wired: AI + Cloud Leaders | Nitin Wagh, Karini AI | AI + Cloud Leaders

Clips
More from theCUBE + NYSE Wired: AI + Cloud Leaders

Nitin Wagh

Co-Founder & Chief Executive Officer

Karini AI

play_circle_outline Revolutionizing Business with Karini AI: Empowering Experts Through No-Code and Low-Code Generative AI Solutions

play_circle_outline Karini AI deploys within customers' AWS VPC for enhanced security and data privacy.

play_circle_outline Discussion on SageMaker’s evolution and its importance in AI infrastructure management.

play_circle_outline Historical context of Karini AI's evolution from RAG to agentic workflows.

play_circle_outline Collaboration with AWS unlocks strategic opportunities for growth and marketing.

Info
Transcript

Nitin Wagh, Karini AI | AI + Cloud Leaders

Nitin Wagh

Co-Founder & Chief Executive Officer Karini AI

In this exclusive segment from theCUBE + NYSE Wired: AI + Cloud Leaders, Karini AI CEO and co-founder, Nitin Wagh, joins theCUBE’s John Furrier to break down how enterprises are accelerating generative AI deployment with Karini’s no-code, low-code platform. Drawing on his deep AWS and SageMaker experience, Wagh reveals how Karini is helping industrial, legal and engineering firms automate complex business workflows – delivering measurable outcomes like $1M+ in savings through ERP exception handling and agentic RFP response solutions.

Wagh outlines Kari... Read more

explore Keep Exploring

What is the mission and focus of the company mentioned in the text? add

What challenges do enterprises face when choosing between different technology vendors and solutions for their needs? add

What can you share about your experience and insights regarding the evolution of SageMaker and its relation to other components in the AI infrastructure stack? add

What was the motivation behind starting the company and the evolution of its focus on RAG and agents? add

What developments and agreements have occurred for the company in the first half of the year? add

bolt Powered by CUBE AI

Nitin Wagh, Karini AI | AI + Cloud Leaders

search

>> Hello, welcome back to theCUBE here at the New York Stock Exchange Cube Studio. Of course, Palo Alto and New York connecting Wall Street and technology together, part of the NYSC Wired. Nitin Wagh is here, he's the CEO and co-founder of Karini AI. We were just at the AWS Summit, we're back in our studio. Nitin, thanks for coming in.

Nitin Wagh

>> Yeah, thanks for having me.

>> Congratulations on the venture.

>> Just talking about your history at AWS, Databricks now, this venture. You've got to look at the AI wave right now, you've kind of seen it from the inside, early days of AWS. Now you've got the Pin, big partner. Talk about the company. What do you do?

Nitin Wagh

>> Yeah, so Karini, our mission is to put generative in production at enterprises. So to do so, what we do is, we provide a no-code, low- code platform sort of abstraction on top of the AWS infrastructure. So not just IT, but also line-up business IT, business experts can build, operate, and manage generative AI. And we focus on time to market. A lot of our customers are from marketing department. I'm talking to lawyers, I'm talking to HR experts. In my life, I was always talking to data scientists, but with generative AI, guess what? We are talking to more domain experts and they are quite essential for successful generative AI projects.

>> You're targeting real companies.

Nitin Wagh

>> We are targeting real- - Average

>> personas, a business person.

>> So you're kind of hitting that productivity business transformation use case.

Nitin Wagh

>> All of our projects are about business productivity transformation, and they're mostly about business process automation. For example, some of the use cases we are doing at Space- Lok, Southeast Texas, which is a huge fluid parts manufacturing for oil and gas, aerospace. So we are doing sort of order-to-cash, ERP exception, business process automation. Customer said it saved them a million dollars just with that multi-agentic workflow. At another company we are working with, legal tech company, where we are doing legal research, precedent citation matching. Then with the engineering and construction company who responds to thousands of RFPs per year, we are automating the RFP response generation solution, which is a super hard use case. You can imagine writing an essay and it has to be perfect, it has to be a winning combination? Right, so-

>> You don't want to miss a zero on those bids.

Nitin Wagh

>> No way. - Accuracy is key. Well, let's zoom out.

>> What's the product? Explain the product, how it's consumed.

>> Obviously you're making it simple and easy to use agents. Okay, is it an agent platform? Take me through how the product is deployed. What is the key features?

Nitin Wagh

>> Yeah, yeah, definitely. So if you think about an enterprise, today, there are a lot of vendors. There are some vendors who are really good at model serving, observability. Clouds themselves have their own offerings and native services, or there are agentic frameworks, these Lanch and Llama index. So there's just a lot of noise, but then these are siloed offerings. They're really good in themselves. So enterprises are then often tasked, "Hey, do I go to a SAS and give my data to them? Or do I build an agentic platform? " This is a classic chaos within enterprise, and they end up starting to build a platform. And that takes millions of dollars, many, many months. By the time they build it, time has passed.

>> It's out of spec. - Yeah, it's like

Nitin Wagh

>> things have moved on, games have changed. So our value proposition is very simple. You deploy Karini AI, which comes with all the capabilities, such as built-in observability, agent building experience, tracking, FinOps element, knowledge base and knowledge graph creation. So all the quintessential elements that makes up for the agentic platform within Karini, you deploy within your own VPC. So we are not selling a SAS. We have a SAS, but that's good for demos in POC. Our model is we deploy in customers' AWS account, in their VPC, so it can be entirely air-gapped, and then we enable their business users to build their own private AI. So that is where we-

>> Are you guys only AWS?

Nitin Wagh

>> Currently we are only on AWS, but we built the platform on Kubernetes as a backend, so it can be-

>> It can be multi-cloud. Yeah. So one of the customer's cloud, they have another cloud, you can be on that?

Nitin Wagh

>> So currently our control panel only runs on AWS, but then they can connect to other model providers or other type of offerings. Let's say we have customers, sometimes they want to use OpenAI or Azure OpenAI, they can connect from our cloud. But again, we are-

>> So you can play within Bedrock and SageMaker,

>> and the VPCs, air gap, everything on Amazon. Great. But OpenAI is not in Bedrock. It's okay, that's just a call.

>> Make an API call. We also support Google Vertex.

Nitin Wagh

>> There are also private providers like Fireworks. So I think we believe in... We have XSS tier. So we believe in customer choice, so let the customer dictate.

>> Talk about your background at AWS. We were talking before we came on the podcast here, you were involved in SageMaker.

Nitin Wagh

>> Yes. - Which has gone through a transformation.

>> I kind of like what it is now. I call it the power user, the advanced mode, but it really is tuning a lot of the infrastructure piece underneath Bedrock. Then there's a Bedrock piece and AgentCore looked really good yesterday, Kiro looks fantastic. What's the evolution of SageMaker and how does that stack? Because a lot of companies are coming out in this AI stack, I call, for lack of a better word, the infrastructure is great.

Nitin Wagh

>> Yeah. - So you've got to tune that too.

>> You've got to play with to get that operating properly, and yet all these tools are coming out to manage things like tokens. So you're seeing a lot of customization with code generation around tools.

Nitin Wagh

>> Yes. - I was talking to a customer who said, "Yeah,

>> we're more excited less about agents. We love agents, but it's the tooling that allows us to manage the token costs."

>> Yeah.

>> Why go to a token call when you can build an abstraction

Nitin Wagh

>> in the LLM and manage non-related calls? Why would I call an LLM to do mass? Why waste tokens on that? So they're starting to get more honed in on the design of the system on top of Bedrock and then SageMaker. How do you see that? Because I think that's a story that's not being told right now, where you've got SageMaker tuning the infrastructure and then you've got the Bedrock side where the developers are managing how to configure the LLMs, whether it's going external to an API or managing it within Bedrock. What's your reaction to that? What do you see there?

Nitin Wagh

>> Yeah, yeah. So what we have seen is, when you start a use case, again, these are the tools that you should use based on your use case and your problem you're trying to solve. There's no need to analyze, parallelize too many, and typically when you need fine-tuning, right? You'd never start with fine-tuning. If you start with fine-tuning, you're making a mistake. When we start with our customers, we typically start with Bedrock. I mean, Karini platform, you select a problem, write a prompt, write an agent, and then of course, as you said, tool ecosystem is very important. We always try to push down things, because I don't want LLM to do processing. LLMs are not built for doing data crunching. For example, I don't want LLM to do knowledge graph, query execution. That is better done in Amazon Neptune. So we have actually built a tool that is a super tool, you can say. LLM creates a plan, "Hey, extract from OpenSearch some of the records, go to knowledge graph, build this query, do the transformations. " All of that is happening in tool, so you can free up LLM to just create a plan.

>> So leverage the reasoning piece of-

>> Reasoning piece rather than doing

Nitin Wagh

>> the heavy data crunching. So what we do is, to your other question, so we typically start with Bedrock, with the on- demand model, and then we sort of run the trial, typically take golden question, answers from customers, and then we have a evaluation harness. And if you see that, all the techniques, for example, we also have prompt optimization. So you can optimize the problem for a particular task, find the model, write a combination, but sometimes you still cannot get to your goal. You always have outlier edge cases, and that's when we definitely talk about fine-tuning, and that's where we have built the abstraction on top of SageMaker. You can fine-tune the model and then you register that model into Bedrock for inference, because Bedrock is amazing for inference, right?

>> Yeah. - You don't have to manage, you just import the

Nitin Wagh

>> model package and boom, you have an API that you can register from Karini and it just magically works.

>> Yeah, it's having that AWS knowledge. It's almost like you're like a product team inside AWS, but you don't work there anymore.

Nitin Wagh

>> Yes. - I love this abstraction with SageMaker.

>> So you're essentially automating SageMaker to work with whatever is coming in off the top. You're basically building kind of a runtime, your own kind. That's a bad word. Maybe more of an abstraction intelligence layer.

Nitin Wagh

>> Yeah, it's like a dev loop. We built a dev loop which customers don't see. It's literally like, "Hey, you want to sort of fine-tune a model? " We launch a job in SageMaker. Because it's hard. Fine-tuning, while it's sounds simple, it's the whole developer loop. It's very time-consuming. I don't think it's hard, it's time-consuming. All the pieces are there, but we just want to make it faster so that time to market is easy. So you fine-tune a model, we launch a job in SageMaker through API, and that knowledge definitely helps. My co-founder, she's also from... Ex-AWS, she's an ML specialist. So it's a lot of AWS knowledge.

>> You have the working knowledge of all the buttons

>> and knobs to push in SageMaker. Again, I call SageMaker the power mode, it's like the advanced mode. Don't go in there unless you know what you're doing. You'll blow something up, if you will. Okay, so I love that angle. So I want to ask you a question since you're Amazonian, former Amazonian. We'll use Amazonian speak for a minute here.

>> Yes. - So you provide all the undifferentiated heavy

>> lifting around managing Bedrock, SageMaker,

Nitin Wagh

>> AI infrastructure, which is SageMaker interfaces into policy-based, whatever you want to do, managing the infrastructure with the logic in the dev loop. So when you go to the customer, say an engineering company who's just trying to win bids, they're like Excel users or using sheets, that's what they use.

Nitin Wagh

>> Yes. - So you go to them

>> and say, "Look, we've got you covered. Import your logic. " How are they interfacing into your dev loop? So I mean, you're doing probably some services with them, right? So I'm imagining that's part of the motion, is you kind of come in, you deploy, but how do I leverage? Okay, here. Here's a spreadsheet, how do I inject my logic into the Bedrock?

Nitin Wagh

>> So yeah, I think it's a little nuanced question, how we get an answer. Bear with me. So typically what we do is, we understand the business problem and then map it out to a technical. What is the solution, right? AI. And we have an excellent AI engineering team. We are not a services company, a product company, but what we believe is, we want to make a customer successful. So we also, during our first use case with the customer, we help them deploy that and take it to production. Because only a successful customer can get us to a next use case, right? So that's very important. So we map that business use case to a technical sort of AI problem, and then we call it as a recipe. A recipe is like a canvas. Or you can say it's a workflow under the hood, right? It's an agentic workflow. Okay, now user wants to ask questions, so it's a chat. I want to send the API, it's a web book type.

>> Provisioning a workflow, basically.

>> It's a workflow. It's a workflow.

Nitin Wagh

>> And a workflow can have different types of node. It can have a prompt node, agent node, it can have a custom function. And then we map out that process, and then those users understand the process they may not understand. For example, in the engineering company, they have historical responses, they have bid data, they have resumes, and they have all the win history, win-loss history. So all these things they understand, so we just map that out on a recipe canvas, and then we take that process. Okay, we get alignment, and then we start building. And building is done, you go to a prompt playground and build those individual agents, test them, select the right model, version control them and bring the real agents back to the canvas and boom.

>> I like how you said you're not a services company, you're a product company. I love that. But again, Swami, when he gave his keynote yesterday, he had the big slide up there that said service as software. That's essentially what you're doing. Your service is the software.

>> Service is the software.

Nitin Wagh

>> So that's not software as a service, that's SaaS.

>> You have a SaaS demo platform, but that's just where the world is going. I mean, it looks like a professional search. Yes. But you're not hired to be a consultant, you're hired to essentially deploy agents. So you provision the service as a product team.

Nitin Wagh

>> Totally. So I mean, of course we are a product company.

>> Of course. Yeah, yeah. - A lot of people confuse us.

Nitin Wagh

>> So what we believe is we want to partner with the SIs,

>> system integrators, GSIs, and those partnerships are forming. Because I think their model is also changing, right? They have to learn the new ways. So the traditional way, we have actually done multiple projects where things have gone wrong with that traditional approach. And with our approach, things are faster, customer is a little bit enabled. So I think they have to also think about changing their model. Because with Karini, you don't have technical debt, there's nothing left. You can continue to iterate. A traditional approach, if you write code, it becomes a technical debt, because soon it'll be outdated. For example, when MCP came along in Jan, every customer of us is thinking about, "Hey, what do we do? Should we get rid of all the agent workflows? " I was like, "No, let's not do that. Let's understand, is it really helpful for you to implement MCP? " And MCP had a lot of security holes back then. So we took a very careful approach where we said, "Okay, why don't we build an MCP registry where an administrator within enterprise can only approve the tools or the servers, MCP servers that are authorized, they're vetted and validated security-wise."

>> They're trusted.

Nitin Wagh

>> They're trusted, and then you add them centrally

>> and then let the builders have access. And as admin, you cannot control, "Oh, there's a security vulnerability, I'm going to disable this and it's just-"

>> It's a great protocol, but you have to take careful consideration for making it a node in the network.

Nitin Wagh

>> Because there's just so many open source MCP tools, it's just tough to understand... It's like , so many models. Which model you can trust to put in your enterprise?

>> Yeah, I mean it's not technical debt,

>> it's just bad design.

Nitin Wagh

>> Just bad. - So technical debt is great,

>> then you're talking about that, but then the design side. All right, so I have to ask you, this is one of the things we've been talking about on theCUBE a lot, is that we're entering a systems revolution. Something that I've been saying for years. Now we're seeing it play out at Summit, which is kind of like a mini re:Invent in my mind, because even though we're midpoint, kind of at the halfway point of the year, it feels like a lot has happened in the first six months. And then they had significant news. I mean, Nova customization, obviously AgentCore, even Jassy did a post this morning on that. He did a little video. And then obviously you've got the marketplace making a dedicated section for agents and tools. So that's significant. Over 100 people launching with it, that's pretty large considering their last launch with SAS was only a handful. Chris Cruz and I talked about that yesterday. So if I look at AgentCore, the builders now have to be systems thinkers. What's your take on this? Because I think this system mindset is an Amazonian concept, distributed computing, old school computer science principles are in play.

Nitin Wagh

>> Yeah, I mean AgentCore is a brilliant idea. If you look at all those cubes that are out there, that is actually a Karini platform. It's just that we have an abstraction, no-code, low-code abstraction on top of those AgentCore. So the beauty is, AWS always, I was part of the other side, so I know AWS has thousands of engineers, they build beautiful things. Observability, for example, we have our own observability where we package a data warehouse under the hood and then sort of stream. So that's how customers get FinOps and other things. Guess what? Now AWS has observability, we may swap out that piece that AgentCore already provides because it's a siloed capability, which is really nice for someone like us.

>> You get optionality. - Yeah, we get optionality.

>> Yeah. - And then we can push down.

Nitin Wagh

>> For example, within Karini, we started

Nitin Wagh

>> with having our own OpenSearch for vector database,

>> but soon we realized, why manage OpenSearch when Amazon has Amazon OpenSearch? So we sort of migrated all our customers to Amazon OpenSearch. So we will sort of start swapping out some of the components as they mature. It's very important that these components are mature, because we have customers in production, we want to have those mature components.

>> You're essentially refreshing the stack

>> and the capabilities with best capable, best of breed in the platform.

Nitin Wagh

>> It's the platform. Because it's all about skill, right? I mean, I know Amazon is really good at scale. We have some scalable workload. For example, in legal tech, working with a customer, they have three million precedents. So that's a lot of data. You're talking about probably 15, 20 days of processing time. I would rather outsource that to a managed service like AgentCore.

>> Yeah, yeah. All right, talk about the business.

>> When you started the company, obviously you guys came together. What was the original vision of the company, the origination, and where are you now? What has been the biggest change? Just the robustness of the software. scalability, demand? From the origination point to now, take us through that.

Nitin Wagh

>> Yeah, so when we started the company, actually, Deepali, the co-founder, she started the company, I joined her later, but the idea was she was one of the early engineers in the machine running, she knew about Bedrock, but the idea was AWS is really good at creating developer tools, but what if we enable the business experts? And that was a missing gap. So the goal was, when we started, I think it was about RAG, right? There were no agents.

>> Exactly. - We just build a really awesome, easy

Nitin Wagh

>> to use RAG you can build in 10 minutes. And customers just loved it. And then of course, agent came along and we quickly pivoted to agents, but we always kept our theme as a platform. A lot of our other companies, they pivoted from different businesses to platform, or platform to some other business, vertical business. So we always kept our-

>> Well, you still have the RAG now?

Nitin Wagh

>> We still have a RAG, but

>> now it has changed into agentic RAG.

>> I mean, it's really not a pivot. A pivot is a stop and turn. You guys just extended off to agents because it's a natural-

>> It's a natural .

Nitin Wagh

>> We started with RAG, we added agents, then we added knowledge graphs. And then I always believe that AI is only as good as the data it can see. So RAG is always important. And then there are different types of RAGs. There's a knowledge graph, there's knowledge base. And we always believe that if you can industrialize the system of record, which is your knowledge graph or knowledge base, that is going to be phenomenal. So we have been working very hard in engineering under the hood, because people see this platform, they see it's like, okay, no-code, low-code canvas. I always tell our customer, "It's also a workflow under the hood because we also migrated from sort of a siloed single load to distributed RAG. " We raise the best thing in technology that can happen because I want to scale to millions, millions of records. Customer, they should not be upper bound.

>> Great to have you on and I really appreciate you coming into our studio. I have to ask you, if you had to describe the biggest surprises or activities that have happened in the first half of the year, what would they be? What were some of the big moments in this year that kind of changed the game, move the needle, if you will, or raise the bar? And then what's your plan for the second half?

Nitin Wagh

>> Yeah. Is it for Karini? Or- - Karini

>> and just the industry and you guys. Both.

Nitin Wagh

>> Yeah, so I think with Karini we are actually working with meaningful GenAI use cases across the board with different domain experts, variety of industry units. We're really good at manufacturing. We have actually captured and done all types of use cases for manufacturing, automotive, and industrial, which is typically not a leading sector, but they're leading in GenAI.

>> They got a lot of labeling. - They've got a lot

Nitin Wagh

>> of labeling, a lot of the data, machine data, a lot of manufacturing, unstructured data. What we call unstructured data. We are having amazing success there. So first half has been really good for us, because we do strategic collaboration agreement with AWS, which unlocks a lot of doors for us, co-marketing events. That's why probably I'm here. We did GenAI competency. That was late last year. Then we just launched agent yesterday. We are right next to Anthropic in terms of that slide, but we launched our first agent and we learned a lot about agent. We were on marketplace, but this is our first on-demand agent where you understand billing, metering, and all those things. Now we have all the orchestration, we can build many more agents. So second half of the year, what we are going to focus on is solutions. So for example, we have this order-to-cash use case. Now, maybe we'll come up with a solution for ERP exception as a whole so it can solve procure-to-pay, order-to-cash, record-to-remit, all these problems. So we are going to focus a lot on verticalized use cases, but we will still continue to keep our platform theme. But it'll be a sort of application.

>> You're going to continue to get that horizontal use case

>> of all GenAI wherever possible, and then start targeting the vertical apps or use cases where the domain expertise are, where you can import or have the clients bring their workflows in?

Nitin Wagh

>> Yeah, we'll give them a starter pack and then we believe AI has to be customized for their needs. Not every customer is same. If someone is telling you, "Hey, I can just put one agent that will solve all world hunger, " it is not going to happen. You have to look at the data. Data has veracity, it has variety, so you have to customize that AI.

>> Well, great to have you on. Final question. What do you think about the Summit this year? It was kind of a mini re:Invent. I think that's my view. You probably agree with that. Usually it's active anyway, it's a big event. It's freeing. And we're in New York, everyone is here.

Nitin Wagh

>> Yeah. - What's your takeaway?

Nitin Wagh

>> I've been attending, I've spoke at mini

>> Summits, re:Invent. I think this is surreal, because all the launches that I've seen yesterday, and Swami was very energetic. Even Rohit, the Nova launch, I was very excited about Nova's inference that is on-demand with fine-tuned models. That has been a key challenge for customers and I've been very vocal about that and they finally launched it. So we are going to use S2 for index. I mean, that's huge.

>> Yeah, it's huge. - A customer who's processing three

Nitin Wagh

>> million indexes, why not use S3? It's so cheap.

>> It's so close to the story.

Nitin Wagh

>> All the launches are amazing. I think this is more than

>> re:Invent, I'm sure like-

>> A lot of meat on the bone on this one. I mean, I've got to say, a very strong... I said to the AgentCore product lead, I said, "If this is the midpoint reveal... I mean, re:Invent, usually they save the best for re:Invent. So if this is a teaser, re:Invent is going to be massive.

Nitin Wagh

>> It's massive. I can't wait to see.

>> Super intelligence coming at the re:Invent.

Nitin Wagh

>> I'm sure they will innovate a lot on Nova. I mean, a lot of the offerings, typically AWS does is like MVP, but it's a surprise that they launched it as a GA offering. I was thinking it's preview, but it looks like everything is GA, which is surprising. So they're moving very fast.

>> Did you get access to some of the betas?

Nitin Wagh

>> Yeah, we got some access to the betas.

>> Yeah, especially Nova, so...

>> Well, then congratulations. We'll keep following you and thanks for coming in and congratulations. Thanks for coming on theCUBE. I'm John Furrier, we're here in our New York Stock Exchange CUBE Studio, of course, bringing all the action here with AWS Summit happening this week in New York. And of course, this is our media week, AI meets cloud. As GenAI starts to go into production, starting to see real use cases and visibility into the economics. The value extraction is starting to be there. Obviously the value creation is full throttle. Again, thanks for watching. We'll be right back.