This discussion examines governance for agentic systems and the operationalization of policies and oversight for artificial intelligence in the enterprise. Navrina Singh of Credo AI appears on theCUBE and NYSE Wired: Mixture of Experts with hosts Gemma Allen, John Furrier and David Vellante.
Singh outlines a six-year mission to build infrastructure of trust for AI, explaining governance as code and the importance of discovery, policy codification and continuous monitoring. They address shadow AI, agent metadata standards and how enterprises translate intentions into enforceable controls across model, dataset and agent inventories. They emphasize integrating governance with security and monitoring, using standards such as A2A agent metadata and balancing capability, control and cost. Market projections and recent shifts underscore growing enterprise demand for verifiable governance in regulated sectors and highlight the relevance of governance as code, AI trust and compliance strategies.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For theCUBE + NYSE Wired: Mixture of Experts Series
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for theCUBE + NYSE Wired: Mixture of Experts Series.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: Mixture of Experts Series
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: Mixture of Experts Series. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Navrina Singh, Credo AI
This discussion examines governance for agentic systems and the operationalization of policies and oversight for artificial intelligence in the enterprise. Navrina Singh of Credo AI appears on theCUBE and NYSE Wired: Mixture of Experts with hosts Gemma Allen, John Furrier and David Vellante.
Singh outlines a six-year mission to build infrastructure of trust for AI, explaining governance as code and the importance of discovery, policy codification and continuous monitoring. They address shadow AI, agent metadata standards and how enterprises translate intentions into enforceable controls across model, dataset and agent inventories. They emphasize integrating governance with security and monitoring, using standards such as A2A agent metadata and balancing capability, control and cost. Market projections and recent shifts underscore growing enterprise demand for verifiable governance in regulated sectors and highlight the relevance of governance as code, AI trust and compliance strategies.
>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, co-host here with David Vellante, my co-host. Welcome back to theCUBE Studio here at the New York Stock Exchange. I'm Gemma Allen, co-host of NYSE Wired's Mixture of Experts. And we are going to have a conversation now about how AI is getting very good at just about everything. It can make decisions, interact with customers, move information, write code, and increasingly act autonomously inside a business. But here's the uncomfortable question: who is actually keeping track of what all these AI systems and agents are allowed to do? Navrina Singh has spent the last 6 years building Credo AI around that very problem, and now she's taking it to the next frontier: governing AI agents as they start to act on our behalf. So when AI goes from giving us answers to actually making decisions, who sets the rules? Navrina, welcome to NYSE Wired.
Navrina Singh
>> Thank you so much for having me, Gemma.
Gemma Allen
>> So, we are certainly at a very interesting time in the world of enterprise tech and enterprise AI, right? Not just from the perspective of tech economics, but from the perspective of risk efficiency. There's a lot of narratives out there that are contested and non-contested. But I think what we can all agree on is that we are entering the next frontier from a risk perspective. Right. We have seen that over the last couple of weeks with Hugging Face, with Mistral, maybe, may or may not be true just last week. you're essentially saying that we need to have very strong, very intelligible governance in place, both technically from the perspective of platform and also from a policy perspective, I guess. Help me unpack that. Help me understand what exactly Credo AI can do for an enterprise.
Navrina Singh
>> Absolutely. So, Gemma, we've been on this quest for the past 6 years to really build the infrastructure of trust because the thesis that we had was as AI systems become extremely capable, which we've seen, from answering questions to now fully making autonomous decisions, what becomes central is how do you ensure that within your organization's goals and intentions, these systems are behaving as they should be? So when you think about governance, governance is not just about risk management and compliance management. They are really important components. But a big part of governance, especially in Fortune 500 companies, is do I know where AI exists? If I know where AI exists, can I put the right intentions and guardrails and policies around it so that it behaves according to my objectives? And when it doesn't, can I intervene? So governance really is how do I define what good looks like for my business and operationalize that? So where this confluence of policy and technical capability comes in is, as you can imagine, and we've seen this especially in the last week, Do you need a federal framework or do you need independent evaluations or should you be doing your own testing and evaluations yourself? So there's a spectrum of trust that is getting created from the responsibility that you as a foundation model provider have to you as an independent evaluator have to you as an enterprise have as you are building capabilities around these frontier models. And so within all those different parameters of trust, you have to define those policies and operationalize it. That is what Credo AI does is governance as code. We take your intention, we codify that into controls, and we make sure wherever your AI is being built, bought, and deployed, you're guiding those systems appropriately.
Gemma Allen
>> I mean, if we are to take the philosophy of Jensen, right, he thinks that companies, frontier labs, and also enterprises should govern themselves, right? First and foremost, that was kind of his key takeaway from that, right? Regulation debate last week. But when we think about governing yourself as an enterprise, not just in the world of AI, but over the last 15 to 20 years, right? A lot of it is actually lost at the discovery phase alone. There are a lot of enterprises that don't fully understand the scope of their environments from a tech perspective, right? Because you see so much scope creep. we saw that in the move to cloud. That alone was a challenge, right? Folks need to understand what exactly they had and what they could migrate.
Navrina Singh
>> Yeah.
Gemma Allen
>> When we think about AI, talk to me a little bit about what sort of discovery is happening and what sort of rogue activity is happening. we have folks come on, they say we have builders and they're using these AI-generated tools to create and they want API keys and they're creating some phenomenal content. And I'm like, wow, there's also a phenomenal level of risk there.
Navrina Singh
>> Yeah.
Gemma Allen
>> So help me understand what's actually happening inside enterprise? Are companies trying to figure out, okay, what exactly is in scope here and who's building what and where?
Navrina Singh
>> Yeah, so let's unpack that. So one, on Jensen's comment, I think there's a spectrum of trust that everyone's sort of missing, and it's really important to nail that down. I don't think the answer is just testing and evaluation that an organization does or having a regulatory framework. There's an entire set of things that you need to be doing to make sure that these highly capable systems actually reap the benefits and the ROI that we all are expecting. And we can unpack that a little bit more in a bit. But what's really happening within an enterprise is a great question because enterprises are not just taking in these frontier capabilities, they are asking themselves within the context of my use, if I'm a healthcare organization and I'm using agents to really help with patient care decisioning, that's a very different set of guardrails than if I'm gonna be using these frontier models or even open source models to build a marketing application or to build a customer service application. So the first thing that's happening when the frontier hits enterprise is context becomes paramount because within the context you can actually define what that good looks like. So let's take this healthcare example. You've built a patient-serving agent and this patient-serving agent is going to give you some guidelines on what kind of healthcare benefits you have. The question a business is asking is, what is this agent allowed to do? Can it just guide the patient around all its healthcare benefits? Can it actually prescribe those benefits? Should it be accessing patients' private information or should it not be? So all those decisions around access, identity, what it is allowed to do when it does do something which is outside the bounds. Should you escalate? Should you block? Should you allow it to happen? All those configurations basically come in from a policy. And so Credo AI is trying to translate those policies into actual configurations that these agents can follow and we can take appropriate actions and intervene when the agents are performing anything outside of it or outside of that.
Gemma Allen
>> So if I'm correct, this is essentially a command center, right? It gives you full visibility over what's happening across your entire stack, your environment, let's say, in the world of enterprise tech. It also gives you a certain amount of control. Does it give you the opportunity to— I don't want to call it a kill switch, but can it help you prevent potential challenges or threat vectors? It's not a cybersecurity play per se, right? It's one step back from that. Help me understand how exactly the technology works. Where does it draw the line, I guess?
Navrina Singh
>> Yeah, great question, Gemma. And this is where the market landscape is not only very vibrant but also getting confusing. So governance, or as you describe, command center, is basically that layer of trust that sits on top of your data security stack. So we have to be able to orchestrate across everything like Azure AI Foundry, all the way to maybe security that you might be getting from Splunk (a Cisco company). And we aggregate all that information to really understand one, Where is AI, whether it's shadow or sanctioned AI, being used in your organization? So this is where we'll plug into your security tooling to say we've just discovered some agents that Gemma built. Should these be sanctioned to use within the context of New York Stock Exchange? Right after that discovery, Credo AI, because of the signals we are able to get from your agent platforms, we can do first level of risk analysis. What kind of risk this agent, potentially can cause. Is it an identity risk? Is it potentially a fairness issue because the agent hasn't baked in demographic parity? So we do that initial risk analysis. And then the third thing, which is really important is— and then we were just discussing this— when you think about multinational organizations, you are deploying agents all over the world. But when they enter the boundaries of those different countries, new set of regulations, a new set of standards now apply to it. So Credo AI is now able to surface for this agent, for the system that you've built. Here are the set of requirements you absolutely need to meet if you're launching the system in Europe, if you're launching the system in Australia. And then we do this continuously. And this is where we have to hook into monitoring platforms because once you've launched the agent, how do you make sure that all the policies that you've provided to that agent are actually happening in the operational realms. So we'll hook into your monitoring platforms. We are taking signals back from those monitoring platforms to really see did the agent at any point violate the conditions that we have configured it for and then take appropriate action.
Gemma Allen
>> So I'm going to ask you a very— what might seem like a basic question, but help me understand this. Let's say healthcare. You're a nurse. You have a Muse. Okay. We heard a lot about Muse this week. You say to it, hey, go in and check the roster and see if there's any overtime opportunities for me. That system is interacting with, your work system to a certain level of credentials and authentication, right? What is the unique authentication by which you track Agentic and agent activity? Is it on a form factor device? Is it based on a domain? How are you actually really finding and catching these operators, these actors within environments?
Navrina Singh
>> Very difficult problem. So one of the things to again distinguish is we are not on the threat side. We are on the governance and oversight side. What that means is when you think about a three-layered cake, your organization level, your use cases or entity level, and then your runtime level, what does good look like? That's what governance defines. The actual threat analysis happens in your ops layer, which is the security and the monitoring layer. And that's why there's a better together that needs to happen. But one of the things that I do want to address in your question is what is becoming even more paramount is the need for standardization. So for example, there's a specification that's been created by Google called A2A that defines very specifically what your Agent Card or the metadata that needs to be associated with every agent. Needs to be. This has— okay, is this Gemma's agent? Yes. How do we associate it with Gemma? What is it allowed to access? Which databases it should not be allowed to access? So all the critical metadata that defines what is that role of that agent. So think about, that agent sort of like an employee that you're bringing in. You want to make sure that you are not only onboarding that employee with all the policies that are important for that employee, to be successful, but to be able to effectively onboard and in some cases offboard them if the agent or the employee doesn't perform right. And the only reason you'll be able to offboard or fire that employee is if they've done something wrong. So violation of those policies is something that Credo AI monitors.
Gemma Allen
>> Well, it is certainly an interesting world if you think about just identification alone. But okay, so moving on then, Credo. So you've been in this business before we all were saying ChatGPT, generative AI, you've been in it from the very, very beginning, right? But a lot has changed in a relatively short space of time. Talk about the business model here. Is this usage-based? Is it environment scope? How do you actually go out to an enterprise customer or prospect and say, okay, here is the scope of what we can do for you? Because it seems so— it's pretty monolithic in some respects. And then talk a little bit about what you're seeing from the perspective of TAM. It seems as though right now everyone's claiming that they are going to meet the agentic future tomorrow, right? How real is that? How much is it driving your TAM, Navrina?
Navrina Singh
>> So Gemma, I started on this journey almost 6.5 years ago. We created the AI governance category, and when I started Credo AI, it was a $0 TAM, but there was a lot of 100% conviction behind the idea that as AI capabilities advance, we need to really draw down on our governance debt, which is going to arise from a couple of different reasons. One, because we won't have systems that always have human in the loop. These are going to be fully autonomous systems. And when that happens, how do you bring in the right oversight? Secondly, the AI literacy and capability of individuals providing oversight of these systems will not be able to keep up with everything that's happening in AI. And we've seen recently, just in the last week, everything around recursive self-improvement. If you're using AI to generate more AI, how do you actually provide the right oversight? So a thesis we had 6 and a half years ago was, as these systems advance, how do we ensure that we have the right accountability and oversight built in from the beginning? So it was a $0 TAM right? Now Forrester is projecting it's going to be about a $50 billion market by 2032. So you can see that the market has just shifted because it's not about the AI capabilities anymore. It is about can I trust that capability to work within the context that is important to me? And so as we work with the enterprise clients, what's interesting is there's an equation that's falling into place. It's the capability, it's the cost, and it's the control. And they have to make a trade-off between do I want the best-performing, highly capable models at what cost? As you can imagine, token maxing is no longer a thing right now.
Gemma Allen
>> Thank God.
Navrina Singh
>> Yeah, thank God for that. But it's all about value maxing now, right? So how do I get the maximum value from the capable models, whether they're coming from Frontier or whether they're coming from open source? And then there's a pretty big question around control. How can I trust that these systems are going to behave on my behalf as an organization so that I can continue to engender that trust with the stakeholders, whether it's a customer, whether it's a board, whether it's my employees. So this capability, control, and cost trade-off is something that our customers are actively dealing with, and that's what's causing the TAM to continuously increase.
Gemma Allen
>> And in terms of the business model, how do you— what is the model here? How do you sell this? Is it seats? Is it usage? Is there something different?
Navrina Singh
>> Yeah, no, great question. So Credo AI has a platform and it is based on subscription. So our SaaS platform, we charge based on number of entities you're governing. And an entity for us is— it could be an agent, it could be a use case, it could be a dataset, it could be a model. Because as you think about the AI bill of material, you as an organization are buying a third-party application or building your own agentic system, you can actually unpack that bill of material pretty easily, right? This is my end application and here are the models that are powering it. Here are the datasets that are powering it. Here are the third-party vendors that are powering that particular application. So we charge based on how many entities are you governing, because for us it is getting to the outcome of trust in this system. So we don't charge based on seats or number of users. We want everyone in your organization to be governing these systems. And then lastly, as you can imagine, we serve a lot of regulated clients and we serve a lot of agencies, federal agencies as well. So in addition to our SaaS platform, we also have on-prem deployment, which is becoming even more crucial and critical because everyone has consequential use cases when they are looking at national security to very sensitive use in insurance or financial services. So on-prem becomes really critical. So we have a slightly different business model for on-prem.
Gemma Allen
>> I'm going to ask you a cheeky question. We heard a lot this year about the SaaSpocalypse, right? About how frontier models are going to become this OS layer of enterprise. Do you think any one model is winning and dominating the enterprise? Like Anthropic has certainly marketed itself as the front-runner. What are your thoughts? Do you think that there's a whole lot of everything happening or are you seeing some clear gains for certain players?
Navrina Singh
>> You know, enterprises are a very interesting set of consumers because they want optionality. And so it goes back to the trade-off that I was just mentioning around capability, cost, and control. We are not seeing one winner anymore. We were in the beginning of the year, Anthropic was doing really well around enterprise use. But what we are seeing now is not only, you know, other frontier labs keeping up, but open source really becoming front runner. We are seeing organizations asking themselves the question, do I really need to be paying so much for a frontier lab when I can actually get the same capability at 1/10 the cost from open source? And can I then in that case embed more controls? So I think it's really a question of tradeoffs. No clear winners yet, but there is a very interesting question around how do I get the ROI from my AI, frontier AI use and adoption, but really focused on this trust layer. And trust is becoming the competitive advantage and moat in all the enterprises that we are working with.
Gemma Allen
>> It is so interesting because I can tell you earlier this year we had some very talented sets of people on this show say corporate America will never adopt open-source models. That will never happen. Yeah, I know things are just changing and shifting so quickly. So Navrina, last question to you. Talk about your own business plan, the strategy. You guys have raised a Series A. I'm sure like in every business, burn rates are happening at the speed of sound too, right? We think of token maxing as one such example. But what is ahead for you and the team? It sounds like you certainly have a great inflection point here from the perspective of your timing.
Navrina Singh
>> Yeah, as pioneers, sometimes you can be seen as really stupid because we were so early, so ahead of the market. But our moment has really come in the past 2 years and especially this year. We are just coming out of our third record-breaking quarter. And what we are seeing is this accelerated demand on the enterprise side because enterprise leaders are getting asked this question: Is there an ROI here with the frontier LLMs? Is there an ROI with the AI? So show it to me. And a big part of showing it to me is not just trust us, it's you have to demonstrate through evidence and verification. And that's where Credo AI comes in. So we are expecting more record-breaking quarters ahead of us and hopefully a big fundraise coming up as well.
Gemma Allen
>> Well, Navrina, I certainly wish you all the best. Hope to see you back on NYSE Wired again in the not-too-distant future.
Navrina Singh
>> Yeah, thank you so much for having me, Gemma.
Gemma Allen
>> I'm Gemma Allen here at theCUBE Studio at the New York Stock Exchange. This is NYSE Wired's Mixture of Experts. Thanks for watching.
>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, co-host here with David Vellante, my co-host. Welcome back to theCUBE Studio here at the New York Stock Exchange. I'm Gemma Allen, co-host of NYSE Wired's Mixture of Experts. And we are going to have a conversation now about how AI is getting very good at just about everything. It can make decisions, interact with customers, move information, write code, and increasingly act autonomously inside a business. But here's the uncomfortable question: who is actually keeping track of what all these AI systems and agents are allowed to do? Navrina Singh has spent the last 6 years building Credo AI around that very problem, and now she's taking it to the next frontier: governing AI agents as they start to act on our behalf. So when AI goes from giving us answers to actually making decisions, who sets the rules? Navrina, welcome to NYSE Wired.
Navrina Singh
>> Thank you so much for having me, Gemma.
Gemma Allen
>> So, we are certainly at a very interesting time in the world of enterprise tech and enterprise AI, right? Not just from the perspective of tech economics, but from the perspective of risk efficiency. There's a lot of narratives out there that are contested and non-contested. But I think what we can all agree on is that we are entering the next frontier from a risk perspective. Right. We have seen that over the last couple of weeks with Hugging Face, with Mistral, maybe, may or may not be true just last week. you're essentially saying that we need to have very strong, very intelligible governance in place, both technically from the perspective of platform and also from a policy perspective, I guess. Help me unpack that. Help me understand what exactly Credo AI can do for an enterprise.
Navrina Singh
>> Absolutely. So, Gemma, we've been on this quest for the past 6 years to really build the infrastructure of trust because the thesis that we had was as AI systems become extremely capable, which we've seen, from answering questions to now fully making autonomous decisions, what becomes central is how do you ensure that within your organization's goals and intentions, these systems are behaving as they should be? So when you think about governance, governance is not just about risk management and compliance management. They are really important components. But a big part of governance, especially in Fortune 500 companies, is do I know where AI exists? If I know where AI exists, can I put the right intentions and guardrails and policies around it so that it behaves according to my objectives? And when it doesn't, can I intervene? So governance really is how do I define what good looks like for my business and operationalize that? So where this confluence of policy and technical capability comes in is, as you can imagine, and we've seen this especially in the last week, Do you need a federal framework or do you need independent evaluations or should you be doing your own testing and evaluations yourself? So there's a spectrum of trust that is getting created from the responsibility that you as a foundation model provider have to you as an independent evaluator have to you as an enterprise have as you are building capabilities around these frontier models. And so within all those different parameters of trust, you have to define those policies and operationalize it. That is what Credo AI does is governance as code. We take your intention, we codify that into controls, and we make sure wherever your AI is being built, bought, and deployed, you're guiding those systems appropriately.
Gemma Allen
>> I mean, if we are to take the philosophy of Jensen, right, he thinks that companies, frontier labs, and also enterprises should govern themselves, right? First and foremost, that was kind of his key takeaway from that, right? Regulation debate last week. But when we think about governing yourself as an enterprise, not just in the world of AI, but over the last 15 to 20 years, right? A lot of it is actually lost at the discovery phase alone. There are a lot of enterprises that don't fully understand the scope of their environments from a tech perspective, right? Because you see so much scope creep. we saw that in the move to cloud. That alone was a challenge, right? Folks need to understand what exactly they had and what they could migrate.
Navrina Singh
>> Yeah.
Gemma Allen
>> When we think about AI, talk to me a little bit about what sort of discovery is happening and what sort of rogue activity is happening. we have folks come on, they say we have builders and they're using these AI-generated tools to create and they want API keys and they're creating some phenomenal content. And I'm like, wow, there's also a phenomenal level of risk there.
Navrina Singh
>> Yeah.
Gemma Allen
>> So help me understand what's actually happening inside enterprise? Are companies trying to figure out, okay, what exactly is in scope here and who's building what and where?
Navrina Singh
>> Yeah, so let's unpack that. So one, on Jensen's comment, I think there's a spectrum of trust that everyone's sort of missing, and it's really important to nail that down. I don't think the answer is just testing and evaluation that an organization does or having a regulatory framework. There's an entire set of things that you need to be doing to make sure that these highly capable systems actually reap the benefits and the ROI that we all are expecting. And we can unpack that a little bit more in a bit. But what's really happening within an enterprise is a great question because enterprises are not just taking in these frontier capabilities, they are asking themselves within the context of my use, if I'm a healthcare organization and I'm using agents to really help with patient care decisioning, that's a very different set of guardrails than if I'm gonna be using these frontier models or even open source models to build a marketing application or to build a customer service application. So the first thing that's happening when the frontier hits enterprise is context becomes paramount because within the context you can actually define what that good looks like. So let's take this healthcare example. You've built a patient-serving agent and this patient-serving agent is going to give you some guidelines on what kind of healthcare benefits you have. The question a business is asking is, what is this agent allowed to do? Can it just guide the patient around all its healthcare benefits? Can it actually prescribe those benefits? Should it be accessing patients' private information or should it not be? So all those decisions around access, identity, what it is allowed to do when it does do something which is outside the bounds. Should you escalate? Should you block? Should you allow it to happen? All those configurations basically come in from a policy. And so Credo AI is trying to translate those policies into actual configurations that these agents can follow and we can take appropriate actions and intervene when the agents are performing anything outside of it or outside of that.
Gemma Allen
>> So if I'm correct, this is essentially a command center, right? It gives you full visibility over what's happening across your entire stack, your environment, let's say, in the world of enterprise tech. It also gives you a certain amount of control. Does it give you the opportunity to— I don't want to call it a kill switch, but can it help you prevent potential challenges or threat vectors? It's not a cybersecurity play per se, right? It's one step back from that. Help me understand how exactly the technology works. Where does it draw the line, I guess?
Navrina Singh
>> Yeah, great question, Gemma. And this is where the market landscape is not only very vibrant but also getting confusing. So governance, or as you describe, command center, is basically that layer of trust that sits on top of your data security stack. So we have to be able to orchestrate across everything like Azure AI Foundry, all the way to maybe security that you might be getting from Splunk (a Cisco company). And we aggregate all that information to really understand one, Where is AI, whether it's shadow or sanctioned AI, being used in your organization? So this is where we'll plug into your security tooling to say we've just discovered some agents that Gemma built. Should these be sanctioned to use within the context of New York Stock Exchange? Right after that discovery, Credo AI, because of the signals we are able to get from your agent platforms, we can do first level of risk analysis. What kind of risk this agent, potentially can cause. Is it an identity risk? Is it potentially a fairness issue because the agent hasn't baked in demographic parity? So we do that initial risk analysis. And then the third thing, which is really important is— and then we were just discussing this— when you think about multinational organizations, you are deploying agents all over the world. But when they enter the boundaries of those different countries, new set of regulations, a new set of standards now apply to it. So Credo AI is now able to surface for this agent, for the system that you've built. Here are the set of requirements you absolutely need to meet if you're launching the system in Europe, if you're launching the system in Australia. And then we do this continuously. And this is where we have to hook into monitoring platforms because once you've launched the agent, how do you make sure that all the policies that you've provided to that agent are actually happening in the operational realms. So we'll hook into your monitoring platforms. We are taking signals back from those monitoring platforms to really see did the agent at any point violate the conditions that we have configured it for and then take appropriate action.
Gemma Allen
>> So I'm going to ask you a very— what might seem like a basic question, but help me understand this. Let's say healthcare. You're a nurse. You have a Muse. Okay. We heard a lot about Muse this week. You say to it, hey, go in and check the roster and see if there's any overtime opportunities for me. That system is interacting with, your work system to a certain level of credentials and authentication, right? What is the unique authentication by which you track Agentic and agent activity? Is it on a form factor device? Is it based on a domain? How are you actually really finding and catching these operators, these actors within environments?
Navrina Singh
>> Very difficult problem. So one of the things to again distinguish is we are not on the threat side. We are on the governance and oversight side. What that means is when you think about a three-layered cake, your organization level, your use cases or entity level, and then your runtime level, what does good look like? That's what governance defines. The actual threat analysis happens in your ops layer, which is the security and the monitoring layer. And that's why there's a better together that needs to happen. But one of the things that I do want to address in your question is what is becoming even more paramount is the need for standardization. So for example, there's a specification that's been created by Google called A2A that defines very specifically what your Agent Card or the metadata that needs to be associated with every agent. Needs to be. This has— okay, is this Gemma's agent? Yes. How do we associate it with Gemma? What is it allowed to access? Which databases it should not be allowed to access? So all the critical metadata that defines what is that role of that agent. So think about, that agent sort of like an employee that you're bringing in. You want to make sure that you are not only onboarding that employee with all the policies that are important for that employee, to be successful, but to be able to effectively onboard and in some cases offboard them if the agent or the employee doesn't perform right. And the only reason you'll be able to offboard or fire that employee is if they've done something wrong. So violation of those policies is something that Credo AI monitors.
Gemma Allen
>> Well, it is certainly an interesting world if you think about just identification alone. But okay, so moving on then, Credo. So you've been in this business before we all were saying ChatGPT, generative AI, you've been in it from the very, very beginning, right? But a lot has changed in a relatively short space of time. Talk about the business model here. Is this usage-based? Is it environment scope? How do you actually go out to an enterprise customer or prospect and say, okay, here is the scope of what we can do for you? Because it seems so— it's pretty monolithic in some respects. And then talk a little bit about what you're seeing from the perspective of TAM. It seems as though right now everyone's claiming that they are going to meet the agentic future tomorrow, right? How real is that? How much is it driving your TAM, Navrina?
Navrina Singh
>> So Gemma, I started on this journey almost 6.5 years ago. We created the AI governance category, and when I started Credo AI, it was a $0 TAM, but there was a lot of 100% conviction behind the idea that as AI capabilities advance, we need to really draw down on our governance debt, which is going to arise from a couple of different reasons. One, because we won't have systems that always have human in the loop. These are going to be fully autonomous systems. And when that happens, how do you bring in the right oversight? Secondly, the AI literacy and capability of individuals providing oversight of these systems will not be able to keep up with everything that's happening in AI. And we've seen recently, just in the last week, everything around recursive self-improvement. If you're using AI to generate more AI, how do you actually provide the right oversight? So a thesis we had 6 and a half years ago was, as these systems advance, how do we ensure that we have the right accountability and oversight built in from the beginning? So it was a $0 TAM right? Now Forrester is projecting it's going to be about a $50 billion market by 2032. So you can see that the market has just shifted because it's not about the AI capabilities anymore. It is about can I trust that capability to work within the context that is important to me? And so as we work with the enterprise clients, what's interesting is there's an equation that's falling into place. It's the capability, it's the cost, and it's the control. And they have to make a trade-off between do I want the best-performing, highly capable models at what cost? As you can imagine, token maxing is no longer a thing right now.
Gemma Allen
>> Thank God.
Navrina Singh
>> Yeah, thank God for that. But it's all about value maxing now, right? So how do I get the maximum value from the capable models, whether they're coming from Frontier or whether they're coming from open source? And then there's a pretty big question around control. How can I trust that these systems are going to behave on my behalf as an organization so that I can continue to engender that trust with the stakeholders, whether it's a customer, whether it's a board, whether it's my employees. So this capability, control, and cost trade-off is something that our customers are actively dealing with, and that's what's causing the TAM to continuously increase.
Gemma Allen
>> And in terms of the business model, how do you— what is the model here? How do you sell this? Is it seats? Is it usage? Is there something different?
Navrina Singh
>> Yeah, no, great question. So Credo AI has a platform and it is based on subscription. So our SaaS platform, we charge based on number of entities you're governing. And an entity for us is— it could be an agent, it could be a use case, it could be a dataset, it could be a model. Because as you think about the AI bill of material, you as an organization are buying a third-party application or building your own agentic system, you can actually unpack that bill of material pretty easily, right? This is my end application and here are the models that are powering it. Here are the datasets that are powering it. Here are the third-party vendors that are powering that particular application. So we charge based on how many entities are you governing, because for us it is getting to the outcome of trust in this system. So we don't charge based on seats or number of users. We want everyone in your organization to be governing these systems. And then lastly, as you can imagine, we serve a lot of regulated clients and we serve a lot of agencies, federal agencies as well. So in addition to our SaaS platform, we also have on-prem deployment, which is becoming even more crucial and critical because everyone has consequential use cases when they are looking at national security to very sensitive use in insurance or financial services. So on-prem becomes really critical. So we have a slightly different business model for on-prem.
Gemma Allen
>> I'm going to ask you a cheeky question. We heard a lot this year about the SaaSpocalypse, right? About how frontier models are going to become this OS layer of enterprise. Do you think any one model is winning and dominating the enterprise? Like Anthropic has certainly marketed itself as the front-runner. What are your thoughts? Do you think that there's a whole lot of everything happening or are you seeing some clear gains for certain players?
Navrina Singh
>> You know, enterprises are a very interesting set of consumers because they want optionality. And so it goes back to the trade-off that I was just mentioning around capability, cost, and control. We are not seeing one winner anymore. We were in the beginning of the year, Anthropic was doing really well around enterprise use. But what we are seeing now is not only, you know, other frontier labs keeping up, but open source really becoming front runner. We are seeing organizations asking themselves the question, do I really need to be paying so much for a frontier lab when I can actually get the same capability at 1/10 the cost from open source? And can I then in that case embed more controls? So I think it's really a question of tradeoffs. No clear winners yet, but there is a very interesting question around how do I get the ROI from my AI, frontier AI use and adoption, but really focused on this trust layer. And trust is becoming the competitive advantage and moat in all the enterprises that we are working with.
Gemma Allen
>> It is so interesting because I can tell you earlier this year we had some very talented sets of people on this show say corporate America will never adopt open-source models. That will never happen. Yeah, I know things are just changing and shifting so quickly. So Navrina, last question to you. Talk about your own business plan, the strategy. You guys have raised a Series A. I'm sure like in every business, burn rates are happening at the speed of sound too, right? We think of token maxing as one such example. But what is ahead for you and the team? It sounds like you certainly have a great inflection point here from the perspective of your timing.
Navrina Singh
>> Yeah, as pioneers, sometimes you can be seen as really stupid because we were so early, so ahead of the market. But our moment has really come in the past 2 years and especially this year. We are just coming out of our third record-breaking quarter. And what we are seeing is this accelerated demand on the enterprise side because enterprise leaders are getting asked this question: Is there an ROI here with the frontier LLMs? Is there an ROI with the AI? So show it to me. And a big part of showing it to me is not just trust us, it's you have to demonstrate through evidence and verification. And that's where Credo AI comes in. So we are expecting more record-breaking quarters ahead of us and hopefully a big fundraise coming up as well.
Gemma Allen
>> Well, Navrina, I certainly wish you all the best. Hope to see you back on NYSE Wired again in the not-too-distant future.
Navrina Singh
>> Yeah, thank you so much for having me, Gemma.
Gemma Allen
>> I'm Gemma Allen here at theCUBE Studio at the New York Stock Exchange. This is NYSE Wired's Mixture of Experts. Thanks for watching.
Gemma Allen