Media Week NYC theCUBE + NYSE Wired | Pamela Gupta, Trusted AI

Clips
More from Media Week NYC theCUBE + NYSE Wired

Pamela Gupta

CFO

Trusted AI

play_circle_outline Unlocking Trustworthy AI: De-Risking Adoption and Identifying Key Factors for Trustworthiness

play_circle_outline Shift from predictive AI to generative AI poses new risk factors

play_circle_outline Optimizing AI Strategy: Cultural Workforce Involvement and Risk Assessment for Enhanced Adoption Success

play_circle_outline Red teaming AI models and data sets for risk management

play_circle_outline Need mechanisms to handle risks identified by AI, involving experts and strategic planning.

play_circle_outline Building Trustworthy AI with Passionate Hires for Strategic Adoption and Deployment

Info
Transcript

Pamela Gupta, Trusted AI

Pamela Gupta

CFO Trusted AI

TheCube is at theCube East in New York City, the new studio set by the New York Stock Exchange in partnership with NYSC and NYSE WIRED community. CEO of Trusted AI, Pamela Gupta, discusses the importance of trustworthy AI and the eight essential pillars she identified for de-risking AI adoption. She emphasizes the need for organizations to understand and address the risks associated with AI strategically. The conversation covers the evolving nature of AI, particularly with generative AI, and the alignment of AI strategies with business goals to build trust an... Read more

explore Keep Exploring

What are the eight essential pillars of trustworthy AI according to the speaker's definition? add

What are the differences between predictive AI and generative AI, particularly in terms of threat models and threat vectors according to data scientists? add

What are the steps involved in conducting an AI risk assessment and algorithmic impact assessment? add

What is the target audience for Trusted AI in terms of risk management and who is responsible for red teaming data sets in cybersecurity? add

What considerations need to be taken into account when developing mechanisms for handling policy or risk variables in AI systems to prevent potential long-term consequences? add

What qualities are you looking for in individuals interested in adopting AI technology and how can they reach out to you for potential opportunities? add

bolt Powered by CUBE AI

Pamela Gupta, Trusted AI

search

>> Hello everyone, welcome back to theCube. We are here in New York City at theCube East, our New York Stock Exchange new studio set in partnership with NYSC, in collaboration with NYSE WIRED community. theCUBE is here. I'm John Furrier, your host. Behind us is all the action, bringing Wall Street and Silicon Valley together. Pamela Gupta here, she's the CEO of Trusted AI. One of the top conversations this week, it's been climate week, it's been UN General Assembly. And again, one of the topics that's been consistent across all has been governance. How do you handle AI? And then private AI is weaved into the conversation. At the end of the day, it's all about the data and trust. Pamela, thanks for coming on theCube.

Pamela Gupta

>> My pleasure. Thank you.

>> So can you talk about your company Trusted AI, what you guys do? Let's set the context. Give us a little bit of a start here.

Pamela Gupta

>> So what we're doing is helping companies de-risk their adoption of AI by identifying what are the trustworthiness factors that they need for de-risking. So to give you an example, five years ago I defined eight essential pillars of trustworthy AI, which actually coincide with what NIST had created a few years later on. And this is a practitioner as well as a strategic approach to how do we assemble a actionable strategy to define the risk by context and address them. So what are those eight pillars? Security and privacy, right at the foundation, because that is really the essence and essential. And then we go into more AI specific domains such as transparency, are you interacting with a automation system or is it an automated decision, et cetera, so are you interacting with the computer, explainability, so those are specific to AI. And then it goes more into the actionable components, audit, accountability, regulations and bias.

>> Those are the eight pillars.

Pamela Gupta

>> Those are the eight pillars.

>> Okay. So from privacy to bias seems to be the spectrum.

Pamela Gupta

>> Correct.

>> When did you put that together?

Pamela Gupta

>> This was in 2019.

>> Okay. Now let me ask you a question before we start diving in. How much has changed? I saw the latest NIST report, not bad, really good, they're doing a good job. So you're starting to see standards recommendations coming in. Obviously, now security fields being implemented, guardrails as well. Everyone's talking about guardrails. How much has changed since the original eight pillars in your mind? What's the big action?

Pamela Gupta

>> Yeah, that's interesting too, let's take a look. So what has stayed constant is those eight pillars. Nothing has changed in terms of these can pose a risk, and how are we going to address them? That has not changed. What has changed is the going from predictive AI, or those decision banking systems to now generative AI. And even if you talk to data scientists for example, I was just at the Pentagon and just data scientists in general even, the threat model and the threat vectors for both those types of AI are not the same, so that's what's different. And then we'll talk more about why can that be a problem?

>> I really think you're bringing up one of the most critical points that I'm seeing in our reporting in our CUBE research team has looked at, is that you hit the nail on the head with the comment predictive is different than generative. And the reason why is that in all the verticals and everyone where they're adopting AI and transitioning to this new wave, everything's changing both on the back end and the front end, user experience changing on the back end. But what's interesting is that if you can't just adopt a new IT strategy without culturally nailing the processes. You can't change the processes and the user experience without understanding the back end implications of scale because the role of data is involved. This is first time in my career I've never seen many cycles of innovation, both of the back end and the front end, both really changing radically at this point in time. And I think predictive was different and good, but different than this they're not mutually exclusive either. And so okay, this gen AI is exploding those eight pillars in action is more activity, because the new thing is here, but the old didn't go away either. So people aren't grokking that. And this is I think a very nuanced point, but you can't just kill the old to bring in the new. It's actually not going away, it's more just as relevant than the new one. So this is a cultural moment. Share your thoughts on this if you have any, because this is what I find to be the consistent pattern in process, and overall transformation is that it isn't this or that, it's both. And if you don't do both, you can't capitalize by preserving the best of the existing while bringing in the new.

Pamela Gupta

>> Yes. So that's why our focus on de-risking, because the risk is going to be different, and the implementation is going to be different, and then there is that fear of missing out, that adoption ramping up across the board. Show us how we can use this. So I call it a lot of the it's tail wagging the dog, because a lot of times there's that fear of missing out, so you're like, "We have this data, what can we do? Show us what we can do and do it fast." So those old concepts of fast fail, fail fast, et cetera, those are not going to work the same way. That culture you talked about seeing a lot of hostility in companies right now between the workforce because they don't want to adopt AI, and the field it's going to be replacing them. So we are seeing a lot out there, which is beyond the risk of the technology, and not fully getting a handle on that is that culture, and that adoption within the company of what do we need to do? How do we do it? And so we are trying to solve for that along with the industry. And just to quickly touch on what that means is when we are looking at companies, we're looking at upskilling the labor professional, so they don't feel their job's going to be done better by a machine, but it's how do we use AI to do our job better? And that's really a critical point in a lot of companies too.

>> One of the things I see too, and to your point is there's a lot of weight washing too, they jump on the bandwagon, "We're private AI." I think private AI is legit in my opinion. I think people should look at AI from the lens of a power law. You're going to have popular open public things, and then specialty models that will be either public or private. And I think there's a lot of discussion around data protection. Again, back from old data protection mechanisms to new data protection mechanisms hit those pillars. So obviously a big discussion, which pillar of the eight in your opinion is the most robust in discussion? Because remember there's a fear of missing out, also a fear of getting breached or fear of loosing. Not winning out, but my risk management might blow up, I might get screwed over, so I might miss out on fear. Half the group's missing out, fear of missing out, the other one's half are getting scared, to get a risk management piece. So which of the pillars is the most explosive and disruptive enablement or conversation?

Pamela Gupta

>> So I would say accountability. If we start with that, that is a fundamental thing. If you are developing something who at the end owns the responsibility. And so that's one thing we start with is taking a look at defining the stakeholders within a company of who should be contributing to that. So I would say as a pillar, that is easier to obtain, right? Because you can if you are driven towards it, you can obtain that quite easily. Who should be involved, is it the business team? Is it the legal team or the privacy team? The security team, the data science team, who needs to be involved? And that could be across different verticals, different industry could be easily identified.

>> Go ahead, go ahead, continue.

Pamela Gupta

>> Where it becomes more of a challenge is not having the expertise to recognize what the other pillars are. For example, security. I'll tell you a funny story. The reason I even started on trustworthy AI is because I come from a background of cyber security, and I was trying to see what do we need to do to secure AI. And then I realized that as I started doing more research, there was a lot of material on how to use cyber security for AI, or AI for cyber security, but not how to secure AI. And I realized that cyber security by itself is not enough to secure AI, and even if you just take cyber security, that is very different from conventional system security. You just think like data provenance, where's your data coming from? Application security, not even part of security teams for a lot of companies. So just even handling cyber security for AI, that is something that cannot be skipped over, and is extremely important. Ray AI framework-

>> Supply chain by the way, again, that's data, not lineage data, where did it come from? How do you explain it to your pillar?

Pamela Gupta

>> .

>> That could be hacked.

Pamela Gupta

>> Right. Right. So there are so many additional vectors of risk that we really cannot do it in silos. So those eight pillars are good, but we have to look at them to get-

>> Pamela, I got to say if I'm the customer, pretend I'm the customer, or someone looking for help, my mind's blown. It's just like mind-blown, "Oh my god, there's so many vectors of attack. What do I do? Where do I start?" So I think there's a lot of education and I always have a phrase, you don't play chess unless what checkmate looks like. You got to know what you're dealing with before you can understand it. You can't just jump in and throw words around and cliches or consultants, you got to get your hands dirty with it. So what's your advice? Are people doing that? NYSE, Anand runs the center of excellence, and he and I had that conversation. He's working through practical examples so he understands it better. And so there's a lot of that going on, but it's hard. What do you advise people? Because people want Trusted AI, but their minds are blowing, "They're like too many things, where do I start? How do I prioritize? How do I even understand it?" So what's your advice here and what's your opinion?

Pamela Gupta

>> So the first thing we start with is really doing that risk assessment. What is the AI risk assessment? What is the algorithmic impact assessment? What is the company vision and what is the intended outcome? What are you looking to do? And from that defining who needs to be involved, who needs that seat at the table to define what goes into that? What should that end product look like? And at the end of the day, what are the risks that can affect you from achieving that? So that is going to be contextual. If it's in healthcare, regardless of there's critical infrastructure, whatever it may be, it's extremely contextual. But to your point as to where do you start, it's by knowing what you are trying to achieve. And then out of that, what is the uncertainty that you need to be prepared for? So out of those eight pillars, that'll be really important to see is there a privacy concern, corral all this data, aggregate all this data and then create a model. Are we at risk for harm coming from data exposure of what we have seen? We've seen algorithmic failures already. As we are building the ship, we're trying to steer the direction, we're trying to fix it. But in terms of how do we do this, it's really simple actually. You do your inventory of what are those AI, do your AI inventory right at the start, where is it internal or external? And what kind of data is involved, data protection, and data security, privacy, extremely important. So nothing different there. So we've done this, we can do this. But the only thing to watch out for is what is specific to AI? And that for example, such as generative AI. Just to give you one example, when we're looking at even threat vectors or security risk factors, when it comes to data that can be created based on probability. So these are all statistical models, generative AI, how it's producing output, it doesn't have any idea of context, but we as humans do, so we have to take that into account. Even if you set up a red teaming plan for testing, is your system secure? That's a very small part of how it can be a risk for you, because even in creating that red teaming plan, or security testing, or whatever the risk might be the pillar that you're looking at, and is important for you, if you put in a high risk environment then your calculations change. So you really cannot put a system that can fabricate information and come to decisions, and be a part of a automated decision-making system. So those are the kind of things.

>> You bring up a good use case there. And I think that's illustrated the potential downside scenario of not testing at scale, or red teaming the models and AI, because that's where it could really go off the rails. So most people don't think of red teaming as an in AI issue, that's mostly a security issue. Are people doing that now? Are they red teaming data sets? And who does that risk management? Because again cyber is interesting, and you come from a cyber background, I see why they all connect because it's red teaming, and the risk management is elevated itself so much in cyber that this falls right into the data. Cyber security is a data risk management problem. At the end of the day there's a lot of things going on, but now AI is both. So it's kind of a cyber problem set, but gen of AI is just another application. So we've seen the AppSec application security movie before. So this is interesting, who's the target for Trusted AI, the CFO, the CISO, CEO, board? Do you see a pattern emerging there? Because certainly everyone wants to be de-risking. So you had me at de-risking from the customer.

Pamela Gupta

>> So it's really essential to have those right stakeholders in that conversation because what I like to tell the people I'm working with is this is not us against different teams, or different teams can do this alone. We really have to figure out where, and sometimes we even be testing for the same things. Making sure their systems are staying on track, and then we haven't even covered other pillars like regulations for example, or audit. Audit is a completely different, it has implications which are vastly different from the existing way of auditing systems. We didn't have to do this kind of step-by-step, what decisions are we making, how do we track all that? Because it is learning on data, so it is going to keep changing its dynamic. So I think the more complex something is, and complexity as you know is the enemy of risk, is the enemy of security, so the more we look to simplify it. And that is by creating that center of excellence, which goes with what is the business strategy, what is the business outcomes that you want, what are the trust KPIs that you need? And then the other very important part is how do you bring the rest of the workforce into it, so that they can do their work better and they don't feel that this is going to be replacing them. Because I've seen immense hostility in some companies where that would be a big roadblock to you creating the right and effective solution, because the company personnel are not contributing the right information to go into it to make it more robust here.

>> I think you nailed the cultural aspect there. That's awesome. I think that's key culture of the workforce, the users. Because users in the loop is going to be key as reasoning steps up. Causal AI is right around the corner. We're right now dealing with probability outcomes. Talk about the company Trusted AI. Let's get into what you guys are doing. We'll use the rest of our time here on the company that you're the CEO of, what's your mission? Who are you're talking with? What are some of your customers? Who's working with you? Who's the customer, and what are the sort of the work products that you guys are doing?

Pamela Gupta

>> So we are pure consulting company. We have a lot of company strategic partners such as platform providers of AI, risk management platform providers, those who are doing auditing for. But we are a pure strategy creation company. We want people, organizations, we want to teach them how to fish, we are not catching fish for them. And so some of the things that I'm mentioning that is coming directly from working with customers, figuring out where is that roadblock going to come from besides... Absolutely no negotiation on what are those risks, and how those need to be process mapped out, but then creating that strategy and that process for the company, so that they can have that capability in place from the right people involved, the technology involved people, and process involved. And what we do is we create a certification seal, which is not for by system, by system. It's for saying that, "Yes, we work with them, we have a process and capability in place in the organization, where they can take in any AI systems that are internal or external because external vendors," were deployed. So how do you have the right risk assessment and management process in place?

>> I love your phrase I don't teach them how to fish, I don't fish for them. Classic professional services motion, congratulations with the great service. My question is not to make fun of the fishing example. I use it all the time. Do they even know what fish looks like? And do you have to take them out and show them where the fish are? That's where they are, now do it.

Pamela Gupta

>> All the time.

>> Because we're in a new world where I don't know the difference between the fish, are they the right fish? So this is a real part of this why I was saying we're getting into this practitioner. That's why I like the center of excellence approach as a stepping stone to building the teams and the culture and the mechanisms. What's their view on the fish? Do they say what's a fish, or when do we fish? How do we fish?

Pamela Gupta

>> Sometimes it is an eye-opening moment where it was, "Oh, you mean this can be a risk? There are risks involved?"

>> You shouldn't have said that, now I have to go through the process.

Pamela Gupta

>> Right. And then they're all ears, then you have your attention once you show them what is a very contextual, what is different between... Sometimes you'll hear things like we already have policy, we have a governance process.

>> We're all good.

Pamela Gupta

>> We're good. And no, but there are a lot of differences, and AI is not one single static system. It's not like deploying a facial recognition system for ID access. It's not one system, it's growing by data, it's learning, it's adapting.

>> It's not a one and done, bye-

Pamela Gupta

>> It's not....

>> and deploy. That's your point.

Pamela Gupta

>> Right.

>> It's not set it, forget it.

Pamela Gupta

>> Absolutely not.

>> This is holistic and also always on because it's generating new things. What's the top areas to get their attention? If you had to stack rank the top few signals that gets their attention on risk, what's the top three that jump out at you that they react to, "Go, whoa, I didn't know that. We're good."

Pamela Gupta

>> I think just to take generative AI as an example, the biggest eye-opener is that, oh, they can be wrong input. Somebody can be trying to put in wrong things. So a malicious input, or even untrained input that can cause really unpredictable and heinous outcome. So for example, whether it's you may have heard some cases the National Eating Disorder Company Society in the US, where deployed a chatbot for giving advice when people call in for their hotline. And earlier they had a chatbot which was static, but then they replaced it with generative AI and somebody called in, or that's what the reports are to say that they had a weight problem and they say, "Well, you should be watching what you eat." And so to give that advice to somebody who's bulimic, who's already could have a deep eating disorder, that's the last thing they need to hear is that they may have somebody validating that you have a eating disorder, and you need to take care of your weight. So those are the kind of things that's a high-risk situation where it's directly interfacing, and you don't know what the system will come up with. So if you put it in a area where it's directly interacting with a customer or a user, then that can be harmful.

>> So this is where AI scales nicely, where you can operationally put these identifier risk . So this is what gets their attention, then the action is you actually have to develop a mechanism for handling that policy, or that risk variable.

Pamela Gupta

>> So there's always an expert involved. And then there's the enterprise side of how do you build systems that you're not putting your intellectual property in, so we go into all that. We will work with the clients and see what is that architecture that they need, what is the design they need. And how do you prevent them from just looking at the short term, and not looking at it strategically, because once you put it in there you really can't pull it back. You can't unravel that.

>> And you got to have teams. Well, Pamela, it's been great conversations. We had a great backdrop here. Final word, look at the camera and give the plug for Trusted AI. Love your mission. It's just early days, it's just beginning, again, you've got a lot of body of work going into this. And now with the pillars activated in the market now up and down that pillar stack, what are you guys doing? What's the pitch? What are you guys looking do, are they're going to hire? Give a plug for the company and your mission.

Pamela Gupta

>> Yes, so absolutely. So what we are looking at doing is helping adopt AI because this is an extremely powerful technology, and we want to get the benefit from it. So we are looking for people who are passionate about leading with AI, want to help people with creating that right people process technology. Because all said and done, that's what's going to help us stay ahead. We have all the stop of AGI and other aspect. First, lets at least get responsible and trustworthiness in our AI system that we are deploying, and know what we are doing. It is going to be with intention. If you are interested in working with us, definitely reach out to me pamela.gupta@trustedai.ai or on LinkedIn. Always hiring and looking at growing, but we are a pure play strategy creation company.

>> Yeah, you are teaching them how to fish, not fishing for the customers. Great advisory. If you are in need of help, trust AI, it's hot. Private AI in the enterprise is a category, and It will be very relevant, we're already seeing early returns on that. And AI in generally can't be ignored, risk management, it hits every aspect of the process in a company. And of course we got to be on it. And we're doing our part here in theCube at NYSC, as we launch our EastCcoast hub here in the NYSE. As we open up our access point, as we call it for the network. Excited to cover the tech scene here in New York, and up and down the East Coast, and connect that to Silicon Valley where our home is, and build that network for the Cube, and meet new people, and fresh voices. The world needs expertise and of course feel free to contact us. My name is John Furrier, host of theCUBE. Thanks for watching.