Databricks Data + AI Summit 2024 | Anand Pradhan | ICE

theCUBE.net

Clips
News
More from Databricks Data + AI Summit 2024

Anand Pradhan

Head of the AI Center of Excellence

ICE

Guardrails for AI: How ICE’s AI Center of Excellence ensures compliance and innovation

Given the growing interest in artificial intelligence for its ability to enhance human capabilities, establishing governance guardrails is crucial, as demonstrated by Intercontinental Exchange’s AI Center of Excellence.As a testing ground for new AI use cases, the AI Center of Excellence plays a key role in the incubation process, ensuring that established guidelines — such as data validation, AI model governance, data governance and distribution — are diligently followed, according to Anand Pradhan (pictured), senior director of technology at the AI Center of Excellence at ICE. Pradhan also runs the NYSE Launchpad Lab

play_circle_outline Enabling AI Advancements: The Impact of ICE's AI Center of Excellence and Anand's NYSE Systems

play_circle_outline Purpose of the AI Center of Excellence as an internal R&D group

play_circle_outline Empowering product and application delivery teams for AI use cases

play_circle_outline Unveiling the Challenges of Regulation and the Crucial Role of Explainable AI in a Regulated Industry: A Deep Dive into Productivity Improvement Use Cases

Info

Anand Pradhan | ICE

Anand Pradhan

Head of the AI Center of Excellence ICE

search

>> Welcome back everyone to theCUBE's live coverage here in San Francisco. I'm John Furrier, host of theCUBE, here with Savannah Peterson. She and I, Rob Strechay and the entire CUBE team is here. We've got Anand, he's a friend of theCUBE, senior director of technology at ICE. ICE is the parent company of the New York Stock Exchange. Anand, great to see you. Thanks for coming onto theCUBE.

Anand Pradhan

>> Thank you. Happy to be here.

>> So, we've known each other for a while and it's the first time on theCUBE. Welcome with me. Thank you for coming on the show.

Anand Pradhan

>> Thank you.

>> Talk about your background because you've told me you built a lot of stuff at the NYSE, a lot of the systems, a lot of the code that power the exchange. ICE has a lot more going on than just the NYSE. It's a lot of things going on the systems there. You just started the AI Center of Excellence at ICE that you're running, and it's got a lot of action going on. So, take us through what you've been building in the past and what is the AI Center of Excellence?

Anand Pradhan

>> Yeah, I started my journey as a technologist more than two decades ago back in India. Worked there for five years, moved to the States as part of the New York Stock Exchange. I was part of the development team. But the interest has always been how the system works at the lowest level, try to understand everything, how to optimize. And that plays a role in the story leading up to the AI. Worked my way up building many of the system that powers the New York Stock Exchange. Played pivotal role in rebuilding the systems. Have been part of the Pillar development platform or trading platform that powers all our exchanges at New York Stock Exchange, as well as market data systems. Truly, a world-class sophisticated distributed computing system, at a lowest latency. And we use InfiniBand, which is basically used in AI training, so we understand a thing or two about low latency, high-volume usage. I also led the team to build the first hybrid cloud infrastructure and application for NYSE. So, I built a system for NYSE surveillance for the regulators. We mine data for 650 billion transactions every single day to find patterns. Once I finished that-

>> How many signals per day?

Anand Pradhan

>> 650 billion.

>> Billion with a B?

Anand Pradhan

>> B.

>> That's a lot.

Anand Pradhan

>> At microsecond-level latency. That means that you have ton of dense nanosecond granularity data, and you have to find patterns in the data set to figure out who is trying to game the market.

>> Seems a lot of vector is involved, a lot of embeddings, a lot of math.

Anand Pradhan

>> So, the embedding and math. So, math definitely is the core of all these things, but embedding comes later on.

>> Continue. I didn't mean to interrupt. We'll come to that.

Anand Pradhan

>> So, basically, once we finished all the data mining and we didn't have the day-to-day operational challenges after the cloud migration, we started focusing on machine learning. So, we build our deep learning models and things like that for the time series data. And then, ChatGPT happened, everybody pivoted towards transformer models. So, we build new sentiment analysis to correlate with the regulatory data or surveillance output that is coming out to figure out how we can auto resolve some of the alerts. That's where Vector Database comes into picture, combination of multiple model, there is a pipeline and things like that. But then, fast-forward right now, I moved to lead the AI Center of Excellence for our parent company to increase our collaboration across the company observability. There are similar use cases. How do we build it right? Make sure we have the accuracy, explainability, and all the things that goes without saying in our work? So, that's my journey.

>> Awesome. So, well, congratulations. It's fascinating and I can't wait to get into the weeds on some of the questions because there's a lot of data. Real time has been a big part of what you're doing. We haven't heard a lot from Molly and the Databricks team about real time yet. They have to deal with real time, but they're doing more like solve the big lake problem first, but real time matters in AI. Can you share your opinion and commentary on why real time's so important? I mean you deal with it every day, it's in your DNA and your bloodstream at ICE, but mainstream enterprise's now are going to be dealing with real-time data. How is that going to change the game?

Anand Pradhan

>> Oh, it's a big thing. So, I was explaining someone last week how the low latency comes into picture. So, let's say you type something for a search and the response takes 10 second to come back, you would give up on that search engine, right? So, there is a latency of 10 second in that case. So, you have to improve every step of the way so that it is faster. Now, to your question about real time when things are happening around the same time, you need the information around that. So, human surveillance is one big use case. As soon as something is happening, if you can correlate the data and find patterns, that's of immense value. So, the time to act on something that is happening with human surveillance is a big thing for any real-time use cases that is out there in the world.

>> Yeah, and low latency is critical.

Anand Pradhan

>> And the way to think about this is that so LLM cannot be a sledgehammer. So, you have to build purpose-built models, small language model or classification model, deep-learning model to figure out how to use all these things as modules, rather than thinking that ChatGPT will solve all the problems. So, yeah.

>> So, when people say neural network, "Okay, AI is a neural network knowledge graph." A knowledge graph is a knowledge graph. We know what that is. But neural network has kind of, I won't say the term's been abused, it might've been misused a lot. What is a neural network in context to AI? Can you simply explain what that is?

Anand Pradhan

>> Okay. So-

>> Is it a format? Is it a database? Is it graphs? Is it like math instead of keywords? I mean neural network, is it essentially an actually instrumented node in a graph? What is a neural network?

Anand Pradhan

>> Okay, so let me dumb it down. So, we know about human brains and neurons, basically. So, let's say you have a picture of number nine and you are saying that this model can understand that this letter that you have scribbled is actually nine. So, what happens is that each of the neurons, which is essentially making a decision whether a pixel fits in that number nine, let's say there is a 28-by-28 pixel number nine, and you lay that in a flat neuron in the first layer, each of the pixel will make a decision. So, think neuron as a number between zero and one, and the activation is there for the neuron to make a decision. So, that creates your first layer of neural network. And the last layer of neural network would basically tell us whether that is number nine. And in between that, there are multiple layers which are there. Then, each of these layer basically route for the next layer and that becomes your weights. So, essentially, it's a thing in memory which is storing a number between zero and one. That's how I would like to simplify it.

>> And so, it allows you to do things much faster, take that 6 billion signals?

Anand Pradhan

>> 650 billion signals. Yeah.

>> 650 billion. Sorry, I missed a few zeros on that one. But that's a lot of large data. So, neural network allows you to do things faster, is that the way to think about it?

Anand Pradhan

>> I don't think neural net would get to a level where we can, at least not at this day and age, can do the real-time 650 billion transaction inference-wise. Like Ali said today, inference is the thing. Training is one part of the problem, but the main use case is inference, right? The way low latency or any of the high-volume transactions work and the way neural net work, there are completely different use cases for that. But you can always find patterns in the data which can make into the neural net to make some decision or some output.

>> Yeah, it's just another way to deal with information. In this way, it's not semantic or keyword. The number nine is actually not a number in the mind of the neural net. It's just becomes nine because the brain figures it out or the neurons, you said. All right, so let's get to the ICE situation. ICE is the parent company of NYSE. You're running the AI Center of Excellence. What is that organization? Is it like a think-tank? Are you building stuff? Do you bring customers in? Do you interrogate people and find out what they want to buy? I mean what do you do there? I'm only kidding on the last part. But what is the AI Center of Excellence?

Anand Pradhan

>> So, as the name suggests, it is a center for excellence. So, what does it mean is that let's say there are various use cases. How do we make sure that we implement the AI use cases properly, so that each of the process is followed so that we get some value addition first of all, and it is implemented the right way. And the way to think about it is that is each of the process in data validation, data governance, AI model governance, distribution and all the training exercise that is there and all the bells and whistles that is needed is covered. That's one thing. The other thing is having a strategic vision in the company to lay out the foundation, so that multiple AI applications or projects can run with the same vision is the key for the Center of Excellence. So, it's an internal R&D group, so-

>> So, your stakeholder inside the company right now?

Anand Pradhan

>> Yes.

>> Any external customers or just all internal for now or stakeholders, whatever you call them?

Anand Pradhan

>> So, basically, our idea is to empower the product and application delivery team for their AI use cases. I don't think we will go to a level where we will directly interface with the external customer. But think like let's say, oh you need to build something which is AI application related and you are getting started from scratch, you don't have any idea. So, they would end up in this group, collaborate with us, bring their idea-

>> It's kind of incubation meets R&D, meets commercializing a new idea. It's kind of like guardrails versus... I see some organizations where there's a big elbowing each other out for who's going to be the AI czar, right? "I'm in charge of AI." "No." Everyone either gets their own AI or there's a point person leading a group to understand it, build guardrails, lead innovation.

Anand Pradhan

>> Correct. So, basically, it is innovating without compromising on the first principle, right? And each of the team have the liberty to build their own thing. The only thing is that if they need any help... And some teams would be further along. So, let's say we have built our model, but in that case, are we following all the processes which are needed? AI regulation is one thing that is fast evolving. You would see in news about AI regulation every single day. So, our job is to make sure that everything is implemented by the book. And we are a highly-regulated industry. Accuracy, explainability, observability, all these things matter a lot. So, as part of my previous role, we had built many of these things. So, the idea is to take that knowledge forward and because I have enterprise application delivery experience, it's a combination of all those factors matters.

>> Great. So, great leadership opportunity to lead and help the organization. What are you building now? How do you look at the market? Because ICE obviously has a lot of data. You work with external data suppliers as well. Obviously, you're an exchange. We hear a lot about LLM being worried about privacy leakages, intellectual property. So, people are building their own private models or small language models or small foundation models, and then connecting them to others. Bloomberg was reported last week when we were in New York at the AWS Financial Symposium, apparently it was said publicly, so it's not like it's a secret. The head of AI there said, "We did BloombergGPT." They were proud that they did it, but they actually ended up rewriting it because they actually went over their skis a little bit. And by the way, they don't mind. They knew that that might happen because they were first. Everyone's bloody through the wall, first one through the wall gets cut up. So, he said was, "Look, we don't have to reinvent a lot of stuff, we can leverage from other models." And his point was, "We did all that. We learned a lot, but we actually learned that we don't need to do all that stuff." What's your position on this and how do you think about that from a first principle standpoint or just from an industry perspective? Sometimes you don't know what you don't know, but sometimes some things are clear as day. Why build it if you don't have to? And is it in your core competencies. What's your thoughts on this?

Anand Pradhan

>> So, yesterday I had a meeting with chief scientist of Mosaic ML, Jonathan, and his vision and my vision resonated, because he said that, "Start small and then understand the entire process. And if you have not started the journey, start with something very small, so that before burning your finger, you will learn about that." So, we have done all the things we have done. Fine-tuning, we have used a foundation model and so on and so forth. The idea is that before even you go and start thinking about building your foundation model, there are so many open source model out there, which can be fine tuned. We have built small language model based on transformer architecture ourselves. So, it's nothing new that is happening, but we are further along. That's-

>> You are building your own model?

Anand Pradhan

>> We did build our own model for news data analysis and that can run on a GPU. It's BERT-based architecture, which is basically the first generation of transformer model. And the idea was that how do we optimize the cost? We always have some constraint and the innovation comes from the constraints basically. So, my vision or my view is that do it, do it properly start at a smaller scale, learn your way up before start thinking about investing ton of money.

>> So, I just checked my perplexity query about what a frontier model is because we were talking before we came on camera, what's a frontier model? What's the founding... All these different names. A frontier model is one that's ahead of the pack in performance larger than the other large language models. So, it's the leader basically. You mentioned BERT and there's leader boards now coming out. So, I want to ask you about this idea of leader boards and the race. Ollie showed on stage, "Okay, open source is catching up," meaning to the proprietary models, which they don't like to be called proprietary because OpenAI and Anthropic, it's their language model. It's technically proprietary, but we can all use it. What's your feeling on this? Do they merge together? Are there situations for each one that work well? Will open source surpass? Will it flip around? Will they may coexist? What's your thinking on this one? What's your opinion?

Anand Pradhan

>> So, my opinion is that we need to have control over our data and assets end-to-end. Many of the cases open source model would work for the kind of use cases we have right now. For some of them, it may not work and you have to go with the enterprise solution, maybe OpenAI and Anthropic on AWS and things like that. But in those cases, we make sure that we have coverage for data privacy and our cyber-sec team, they validate the entire pipeline to make sure. And also, we have to have agreement with them that they don't use our data for training the model further. So, those kinds of things are the first principle in my opinion for any of the things.

>> What's the coolest thing you're working on right now?

Anand Pradhan

>> I mean every day is a fun day for me. I'm a hardcore technologist, so when I was building the low-latency platform, I was very excited. Now, it's kind of a similar thing, how can we scale any of the things? So, I love technology. I love to-

>> What's the most exciting thing that gets you excited? The fact that it's unknown territory, that it's changing so fast, or is there a specific root-level challenge or problem to solve?

Anand Pradhan

>> Yeah, look, basically not so much about how things are changing fast, but how things work. How do we make sure that we are building the cost-effective solution because we need to have the right infrastructure, right tooling around it? So, that's my main focus right now. That keeps me up at night to think about the solutions.

>> Final question for you. You mentioned regulated industries, explainable AI, whatnot. What's the main challenge around regulation or the pressure for regulated industries with AI? What are the main challenges and explainability means... Explain what's behind that. What are some of the challenges that you see? I mean you guys are living it.

Anand Pradhan

>> First of all, we need to understand where AI is being used, right? Is large language model used for making pricing decisions? That will be a strict no from our side. So, we don't use for our critical mass of trading and decision making and things like that. Where we use is productivity improvement use cases, so where you can improve the productivity kind of thing. The other thing is are we doing everything by the book? Do we have audit trail? Do we have observability? Do we have monitoring? Did we validate the accuracy of it? All those kind of things. So, that is of paramount importance for us.

>> Yeah, you guys hosted a really great event in Silicon Valley and I was talking to some of the technologists, and others that I've known for years. And I asked all the funds that have been through multiple ways of innovation, "What about this wave?" And they're like, "Oh, it's going to be productivity right out of the gate." And the comment was it was like email. Email and Office suite was like an instant step function change from what was before. And then, it's going to probably go into some sort of more programmatic agents, and then ultimately some intelligence. But he thinks it's definitely going to be the low-hanging fruit now on productivity, similar to what email did in say word processing and spreadsheets because the alternative was manual, and now it's magic compared to what it was. Do you agree with that?

Anand Pradhan

>> I mean, that, and also you have to think about what are the things you do manually? Like you go to Perplexity and then search for something, it gives information. So, if you take that example and apply that in various other workflow use cases, that can be automated. It can give better results. So, there are a lot of use cases like that, which still, in my opinion, can be improved by a lot.

>> Thank you for coming on theCUBE. Great to see you.

Anand Pradhan

>> Thank you. Thank you for having me.

>> Hardcore technologist on theCUBE. We love it. Deep dives, that's our format. theCUBE is live here on the show floor at Databricks AI and Data Conference. I'm John Furrier, the host of theCUBE, with Savannah Peterson here, Rob Strechay, George Gilbert, the entire SiliconANGLE team coverage. We'll be back with more after this short break.

Anand Pradhan | ICE

search

>> Welcome back everyone to theCUBE's live coverage here in San Francisco. I'm John Furrier, host of theCUBE, here with Savannah Peterson. She and I, Rob Strechay and the entire CUBE team is here. We've got Anand, he's a friend of theCUBE, senior director of technology at ICE. ICE is the parent company of the New York Stock Exchange. Anand, great to see you. Thanks for coming onto theCUBE.

Anand Pradhan

>> Thank you. Happy to be here.

>> So, we've known each other for a while and it's the first time on theCUBE. Welcome with me. Thank you for coming on the show.

Anand Pradhan

>> Thank you.

>> Talk about your background because you've told me you built a lot of stuff at the NYSE, a lot of the systems, a lot of the code that power the exchange. ICE has a lot more going on than just the NYSE. It's a lot of things going on the systems there. You just started the AI Center of Excellence at ICE that you're running, and it's got a lot of action going on. So, take us through what you've been building in the past and what is the AI Center of Excellence?

Anand Pradhan

>> Yeah, I started my journey as a technologist more than two decades ago back in India. Worked there for five years, moved to the States as part of the New York Stock Exchange. I was part of the development team. But the interest has always been how the system works at the lowest level, try to understand everything, how to optimize. And that plays a role in the story leading up to the AI. Worked my way up building many of the system that powers the New York Stock Exchange. Played pivotal role in rebuilding the systems. Have been part of the Pillar development platform or trading platform that powers all our exchanges at New York Stock Exchange, as well as market data systems. Truly, a world-class sophisticated distributed computing system, at a lowest latency. And we use InfiniBand, which is basically used in AI training, so we understand a thing or two about low latency, high-volume usage. I also led the team to build the first hybrid cloud infrastructure and application for NYSE. So, I built a system for NYSE surveillance for the regulators. We mine data for 650 billion transactions every single day to find patterns. Once I finished that-

>> How many signals per day?

Anand Pradhan

>> 650 billion.

>> Billion with a B?

Anand Pradhan

>> B.

>> That's a lot.

Anand Pradhan

>> At microsecond-level latency. That means that you have ton of dense nanosecond granularity data, and you have to find patterns in the data set to figure out who is trying to game the market.

>> Seems a lot of vector is involved, a lot of embeddings, a lot of math.

Anand Pradhan

>> So, the embedding and math. So, math definitely is the core of all these things, but embedding comes later on.

>> Continue. I didn't mean to interrupt. We'll come to that.

Anand Pradhan

>> So, basically, once we finished all the data mining and we didn't have the day-to-day operational challenges after the cloud migration, we started focusing on machine learning. So, we build our deep learning models and things like that for the time series data. And then, ChatGPT happened, everybody pivoted towards transformer models. So, we build new sentiment analysis to correlate with the regulatory data or surveillance output that is coming out to figure out how we can auto resolve some of the alerts. That's where Vector Database comes into picture, combination of multiple model, there is a pipeline and things like that. But then, fast-forward right now, I moved to lead the AI Center of Excellence for our parent company to increase our collaboration across the company observability. There are similar use cases. How do we build it right? Make sure we have the accuracy, explainability, and all the things that goes without saying in our work? So, that's my journey.

>> Awesome. So, well, congratulations. It's fascinating and I can't wait to get into the weeds on some of the questions because there's a lot of data. Real time has been a big part of what you're doing. We haven't heard a lot from Molly and the Databricks team about real time yet. They have to deal with real time, but they're doing more like solve the big lake problem first, but real time matters in AI. Can you share your opinion and commentary on why real time's so important? I mean you deal with it every day, it's in your DNA and your bloodstream at ICE, but mainstream enterprise's now are going to be dealing with real-time data. How is that going to change the game?

Anand Pradhan

>> Oh, it's a big thing. So, I was explaining someone last week how the low latency comes into picture. So, let's say you type something for a search and the response takes 10 second to come back, you would give up on that search engine, right? So, there is a latency of 10 second in that case. So, you have to improve every step of the way so that it is faster. Now, to your question about real time when things are happening around the same time, you need the information around that. So, human surveillance is one big use case. As soon as something is happening, if you can correlate the data and find patterns, that's of immense value. So, the time to act on something that is happening with human surveillance is a big thing for any real-time use cases that is out there in the world.

>> Yeah, and low latency is critical.

Anand Pradhan

>> And the way to think about this is that so LLM cannot be a sledgehammer. So, you have to build purpose-built models, small language model or classification model, deep-learning model to figure out how to use all these things as modules, rather than thinking that ChatGPT will solve all the problems. So, yeah.

>> So, when people say neural network, "Okay, AI is a neural network knowledge graph." A knowledge graph is a knowledge graph. We know what that is. But neural network has kind of, I won't say the term's been abused, it might've been misused a lot. What is a neural network in context to AI? Can you simply explain what that is?

Anand Pradhan

>> Okay. So-

>> Is it a format? Is it a database? Is it graphs? Is it like math instead of keywords? I mean neural network, is it essentially an actually instrumented node in a graph? What is a neural network?

Anand Pradhan

>> Okay, so let me dumb it down. So, we know about human brains and neurons, basically. So, let's say you have a picture of number nine and you are saying that this model can understand that this letter that you have scribbled is actually nine. So, what happens is that each of the neurons, which is essentially making a decision whether a pixel fits in that number nine, let's say there is a 28-by-28 pixel number nine, and you lay that in a flat neuron in the first layer, each of the pixel will make a decision. So, think neuron as a number between zero and one, and the activation is there for the neuron to make a decision. So, that creates your first layer of neural network. And the last layer of neural network would basically tell us whether that is number nine. And in between that, there are multiple layers which are there. Then, each of these layer basically route for the next layer and that becomes your weights. So, essentially, it's a thing in memory which is storing a number between zero and one. That's how I would like to simplify it.

>> And so, it allows you to do things much faster, take that 6 billion signals?

Anand Pradhan

>> 650 billion signals. Yeah.

>> 650 billion. Sorry, I missed a few zeros on that one. But that's a lot of large data. So, neural network allows you to do things faster, is that the way to think about it?

Anand Pradhan

>> I don't think neural net would get to a level where we can, at least not at this day and age, can do the real-time 650 billion transaction inference-wise. Like Ali said today, inference is the thing. Training is one part of the problem, but the main use case is inference, right? The way low latency or any of the high-volume transactions work and the way neural net work, there are completely different use cases for that. But you can always find patterns in the data which can make into the neural net to make some decision or some output.

>> Yeah, it's just another way to deal with information. In this way, it's not semantic or keyword. The number nine is actually not a number in the mind of the neural net. It's just becomes nine because the brain figures it out or the neurons, you said. All right, so let's get to the ICE situation. ICE is the parent company of NYSE. You're running the AI Center of Excellence. What is that organization? Is it like a think-tank? Are you building stuff? Do you bring customers in? Do you interrogate people and find out what they want to buy? I mean what do you do there? I'm only kidding on the last part. But what is the AI Center of Excellence?

Anand Pradhan

>> So, as the name suggests, it is a center for excellence. So, what does it mean is that let's say there are various use cases. How do we make sure that we implement the AI use cases properly, so that each of the process is followed so that we get some value addition first of all, and it is implemented the right way. And the way to think about it is that is each of the process in data validation, data governance, AI model governance, distribution and all the training exercise that is there and all the bells and whistles that is needed is covered. That's one thing. The other thing is having a strategic vision in the company to lay out the foundation, so that multiple AI applications or projects can run with the same vision is the key for the Center of Excellence. So, it's an internal R&D group, so-

>> So, your stakeholder inside the company right now?

Anand Pradhan

>> Yes.

>> Any external customers or just all internal for now or stakeholders, whatever you call them?

Anand Pradhan

>> So, basically, our idea is to empower the product and application delivery team for their AI use cases. I don't think we will go to a level where we will directly interface with the external customer. But think like let's say, oh you need to build something which is AI application related and you are getting started from scratch, you don't have any idea. So, they would end up in this group, collaborate with us, bring their idea-

>> It's kind of incubation meets R&D, meets commercializing a new idea. It's kind of like guardrails versus... I see some organizations where there's a big elbowing each other out for who's going to be the AI czar, right? "I'm in charge of AI." "No." Everyone either gets their own AI or there's a point person leading a group to understand it, build guardrails, lead innovation.

Anand Pradhan

>> Correct. So, basically, it is innovating without compromising on the first principle, right? And each of the team have the liberty to build their own thing. The only thing is that if they need any help... And some teams would be further along. So, let's say we have built our model, but in that case, are we following all the processes which are needed? AI regulation is one thing that is fast evolving. You would see in news about AI regulation every single day. So, our job is to make sure that everything is implemented by the book. And we are a highly-regulated industry. Accuracy, explainability, observability, all these things matter a lot. So, as part of my previous role, we had built many of these things. So, the idea is to take that knowledge forward and because I have enterprise application delivery experience, it's a combination of all those factors matters.

>> Great. So, great leadership opportunity to lead and help the organization. What are you building now? How do you look at the market? Because ICE obviously has a lot of data. You work with external data suppliers as well. Obviously, you're an exchange. We hear a lot about LLM being worried about privacy leakages, intellectual property. So, people are building their own private models or small language models or small foundation models, and then connecting them to others. Bloomberg was reported last week when we were in New York at the AWS Financial Symposium, apparently it was said publicly, so it's not like it's a secret. The head of AI there said, "We did BloombergGPT." They were proud that they did it, but they actually ended up rewriting it because they actually went over their skis a little bit. And by the way, they don't mind. They knew that that might happen because they were first. Everyone's bloody through the wall, first one through the wall gets cut up. So, he said was, "Look, we don't have to reinvent a lot of stuff, we can leverage from other models." And his point was, "We did all that. We learned a lot, but we actually learned that we don't need to do all that stuff." What's your position on this and how do you think about that from a first principle standpoint or just from an industry perspective? Sometimes you don't know what you don't know, but sometimes some things are clear as day. Why build it if you don't have to? And is it in your core competencies. What's your thoughts on this?

Anand Pradhan

>> So, yesterday I had a meeting with chief scientist of Mosaic ML, Jonathan, and his vision and my vision resonated, because he said that, "Start small and then understand the entire process. And if you have not started the journey, start with something very small, so that before burning your finger, you will learn about that." So, we have done all the things we have done. Fine-tuning, we have used a foundation model and so on and so forth. The idea is that before even you go and start thinking about building your foundation model, there are so many open source model out there, which can be fine tuned. We have built small language model based on transformer architecture ourselves. So, it's nothing new that is happening, but we are further along. That's-

>> You are building your own model?

Anand Pradhan

>> We did build our own model for news data analysis and that can run on a GPU. It's BERT-based architecture, which is basically the first generation of transformer model. And the idea was that how do we optimize the cost? We always have some constraint and the innovation comes from the constraints basically. So, my vision or my view is that do it, do it properly start at a smaller scale, learn your way up before start thinking about investing ton of money.

>> So, I just checked my perplexity query about what a frontier model is because we were talking before we came on camera, what's a frontier model? What's the founding... All these different names. A frontier model is one that's ahead of the pack in performance larger than the other large language models. So, it's the leader basically. You mentioned BERT and there's leader boards now coming out. So, I want to ask you about this idea of leader boards and the race. Ollie showed on stage, "Okay, open source is catching up," meaning to the proprietary models, which they don't like to be called proprietary because OpenAI and Anthropic, it's their language model. It's technically proprietary, but we can all use it. What's your feeling on this? Do they merge together? Are there situations for each one that work well? Will open source surpass? Will it flip around? Will they may coexist? What's your thinking on this one? What's your opinion?

Anand Pradhan

>> So, my opinion is that we need to have control over our data and assets end-to-end. Many of the cases open source model would work for the kind of use cases we have right now. For some of them, it may not work and you have to go with the enterprise solution, maybe OpenAI and Anthropic on AWS and things like that. But in those cases, we make sure that we have coverage for data privacy and our cyber-sec team, they validate the entire pipeline to make sure. And also, we have to have agreement with them that they don't use our data for training the model further. So, those kinds of things are the first principle in my opinion for any of the things.

>> What's the coolest thing you're working on right now?

Anand Pradhan

>> I mean every day is a fun day for me. I'm a hardcore technologist, so when I was building the low-latency platform, I was very excited. Now, it's kind of a similar thing, how can we scale any of the things? So, I love technology. I love to-

>> What's the most exciting thing that gets you excited? The fact that it's unknown territory, that it's changing so fast, or is there a specific root-level challenge or problem to solve?

Anand Pradhan

>> Yeah, look, basically not so much about how things are changing fast, but how things work. How do we make sure that we are building the cost-effective solution because we need to have the right infrastructure, right tooling around it? So, that's my main focus right now. That keeps me up at night to think about the solutions.

>> Final question for you. You mentioned regulated industries, explainable AI, whatnot. What's the main challenge around regulation or the pressure for regulated industries with AI? What are the main challenges and explainability means... Explain what's behind that. What are some of the challenges that you see? I mean you guys are living it.

Anand Pradhan

>> First of all, we need to understand where AI is being used, right? Is large language model used for making pricing decisions? That will be a strict no from our side. So, we don't use for our critical mass of trading and decision making and things like that. Where we use is productivity improvement use cases, so where you can improve the productivity kind of thing. The other thing is are we doing everything by the book? Do we have audit trail? Do we have observability? Do we have monitoring? Did we validate the accuracy of it? All those kind of things. So, that is of paramount importance for us.

>> Yeah, you guys hosted a really great event in Silicon Valley and I was talking to some of the technologists, and others that I've known for years. And I asked all the funds that have been through multiple ways of innovation, "What about this wave?" And they're like, "Oh, it's going to be productivity right out of the gate." And the comment was it was like email. Email and Office suite was like an instant step function change from what was before. And then, it's going to probably go into some sort of more programmatic agents, and then ultimately some intelligence. But he thinks it's definitely going to be the low-hanging fruit now on productivity, similar to what email did in say word processing and spreadsheets because the alternative was manual, and now it's magic compared to what it was. Do you agree with that?

Anand Pradhan

>> I mean, that, and also you have to think about what are the things you do manually? Like you go to Perplexity and then search for something, it gives information. So, if you take that example and apply that in various other workflow use cases, that can be automated. It can give better results. So, there are a lot of use cases like that, which still, in my opinion, can be improved by a lot.

>> Thank you for coming on theCUBE. Great to see you.

Anand Pradhan

>> Thank you. Thank you for having me.

>> Hardcore technologist on theCUBE. We love it. Deep dives, that's our format. theCUBE is live here on the show floor at Databricks AI and Data Conference. I'm John Furrier, the host of theCUBE, with Savannah Peterson here, Rob Strechay, George Gilbert, the entire SiliconANGLE team coverage. We'll be back with more after this short break.