Databricks Data + AI Summit 2026 | Anand Pradhan, Intercontinental Exchange

Clips
More from Databricks Data + AI Summit 2026

Anand Pradhan

Head of the AI Center of Excellence

ICE

play_circle_outline ICE AI Center of Excellence: On-Prem Hybrid Stack (NVIDIA, VAST, Databricks) for Massive Financial Unstructured Data and Agentic Apps

play_circle_outline Data governance: audit trails, catalogs, provenance, benchmarking

play_circle_outline Managing Single-Copy Data and Scalable GPU Compute with Kubernetes: Access Control and Low-Latency AI Pipelines

play_circle_outline Recording agent conversations, logging, and compliance requirements

play_circle_outline Production workloads at scale versus prototypes and technical rigor

Info
Transcript

Anand Pradhan, Intercontinental Exchange

Anand Pradhan

Head of the AI Center of Excellence ICE

In this interview from the Databricks Data + AI Summit 2026, Anand Pradhan, vice president and head of the AI center of excellence and mortgage data at Intercontinental Exchange, joins theCUBE's John Furrier to discuss how the company is building a governed, production-ready AI stack across on-premises infrastructure and cloud. Pradhan details ICE's hybrid architecture — combining NVIDIA hardware, VAST Data and Databricks Unity Catalog — to process millions of pages of unstructured data across mortgage servicing, real estate, fixed income and multilingual pip... Read more

explore Keep Exploring

What is your organization currently working on? add

What is the process and governance you use to build an AI feature — including data/catalog management, model selection and benchmarking, responsibility and access, and the steps from experiment to production? add

What considerations and architecture do you use when designing an AI/ML data and compute stack—covering data management and governance, scalable compute (containers/Kubernetes), GPU data feeding, and inference (streaming vs. batching and APIs)? add

How does intelligence in agent systems arise, and what data architecture and storage considerations are needed to record, analyze, and scale agent conversations? add

What are the challenges and requirements for taking AI prototypes into enterprise-grade production at scale? add

bolt Powered by CUBE AI

Anand Pradhan, Intercontinental Exchange

search

John Furrier

>> Welcome back over theCUBE's live coverage here on the show floor of the Databricks event in San Francisco, the Data + AI Summit. It's theCUBE and the NYSE Wired program. theCUBE Original, again, the open network of innovators here on the ground. This event represents the accumulation of many, many years of Databricks. Been following them since their founding. Now, they're at the precipice of going global. 31,000 people are here. We predict that it's going to bust out in the season, probably have to move to Las Vegas. But someone who's been part of theCUBE and the NYSE Wired program and network, Anand is here. He is the head of ICE, Aurora Intercontinental Exchange team. ICE, Intercontinental Exchange. He's been an AI leader and part of the early original gangsters or OGs. Anand, great to see you. I know you work really hard and you had a very busy schedule. Thank you for taking time out to share your analysis. But I'm first more interested in what you're working on right now because you always have great line of sight on the innovation at the Intercontinental Exchange team. You are championing the Center of Excellence, all the AI teams. You're a builder, you're an engineer, you like to innovate. So you got your hands in a lot of cool things. So first question, what are you working on right now? Share please.

Anand Pradhan

>> Thanks. By the way, thanks for having me. I look forward to meeting you guys. I love the way you have grown the platform. What are we working at? We are working at many fronts right now. One of our core business is how to handle the data, process the data, create data assets and provide services to our customers. How do we do it in a more compliant way, but more efficiently using AI? That has been our push from the beginning. We are building our on-prem ecosystem with NVIDIA Stack, VAST Data, putting Databricks Unity Catalog on top of it for our analytics. So it'll be a truly on-prem cloud hybrid ecosystem. I had shared this before, but what is the usage of that? So when you are handling millions of pages of unstructured data, whether it is in our mortgage ecosystem or our realistic data processing or our fixed income and data services, multilingual data processing, you need and have a hyper optimized ecosystem. So that's what we are focused on, but one of the things that has happened in the last year or so is the agentic AI. So agentic AI, if people understand to go deep into the agent layer, how the things are working and how do we put agentic AI on top of our AI stack to further optimize this ecosystem. And that also empowers our customer facing things. For example, on our servicing platform, which is called MSP, which is the largest servicing platform that we have. So we are putting AI chatbot, voice agent, as well as launching chat application for our loan origination ecosystems. So across the board, we are building all these embedded AI features for our applications, services and things like that. But behind the scene we have a big pipeline of data and underneath everything is the hardware infrastructure, analytics, so on and so forth.

John Furrier

>> I was talking to Jonathan Frankel, chief AI scientist, you know him.

Anand Pradhan

>> Yeah.

John Furrier

>> There seems to be a pattern emerging and he kind of pointed it out and you just reminded me to bring it together. I'm just kind of riffing in real time. The good AI teams have two things going on. You got some stuff being deployed and managed and technology innovation being crafted and put into production. And then you have a behind the scenes team. He's running the lab, so he shared ... And he wasn't really involved in a lot of announcements like last year he did the whole evaluation thing. But his team is working tirelessly behind the scenes because there's a lot of other optimization. That's what you just said. Take us through some of that because this is where Databricks seems to fit because it's not a shim layer, it's a data layer. And your use cases on-premises, you got a lot of data feeds, a lot of security, a lot of proprietary data. This is something that you're doing too. What is that behind the scenes work that needs to be kind of done and that's where agents are going to play, but it's also going to connect to the front end or the service layer on the front?

Anand Pradhan

>> Right. So let me walk you through the process. For example, let's say we want to build an AI feature. We need to make sure we have the necessary accuracy, the guardrails, all the audit trail of the information and that is our part of our governance. So in order to achieve that, you need to have the catalog information. Where is the data coming from? What are the models we are using? What are the benchmarking we are using? So we have standardized that for the most part on Databricks platform, using the Unity Catalog. And I like Databricks guys, because they open source a lot of the things. So let's say you have a hybrid solution, you can take their open source, use it on-prem, then also use Databricks, managed services, and stitch it together. Now, what I'm saying is that all the pieces that goes behind the building block of building an agentic layer or AI layer is, how do you manage your data assets? Who is responsible for the data? Do we have access to the data? What models we are using, what benchmarking solutions we are using? The whole harness layer is super important. So that's what we are focused on to standardize those things. For the application layer, when they come to our shop, we say that these are the models, this is the platform, these are the data assets that you can use. And this is the process you have to take an idea, experiment, implement, go to production. So that's the process we have. And then governance and all those things plays a part in that ecosystem.

John Furrier

>> Okay. I want to go down a trip and memory lane with you. We haven't met yet. Let's rewind back to 15 years ago, the Hadoop era.

Anand Pradhan

>> Yeah.

John Furrier

>> That's the era that storage and compute had separation. Then you got the cloud era, the elasticity, the lakehouse was unification. The AI era is about context, reasoning, autonomous execution. Okay. That's what you see on slides. But I want to talk about the storage and compute execution, because what's come out on theCUBE here at this event is what that means from an AI perspective. When you have data sets that you can actually fork and branch and you have agents doing this, it's not human. So the time to do it is not weeks. There's a lot more things you're doing. And I know you've been doing a lot of work around AI storage, moving data around superfast. Take me through your thoughts on this. I want to hear your vision because this is coming up as a key computer science AI principle in computer systems right now for AI.

Anand Pradhan

>> Yeah. At a fundamental level, what we need is access to the data and you cannot have duplication of data. So you need potentially a single copy of the dataset for better data management and governance, then the access control of the data. So that goes into the data layer. And how does it work? So you have the data infrastructure. On top of that, there is a software layer which manages all these things that is part of your data stack. But then in order to do the computation, whether it is API agents, your trial and error compute that you needed, which is like pre-processing, post-processing, all those thing, that needs to be a scalable compute layer. So with container and container orchestration like Kubernetes and OpenShift kind of layer that provides that scalability, and that needs to have access to the data. So whichever data solutions can provide faster access to the compute, that's the way to go, and that is our focus. Then once all these things happen, you need the GPU layer to have faster access. So there is a data retrieval and then feeding mechanism going into the GPU layer. And on top of that, once the feature gets built out and you put it to inference, then you have to think about whether it needs streaming or batching, what is the API layer and so on and so forth. So that's the typical thought process we have and it is pretty standard in computing terms, but not many people have done it right.

John Furrier

>> So Dave Vellante and George Gilbert, they do a big deep dive in Breaking Analysis on and we've covered system of records, system of analytics, lakehouses. Now we're in a system of intelligence. I think the next level of system of execution, they call it system of agency. Not agency, but like empowering.

Anand Pradhan

>> Yes.

John Furrier

>> When you're starting to operate at a system of intelligence, that's where agents start to shine. So connect the dots for us. As you embark on that next journey to go from systems of intelligence to execution, the workers will be digital. There's going to be humans in the loop, but the architecture sets the table. What's your vision on how you get there? You just talk about the stack. Do you see something emerging from the show here or observations that you're seeing? Is it working with the storage vendors? What are the things that you need to do to maintain that intelligence?

Anand Pradhan

>> Yeah. I mean, if you dig deeper a little bit, so how does the intelligence come? It's you have to track all the conversation that is happening in the agent layer. So let's say I ask an agent to do an action that, "Hey, build something for me using these datasets kind of thing." So that conversation needs to be recorded. Every step that is happening inside the agent needs to be recorded. So that means our data architecture needs to be solid enough to first write these data fast enough, have enough access control, then do the pattern recognition in the data to give the insights. So Databricks does a good job of putting it all together and analytics on top of it, but behind the scene, you need to have faster access. Now, many people can use typical cloud, whether it is AWS, Google, Azure, you name it, in their storage bucket, but then if you have to optimize it, you have to think about your on-prem deployment because you are talking about petabyte scale storage. Yeah. So VAST plays a huge role for us in that. We also have Qumulo for mortgage ecosystem. So we use a combination of storage appliances and storage solutions depending on what needs to be used where, but the data intelligence comes if we-

John Furrier

>> Saw a great demo from a Qumulo at Cisco Live. They're using agents to identify malware fast within a second or two hitting an agent from the Cisco side and then quarantine on that network node in milliseconds. That is unheard of. So you start to see the advantages of what's going on behind the scenes with agents. What are you most excited about when you look at Databricks to this week from your purview? Because you're laying out the future for ICE and it's going to look a lot like what I just said, low latency, faster activities, actors being taken with trust and security built in with governance. That's the future here. That's the current situation. What's the reaction to Databricks' performance today?

Anand Pradhan

>> Yeah. So how should we think about the future? So anytime you are building something, it needs to be future protected, so then you have to focus on the fundamentals. And the most exciting thing for me, the pace at which all the technology updates are coming is unbelievable. It's exciting at the same time very stressful, because you have to filter through the noise. And if people have not spent enough time in the low level technology, it's hard for them to realize what is happening inside the agentic AI layer kind of thing. The other thing to focus on is, can you keep it vendor independent because nobody wants to be in that vendor lock-in thing.

John Furrier

>> Choice is a huge thing for them.

Anand Pradhan

>> Correct. So basically, this is what I like about Databricks, because they open source a layer of it, which can go into any vendor software and still works with Databricks. One of our data intelligence platform we rolled out to our customers right now is on Databricks, because we are using them as a managed service to provide intelligence layer, Genie apps and things like that. We have been working closely. So it is pretty exciting. If you have your data layer correctly sorted, filter through all the data, have managed datasets, building an app for analytics is quick.

John Furrier

>> I love the Databricks people because the team, the executive team, firstly, the first principles. It's been true from day one. I will give them props for Databricks. I'll give you props for this because it's worth noting. They have played the long game. I mean, they're making a lot of money now. Okay, great. But I don't think they're focused on just the money making. Look what they did with open tables. Look what they did, the open source goodness coming out. They're very developer-centric. So definitely playing the long game a lot like what you're doing. It's not about how big a purchase order could come in. They're really focused on partnering at the tech level. I think they see the bigger picture and the transactional piece that came out today, I talked to Ollie and the team about that three years ago and I know our research team was talking ... So it's always been on their radar, just keep adding stuff to the package, meaning the open source and the platform. Now, it's going to be time to see growth. What do you expect from the industry as a result, not just you guys, as a proxy, your view? I see EY over there., they're going to do some good business. I mean, all the vendors are here on the floor, suppliers, customers end up winning, but it's an ecosystem. What's your view of the ecosystem and how this is all going to play out for Databricks?

Anand Pradhan

>> I think they are rightly placed and it goes back to the first principle. I mean, early today, I said that it all started from the open source Spark and that ethos is there. So whenever I see fundamentally what they're doing, their focus is in the right place. And because of the AI related changes which is happening in the industry, I feel like they're really well-placed, because they understand what it takes to build a model, they understand the ecosystem, they understand the analytics layer and they have the flexibility to open source things so that it works in open formats, which is like vendor independent. I mean, from what I can read, they are well focused on, "Hey, this is our strength. Let's focus on that and let's grow with it." That kind of thing.

John Furrier

>> I got to say, when they did that, I said to Dave Vellante, we were actually covering the live event when they did the open formats and that was a strategic move because we saw then that this is going to take the drama out of formats. They basically cannibalized their own business to grow the category. That was a bold move. And I just had last year when you came on as well, Pete Sonsini came on, he's a board member of Databricks, he's a good friend. He was telling a story where during the Spark era, talking about Hadoop era, Hadoop was a great idea, but it was hard. Spark, they took the chance on Spark, but there was a moment where Databricks almost went out of business and Pete Sonsini told that story here on theCUBE where they weren't really making any money and they were going to go out of business. Just think about that. They were this close to going out of business. He led the B round when he was at NEA, now he's at Laude Ventures, Laude Institute. They're doing a lot of great stuff there and then it kicked in. This is the playbook for AI. You got to be patient. If they don't make it through Spark, which they did, and they came out with flying colors with the data lake concept, which Uber kind of did, but they did that, so it's so good. They don't make that. A lot of people give up as a manager. How do you handle that patience with you and your team? Because you want results, speed is a game, but sometimes you just got to have the awareness of saying, "Whoa, whoa, let's just think, bake in the oven a little bit more. We just got momentum." How do you distinguish between real momentum and time to abandon?

Anand Pradhan

>> Yeah. Do you see more gray hairs? Yeah. So it's tough, because people think that it's magic. Although it seems like magic, you still have to be real focused on not only the prototype, but how does it work in your enterprise grade scale, right? Do you have check-pointing? Do you have logging? Do you have-

John Furrier

>> You got to do the work....

Anand Pradhan

>> security? Yeah.

John Furrier

>> You got to do the work. You got to do the work.

Anand Pradhan

>> So that conversation is always tougher, basically. It's easy to get started, but in order to build an enterprise grade system ... Look at the scale at which NYSE operates, right? And same goes for ICE, other businesses. If you look at the scale, our mortgage pipelines are operating. It's unbelievable, the teams, what they have built. So in order to embed AI feature into any of these applications, first of all, you have to look at, "Are we really solving a problem for someone by doing this thing, or it is more of a hype and we are changing the hype?" So once we are passed that, then you have to focus on the accuracy and how do we justify the accuracy of it, what kind of feature we build. But yeah, it takes a lot to convince people that building the prototype-

John Furrier

>> Yeah, it's hard because the new North Star in this era is production workloads at scale. That's the new part. I mean, you guys, NYSE and ICE, you guys are at scale, but all companies, when the data starts coming in, it's at scale at some level.

Anand Pradhan

>> Yeah.

John Furrier

>> All right, final question. I know you got to go. Appreciate your time. I know personally from observing your work role at ICE, you meet a lot of companies. What's been the biggest change in the past year in terms of how people approach you for business? And has it been the scale? Has anything changed? Have you noticed any pattern from what's good and what's not good? You don't have to give any names, but is there a certain approach as the way they dress up for the party so to speak with their tech, their evaluations?

Anand Pradhan

>> You're talking our customers, or prospective customers?

John Furrier

>> Prospective suppliers that come and say, "Hey, I want to sell you something." Or, "Hey, I want to partner with you."

Anand Pradhan

>> Yeah, you're right. I hope I'm right that our impression as a technology company has come along right now because we used to do a lot, but we wouldn't speak about it. For example, if you look at the security poster we have, it's commendable what at scale the team is doing. And if you look at the team, there are some geeks in there.

John Furrier

>> You guys are target for hackers.

Anand Pradhan

>> Yeah. So I think we have a good sort of impression in the market that we are a good technology partners. Our customers trust us for that. We have a long history of building reliable products and services. So that has been our moat and then we continue to invest in innovative things. So we are not like a traditional technology company sitting on our laurels kind of thing. We are constantly pushing ourselves to take it to the next level with new features, maintaining all the regulatory requirements, security guard rails, governance.

John Furrier

>> It's a critical infrastructure.

Anand Pradhan

>> Yeah.

John Furrier

>> Anand, thank you for coming on theCUBE. As always, we'll make it a tradition. Love your commentary and insights. Thanks for coming on. As always, great to see you.

Anand Pradhan

>> Thank you. Thank you. Great to have you.

John Furrier

>> I'm John Furrier, your host of theCUBE. We are here on the show floor as part of theCUBE and our NYSE Wired program, a CUBE original, an open network of innovators coming together to celebrate but also drive the innovation. There's so much opportunity, production workloads at scale. As agentic comes to the scene, all the power from the supercomputing, the data layers, you get that governance right, you get that security right, the agents can be very, very productive at scale. We're going to see a lot more of this for sure. Stay tuned. More live coverage from theCUBE after this short break.