Data + AI Summit 2025 | Patrick Wendell, Databricks

Clips
News
More from Data + AI Summit 2025

Patrick Wendell

Cofounder & VP of Engineering

Databricks

Inside Databricks’ AI revolution: 10 key takeaways from the Data + AI Summit

Artificial intelligence is becoming more intuitive, interactive and embedded, not just in systems, but in the way companies build, think and create. From live coding and autonomous agents to unified governance and automation, Databricks Inc. is moving AI closer to the application layer and aligning it with developer intent.TheCUBE Research’s John Furrier and George Gilbert break down the keynote analysis during Databricks’ Data + AI Summit.“You sort of tell the AI engineer, the software developer, an agent, what you want to do,” said George Gilbert, principal analyst for theCUBE Research. “In the past, all it did was generate code

play_circle_outline Importance of an interoperable mindset in open-source strategy for Databricks.

play_circle_outline From Early Career to Databricks Growth: Exploring Mosaic, OLTP, and AI Integration for Enhanced Online Storage Solutions

play_circle_outline Navigating the Evolving Data Platform Landscape: Timing Strategies for Successful Enterprise Cloud Adoption

play_circle_outline Embracing AI Growth: How Agent Bricks Streamlines Application Deployment and Boosts Production Efficiency in Enterprises

Info
Transcript

Patrick Wendell, Databricks

Patrick Wendell

Cofounder & VP of Engineering Databricks

Patrick Wendell, co-founder and vice president of engineering at Databricks Inc., joins theCUBE’s John Furrier at the Databricks Data + AI Summit 2025. The conversation offers a behind-the-scenes look at Databricks’ evolution from data services to full-spectrum AI leadership.

Wendell reflects on the company’s journey and the milestones shaping its future, including acquisitions such as Mosaic and the integration of online transaction processing capabilities. He shares how Databricks continues to prioritize open-source foundations and seamless integra... Read more

explore Keep Exploring

What developments in open-source initiatives at Databricks have occurred over the past two years, and how do they relate to recent advancements in AI and new applications? add

What are the key components and developments of a comprehensive data platform? add

What factors influenced the growth and success of Databricks in the context of AI and market timing? add

What observations were made about the current market for generative AI and the challenges of implementing it in production? add

bolt Powered by CUBE AI

Patrick Wendell, Databricks

search

>> Welcome back everyone to theCUBE's live coverage here in San Francisco for Databricks's Data + AI Summit. I'm John Furrier, host of theCUBE. We've been covering Databricks for a long, long time, but my good friend Patrick Wendell's co-founder and VP of engineering at Databricks, just riffing from the days back when we hung out together in the old office of Cloudera.

Patrick Wendell

>> Yeah, we were joking we both had less gray hair and were early in our careers and that was just as you were getting started with this thing and I was about to found Databricks, so those were crazy times.

>> Two co-founders. I'm the host of theCUBE and you're the co-founder of Databricks.

Patrick Wendell

>> Yeah.

>> Congratulations.

Patrick Wendell

>> Yeah, thank you so much and thanks for being here.

>> Great. We're super excited. I just personally love the mojo of the keynotes. I mean, if you go back two years ago here at this event, it really was the open-source part of Databricks is saying, "Hey, let's get more into open. We're going to take the first move at opening up and creating an interoperable mindset." Obviously, some DNA at Berkeley goes back in the first principles, but that was the beginning, okay? Now, fast-forward, you got a lot of AI action, what I saw on stage today with the apps and also the Agent Brick, really great direction. The Mosaic comes in. I'll have Jonathan on later. Naveen wasn't available, but Jonathan and I have interviewed in New York, but the piece parts aren't together. Explain how this is all happening, because you got Mosaic, you got the new app, you got OLTP with Postgres. What are these elements, these puzzle pieces? Why is it working? Why is this AI equation working for Databricks?

Patrick Wendell

>> Yeah, yeah, thanks. Great observations. I mean, the way we like to think about it is that there's a few key pillars of what it means to really have a data platform, and one is the area we've focused on for a long time, which is your core data ingestion, ETL. I mean, Spark has been used for that for a decade or more now, but over time we've expanded. And we saw a big expansion a few years ago into data warehousing, SQL type analytics. That's sort of well on its way. And then you saw a couple years ago we acquired Mosaic, and that's really become the foundation of our AI capabilities. And this year, it's all about online transaction processing and production databases through the Neon acquisition and integration. And what's interesting is that not only do we kind of need this to round out the whole story for our customers, but increasingly, these things are synergistic with each other. I mean, almost every online application now uses AI, so the online storage, it can't really be separated from the models and the model building. That all needs to happen in the same place. So we see these as highly synergistic as part of the broader data platform.

>> I remember, first of all, the unification, anyone who knows operating systems knows that you got to have a unification layer. But one of the things you guys had with Delta, and at that time, you're getting a lot of FUD in the market around, "Well, proprietary tables interact, there's other parquets and other things out there," but really getting behind Iceberg and letting customers store the data, it wasn't a storage mindset. It was very much a data OS mindset, which is one of the themes that we were commenting in our keynote analysis. It's like an AI operating system, if you think about it.

Patrick Wendell

>> Yeah, that's exactly right. I mean, if you look first of all as it pertains to open, our view has always been the core pieces need to be open. And I mean, you saw that even when we started Databricks. That was we started around Spark and if you look, the catalog is open, the storage format's open. We had a situation where there was competing standards at the storage layer and we just said, "Look, we want to make that not an issue for our customers. You choose your storage format and we'll support it." So that's been kind of our philosophy always is-

>> Oh, by the way, moving data around costs money.

Patrick Wendell

>> Yeah, for sure.

>> Leave it where it is.

Patrick Wendell

>> Let's avoid that headache. So yeah, I absolutely see it as an operating system for your data. And in that operating system, we want the core pieces to be based on open standards.

>> And if you look at what the challenges are with customers right now, you have two things happening. You got the emerging, what I call the new generation developer, the young gun or seasoned systems person. This seems to be kind of two people, old systems guys like me, and now you, you're getting a little older now, but back in the '90s, Berkeley was a revolutionary in the systems revolution. I mean, they were the catalyst.

Patrick Wendell

>> Yeah. It's funny because when you said young gun, at first I thought, "Yeah, that's me you're talking about," and then I'm realizing I'm not in that category anymore. But Berkeley, Berkeley has a history of these core foundational contributions and also a history of open source. And it might be the hippie vibe coming out of Berkeley, I don't know, but I think that's been a really successful formula from the university for these foundational platforms.

>> One thing I love about revolutionary startups, whether it's coming out of Berkeley or just a company, is that to do something radical, you have to be uncomfortable with not knowing what's going to happen next. And I think if you look at Hadoop, we all thought, "Oh, big data's here, finally." And then it just kind of stunted at birth. And then Spark came in and then it just hit a little lull. Pete Sunstein was talking about that when he was just on around the series CD part of the company was like, "Okay, no one's getting it yet. We haven't hit escape velocity." That was a 2015 timeframe, 2014, but then cloud was exploding. So that was a critical point in the history of Databricks because then enterprise moves to the cloud, now they have a lot more data. What's your perspective on this point of history? I know it's really nuanced, a little bit of look back, but that was an inflection point and that tells us a lot what's going to happen now because now you have AI growth going on.

Patrick Wendell

>> Yeah. So it's very similar. I mean, in some sense, even the early days of Cloudera before Databricks, everyone kind of knew what the long-term was going to look like. In fact, they called the company Cloudera thinking initially they would do a cloud managed offering, which they didn't end up doing at that time. But I think what was not anticipated is that it took a while for the future to become the present. It took the cloud being a major technology transition that allowed a lot of these companies, since they were rethinking things for the cloud anyways, they took a chance to rethink their data platform. And a lot of it is just the timing of when these technologies take hold in the market. And I think as Pete told you, we had this almost existential moment in 2015 where the product was there, we had the vision, but the market was kind of just starting to turn. And honestly, if that turn hadn't happened at just the right time, I don't know if Databricks would be the company it is today. So we obviously try to build great products, but you're also counting on these trends happening in the market and hitting the timing correctly as well.

>> So as a co-founder, it's kind of legendary status now as you guys look at the size and value of the company. I mean, there's almost a pinch me moment of like, "Okay, let's keep my head down, keep jamming." I'm sure you have that. You guys are very humble. Even Ali's admitting he made some bad calls on stage, which is fun to watch because that is the culture. But now you look forward, you're driving an AI step function and this is what we see. Again, young programmers are coming in, solo founders, two or three folks building a company, enterprises changing how they do things. What's next in the AI vision? Can you share your thoughts as you drive this forward? Because there's a lot more stuff coming from this. If you take just the direction of Agent Brick, Lakeflow, this is just the beginning of new mechanisms.

Patrick Wendell

>> Yeah. So I think the way we look at this market, I mean, first of all, there's this Cambrian explosion of you just look around the expo hall. There's a ton of new ideas, a ton of startups, and I think the next 48 months is going to be about distilling the stuff that works from the stuff that doesn't work. And certainly we see that amongst our customers. That was the inspiration behind Agent Bricks is that we had thousands of customers doing some kind of gen AI and we saw what delineates the ones that work from the ones that don't work. And it turns out that making gen AI application ship into production is hard, and that's good in some ways. It means that you can differentiate yourself from your competitors by doing it, but it's very similar to the early days of data platforms. If you remember, even at the early days of Spark, there was a lot of do it yourself, a lot of having to build everything from scratch. You needed PhDs to get anything done. And over time, Databricks and other companies simplified what it was like to do that. And we're in the middle of that simplification for gen AI.

>> And that's the same first principles you're applying to now.

Patrick Wendell

>> Almost identical. I mean, obviously the context is different, but if you want something to have very broad impact, you need to make it simpler in a lot of ways. On the consumer side, ChatGPT was that. I mean, those models already existed, these chat models. They put it in a really simple nice interface that even my mom and dad use all the time as non-technical users.

>> Like, "Mom and dad, that's what kind of I do for AI."

Patrick Wendell

>> So on the consumer side, that was the vehicle for simplifying this. On the enterprise side, we think it's going to be something like Agent Bricks.

>> Yeah, and also Lakeflow is interesting because now you can match declarative pipelining of data to the identified workflows.

Patrick Wendell

>> And you just had my colleague, Michael, on. We've been thinking about the interfaces for everything are being revisited with gen AI, including for core data pipelines.

>> On the enterprise side, on the ages, what should people start doing now for AI readiness? Because sometimes they're going to be ready and we're seeing in regulated industries, they've already been labeling data for years for the wrong reasons.

Patrick Wendell

>> Yes.

>> I mean, for their reasons, but now they're good for AI or for young companies like ours. I'll give you an example. We have our video data lake, our media brick I call it. But what do we do? I plug it right into the agent builder. Can I start building agents? If my data's together, what's the vision of connecting into Databricks?

Patrick Wendell

>> Yeah. So first of all, you actually demoed it to me right before this interview and you guys have a pretty nice little gen AI solution you've built, and I think you've done step one, which is what we see in all of our customers, get the relevant data together, because what differentiates your product is your data. Only you have that data, and that's the same for our customers. So even that is easier said than done. When the data scale gets large, getting it together, having the right access control permissions, dealing with sensitive data and who can see what, even that's extremely difficult. But once you have your core product, then the next step is about optimizing. It's running that optimization loop. How do I get the quality to be better and better and better? And that's where Agent Bricks comes in. We're trying to help you do evaluation. When your customers are using it, are they happy? Are they getting the right results? Are you sometimes hallucinating? So kind of getting that evaluation and then optimization loop to make your thing better and better as you get more data, that's really what we're focused on.

>> So I would then have to identify a workflow that I want to get an agent for, get the agent data into the agent brick and play around.

Patrick Wendell

>> Exactly. Yeah, exactly. And I think we have bricks that are focused on this kind of knowledge-based Q&A where a lot of our customers, they have hundreds or thousands of documents internally, they want to let their employees interact with that corpus of information, and doing that without hallucinations, without errors is actually a little bit tricky.

>> What about tooling? Because one of the things I always kind of roll my eyes when I try to figure out what to do or people tend to, is that, okay, before I'd connect to an API, API, no problem, but now the data connections, there's issues around lineage and authority and delegation and intelligence. No one wants to connect to a black box. So how do people build tooling faster? That seems to be a hot area right now in tooling.

Patrick Wendell

>> Yeah, it's really tricky and especially since a lot of the promise of gen AI is getting all these systems together, I think we're looking at... There are some evolving protocols. There's MCP, the Mosaic, not Mosaic-

>> Model context protocol.

Patrick Wendell

>> Model context protocol. And I think the solution to that is standard interfaces where systems can talk to each other, they can do authentication, they can share data so that the job of integrating this data just becomes less of a headache.

>> How should people build their AI teams knowing that this agent builder... You guys really got a letter grade A from me on the keynote. Agent Brick to me is directionally super relevant. It'll evolve, but okay, let's say I buy into that. What do I do? How do I set up my team? What's your vision knowing what you know? How would I want to organize my team to take advantage of this?

Patrick Wendell

>> So I think that in the early days, really in the last 24 months, many of our customers went through a few phases. The first phase is they tried a million ideas in gen AI. Every team tried an idea. And we even did that at Databricks. We had a lot of ideas. I mean, you see the ones that made it all the way, but there was a lot of experimentation. It's very important after that first stage to consolidate and choose a few efforts that you think have legs where you think there's really obvious value to the end users. You have unique data that's going to let you differentiate. So a lot of what we're doing actually in the last year is helping kind of pick the really high ROI projects and then put firepower behind those teams. Gen AI talent's also hard to find. You don't have everywhere in the company people that have a lot of experience building this stuff. So I think you've got to let those teams focus and then give them tools like Agent Bricks that can accelerate their own time to market.

>> All right. What are you most excited about right now in terms of what you're working on? Obviously AI. Can you be specific on your scope of your job right now?

Patrick Wendell

>> Yeah. First of all, I have two jobs. I'm a founder and as a founder you kind of... It's like a little-

>> Celebrity status.

Patrick Wendell

>> Well, that's the nice version of it, but it's a little bit like being a parent. If your kid's sick, no matter what's wrong with them, you got to help. So I certainly spend a lot of time in all parts of the company, but my day job is supporting our engineering team around our AI products. And there I think the thing I'm excited about is that we're in this transition from experimenting to production. And I think we've tried to highlight at this conference a lot of our customers in production. Fox Sports has a whole area about their AI apps that they're doing here. We've got MasterCard, we have Block. And so it's just exciting to see after all the years of hype and very high expectations, you're starting to see the value come out. And I think that's the most exciting thing for me.

>> You guys, it doesn't hurt. It's a good business model, makes things easy to use, simple and reduce the steps it takes to do something.

Patrick Wendell

>> Yeah. And I mean, when you're in the middle of a technical transformation, making things easier to use is always valuable. So I think that's been our philosophy.

>> What's the area you're working on to make it easier now? Again, is it ETL? Is it the AI spinning up agents? Is it the data management? Is it the governance? What's the key area that needs to simplify?

Patrick Wendell

>> Certainly AI agents are a topic I think about, but I think overall, we're actually in a company transition, moving more into just generally online services. So I think people think of your data stack initially as, "Oh, that's your offline pipeline, runs every night. It collects all the relevant data, powers some dashboards." But I think the big change we're seeing that I'm most excited about is real online production applications. You see that with Databricks apps, you see that with our OLTP database, you see it in the AI agents, and it's been fun for us to help our customers go all the way to the end basically.

>> It's interesting. OLTP was a very much a niche business for finance, online transaction processing, big mega fault-tolerant machines, hardware, software. But if you think about micropayments back in the day or agents talking to each other, these are transactions.

Patrick Wendell

>> Absolutely.

>> I mean, this is OLTP characteristics. How does OLTP change in your mind? Because this is a computer science concept that was once for the high performance, big money transfers, actions.

Patrick Wendell

>> Yeah, I mean, the famous example is airline tickets because you don't want to book two people on the same seat.

>> Database locking and all that high end-

Patrick Wendell

>> It's all about that stuff. So I think we're sort of seeing a rejuvenation of that area, but with much different types of applications, AI-based applications, applications where they're creating new databases on a very frequent basis, so some of the original tenants of OLTP are still there, especially around the transaction safety, and if we store a record, it really needs to be there. There can't be data errors, but the need for elasticity and for sort of very dynamic provisioning is fairly new.

>> And having that observability around those features too, that brings that whole conversation we heard today, right?

Patrick Wendell

>> Exactly. Yeah.

>> All right, so I have to ask a final question. First of all, great conversation. Great to have you on. We'll certainly get on again for sure. Love the co-founder throwback historian and leading engineering conversations. Computer science right now is so hot. I mean, Jensen Wong is the CEO that I've heard on stage. He says the word computer science at least three times. So there's a whole big computer science wave, but it's also being democratized at the same time. What computer science areas, Patrick, do you see right now that are super hot that people watching, whether they want to rescale or reshift or just double down, where are the hot computer science areas in terms of either PhD work or just action?

Patrick Wendell

>> Yeah, that's a great question. So first of all, there is this big question mark, are coders going away? Are we the last generation of software engineers? And I think maybe in the very long term, but no time soon. In fact, I was listening to an interview with the Cursor CEO, and his view is similar to mine, which is that it's very much augmentation right now. It's helping those people be more efficient. So I don't think the fundamentals are going anywhere. I think that I still would strongly recommend-

>> It's distributed computing, it's operating systems.

Patrick Wendell

>> Yeah, exactly-

>> Networking, database.

Patrick Wendell

>> Programming languages, designing good abstractions, dealing with fault recovery. That's not going anywhere.

>> too.

Patrick Wendell

>> So the good news is if you have a computer science degree, don't worry. It's still pretty valuable.

>> Yeah. Mama, I did good.

Patrick Wendell

>> Yeah.

>> It's going to pay off someday. Thanks so much, Patrick. Good to see you.

Patrick Wendell

>> Yeah, thanks for having me. And as always, thanks for your participation.

>> Congratulations again, continued success. Big fan of you guys, and as co-founder, I know it's not an easy road being a co-founder.

Patrick Wendell

>> Yeah, we'll more kindred spirits in that regard.

>> All right, bringing the co-founder perspective here in theCUBE. I'm John Furrier. I'll be back with more live coverage after this short break.