Cloud AWS re:Invent Coverage | Naveen Rao, Databricks

Clips
News
More from Cloud AWS re:Invent Coverage

Naveen Rao

VP of AI

Databricks

Exploring Databricks’ vision for an AI-driven data intelligence future

Over the last decade, a clear trend has emerged. Companies that used to store data without a clear plan for use are now uncovering untapped value through AI-driven data intelligence.In an industry hurtling toward a new frontier, Databricks Inc. is redefining the possibilities of data and AI, according to Naveen Rao (pictured), vice president of artificial intelligence at Databricks.Discussing AI-driven data intelligence with Databricks’ Naveen Rao.“For the last 20 years, we’ve been building data technologies, and the promise had always been, ‘If you log it, you’ll figure out something really amazing to do with it,'” Rao said. “I think now we’re getting there

play_circle_outline Development of gen AI and its impact on data architecture and investments

play_circle_outline Importance of foundational work in data warehousing and data intelligence platform

play_circle_outline Building the Foundation: Navigating the Evolution from Data Warehousing to Lakehouse and Gen AI Technologies

play_circle_outline Unleashing Gen AI: The Evolution of Data Technologies for Value Extraction and Intelligent Platforms

play_circle_outline Relationship and partnership with AWS, focusing on AI compute and hardware optimization

play_circle_outline Databricks' relationship and collaboration with AWS

play_circle_outline Databricks' 100% serverless milestone and relationship with AWS

play_circle_outline Vision for AI in next 1-3 years: building end-to-end solutions, unlocking value, new applications

Info
Transcript

Naveen Rao, Databricks

Naveen Rao

VP of AI Databricks

At AWS re:Invent 2024, Naveen Rao, VP of AI at Databricks, discusses the evolution of data technologies over the past 20 years. The concept of "gen AI" is introduced, highlighting the ability to extract institutional knowledge from data to improve business operations and create new experiences. As data complexity decreases, opportunities for automation and insights emerge. The conversation also touches on the significance of open-source models for developers and the shift towards serverless computing. The partnership between Databricks and AWS is emphasized, ... Read more

explore Keep Exploring

What has been the evolution of data technologies over the last 20 years and how are they now being used to improve business operations and create new experiences? add

What is the concept of the data intelligence platform and how can it be used to extract valuable insights and improve various aspects of business operations? add

What is the concept of a data intelligence platform and how can it be used to leverage unique data for improved products, customers, and operations? add

What is the concept of the data intelligence platform and its importance in utilizing unique data for creating better products, customers, and operations? add

What partnership does Databricks have with AWS and how is gen AI involved in their recent deal? add

What is Databricks' relationship and future plans with AWS in regards to AI compute and partnership agreements? add

What is the significance of the partnership between Databricks and AWS, and how does AI compute play a role in it? add

What advancements do you anticipate in the development of solutions over the next one to three years? add

bolt Powered by CUBE AI

Naveen Rao, Databricks

search

(upbeat music)

>> Welcome back everyone to theCUBE's coverage here in Las Vegas. We are at AWS re:invent 2024. I'm John Furrier, host of "theCUBE." We're here, you know, four days, basically three full days at a half a day on day zero. This is really day one, the kickoff of all the keynotes. Naveen Rao's here, the VP of AI at Databricks, former entrepreneur Mosaic. Before that, you know, the company sold to Intel. Naveen's theCUBE alumni, going back, I think, 2015. But great to see you. You're in the hot seat in the industry. Databricks continues to impress and thunder along.

Naveen Rao

>> Yeah. >> You know, from the Spark days and now data lakes and now generative AI is just really kind of lifted the value of the data and then the architecture and the investments in data lakes. You're seeing open table format, you're seeing developers coding with data, and obviously model diffusion. I mean, all the innovations coming in off the core data. You're in the center of the action.

>> Yeah, I mean, you know, I think for the last 20 years we've been building data technologies and the promise had always been like, if you log it, you'll figure out something really amazing to do with it. And I think now we're kind of getting there. It took a while, but now I think we're at a point where we do have some ideas of how to do something interesting like improving your business operationally or even new experiences.

>> You know, yeah, we used to have an expression, this is back in the old big data Hadoop days before that crashed, but that really spawned that kind of, when that kind of went away, also became the fertilizer for that next wave,

>> Naveen: That's right.

>> Which was, you know, all that entrepreneur big data thinking was the people who were data hoarders, I call them, like they just store data just in case. I mean, if you have been storing data and just parking it away, it's coming back in big value. And now that opened up, I think you go back maybe seven years ago and you started to see people go, "Okay, I got to get some sort of reign around my data. It's not the data warehouse. I got to think differently."

>> Naveen: That's right.

>> And then you fast forward today, it's been the pioneers. What have you been seeing as you come in and drive the AI agenda to Databricks, which did all that Lakehouse, all the great work, I call it the foundational work, but now you're starting to see new formations, more expansion. What's been the key to success for this part of history?

>> Well, I think, like you said, the foundational work. So the foundational work actually goes back to data warehousing where we organize the data, govern it, log it, monitor it, then started to add capabilities with compute on top of it. We invented at Databricks called the Lakehouse. And really now we've extended that further. I would argue that we're still in a foundational building. We're building a new foundation with gen AI and I'll talk about in a second, but we call this the data intelligence platform. How can I use my data to customize intelligence and can I build new capabilities that others can't because I have some unique data. Like those data hoarders, right? They got this wonderful institutional knowledge in their data. How do I extract that and then make a better product or better customer or better operations? That's really the name of the game right now. We're weighing that foundation.

>> Yeah, and you're seeing, I would agree by the way. We've been covering this and not, first of all, we love the whole data layer evolution, but all the core engineering's going down at the physical layer. You look at the chips Trainium2 now, it's generally available. You got ultra clusters, you've got NVIDIA with the monster I called the God box on-prem. It's the scale of infrastructure that's showing some unique value that it's hard to just, you can't just generate trajectory this economies of scale. So if you had, you're at a scale like an AWS or an NVIDIA or say Databricks, you're seeing things that are, I would say, a first generation scaled system.

>> Naveen: Yeah. >> What are you guys seeing at Databricks? Because you guys have that unique position. One, Databricks is a company with the platform, but also your customer base. You mentioned JP Morgan Chase, they're going to be coming on "theCUBE," they're just on stage. You got enterprise with significant value in data. And they're not going to just go to Open AI and say, "Hey, use it." They're going to want to build their own.

>> Naveen: Yeah. >> On-prem or on the cloud or in cloud operations, this is where everyone's going.

>> Yeah, I mean, what we found is something like 85, 90% of these gen AI projects never went anywhere. And it's not because things didn't work to some degree, but it's that this, the plumbing just wasn't there. Like how do I do things securely? How do I verify that the data that is super important and critical to my business doesn't go places. It shouldn't, how do I, you know, get access controls and monitoring worked out? So we call this the governance lawyer, and Databricks really had invented this for data. We've basically co-opted that and extended it for gen AI. Then, so that's the kind of table stakes before you can get anything into production. The next thing then starts becoming quality. So I think we had the wrong mental model in 2023, where everyone thought gen AI or LLMs were kind of magic in a box. And it turns out it's not. It turns out you actually have to engineer things to make them work reliably. And so I think 2025 is going to be exciting because now we can have these tools where customers can actually start to experience, engineer those experiences and get the performance required to support an application.

>> Yeah, and you know, remember the days when, I think when you first came on "theCUBE," we're still kind of in the infrastructure as code kind of honeymoon of DevOps and DevSecOps, you know, APIs that connect systems, but now with generative AI, they're connecting data.

>> Naveen: Yes. >> And so now you have to think of it differently. You can't just have a black box API, what's behind that from a security standpoint. And then there's all these like tunings with models, you know, weights and fine-tuning with different configurations. With the advent of say, inference in this kind of agent layer, you're starting to see that you can abstract away all that complexity.

>> Yeah.

>> How do you guys view that?

>> Because I think that's the next progression of data lakes where, okay, I got the foundation, now I got to start stitching together some value propositions that are real gettable and there are use cases out there that people are getting and from, you know, basic retrieval to some rudimentary stuff, but that they're not moving the needle yet, but they're starting to see it because now the data complexity of managing it goes away.

>> That's exactly right. So again, it comes down to that promise, the initial promise. I log it all and it will come. Well, I'll figure out a way to make it work. That's still hard, right? I think up until very recently, you still had to have someone who's a data engineer. That person had to be able to write SQL queries and organize data, do ETL, like it was fairly technical. Now what's happening is we can start to automate those components. And when we automate those components, now we can start to focus on the insights I derive. And so I think we're slowly but surely moving toward that world. And gen AI is a big step function in that trajectory.

>> You guys have been a big proponent of, I call open-source for data, and obviously every event Databricks' always been hardcore on this. Obviously, the Berkeley pedigree has a nice edge there with it. They've done a lot of things that have moved the needle in the industry, but when you start thinking about the developer relationship to the infrastructure, It's always been, you know, infrastructure as code. We used it before, we were talking about that, but like, I have a developer, okay, I got a database.

>> Naveen: Yeah. >> The old school, I code, I store stuff in a database. Now it's not database, it's a large system.

>> Naveen: Yes. >> It's a lake. It's multiple things now. As a developer, as you start to see the open-source models kind of fuel the appetite of these new open-source developers.

>> Yeah. >> You're starting to see some patterns. What are you seeing with developers right now in their relationship? Is Lambda the model where, hey, just want to function, I don't need servers. Is inference the Lambda-like, the feature, what's the developer vibe right now? How do you guys see that? Because that's the code that needs to be cracked.

>> Naveen: Yeah. >> Because once the developers have this simplicity of just coding, whether it's code assistant or engineering down to the kernel.

>> Yeah, so I think what companies want and what developers want is control over the things they need to control. Sometimes maybe that extends a little bit too far and they do things they shouldn't, but overall, I think it's about providing the right knobs and and control that you can very clearly specify. And that's code, really. And so I think serverless is a huge step forward. I actually, I think serverless is to cloud as cloud was to on-prem. It frees you up from all this management layer of things. And so Lambda, I think, was a good first step there. I'm not sure if just the function was the right way to do it, but we have our own version of serverless where now I'll be able to like spin up MGPUs not have to deal with any of the resiliency or networking, all that stuff and just access it from a notebook. That's hugely powerful. This is a really cool tool. I'm excited about within Databricks. So I think we have this capabilities of like, basically freeing the developer from those banalities (laughs) of dealing with the infrastructure to allow them to focus on the business logic of what they care about.

>> Yeah, I interviewed Ali. I can't remember which month it was. I think it was before the event. He was so excited to say we're all serverless now.

>> Naveen: Yeah. >> Databricks is now 100% serverless.

>> Naveen: On all the new products, yeah.

>> On all the new products. So that's a huge milestone. Talk about your relationship with AWS. You guys have always had a great relationship. I know Ali talks about it too, but now with Matt on stage talking about some of these new table buckets, S3 table buckets. They got Iceberg table support in the tabular stuff they're doing and other things. S3 metadata. So a lot of storage. You've seen a lot of storage activity. And then also he's introducing, and Swamy will talk about the agents, talk about your relationship with AWS and here at re:invent what you see for your customers and what you guys are doing at Databricks.

>> Yeah, we have a long history with AWS. It was actually the first cloud that was supported by Databricks. And it actually is the biggest still, over a billion in revenue that we generate on top of AWS. So clearly, very important to us. We actually just recently signed, you know, a five-year deal, solidifying that partnership. We both, us and AWS, fully anticipate that we will burn through that deal in less than five years and renew it early. Because things are just, are actually accelerating, interestingly enough. As part of that deal though, gen AI, which is, you know, obviously the thing that I think about every day, is a big part of it. How we deal with AI compute, as you mentioned, I mean, this starts becoming front and center because it's the cortex, it's the thing that actually can extract this value. We want to have a diversity of different solutions like Trainium and Inferentia. And AWS is investing heavily on. And so we're working with them as partners on this. Of course, we're to be working with NVIDIA and you know that's a gold standard of compute here. But, yeah, I think this is very exciting for us going forward. And there are lots of different places where we intersect, like Bedrock, we integrate into, and we get an anthropic models through this. There's a lot of touch points with AWS.

>> I was talking to Matt Garman and Dave Brown.

>> I just posted my conversation with Dave Brown as well. The idea of getting close to the hardware reminds us about the '90s when you had to kind of do memory management and all these like hardcore assembler, you know, machine coding and then-

>> I grew up in that world.

>> (both laughing)

>> Okay, so we're back.

>> I mean the value extraction down

>> at that level is significant when you start to integrate with the abstraction layers above it. What you can tune there, talk about how important that is for the developer, because with code assistance, the alpha developers are going straight down to the hardware.

>> Yeah, I think what no one seems to understand, at least at the beginning of the AI revolution, was that AI is a fundamentally different kind of workload. It's not like SaaS where the business logic lives mostly in the software. The value actually comes from the computations that are happening in the hardware. And when you're running a neural network on a piece of hardware, it takes all of it. It's using all the hardware. So you can't time slice in the same way. So the fundamental economics are a bit different and you know, NVIDIA's a big winner here for that reason.

>> Yeah, Naveen, great to have you.

>> I know you're super busy. I know you got 30 seconds left here on the segment. Before you go, you vision for AI as you look back, you've been an entrepreneur, now you're at Databricks. You guys are doing great, business performance, the technology's doing great. What's your vision for AI, folks watching out there, putting their customer excellence centers together, starting to figure out what benchmarks matter. There's no standards yet. They're just trying to get their arms around it. What's your vision? How would you talk to them out there about how you view AI right now?

>> Yeah, in the next one to three years, we're going to be able to build solutions end-to-end that are not just one LLM or one model. They're actually a composition of many things. And it's basically separating out the deterministic components like database lookups, function calls to these non-deterministic things, which are like squishy human language. And I think now we have this paradigm where can combine the two and build really new applications. So there's going to be tremendous unlock of value here from data and new applications we can build. So I think the next three years going to be very exciting. 2025 is really the beginning of it for me. It's when, I kind of joke, it's when the shit gets real.

>> (laughs) That's fun.

>> I got to tell you, I've never been more excited up and down the stack of action, get the code development up top with code assistance, but then still the human in the loop is going to get down and dirty and more old school up like.

>> Naveen: In some ways, yeah.

>> Yeah.

>> Enable us to be faster in those old school ways.

>> Full circle (chuckles)

>> Yeah.

>> Assembler, registers core dump, debugging. We need agents for those.

>> Exactly, yeah.

>> Naveen, great to see you.

>> Thanks for coming on "theCUBE," appreciate it.

>> Thanks for having me. >> Naveen Rao, VP of AI at Databricks. Been in the industry for a very long time and AI early and now as it starts to go next level and it gets real. He's that leading the helm over at Databricks. I'm John Furrier for "theCUBE," thanks for watching. (soft music)