Cloudera Evolve 2024 | Priyank Patel, Cloudera

Clips
News
More from Cloudera Evolve 2024

Priyank Patel

VP of AI and Ml

Cloudera

Three insights you might have missed from the Cloudera’s Evolve24 event

As the digital world grows more complex, organizations find that hybrid data management is the key to unlocking new possibilities. Bringing together diverse data streams and harnessing their full potential is no longer a luxury, but a necessity for staying competitive. The ability to blend proprietary and cloud-based data is helping organizations turn raw information into actionable insights, allowing them to fine-tune their AI capabilities and push the boundaries of what’s possible.Unified platforms such as Cloudera Inc

How Cloudera AI Inference is accelerating scalable AI with Nvidia NIM microservices for enhanced model deployments

As artificial intelligence drives faster insights and real-time decision-making across the enterprise, the Cloudera AI Inference service, designed to operationalize machine learning at scale, is gaining traction.To boost large language model performance and the private deployment of models, the Cloudera AI Inference service uses Nvidia NIM microservices and accelerated computing, according to Priyank Patel (pictured), vice president of artificial intelligence and machine learning at Cloudera Inc.Cloudera’s Priyank Patel talks to theCUBE about the transformative power of the Cloudera AI Inference service

play_circle_outline Cloudera EVOLVE24 coverage in New York City

play_circle_outline Cloudera's AI Inference Service Collaborates with NVIDIA for Advanced Enterprise Solutions

play_circle_outline Internal AI teams at Cloudera building assistants and capabilities

play_circle_outline Emerging role of GenAI builders and simplified skill sets

Info
Transcript

Priyank Patel, Cloudera

Priyank Patel

VP of AI and Ml Cloudera

The discussion at Cloudera EVOLVE24 in New York City focused on integrating NVIDIA's NIM microservices with Cloudera's AI Inference service. The goal is to efficiently run AI models on public clouds or on-premise, addressing the challenge of balancing security and compliance with AI innovation. The evolving role of developers in AI, simplifying skill sets and upskilling talent, was also discussed. Navigating the various frameworks, models, and deployment options in AI was highlighted. The concept of "accidental architecture" in data platforms, balancing centr... Read more

explore Keep Exploring

What is the buzzword of the year, if not the millennia, according to Rebecca and Bob? add

What is the announcement that was made by Cloudera regarding their partnership with NVIDIA and the integration of NIM microservices into their software stack? add

What are the two streams within the speaker's role, AI in Cloudera and AI with Cloudera, and what is the difference between them? add

What term has been internally used at Cloudera to refer to individuals who build AI systems, intentionally not calling them developers or scientists? add

bolt Powered by CUBE AI

Priyank Patel, Cloudera

search

Rebecca Knight

>> Good afternoon everyone, and welcome back to theCUBE's live coverage of Cloudera EVOLVE24. We are live in New York City. I'm your host, Rebecca Knight, sitting alongside my co-host and analyst, Bob Laliberte. Bob, let's talk about the buzzword of the year, if not the millennia, AI.

Bob Laliberte

>> What is this AI you speak of, Rebecca? I think I've heard of it, yes.

Rebecca Knight

>> A good segue, a good segue to welcome our next guest, Priyank Patel is the GM of enterprise AI at Cloudera. Welcome, Priyank.

Priyank Patel

>> Thank you. Thanks, Bob. Thanks, Rebecca.

Rebecca Knight

>> So you have a big job, let's face it. Tell our viewers a little bit about what your focus is as the GM of enterprise AI.

Priyank Patel

>> So I work with our product teams, engineering as well as products, to deliver our services that our customers are going to use and deploy and build their own applications with AI so that AI does not just remain a buzzword, but it actually delivers value. And my focus in this role is always being customer-first and to make sure that they are spending the money that they are, they're spending the time that they are with us and with the technology to get the eventual business value that they do, and we are seeing amazing results of that all throughout.

Bob Laliberte

>> Awesome. That sounds great, and I think one of the reasons why we wanted to talk to you today is that you might've had an announcement with one of those other companies that plays in AI a little bit. NVIDIA, I believe they're called. And so I'm wondering if you can talk, so clearly everyone knows NVIDIA. Everyone's seen their technology. Everyone's been talking about their partnerships with them. You've got an announcement today. Can you elaborate a little bit on what it is you're announcing?

Priyank Patel

>> Yes, yes. When you think of NVIDIA and when most of your viewers and everybody here thinks of NVIDIA, we think of them as a hardware company. The company that makes GPU and the hardware that is the core to AI. The announcement we made today has actually nothing to do with the hardware. What we are integrating is the software stack that the NVIDIA team has built out, something called NIM, NIM microservices, and that's essentially a full stack. It's an integrated hardware software layer that sits above their GPUs. About a year ago when we started working with this tech, I remember clearly going to my teams and asking, we have these NVIDIA GPUs, and with a lot of effort, we had gotten our hands on it. Though that's a separate discussion we can have. And I was like, we need to be able to run these models that we want to experiment on, and we were at that time experimenting a lot more. We want to be able to run these in the most efficient manner because there's a finite resource and it's a scarce resource and it's an expensive resource. Let's make sure we have the best performance. They tried for a significant amount of time to optimize. We got better. But then we realized that if you are going to try to optimize on somebody's hardware, you are better off learning from what they're doing with their software as well, which is when NIM came in and NIM got us started and attracted to it. We learned more of what goes into the NIM, and that really formed the basis of a service that we went general which you hear of it today. That's the Cloudera AI Inference service. It's the model serving offering from Cloudera that works anywhere on public clouds as well as on-premise, and fundamentally enables our customers and enterprises to have private endpoints for AI to be able to build and run AI privately. That's the announcement.

Bob Laliberte

>> Excellent, and I think what's really compelling to me is that it started as an internal service. You were using it yourselves before bringing it out to your customers because then it's not just, yeah, well, this isn't something we just put together and we're going to meet in the field to sell. This is something you've already worked with, you've tightly integrated it with your solution, and you're able to now bring it out and have your customers gain benefit.

Priyank Patel

>> Absolutely, absolutely.

Rebecca Knight

>> Drinking your own champagne.

Priyank Patel

>> A hundred percent.

Rebecca Knight

>> So what have you found in it in terms of working with your own teams in terms of the benefits and what makes it so special?

Priyank Patel

>> So the internal teams that work on it, we have two streams in my role that I drive. One is called AI in Cloudera. The other is called AI with Cloudera. AI with Cloudera is about us building the best platform for our customers to build their AI applications with. But AI in Cloudera is about us infusing AI within our platform without our customers ever needing to know about it. And that means there are dozens of teams internally within our organization who are building the copilots, the assistants and the capabilities that would ease the regular day-to-day user of the Cloudera platform. And as you know, Cloudera manages a significant amount of data estate both on-premise and the cloud. Our last estimate is that about 25 exabytes of data under management. That means there that many number of data users, whether it's data engineers, data analysts, data scientists who are actually interacting with our platform on a day-to-day basis. And the teams that are working internally are essentially building these assistants to enable those personas to do the tasks faster. That's what became the first use case for us, or the trigger point for us to build an efficient service to even serve out this system.

Bob Laliberte

>> I think that's so important because we've heard so many times with the rush to AI and board-level visibility and the spending on it, but now everyone's looking for what's the ROI?

Priyank Patel

>> That's right.

Bob Laliberte

>> So being able to provide your users with ways to more effectively deploy and get results faster is something that we see on and on again. Everyone's looking for give me those validated designs, give me the pre-engineered designs, give me the tightly integrated solutions so that I can accelerate the time to value with my AI solution.

Priyank Patel

>> A hundred percent. While we were doing this, obviously the world of AI, it would be an understatement to say that it evolved. It has exploded, and our customers have also, when we learned from our customers as well. While we were building this, our customers have come and told us that they are not just building applications of the kind that we are most used to, around chats and documents and Q&A, but they've started building agents which are again powered by models, and they have started to move towards these smart, think of it like bots which are taking action on behalf of the end user, right?

Rebecca Knight

>> Right.

Priyank Patel

>> Sorry, go ahead.

Rebecca Knight

>> No, I was just going to say, I mean that's such a fascinating point that you bring up because, as you said, you're working with customers who are using AI already in their jobs.

Priyank Patel

>> That's right.

Rebecca Knight

>> But we're also using it as ChatGPT has exploded in our personal lives, and we're using GenAI in so many different ways, which perhaps makes us more demanding customers as customers of in our personal lives, as well as when we are looking at service providers in terms of what we are demanding, what we need, what we want to see.

Priyank Patel

>> Right, absolutely. And I think the most visceral reaction I have is when I actually go to ChatGPT, I get 90% good answers and the 10% that I get which is not good, which is not relevant or aligned with what I want, it makes me realize that this is after all, as was mentioned earlier by my co-presenter on the stage from NVIDIA, she said models are after all new employees, they don't know anything about your company, about your context, about, in my case, my life. So they're not contextualized yet. And that really is the opportunity, obviously for companies such as us and Cloudera, this is the opportunity that we can solve our customers in the enterprise.

Rebecca Knight

>> Exactly.

Bob Laliberte

>> Yeah, absolutely. It's that ability to leverage your private data to take those models from 80, 85% efficacy to 95 or 100% efficacy. And you had mentioned also the agent, agentic AI, one of the really buzzwords that are coming out now talk about how do you leverage those agents to drive greater efficacy, to drive greater context.

Priyank Patel

>> That's right, that's right.

Rebecca Knight

>> So security and compliance is a key differentiator of this. Can you talk a little bit about what you were hearing from customers in terms of their pain points and what problems this solves?

Priyank Patel

>> Yes. So the fundamental pain point is to be able to understand that what data I'm giving to the model is, or to be able to say with certainty that the data that I'm feeding the model is not leaving the perimeter of security that I am comfortable with that that has been defined, and that is first and foremost about not leaving the perimeter. And that's where oftentimes when customers are rushing fast with POCs, experiments, they're using third-party services and making calls elsewhere. For that matter, when I'm using OpenAI, I am oftentimes uploading documents that I shouldn't be always uploading from my laptop onto the chatbot over there assuming that it won't be used otherwise, but that's not a liberty that you can take in the enterprise setting. And that's the first and foremost security and governance part. From there onward, the questions that we are hearing are more around how do we govern which parts of the data because it is also not the case that once it is inside the perimeter of security that you can have all the data go over there. There is PII data. There is HIPAA compliance data. There's compliance around data that has matured over multiple decades, and to ascertain what part of this is fair game for the model versus not is I want to say it's an open question and it's a real concern and everybody's evolving trying to figure this one out. This becomes a very strong point of discussion with our customers. It's a balance. Are you worried about it so much that you never move a finger, or are you willing to let a few experiments happen and then find the middle ground? And customers tend to struggle around it and that's the balance.

Bob Laliberte

>> No, that's a great point. There's always been that struggle between security and performance or security and agility and so forth.

Priyank Patel

>> Security, yeah.

Bob Laliberte

>> So absolutely.

Priyank Patel

>> I mean, the best way to be secure is not to do anything.

Bob Laliberte

>> Correct.

Priyank Patel

>> Naturally, you are guaranteed to be secure, just nothing.

Bob Laliberte

>> Absolutely. So one of the questions I wanted to bring up to you is how do you see some of the developers starting to work with AI? You talked about NIM's software, et cetera. How is that impacting the developer?

Priyank Patel

>> Amazing. So we used to think, I mean we've been building machine learning and data science platforms at Cloudera for over 10 years now. When we started out two years ago, the core competence of actually building these AI systems was with the data science teams, the AI teams, the machine-learning teams because that's the technology evolution of these deep learning networks. As it has progressed to now, we see, and we internally use the term GenAI builders, intentionally not calling them developers, intentionally not calling them scientists because we think that there is a simplification of the skill set and upleveling of skill set that has gone through in the industry. Consequently, there's a lot more access to such talent that customers and our clients have. And that is the kind of persona, if you may, that we are building our products to. What they're faced with is a massive explosion of hundreds of frameworks combined with thousands of models combined with yet another hundreds of places to run it and trying to find the combinatorial fix or mix over here which is going to be the best for that use case. So it is a daunting task if you are out there and trying to make sense of it. You are essentially trying to catch a moving train every morning you wake up. It's true.

Rebecca Knight

>> It's so true. One of the things that we heard on the main stage this morning was this idea of accidental architecture and how data platforms are really evolving. I'm curious to hear your take on what you're seeing and what you're hearing from customers in terms of how they're grappling with these challenges as these platforms change.

Priyank Patel

>> Yes. It's a great phrase because it captures what's really happening. The accidental architecture, it's not planned. It happens because business priorities have to be met, whether it is through an acquisition that happened in a different country, whether it is just a different team that decided to go on a different path for a couple of years and now we are suddenly stuck with these architectures.

Rebecca Knight

>> I feel like my house is almost a product of accidental architecture.

Priyank Patel

>> We'll talk about that on a different podcast, let's talk about that. All right, so it's true that the data architecture is also evolving. There's always this push and pull, and I've been in this space for a while. At the data level, you either gravitate too much towards being centralized or you gravitate too much towards being decentralized. Meaning you are always in multiple disparate systems, or you have this vision that everything is in our system. If you work in this space enough, you know that it's neither of the extremes. It's somewhere in the middle, which is there's gravitational pull towards going to a centralized architecture. But there's a reality, and we have had customers tell us, "I will not move the data out of this old system because the migration cost is more than what I can do anything with it. So it just does not make economic sense." So now what? So make it work. At this point, this is the steady state, make it work with this. And that's where data meshes and the approaches to actually access data in disparate systems yet provide a single semantic view comes up. It gets multiplied exponentially in the world of AI because think about it, where do the documents, where does the unstructured data that goes into these models sit in an organization? It's in multiple disparate systems, and some might say it's not even accidental. That is in fact planned that I am hosting my PDFs over here. I'm hosting my documents over there. My website assets are over there. All of these need to be given to the model, and you have to work in an environment which is multi-system. And that's needed for the AI systems, for AI to be built as well.

Rebecca Knight

>> Totally.

Bob Laliberte

>> Yeah, absolutely. The complexity only continues to increase as the IT environments and application environments become far more distributed. And so having that ability to unify all that data without centralizing all that data is hugely important.

Priyank Patel

>> It's hugely important. It's definitely important. It does take intentional effort. One of the customers, AbbVie, who spoke on our main stage, a lot of the amazing outcomes that they shared with us in the 20 minutes that they spoke is built on multiple years of building that underlying knowledge base, and that came with an intentional effort. At that time that they were building it, the results of what they shared today with respect to the life-saving advances that they're able to make, you could not have imagined at that time. You just had to do the centralization of that knowledge, and that's what they did.

Bob Laliberte

>> And I think that's super impressive. Obviously when you're saving lives as well, it's even more important. It was interesting, we had a guest on earlier who runs a small international airport, and it was interesting to hear you compare that massive organization with all sorts of data to also being able to benefit an organization like his. And he was able to come in very quickly, tie in five or more different disparate data sources, unify them all and drive real value very quickly with one person, essentially. So that's great to hear the contrast between the small... If you're out there and you're thinking, hey, we're too small to do this, you're not.

Priyank Patel

>> You're not.

Bob Laliberte

>> And if you're that large, maybe it takes a little bit more thinking and a little thought going into it, but the opportunity is there as well.

Priyank Patel

>> The opportunity is there. The results are, I mean, it's mind-boggling. You can't match what they said, but it's mind-boggling results.

Rebecca Knight

>> Well Priyank, thank you so much for coming on, a really, really interesting conversation.

Priyank Patel

>> Thanks, Rebecca. Thanks, Bob.

Bob Laliberte

>> Thank you.

Rebecca Knight

>> I'm Rebecca Knight for Bob Laliberte, stay tuned for more of theCUBE's live coverage of Cloudera EVOLVE24. You're watching theCUBE, the leader in enterprise tech news and analysis.