Cloud AWS re:Invent Coverage | Debojyoti Dutta, Nutanix

Clips
News
More from Cloud AWS re:Invent Coverage

Debojyoti (Debo) Dutta

Chief AI Officer

Nutanix

Simplifying AI: Nutanix brings ‘GPT in a box’ to Amazon’s EKS

Nutanix Inc. is strengthening its collaboration with Amazon Web Services Inc. by running its Enterprise AI solution on top of Amazon’s Elastic Kuberne...

play_circle_outline Nutanix's transition from on-premises to cloud and focus on Enterprise AI.

play_circle_outline Simplifying AI inference with Nutanix Enterprise AI, offering consistency and control.

play_circle_outline Need for inference closer to data to make decisions, with consistent operating model.

Info
Transcript

Debojyoti Dutta, Nutanix

Debojyoti (Debo) Dutta

Chief AI Officer Nutanix

Palo Alto Studios discusses organizations using AWS for hyper-scale cloud and AI's role in achieving ROI on data. Debo Dutta, VP of engineering for AI at Nutanix, talks about their transition to the cloud and focus on a consistent platform for apps and data management. Nutanix Enterprise AI simplifies inference process by easy and secure model deployment. It now runs on AWS Elastic Kubernetes Service for a common operating model. Internal use cases include support bot for SREs and generative AI for software developers' efficiency. Nutanix aims to support semi... Read more

explore Keep Exploring

What are some notable changes that have occurred for Nutanix in the past couple of years? add

What is Nutanix's new offering in Enterprise AI, and how does it simplify the inference process for customers? add

What is the importance of running inference closer to where decisions need to be made? add

bolt Powered by CUBE AI

Debojyoti Dutta, Nutanix

search

>> Hello and welcome to our Palo Alto Studios. I just flew in from Vegas where we're still on the ground, interviewing people as well. We have a great Kube conversation going on here exclusively as part of our coverage of AWS's re:Invent, Rob Stretching, managing director with Kube research. In this session, we'll explore what organizations that are using AWS for their hyper-scale cloud are looking to do when it comes to getting a return on investment on their data and how AI most certainly is playing a major role in this. Today I'm joined by Debo Dutta, who's the VP of engineering for AI at Nutanix. Welcome, Debo.

>> Thank you for having me here on your show.

>> Yeah, I think again, there's so much exciting news coming out of Las Vegas this week. I'm overwhelmed by the hundreds of announcements. I think there was something like 40 different launches or something like that, some really exciting stuff. But getting started, let's take a step back and look at the state of the market, because I think a big piece of this is people are really trying to understand how cloud is tracking. Not even just cloud, but actually cloud operating models and the adoption of those services. One of the things that has been an inhibitor to building AI in the cloud has been the fact that over 84% of data that organizations want to use in their models is on-premises. That leaves most organizations to have to move or copy data to cloud if they are planning to train or do inference. Let me bring you into the conversation Debo, because I think again, Nutanix really is taking a different approach. You guys started out on-premises and have gone to the cloud. You've had again started out with a storage framework, then into virtual machines, into containers, and now into AI. Give people a little bit of a perspective of your background and what you're up to at Nutanix because you have a really unique role there.

>> Yes, sir. Of course. I lead all of AI engineering in Nutanix, which means I do two things. I lead all the engineering for Nutanix Enterprise AI that we can go into, which is part of GPT in a Box solution. Also, I have a dual role to ensure that Nutanix can be made more efficient with the user generative AI.

>> Yeah, I think that to me is the real exciting thing because you're sipping your own champagne as it would be.

>> Absolutely.

>> I think that again, folks might only have known Nutanix from the hyper-converged infrastructure market. Give a snapshot of where things have changed for Nutanix in the past couple of years here.

>> Sure. Actually, I'll give you a quick overview of how we've morphed. We we were a hyper-convergent infrastructure company, then we morphed to a cloud platform company, a hybrid cloud platform company, and now we are morphing into a cloud-native company to manage modern apps, including AI of course and all the data that enterprises have under their control. In this transition, what we've noticed is that people are really embracing for the modern apps, AI, the AI agents, two major things. One is that they would like to have a consistent platform for running all their apps, including their AI apps. The second thing is that they would like to have a consistent view of managing their data.

>> I think that to me fits really well with a lot of the announcements that are going on and how really I think organizations that I've been talking to over the course of a week in Vegas, really I think the announcements that you've had, especially around Nutanix Enterprise AI, really fit well with a lot of the talk that's been going on down at re:Invent. I want to take us through what Nutanix Enterprise AI is and really how it fits into that ecosystem, because I think people will be a little bit surprised if they haven't heard of it before.

>> Nutanix, the Enterprise AI is a new offering from Nutanix, which is part of our GPT in a Box overall solution where our goal is to land our customer's generative AI workloads on Nutanix. Now, if you double-click what's happening there, one of the key components of that generative AI application development lifecycle, whether you do agents or rag, is inference. We are focused in Nutanix Enterprise AI on how to do inference really well for the enterprise. The way the product works is very simple. We simplify the entire lifecycle of inference for our customer. A customer can go and choose any model from Hugging Face or from the Nvidia catalog and then deploy the model very easily with a couple of button clicks. What you have is a model endpoint running with an API in front that talks like OpenAI. With that, customers can now build their chatbots against those and we give them enterprise-grade control, like RBAC and also they can access these models with access tokens so it's safe, secure. Essentially, what we've given them is simplicity and we've given them control and we've given them predictable price instead of per token-based pricing. What's new right now is that we have extended this whole offering on EKS, and that's what's exciting to me and to our customers.

>> For those who don't know, it's Amazon's AWS, Elastic Kubernetes Service. I got to interview Barry Cooks who was the lead of that, and I think they're doing some really neat things there as well that are very complimentary to exactly what you're talking about. I think it's actually really very interesting how these two things have come together at this point in time, because a lot of people are trying to figure out how do I do inference? Because they're looking at it and going, yes, I have to train or fine tune the models and put guardrails around it. But I'm really worried about data exfiltration and the security that it goes around that. I'm sure that there's some interesting use cases that you've guys have been using internally with your other hat on. I want to help people understand, because I think that's a real key to a lot of organizations, where to get started. Nutanix has always been good about having that easy button to get started.

>> Absolutely. When we looked inside to our own use cases, which was pretty much the similar stuff that we heard from customers, we found that there is a need to improve our support experience. What we did internally is to create a support bot. We called it SupportGPT, which would help our own SREs get to information faster so that they could give our customers delightful support that we are known for. Another use case that we are currently working on is how do we make our software engineers more efficient? While we still are in the early phases of that, but we figured out many several use cases in the entire software development lifecycle where we can use generative AI to move things faster.

>> I think what's really interesting, again, you wearing these both hats, you see there and you're sipping your own champagne. What are some of the internal use cases that you see Nutanix really diving into that you're using internally around generative AI?

>> Yeah, it's been so exciting as a journey internally. We've seen several use cases. In almost every part of Nutanix, we see a use case, but we prioritized and we picked up two use cases in the last year or so. One is to create a chatbot internally to help our SREs find information really quickly and then provide the delightful support that Nutanix is known for. The second use case is a little bit more early, but we are using generative AI to help our developers go faster, which means it's not just the code generation, but the entire lifecycle, which includes testing, code understanding, code review, and also code explainability, and things like that. I think this is just the beginning. We are finding out that there's so many use cases, it's just that we have to prioritize them and make sure that these are of high quality, the use cases that we build, because one of the biggest challenges in building internal use cases or external use cases is data quality and the ability to actually have good outcomes and results from these data science experiences.

>> I love those ideas that help people really focus on where they can get started through examples that people are being successful with. Obviously, sipping your own champagne and helping your software developers be more efficient in their production of code and as well as your SREs, your site reliability engineering team, I think that is key to organizations and pretty much transferable to almost every company in the world.

>> That's correct. Yeah, absolutely.

>> Yeah, and I think that was a big conversation around that is how do you do that but keep your own style, your own code base so that people become more efficient with Nutanix's way of building software versus just generalized? Also, I would assume that this helps Nutanix from a perspective that you can control the data that's in there so you don't have to worry about other IP infringement or anything like that.

>> That's correct. That's a very tricky problem to solve actually. You've hit the nail on the head, because I know a lot of people are generating code today, so we've taken a very conservative stance on not generating any code that ships to our customer, because we want to make sure that it is completely conflict-free in terms of intellectual property. What we do is we use generative AI in the other parts of the software development lifecycle, like code understanding, code review, where we are not actually inserting generated code. We are still not there yet.

>> Well, I think that's a great takeaway for customers that are out there and organizations that are saying, "Hey, I want to use this." In fact, it's funny, I was talking to a couple developers this week, just earlier this week, and that was a big piece of what they were looking at was the fact that they were not trying to generate code per se out of that, but they were trying to figure out, "Okay, where do I get started? Do I start with code review or QA" or what have you? I think that's a great lead-in to there. Now, I don't want to bury the lead here because it is re:InventWeek and things of that nature. Why was it important to yourselves and to your customers to really allow Nutanix Enterprise AI to run on top of EKS?

>> Well, that's a very good question. We actually realized that many of our customers are also customers of AWS EKS, and they're using EKS for running all their workloads. You want to typically co-locate your AI where your other workloads are, where your data is. The other thing is what we realized is most of our customers want a common operating model, they want consistency. Most of our customers, including ourselves, we are hybrid in terms of where our computation is, where our data is, and where the AI is going to run. When you are running AI on-premise as well as on AWS, you need to be consistent in terms of your tooling, your user interface of the tools that you use. That way, your enterprise IT administrator can become the new AI admin and have a consistent view of how to manage the entire sprawl. That's one of the key reasons why we did it.

>> Yeah, I can see, because again, we were at KubeCon and we talked to you guys at KubeCon as well, and I think ... That was almost a month ago now it feels like yesterday, but yesterday was ...

>> That's a .

>> Yeah, I know. But when we start to look at it, there's platform engineering and people where it used to be VM admins, storage admins, and network admins. Now they're all coming together again as these platform engineers wearing different hats. Oh, by the way, there's ML ops and AI ops.

>> LLM ops.

>> Yes, and LLM ops. There's Ops everywhere. I would say that that has to be also a focus to your point of why. Because with 84% of the data being on-prem, it seems to me that that's why you would want to do that and have that data, so you can go and build your models and train your models, but when you're doing the inference, you want it closer to the people who are actually doing the work.

>> Yes, we believe that inference needs to be done closer to where the decisions have to be made. When you train the model, you have to run that workload closer to the data which you want to use for your training or fine-tuning. We find that for most of our use cases and our customers, that's also a very key thing. They want inference to be done all the way to the edge in certain use cases. Some of our customers are running most of the things in the data center, and they'll do it in the data center, and then they have a hybrid, so they're running it in cloud. For a spectrum of use cases, you want a consistent operating model, a simplified view of inference from the edge all the way to the cloud. That covers your entire gamut of installation of your enterprise IT.

>> Yeah. In fact, that came up. We were having a conversation as analysts got to have a conversation with Matt Garman yesterday, and that came up as a big ask for customers is that really how do you bring AI everywhere, not just all centralized? Because not everybody's, in fact, I would say the vast majority are not trying to go and build the next ChatGPT themselves to productize that as an offering. They're using chat as a way to, like you said, enhance their customer service experience, their SREs, so they can get to the answers and code faster. Is that where you see the center of gravity of where you're aiming this solution at?

>> That's a interesting question. I feel that we will start at simple use cases like what we just talked about, customer support, generation of content, whether it's code or PowerPoint presentations or even documents and also private document search, and of course security. Everybody wants to use generative AI to be more secure. A lot of our use cases around these issues, but that's I think just the beginning, that's phase one. In phase two, I believe that we'll see a lot of semi-autonomous agents, AI agents who will do certain tasks on their own. I think that's going to be, that's my personal-

>> I was going to say, is this where you're going with the product or where you see the product going over say the next 12 months is agentic?

>> No, I think we have to support agentic workloads, but with this Nutanix AI product, what we realized is the more people do agents, the more people do RAG and other use cases, there's going to be one constant thing, the more inference you'll need. We want to first give our customers delightful experience in just doing really high-quality inference, and give our customers the choice to run it anywhere on EKS, on-premise, at the edge, and also a variety of models, whether it's NVIDIA NIM or whether it's models on Hugging Face like Llama 3.2. It's important for customers to know that they can have one simple consistent operating model for running models. Then with the product, they can exactly see who's using which model. Like the admins, they have full visibility, full control. I think that's important in the enterprise.

>> It's super important I would say, because you want to understand how it's being utilized. I think again, it's really important to bringing it back, that full circle of development as you start to iterate through and you get that flywheel going.

>> Yes, exactly.

>> This has been great. I know people can go and find more out at Nutanix.com/AI. Great landing page there. I went and checked it out after the launch.

>> Thank you.

>> Thank you for coming on board, and I really appreciate you coming here. Again, it's been crazy with all of the announcements. I think what you guys are doing really ties up really well with what's going on at re:Invent this week.

>> Thank you very much for having me here again.

>> Yep. Thank you for watching this episode of the Kube Live from Palo Alto and Las Vegas, where we're breaking it all down for you. We got everybody on there from Andy Jassy all the way to the heads of all of the different services, plus all of the partner ecosystem like Nutanix, who's building and integrating in to give that customer a better experience. Stay tuned for more live from Las Vegas and from here at the studio and from some remote as well. We'll see you soon.