KubeCon + CloudNativeCon NA 2024 | Arun Gupta, Intel

Clips
News
More from KubeCon + CloudNativeCon NA 2024

Arun Gupta

VP & GM, Open Ecosystems

Intel

Intel embraces open-source ethos with AI microservices platform

Open-source artificial intelligence projects are gaining popularity, and Intel Corp. is taking advantage of the momentum with their composable AI platform.Open Platform for Enterprise AI, or OPEA, was launched by Intel this year as a vendor-neutral project with the Linux Foundation. Its potential is immense, according to Arun Gupta (pictured), vice president and general manager of developer programs at Intel.“When you’re building a gen AI application, you need a whole bunch of microservices,” he explained. “The first combination of OPEA is all of those about 30-plus microservices. These are all cloud-native, so they’re published as containers and you can run them anywhere

play_circle_outline Optimizing Gen AI Applications with Customizable Integrations and Vendor-Neutral Microservices from OPEA Project

play_circle_outline Prioritizing features based on technical steering committee validation and community feedback.

play_circle_outline Partnerships with companies like AMD for CI-CD validation and performance optimization.

play_circle_outline Focus on CPU optimization for inferencing at the edge and data center.

Info
Transcript

Arun Gupta, Intel

Arun Gupta

VP & GM, Open Ecosystems Intel

At KubeCon North America in Salt Lake City, Savannah Peterson and Rob Strechay discuss the OPEA project with guest Arun. OPEA is an open platform for enterprise AI created through a collaboration with 15 partners. It aims to provide a customizable stack for building gen AI applications with over 30 microservices available in containers. These microservices can be combined to create blueprints for various applications, including a RAG chatbot. The project is vendor-neutral and overseen by a technical steering committee with 11 members. Arun emphasizes the impo... Read more

explore Keep Exploring

What is the significance of OPEA in building a gen AI application and how does it involve a combination of microservices and blueprints? add

What governance model is in place for the party project, specifically regarding the technical steering committee and the roadmap for OPEA? add

What benefits does AMD bring as a partner in terms of validation and performance optimization on Xeon processors, along with the inclusion of their GPUs in CI-CD validation? add

What is the most prevalent commodity across hyperscalers data centers, and what direction are customers liking in terms of inferencing models? add

bolt Powered by CUBE AI

Arun Gupta, Intel

search

Savannah Peterson

>> Good evening, Cloud community and welcome back to Salt Lake City, Utah. We are here coming to the conclusion of our three days of coverage at KubeCon North America. My name is Savannah Peterson, joined here by the fabulous and ineffable Rob Strechay. Rob, we're just barreling through the day.

Rob Strechay

>> Today has been great. I mean, the community-

Savannah Peterson

>> It's actually going by pretty fast....

Rob Strechay

>> and it's very fast and very interesting and wide-ranging, from the highest level applications with AI. Now we're going way deep down, which I love and get to where the rubber meets the road or the software meets the data center, depending on how you look at it.

Savannah Peterson

>> Hey, I love it.

Rob Strechay

>> Or cloud.

Savannah Peterson

>> Yeah, however you want to call it. Speaking of someone who's going to help guide us who this conversation, one of my favorite KubeCon guests to have on the Arun, welcome back.

Arun Gupta

>> Thank you. I'm very happy to be here.

Savannah Peterson

>> Yeah, your smile always makes me smile. You're the perfect, at the end of the day, good vibes, good energy. You all have made some announcements and done some interesting things since the last time you and I had the chance to catch up in Chicago. Can you give us a little refresh of where we're at right now?

Arun Gupta

>> That's right. So, back in April at part of the Open Source Summit, we announced this new project called OPEA, Open Platform for Enterprise AI. And this is, as we started looking at different gen AI efforts happening inside Intel with our customers and partners, we realized lots of customers and developers are building these open source gen AI stack very bespoke, but very similar, in a way. So, what we did is we launched this project with a help of about 15 partners and we gave this to Linux Foundation. And so this is a vendor neutral third party, with LF AI & Data Foundation and it's a project that sits over there and we're very excited about it.

Savannah Peterson

>> Well, I can imagine. And I am curious, so open source, very important to Intel clearly. Talk to me more broadly about your strategy, what being here means, how you're implementing open source technology within the company.

Arun Gupta

>> Yeah, so I mean open source has been very essential part of Intel. As you said, I was looking at our history, first contribution goes back almost 40 years to GNU Compiler.

Savannah Peterson

>> Wow.

Rob Strechay

>> Wow. Yeah, that's-

Arun Gupta

>> So, it's a long history. And over the last 20 years, we have only accelerated that much more. We are one of the top contributors to PyTorch, TensorFlow, Kubernetes, OpenJDK, and these are massive projects. And the reason we contribute to these projects is because our customers who are sitting in data center, hyperscalers, laptops, edge network, they expect that these 300-plus open source projects we contribute to, they're optimized and leveraging the latest instruction. So, we contribute straight up stream changes so that the customers have a very delightful experience.

Rob Strechay

>> And I think one of the challenges, and I think you touched on it, was again, there's so much open source. I mean, the open source AI day yesterday was packed and there's so many down in the project pavilion and all the way down there. I mean, it's great to see so many companies and projects starting up, but that also comes with complexity and things of that nature. So OPEA is really around helping determine an opinionated stack. Is that how we should read that and how people should phrase that?

Arun Gupta

>> That's right. Opinionated to begin with, but fully customizable. Think about batteries included, but replaceable. So, that's the terminology that I like to use.

Savannah Peterson

>> I like that.

Arun Gupta

>> What OPEA is, think of it as when you're building a gen AI application, you need a whole bunch of microservices, embeddings, retrievers, rerankers, LLM, vector database. So the first combination of OPEA is all of those about 30 plus microservices. These are all cloud native, so they're published as containers and you can run them anywhere. Now, the microservices by itself are not very helpful. What's really impactful for developers and the end customers is blueprints that sit by building those microservices together. So, we have gen AI examples, 20 plus of those, which you probably saw on the keynote this morning. So one of them is a RAG chatbot. Now you want to build a RAG chatbot, we are giving you a Docker Compose file or a helm chart. So, it would run on any compute instance anywhere you have Xeon or whatever it runs. And then with a single click, you can deploy your RAG chatbot. Now that opinionated stack, as you talked about, comes with Redis, TGI as a backend server, TEI as an embedding, and so on so forth. But within OPEA, what we have done is we have done integration with, instead of TGI, hey, you want to use BLLM? Sure, that integration is done. Instead of Redis, you want to use Quadrant, Chroma, Milvus, even Pinecone for that sake, which is not open source, but the idea is as an open platform, so it's just an API contract that you got to honor. So we provide that diverse and wide set of integrations for you, which makes it very easy for customers to get started and bring those levers on. So, some customers like, "Oh, I don't want to change the levers, I'm good with this." And some customers do want to get into the details and change the levers to customize, because their source of truth, maybe something else, not Redis necessarily. So that gives them that flexibility.

Rob Strechay

>> That would seem like, again, you're giving optionality still to the end user, but you're leading them to that, so they don't have to be an expert in every step of the way. Because we've been talking and to a number of people today and it's like, "Hey, I'm an infrastructure person, not a data scientist. My data scientists want to work in PyTorch or something like that, but I need to give them the right platform, TensorFlow or what have you, so they can build the algorithms and start to build out what they need to do." And this is more than, even though you have the chatbot, it helps with more than just gen AI, it's helping with AI in general, it would seem as well.

Arun Gupta

>> Correct. Yeah. So I mean, if you think about from the data scientist perspective, they train these models, fine tune these models, but that's a training story. But once the model is trained, the model is available, all you do is you do that integration with OPEA because end of the day, you're not going to use the model as is, right? You're going to integrate the model as part of a blueprint, bigger blueprint. Model is just only one part of the solution. But when you're building the solution, what are the Lego pieces around it to build that beautiful castle that you want to build? And then-

Savannah Peterson

>> I like where this is going. I'm here for the castle.

Arun Gupta

>> How about that? And I don't like my castle door to be red. I want it to be green. You can do that today. So, that's what OPEA provides.

Savannah Peterson

>> How are you prioritizing? I can imagine, I mean, you work with the biggest companies in the world, and I can imagine a lot of people want a lot of different things. You mentioned that people were building similar stacks, but when it comes to the features that you're developing, how do you prioritize that and what's the feedback loop like with the community?

Arun Gupta

>> Yeah, so OPEA is a vendor-neutral third-party project sitting with LF. So, the governance model is very clear. There is a technical steering committee, 11 seats on the technical steering committee, out of which only two are Intel because we were the initial contributors.

Savannah Peterson

>> Wow. Cool.

Arun Gupta

>> That was super important and very intentional to begin with.

Savannah Peterson

>> Love that.

Arun Gupta

>> Because we want that TSE to be defining the roadmap for OPEA, even though Intel is putting the engineering efforts and other partners are putting into, but it's a very vendor-neutral, that method. So, essentially the roadmap is created, then it's validated by TSE, they give a thumbs up and then we go ahead and execute on it. Now, other exciting part is when OPEA was launched, we had about 15 partners. Now we have about 45 plus partners. We have seen that grow dramatically. One of the partners that we are particularly excited about is AMD. Now AMD is a partner. It's an Intel-created project. So of course we do the validation and performance optimization on Xeon. With that, AMD gets all the free goodies, but now they're bringing their hardware as part of the CI-CD validation part of it. So it's not just, you know, AMD, CPUs, but their MI-300, which are their GPUs that they're going to enable and validation on. So from a customer perspective, what they're getting is a mix of everything that they want to see, because otherwise you will have to do all of that validation, but now with OPEA, you get that validation by itself.

Savannah Peterson

>> That's exciting.

Arun Gupta

>> It's very exciting actually.

Rob Strechay

>> Do you see, and again, we've been seeing a trend where, rightfully so, people aren't building out their foundation models. Mostly. I mean, the big guys are going to still do that, and that's for their own reasons. But most people are looking at it, they're maybe training them, fine-tuning it then like you said, they're building out their rag and things of that nature, but then they're looking to do inference. And it would seem, again, people are looking for efficiencies at the edge in particular, and that's where CPUs are. I mean, I don't think people are going to be putting GPU... I mean, maybe they're putting GPUs everywhere, but cost-wise, people are looking at how do they use... Are you seeing a lot of some of the work that's happening in there to really optimize inference at the edge? And for those other use cases that are not these massive training models and things of that nature?

Arun Gupta

>> Not just at the edge actually all across, even in data center as a matter of fact. Because it's always a trend, right? The pendulum swings too hard to the right. Sometimes it's, "Oh, I need to write a hello world and I need a GPU for that." Let's think about it.

Savannah Peterson

>> Yeah.

Arun Gupta

>> Let's think about that for a second. Do you really need it?

Rob Strechay

>> Yeah.

Savannah Peterson

>> Hundred percent.

Arun Gupta

>> So what you are seeing from our customer base is, if you want to run a billion parameter model, a 5 billion or a 7 billion parameter model, and you only want to do inferencing with a RAG, that runs really well actually on a CPU.

Savannah Peterson

>> Yeah.

Arun Gupta

>> And guess what? The most prevalent commodity across hyperscalers data centers is Xeon. So in that sense, a lot of our customers are liking the direction that we are taking that, yeah, bring on this OPEA capabilities. We're going to do the inferencing on the CPU, whether it runs in data center, whether it runs in a cloud, whether it runs on the edge. So all of that capability is there. So, I think that's the beauty of it.

Savannah Peterson

>> That is beautiful. We've had Ryan Tauber who runs the Xeon team on the show and always have really exciting conversations. I think there's something really, is compelling there. And I am really excited to see what starts to happen as we see AI adoption really hits scale across different verticals and a lot of different things happen. I want to take the conversation a very different direction, and I want to talk about your fabulous facial hair.

Arun Gupta

>> Thank you.

Savannah Peterson

>> Because you always have the most striking Movember aesthetic of anyone. You impressed me at last KubeCon. It is Movember. We do care obviously about men's health. The team is entirely men except for me, so I better care about men's health. But I love how passionate you are, and I know it's a very personal mission for you. So, can you tell us why you're such an avid participant?

Arun Gupta

>> Yeah, I think the whole mission of Movember is no shave November basically, and I've been a public speaker for a very long time. For over a decade, I have leveraged this stash to really make a statement about men's health. And oftentimes when you are a man, you think I'm more masculine, more macho. Hey, you know what? I don't want to be hung up on a poster in my family room. I want to play with my kids on the family wall floor. I want to play on the floor. So essentially this is a statement about men's health, mental, social, and physical. And our surgeon general, Vivek Krishna Murthy has declared men's health as a pandemic and he talks about how men should not worry alone. As a man, we have a tendency, "No, I'm going to figure it out." Hey, you know what? If this is something bothering you, this is probably bothering very many other men in your family as well. Or talk to your partner or talk to your brother or talk to your father or talk to your son, whoever, or any other female or any other person that you trust. Don't worry alone. So, that's one part of it. The second part of it is we were talking about self-care. Make sure you take care of yourself. Because KubeCon are hectic schedules, but I took out time this morning, 50 minutes, for an intense workout, and that's what allows me to sustain because I've had me time and I'm physically ready to manage it. And then the social element of it right here itself, you see so many people. To me, hugs are very therapeutic.

Savannah Peterson

>> I'm with you there.

Arun Gupta

>> Just having a hug, just rubbing a little bit back on the back, it goes a long way. And again, it's everybody's comfort level, so I seek that permission first. But then to me, that's very... So I think those are simple things that I care about it and everybody in their life has a man, a father or a brother or a partner or a friend or a colleague. Let's make men more healthy as well.

Savannah Peterson

>> I love that. And so wonderfully stated, I'm happy I'm a part of your therapy plan. You came up and gave me a big hug right away, and it made me very happy as well. I'm on the same team. I'm curious, because I think this is a really important conversation that we don't talk about enough, particularly at the enterprise tech level, self-care isn't always the hottest topic in our interviews. How do you promote a culture of compassion and empathy and self-care like you just described within your own team?

Arun Gupta

>> I think it's to be a role model. My charter in the team is a lot bigger. I run the developer programs for all of Intel and basically saying that, "Hey, you know what? I can't take a meeting from 6:30 to 7:30 in the morning because that's my running time, and I'm not going to do that. And I'm not going to change it." Declining meetings so that your team feels empowered to do that too. And respecting that. I am going to take a 45-minute lunch break and I'm not going to be on a call because I need to fix up a salad and I need to go for a walk around the block. I am going to wrap up by 6:00 P.M. and I'm not going to take a late night call because I need to do the dinner because if I'm not going to do it, I'm not going to feel happy. So, I think creating those boundaries for yourself goes a long way and guiding your team that it's okay to do that for yourself. I think that's the way to go for it. And really, self-care is what this boils down to.

Savannah Peterson

>> Those boundaries, yeah.

Rob Strechay

>> And leading by example. I think that's a big piece of it.

Arun Gupta

>> Oh, my God.

Rob Strechay

>> I've done similar stuff for my teams as well, because in some ways I've also said, "Don't be me," sometimes because our travel gets to be so extreme and stuff like that. But again, it's really interesting, and I appreciate you doing that as well.

Arun Gupta

>> Well, I mean, you were talking about, in a plane, they say, "Put your mask on first before you put others." Think about this way, the advice that you give to your friends and family to live a healthy life, do we follow that same advice for ourselves?

Rob Strechay

>> Yeah, exactly.

Savannah Peterson

>> It can be very tricky.

Arun Gupta

>> To talk to yourself as your best friend and live that healthy lifestyle.

Savannah Peterson

>> I love that. One of the things that I do for most events, I actually didn't for this event for a variety of reasons, but I always do my nails based on the color of the event. And I make sure that I go get my nails done during business hours because that is not only self-care, it's also a part of my job and a part of my uniform, and I very much normalize it with the guys. I'll ask the customers, and I'm on my way to the nail salon and I'm like, "Yeah, this is a part of the deal. I shouldn't be having to take my personal Savannah time to do something that is a part of the aesthetic that people comment on the whole time." So I just-

Arun Gupta

>> I really respect that. No, I think that's super important. And calling it out.

Savannah Peterson

>> Yeah.

Rob Strechay

>> Yeah.

Arun Gupta

>> Calling it out. that is the norm.

Rob Strechay

>> Absolutely.

Savannah Peterson

>> Yeah. No, I always love chatting with you, but I think if folks took nothing else... Well, obviously they should go check out OPEA, but if they took nothing else from this though, think about that-

Rob Strechay

>> There was a little AI thing going on, but-

Savannah Peterson

>> There is. There is, but frankly, that AI thing doesn't matter unless we're all feeling great about it and living our best healthy lives, and we all want AI to make us healthier humans. All right, Arun, last question for you because we could just go on and I can tell I'm at the stage of the day where I just want to take the train with you on the self-care path. When we have you back on the show, which we obviously will since we're here back together again, what do you hope to be able to say a year from now in Atlanta, for example, or in London that you can't yet say today?

Arun Gupta

>> So, my hope for OPEA project is I want to have these OPEA solutions available in all five hyperscalers, in their marketplace. Because right now, the solutions are validated on the hyperscalers, but it's a bit of a manual step. I want a single-click deployment. I want to be able to see a wide set of integration, which are well-documented, a great customer adoption, single-click. I want to deploy on AWS, on Azure, on GCP, on Oracle. Pick a favorite cloud provider of your choice and it's available. I think that would be a good measure because end of the day is building that mind share and making it simple and easy for developers and customers to deploy these applications, because if you do the education well, selling will happen.

Savannah Peterson

>> Love that. And we look forward to hearing about all of those single-click deployments when we have you on the show next time. Arun, thanks for being the best and fantastic. Are you going to participate in Movember one of these times?

Rob Strechay

>> I haven't shaved this in a while.

Savannah Peterson

>> It's true. It's true.

Rob Strechay

>> Well, actually, this is longer than normally.

Savannah Peterson

>> You're kind of like in a perpetual Movember. about self-care.

Rob Strechay

>> We'll see if I can get to December without touching this stuff. That stuff, we'll see.

Savannah Peterson

>> We'll see.

Rob Strechay

>> Anderson may have problems with my stash in the microphone at some point in time, but we'll see.

Arun Gupta

>> It's calibrated for that.

Rob Strechay

>> Yeah, he's good with that. They're professionals. They're the best.

Savannah Peterson

>> You just got to start shaving a little bit of a line right here just for the mic.

Rob Strechay

>> Funny.

Savannah Peterson

>> I need to do that as a joke at some point. Anyway, thank you for joining me, Rob, Arun. Fabulous. And thank all of you for tuning in wherever you might be. We're here in Salt Lake City, Utah coming to the end of day one of KubeCon North America of our three days of coverage. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.