TheCUBE Research’s Rob Strechay and Savannah Peterson speak with Bobby Allen, cloud therapist at Google Cloud, and Brandon Royal, senior product manager for Google Kubernetes Engine, for a Google Cloud Primer on AI and K8s, detailing the transformative intersection of AI, Kubernetes and GKE.
In this video, learn more about:
• How to get started in AI
• What it means to train a model
• Different model sizes
• The importance of inference
• How to evaluate the many models available
• The current state of AI
• Optimizing containers and AI at any level of scale
• "Farm to Table AI"
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Google Cloud: Passport to Containers. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For Google Cloud: Passport to Containers
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Google Cloud: Passport to Containers.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Google Cloud: Passport to Containers. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to Google Cloud: Passport to Containers
Please sign in with LinkedIn to continue to Google Cloud: Passport to Containers. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Google Cloud Primer on AI & K8s with Bobby Allen and Brandon Royal
Join theCUBE and Google Cloud for a special series navigating today’s AI landscape for container optimization at scale. Gain from our analyst-led deep dives exploring what it means to train a model, how to evaluate the countless models available in the market, and why AI is the sauce but isn’t necessarily the main dish.
In this video you will learn: - how to get started in AI - what it means to train a model - different model sizes - the importance of inference - how to evaluate the many models available - the current state of AI - optimizing containers and AI at any level of scale - "Farm to Table AI"
Google Cloud Primer on AI & K8s with Bobby Allen and Brandon Royal
TheCUBE Research’s Rob Strechay and Savannah Peterson speak with Bobby Allen, cloud therapist at Google Cloud, and Brandon Royal, senior product manager for Google Kubernetes Engine, for a Google Cloud Primer on AI and K8s, detailing the transformative intersection of AI, Kubernetes and GKE.
In this video, learn more about: • How to get started in AI • What it means to train a model • Different model sizes • The importance of inference • How to evaluate the many models available • The current state of AI • Optimizing containers an...Read more
Savannah Peterson
Principal Analyst & HostSiliconANGLE Media, Inc.
HOST
Rob Strechay
Dir./Principal Analyst & HosttheCUBE Research
HOST
Bobby Allen
Cloud TherapistGoogle Cloud
Brandon Royal
Senior Product Manager, Google Kubernetes EngineGoogle
Google Cloud Primer on AI & K8s with Bobby Allen and Brandon Royal
search
Savannah Peterson
>> Hello, cloud community, and welcome to a very special educational segment we're doing with the Google Cloud team. My name is Savannah Peterson, joined by my fabulous cloud wingman, Rob Strechay.
Rob Strechay
>> Love being in the cloud. Love being up there with you, flying high.
Savannah Peterson
>> Likewise, Rob. We have quite a view here from the stratosphere in our world.
Rob Strechay
>> Absolutely.
Savannah Peterson
>> And sometimes getting the high-level view and hanging out with all the thought leaders we do, we really need a conversation that grounds the technology and the hype and the movement that's going on right now, which is-
Rob Strechay
>> Totally agree....
Savannah Peterson
>> exactly why we have the two fabulous guests that we have with us. Bobby, who might as well be sitting here, he's on the CUBE so much these days, which I love. Welcome back, Bobby.
Rob Strechay
>> Thank you for having us.
Savannah Peterson
>> Yes, and Brandon, new face, bright face, very delighted to have you here with us.
Rob Strechay
>> My first time. Yeah, I'm so happy to be here. Thanks for having me.
Savannah Peterson
>> We'll try and make your first time as painless as possible, Brandon.>> I'm very excited for it.
Savannah Peterson
>> So before we dive into some of the technical stuff, Bobby has hyped you up a lot.>> Okay. All right.
Savannah Peterson
>> In the best way, and I'd love for you to give the audience a little bit of your background, because you're an OG here, you've got a wealth of technical knowledge.>> Yeah, yeah. I've been in the cloud-native space for quite a while, almost 10 years. I've been doing the sort of container thing since 2016, 2015, and sort of been following this sort of evolution of containers and Kubernetes and this intersection of machine learning and AI, which I don't know if you've heard is kind of a thing now. So, it's quite exciting. So it's a fun time to be working on AI and cloud-native technologies and Kubernetes and AI infrastructure. That's really what I bring together is really sort of the combination of those things. And what I do at Google is I'm a product manager on the GKE team, so I primarily work with customers who are doing large-scale training, large-scale inference, machine learning operations, all those things on top of Kubernetes and GKE. So it's a lot of fun. It's very exciting. Every day is new. Technology is moving incredibly fast. But yeah, it's a lot of fun.
Savannah Peterson
>> Yeah, it is a lot of fun. And speaking of those acronyms you just talked about, Bobby, GKE and AI, having a moment.>> They are. We are.
Savannah Peterson
>> Yeah.
Savannah Peterson
>> We are for sure.
Savannah Peterson
>> Tell me what's going on. What are some of the trends you're seeing, the velocity you're seeing? Why is this conversation so important to have right now?>> I think, well obviously it's been kind of the year or the past 18 months of AI, like everybody talks about it. You can't go anywhere without someone sprinkling some AI on a little bit of everything. I think even grandmas and grandpas are messing with AI at this point. Like, "Help me get on the Gemini thing or the chat GPT thing," or whatever. So everyone's kind of seeing it now. It's bleeding into our personal lives, not just our corporate lives. And I think that's why it's kind of everywhere. But I think what we're also seeing is that it's becoming something that, it's futuristic, but it's also bleeding into everything. Like kids to grandparents, again, the range of people that want to play with this stuff, that want to touch it, they want to understand it, they want to get their mind wrapped around is this a next year thing? Is this a tomorrow thing? I think people can see the pace or feel the pace speeding up every day. Every week there's something else coming out and they're kind of like, "What's next that I might... What's the new hotness that I need to make sure that I'm up on?"
Savannah Peterson
>> Yeah, exactly. What is the new hotness? And speaking of getting up to speed, training, you and I were talking about track and field last night. Training can mean a lot of different things. Everyone uses the term here, throws it around, but Brandon, what does it really mean to train a model?>> To train a model?
Savannah Peterson
>> Yeah.>> Absolutely. So training model is really all about taking some data. I mean, we're using a fundamental set of technologies that have actually been around for a while. We're using deep learning and neural networks essentially under the covers to make models that can do predictions. And now if you look at AI as we define it today, sort of modern AI, we use things like large language models or LLMs, and those are technologies that can be pre-trained with a whole bunch of data. Think of all of the knowledge of the internet and human civilization codified into a single model that can then be delivered to people all open, generally open source or are available on the open internet. So, it really varies quite a bit. It could be training a very small model with a very specific set of information, specific set of data that then has a prediction outcome. Or it could be all the way up to very large language models that are doing incredible text encoding, text generation or even image models. It really varies a lot by use case, but training is really all about taking that data and making something valuable out of it.
Savannah Peterson
>> Oh, go ahead, Rob.
Rob Strechay
>> So what do you see as the difference between, and what is inference and then what is fine-tuning and how did those play with training? Because I think a lot of people think, "Well, I trained the model and I'm done."
Savannah Peterson
>> Or, "Now I'm going to achieve ROI."
Rob Strechay
>> Yes, or, "I'll be at ROI," which is even bigger question.>> Yeah, it's a really good question. So a model is really only valuable when it's in the hands of its users. It's just like software, right? We talk about how much time we spend building software in our development environments. It's really not until it's in the hands of a user that it delivers actual value. And the same thing is true with inference, right? A model is only valuable until we can put it behind an API and make it available to do something interesting. And so that's really what inference is all about. It's taking a model, whether it's a large language model or a diffusion model for images or a simple model and providing an endpoint by which we can expose that to users. And that's really where the fun and interesting stuff happens. Inference used to be really just a part of the model life cycle. So you train a model, you evaluate the model, you test the model, and then eventually you take all that work and then you put your model out there and you deliver it, which is the model that we've kind of been used to for the last several decades. But now with large language models, we can take off-the-shelf models and basically just start with inference, which is sort of a total paradigm shift. I can take a Llama model as an example, I could take it off the shelf and I could do something really valuable with it. So, inference is becoming more and more critical to businesses that are looking at deploying AI models in their platforms.
Savannah Peterson
>> You just brought up a really important point and I'm glad you said that. Taking things off the shelf. I think also when people think about starting their AI journey, which I know we talk a lot about on-ramps and how to get people on the right path, they think, "Oh my gosh, I've got to have all this compute, I've got to have all this data. It's got to be clean so that I can put it into this model to be able to do this," which is true if you're making your own model and you're doing your own AI, however, there are so many different off-the-shelf models that you can run to then flip the switch. How do you help people that you're working with navigate that solution? I imagine that's a conversation you have, to build it or the use what's already there or maybe grab something and then add on top of that. What's that conversation like?>> Yeah, it's a really good question. It really comes down to evaluations. Evaluations are now like a sub-domain in and of itself. How do you evaluate the performance of a model? And it used to actually be pretty simple. To evaluate performance, you give it the same sort of input and it gives you a similar kind of output and you can validate, is that better or worse? But now that we're getting into large language models, doing that evaluation is quite human, it's quite subjective. And so over the last 12 to 18 months, this sort of evaluations as a sub-domain has really continued to evolve. And there's some amazing frameworks work that's happening in the open community, work that's happening with partners of ours like Hugging Face where they're providing leader boards and tools. So you can now actually see, "How does my model perform against the common open benchmarks that are available?" So now I can say, "How does Gemini perform against this particular model?" Then of course there's the use case specific stuff. That's the general purpose. Then there's also, okay, now I bring it into my organization, how does it perform for my domain specific tasks? It is a complicated answer to a simple question, but I think it's evolved in a really meaningful way to make more valuable.
Savannah Peterson
>> I'd love to jump in and just to take a step back to kind of come get the people who may think we're going all the way kind down the rabbit hole as Brandon has given us some great information, but every AI conversation can go in so many different directions. And so when I talk to customers, I try to kind of categorize which type of AI conversation do we want to have? Are we talking about what you're building, how you're building or how you're operating? And so Brandon is going down a path really of what we're building. Are we building models, training? Are we hooking models to applications, inference where we're leveraging things off the shelf or tweaking those? There's how we're operating where we might use AI to optimize our resources or security holes or things like that. And then there's how we build where we're putting AI into our software development lifecycle process. So, this is really about what we're building, not how we're building or how we're operating.
Savannah Peterson
>> Staying there for a second, Bobby.>> Okay.
Savannah Peterson
>> Where are we with the state of AI? Everyone's obviously scrambling for it. I'm sure you're both talking about it all day long. I know you have a really good analogy for this.
Savannah Peterson
>> So I have to go back to kind of the way that I think about this. And so my statement and then a metaphor. So the statement is AI's not the thing, AI's the thing that makes the thing better. So if we were talking about food. AI's not the dish, it's the sauce or the spice that makes the dish better. So back to what Brandon talked about with training and inference. If you're training, you're someone who's creating the sauce, if you will. But most of us are not going to be creating sauces, we're going to be using sauces, which is closer to inference to inject that into some dishes that hopefully are going to taste good for the people eating at our dinner table.
Savannah Peterson
>> Yum.
Rob Strechay
>> Staying with that analogy, and you can both play off of this I think, is that there's a lot of sauces out there and I think Kubernetes is having its moment and GKE and where you start to look at it because there's so many tools that you can bring into a Kubernetes environment, but there's a lot of tools. And I think this goes back to your which sauce do I pick? What menu? Who am I serving? Do I have a side of fries with that kind of thing? Help people break that down.
Savannah Peterson
>> It's a good comment too. I think it comes down to what is the safe choice that you ultimately need to make? And I'm admittedly a little bit biased since I've been in the Kubernetes ecosystem for a while, but Kubernetes is the safest choice when we think about open frameworks for large language models or for AI models or for MLOps or really anything you're doing with AI because you have so much choice and flexibility around the ecosystem. And so that's I think the exciting part. You're right, there are a lot of choices and how we rationalize those choices sort of depends case by case, but we're starting to see some consolidation around common patterns and frameworks within the ecosystem. And it's something that we as Google have been working on quite a bit, helping not only on the Kubernetes ecosystem but also contributing to open source frameworks that just make running these models, training the models, doing inference of the models better, more efficient on top of Kubernetes. And so I know it didn't exactly answer your question, at least not using the food analogy. Analogies are not my thing with food, but I'm a little hungry now.
Rob Strechay
>> Sorry about that.
Savannah Peterson
>> Rob has that effect on -
Savannah Peterson
>> I know, I know.>> But you all can see why I'm so glad that Brandon and I get to work together because then I can go ask smart people like him, "Help me unpack some of these terms."
Savannah Peterson
>> Yeah. And I want to hang out here for a second because I think, and I say this with love to our entire industry, I think we've got a couple different communities at different levels of maturity in terms of both the adoption of Kubernetes and AI. I think AI actually very much accelerating the adoption of Kubernetes, but people still getting just started on their container at scale journey. And as they're thinking about AI at scale, it's particularly challenging. So what's your advice, if I'm a strategist, maybe not as deep on the technical side or someone in the C-suite even, evaluating what we're going to do in 2025, looking forward, what's your advice for getting started and not getting overwhelmed? Because there's a lot.>> I think there's a lot of lessons we can actually learn from the non-AI workloads in Kubernetes over the last seven years. We can really look at... Maybe taking a quick step back. So if we think about what worked really well for Kubernetes, we started with this world where it was only stateless applications and there was a very small sub-domain of applications that we can run in Kubernetes, but things matured and now we can run databases and all kinds of stateful kinds of applications. And so I think about the same kind of patterns that we're starting to see now. Things that were really hard even 12 months ago in Kubernetes are becoming increasingly simpler and easier. So we're starting to reduce the friction as a community and as cloud providers is, what we do in GKE is really just helping customers to onboard faster, to reduce the operational friction and really optimize for whatever level of scale you're at. If you're doing something really small, we can help you scale there. Or if you're some of the largest customers in the world that are doing large-scale training and inference, we can help you there as well. But for a lot of customers, back to your question, where do you get started? I think you can use similar patterns there. Start with an API. There's APIs that are available and you can use them really quickly. Use an off-the-shelf Gemini model with an API, get started, iterate. But if you want portability and you want flexibility, that's where you can start to lean on that Kubernetes community.
Savannah Peterson
>> I think that's a great point. Bobby, I see you over there.>> Yeah, I'd love to jump in there. So there's so much that's happening right now and I just want to say for the listening audience, I work at Google and I can't keep up with all this stuff. So if you're at home feeling like this a lot, it is a lot, right? So it's not just you, I'm not gaslighting you. It is a lot. But I've got a mental framework that I don't want to share with the audience, it's based around food, unfortunately.
Savannah Peterson
>> It's not unfortunately. We love food.>> I do talk about stuff besides food, but this is kind of my mental framework to organize a lot of the technology that's coming at us. I call this farm to table. So there are four buckets, there are four questions I think we need to ask ourselves in terms of whether this is something I need to dive into right now. It's what's in the ground growing? What's in the fridge, resting? What's on the stove cooking? And what's on the table, cooling? The things that are in the ground growing need resources, sunlight, water, it needs a lot of work. It is got potential, but it's not what it's going to be yet. Things in the refrigerator are ingredients, they're not recipes. They need assembly, they need instructions for how to put it together. Things on the stove are not ready. You can smell it, you can see what it's going to be, but it needs time and labor. And then the things that are on the table just need an appetite. They're ready to go now. So I put things into those buckets. And so you also have to think about your audience, right? Because you wouldn't give a potato to a toddler. They want a tater tot not a potato in the ground. So we've got to think about the maturity of the product or the feature and the audience to make sure we're right-sizing those.
Savannah Peterson
>> That analogy... How long do you spend thinking about food analogies? Do they just come to you naturally?
Rob Strechay
>> Usually when I'm working out.
Savannah Peterson
>> I love that. When you're, you're training the analogy part of your brain as well.
Rob Strechay
>> Exactly. Yep. Natural intelligence.
Savannah Peterson
>> Yes.
Savannah Peterson
>> Couldn't resist the opportunity there. All right, I got two more questions for you. When we're looking towards 2025, I mean a lot has changed in the last year, the last two years in particular. We don't even need to recap it. We're living in a very wild time as technologists. What are some of the trends we're going to be seeing in 2025? You're right there on the precipice.>> Yeah. I mean, one of the things we're going to start to see a lot more is distributed computing and the role of distributed computing for AI/ML. So distributed, taking a model and distributing it across, whether it be multiple nodes or multiple regions. We now have user base around the globe. So distributing models in a more sort of efficient way is going to become I think much more of a trend. It used to be something that, well really up to this point, it's really the more advanced organizations that are doing that kind of work. But now as frameworks are starting to mature, it's easier and easier to do distributed training and distributed inference across Kubernetes clusters or nodes or whatever it happens to be.
Savannah Peterson
>> It's going to be easy and distributed.
Savannah Peterson
>> Easy and distributed, and you can bring it to your users more easily. For sure.
Savannah Peterson
>> I love that. Bobby?
Savannah Peterson
>> I think there's going to be a level of maturity in enterprises about how they vet models versus how they roll them out. And so there's so many models or sauces on the shelf now that you're going to have, in my opinion, many sauces, few people, so kind of a tiger team evaluating what are the different things we want to have on the menu. So many models, few people, will roll out to fewer models, many people. So how do we vet it and then how do we roll it out and scale it out? And I think people are going to stop asking for sauces by brand name. They're going to say, "I want a smoky sauce or sweet sauce or this type of model or that type of model." And so we'll be abstracted away somewhat by the model we're calling and it'll be about the function it delivers to the end user.>> And to build on that a little bit more, I think this pattern of model as a service->> Yes, yes..... >> I think is going to become much more of a thing.
Rob Strechay
>> Huge, huge.>> Especially within enterprise organizations. I think we've started to see this recognition that there's amazing frameworks and models out there that I can now start to consume. So as an enterprise, I now need to deliver that as a service to my business. And I need to think about what is the variety of model based on the tasks that I want to perform. But I also now need to think about how do I iterate through those models? Do I have a new version? How do I get that out to my users as quickly as possible? And this is where again, I come back to the cloud-native ecosystem. We've solved that problem pretty well, I think for modern software. And I think we can bring a lot of those learnings back to AI/ML and deliver an amazing models of service platform to our users.
Savannah Peterson
>> I think that's a great point. I'm glad you brought that up. Okay, I am going to ask one final question here for this segment. My whole thing at theCUBE is putting the human in tech and we talk about a lot of really cool features and solutions and tools, but I'm going to ask you both. And I asked you a question about your family at the dinner table last time. This is a slightly different version of that, Bobby. What do you hope personally, could be for your family, for your life, for your hobbies, anything. What do both of you hope that are AI revolution that we're amidst, solves or makes better for the world? Bobby, I'm going to you first. King of the analogies.>> I want AI to feel like it's something we can all participate in. Maybe in different stages, right? So I'll give a non-food analogy for a second. All of us can't sing. Some of us are working a soundboard. Some of us are in the audience listening to people that can sing or buying the records of the people that can sing. Some of us are making microphones. We're all participating in that music supply chain though. And I want everyone to feel like I've got a role to play in this AI journey because this is going to be something that touches all of mankind. Let me just find my part and play that role. I'm not mad that I'm not on stage singing. I might be the band or the AV person, but I've got a role to play. I want everyone to feel like they can play a part.
Savannah Peterson
>> You and I both care about inclusion. I know it's a passion and everyone should be included.>> Yes.>> We talk about productivity as such, like a mechanical thing. But I think about productivity as such a human thing. How do I remove toil in my life?
Savannah Peterson
>> Yes.>> And that is something I really am an incredibly optimistic about the power of AI. Because there's some things, let's just be honest, we just don't like doing. Who likes it, and I struggle with this personally, a blank sheet of paper? You're starting to write a paper or an essay or an email and you have a completely blank sheet of paper. Everyone knows how difficult that is. It's particularly difficult for me, but that's the kind of toil we can start to remove or reduce from our lives. So in that respect, I love identifying the things that if I could just make that a little bit easier, I could make these other qualities of what I bring to society that much stronger, that much easier. So I'm super optimistic about sort of leveling the playing field and helping to smooth out the toil and human lives.
Savannah Peterson
>> I love that. Smooth out the toil. That is great. As someone writing a book right now, the amount of times I sit there staring at the blank page and the cursor just blink, blink, blink ->> Exactly. Exactly.
Savannah Peterson
>> I feel that. And I think that I'm now, as you said that I'm thinking, "Oh, how could I use AI to get me rolling a little bit to get out of that, kind of that first step's always the hardest, right? Of anything, as we all know.>> That's right. Yeah.
Savannah Peterson
>> So, wow, this is so cool. I'm so excited to continue this conversation in 2025.>> Likewise.
Savannah Peterson
>> And continue to build on the education. Bobby, thank you for bringing Brandon to us. This is outstanding. And Brandon, you really didn't disappoint. You lived up to the hype, man.
Savannah Peterson
>> Thank you so much.
Savannah Peterson
>> You're so good, we're going to have to keep doing it.>> It was a pleasure being here.
Savannah Peterson
>> Yes, yes it was. And Rob, thank you for hanging out.
Rob Strechay
>> I am glad to be here.
Savannah Peterson
>> I know. We just get to sit and learn from the smartest people on earth.
Rob Strechay
>> I think this is going to be wonderful and I guess we're going to be able to help a lot of people understand this stuff, which is great.
Savannah Peterson
>> I think so. And we're all stronger together when we collaborate and teach and educate. And I hope that for all of you, wherever you might be watching, you feel comfortable reaching out to folks like Bobby and Brandon, or Rob and I, to ask your questions about AI, to talk about your Kubernetes journey or container journey, wherever you might be. To Bobby's earlier point, we're not gaslighting. We really all are here to help. These guys in particular are super helpful. So there's no stupid questions and no one has it all figured out yet. Even the smartest people at the helm of this magical ship that we're all riding on. Thank you for tuning into this fantastically special educational segment with Google. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.
