Chris Wolf, Global Head of AI at Broadcom, and John Furrier discuss Private AI and its cost-effective, customer-centric approach. Private AI allows control and privacy over data, addressing concerns around security and enabling seamless integration of AI services. The focus on inference workloads and generative AI appeals to customers looking to maximize GPU infrastructure. The ecosystem is evolving with a focus on domain-specific models and AI governance. Customers seek deep insights and intelligent infrastructure placement. VMware's Cloud Foundation positions itself as a platform for all applications, emphasizing control and flexibility. The enterprise adoption of AI is gaining momentum, with VMware driving the development of advanced AI services. A focused approach starting with trusted partners is recommended for implementing Private AI. Quick wins in addressing specific data issues are key, with the ability to run an AI application in two weeks highlighted. Wolf emphasizes the importance of reliable infrastructure before scaling AI services.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value. If you don’t think you received an email check your
spam folder.
Sign in to VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value.
Thanks for confirming your account. Now you can access VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value with this email address.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value. If you don’t think you received an email check your
spam folder.
Sign in to VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value
Please sign in with LinkedIn to continue to VMware Cloud Foundation Transformed: A New Era of Private Cloud Innovation and Value. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
VCF Al Tech Solutions & Strategic Offerings
Chris Wolf, global head of AI and advanced services, VMware Cloud Foundation Division, at Broadcom, talks about the transformation of VMware Cloud Foundation. He also explores the future of VCF, the impact of AI and the innovative strides Broadcom is making in the industry. Wolf shares the success of the company’s private AI initiative, its collaboration with Nvidia and the strategic investments driving Broadcom’s AI infrastructure. Discover how privacy, control and cost-efficiency are at the core of their strategy, and learn about the agility and flexibility that Broadcom's platform approach offers in an evolving ecosystem.
>> Hello. Welcome to this special CUBE presentation here in Palo Alto. I'm John Furrier, host of theCUBE with a special VMware Cloud Foundations transformed presentation. We have a series of guests come in. We're going to unpack where VCF is going, where Broadcom is taking the next level generative AI position. I'm going to bring that to you. Chris Wolf is here, who's the Global Head of AI and Advanced Services. Good to see you.
Chris Wolf
>> Likewise, John. Good to see you.>> So Global Head of AI and Advanced Services, that's a big title. So that means you're going to unpack the entire Broadcom VMware program out to the customers. So last year this time you launched Private AI here on theCUBE.
Chris Wolf
>> We did.>> You guys were the first. And at that time it was clear to us in our power law that we started seeing the smaller models and especially models kicking in. That played out this year. Congratulations to you guys for building the category. Private AI is the hottest topic. The hardware manufacturer like Dell, HP, they're building systems for this, is a big part of the industry, so congratulations. How does it feel?
Chris Wolf
>> It feels great. I mean, it really comes down to listening to your customers and understanding where their needs are, and then also being smart about finding what your market fit is so that all those things came together. It was probably a year effort for us to get to the concept of Private AI and fully have it vetted and baked. But here we are, we're seeing the market react really strongly.>> I remember the early conversation, you guys were very clear on this, had the vision, so great. You saw the North Star, but you saw the reality of where the market was. The consumer side was booming, the proprietary models, multimodal, foundation models now called pioneer models, I guess, but the enterprises weren't leaning into it. You guys knew that, you saw it. And since then you've made a lot of investments. Can you take us through what happened, what you invested, and what was the bet and where are we right now?
Chris Wolf
>> Yeah. So what we were betting on was that privacy and control of data really mattered to customers. The ability to bring the AI model to wherever their data resided would matter to customers. And even last summer, we had some real insights in terms of operating AI services in an enterprise data center because we were doing that ourselves. And we were seeing that some of our internal services were roughly one third the cost of comparable public cloud services. So we knew we were onto something that it wasn't just about getting these benefits, but you were getting a lower cost as well. So that was what really grounded our strategy. We partnered with Nvidia. We announced Private AI foundation with Nvidia at Explore last year, and now in just last month we went generally available. We're seeing significant customer traction, really getting behind what we're doing. You get the best of Nvidia, the best of VMware and Broadcom technologies together in a unified platform.>> Yeah. I saw in the news recently the founder who left OpenAI starting a new company and they're going to focus on super intelligence at a Supercloud event. When you were here, you were seeing that direction where LLMs and large language model and foundation in the enterprise were different. What happened in your mind that tipped the scales that said, "Okay. This is happening. We're going to double down on this?"
Chris Wolf
>> Yeah. We saw the trend for sure, and we saw that we had a very specific space that would really fill in a key gap. And we also saw what our approach was going to be very complimentary, because at the time, most of the ecosystem players, they were going full stack like, "Bet on our software stack all the way down through our hardware, make a bet with us." And we were thinking, "No way," the space is moving so fast that you can't just bet on a single platform or a single solution. So we said we need to take a platform approach, bet on an AI infrastructure platform that's going to give you some optionality so that as something changes, you can quickly adopt it. And to give you an example, one of our top internal AI services, we've changed the foundation model three times in the last nine months just because we keep getting better results. And the platform approach is what gives you that agility, and that's a key reason why folks are betting on us.>> It's interesting. The big conversation, a lot of the conferences here said the general AI is changing the stack for integrating that stack for customers. There's obviously compute and horsepower and all that jazz, but you've got the data stack and then you also got this new model stack emerging. I mean, generically speaking, the models are now decoupled from compute and data. As they start to bring in neural networks, you start seeing different infrastructure targeted for those workloads that will be running on neural networks. What's your take on that and how did that interface with Private AI and your efforts?
Chris Wolf
>> Yeah, yeah, sure. So we've taken a pretty broad ecosystem approach. One thing that really works for us is that we're not opinionated in those higher level models or AI services. So it's really allowed the AI community to want to partner with us because we're not a direct competitor. Whether you're an AI ISV, you're producing models, et cetera. We want to be the runtime, and that's where we've been really successful. So if we step back and tease that out, you get the Nvidia inference microservices, you have all of the automation that sits below that, and that's been one of the struggles for AI. It's easy to say, "Run an AI app." Well, how do you automate the deployment? How do you automate beyond day two, right? How do you secure it? How do you add governance and controls? These are the real challenges that folks face. And then the other issue is, I can't just go out and buy all this AI infrastructure. I have to figure out how do I effectively use it. That's been another key area that's driven our early adopters is, "If I just use pure physical capacity, oftentimes I might claim a GPU, but I only need a fraction of the GPU." So being able to virtualize and share GPUs, which are hard to come by is really important.>> Well, I think the workloads have to all include GPU management, because now the infrastructure that you guys have been doing for many, many years has been an IT backbone and fabric, but IT is changing. We hear Jensen Huang say this, and almost all his keynotes is that, "If you're in IT today, you got to lean into generative AI because it's a net new category."
Chris Wolf
>> Yup.>> So the category is new, so it's generous, so it's going to change the face of it. So they got to lean into automation, rethink those stacks. So given all that, which I'm sure you agree because we're on the same page on that one, but Private AI is key to that. What's the status of the Private AI right now for you guys? Is it shipping? Is GA? You're announcing GA, and what's the relationship with NVIDIA?
Chris Wolf
>> Yeah. So we've launched our GA product last month. We've seen tremendous customer traction. In terms of a new product launch this is the most exciting product launch in terms of just pure traction I've ever seen in my career. So that's been really exciting for us just to see the customer demand. And now what we're seeing is a couple different classes of customers. So the early adopters of AI that have been running services in the public cloud, they're coming to us for a few different reasons. They might've had an on-prem solution that's been on bare metal, and you have a lot of challenges in terms of, how do I manage resource scheduling? How do I manage availability? How do I intelligently connect an application to the right amount of infrastructure capacity? How do I reclaim capacity? These are really difficult problems to solve. And VMware has had technology such as DRS, our distributed resource scheduler for decades, right? So being able to expand on the core technology and the foundation we have has allowed us to add all of this value to AI infrastructure that's attracted customers. The other thing that's really brought them in is cost. So we're seeing customers tell us, "To run the AI service on-prem could be sometimes three to 5x less expensive than running that comparable service in the public cloud." And it's not for everything. I'll use the cloud services for training models where I need that bursty capacity when I want to do inference workloads and just apply the AI model to an application, the cost savings is significant. So we have one class of customers, which is that early AI adopter, they've had experience running GenAI for maybe two years now, and they're like, "Cost is my biggest issue. Help me with that. Huge." Or they have their own on-prem stack and they have the same issues. They struggle with resource management, all these things, and we can give them all of that. So there's tremendous value there. And then you also have this other class, which is they want to get their AI win. Every organization has a document summation, document type search use case, get my support text faster access to the meaningful data they need to be able to close tickets quicker. Everybody has that use case. There's even a college we're working with and their use cases is, "Our campus is huge. And when our students schedule classes, it's very difficult for them to figure out, well, this class is available, but can I actually get to it on time?" So it's a cool language model use case to do these types of things as well. So we have that gambit from financial services to public sector organizations where they have proprietary data sources, our own proprietary that a contractor built for them decades ago, right? So bringing the model to the data is just a huge value to them. And then the other side too is on the partner side, ISVs love us because we're non-competitive, right? We can easily approach a customer together and help to solve problems. So all of that is driving the momentum. So we have lots of customers use cases around document summation, code development, contact centers or the types of things we're seeing. But again, it's the value of the privacy and control and the total cost of ownership being lower than other services, that's been really attractive.>> You talk about the customers and you get the ecosystem a second, and that's a big part of this. But on Private AI, you mentioned a bunch of things there. There's different orientations to how people are leaning in, but the number one thing that we hear is costs. How do I scope the costs? What am I dealing with? Because there's so much data in every use case. I mean, whether you're a large campus or a school or hospital or healthcare, there's data everywhere. And that's the key thread here with Private AI that we're seeing. What's your reaction to that? Because data seems to be the main criteria, and privacy and security, obviously that's key for all enterprises, but their data now is their crown jewel. So Private AI is aligning with the data and also their challenges with say, security and privacy. How are you guys addressing that with Private AI? How's your architecture meet them where they are on scoping understanding scale, and then what the use cases are going to be developed?
Chris Wolf
>> It's really key. So first, I don't have to take data out of one of my own internal data sources that I control and then move it to somebody else's perhaps proprietary data source to gain the benefit. And there's also just genuine concerns organizations have like, "Is this data now going to be help it to influence a model that might benefit my competitors?" There's other challenges such as, "If I'm even just passing code to an external service, my customers that I sell to might have challenges even trusting my code at that point." These are just genuine tactical issues that organizations are faced with. So they even want stacks even for code development where they have full control, they have full isolation of those as well. And these are things that they can prove to an auditor, but bringing the model adjacent to the data sources creates a lot of agility and flexibility for the organizations. And then they can quickly stand up using retrieval-augmented generation architectures. It's very easy to collect the data that you need to feed to a foundation model and get good results. And we're even seeing now the models are getting so good. One of the ones we use is Mixtral 8x7B internally, and we're finding with very little fine-tuning, you're getting accuracies with RAG in the 70s. And that's incredible. And what it means is you don't have to have this massive team of data scientists to get these benefits of AI. It's becoming more and more consumable for the enterprise even if they don't have that experience.>> Yeah. It's interesting. We just did that segment on theCUBE last week around how the neural network infrastructure of vectors and tokens have changed, retrieval, all kinds of how data's being used. And the enterprise AI story was pretty much like a year ago, "Well, the enterprise doesn't..." People kind of body swerving that. Now with Private AI, people see immediate benefit, low-hanging fruit, retrieval-augmentation generation or RAG. You've got new ways to bring in say PDFs or images. I mean, it's off the chart. There's use cases everywhere. So the point is there's no shortage of where to attack and test. And so everyone's leaning into that and they're getting into it for that reason. The question then comes in and every company we talk to asks the same question, what does AI mean to me? No matter what the entry point is development, how do I do development better? How do I make my networks run better? How do I run my workloads? So the question I want to ask you is, how does Private AI impact your customers that Broadcom and VMware? Because their infrastructure, they're managing workloads but also have a developer front end. So is it everything or where do you lock in? What's the lock-in position?
Chris Wolf
>> Yeah. Yeah. For us, I wouldn't use the term lock-in.>> Oh, I don't know what you mean by lock-in . What do you settle?
Chris Wolf
>> We can expose our infrastructure through a Kubernetes API or through a Ray API and lots of other open source constraints. And that's important because we want our customers to run on our stack because we're the best platform for them, not because we're doing anything unnatural. But we have so much experience driving platform services. VMware has always been the platform you can trust to run a whole variety of applications, and we're expanding that to artificial intelligence applications and doing really slick things in terms of how you schedule and manage access to GPUs and how you automate all of that. So that's been really essential for the customers that have been onboarding with us is that they can get very quick time to value. So I think those are some of the areas that have influenced at least the customer decision to partner with us. But again, it's control of the data. It's privacy and it's->> It's innovation everywhere in the workload, workload management, enabling developers. You mentioned Kubernetes, much as lock-in by the way, I didn't mean lock-in like it's a lock-in spec.
Chris Wolf
>> Yeah. Yeah. Yeah.>> Where the focus is? Where's the beach core beachhead for Broadcom, VMware with your customer? Because there's extensibility with Private AI, and that's basically what you're getting at.
Chris Wolf
>> Yeah. Yeah. Yeah. So it's the extensibility, it's the flexibility to run all of these different services to very quickly pivot is one, but then on top of that, it's inference. Because if you look at some of the stats out there, the focus on AI often is on the training workloads. And that's 4% of the actual compute cycles being driven to AI. The rest is runtime and it's inference. And that's where we're seeing customers. I was talking to one of the largest global banks yesterday. They have a separate DGX cluster they're using for training, fine, but their inference decision is to run inference on their VMware platform. Because we're going to give them the best economics, the best total cost of ownership, the best reliability, and the way that we can help them to manage and maximize their GPU infrastructure is something they just can't do with anybody else.>> And I think that's key, because that's the scale needed. The other piece that comes up I want to get your thoughts on is that you mentioned runtime earlier. Generative AI is generative. It's not the old school way of pre-program, this is what Nvidia leaning into with their narrative is saying, and he's not wrong. I mean, it's a new category, it changes IT because the data changes, right? So that's one. How do you see that playing out from Private AI? Is that the avenue you're getting the most traction in is the fact that people are saying, "Hey, I could have inference and reinforced learning and at runtime?" Is that a piece of that? How would you react to that?
Chris Wolf
>> Yeah. I can have it adjacent to my data. I can get quicker time to value because I don't have to plan any types of data migration projects. That's really key. So you're definitely on the right track there in terms of where the value is with Private AI. It's that model adjacency to the data. And again, it's not just about what you can even do from a pure technology sense, right? It's, what is your legal team's comfortable with? What is acceptable from a regulatory compliance perspective? All of these things really matter and drive that decision. And then you start to look at just even doing things like segmenting models based on access groups or segmenting your data indexes based on access groups, which is a very effective way to implement the right levels of privacy controls as well.>> Chris, I want to get your thoughts. You've been in many industry cycles. You've seen a lot. This is a big one. And you just talked about the customer impact, which by the way is awesome. The ecosystem's changing too. You're seeing the platform shift happen, but also the ecosystem shifts with it. How is the ecosystem changing? How are you guys rolling with that? Can you point out some successes and some changes or what's changed in the ecosystem for the better? What's the key ecosystem impact this? Because there's more APIs everywhere connecting people. The new data models are out there. You got model stacks. What's the ecosystem impact? How are you guys in the middle of it?
Chris Wolf
>> It's been exciting. I think there's a couple flavors of it. You hit on one earlier, which is domain specific models and getting down to models like VMware, we're a software development company, right? I don't need an AI model that is an expert in healthcare, right? I don't need that information, that just adds to model bloat. So I can get to focus models on what I need and be very successful with that. So you're seeing that traction starting to really influence some of these applications. So that's one. In our case, we have a very clear demarcation point between the value we're providing. We're stopping at those AI infrastructure services. So if you're an ISV doing app development, you're doing AI models or open source or whatever we're, the company to partner with because we can approach the customer together where we have joint value. Same thing with our SI partners. We're seeing system integrators really gone onto us because there's a very clear place where they're adding value and can support the customer through their entire application lifecycle. So that's driving a lot of this ecosystem around what we're doing. And to me that's exciting. We are really a great enabler for these ISVs coming up because in some cases it can be more difficult for them to work with a cloud provider because the cloud provider has their own AI stack that they're also trying to sell. We're really work with us because, again, we're neutral. We can work together and solve customer problems together.>> You're attracting ISVs because they can get access to the data and enhance with GenAI, the new development environment will be leveraging that. Does that be true?
Chris Wolf
>> Yeah. Yeah, for sure.>> Which trends are you tracking right now as you look forward? You got VMware Explore, I'm sure you don't want to reveal all the surprises that you got planned. I'm sure you got a few surprises. Can you give us a little teaser on that? And also what are you tracking right now that's going to be extending, accelerating the GenAI and Private AI and foundation?
Chris Wolf
>> Yeah. I think the maturity of services is really important. So there's a lot of velocity in open source communities around data collection as an example. So the hard part with retrieval-augmented generation had been actually indexing and collecting data to be able to feed to a model. And years ago when we first started down this path internally, we had to build our own data collectors because they just didn't exist. And today, there's great projects out there, whether it's like LangChain, Lambda index, there's others that can really help to make that far more turnkey for organizations. So that velocity there to help with the attainment of AI, I think is important. The other areas are really, for us, it gets into our core. It's providing AI model governance. It's providing really deep insights into how to intelligently place an AI application on the right infrastructure the first time and taking the guesswork out, right? This is the stuff in terms of really advanced workload management. We've been great at this stuff, right? So this is what's really going to help. So it's driving those efficiencies, lowering the cost. And the other thing we haven't even mentioned that's important here is carbon footprint. People get concerned about the power footprints to their racks for AI. And when you start to get into these more smaller domain specific models, guess what? You can start to run AI applications without having to completely change that power dynamic of your data centers, which can also make a huge difference. Again, it's going to depend on the application. Some applications are going to need dozens of GPUs, don't get me wrong, but there's others that you might get away with four or eight GPUs and it's going to be fine for the use case.>> It's interesting watching VMware over the years, you've been basically running IT for operations and companies operations are changing radically too. As head of AI, you're looking at that 20 mile stair down the road. What's your priorities now? You're going to build in these new advanced services. What are your priorities?
Chris Wolf
>> Yeah. If I say, what is the North Star? We want people to rethink about VMware now, where VMware Cloud Foundation is your platform for all applications. You're going to have the best control, you're going to have the best privacy, you're going to have the most flexibility running modern applications on our stack. So you're seeing a lot of investment in, and Paul talked to you about this as well in what we're doing around Kubernetes, making that simple, what we're doing with AI and making that super consumable. And then you're seeing even with technology such as our data services manager, giving you database as a service just native to the platform. So to run an entire modern app stack with more flexibility and choice is what you're going to see from us going forward, and then even much more so on the AI side. So just really slick things. So if you're not signed up for Explore, you better get there because we're going to show some really cool stuff in the next couple of months. I'd love to talk more about it here, but we have PR sitting next to me and I don't want to get in trouble.>> Well, exactly. And one of the beautiful things when we talked about this off camera before, a lot of people like to compare this shift to the internet and where consumer led and enterprise followed right after and we said, and we discussed it, and it's happening. I want to get your thoughts. The enterprise is happening. It's happening faster. We're in an accelerated cycle where consumer and enterprise are almost neck and neck in terms of adoption. A year ago, people were saying, "Well, AI is chatbots. It's customer support. Okay." It's so much more than that. Share your thoughts on what's happened that made that, why is it accelerating so fast? Is it more data? Is it more mature infrastructure? Is it the fact that people are at large scale? Why is the enterprise AI accelerating faster than most people thought?
Chris Wolf
>> Yeah. I think it's because of the efficiencies that you can gain from AI, that it's making people more productive. It's letting them find the needle in the haystack faster, finding the information they need faster, at least in a lot of these generative AI use cases. And I think what's important about the VMware approach is we're very deliberate in making sure that the human is in the information loop. So all of our AI services, we call it intelligent assist. So AI is there to assist the human. And when the humans start to look at this as a tool that's going to help them to be more productive in their job and help them to find things faster, create content or whatever, that's where it really makes a difference. I mean, one of the data scientists, and he's AI engineer on my team, he just had this experience with this assistant technology that we've been running internally, and he was able to write an entire app in a couple of weeks, and he said that would've taken him probably six months without the code assist technologies.>> VMware Cloud Foundation obviously has transformed. We're in a new era of innovation. What's the one thing that you've learned over the past year that you can share to folks watching, customers or potential customers, they're going to implement Private AI. What's the best practice? What's the best approach? How should they lean into Private AI?
Chris Wolf
>> Yeah. I think that the best approach is to really narrow your focus and narrow your scope. You have to get very specific. Everybody wants to have an AI win. So to me, it starts with focusing on the shore bets, which is why we have Private AI Foundation with Nvidia, right? If you go with trusted partners in your journey, you're going to get good results. For a lot of companies, this is a board level imperative, right? The board wants to see the CEO show an AI win. So starting with us, starting with a trusted platform is one. Taking a platform approach is really important, because you don't want to just bet on that one app stack because you're going to have buyer's remorse next month if you do that, right? So that's the other area is to take the platform approach from us, that's worked out. And then what we're focused on with customers is getting them that quick win. So oftentimes, what is the data issue that you're having, right? What does it take people the longest to gain information to? And this is banks, this is public sector organizations, right? It runs the gambit in terms of healthcare, et cetera. In terms of, how can I help? So you have to narrow your scope, think about one problem, get that AI win, and then understand how you're operating that AI application, then start to move on to other things, right? So that's been our focus this year is getting customers to that first AI win that they can iterate on and then start to look at additional .>> And by the way, that's an easy win because they have data and the data gets them the ability to do it on a private environment. They get the win, they can understand how to operationalize it, scale it, and then everything falls into place.
Chris Wolf
>> Yeah. They can go from zero to a running AI application in two weeks. If you have the right scope, you can get there very quickly and see immediate value.>> And the beautiful thing for you guys, and again, props for being right on the right wave here early, is that all the hardware and all the systems being rolled out now are right into what you're doing.
Chris Wolf
>> 100%.>> All your customers are building faster, smaller, clustered systems, large scale, and then mentioned one of the leaders, others, all the OEM partners. So ecosystem is changing.
Chris Wolf
>> It is. It is. It's moving quick.>> Good. Good.
Chris Wolf
>> Yeah. I think that's the difference. AI has really reset the velocity expectation.>> Well, the value creation is going to be amazing. The app tsunami coming, it's going to be Cambrian explosion and applications, but they got to get their ops lined up, infrastructure teed up.
Chris Wolf
>> That's it. Get your house in order, get a reliable IAS first underneath your AI services, and then you can grow from there.>> Chris Wolf, you heard it here. Head of AI for Broadcom here on theCUBE, breaking down the private AI wave that's here. It's certainly tracking. It's all lining up perfectly to fund and fuel the next generation of value. I'm John Furrier with theCUBE. Thanks for watching.
>> Hello. Welcome to this special CUBE presentation here in Palo Alto. I'm John Furrier, host of theCUBE with a special VMware Cloud Foundations transformed presentation. We have a series of guests come in. We're going to unpack where VCF is going, where Broadcom is taking the next level generative AI position. I'm going to bring that to you. Chris Wolf is here, who's the Global Head of AI and Advanced Services. Good to see you.
Chris Wolf
>> Likewise, John. Good to see you.>> So Global Head of AI and Advanced Services, that's a big title. So that means you're going to unpack the entire Broadcom VMware program out to the customers. So last year this time you launched Private AI here on theCUBE.
Chris Wolf
>> We did.>> You guys were the first. And at that time it was clear to us in our power law that we started seeing the smaller models and especially models kicking in. That played out this year. Congratulations to you guys for building the category. Private AI is the hottest topic. The hardware manufacturer like Dell, HP, they're building systems for this, is a big part of the industry, so congratulations. How does it feel?
Chris Wolf
>> It feels great. I mean, it really comes down to listening to your customers and understanding where their needs are, and then also being smart about finding what your market fit is so that all those things came together. It was probably a year effort for us to get to the concept of Private AI and fully have it vetted and baked. But here we are, we're seeing the market react really strongly.>> I remember the early conversation, you guys were very clear on this, had the vision, so great. You saw the North Star, but you saw the reality of where the market was. The consumer side was booming, the proprietary models, multimodal, foundation models now called pioneer models, I guess, but the enterprises weren't leaning into it. You guys knew that, you saw it. And since then you've made a lot of investments. Can you take us through what happened, what you invested, and what was the bet and where are we right now?
Chris Wolf
>> Yeah. So what we were betting on was that privacy and control of data really mattered to customers. The ability to bring the AI model to wherever their data resided would matter to customers. And even last summer, we had some real insights in terms of operating AI services in an enterprise data center because we were doing that ourselves. And we were seeing that some of our internal services were roughly one third the cost of comparable public cloud services. So we knew we were onto something that it wasn't just about getting these benefits, but you were getting a lower cost as well. So that was what really grounded our strategy. We partnered with Nvidia. We announced Private AI foundation with Nvidia at Explore last year, and now in just last month we went generally available. We're seeing significant customer traction, really getting behind what we're doing. You get the best of Nvidia, the best of VMware and Broadcom technologies together in a unified platform.>> Yeah. I saw in the news recently the founder who left OpenAI starting a new company and they're going to focus on super intelligence at a Supercloud event. When you were here, you were seeing that direction where LLMs and large language model and foundation in the enterprise were different. What happened in your mind that tipped the scales that said, "Okay. This is happening. We're going to double down on this?"
Chris Wolf
>> Yeah. We saw the trend for sure, and we saw that we had a very specific space that would really fill in a key gap. And we also saw what our approach was going to be very complimentary, because at the time, most of the ecosystem players, they were going full stack like, "Bet on our software stack all the way down through our hardware, make a bet with us." And we were thinking, "No way," the space is moving so fast that you can't just bet on a single platform or a single solution. So we said we need to take a platform approach, bet on an AI infrastructure platform that's going to give you some optionality so that as something changes, you can quickly adopt it. And to give you an example, one of our top internal AI services, we've changed the foundation model three times in the last nine months just because we keep getting better results. And the platform approach is what gives you that agility, and that's a key reason why folks are betting on us.>> It's interesting. The big conversation, a lot of the conferences here said the general AI is changing the stack for integrating that stack for customers. There's obviously compute and horsepower and all that jazz, but you've got the data stack and then you also got this new model stack emerging. I mean, generically speaking, the models are now decoupled from compute and data. As they start to bring in neural networks, you start seeing different infrastructure targeted for those workloads that will be running on neural networks. What's your take on that and how did that interface with Private AI and your efforts?
Chris Wolf
>> Yeah, yeah, sure. So we've taken a pretty broad ecosystem approach. One thing that really works for us is that we're not opinionated in those higher level models or AI services. So it's really allowed the AI community to want to partner with us because we're not a direct competitor. Whether you're an AI ISV, you're producing models, et cetera. We want to be the runtime, and that's where we've been really successful. So if we step back and tease that out, you get the Nvidia inference microservices, you have all of the automation that sits below that, and that's been one of the struggles for AI. It's easy to say, "Run an AI app." Well, how do you automate the deployment? How do you automate beyond day two, right? How do you secure it? How do you add governance and controls? These are the real challenges that folks face. And then the other issue is, I can't just go out and buy all this AI infrastructure. I have to figure out how do I effectively use it. That's been another key area that's driven our early adopters is, "If I just use pure physical capacity, oftentimes I might claim a GPU, but I only need a fraction of the GPU." So being able to virtualize and share GPUs, which are hard to come by is really important.>> Well, I think the workloads have to all include GPU management, because now the infrastructure that you guys have been doing for many, many years has been an IT backbone and fabric, but IT is changing. We hear Jensen Huang say this, and almost all his keynotes is that, "If you're in IT today, you got to lean into generative AI because it's a net new category."
Chris Wolf
>> Yup.>> So the category is new, so it's generous, so it's going to change the face of it. So they got to lean into automation, rethink those stacks. So given all that, which I'm sure you agree because we're on the same page on that one, but Private AI is key to that. What's the status of the Private AI right now for you guys? Is it shipping? Is GA? You're announcing GA, and what's the relationship with NVIDIA?
Chris Wolf
>> Yeah. So we've launched our GA product last month. We've seen tremendous customer traction. In terms of a new product launch this is the most exciting product launch in terms of just pure traction I've ever seen in my career. So that's been really exciting for us just to see the customer demand. And now what we're seeing is a couple different classes of customers. So the early adopters of AI that have been running services in the public cloud, they're coming to us for a few different reasons. They might've had an on-prem solution that's been on bare metal, and you have a lot of challenges in terms of, how do I manage resource scheduling? How do I manage availability? How do I intelligently connect an application to the right amount of infrastructure capacity? How do I reclaim capacity? These are really difficult problems to solve. And VMware has had technology such as DRS, our distributed resource scheduler for decades, right? So being able to expand on the core technology and the foundation we have has allowed us to add all of this value to AI infrastructure that's attracted customers. The other thing that's really brought them in is cost. So we're seeing customers tell us, "To run the AI service on-prem could be sometimes three to 5x less expensive than running that comparable service in the public cloud." And it's not for everything. I'll use the cloud services for training models where I need that bursty capacity when I want to do inference workloads and just apply the AI model to an application, the cost savings is significant. So we have one class of customers, which is that early AI adopter, they've had experience running GenAI for maybe two years now, and they're like, "Cost is my biggest issue. Help me with that. Huge." Or they have their own on-prem stack and they have the same issues. They struggle with resource management, all these things, and we can give them all of that. So there's tremendous value there. And then you also have this other class, which is they want to get their AI win. Every organization has a document summation, document type search use case, get my support text faster access to the meaningful data they need to be able to close tickets quicker. Everybody has that use case. There's even a college we're working with and their use cases is, "Our campus is huge. And when our students schedule classes, it's very difficult for them to figure out, well, this class is available, but can I actually get to it on time?" So it's a cool language model use case to do these types of things as well. So we have that gambit from financial services to public sector organizations where they have proprietary data sources, our own proprietary that a contractor built for them decades ago, right? So bringing the model to the data is just a huge value to them. And then the other side too is on the partner side, ISVs love us because we're non-competitive, right? We can easily approach a customer together and help to solve problems. So all of that is driving the momentum. So we have lots of customers use cases around document summation, code development, contact centers or the types of things we're seeing. But again, it's the value of the privacy and control and the total cost of ownership being lower than other services, that's been really attractive.>> You talk about the customers and you get the ecosystem a second, and that's a big part of this. But on Private AI, you mentioned a bunch of things there. There's different orientations to how people are leaning in, but the number one thing that we hear is costs. How do I scope the costs? What am I dealing with? Because there's so much data in every use case. I mean, whether you're a large campus or a school or hospital or healthcare, there's data everywhere. And that's the key thread here with Private AI that we're seeing. What's your reaction to that? Because data seems to be the main criteria, and privacy and security, obviously that's key for all enterprises, but their data now is their crown jewel. So Private AI is aligning with the data and also their challenges with say, security and privacy. How are you guys addressing that with Private AI? How's your architecture meet them where they are on scoping understanding scale, and then what the use cases are going to be developed?
Chris Wolf
>> It's really key. So first, I don't have to take data out of one of my own internal data sources that I control and then move it to somebody else's perhaps proprietary data source to gain the benefit. And there's also just genuine concerns organizations have like, "Is this data now going to be help it to influence a model that might benefit my competitors?" There's other challenges such as, "If I'm even just passing code to an external service, my customers that I sell to might have challenges even trusting my code at that point." These are just genuine tactical issues that organizations are faced with. So they even want stacks even for code development where they have full control, they have full isolation of those as well. And these are things that they can prove to an auditor, but bringing the model adjacent to the data sources creates a lot of agility and flexibility for the organizations. And then they can quickly stand up using retrieval-augmented generation architectures. It's very easy to collect the data that you need to feed to a foundation model and get good results. And we're even seeing now the models are getting so good. One of the ones we use is Mixtral 8x7B internally, and we're finding with very little fine-tuning, you're getting accuracies with RAG in the 70s. And that's incredible. And what it means is you don't have to have this massive team of data scientists to get these benefits of AI. It's becoming more and more consumable for the enterprise even if they don't have that experience.>> Yeah. It's interesting. We just did that segment on theCUBE last week around how the neural network infrastructure of vectors and tokens have changed, retrieval, all kinds of how data's being used. And the enterprise AI story was pretty much like a year ago, "Well, the enterprise doesn't..." People kind of body swerving that. Now with Private AI, people see immediate benefit, low-hanging fruit, retrieval-augmentation generation or RAG. You've got new ways to bring in say PDFs or images. I mean, it's off the chart. There's use cases everywhere. So the point is there's no shortage of where to attack and test. And so everyone's leaning into that and they're getting into it for that reason. The question then comes in and every company we talk to asks the same question, what does AI mean to me? No matter what the entry point is development, how do I do development better? How do I make my networks run better? How do I run my workloads? So the question I want to ask you is, how does Private AI impact your customers that Broadcom and VMware? Because their infrastructure, they're managing workloads but also have a developer front end. So is it everything or where do you lock in? What's the lock-in position?
Chris Wolf
>> Yeah. Yeah. For us, I wouldn't use the term lock-in.>> Oh, I don't know what you mean by lock-in . What do you settle?
Chris Wolf
>> We can expose our infrastructure through a Kubernetes API or through a Ray API and lots of other open source constraints. And that's important because we want our customers to run on our stack because we're the best platform for them, not because we're doing anything unnatural. But we have so much experience driving platform services. VMware has always been the platform you can trust to run a whole variety of applications, and we're expanding that to artificial intelligence applications and doing really slick things in terms of how you schedule and manage access to GPUs and how you automate all of that. So that's been really essential for the customers that have been onboarding with us is that they can get very quick time to value. So I think those are some of the areas that have influenced at least the customer decision to partner with us. But again, it's control of the data. It's privacy and it's->> It's innovation everywhere in the workload, workload management, enabling developers. You mentioned Kubernetes, much as lock-in by the way, I didn't mean lock-in like it's a lock-in spec.
Chris Wolf
>> Yeah. Yeah. Yeah.>> Where the focus is? Where's the beach core beachhead for Broadcom, VMware with your customer? Because there's extensibility with Private AI, and that's basically what you're getting at.
Chris Wolf
>> Yeah. Yeah. Yeah. So it's the extensibility, it's the flexibility to run all of these different services to very quickly pivot is one, but then on top of that, it's inference. Because if you look at some of the stats out there, the focus on AI often is on the training workloads. And that's 4% of the actual compute cycles being driven to AI. The rest is runtime and it's inference. And that's where we're seeing customers. I was talking to one of the largest global banks yesterday. They have a separate DGX cluster they're using for training, fine, but their inference decision is to run inference on their VMware platform. Because we're going to give them the best economics, the best total cost of ownership, the best reliability, and the way that we can help them to manage and maximize their GPU infrastructure is something they just can't do with anybody else.>> And I think that's key, because that's the scale needed. The other piece that comes up I want to get your thoughts on is that you mentioned runtime earlier. Generative AI is generative. It's not the old school way of pre-program, this is what Nvidia leaning into with their narrative is saying, and he's not wrong. I mean, it's a new category, it changes IT because the data changes, right? So that's one. How do you see that playing out from Private AI? Is that the avenue you're getting the most traction in is the fact that people are saying, "Hey, I could have inference and reinforced learning and at runtime?" Is that a piece of that? How would you react to that?
Chris Wolf
>> Yeah. I can have it adjacent to my data. I can get quicker time to value because I don't have to plan any types of data migration projects. That's really key. So you're definitely on the right track there in terms of where the value is with Private AI. It's that model adjacency to the data. And again, it's not just about what you can even do from a pure technology sense, right? It's, what is your legal team's comfortable with? What is acceptable from a regulatory compliance perspective? All of these things really matter and drive that decision. And then you start to look at just even doing things like segmenting models based on access groups or segmenting your data indexes based on access groups, which is a very effective way to implement the right levels of privacy controls as well.>> Chris, I want to get your thoughts. You've been in many industry cycles. You've seen a lot. This is a big one. And you just talked about the customer impact, which by the way is awesome. The ecosystem's changing too. You're seeing the platform shift happen, but also the ecosystem shifts with it. How is the ecosystem changing? How are you guys rolling with that? Can you point out some successes and some changes or what's changed in the ecosystem for the better? What's the key ecosystem impact this? Because there's more APIs everywhere connecting people. The new data models are out there. You got model stacks. What's the ecosystem impact? How are you guys in the middle of it?
Chris Wolf
>> It's been exciting. I think there's a couple flavors of it. You hit on one earlier, which is domain specific models and getting down to models like VMware, we're a software development company, right? I don't need an AI model that is an expert in healthcare, right? I don't need that information, that just adds to model bloat. So I can get to focus models on what I need and be very successful with that. So you're seeing that traction starting to really influence some of these applications. So that's one. In our case, we have a very clear demarcation point between the value we're providing. We're stopping at those AI infrastructure services. So if you're an ISV doing app development, you're doing AI models or open source or whatever we're, the company to partner with because we can approach the customer together where we have joint value. Same thing with our SI partners. We're seeing system integrators really gone onto us because there's a very clear place where they're adding value and can support the customer through their entire application lifecycle. So that's driving a lot of this ecosystem around what we're doing. And to me that's exciting. We are really a great enabler for these ISVs coming up because in some cases it can be more difficult for them to work with a cloud provider because the cloud provider has their own AI stack that they're also trying to sell. We're really work with us because, again, we're neutral. We can work together and solve customer problems together.>> You're attracting ISVs because they can get access to the data and enhance with GenAI, the new development environment will be leveraging that. Does that be true?
Chris Wolf
>> Yeah. Yeah, for sure.>> Which trends are you tracking right now as you look forward? You got VMware Explore, I'm sure you don't want to reveal all the surprises that you got planned. I'm sure you got a few surprises. Can you give us a little teaser on that? And also what are you tracking right now that's going to be extending, accelerating the GenAI and Private AI and foundation?
Chris Wolf
>> Yeah. I think the maturity of services is really important. So there's a lot of velocity in open source communities around data collection as an example. So the hard part with retrieval-augmented generation had been actually indexing and collecting data to be able to feed to a model. And years ago when we first started down this path internally, we had to build our own data collectors because they just didn't exist. And today, there's great projects out there, whether it's like LangChain, Lambda index, there's others that can really help to make that far more turnkey for organizations. So that velocity there to help with the attainment of AI, I think is important. The other areas are really, for us, it gets into our core. It's providing AI model governance. It's providing really deep insights into how to intelligently place an AI application on the right infrastructure the first time and taking the guesswork out, right? This is the stuff in terms of really advanced workload management. We've been great at this stuff, right? So this is what's really going to help. So it's driving those efficiencies, lowering the cost. And the other thing we haven't even mentioned that's important here is carbon footprint. People get concerned about the power footprints to their racks for AI. And when you start to get into these more smaller domain specific models, guess what? You can start to run AI applications without having to completely change that power dynamic of your data centers, which can also make a huge difference. Again, it's going to depend on the application. Some applications are going to need dozens of GPUs, don't get me wrong, but there's others that you might get away with four or eight GPUs and it's going to be fine for the use case.>> It's interesting watching VMware over the years, you've been basically running IT for operations and companies operations are changing radically too. As head of AI, you're looking at that 20 mile stair down the road. What's your priorities now? You're going to build in these new advanced services. What are your priorities?
Chris Wolf
>> Yeah. If I say, what is the North Star? We want people to rethink about VMware now, where VMware Cloud Foundation is your platform for all applications. You're going to have the best control, you're going to have the best privacy, you're going to have the most flexibility running modern applications on our stack. So you're seeing a lot of investment in, and Paul talked to you about this as well in what we're doing around Kubernetes, making that simple, what we're doing with AI and making that super consumable. And then you're seeing even with technology such as our data services manager, giving you database as a service just native to the platform. So to run an entire modern app stack with more flexibility and choice is what you're going to see from us going forward, and then even much more so on the AI side. So just really slick things. So if you're not signed up for Explore, you better get there because we're going to show some really cool stuff in the next couple of months. I'd love to talk more about it here, but we have PR sitting next to me and I don't want to get in trouble.>> Well, exactly. And one of the beautiful things when we talked about this off camera before, a lot of people like to compare this shift to the internet and where consumer led and enterprise followed right after and we said, and we discussed it, and it's happening. I want to get your thoughts. The enterprise is happening. It's happening faster. We're in an accelerated cycle where consumer and enterprise are almost neck and neck in terms of adoption. A year ago, people were saying, "Well, AI is chatbots. It's customer support. Okay." It's so much more than that. Share your thoughts on what's happened that made that, why is it accelerating so fast? Is it more data? Is it more mature infrastructure? Is it the fact that people are at large scale? Why is the enterprise AI accelerating faster than most people thought?
Chris Wolf
>> Yeah. I think it's because of the efficiencies that you can gain from AI, that it's making people more productive. It's letting them find the needle in the haystack faster, finding the information they need faster, at least in a lot of these generative AI use cases. And I think what's important about the VMware approach is we're very deliberate in making sure that the human is in the information loop. So all of our AI services, we call it intelligent assist. So AI is there to assist the human. And when the humans start to look at this as a tool that's going to help them to be more productive in their job and help them to find things faster, create content or whatever, that's where it really makes a difference. I mean, one of the data scientists, and he's AI engineer on my team, he just had this experience with this assistant technology that we've been running internally, and he was able to write an entire app in a couple of weeks, and he said that would've taken him probably six months without the code assist technologies.>> VMware Cloud Foundation obviously has transformed. We're in a new era of innovation. What's the one thing that you've learned over the past year that you can share to folks watching, customers or potential customers, they're going to implement Private AI. What's the best practice? What's the best approach? How should they lean into Private AI?
Chris Wolf
>> Yeah. I think that the best approach is to really narrow your focus and narrow your scope. You have to get very specific. Everybody wants to have an AI win. So to me, it starts with focusing on the shore bets, which is why we have Private AI Foundation with Nvidia, right? If you go with trusted partners in your journey, you're going to get good results. For a lot of companies, this is a board level imperative, right? The board wants to see the CEO show an AI win. So starting with us, starting with a trusted platform is one. Taking a platform approach is really important, because you don't want to just bet on that one app stack because you're going to have buyer's remorse next month if you do that, right? So that's the other area is to take the platform approach from us, that's worked out. And then what we're focused on with customers is getting them that quick win. So oftentimes, what is the data issue that you're having, right? What does it take people the longest to gain information to? And this is banks, this is public sector organizations, right? It runs the gambit in terms of healthcare, et cetera. In terms of, how can I help? So you have to narrow your scope, think about one problem, get that AI win, and then understand how you're operating that AI application, then start to move on to other things, right? So that's been our focus this year is getting customers to that first AI win that they can iterate on and then start to look at additional .>> And by the way, that's an easy win because they have data and the data gets them the ability to do it on a private environment. They get the win, they can understand how to operationalize it, scale it, and then everything falls into place.
Chris Wolf
>> Yeah. They can go from zero to a running AI application in two weeks. If you have the right scope, you can get there very quickly and see immediate value.>> And the beautiful thing for you guys, and again, props for being right on the right wave here early, is that all the hardware and all the systems being rolled out now are right into what you're doing.
Chris Wolf
>> 100%.>> All your customers are building faster, smaller, clustered systems, large scale, and then mentioned one of the leaders, others, all the OEM partners. So ecosystem is changing.
Chris Wolf
>> It is. It is. It's moving quick.>> Good. Good.
Chris Wolf
>> Yeah. I think that's the difference. AI has really reset the velocity expectation.>> Well, the value creation is going to be amazing. The app tsunami coming, it's going to be Cambrian explosion and applications, but they got to get their ops lined up, infrastructure teed up.
Chris Wolf
>> That's it. Get your house in order, get a reliable IAS first underneath your AI services, and then you can grow from there.>> Chris Wolf, you heard it here. Head of AI for Broadcom here on theCUBE, breaking down the private AI wave that's here. It's certainly tracking. It's all lining up perfectly to fund and fuel the next generation of value. I'm John Furrier with theCUBE. Thanks for watching.
