In this video, we explore the dynamic landscape of artificial intelligence with Josh West, global artificial intelligence ecosystem leader and distinguished architect at Red Hat. Filmed at the New York Stock Exchange's AI Factories event, West discusses the pivotal role Red Hat plays in shaping the future of AI, with particular emphasis on private AI and open-source technology. They are joined by Dave Vellante of SiliconANGLE Media in a conversation that uncovers significant industry trends.
Josh West brings over two decades of expertise at Red Hat, where they now spearhead the global AI ecosystem by forming strategic partnerships with technology providers. The discussion, hosted by Vellante on theCUBE Research, focuses on how Red Hat contributes to the "AI Factories: The Data Centers of the Future" initiative. It showcases the integration of private AI technologies within enterprise information technology environments and their impact on financial institutions.
Key insights from the conversation include the shift towards private AI solutions at on-premises data centers and the collaborative efforts with companies such as Dell and NVIDIA. West states that aligning with open-source projects provides clients with the flexibility and stability necessary for fostering innovation while maintaining regulatory compliance. This session further explores the importance of customizable AI platforms based on Kubernetes technology.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. If you don’t think you received an email check your
spam folder.
Sign in to AI Factories - Data Centers of the Future.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for AI Factories - Data Centers of the Future
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for AI Factories - Data Centers of the Future.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. If you don’t think you received an email check your
spam folder.
Sign in to AI Factories - Data Centers of the Future.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: AI Factories - Data Centers of the Future
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Josh West, Red Hat
In this video, we explore the dynamic landscape of artificial intelligence with Josh West, global artificial intelligence ecosystem leader and distinguished architect at Red Hat. Filmed at the New York Stock Exchange's AI Factories event, West discusses the pivotal role Red Hat plays in shaping the future of AI, with particular emphasis on private AI and open-source technology. They are joined by Dave Vellante of SiliconANGLE Media in a conversation that uncovers significant industry trends.
Josh West brings over two decades of expertise at Red Hat, where they now spearhead the global AI ecosystem by forming strategic partnerships with technology providers. The discussion, hosted by Vellante on theCUBE Research, focuses on how Red Hat contributes to the "AI Factories: The Data Centers of the Future" initiative. It showcases the integration of private AI technologies within enterprise information technology environments and their impact on financial institutions.
Key insights from the conversation include the shift towards private AI solutions at on-premises data centers and the collaborative efforts with companies such as Dell and NVIDIA. West states that aligning with open-source projects provides clients with the flexibility and stability necessary for fostering innovation while maintaining regulatory compliance. This session further explores the importance of customizable AI platforms based on Kubernetes technology.
Global AI Ecosystem Leader & Distinguished ArchitectRed Hat
In this theCUBE + NYSE Wired segment from AI Factories – Data Centers of the Future, theCUBE’s Dave Vellante sits down with Josh West, who leads the global AI ecosystem at Red Hat, to explore how open source and private AI are reshaping enterprise infrastructure. West explains his remit spanning partnerships from silicon to ISVs and public clouds, and details why the “third wave” of AI – on-prem inference with full-stack control – is accelerating as enterprises bring AI to their data. He outlines Red Hat’s role at the core of modern AI factories: Linux and Ku...Read more
exploreKeep Exploring
What is the professional background and current role of Josh West at Red Hat?add
What is Red Hat's role in the evolving landscape of AI and its deployment in local data centers?add
What are the current trends and challenges in building on-prem AI stacks in financial institutions, and how does Red Hat fit into this landscape?add
What are Red Hat's efforts regarding the support and maintenance of popular open source projects for customers?add
What role do open standards and open-source projects play in ensuring consistency across different technologies and platforms?add
>> Hi everybody. Welcome back to New York Stock Exchange. You're watching theCUBE plus NYSE Wired's AI Factories: The Data Centers of the Future. Josh West is here from Red Hat. He sits right at the intersection of private AI and open source. And we're super excited to have you here, Josh. Thanks so much for coming on.
Josh West
>> Thanks for having me.
Dave Vellante
>> So you're in this area, you live in this area, you work a lot in New York, you work with the big financial institutions. What is your exact role at Red Hat?
Josh West
>> I live in northern Jersey. I've been at Red Hat for 19 years, and this used in my backyard where I covered financial services and I was a distinguished architect for the largest universal banks and card networks and otherwise. And I recently moved over to lead the global AI ecosystem where I'm managing the go-to-market and the partnerships that we have with everything from the silicon providers to the ISVs that run on top of Red Hat's platform, the public cloud providers and more as well.
Dave Vellante
>> Red Hat of course, is the poster child for open source, the most successful open source project in the history of open source projects, one could easily argue. So what do you see Red Hat's role in this new AI era, particularly as it relates to private AI? And we're talking about AI factories and the build out of these future data centers. How do you see Red Hat's role fitting there?
Josh West
>> Well, I certainly see that AI has started with research and centers where things are consumed as a service or models are coming out of research environments or OpenAI or Gemini with Google. And those are possibly consumed through services you you're getting from Salesforce or Workday or things you've already purchased. Then people are consuming services from public clouds through a token-based interface. But this third wave is really starting now where people are running AI inference in their local data centers so that they have full control of the stack all the way from the hardware up to the developer experiences and the connections to their enterprise systems, the connectivity that's there. Also, really the overall software development lifecycle. So Red Hat is founded in open source. I'd say a lot of AI happens because of AI. Most that is running on top of Linux. Any scalable AI system is running on Kubernetes. It's the heart of Red Hat's OpenShift technology. And so Red Hat's working with a broad ecosystem. And of course we're going to talk today about Dell and NVIDIA and how we're working closely together to provide this private AI foundation. It's really going to start on-prem and extends consistently to public clouds and give you that option for what you have control over for AI overall.
Dave Vellante
>> Yeah, we'll definitely give the love to our generous sponsors, but I want to ask you about what's happening with enterprise AI specifically, because we know that a lot of the action happens in the cloud. I mean essentially the cloud and the hyperscalers and Meta and X AI, the neoclouds, they're sort of funding all the enthusiasm. We see all the green and the stock market. And when we talk to enterprises, as you point out, I like to call it bringing many, it's not my term, but bringing the AI to the data. We certainly see that trend, but the stacks aren't as rich on-prem as they are in the cloud, so they have to be built out. And so what we're seeing is a lot of financial institutions in particular, they've got the technical skills and they're actually building their on-prem AI stacks. What do you see that? Can you confirm that? And where does Red Hat fit? What do those stacks look like?
Josh West
>> Well, everything starts from that foundation of the actual hardware coming from the GPUs, NVIDIA, OEMs like Dell that are composing these full racks and solutions that our customers can get into. They've already built out these large environments for virtualization containers and otherwise. And now as AI enters, a lot of that started as very single purpose stacks inside of the data analytics teams where people were consuming clusters for their early pilots of GPU access, really for machine learning and predictive AI. And they started dabbling in generative AI on-prem too. But as we've heard from the MIT study and across the, I think everyone's seen on the internet these days, a lot of those early experiments have failed per se, but really that's actually really healthy way to get started with any technology thing. It's business experiments lead to learning those lessons and evolving something that is going to be truly guardrailed, controllable for getting the production. And knowing that you're not going to impact your business in a negative way, but you're going to actually grow the business in a way that you're actually dealing with change management throughout the environment. So I think a lot of folks that I actually saw myself working in financial services, they started with a single purpose stack from one vendor. Then they said, "Hey, we need to run this software a slightly different way, and so we're going to build all this stuff ourselves from open source, because we're so deeply technical." And that doesn't work because the technology is changing so fast. And so Red Hat is working with the other providers to make sure that these most popular open source projects are steering to a way that is supportable and maintainable for our customers to have that foundation to run their business on long-term, not just to move fast on as well. So just to close off that point, we saw that actually happen at some of our earliest customers. They went through those phases and now they're based on Red Hat's, OpenShift AI technology, working with NVIDIA and Red Hat together to provide a software stack that they can maintain long-term and distribute across the world and to sovereign countries on the public clouds and in their big strategic data centers too, really blend in with the rest of their enterprise IT environment as well.
Dave Vellante
>> Josh, Jensen at GTC in March, he laid out sort of the three vectors of growth for AI, AI in the cloud, AI in the enterprise, and AI in robotics. And the deal announced last week between Intel and NVIDIA, we felt was significant in accelerating that middle, the enterprise AI. Because while to your point, there's been a lot of experimentation, experiments fail. That's the whole point to your point, but much of the stack, the AI stack today has been highly vertically integrated. I mean, Z itself, it's the, I called it poster child before, but it's the poster child for vertical integration. NVIDIA, you could take the entire CUDA stack, you could buy the whole NVIDIA, and people have said that, "We bought the whole NVIDIA stack, but we want alternatives. We have expertise in other areas or use cases." You could argue the hyperscalers have their own sort of stack-like vertically integrated system. So I want you to talk about why open source is so critical for AI, how it preserves optionality for customers, and how is Red Hat working with enterprises to make sure that that choice and portability of preserved specifically?
Josh West
>> Yeah. In the old days, consistency or something that could make something similar or the same on one technology versus another used to just be open standards. Everyone would comply with a specific regulation or a specification. Open source brought the idea that we can all be based on the same projects, engineering and testing our technologies to work with specific chips. Like the Linux Kernel is a great example of something that works on IBM Z, it works on x86, it works on ARM, it works on RISC-V and all these other new emerging standards too. The same thing goes for GPUs. GPUs are diversifying in so many ways, but there's a common way that you can deal with inference. And so Red Hat is leading the project called vLLM. We see that's used in public clouds. It's used by other inference providers and NVIDIA themselves in their Dynamo release too. So we're basically the core contributors to those projects, making sure that that is going to be the heart for everyone to run inference and Linux operating system, and if you zoom out, Kubernetes at scale to run an entire platform. So whether we're working commercially together or not, open source is the thing that provides consistency and stability for those customers. And so that's how we see it. We don't have to force people to do things, but we believe that people understand Red Hat is leading the charge to shape and steer that to a way that you can consistently get to an enterprise operable, secure, safe and sound standard environment. And then the choice of running IBM Z, IBM Power, x86, ARM, RISC-V or all the other chips that are coming out, or NVIDIA, Cerebras, AMD, Intel GPUs, Gaudi GPUs, or either the public cloud, the ones it's going to really come down to. You want something that is going to be the same across those. And open source allows you to have that consistency so you can build something once and know it's going to work to a large degree and in another spot or in your future hardware that you're blending into your environment to.
Dave Vellante
>> Write once, publish many and so many... Such a powerful metaphor. I want you to put your architect hat on for a moment and put you in the context of I'm developing a private AI sort of on-prem capability for my organization. One of the biggest challenges because I can go to the cloud, I get multiple data stacks, I get multiple governance capabilities, I get every application there, every SaaS application under the sun. So I have this menu of services that I can spin up through APIs that are pretty well understood by developers. And now bring that on-prem, it seems like there's work that has to be done to get there, but the benefit of getting there is A, I can bring that AI to the data. I don't have to move the data into the cloud. And B, I've got proprietary data that I want to apply for competitive advantage, and I don't want it leaking into the public internet or LLMs. Not that the cloud guys are doing that, but just the whole data movement, the data migration, the security, the privacy makes me feel better at night. So what are the challenges from an architectural standpoint of building that out and how are you guys helping?
Josh West
>> Sure, yeah. Well, certainly I strongly believe in moving AI to the data, not just where it's stored, but also where it's emitted as well. So generative AI is a little bit different than machine learning, where historically you had to centralize data to do machine learning or data analytics on that data within a central pocket of the firm. A lot of that moved to public cloud because it is elastically scalable and you were able to better have data storage tiered for a cost model as well and get more rich tools to process the complexity of that data. When we talk about generative AI, we're not just looking at all the data that's there. You're actually condensing your domain knowledge, your domain, your business data, the knowledge of the internet into something that is a really relatively small version that it might be a trillion parameters or it might be a couple billion parameters, but it's actually something that is a condensed version of all this knowledge and way to reason on your business environment. And so you can apply that to data in a small context. Could be a single document. It could be one web request that's coming in emitted from a system that's within one divisional application, or you can apply those models to data at scale and data warehouses that probably still running Cloudera, Hadoop, or Spark in your on-prem environment. And also you might be connecting to Snowflake or Databricks through MCP from your local environment. One thing we found is that a lot of regulated industries, at least, they're not able to contact from the public cloud into the data center. And so you're going to need data, you're going to need AI inference, close insider environment that can reach out to the public cloud, but it's difficult to get it to happen.
Dave Vellante
>> Sorry, you're saying that's a regulatory issue, is that right?
Josh West
>> Regulatory the way at least banks and other financial firms-
Dave Vellante
>> A compliance-
Josh West
>> industry-
Dave Vellante
>> A best practice even?
Josh West
>> Yeah. Well, I'd say if you're the Department of Defense, Palantir is a partner that runs on top of OpenShift in their environment because they're usually in those very, very private environments that are completely disconnected from the internet. A financial firm is connect the internet, but really traffic only goes one way, which is outward. Then like any large firm has to operate in different parts of the world that might have, we're seeing this everywhere, sovereign regulations where data isn't allowed to leave the boundaries of that country. So even like payment systems in India, you got Saudi Arabia, which is a huge place where there's so much infrastructure coming out for AI. There's at least 14, 15 countries where data has to live in a private world, at least can't leave the country. And some of those countries don't have clouds like Turkey, for example. And so there's best practices. There's also just a reality, you're going to need some blends of local, private, sovereign controlled AI and also public cloud that's going to move fast and give you all those new services that you were mentioning too. They're going to innovate fast, but what you're going to have that's stable as your foundation and run your business consistently, you're probably going to want run that steady state on-prem and your own GPUs with a software stack that manages all that complexity for you, which is really what Kubernetes is, OpenShift and NVIDIA and the rest of us are really good at together.
Dave Vellante
>> I want to juxtapose cloud and on-prem because in the early days of the cloud and the ascendancy of the cloud, the cost structure was better. The facile nature of cloud spinning up virtual machines and such a rich ecosystem that developed and on-prem was at disadvantage because it was very labor-intensive. By premises, that's dramatically changed. Depending on the workload, you can make a strong case that on-prem is could be significantly less expensive. Maybe you don't have the richness of the tooling, but it's pretty cost-effective. It's closed the gap significantly. Now, the cloud guys would argue on their sovereign AI argument, the cloud guys might argue, they're trying to argue that eventually we'll get there with virtual private cloud, virtual private AI, we'll be able to convince those governments to use the cloud. My personal opinion, love to get your thoughts on this, is they're going to have to put clouds in those regions and that's how they'll deal with it. So it seems to me the advantage of the pendulum in that regard swings to the on-prem. So to the extent that they, so they've got the cost piece down, you can pretty efficiently manage infrastructure these days. They've got to work on the stack. They got to have more stack optionality. And then there's, my question is startups where all the innovation occurs and startups have been, if you went to a VC and said, "Hey, I'm going to build an on-prem solution." They would've kicked you out 10 years ago. And so the startups haven't historically had a great route to market to on-prem. Partnerships give them one. How do things like Kubernetes address that gap?
Josh West
>> Yeah, I mean, first component, you mentioned that technology historically was so difficult, because it's so labor-intensive. That was because everything that you needed to deploy and configure was done by individuals running scripts, typing at a command line or clicking buttons on a screen. And Kubernetes drastically changed that, where you can just declare, you know what hardware you have, you can declare what configuration you want. And Kubernetes itself pulls its dependencies altogether in a way that you can compose the infrastructure that you need. And so Red Hat's 11 years in the making, has made that so strong that you can have just a bunch of Dell machines deploy a full cloud that you can run virtual machines, containers, and AI workloads on that with a single push of a button. And that's through all the hard work that we've done with NVIDIA and Dell and all the other players in these upstream open source projects. And not us alone, everyone in the ecosystem contributing to the CNCF and Kubernetes overall have led to this technology. You can run basically your own cloud in your own data center that is very rich in nature. And Red Hat has, when you set all those other capabilities on top, public clouds have a gradient of shared responsibility. You can consume a virtual machine, you can consume bare metal nodes on a public cloud. That's probably the most expensive option. Or you can consume their highest level abstractions, things like serverless frameworks, and basically the amount of responsibility that you want, the more that they're going to charge you for access to that because they're basically doing timesharing of access to hardware. You can actually get a much better price if you have control over your infrastructure by running your own software stack and deploying your own cloud as long as you're keeping that evergreen and keeping up to date and making sure you're following the latest innovations. And so like Red Hat and NVIDIA, we're providing a supply chain of all these new innovations that you can keep up to date and really provide your business an experience that you can have a renaissance of innovation on top, again, where you want to deploy it. So I would never pit Red Hat against a public cloud. I would say we're complimentary to them. We're actually partners with all the, at least five major public clouds where our committed spend can be worked or we can draw down on the committed spend and vice versa. So they're not competitors at all. We're not going to be pitted against them. I say that we're all in the same boat trying to drive innovation. That point that you made about, wouldn't all just be public clouds, they'll get into all those other countries eventually. JP Morgan Chase, their CISO just released an article describing the challenges of public cloud and how there's still a lot to be desired with they want to run things on-prem. They've even described that they're going to run a lot of their banking services in their own data centers and build their own cloud. And I think that's important to note. And I guess Michael Dell, again, Dell's our partner, he said that the data center investments are tripling over the next five years as well.
Dave Vellante
>> They could see it in there already.
Josh West
>> Yeah, yeah. This isn't going to go away. It's going to blend. It's like it really comes down to how much responsibility do you want to own that infrastructure for which use cases. It's not a binary decision. It's a, which use cases do you want to have full control environment. It's probably the highest volume use cases and where you have this most strict compliance considerations or security considerations as well.
Dave Vellante
>> The Brits call it horses for courses.
Josh West
>> Yeah.
Dave Vellante
>> All right, last question. If you look out toward the end of the decade, how do you see the data center stack evolving? Will Kubernetes be that orchestration layer for AI factories or do you envision a new abstraction for agentic and agents and AI?
Josh West
>> Yeah. Microsoft or Azure has said that Kubernetes is going to be the foundation for AI. If you think about when people talk about AI, it seems like one big model. One big brain is going to rule everything, but think about how any organization runs. They try to have change management to isolate change, to only have one thing tweaked at a given time. So if you're an enterprise, you put all your eggs into this one model basket, what happens when that model changes or search, interpreting the things in different ways? You want intricate control over how you're interpreting the summary of customer requests coming in. And that's going to be one model. Single purpose is going to respond in a very specific way. Calling a ServiceNow and opening a ticket, and that's going to be tested revisioned and change managed in a very specific way. It's only be one big model to be many different flows. So we believe that, I think NVIDIA has been talking about this too, there's going to be a lot of smaller models, reasoning models that are reinforcement learned to get better and better at single purpose tasks. When you think about the history of Unix and Linux, everything is a single purpose process. Do one thing really well, and I think that's going to become true of agents as well. Going to be vertically really strong at evaluating cybersecurity for one use case or really strong in evaluating doing customer calls and solving a customer satisfaction issue in a call center, so on and so forth. So what really matters is that people are really thinking about how should we be changing the day in life of the employees? And how can we reach our customers in entirely new ways? That's going to happen on a platform where you have all the capacity and you can run wherever you need. And it's great to see every vendor is working together to provide this one foundation or an AI platform that anyone can consume anywhere in the world.
Dave Vellante
>> Yeah. To your point about small language models, early in 2023, we published, theCUBE Research published the Power Law of Gen AI. It was theoretical at the time, but the theory is proving out. Josh West, really appreciate your time and your architect brain coming on theCUBE-
Josh West
>> Thank you. Thank you.
Dave Vellante
>> And NYSE Wired. Thank you.
Josh West
>> Good to meet you.
Dave Vellante
>> All right, good to meet you. Okay. We're seeing a productivity boom here powered by AI factories. Red Hat's a big part of that, as is open source. This is Dave Vellante for John Furrier, and NYSE Wired team, AI Factories: Data Centers of the Future. We'll be right back right after this short break.
Dave Vellante