Name: Deep Dive: Cisco Secure AI Factory with NVIDIA Expands to Rack Scale
Uploaded: 2026-08-25T12:00:00.000Z
Duration: 21 min 40 s

Clips
More from Cisco Secure AI Factory with NVIDIA Expands to Rack Scale

Will Eatherton

SVP, Data Center, Internet & Cloud Infrastructure Engineering

Cisco

Marc Hamilton

VP, Solutions Architecture and Engineering at NVIDIA

NVIDIA

play_circle_outline Cisco & NVIDIA Rack-Scale AI Factory: Liquid-Cooled HGX/MGX/NVL72/Vera Rubin, Nexus One Management, Spectrum-X Networking

play_circle_outline Securing On-Prem ERP and Financial Data for Model Fine-Tuning with NCCL GPU Scaling Across Racks

play_circle_outline Inference as primary revenue driver; disaggregated serving, KVCache, prefill and decode pools

play_circle_outline Cisco Silicon One used for front-end networking and enterprise feature set

play_circle_outline Edge and telco use-cases: campus, hospitals, 5G, far edge, AI in space potential

26:39

Info

Deep Dive: Cisco Secure AI Factory with NVIDIA Expands to Rack Scale

Will Eatherton

SVP, Data Center, Internet & Cloud Infrastructure Engineering Cisco

Marc Hamilton

VP, Solutions Architecture and Engineering at NVIDIA NVIDIA

(INTRO)

John Furrier

>> Hello, I'm John Furrier, host of theCUBE here in our Palo Alto studios. We have some experts from NVIDIA and Cisco. They've come together to deliver Cisco's secure AI factories with NVIDIA expanding to rack scale. Cisco's SVP and head of networking engineering, Will Eatherton and Marc Hamilton from NVIDIA, VP, solution architecture and engineering. Guys, it's great to see you in person here at the studio. Thanks for having me. We just had a great conversation around the announcement, did a whole analysis of it, but there's some really important pieces of this that I really want to get more data on, which is the relevance and the importance of the relationship between Cisco and NVIDIA around the certification of cloud scale, rack scale systems for AI factories. This is not something you can just fill out a form and get certification. This is like a serious high bar of engineering involved because AI factories, huge demand, they're deploying fast, they got to deliver that first token, which is revenue, which we discussed on the last conversation. It takes a lot of work. Take us through the relationship and some of the moving parts on how this comes together because you're connecting networking conventionally with the modern era in the rack scale, scale up, scale out, and scale across. So, it's a perfect opportunity. Sure.

Will Eatherton

>> So let me start just with the core architecture of what we've announced today. So a couple of parts. First of all, from an overall basis, we do have now full rack scale compute with liquid cooling. And that is in all the form factors, HGX, MGX, the NVL72, and then moving forward to Vera Rubin. And so that's a core base component. And then from a Cisco software standpoint, and then in the rack scale we're putting in Cisco switches into that rack scale, so there is a consistent management layer across every part of scale out management across the entire cluster. Then from a networking standpoint, on the scale out portion, connecting GPUs, we have a stack from Cisco where we're taking the NVIDIA Spectrum silicon, we're putting our NX-OS or SONiC on it, running on top of that, and then we use that to connect the GPUs. On the front end, we use Cisco Silicon One, again with the same software, NX-OS, SONiC, and then we have an option there where, for instance, the customer's doing high-performance networking with storage, VAST is one of the key partners we have there, that can then use a Spectrum-X license in order to run the adaptive routing protocol. And so these are all key components. We also have customers then that will do data center interconnect and other routing capabilities. And so we then can bring that in with similar software. And so this is all of the core aspects is that it's the underlying cluster. It's then a network OS layer that from Cisco we're bringing in. And then we do a common management layer. So we have our Nexus One, which is how we do our day-to-day operations and management and monitoring. And then we have Cisco Cloud Control, which is where we bring in agentic ops across the whole solution. And so, again, from a Cisco standpoint, we're bringing in software sales and support for this whole cluster. And that's really been key in the partnership with NVIDIA because they're looking for outcomes for the customers that is parallel to the stack that they've put together with all the right technology components and then all of the RA, which is what we're going to get into next.

John Furrier

>> Mark, talk about some of the mechanics on the NVIDIA side. Obviously, this is connecting, traditional, existing, and Cisco's legacy. You see what they've done in the history books. They made the market. You guys on the AI factories, it's kind of like the land and expand. AI factory has to land and then expand and technically they got to connect. And that's why I love the accelerated computing tagline for NVIDIA because you're accelerating. You're not disrupting anything. You're actually embedding in and connecting in with things. It's not like kill the old to bring in the new. It's a whole mindset. And it's very nuanced, but it's an NVIDIA-first principle.

>> The AI factory is a new concept, right? Generating tokens, generating revenue. But that AI factory needs to sit within your existing enterprise infrastructure, right? The lifeblood of AI is data, right? Right. And why is that? Well, when you go and use ChatGPT or Gemini, those are great models. But remember, those are trained on basically public data, everything on the Internet. What your Cisco traditional enterprise network has access to is all of the data in your enterprise that is not out on the Internet, your ERP or financials, your marketing data and the like. And so what is the importance then of the reference architecture? Let's dive a little bit into detail here. It's much more than even a rack scale solution. This is really an AI factory -wide solution. And just think about it, whether you're installing 1 ,000 GPUs in an enterprise or 100 ,000 GPUs in a Neo cloud. When you're doing some post -training for fine -tuning of that model, you may have any two GPUs out of that thousand or out of that hundred thousand need to communicate together in lockstep. In the NVIDIA software that every AI model, every AI application uses, one of the layers of NVIDIA software is called NCCL, the NVIDIA Collective Communications Library. So think about this. You have two pieces of software that have no idea how they're connected across these 100 ,000 GPUs. They might be connected within the same GPU board, compute tray. They may be connected on that NVL72 rack by NVLink networking. They may be connected by the Cisco Spectrum-X scale -out networking. And in fact, as you get into the agentic world, you may have to call out to your agent, sitting in a sandbox or doing some tool calling that is sitting on your front-end network. And now you're running this and something slows down. Where on earth do you even know how to go in and look and debug this and optimize this across this stack if you're pulling together a bunch of different pieces from a bunch of different OEMs that have never put this all together before? And that really is the power of the reference architecture that Cisco is fully adopting and bringing to their customers through a single sale.

John Furrier

>> And the work that you guys have done to solve that complexity problem you just laid out, which, by the way, by itself, in isolation is hard. Now imagine the volume of traffic. So you put traffic into it, which you guys know a lot about at scale, and what they've done, this is where it connects.

Will Eatherton

>> Is that right? Right. And I think one of the key things in Mark's team from the NVIDIA aspect, some of the key things we've seen and our customers have found very important is, for instance, you mentioned the troubleshooting and the performance. You guys have put together a library of significant scripts and other intellectual property around how to do the checkout, the verification, the final output during the bring-up phase. and that really, as we've iterated, in fact, some of it was a little challenging to work through, that we make sure we take those scripts and we make the changes, for instance, with our Nexus aspects and that we'd then be able to support our customers at scale globally with the same type of technology. And so as part of that, we've gone through a whole joint checklist around the cluster that Cisco maintains in order to help do testing and benchmarking, those script aspects, putting together a team, Cisco Validated Designs (CVD), that will be the front lines for doing the checkout and then be able to work with NVIDIA when we see something that is not Cisco specific. And so, it's been a lot of work for us to go through these pieces together.

John Furrier

>> Cisco, you guys have had a lot of experience dealing with multiple networks with different capabilities, which you basically laid out. When you get into the killer app, which everyone's now realizing it's inference. Yeah. Training's certainly going to happen, but inference is where the intelligence in tokens turns to revenue, and as NVIDIA brings intelligence into things with AI factories, that's going to be a very important thing. How do you guys, with all those products that you have, Silicon One, Nexus, and a variety of stuff, how does that connect in? Because this is the inference, this disaggregated serving, caching around, you guys been there, done that, but now you got pre -fill and decode pools. You got weight model services going on. All the stuff that we've been covering with KV cache and all the coolness, it's all networking. Yep. So you've got all that going on in the factory, and then you come out and say, let's connect it to the world or the enterprise, unlock that data. That's value to the customer. Yep. From your perspective, you're enabling that with NVIDIA. How do you, what's the main story? Is it Silicon One? Is it Spectrum-X? So you guys just explain that to me like I'm a five -year -old.

Will Eatherton

>> So again, from a technology standpoint, it's really the union of the two from a Cisco standpoint. When we talk about conventional enterprises and the kinds of feature sets they need across L2 and EVPN VXLAN, there's a broad expectation, VMware -based systems and so on, that's been there for years, that continues to evolve. Storage and networking for broader enterprise, all the data that Mark mentioned that needs to be pulled in for use by the AI cluster. So all of those aspects are things that we certainly have experience with. And the biggest transition we've been seeing recently is that customers, we've had things that maybe historically have been fragile with MLOps and so on. Now, AgenticOps is a big transition. So from a Cisco standpoint, we've added Cisco Cloud Control, which brings in an AI Canvas, which then allows across security aspects, the firewalling, AI Defense, those functionalities, along with all those infrastructure components, which does spread across the Cisco Silicon One with the breadth of feature set, the Spectrum ASIC from NVIDIA, and then the software. And then as we build up the stack, I think one of the aspects we work with NVIDIA on is providing the NVIDIA AI Enterprise software package so that our customers have that capability on top of our overall system. So yeah, I think, to your point around moving to inference, I run a fairly large software engineering team, and I would say we have been using a lot of the latest AI development tools. When I look at the run rate of spend we had coming out last month, it was very significant. So one of our goals going through this next year is to work towards 50 % of our inference being with on-prem AI tokens, using our own infrastructure, and then 50 % leveraging the providers and really divvying that up based on different use cases. but we're at a point where you have to start leveraging on-prem because the cost structure is just going up and to the right.

John Furrier

>> Well, the enterprise piece is different. The neocloud piece is different from a scale standpoint, but it's kind of the same game. Pretend I'm a neocloud. I was just talking to a neocloud. I can't say their name, but they've got a $10 billion contract. They ring -fence it with finance. They're doing all these things. They have a specific scope. They're going to stand that up. They may buy a rack scale, but these guys are going to build their own, but they're going to go with the AI Factory. they got to move fast what's the data centric view because this is like a mega data center build out that's i won't say custom but purpose built for being a supercomputer the building and this is where i think this connects okay do i call a sales rep in from Cisco and i got NVIDIA i know what i want to do take me through that what is the pitch what do i need to think about i'm an architect i'm not a business guy i don't know i got to build this thing out, the power Now we've got to stand it up.

Will Eatherton

>> So a lot does start with the facilities and the underlying goals, because that can dictate a lot of the architecture. We have NeoCloud customers that, for instance, have started with clusters that are using some of the 8G, 8GPU, MGX, HGX, that are putting in sort of isolated clusters that they've already sold. That's a very different use case from a customer who, from the beginning, is thinking about multiple sites, is thinking about routing interconnection, that is thinking about facilities where they need rack scale and they need those capabilities, working with the right software stack. So for instance, we've worked a lot with Rafay and an integration layer and how you're going to orchestrate the services and how do you want to think about what does tenancy mean? Does tenancy for a NeoCloud mean just their direct customer or increasingly we're seeing requests of, hey, I want tenancy to extend to sub-orgs that are part of that customer. So there's many different ways that this can go. Tenancy is a really interesting discussion.

John Furrier

>> It is an evolving discussion. now you have multi-tenancy. What does that even mean? Is that multi-workload tenancy? Multi -company tenancy? The whole world's gotten completely flipped upside down, but that's the state of the art. Yes.

Will Eatherton

>> So, again, from a NeoCloud, it will start as simple as my end customer, their end customers that they may have multiple of are going to have some partitioning, and they want to be able to have from an accounting, a security, a monitoring standpoint, they want to be able to divide based on the end customer. But the new thing that we're seeing is multiple levels where their end customer may have different departments. They may have production versus development. And so they now want to be able to subdivide and be able to have different behaviors, different segmentation from a security standpoint. And so that is the new requirements we're starting to see. And this is, again, where, InfiniBand's had architectures to support. And now with Ethernet, we're starting to add those technologies in also. So it is a fast moving area.

John Furrier

>> Mark, what should people be thinking about the long view as they want to, in the short term, stand up the factories? Certainly the software stack is going to be really critical to that. As you look out, the next Pareto curve is coming. I'm sure Jensen will put up a nice new one on the next one. But you now have policy. You almost have multiple Pareto curves, multiple capabilities. This is networking. Policy has been around from day one. So it's all happening at large scale inside the clusters and outside.

>> Right. Well, I think to any enterprise CIO, agentic can be very terrifying, right? What could go wrong when a user gives their ERP or their CRM access to an AI agent. And so one of the big areas is around having that policy around agentic and how do you set up a safe sandbox to let your employees experiment and start running in agentic. So, again, that's one of the things that parts of the NVIDIA AI Enterprise software stack, such as NeMo Guardrails, provide as part of the Cisco solution. The second thing is that, again, we're just at the start of the AI age. Coding, it makes perfect sense. that software developers adopted this new technology early. But today, with agentic frameworks, anyone in the enterprise can go off and start being a developer and building their own agents. And as they start to do that, the AI is not limited. Your inference is not limited by how fast you can type to ChatGPT. It's limited by how many agents you have and how fast your agents type, right? And they type much quicker, right? So this is why driving down the cost, right, is going to be continually important. And driving down the cost means having that optimized hardware infrastructure, but also the optimized software infrastructure. If you look at the Blackwell generation of GPUs, over the lifetime of that product, we've driven down the inference costs by X factors, not 1 or 2X, but 10X, 20X, 30X, by going through and doing software optimization. So being able to continuously upgrade that AI factory once you install it is super important.

John Furrier

>> Yeah, and I think what I love about this market is that there's not one boilerplate approach. You can have CoreWeave vertically integrated, you can have, I see Nscale, they're doing well, you got Nebius, you got all these Neo clouds, they're deploying massive CapEx. Not every enterprise can get a big, a billion dollar building in Texas and build out a mega data center. and make an AI factory. So you're going to start to see that service layer connecting in and then you got upstarts like a company we covered, Argentum, which is taking a different approach. They're doing something, they're not trying to vertically integrate, they're trying to be just a refinery, if you will, of intelligence. So there's no one approach. At the end of the day, how fast can you get the infrastructure? Okay, so that's on the NeoCloud side. On the enterprise side, I don't have a billion dollars or maybe I don't have $10 billion to spend on an AI factory but I'll use a CoreWeave. I'll tie into a hyperscaler. I'll get my stuff on-prem. Maybe I have a mini AI factory. I saw some of the new solutions from NVIDIA. I could see everyone's desktop having an AI factory on it. So this is kind of where I see it going. Your thoughts? Because this is right in your wheelhouse, because you know all the routes.

Will Eatherton

>> You know all the plumbing to get to the factory. A few things there. I think certainly desktop is something we're very interested in, And we are seeing that is an interest for a lot of our customers, which is not only around the devices that they would have, but it also plays into, from a campus networking, cabling and interconnection, the security that needs to happen there and scale, a lot more scale, much quicker on campus. So that's been a whole topic. For enterprises overall, I think this is why having a range of form factors is really important. So certainly there's a lot of attention to a liquid cooled rack scale. we still see a lot of interest in, HGX, MGX, the 8GPU form factors. Folks that will put those in, maybe 128, 256 GPUs to start. But they are looking for commonality architecture -wise between what that looks like deployed in their enterprise data center, maybe it's a more conventional data center and their spacing is out, and what is the architecture, the security services, and so on of the NeoCloud that they're partnering with. And so that is a transition that we are increasingly wanting to make as seamless as possible for our enterprise customers. They shouldn't overly, there is obviously going to be a cost difference, but they shouldn't overly care from a integration standpoint, whether those GPUs are sitting in their data center or whether it's in a colo.

John Furrier

>> I mean, there's no doubt in my mind that enterprise will have AI factories on -prem. It'll be rack scale. I was just covering some work that you guys have been doing in healthcare. You have the AI operating room with a project going on. So you will have a supercomputer in the hospital. No brainer. But branch office, maybe we're tapping the factory and we're going to the service that looks like a single tenant to you, but it's a multi -tenant NeoCloud. So this is kind of where it gets kind of interesting because now you're going to have all the benefits of security and end-to-end networking. This is something that's kind of exploding with agents. What's here? How does that play out? Any thoughts on that? Because that's where most people see it going.

>> Well, AI is definitely going to extend to the edge, right? And there's different reasons for that. Sometimes you want to keep your data at the edge for policy security reasons. Sometimes, whether it's a self -driving car or a robot in a factory, just the latency requires you to have that at the edge. So again, the natural movement is to bring AI to the edge through telcos, the people that have all this 5G infrastructure. They got networks, they got power. That's right. And they got facilities. And they often have Cisco networks. What do you guys call it? Shell? What's the term? Shell? Land power shell. Land power shell. They got all that. And then the ultimate land power shell, and Cisco isn't quite in this space yet, but I wouldn't hold them back is of course AI in space, the ultimate edge.

John Furrier

>> Yeah, it's ultimately, it's all about the edge, far edge. Guys, thanks so much, I really think this helps clarify and also illuminates the future opportunity is that intelligence, not just silicon and GPUs and components, but the demand for intelligence we think will continue into areas that we don't even know yet. But you got to connect it, low latency, relevant data at the right place at the right time, and I think the intelligence will figure it out. Thanks for sharing. That was fun geeking out. All right, we're at theCUBE here doing our part, doing the deep dive, bringing all the data to you here with our mixture of experts. I'm John Furrier, host of theCUBE. Thanks for watching.

Deep Dive: Cisco Secure AI Factory with NVIDIA Expands to Rack Scale

(INTRO)

John Furrier

Will Eatherton

John Furrier

Will Eatherton

John Furrier

Will Eatherton

John Furrier

Will Eatherton

John Furrier

Will Eatherton

John Furrier

Will Eatherton

John Furrier