SC25 | Saurabh Kapoor, Dell Networking

Clips
More from SC25

Saurabh Kapoor

Director Product Management, AI Networking

Dell Networking

play_circle_outline Enhancing AI Performance: Dell's Lossless Network Fabric and RDMA Over Converged Ethernet Solutions

play_circle_outline Dell's partnership with Nvidia introducing InfiniBand and Ethernet for scalable AI infrastructure.

play_circle_outline Networking solutions enabling enterprises to achieve hyperscale-level performance without massive capital investments.

play_circle_outline Enhancing Enterprise Networking: The Power of SONiC and Spectrum-X Integration for Unmatched Flexibility and Choice

play_circle_outline Empowering Enterprise Customers: Dell's Comprehensive Services for a Seamless Transition to AI Solutions

Info
Transcript

Saurabh Kapoor, Dell Networking

Saurabh Kapoor

Director Product Management, AI Networking Dell Networking

In this SC25 interview, Saurabh Kapoor, product manager of Dell Networking at Dell Technologies, joins theCUBE’s John Furrier and theCUBE Research’s Jackie McGuire to discuss why networking has become the "operating system" of the AI era. Kapoor details the unique demands AI workloads place on infrastructure, citing the need to manage "elephant flows" and bursty traffic through lossless fabrics and RoCE v2 technology. The conversation highlights how the Dell AI Factory with Nvidia is optimizing compute, storage and networking to prevent packet loss and ensure... Read more

explore Keep Exploring

What are the key considerations for optimizing infrastructure capabilities when running AI training jobs? add

What are the benefits and technological advancements associated with the use of AI infrastructure and open-source networking technologies in modern data centers? add

What role does SONiC play in extending hyperscale networking capabilities to enterprises, particularly in the context of AI and budget constraints? add

What is the significance of SONiC in networking and how is it transforming the environment for enterprises compared to hyperscalers? add

What kind of support and services does Dell provide to customers in their AI journey? add

bolt Powered by CUBE AI

Saurabh Kapoor, Dell Networking

search

John Furrier

>> Welcome back everyone to theCube's live coverage here at SuperComputing 2025. I'm John Furrier, your host of theCUBE with Jackie McGuire From theCUBE Research, breaking down all the AI infrastructure. You've got compute, you've got networking, you've got storage, all coming together in the new AI architecture that's scaling up, bringing in the agents and bringing in all the benefits we're seeing with AI native. Saurabh Kapoor is here as Product Manager of Dell Networking, Cube alumni. Great to see you again, thanks for coming on theCube.

Saurabh Kapoor

>> Thank you, John, Jackie.

Jackie McGuire

>> Nice jacket.

Saurabh Kapoor

>> Good to be here. Thank you.

Jackie McGuire

>> Sporting the Dell colors with the jacket.

John Furrier

>> He's on brand.

Jackie McGuire

>> I appreciate the rizz.

John Furrier

>> The quote from Nvidia that they put on their LinkedIn was from last weekend when I sat down with Gilad, you saw the interview. "Networking is the future of AI." It was in context to what he was talking about, I'll say. The whole operating system of these large scale super computers, AI factories, is the networking and the data around the chips and all that stuff. So one, you're in network, so you probably don't disagree. But how has AI networking changed? And what is Dell doing to solve some of the bottlenecks? There's now a lot of new things coming out, Nvidia's showing their NVLink. We'll see how that plays out. That's all connected to KV Cache, Dynamo, what's your take? How are you guys solving the networking bottlenecks?

Saurabh Kapoor

>> You were spot on, John, when you said, "Well, networking is cool again, thanks to AI." There's a realization how critical networking is. But the element here is the uniqueness. When we look at AI workloads, networking characteristics are so different. You're looking at Elephant Flows, traffic, links that can get saturated in microseconds. Low entropy use cases. It's all about how do you address those characteristics. It's unlike the traditional infrastructures and workloads where you were not looking at these lossless fabric capabilities, highly optimized infrastructure. So we're helping drive that innovation. Dell AI Factory with Nvidia has been the thought around it. How do we bring compute storage networking, making sure that these infrastructures are highly optimized for AI infrastructures, workload spanning, training and fencing, fine-tuning, you name it, and then building those capabilities.

John Furrier

>> Define what lossless network fabric actually means, because he's mentioned lossless. What does that mean? Because it's got new context now.

Saurabh Kapoor

>> Yes. So when we talk about AI, you're looking at capabilities that infrastructure requires, you're running training jobs, and you're looking at job completion timelines and things like that. So you want to make sure that the protocols, the capability, we call it RDMA over Converged Ethernet, RoCE v2, a technology that helps you build that lossless capabilities. You're not losing those packets in the infrastructure. Because if you lose the packets you're talking about capabilities that the job completion timelines increase, and your infrastructure is not getting peak performance. So capabilities to address explicit congestion notification, enhanced transmission selection, data center bridging, all of that coming together to give you that highly optimized infrastructure that's not dropping packets. You highly optimize to deliver those capabilities.

Jackie McGuire

>> How do you do things like your traditional network prioritization and things like that at that scale and velocity? How have you had to adapt the way you think about networking for those crazy, like you said, bursty, it's not traffic that's predictable, right? It's this insane amount of bandwidth needed, and then how do you contain the costs when that comes back down?

Saurabh Kapoor

>> Yes, yes. So there's a few ways we look at it. One is choice of technology when it comes to AI infrastructure, networking perspective. So with Nvidia, we have expanded the partnership to bring in InfiniBand and Ethernet based technologies that help deliver scale capabilities across ecosystems. We build the largest of our factories, is the GPU farm. So you could have distributed and fencing use cases. Now, what it does is, especially with the Ethernet, it's a highly commoditized market. You bring in these next-gen software technologies. SONiC is one of the big focus areas, software for open networking in the cloud. It's an open source technology. We've put the Dell capabilities around it, adding innovation and flexibility. We have customers who deployed SONiC into the environment, and given that open source element, you've been able to reduce the cost of ownership by almost 40 to 50%. So depending upon the scale and capabilities that you're looking at, that could translate into millions of dollars of savings overall from infrastructure perspective.

John Furrier

>> Dell, you guys are extending SONiC into the Spectrum-X. Okay. So how does that work? Because hyperscalers have figured out the networking at large scale with the purpose-built interconnects. I mean AWS, Meta, they're all there. So how does an enterprise get hyperscale level AI in the factory? What does it specifically that SONiC and Spectrum-X, do? What do you guys... How do you make that work? Because the enterprise all want that, but they don't have the CapEx budgets.

Saurabh Kapoor

>> Right. Right. I'll take you a little bit in the time, right? So back in the early nineties we saw Linux and Windows came into existence. One common operating system running across multiple compute hardware ecosystem. The same thing is happening with networking now, with SONiC becoming the Linux of networking. Where it could have one common operating system that runs across multiple silicon and hardware architectures. That gives scale agility that the hyperscalers wanted in their environment when they started cloud and now expanding into AI. What Dell is doing with SONiC is extending those capabilities from the hyperscaler ecosystems out to everybody else. It could be large enterprises, telcos, they do two cloud providers, and beyond that. So that means that now need enterprise level capabilities. You need the VX standard, UPN, multi-chassis lag and all of that innovation. And above that you need enterprise support. Why? Because those hyperscalers have army of engineers to champion those environments. But when it comes to enterprises, you're looking at capabilities, best-off open source enterprise partner support, making sure those infrastructures are up and humming while they focus on end-business applications. So that's the element that Dell brings into the ecosystem. We had a portfolio with the partner. We expanded that to Nvidia Spectrum-X capabilities. Just this week we made that announcement about Spectrum-X with Dell SONiC, compliant with Nvidia Cloud partners solution architecture, bringing the best practices, the innovation that Nvidia did for Spectrum-X, and now championed with SONiC.

John Furrier

>> Which Nvidia switches do you guys work on?

Saurabh Kapoor

>> So we focused on AI fabric capabilities, so Spectrum-4 platforms, auto-band management capabilities, platforms that connect your East-West traffic, North-South traffic. We have all of that. We working with Nvidia for the next gen technologies that they plan to roll out for next year. I won't steal that thunder. So there are a lot of good things that Jensen spoke about at GDC and different forums. So we're looking to bring in all new technologies in Spectrum-X that they're bringing from silicon photonics and air-cooled, liquid-cooled platforms and more.

Jackie McGuire

>> I think this, well, one of the things we were talking to Seamus about a couple of segments ago, was the fact that we always hear from Dell customers that you act as a partner. So they always say, "We partnered with Dell to X, Y, Z." And we were hearing that with Seamus in terms of asking for help with what you need. I think one of the other things that I have really noticed about Dell is you've really embraced open source technology. Do you feel like those two things go hand in hand and that because you more closely partner with your customers at a lot of other companies. You, by embracing open source technology, you get closer to where they're at and are able to more effectively partner with them. We hear that all the time, and I think SONiC's a good example of this is open source technology, and you guys also always emphasize choice. So do you feel like that's part of the Dell culture or is that just an evolution that you've had to make because customers need more support?

Saurabh Kapoor

>> Yes, it's a combination of two, right? So Nvidia has been an early player in the SONiC when all the community came together to champion SONiC as technology supporting the hyperscaler engagements. Nvidia has been engaged in the SONiC journey since the very big day. What we decided to do was now take that capability that Nvidia did in the open source world to now extend it to everybody out there, the enterprise space. Because there's an element, as I said, the support capabilities are something that the enterprises want before they champion open source capabilities. So our partnership with Nvidia started with the Dell AI factory with Nvidia last year. We continue to add Spectrum platforms, the software stack, Omniverse, NIMS services, et cetera. And this year is all about how do we extend and bring more choice and flexibility when it comes to network operating system. Extending SONiC capabilities for the market that is looking for a choice and flexibility when it comes to network operating system, delivering that as part of the Spectrum-X solution as well.

John Furrier

>> You got the silicon independence, you got the open source. Talk about the Nvidia Dell AI factory and how you're aligning with their tech or their reference architectures. Because the confidence of AI factories right now is, I say the enthusiasm is high, but as the deployments move from super big Neoclouds to large super large scale enterprises, the mainstream large scale enterprises will start adopting. I don't think they have the confidence yet because the NIMS are out. You get NEMO. So you have the ecosystem coming up fast. So what's that alignment with Nvidia? Can you share how you guys co-design with them?

Saurabh Kapoor

>> Yes. We co-design, co-engineer some of these solutions with them. Nvidia does a great job in building those reference architectures, right? You've got enterprise reference architectures, Nvidia cloud partner solutions. So they have reference architectures that define what good looks like for East-West, North-South capabilities out-of-bound management connectivity and the software stack that sits on top of it. What we are doing is taking that best practices, building the software stack up. So when we decided to extend the SONiC capabilities out for Spectrum-X, we reference those, reference architectures. What those capabilities mean from a software solution perspective, expanded SONiC capabilities to support those capabilities and now offer this. Then you see now Nvidia Spectrum-X with Dell SONiC compliant with Nvidia cloud partner. What it does for those end users is they're able to now expand and build on those capabilities and get the choice of operating system when it comes to how they want to build those infrastructures. So bring in that open source element around it. I'm sure you heard Gilad, and I saw one of those videos interviews that you do with Gilad and talking about SONiC. So this is Dell adding more to it and in championing Nvidia capabilities.

John Furrier

>> I mean, Gilad from Nvidia, he's the networking lead, senior vice president. He was very candid about networking, although I've been saying KVCache is the operating system for the AI factory because Jensen said that last GTC. I'm now corrected from Dion from Nvidia who said, "No storage is in there too because you now have storage is in the fabric. It is network." But KVCache is networking basically. So how could it be an OS? So I say, "Okay. Dynamo is the OS because you now got KVCache and you've got storage fabrics coming together." So how are you guys rolling that out from a software stack standpoint? Because everybody wants AI ready systems. 1) What does that mean from your standpoint and what is Dell doing specifically to make these things stand out? Because you can't drop, ship an AI factory. It's not like dropping a server on a loading dock and racking it up. You got to actually deploy it with a software stack. What's your take on AI ready?

Saurabh Kapoor

>> All right. When it comes to AI, we've noticed three common things that are across the customers. They're looking for time for AI, as you said, how quickly they're able to get these infrastructures up and humming. Time for the first token. Time for first training. The second element is trust. They want partner technologies solutions that are validated at scale so that they can do watchings and repeat and move quickly in terms of delivering those end outcomes and use cases. Third is ease of management, right? You mentioned some of the software frameworks, the capabilities that we bring in with Nvidia. So bringing all of those capabilities together, Dell AI factory with Nvidia is all about bringing the right infrastructure, creating an ecosystem of software solutions, the right partnerships there. Bringing the data capabilities, the proficiency that we've built over the years from protecting, managing, securing data, bringing that AI security element. Building services around it from day zero to day two with managed services and then boiling it down to bunch of outcomes and use cases so that it drives to a certain output.

John Furrier

>> So talk about the life cycle and management side of it, because one theme this week that's come up multiple times, Jackie and I and Dave were talking many folks, is GPU starvation, okay? GPUs have to be overclocked. All the best deployments are clocking them close to 90% plus utilization. Most are around 60 to 70, some say lower. That's probably accurate, say 65. So maximizing the GPUs is a huge thing. So you need telemetry, you need to have technology, whether it's AI or management. What's your take on maximizing the GPUs?

Saurabh Kapoor

>> You're spot on there. In order to maximize your proficiency and efficiency on compute, you need to make sure that networking is done right. And the analogy I take is for core infrastructures, let's suppose you're operating at 10 miles per hour. You hit a speed bump, you'll manage it. But when you're operating at 100 miles per hour, you hit a speed bump. It's catastrophic, right? So that is the element when we look at AI infrastructures. Because you're running at peak capacity, you need highly optimized infrastructures and next-gen fabric automation and observability capabilities so that you're looking at predictive capabilities instead of the standard SNMPs and syslogs that are more reactive way of managing and monitoring your fabric. You're looking at more predictive way of managing and monitoring those fabrics. By running AI ops, you're running that correlation across what is my performance on compute? What is my correlation across networking, bandwidth utilization, making predictive actions so that you're better managing it. The other element is optics, right? Bringing in predictive optics, health monitoring, making sure the right connectivity, conflict validation, connectivity, validation, and making sure that all of that is humming as expected.

Jackie McGuire

>> I wanted to ask about, bringing it back to networking. You had just mentioned fabric and Dell's been pretty pragmatic about the fact that customers will have kind of a mishmash of different networking. So you have a technology to deal with that, this Smart Fabric. Can you just talk briefly about what that is and how it works and how it helps customers who don't have the perfect single networking tool for everything, how it helps them bring all of those together?

Saurabh Kapoor

>> So what we've done with Smart Fabric Manager is create validated blueprints, right? So as I said, time for AI, how do you drive the first token, get these infrastructures up and running quickly. With Smart Fabric Manager, we have gone with a blueprint-based model where you capture a certain intent. What does that look like from an outcome perspective? We package that in a blueprint that customers can take and build those infrastructures out quickly. Then drive it to the next level of management, lifecycle management. You're rolling out new releases, new capabilities, quickly upgrading those infrastructures without any data plane downtime. How do you manage and monitor those capabilities? And not just networking, as you said, fabric is all about end-to-end connectivity from GPUs to the NIC all the way to the ASIC so that you are able to watch and monitor the telemetry flow, the packet flow, take corrective actions, get better at day-to-day operations.

John Furrier

>> So networking is fundamental for AI. Okay, we check the box on that. But you guys are trying to come out with this notion of unified, open and automated networking. That's the holy grail. It's complex. So the question I have is, how are you guys simplifying it so that people can stand these factories up, configure them? What's the-

Saurabh Kapoor

>> That's how Michael started the company. Choice and flexibility is the North Star vision for all our solutions and product capabilities, right?

Jackie McGuire

>> A like Little bit of viral marketing, right?

John Furrier

>> But the unified make it open and automated. That's really hard. So there's a lot of complexity to be struck out in that tagline. What are you guys doing specifically to make it simpler?

Saurabh Kapoor

>> SONiC. So SONiC with its architecture, the switch abstraction interface, our ability to run a common operating system across multiple silicon architectures with our partner solutions helps us get to that choice and flexibility. So we have a stack that is running on Broadcom. We added capabilities to it. We added feature functionality that was specific to the Nvidia cloud partner architectures, enterprise reference architectures, expanded those capabilities, creating a unified software stack that the customers are able to run on multi-silicon ethernet architectures. Now this gives them choice and flexibility in how they want to build those reference architectures. They're able to get the platform choice on how they want to support those infrastructures. Connectivity across any GPU, any NIC with SONiC based a unified architecture. That's how Microsoft started their ecosystem with Azure services. That's how some of the largest cloud AI platforms are doing it, and we are now making it accessible and available for everybody else.

Jackie McGuire

>> I think services can be understated. So I want to say I had put in my 2025 predictions when we got together to do our research stuff at the end of 2024, services is going to be one of the fastest growing TAMs because nobody wants to hire a whole bunch of people to get this out the door and then have to let them go or not have enough work for them.

John Furrier

>> Well, so that prediction, what's it being added to your prediction? But by the way, congratulations, you're right. Of course you're always right. But the services, a software trend is coming. So this is a super important because now we're talking about AI services in here that are being handled by AI. So the question is, I'm a customer. What's in it for me? What do have to do first? Take advantage of SONiC and Nvidia AI factory. What do I need to do?

Saurabh Kapoor

>> 1800 Dell, just call us.

John Furrier

>> It's too easy. Expand. I mean, it's in the factory. So what? Just buy a Dell factory. What if I had Dell already? I got PowerEdge, I got PowerScale, I got some pre-existing Dell equipment.

Saurabh Kapoor

>> Absolutely. So we are right there with the customers working hand in glove, making sure they're successful in the AI journey. We've built these validated reference architectures as part of different GPU solutions. We continue to jump in Dell AI factory with Nvidia. You spoke about services earlier. We realize and recognize that AI skill set is in a short supply, especially when it comes to enterprise market. So Dell brings in the broadest portfolio from a services' perspective, day zero to day two, and then manage services capabilities from infrastructure assessment to understanding what those workloads mean for the end outcomes, what those use cases look like. Validating and making sure they're ready for production environment, providing 24/7 support globally across the remotest locations on the planet, and then managed services for customers who need it.

John Furrier

>> Okay.

Jackie McGuire

>> And I'm going to cheat a little bit. There's a whole bunch of this information at dell.com/Nvidia-AI. So I happen to be there earlier today.

John Furrier

>> That's a good... Thanks for that.

Jackie McGuire

>> It's a great place to start.

John Furrier

>> Great Plug. Jackie just gave the coordinates. I want to just ask one pointed question around availability. What's shipping? Just give us through the stats on status of the product. Is it shipping, availability? Just got the link from Jackie on how to get more information, status of the open source piece. Give us some stats on the status of where things are now.

Saurabh Kapoor

>> Absolutely. So while we have active customers, customers who have SONiC in production with the current ethernet partner, we made the announcement this week about Dell Enterprise SONiC, now running on Spectrum-X solutions. We have active customer proof of concepts going on, and in a couple of months we are looking to make it generally available as part of the Dell AI factory offer. We have validated these technologies solutions, SONiC across Dell AI Factory, Solution Labs, a bunch of customers running proof of concepts, different skills and sizes from AI training to enfencing, fine-tuning use cases. We have thousands of GPUs connected to these infrastructures. So we're really excited. I mean, we have great results coming out from the lab, the customer proof of concepts, and we're just ready to make it generally available across the board,

John Furrier

>> It's hot. It's a make or break piece of the infrastructure for AI. As these systems get larger, robust lossless network fabrics are table stakes. So thanks for sharing and congratulations. You're in a hot area.

Saurabh Kapoor

>> Thank you for-

John Furrier

>> Networking's cool again. They never left.

Saurabh Kapoor

>> Networking is cool again.

John Furrier

>> Of course, Jackie and I love networking. Of course, theCUBE's doing our part to network you with our community, and I'll share that data at high speed here in the live stream. I'm John Furrier, Jackie McGuire. Thanks for watching.