VMware Explore 2025 | Ram Velaga, Broadcom

Clips
More from VMware Explore 2025

Ram Velaga

SVP & GM

Broadcom

How Broadcom’s new releases weave powerful networks between data centers

Networking devices are the lifeblood that keeps artificial intelligence models running smoothly and without interruption.As AI eats up more space in data centers, Broadcom Inc. has been quick to innovate, releasing a series of products designed to facilitate efficient, low-latency networking.Broadcom’s Ram Velaga talks with theCUBE about networking for AI.“Machine learning is a massive distributed computing system,” said Ram Velaga (pictured), senior vice president and general manager at Broadcom. “What connects all of this together is the network

play_circle_outline Broadcom's VMware Explore: Discussion on enterprise, cloud, and networking innovations.

play_circle_outline Enhancing Machine Learning Efficiency: Broadcom's Approach to Scaling Networking in Distributed Computing Systems

play_circle_outline Tomahawk six device offers 100 terabits of networking, doubling Tomahawk five's bandwidth.

play_circle_outline Tomahawk Ultra switch achieves ultra-low latency at 250 nanoseconds, enabling efficient data movement.

play_circle_outline Network congestion management simplified with higher bandwidth and efficient data center designs.

Info
Transcript

Ram Velaga, Broadcom

Ram Velaga

SVP & GM Broadcom

In this VMware Explore 2025 conversation, Ram Velaga, senior vice president and general manager of Broadcom’s Core Switching Group, joins theCUBE’s John Furrier to dive into the rapid evolution of networking for AI-driven infrastructure. Velaga highlights Broadcom’s latest chip innovations, including Tomahawk 6, Tomahawk Ultra and Jericho 4, that are redefining speed, latency and scalability across racks and data centers.

The discussion unpacks how Broadcom is tackling the distributed computing challenge of AI workloads by ensuring networks never bec... Read more

explore Keep Exploring

What are some key topics and highlights discussed at Broadcom's VMware Explore event? add

What are the key considerations and recent developments in scaling networks for machine learning systems involving multiple XPUs and GPUs? add

What are the advancements and benefits of the Tomahawk six networking device compared to the Tomahawk five? add

What is the significance of the Tomahawk Ultra device in terms of latency and performance in Ethernet switches? add

What is the improvement in the maximum number of GPUs that can be connected using Tomahawk six compared to earlier technology, and what factors influence the scalability of this connection? add

bolt Powered by CUBE AI

Ram Velaga, Broadcom

search

>> Welcome back everyone to theCUBE. We are in Las Vegas for Broadcom's VMware Explore, of course, it's where the industry's coming together. A lot of on-prem, enterprise, cloud discussion, and of course, you can't have large-scale discussions without large-scale technology. Ram Velaga, senior vice president, general manager of the core switching group at Broadcom. Ram, great to see you. Thanks for coming on theCUBE. Here in the VMware camp at their show, I was commenting on our opening and keynote review. Obviously, the business performance on VMware's phenomenal, subscription, profits up, customers are going cloud, maintaining their states. But if you're going to have a cloud, you've got to have networking, we've talked about that a lot here on theCUBE. But of course, you had the big launch this past summer, Tomahawk six, more innovations coming. I'm hearing rumblings of some other stuff coming out in the fall. Of course, you've got SC25, the big show, Open Compute Project event, their summit, tons of events going on all over the industry in semi.

Ram Velaga

>> Yes.

>> What's the uptake? Give us a quick taste of what's life like in the fast lane.

Ram Velaga

>> Well, number one, thank you for hosting us. What's life like in the fast lane is actually amazing. I'd say a couple of years ago, when we came out with our Tomahawk five product, most customers were like, "Hey, can you slow down a little bit? We're still digesting your previous version of the chip," which was Tomahawk four. And then now, we are here with Tomahawk six, roughly 24 months after we came out with Tomahawk five, and we have a long list of customers knocking on our door every day saying, "Hey, when can I get more parts?" So it's exciting times, you're running very, very fast, very cool technologies, great products being built, and more importantly, it's fun to actually see these products getting pulled by the customers and getting into production, so exciting.

>> Yeah, a lot of innovation on the AI infrastructure. What's the latest innovation? I know you've got some chips, always have the latest and greatest props. First, walk us through the latest innovations from Broadcom on the networking side. Can you share the highlights?

Ram Velaga

>> Yeah, look, I think the important thing is the realization that machine learning is a massive distributed computing system, and when it's a massive distributed computing system, you're connecting many of these XPUs or GPUs together, and what connects all of this together is the network. And the important thing about the network is it cannot be the bottleneck, the network has to move the data very quickly between these XPUs and GPUs. So what we are doing is focusing on how do you scale this network to go as fast as it can so that you can get the most out of your investments in these GPUs and XPUs. And so, what that means is you say, okay, how do you get the network that's going to be superfast, and when you think about this massive distributed computing system, you think of it as what's inside a rack, what's across many racks, and oftentimes, this system does not fit inside one data center, you have to go across data centers. So the last three months, what we've announced is products that essentially allow for networking inside a rack and allow for networking outside of a rack, connecting many racks, called scale-out, and then actually connecting multiple different data centers, called scale-out across data centers. So that's the three products that we have launched this summer.

>> Yeah, the scale-out, obviously, to me, it's a simple term, but scale across data centers has become big. Co-packaged optics, which two years ago, I was at your headquarters, Charlie showed us some of the co-packaged optics on the photonics, because that now has been shipped. Obviously, you shipped, I think, two versions, maybe two versions of that. So latency's huge. I know you guys have got to cover it inside the rack and connecting racks, but we're bounded by power, so I have to then go across. What is the latency concerns? I can't get a good answer on the question of, okay, what's the distance on that, can you share the thinking around what's the latest support? Is optics the big part of that, is the CPO piece? How do you do the scale across the data centers, how does that work?

Ram Velaga

>> Yeah. Look, when you look at scale-out across data centers, you may be looking at buildings which are maybe a kilometer apart and sometimes 50 kilometers apart or 100 kilometers apart inside a metro region. So when you're looking at, let's say, data centers inside a metro region about 100 kilometers apart, it's not the optics or the speed of light transmission constrained by any of this optic, that's not the issue. The issue here is now when you have this distance, you need to be able to buffer traffic across these 100 kilometer distances, so that if you have any congestion or if you have any packet drops across these 100 kilometer separated data centers, you're able to retransmit from the switches and not have to go back all the way into your host into the GPU and pull the data down again and retransmit. So what is needed is deep buffering to be able to handle traffic latency that is up to 100 kilometers in distance, and being able to actually, if there is a packet drop, being able to retransmit it without going to the host. That way, you're able to actually get the performance you need when you have these wider distances separating this data centers.

>> So the switches are doing the work?

Ram Velaga

>> Exactly.

>> All right, all right. So give us the latest update on what you've got there in front of you. Is that Tomahawk six? What are some of those chips? Are they Ultra? What is that, Jericho? I always forget, show me what you've got there.

Ram Velaga

>> No, no. Look, I think you know these names more than most people outside of people who work in my division know, so congratulations. So maybe I can walk you through a little bit of chronological order in what we released this summer. So this is Tomahawk six, and this device is 100 terabits of networking. This is twice the bandwidth of our Tomahawk five device, which is very widely deployed in AI deployments all over the world. And what Tomahawk six does is it's not just doubling the bandwidth of a Tomahawk five, every time you double the bandwidth in networking, you reduce six switches, so that's a huge reduction-

>> Say that again, slow down. You reduce how many switches?

Ram Velaga

>> Six. So that math seems very counterintuitive, when you double, how do you reduce six? But believe me, to get the same throughput out of a device that's half the bandwidth of the most advanced device, you need to be able to connect six of them to be able to get that bandwidth. Now, when you connect six of them, you're taking a lot more power, it is a lot of latency, because you have multiple hops that you have to go through on the switch. Oftentimes, they're connected by optics. Optics cost money, they cost power, and they're not necessarily the most reliable things around. So all of this actually even makes the network less reliable. So by doubling the bandwidth, we are effectively making the network so much more simpler and higher performance. And this Tomahawk six is used for essentially scaling out across racks, but customers also use it for scale-up, which is essentially sitting inside the rack. Then what we released was our Tomahawk Ultra device. This is a 50 terabit switch, but the focus of this switch was to get to ultra-low latency. So nobody actually ever imagined that you could take an Ethernet switch and switch packets at less than 300 nanoseconds, or even 250 nanoseconds latency that we are doing here. And usually, people assume, they have a misconception that Ethernet means it's going to be 500 to 800 nanoseconds or even higher. So we did this with the whole purpose of inside the rack, when you have GPUs that are sharing the HPM memory across different GPUs, they want the data to be able to move as efficiently as possible and with little latency as possible.

>> You're saying it's 250 nanoseconds?

Ram Velaga

>> Nanoseconds. That is faster than somebody can blink or even think.

>> Yeah, that's amazing.

Ram Velaga

>> When you actually think about the fact that it's faster than sometimes you can even think or even blink, that's how quickly you are switching 50 terabits of bandwidth.

>> It's a huge amount.

Ram Velaga

>> It's a massive amount. And what is this doing? What it's doing is people otherwise would be looking at proprietary technologies inside the rack, something like an NVLink, and thinking that's the only way they could actually do a scale-up inside a rack. This says, no, you can actually do it with Ethernet. And the beauty of Ethernet is it's an open ecosystem, and you're going to have multiple different vendors and multiple different players that are going to be providing a solution in this space.

>> And when did that ship, just to clarify, when did that ship, come out?

Ram Velaga

>> Forgive me here, but I believe sometime in July is when this product was shipping, whereas Tomahawk six started to ship end of May.

>> Got it, okay, got it. So it's out there.

Ram Velaga

>> Yeah.

>> So OCP's going be a hot show for you, the Open Compute Project's flagship event, that's where all the people are making calls on how to design their data centers.

Ram Velaga

>> Yes.

>> So they're going to want the Ultra-

Ram Velaga

>> Absolutely....

>> close to all the action, and then Tomahawk six to make the connective tissue between all those racks.

Ram Velaga

>> Between the racks, exactly.

>> And up the racks, across the racks.

Ram Velaga

>> Exactly.

>> All right. So when you go scale-up, what does Tomahawk six enable for scale-up, what kind of efficiency? Does it reduce hops? Does it reduce devices? What's the impact? Because I get the scale-up, that's phenomenal. Take some stuff out and more capability, checks the box, good job. Now, what's the scale-up?

Ram Velaga

>> Yeah. So look, when you do a scale-up, there are scale-up racks being created for inference and there's scale-up racks being created for training. Today, when you think about the size of the scale-up domain, it's essentially constrained by what can I fit inside a physical rack because of the distance limitations of copper, and number two, what's the switch that is available for me that can connect the maximum number of GPUs or XPUs. So today, if you look at a technology like NVLink, the maximum number of GPUs I can connect it to is 72, because that is the Radix and the bandwidth of that particular switch. So what we are able to do with Tomahawk six is you're able to connect 512 XPUs or GPUs, each at a 200 gig speed. So now, people look at it and say, "Okay, I'm no longer constrained by a switch on how big a scale-up rack can be. Now, I'm really only constrained by... Do I need to do copper or do I need to do optics? Because if I can get away from one physical rack and essentially run optics and I can put a Tomahawk six device, my scale-up goes from 72 to 512." And that's a mind-boggling improvement in performance, where you have a larger domain of GPUs that are able to essentially work on a workload and share the memory across that.

>> That's where the design decisions come in.

Ram Velaga

>> Absolutely.

>> It's literally copper versus optics.

Ram Velaga

>> Exactly.

>> And that's based on what the situation is.

Ram Velaga

>> Correct.

>> The footprint, what the physical place... Okay, great, cool. All right, now the last chip, what is that, last one?

Ram Velaga

>> Yeah. So now, we've talked about what you do inside a rack, you can use either a Tomahawk Ultra or if you're focused on a Radix, you could use a Tomahawk six inside a rack. But obviously, you will also use Tomahawk six to do scale-out. But then, if you actually look at the data centers, while everybody talks about they're building a gigawatt data center today, maybe two, four gigawatt data centers in the future, the reality is there is also still a lot of existing data centers which are 10, 20, 40 megawatt data centers, 60 megawatt data centers, even 100 megawatt data centers, which are generally within the proximity of a few kilometers, or sometimes even within the proximity of 100 kilometers to each other. So the idea is, hey, why not be able to connect this scale-out across these physical buildings? So when you connect across these physical buildings, now you're stretching the distance on which this traffic is going. You're going, instead of a kilometer, to many kilometers, up to 100 plus kilometers, and that's where you have to deal with things like how do you actually buffer for these longer distances, that's one. Two, you also have to start thinking about, okay, now I'm actually leaving the fibers, leaving the building, do I have to think about security so that somebody just doesn't go and tap into this fiber and now they have access to the information on this fiber? So that's where our Jericho four device comes into play. And what you can see in this Jericho four device is it actually has four HBMs here, you can see, one, two, three, four HBMs sitting here, and that is able to do very deep buffering, so that when you're connecting across these data centers, when there is temporary congestion happening, it's able to buffer the traffic so that it can prevent drops and also prevent having to go back to the host and cause retransmits. Not only that, it does full-speed MACsec encryption. So between buildings, all the traffic is encrypted and you don't have to worry about the security of the data that's leaving these buildings. So now, when you actually-

>> What's the throughput on the Jericho?

Ram Velaga

>> Yeah. So-

>> Because encryption has an overhead. Some people, they want to encrypt everything, but sometimes I've heard I can't encrypt everything because I don't have the storage or the bandwidth. Does that come up?

Ram Velaga

>> Yeah,.so the good thing about this device is we are able to do line rate encryption. So there is no bandwidth impact for the encryption that we are doing. So whether you turn encryption on or off, you get the full throughput of this device. As a standalone device, it's a 50 terabyte router, we call it the AI fabric router. But the important part about the Jericho class of devices are you can actually have many of these pulled together in a system that creates a very large fabric, that is a very massive system, that is thousands of 3.2 terabyte ports is what you're able to actually aggregate together. So the magnitude of this is very, very large.

>> That's good also on the distance piece.

Ram Velaga

>> Exactly.

>> That's the key one for across.

Ram Velaga

>> Across the data centers.

>> You mentioned 100 kilometers, was that a random number or is that-

Ram Velaga

>> No, that is actually the spec, which is in respect to if you are separated by distances as far as 100 kilometers-

>> That's a long distance....

Ram Velaga

>> we understand exactly how long it takes for light to travel that distance. Being able to actually also be aware of the fact that it's a round trip time of the traffic going from one data center to the other, and then being able to actually know that you've acknowledged the receiving of the traffic, and then being able to buffer for that distance.

>> Ram, I've got to ask you, first of all, thanks for the master, it's like a cooking class, you can use this for that, you put a little spice on it, it feels like we're baking a nice .

Ram Velaga

>> There's method to this madness.

>> Okay. So a lot of the dynamics in today's market is footprint. I've got a colo, I need a colo, the demand for space and power, we talked about that before. This is where the builders are starting to have to be really clever. Can you share insight into what you're seeing on a lot of the large scale-up and out across, because there's almost like... I won't say spot building, but like, oh, I got my hands on some space, I need to now build that rack. So then they realize, okay, it's a systems design, they've got to factor in what's over there. So I'm imagining that having Jericho and Tomahawk six and Ultra in there will affect the design of criteria selecting these locations. Did I get that right or am I too...

Ram Velaga

>> Yeah, no, I would say that, to a certain extent, they don't necessarily, at least in the context of a Tomahawk Ultra and a Tomahawk six, they don't necessarily affect the location of the space that they-

>> In the racks.

Ram Velaga

>> Yeah, those are very rack-specific. Generally, the spaces that they pick is more a function of how many GPUs that they want to be able to put in that building. Now, the good thing about this is, to your point, where it does affect it is now they can look at something like a Tomahawk six and say, "In a two tier network..." Because when you build a two tier network with a three tier or a four tier, the network congestion is done much more simpler and the traffic is handled much more efficiently. So when they try to build an efficient data center, then they say, "Okay, you know what? I can put a lot more GPUs inside a two tier network, which means if I can put close to 100,000-plus GPUs, then this is the size of the data center I need to build." So to a certain point, it does extend it, saying, "Okay, look, I can build such a large cluster with the two tier networks, so how large should my shell be?" Now, beyond that, then they tend to also look at where do I have some space available, what can I fit inside that space, now how do I connect a data center A to a data center B, where we are coming in saying, "Okay, you already have an existing problem by virtue of existing data centers, we can solve those problems with Jericho four. So it's both going on.

>> It's both going on, yeah. All right, so can you share what's next? I know it's going to be a very hot and heavy fall season, you've got Open Compute Summit, which is going to be where the builders are, and that's really a nerd fest, people are going to be geeking out, and you've got the speeds, you're living in the fast lane, as I say. But then, you've got Supercomputing, and these are big shows. Any teasers on the announcements coming? I tried to get some data last night from you team. They were all good, they didn't share a word. I'm sensing there's more. Can you give a little teaser?

Ram Velaga

>> Yeah. Look, it's not necessarily a teaser, you know this by now, I think the biggest cusp of what is going to start happening now and in the next couple of years is... To date, you had one vendor, one solution that everybody had to build towards, GPUs from one vendor. What you're starting to see, and I think there is going to be a wave of this that's going to flow through in the next couple of years, is there's a diversification of these XPUs that you're going to see, either those that are being built in-house or new vendors coming into the game and so on and so forth, and all of these are very, very credible machines. By virtue of it, what you're going to see is the demand on networking that allows for a heterogeneity of these XPUs and GPUs is immense. So everything that you're seeing here, products that you're seeing now, is all enabling that, and as you can imagine, there's more and more pieces that are needed to bring all of this solution together, whether it is on the optics front, whether it's the continued switching front, routing front, or the XPUs itself. So I don't think there'll be lack of excitement for the next 18 to 24 months, so I will leave it at that.

>> Talk about open, let's close on some of the open systems that are going on. I know you're a big proponent, obviously Ethernet, you could talk for an hour, probably just riff on that. I was talking with a entrepreneur team, I won't say the name of the company, but they're building a level four autonomous vehicle from the ground up. And so, we talk about heterogeneous versus homogeneous in systems, and there, they say, "Well, we are not connecting to the cloud." So they're an on-car-prem, it's not on-premise, it's the car, the car is, for privacy reasons-

Ram Velaga

>> The machine, yeah....

>> all the data is on the car. So okay, I guess that's homogeneous in the sense of it's on their thing, but it still will want to use parts. So I guess the question is, in that scenario, they would probably need fast networking, because everything's super fast, it's got everything going on. In that case, it's a hardened system. It might want to connect to a network at some point. So then, where does the heterogeneous and then self-contained, in your mind, fit and build? Because I can see people building more and more systems that have purpose-built, so you've got purpose-built systems and you've got just general systems, what's your just thoughts and reaction to that? Because people tend to react to... I know I do, when I hear homogeneous, I think, oh, proprietary, that's just a knee-jerk reaction for me. But sometimes, it might be good to be just in your little-

Ram Velaga

>> Yeah, island. Look, I think there is a place for both. Homogeneous systems work where there's limited scale, and when you look at this car, the car on its own is a limited scale and it's perfectly fine for it to exist on its own. But a heterogeneous system is if you have a massive fleet of these cars and all of them have to coordinate with each other, then it's no longer... I would draw the analogy, it's no longer a PC. See, today, a PC is a homogeneous system, it can be self-contained too on its own. But if you want to build the internet, which is a distributed system, you have to connect this together, and the only way you can do it is it has to be heterogeneous, it has to be a set of open APIs, it has to be where anybody can build it, so that tomorrow, if you don't build it, somebody else can come and take your place. So I think the analogy that you have drawn, I would say the car on its own is a homogeneous system, but the car, for it to be able to work across a multitude of its cars, I bet you it is actually a heterogeneous-

>> Yeah, I think scale's a great point.

Ram Velaga

>> At scale, everything is a heterogeneous system.

>> Ram, great to have you on, great to see you. And again, congratulations, you've got a great set of products there. I'm sure the customers are lining up to build the solutions, taking that stuff out of the network, adding more capabilities. Great performance. Thanks for coming on.

Ram Velaga

>> Well, thank you for hosting us, and thank you for helping us explain what we are building here to the world here. Thanks a lot.

>> Yeah, the world needs to understand how the hardware works. Of course, the fall will be chock-full of action, when you start hearing about Open Compute Project Summit, you also have Supercomputing, we're going to be at re:Invent, you have Mobile World Congress right around the corner. This is going to be a massive build-out of next generation technology for AI generation, the apps are coming, the infrastructure's moving very, very fast. Of course, theCUBE's doing its part to keep track and bring that action to you. I'm John Furrier, thanks for watching.