In this interview from theCUBE + NYSE Wired: AI Factories, Will Chu, executive vice president and general manager of the Custom Cloud Solutions Business Unit at Marvell, joins David Lazovsky, executive vice president and general manager of the Data Center Networking Group at Marvell, to talk with theCUBE's John Furrier about how the shift from copper to optical interconnects and custom silicon is reshaping AI data center economics. Chu traces Marvell's decade-long bet on leading-edge process technology, which skipped the 7-nanometer node entirely to reach 5-nanometer and beyond, as the foundation for the company's current growth. Lazovsky explains how the KV cache bottleneck in large language models is pushing scale-up networks beyond the rack, requiring a shift from copper to optics to hit the bandwidth and latency needed for GPU-to-GPU memory sharing.
The conversation also explores Marvell's Photonic Fabric Memory Module, an optically interconnected memory module that decouples capacity from compute and enables memory farms positioned up to 50 meters from the processor. Chu unpacks why hyperscalers are increasingly turning to custom silicon to capture better performance-per-dollar economics across a market with more than $1 trillion in cumulative spending. Lazovsky details the broader economic shift underway, noting inference now accounts for more than 60% of data center infrastructure builds, up from a third just three years ago, as hyperscaler operating margins climb toward 40%. From a monolithic 100T switch delivering industry-leading latency and power efficiency to a $1.2 trillion annual AI infrastructure buildout, Chu and Lazovsky outline why Marvell's connectivity-first strategy sits at the center of the AI factory economy.
Almost there!
We just sent you a verification email. Please verify your account to gain access to
AI Factories - Data Centers of the Future. If you don’t think you received an email check your
spam folder.
Sign in to AI Factories - Data Centers of the Future.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for AI Factories - Data Centers of the Future
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for AI Factories - Data Centers of the Future.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Will Chu & David Lazovsky, Marvell
David Lazovsky of Marvell, executive vice president and general manager of the Data Center Networking Business Group, and Will Chu of Marvell, executive vice president and general manager of the Custom Cloud Solutions business unit, join theCUBE Research for a conversation produced for NYSE Wired: AI Factories — Data Centers of the Future.
The discussion examines how Marvell's connectivity-first strategy shapes next-generation artificial intelligence, AI infrastructure and data center design. Lazovsky emphasizes optics and scale-up networks to extend shared memory beyond racks; they describe how co-packaged optics, photonics and scale-up networking enable larger memory pools and lower latency. Chu highlights the node-led chip roadmap and platform leverage; they explain that skipping process nodes accelerates performance and serializer-deserializer, SerDes bandwidth. The conversation also addresses high bandwidth memory, HBM and disaggregated memory, Marvell's integrated platform approach and custom silicon investments that support hyperscaler economics and AI factory deployments. Analysts note potential 30-50% power reductions from optical SerDes and underscore the market shift toward inference-driven infrastructure that informs Marvell's updated revenue guidance.
This session provides technical and strategic insights for professionals focused on AI infrastructure, data center networking, optics and custom silicon design, with practical considerations for deployment, power efficiency and scalability.
EVP & GM, Data Center Networking Business GroupMarvell
In this interview from theCUBE + NYSE Wired: AI Factories, Will Chu, executive vice president and general manager of the Custom Cloud Solutions Business Unit at Marvell, joins David Lazovsky, executive vice president and general manager of the Data Center Networking Group at Marvell, to talk with theCUBE's John Furrier about how the shift from copper to optical interconnects and custom silicon is reshaping AI data center economics. Chu traces Marvell's decade-long bet on leading-edge process technology, which skipped the 7-nanometer node entirely to reach 5-n...Read more
exploreKeep Exploring
What is driving Marvell's recent strong business performance and the stock's surge following its Investor Day?add
How did the acquisition of Celestial AI by Marvell affect your role, and how does Celestial’s technology fit into Marvell’s strategy for AI data‑center infrastructure?add
Why did the company skip the 7 nm process node and move directly to 5 nm (and subsequently to 3 nm, 2 nm, 14 Å) for its data‑infrastructure chips?add
How do you define "scale-up" versus "scale-out," and what interconnect and chip innovations (for example, higher-speed SerDes, optics, near‑package or co‑packaged optics) are required to extend scale‑up from a single rack to multi‑rack systems while meeting the necessary bandwidth, latency, and power requirements?add
How much of the U.S. grid do data centers consume now and by 2030, why is moving from copper to optical interconnects important for power efficiency in scale-up (processor-to-processor) networks, and what energy savings can optical/analog SerDes technologies achieve per memory transaction?add
Is it true that networking underpins everything Marvell builds?add
What is this product and how does it affect performance, AI quality, and Marvell's business prospects?add
>> Hello, I'm John Furrier, host of theCUBE. We are here at theCUBE's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. This is part of our NYSE Wired program and open community. It's our AI Factory series where we talk to the leaders who are making it happen. And today we have two leaders here and of course their companies doing incredibly well today in the stock market as they revise their numbers. Dave Lazovsky is EVP and general manager of the data center networking group at Marvell. And Will Chu, executive vice president and general manager of custom cloud solutions business unit. These guys are responsible for a lot of the massive growth going on. And of course, the product mix is great. Guys, great to see you. Thanks for coming back in. CUBE alumni, welcome to theCUBE.
David Lazovsky
>> Thanks for having me.
John Furrier
>> Okay, so guys, big news today. You're up 15 points on the market for your stock. $286 is probably going to close in about 20 minutes. Probably hear the bell here ring. Incredible pop this morning because, you guys had your Investor Day here in Manhattan and quite the performance. This kind of 10 years of vision kind of coming together. I've been to a lot of investor meetings, and when you have everything clicking, you can see it. It's like watching sports. The team's doing great. Take us through this because you got some product we'll get to in a minute, but the business performance is a lot of hard work. But what's the driver to this? Why is this coming together so well? Will, we'll start with you.
Will Chu
>> Yeah, thanks. Thanks for having me. So yeah, I've been at Marvell for 9 years, so I've seen the journey. And we talked about it today at the Investor Day. It's been a journey over 10 years. So back in 2017, we had a mission to be a leader in data infrastructure. And at the time, nobody knew what that was, right? To move, process, store, and secure the world's data with really high-quality chips. And for the past 10 years, we've been investing heavily to make it happen. And so today, I think we've been giving investors the guidance along the way, but today we really upped our numbers because it's all coming together at Marvell. We're firing on all cylinders, right? So we have the custom business, which is mine. We have Dave's data center networking business and we have our connectivity business. Those are all just clicking and we're going to continue to grow market share. And we guided to $8 billion at the midpoint for our calendar '30 or fiscal '31.
John Furrier
>> And Matt Murphy, the CEO, also talked about over that time about, what, $20 billion invested. Was it 20 or 40.
David Lazovsky
>> 40.
John Furrier
>> 40.
David Lazovsky
>> Yeah. 20 organically.
Will Chu
>> 20 organically. And then 20 through— $20 billion through M&A.
John Furrier
>> So, Dave, you're part of the $20 billion, Celestial AI. Last time you were on theCUBE, you were running that company. Now Marvell. You fit into the puzzle because these systems aren't just separate disparate components cobbled together. They're engineered for these AI factories. We're seeing that growth obviously on the infrastructure, but these data centers are becoming industrial intelligence-producing systems.
David Lazovsky
>> Yeah, that's—
John Furrier
>> and you're in the middle of it with the optics and you got the custom. These are the two hottest areas. Custom might have been a bad word maybe 10 years ago. I think merchant silicon was a term. You guys addressed that. The photonics, the integrated chips. This is where the innovation is getting strong. So obviously the stock price is up since they bought you too. So since that acquisition, it just seems that everything's kicking into gear.
David Lazovsky
>> Yes, well said. Marvell is firing on all cylinders. So I did join Marvell with the acquisition of Celestial AI in February of this year. I was previously one of the co-founders and CEO of Celestial. So it's been an amazing journey. Yeah, the stock is, I think, appropriately rerated itself given the magnitude of the growth opportunity ahead of us. And I think the Street is—
John Furrier
>> yeah, you had a good talk about the shift And Will, you were talking before you came on camera about the bets at Marvell. Now, if I looked at the product slides, everything's up and to the right on performance. Can you share the story about the big bet? Because I think this is where companies have the moments. Yeah, I don't mean to go back in time, but I think I want— it's instructive.
Will Chu
>> Yeah, no, no problem. And our president, Sandeep Bharathi, talked about it at the Investor Day. So back around 2018, 2019, we made a decision that if we wanted to do data infrastructure, we needed to be a leader and not a follower. And that meant jumping to the latest process node with the best technology. So we went ahead and we did that. And what that meant for us is we skipped a node, which is very, very rare in the semiconductor industry. So we skipped 7 nanometer. We had products like in 12 and 16 nanometer, and the next logical one would have been 7. We jumped all the way to 5, right? And we got on the first shuttle with our SerDes, and honestly, then we kept going— 3 nanometer, 2 nanometer, 14 angstrom. And so we've been running that playbook, and it's really served as the foundation for our growth to be a leader in the cloud.
John Furrier
>> And the result is the product performance. We saw doubling on some of the higher-end metrics. Yes, the 1.2 or 1.6 terabits.
Will Chu
>> Yes.
John Furrier
>> To 3.2. That was kind of a step function there.
Will Chu
>> Yes. Well, basically what you see is all the data metrics in terms of performance are doubling a lot faster than they used to. Yeah. And so we're keeping up, if not ahead of that trend with our deep investment base at all these different processes.
John Furrier
>> Yeah. Another thing that jumped out at me is the integrated platform. You get a lot of operating leverage. What— explain that for the folks who don't understand the leverage you get from a platform versus bespoke development, or there's benefits there.
Will Chu
>> Yeah, maybe I'll start at a really high level, and we showed it today. So if you look at our revenue mix, about 75 to 80, 90% is in the data center, right? And so that's already, as a company that's super focused on that, we can invest all of our R&D for those type of applications. We're not really distracted by anything else. So that means our SerDes technology, which is core, is meant for the data center. It's not meant for other markets. So we can leverage that across all our products. And that's what you see and that's what's giving us competitive advantage because then we can continue to invest and keep the flywheel going.
John Furrier
>> Dave, you hit a couple trends in your talk and I like the way you mapped the performance of some of the products to the market. We'll talk about KV cache, how that works. But I think that's instructive. Explain this big shift going on because scale up, scale out, scale up, scale out, scale across has been the buzzword. But a lot of people say, oh, we're good with scale up. No, this scale up still needs some work. Scale out always needs work. This is where the networking piece really becomes the operating model. Of the system. Explain that key trend. It's not obvious.
David Lazovsky
>> Yeah, well, that's right. So with the introduction of vLLM, right, from a research team at Berkeley back in 2023 and the implementation of what came out of vLLM, one of the big elements of innovation was KV cache. So what KV cache is, is it's basically the storing of key and value matrices that prevent the need to go back and recompute these things. So you have the information in rapidly accessible, fast memory that enables humans to interact in a context window in a manner that, deals with the impatience that is inherent to human nature.
John Furrier
>> So now for a user, that means when you have a prompt and you're getting ongoing answers, it remembers everything.
David Lazovsky
>> Yeah, you don't have to wait around.
John Furrier
>> Okay. And then, okay, continue. There's another side of it too.
David Lazovsky
>> Yeah, so, to simplify it, that's exactly what's going on. Now the problem is that that KV cache is not infinite. Ideally, it could be infinite, but it is not. So at some point it fills up and it spills over, and one of two things happens.
John Furrier
>> It gets full.
Will Chu
>> It gets full, yeah.
David Lazovsky
>> So either you have to reload the key and value parameters from a different tier of memory down in the memory hierarchy, which adds latency, it's slower. Or you have to go back and recompute, which prevents— basically, it obviates the benefit of having the KV cache in the first place, again impacting latency. So ideally, what you want is as much memory in that KV cache as possible, unlimited. And the problem is that today's XPUs or GPUs have HBM memory.
John Furrier
>> It's like a sports car and the engine just backfires and not enough stuff happening. It literally is sub-optimized the worst you could think of.
David Lazovsky
>> Yeah.
John Furrier
>> Because the GPU is the expensive main actor.
David Lazovsky
>> That's right. That's right.
John Furrier
>> Yeah. It's not firing on all cylinders either.
David Lazovsky
>> That's right.
John Furrier
>> So the KV cache is super important, but no one saw that kind of volume coming. Well, what's the solution?
David Lazovsky
>> Yeah, the solution to lead to the system level point that you made is that you have to interconnect more and more GPUs or XPUs in a shared memory semantic environment. And that memory semantic environment is called a scale-up network.
Will Chu
>> That's what it is.
David Lazovsky
>> So that scale-up network enables any GPU in that network to access the memory of another GPU as if it was its own, right? And ideally, you want that memory transaction to be as quick as if it was on the package and with the same in terms of both latency and bandwidth. Now it's challenging, right? Because what NVIDIA has done is they've moved with leading technology and scale-up networks, which is NVLink and NVSwitch. To a rack full of interconnected XPUs, which is now 72, but it's limited by the reach of copper, right? So the reach of copper at these data rates, which is about 200 gigabits per second, is 2.5 meters. So you're effectively stuck inside the rack.
John Furrier
>> a lot of people try to figure out why the semiconductors are so hot and why rack scale is so popular, why this AI factory thing is going on. When I talk to my friends and family that aren't in tech, they're like, I don't get what's going on. I go, look, there's a lot going on there, but what you just pointed out was you have density and you have distance. So the old school method was, yeah, you use copper, it's right there, it's in the server, you rack them up, you put some cable. I'm oversimplifying, but that's old rack and stack days. Then you had the density engineering, which get everything close, and then that's one GPU. Then you had, wait, I need more of those, right? That's what you're getting. So you're basically putting this into one system. What you just said, you're treating all these GPUs and subsystems as one thing. Okay, now back to the chips. The density is to get everything close on a chip level and then the distance, which is the rack. So you guys are solving kind of the density-distance problem.
David Lazovsky
>> Yeah, well, to be clear, before I turn it back over to Will, we're actually extending the scale-up network beyond the rack, right? Today, the scale-up networks are limited to this roughly 72 interconnected XPUs you can put in the rack. That needs to grow to closer to north of 500. And the only way to do that is to move from copper interconnectivity for the scale-up network to optics. And that's not just rack to rack, it's copper throughout. It's throughout the racks being shifted to optics and across for full connectivity, rack to rack, multi-rack scale-up devices.
John Furrier
>> We're going to get semantic here for a second. Hold on. We'll get to the chips again, but I want to make sure it's right. Scale-up is the
David Lazovsky
>> rack.
David Lazovsky
>> The scale-up used to be just the
John Furrier
>> rack.>> Okay, used to be the rack. That's where the copper came
David Lazovsky
>> in.
David Lazovsky
>> Yeah, but now we are increasing the scale-up domains to multi-rack.
John Furrier
>> So you're essentially co-opting the old scale-out definition. Is that— Well, you're kind
David Lazovsky
>> of.
David Lazovsky
>> If it's 3, it's one
John Furrier
>> thing.>> It's kind of like scale-up.
David Lazovsky
>> It is, but
John Furrier
>> it's—>> What's the
David Lazovsky
>> difference?
David Lazovsky
>> The difference is this. Scale-up networking requires 10x the bandwidth. You have to have the bandwidth to keep pace with a high-bandwidth memory transaction, which today HBM4, HBM4E, 16 terabits per second. You also need the latency, right, to support a memory transaction as if the processor had the HBM in its own package across the network. So that round-trip latency, the requirement there is on the order of 200 nanoseconds.
John Furrier
>> Okay, so good. So basically you're essentially being specific and intentional on the definition not to be a physical definition, but capability.
David Lazovsky
>> That's right.
John Furrier
>> So latency, functionality, power envelope.
David Lazovsky
>> That's right.
John Furrier
>> So whether it's on 2 racks, who cares? It's the same concept. Scale. Now scale out would be— now back to the chips because now the chips help too. So where's the chip innovation come in to make that happen? Because is it the form factor? Is it the speed?
Will Chu
>> Yeah, there's so many things. Okay. But the first I think is the interfaces or the I/O, which is our SerDes technology. And as mentioned, Everybody used copper up until they can't. So everything's moving to optics because the other thing you didn't mention was the speed. So as Dave mentioned, 224 gig, you can only get about 2.5 meters of distance. That's physics. If you want to go faster, the distance gets even shorter. That's what we call the copper wall, right? So for today, long distances, everybody knows you use fiber optics. Because you can't have high speed go over copper for long distances. Use fiber optics for that. As XPU performance and the performance of all these systems keeps going up, the copper wall is hitting those systems down into the chips themselves. This is where we'll need technologies like near package optics and Co-packaged optics (CPO) because you have to basically get over the laws of physics. And as they would say we're not fighting with the electrons, we're just using the photons. In our favor.
John Furrier
>> Yeah, let's bring up the electric field, and we're getting in the weeds here, but it's important because I think this is a significant advancement, which is why you guys are winning a lot of deals. I can see that electricity and optics are two different things. I was commenting when I was looking at the NVL72 rack and I saw those NICs in there. You're going from optical to electrical. There's all these hops. Yeah, that's right, they're hops, and they're also Transmission changes, so you lose energy potentially.
David Lazovsky
>> Yeah.
John Furrier
>> What's the innovation? So are you replacing electricity? What's the benefit? Yeah. On that, can you explain?
David Lazovsky
>> Absolutely.
Will Chu
>> Yeah.
David Lazovsky
>> So you hit on something that's very, very important, right? So let's put it in the context of what's going on in terms of power, right? So at the end of 2024, going into 2025, the United States total grid power consumption that's being used for data centers, roughly 4.5% of the total US grid. Exiting this decade in 2030, it is projected that the data centers are going to consume more than 13% of the US grid. You're probably more up to speed on this.
John Furrier
>> Yeah, of course. And by the way, it's not a steady grid either.
David Lazovsky
>> Yeah, that's
John Furrier
>> right.>> Fluctuates. So there's problems with power supplies and
David Lazovsky
>> UPSs.
David Lazovsky
>> That's right. So it's a big problem, right? So now, why interconnectivity with scale-up moving from copper to optics is so important is because processor-to-processor data transmission, roughly 85% of total data transmission is happening inside of these scale-up networks, only 15% across the scale-out network. And so if you can positively impact the energy efficiency, you can move the needle significantly. So let's talk about specific numbers, right? So we're developing technologies that leverage optics and SerDes, so serializer/deserializer (SerDes), which are analog, that do not require digital signal processors, that significantly reduce the total power consumption. So we can reduce by 30, 40, in some cases 50%, the amount of power consumed in a round trip memory transaction for scale-up network.
John Furrier
>> That's awesome. Let's get into some of the co-packaged optics. We have some props here. Show us this because this is where I think the innovation is. And back in my old brain of dealing with fiber optics was these special connectors. You had to have all this kind of these connector devices, transceivers going on. Talk about what's in this chip because I think this is very interesting.
David Lazovsky
>> Yeah. So what you're looking at here is what we call a Photonic Fabric Memory Module. So, for the first time ever, Marvell has developed optically interconnected high capacity, high bandwidth memory. There are 2 stacks of HBM and this supports a module with up to 8 stacks of DDR5. So it provides you the ability to decouple memory capacity from compute for the first time. The only stacks that are currently available for HBM outside of this device are located 1.5 millimeters away from a GPU or a CPU or an XPU, right? So this technology fundamentally changes the economics of memory. It also provides the ability, again, to separate memory scaling from compute. We integrate these in appliances where there can be 32, 33 terabytes of memory capacity optically interconnected with 150,000 15 terabytes, a memory farm. Basically, it's disaggregated memory. And that can be up to 50 meters away.
John Furrier
>> Interesting. Disaggregation is an interesting term. You look at disaggregated serving, you mentioned KV cache. One of the things that's coming out of that is disaggregated serving that speaks to a similar trend of disaggregated components functionality. What does that tell us? I think that's the— disaggregated serving just tells me that the systems are being optimized as the problems emerge. What are the next disaggregated engineered systems?
Will Chu
>> Yeah, maybe I can talk. There's a whole memory hierarchy. Okay, so let's start within the chip. So within the chip, at Marvell, we make custom SRAM. And what you want is you want high bandwidth at low power. So we talked about our custom SRAM has 17 times more bandwidth at 2/3 of the standby power. Because what you want, as Dave mentioned, KV cache, you got to store a lot of information and you want to be able to have that information immediately, which means you want it in high performance memory. So you have the SRAM, which is in the chip. You have HBM, which is the next layer. Outside of that, you have DRAM, right? And then there's a lot of products, like we mentioned, like CXL, that enables expansion of the memory that kind of disaggregates it, but it's still at a board level, right? And then you have something super innovative like the Photonic Fabric Memory Module that lets you physically disaggregate the memory outside of the tray completely into another box, right? But all of this is what's needed. Eventually you'll go to flash and even HDD to carry all of the AI information.
John Furrier
>> You know, It's so interesting because you guys use a metric that's set— oh, the bell's going to ring soon. Let's check out your stock price here. Let's go and see where Marvell's going to close out today. The bell rings. If you're watching, you'll hear the bell. There it is, up 15.71, to 86 a share. What are we closing at? It's loud. Well, congratulations. You took a lot of your stock in your acquisition. Most of it was stock, I believe.
David Lazovsky
>> That's right.
John Furrier
>> Congratulations.
David Lazovsky
>> Thank you.
John Furrier
>> All right. So this memory fabric farm, this speaks to this— the word fabric. When I hear fabric, explain this to me, because I hear fabric. I don't hear a chip. I hear software. So I've talked to a variety of developers that are working on photonics. They're not building any hardware. And custom silicon is the other big story coming out of Marvell. Obviously, the optics and the networking stuff, clearly that's the case in the data center, in these data centers that are being built. But custom silicon, I didn't have that on my radar, but the software layers seem to accelerate that. So what's this? What's the importance of custom silicon? Because obviously the biggest hyperscalers you guys are partnering with, Can people program to this? How does this fit in? Because a lot of people can have their hands in the pie on these data center deals. Amazon Web Services, Google, Azure, they've all got some heavy hitters up there. They want to have the best custom. I'm sure they're working on a lot of algorithms, math. Explain the custom silicon wave, why it's important now.
Will Chu
>> Yeah. So maybe at a high level, it's economics, right? So if you look at hyperscaler spending, right, in aggregate, we've had north of $1 trillion, right? And if you step back and you want to get performance for every dollar that you invest, once you have a good idea of what your workloads are, which a lot of these hyperscalers do, They can then use that information to develop customized solutions, in this case, custom silicon, to go drive better economics for their business. They get higher performance, lower power, the whole nine yards. That's the real driver, right, for all this custom silicon. If you step back, the hyperscalers in the early days, were buying off-the-shelf servers. We talked about that. And then what did they do when they started to scale? They made their own servers.
John Furrier
>> Yeah, Nitro. That was Amazon's first major breakthrough.
Will Chu
>> They made their own servers and they started making their own chips. This is in the traditional data center world. Eventually, as we move to AI, that supercharged that desire and the requirements have just gone through the roof because the XPU usage is at the center. That complexity continues to rise very, very fast. And that's pulling in a whole bunch of other things we call XPU attach of other customer semi-custom silicon to support these new, I think that's a great insight, Will, because the economics is the story.
John Furrier
>> If you look at the AI factory, which we've been introducing here at Wall Street, of course Silicon Valley knows a lot about it. It was not understood at first.
David Lazovsky
>> Yeah.
John Furrier
>> And if you look at the AI factories and the work you guys are doing, it's not just a data center either, because you just talked about scale up and scale out I'm going to butcher it. Scale up has changed. These are data centers, plural. They're industrial centers producing intelligence.
David Lazovsky
>> Yeah.
John Furrier
>> So the economics, if you back into what you just said, tokens per watt per second, if you're producing good tokens or good intelligence, the product is good. It's like a car. The car runs well. I want to buy those cars.
Will Chu
>> Yeah, I'll just— quick, in the differentiation they get So they all have their own models and that's requiring a certain kind of hardware so they can differentiate their entire infrastructure, not just let's say on the compute side or on the XPU side or the XPU attached side. It's going to extend eventually to the networking side that Dave's doing as well as the connectivity that all has to work together.
John Furrier
>> Yeah. Talk about this because you're built, you're involved with all the people building out all these things. You've got the chips. Amazon and the hyperscalers are already in the game with cloud. So they're like, obviously, already industrial. But, I gave a talk on sovereignty and I basically said there's no real— it's not a new category. It's subordinate to these AI factories because whether you're an enterprise or a country, you're producing intelligence with data centers, which are industrial centers, they're basically industrial. Industrialization.
David Lazovsky
>> That's—
John Furrier
>> this is huge. This is not just, hey, build the data center, get some real estate. Get some power. This is real engineering, talk about that impact because you're directly relating, connecting that to the money.
David Lazovsky
>> Yeah, well, I think the largest hyperscalers, are some of the largest companies in the world. Right. And they are global. Right. They're located in all regions. And to dovetail into what Will is saying, if you look at the $1.2 trillion in CapEx that will be invested in this calendar year, a large percentage of that is being concentrated in the Big Four, right? So you've got most of the Big Four that will invest over $200 billion a year in their own CapEx for data center infrastructure. And just like in custom XPU, they're also building custom networking solutions.
Will Chu
>> So we're—
David Lazovsky
>> what this allows us to do is because it's so concentrated, we can partner with them and we can work very closely as an extension of the teams of these largest hyperscalers to build system-level networking solutions for artificial intelligence infrastructure that is tailored to exactly
John Furrier
>> what—>> is it a true statement if I said networking underpins all of what Marvell builds?
David Lazovsky
>> The networking— we are a connectivity-first company.
Will Chu
>> Connectivity.
David Lazovsky
>> So networking is a means to that.
John Furrier
>> Okay, so let's riff real time because I was talking to NVIDIA folks, I'm like, I really don't like the word networking because it's just, it's so many different definitions. Cisco could be, they said they could be networking. They're old school networking, but they're not networking like this. But you guys are in the distributed intelligence business. Yeah. It's distributed computing. They're connecting nodes, edge, everything, whatever you want to connect, but you're ultimately moving intelligence around or the ingredients to make the intelligence.
Will Chu
>> Yeah.
John Furrier
>> Well, that's a category that's different than saying, hey, we're pulling a wire and connecting point A to point B and moving packets over.
David Lazovsky
>> I think the custom work that we're doing, and Will can talk to this as well, in both the custom XPUs in connectivity and in networking, including the links, the optics, the switches, all of this is moving toward making artificial intelligence infrastructure more and more economically efficient. And so there's two things that
John Furrier
>> are—>> it's part of the operating model. It's like when I hear networking concepts these days, I feel like I'm in operating systems in computer science class. Yeah. Linking, loading. it's a lot more going on than just transmission.
John Furrier
>> Yeah.>> the dumb pipes, as they used to call it in the old days. That's
David Lazovsky
>> right.
David Lazovsky
>> That's
John Furrier
>> exactly >> Explain that, because I think this is a whole new category.
John Furrier
>> Yeah.>> Yeah. Well, or is it? Am I missing
David Lazovsky
>> it?
David Lazovsky
>> No,
John Furrier
>> no.>> Am I getting it right? Help me here. I'm fumbling with
Will Chu
>> this.
Will Chu
>> We talked about this. Okay. So if you look at AI, the first problem they tried to solve was compute. Right? You need to be able to compute things. And then as the models got big, you needed a lot more memory that Dave talked about. But then as things got even bigger, you needed to connect the compute and the memory to give you intelligence. So the real bottleneck today is connectivity. Because if you think of AI, it's like one big brain, but everything's got to be interconnected. If you only have a portion of your brain on and the other doesn't talk to it, you don't have a brain.
John Furrier
>> Yeah.
Will Chu
>> So that's why without all that, a lot of things don't work well. So that's really what we're doing. We're like at the center of AI with our connectivity-first strategy, and we're unique because of our portfolio and our 20-plus years of investment in this area to enable all that.
John Furrier
>> All right. The business performance— I know we got to run a little over time here, but the business performance is off the charts. I look at this, so four years ago, it was, what, $2 point something billion in revenue. This year you're going to be at what, like 15? What's the number this year? Oh, that was just the data center group, I think.
David Lazovsky
>> Yeah, yeah, that's right. So we're tracking to 12 this year. Okay, 20 next year.
John Furrier
>> 70 is a high watermark at the midpoint.
Will Chu
>> 80 was the midpoint.
David Lazovsky
>> 80 was the midpoint of the FY 2031. That's right.
John Furrier
>> That's massive. So that's coming from this thesis. How would you explain the thesis of what we just talked about? Because I think that's the key, that the networking and the connectedness, the ecosystem of Marvell's technology and products.
David Lazovsky
>> Well, I'll touch on it briefly, which is the economics of artificial intelligence infrastructure is now beginning to really take shape. Right. So you're seeing a shift from training-based workloads to increasingly inference-based workloads. Over 60% of all data center infrastructure builds are focused on inference. Inference is the return on the ROI equation. Training is the I in that equation, right? So, and you can see this translates to the operating margins of the world's largest hyperscalers. Some of them were sub-10% operating margin just 3 years ago. Now all of them are approaching 40% operating.
John Furrier
>> This is super important. We're super glad you're here. We'll get to the eye candy and the chip in a second. Explain that again, the economics, because a lot of mainstream media and just people who don't understand kind of looking at these data center builds as a bad thing. What you just said is, yeah, there's a lot of money going in, but the economics, the money is in inference. Explain the order of magnitude scope of the growth of inference and why that ties in the math. Yeah.
David Lazovsky
>> So 3 years ago, 1/3 of total data center infrastructure was focused on inference. Today, north of 60%. And over the course of the next 2 to 3 years, that will rapidly approach 90%. That's where the return is coming from right now. The fact that the largest cloud service providers in the world, which happen to be the world's largest companies, are generating 40% operating margin. What they're doing right now is they are reinvesting that operating profit back into the next wave of artificial intelligence infrastructure. Hundreds and hundreds of billions of dollars, $1.2 trillion in aggregate this year, and that'll grow to north of $3 trillion in aggregate by 20— by 2030.
John Furrier
>> I see it too, but I don't understand why people are putting fear, uncertainty and doubt into the market when you have CoreWeave telling me last week in San Francisco that same thing, training was 80/20. Now it's flipping the other way, but they're both growing.
David Lazovsky
>> Yes.
John Furrier
>> So reinforcement learning is another hot area. These reasoning models are starting to come. You're starting to see that's another part of the inference. So you got training continuing. Yeah. To go. So a lot of people are learning that. So I really appreciate that. Okay. Final, final word. How are you guys feeling about today? Obviously, stocks up. So that's good. That's reward. But in general, overall in the business.
Will Chu
>> Fantastic. Absolutely fantastic. if you just summarize it, as it's all been mentioned, this is the biggest buildout in CapEx in human history. And at Marvell, in terms of our connectivity business, our networking business, and our custom business, we're attached to the fastest growing part of those markets. Yeah, we have the leading technology, we have the customer credibility, a track record of execution. So we're going to grow faster than the market. And this is why you see we really upped our revenue targets because we honestly know we actually have the business in front of us today and we have full control of our destiny to make it happen. Super exciting.
John Furrier
>> Dave, play us out with a description of what this bad boy is right there.
David Lazovsky
>> Yeah. What you're looking at is the world's first fully monolithic 100T switch. So we'd mentioned SerDes earlier. Will was talking about our advanced SerDes, industry-leading SerDes technology. There are 512 200-gig SerDes in a monolithic device.
John Furrier
>> All right, explain what that would— the impact of that stat, the performance of this.
David Lazovsky
>> So what this does is it provides a switch that has the ability to serve, not just scale out at very high bandwidths like we're talking about, but of equal or more importance, Because it's monolithic, we have extremely low latency, 20% plus lower latency and 20% plus lower power than any competitive alternative.
John Furrier
>> So the bottom line is some people are getting bad AI, but this makes better AI. So yeah, good AI comes out of that, better AI.
David Lazovsky
>> And this is great for Marvell shareholders because this product right here is going to be a multibillion dollar annual revenue opportunity.
John Furrier
>> I watched my daughter saying, you get a bad answer, that's bad AI. Yeah. So this Good AI is good intelligence, good tokens, fast speed, data access. Congratulations, guys. Thanks for coming on.
David Lazovsky
>> Thanks so much.
John Furrier
>> This has been a special AI Factories episode. Obviously, the news and performance of Marvell stock today, also Investor Day, but more importantly, this trend of AI factories powering a generation, a shift, an industrial wave that we've never seen before. And the numbers, the math pans out. If you look at the inference and the economics and that's what people are learning. Hope we did our part here on theCUBE. I'm John Furrier, your host. Thanks for watching.