Name: AI Silicon & Hyperscale Infrastructure
Uploaded: 2025-10-17T17:26:00.000Z
Duration: 37 min 49 s

Clips
More from theCUBE + NYSE Wired: GSA Series - Women in Leadership

Sharon Clay

VP, Hardware Infrastructure

NVIDIA

Danielle Shown

Silicon Cloud HW & Infastructure

Microsoft

Olivia Wu

ML ASIC Design Lead

Meta

Manisha Gambhir

Senior Director

Marvell Semiconductor

play_circle_outline Introduction to the panel featuring leaders in AI and infrastructure from major companies.

play_circle_outline Nvidia's role in driving the GPU innovation and meeting compute demand.

play_circle_outline Microsoft's focus on co-optimizing hardware and software across their cloud infrastructure.

play_circle_outline Complexity of integrating AI workloads and scaling data centers efficiently at a large scale.

play_circle_outline Marvell's Strategic Role in Custom Silicon and Interconnect Technologies for AI-Driven Workloads

play_circle_outline Exploration of the relationship between software and hardware in optimizing performance.

play_circle_outline Discussion on the importance of innovative hardware and software in AI advancements.

play_circle_outline Emphasis on collaboration between companies to foster innovation in chip development.

Info
Transcript

AI Silicon & Hyperscale Infrastructure

Sharon Clay

VP, Hardware Infrastructure NVIDIA

Danielle Shown

Silicon Cloud HW & Infastructure Microsoft

Olivia Wu

ML ASIC Design Lead Meta

Manisha Gambhir

Senior Director Marvell Semiconductor

In this #GSAWomenInLeadership segment from theCUBE + NYSE Wired, host John Furrier convenes a powerhouse panel on AI-era silicon and hyperscale infrastructure: Sharon Clay (NVIDIA), Danielle Shown (Microsoft), Olivia Wu (Meta) and Manisha Gambhir (Marvell). The conversation unpacks how reliability-at-scale, disaggregation and hardware–software co-design are reshaping data centers into AI factories. Clay details NVIDIA’s approach to predicting and verifying compute needs end to end – from in-house demand modeling and platform verification to multi-tier network... Read more

explore Keep Exploring

What event is being hosted and who are the panelists involved? add

What are the key considerations and strategies involved in scaling compute capabilities to meet growing demand? add

What is Microsoft's approach to integrating hardware and software to create customer value? add

What are the complexities and considerations involved in creating a scalable and robust data connection for customers? add

What role does Marvell play in the data center ecosystem? add

What are the advantages of hardware and software co-design in optimizing silicon for AI workloads? add

What is the significance of CUDA in the context of software innovation and hardware optimization? add

What are the strategies mentioned for accelerating the mass deployment of AI infrastructure? add

bolt Powered by CUBE AI

AI Silicon & Hyperscale Infrastructure

>> Welcome back over to theCUBE, here in our Palo Alto Studios. I'm John Furrier, your host of theCUBE. We're here for the NYSE, Wired and theCUBE special presentation GSA series. We unpack the latest minds working on the hardest problems to bring AI and infrastructure to the world. We got a great panel, power panel here, AI, silicon, and hyperscale infrastructure. As the race continues to build the fastest compute for the data center impacting silicon, interconnects power and cooling and the software stack, these players are making it happen. I've got a great line of here, Sharon Clay, vice president of hardware infrastructure, Nvidia. Thanks for coming on, Sharon. Appreciate it. Danielle Shown, silicon cloud hardware and infrastructure at Microsoft. Thanks for coming on. Appreciate it. Olivia Wu, ML ASIC architecture at design at Meta. Congratulations on all the success you're having. Great to have you on. Manisha, again here, senior director at Marvell. Thanks for coming on. Appreciate it.

Manisha Gambhir

>> Thank you, John.

>> So first of all, the title of this gets my attention, AI, silicon because we run siliconangle.com website, not related, but we cover silicon. Hyperscale infrastructure, it is the hottest customers who are spending the most money. So everyone sees that in the news. They see all the billions of dollars being spent, 20 billion in North Carolina, 100 billion over here, a billion over there. So much money is being spent out on the infrastructure footprint that it's almost a revolution, like when 4G and 5G connectivity more and trillions of dollars are going to be spent. This is going to set the table for this, all powered by this new systems revolution around the new architectures of large sale computing, GPUs. So serious action, you guys are in the middle of it. So, let's start with Nvidia's GPUs. You guys have really set the table. Jensen talks about this all the time on camera and on stage. The pace of innovation is so high and the hardware is key. GPU's technically a chip and the system, but the software behind it is causing massive innovation. What's the pace of play like?

Sharon Clay

>> Well, first, it's a very careful balancing act to try and bring all these things together and understand where to invest. And so the first thing is to have real clarity on the growth of demand and how to meet it. And so first, we've built in-house models to help us to predict the exact compute cycles required for certain kinds of deliverables and goals. Then we also compute and simulate the compute cycles that can be generated through efficiency optimization that involves the entire stack, as well as operational improvements again through the entire stack, and then of course, procurement. However, while the ability to procure and meet the compute cycles is the first step, reliability at scale is essential. So, what used to be historically a rare fault is now constant at our current scales, and so verification of the entire platform has become very important. And so we have a deep and intensive process, all the way through development and design with verification, and then building test clusters to fully stress the full platform in data centers for their extremes. And then we also internally deploy these full platforms internally and use them to ensure production scale and efficiency across the ecosystem for our external deployments.

>> We really appreciate what Nvidia and you guys have done, so thank you for bringing all that innovation because it has changed the game. I mean whether you call AI factories and the tokens, the scales changed, software, the software innovation. Topic of this panel is the hyperscalers, you sell to all of them. We have one here, Microsoft Cloud was great. We love cloud. Who doesn't like cloud? Cloud brought in with the SaaS generation, changed the nature of the data center. Now, you guys are turning into large scale supercomputers as well. So, Danielle, this is a huge changeover, but it's a continuation of cloud. Tell us what you're working on.

Danielle Shown

>> So at Microsoft, we really believe that everything has a purpose. And so as we move through the silicon layer into the platform layer, and we integrate things into racks and then we apply a software stack over top of it and utilize a network layer, all of these pieces of hardware, and their corresponding software, have to work together. We have to co-optimize at every layer of the stack. We can't achieve customer value at scale by simply focusing on one area of the stack. And so by being able to build things out, whether it's custom silicon, whether we buy it off the shelf, whether we partner with someone else to build it, it really is about creating that differentiating value for the customer and giving them enough options to select from that they can really maximize their profit and performance targets based on whatever ratio they need for their workloads and for their customer's demand.

>> Talk about the cloud scale integration scope, scope, the complexity and the requirements needed to, what's the expression, change the airplane's engine out at 34,000 feet. You're already operating at scale and now you've got a system that has more demand. Talk about the challenge and scope the complexity and how it works.

Danielle Shown

>> Well, it is hugely complex, and frankly, one of the things that we're noticing is a lot of startups are really focusing on what I'm going to call a disaggregation approach, so that you can really have purpose-built hardware and software that comes together in a complementary manner that really allows you to have a robust connection for your customer to their data for whatever their workloads entail. And again, it comes down to you can't do this at just one layer. If you want to scale and you want to scale with a high degree of quality and a high degree of robustness, it all has to come together. It's all interconnected and it all has to be done together, tested together, and ultimately, hosted in a data center together.

>> Well, we're going to put a pin in that because we love how the distributed computing paradigm has scaled up, out and across, and it's the same kind of system design. We'll come back to that. Manisha, talk about Marvell because you guys have a unique position in this game. What's your role in the data center and the chips? Because the system design is the discussion. It's a system design at scale. It's like if we were going to build a server back in the '90s, we'd sit there and say, "What's the motherboard look like? And here's your IO." You do that kind of work. That's happening at massive scale. So, everything's changing, but the game is still the same.

Manisha Gambhir

>> Absolutely. And John, it's a tough question to ask, where does Marvell fit in the whole ecosystem with regards to data center? It's tough for me because what I pondered is where does it not fit? So starting from, I mean we do custom silicon for all our hyperscaler partners, whether they are emerging hyperscalers or established hyperscalers, and what we call as XPU opportunity. And lately, like you mentioned, it's not just the compute, it's what are the companion chips that go next to it. So, we call it XPU attach. It includes the network switches, the memory, the CXL, the expanders. And let's not forget the interconnect. That's where we are leaders as well. I would encourage you, take your microscope, look at the millimeter level, to die to die connection, to put on your visors and look at all the way thousands of kilometers of coherent connections that are all powered by Marvell.

>> So on the interconnects, the hottest area right now, I will say, networking and interconnect, the same thing as networking. We talked about co-packaged ops in the last panel, networking is now the OS. Jensen said that in Nvidia's last GTC. What he meant was it means you can connect more to resource together, and that coordination, that orchestration, you're linking, you're loading. These are OS concepts at scale. So now, this is where the chips come in. So, share your vision for folks that aren't inside the ropes because it's not general purpose anymore. Things are almost custom for the use cases, where the memory sits. You mentioned down to the die level, try to explain that to someone who's not building chips. Why is this so important to have that kind of capability and diversity and performance?

Manisha Gambhir

>> Yeah. So I do, I design chips, I design interconnect. That's my bread and butter. And sitting here I find it, I'm almost witnessing a Cambrian evolution on diversity and number of interconnects. And this diversity is fueled primarily by AI acceleration. I'll give you a few examples. When we look outside data centers, we introduce our technology with regards to coherent light. Our data centers now look like citadels. They are not one building, few buildings, they're tens of kilometers apart. So here, we merged concepts from PAM DSP into coherent technology. So goodness of doing low latency, power efficiency as well as reach into something that was more useful for our customers and drove benefits for them, which is coherent light. You mentioned things like CPO, that's where the whole drama is happening. We are marching towards CPO in our scale-up networks, but as we are progressing towards that, there are other technologies in play like active electric cables, active optical cables, and with the right protocol stacks on PCIe, UALink and the necessary infrastructure to build that networking in.

>> And that brings up the question around resource allocation. If you have a GPU or an XPU and it's expensive, it's engineered to go superfast, and it's not actually running fast. So, this all is back into the design piece. This is where the hardware software piece matter. What's the number one challenge right now for that? Is it interconnect? Is it networking? Is this just an open question? I'll throw it out there. Anyone wants to grab it, the biggest challenge to make everything go faster.

Sharon Clay

>> It's everywhere.

Danielle Shown

>> It is.

Sharon Clay

>> But I'll say that these applications, they want everything. They want very low latency and very high bandwidth to very large data sets. And so this puts tremendous burden on the networking architecture across the data center. And as these things have grown to tens of thousands, heading towards millions of GPUs that we're trying to access, the kilometers is no joke.

>> Yes.

Sharon Clay

>> And so the technology that we can bring, such as a multi-tiered approach that Manisha was referencing, so we have NVLink for that peer-to-peer, GPU-to-GPU direct communication and then InfiniBand and our AI optimized Spectrum-X Ethernet architecture for connecting the thousands of systems, and then HBN networking on the networking chips themselves and special caches on those chips. So, the entire data center is involved in trying to optimize this

>> I think that sounds like a Cambrian Explosion to me, if you want to put it that way. I love that word. Olivia, I want to talk about what you're doing at Meta because Meta jumps onto the scene with massive investments, really strong open source, promoting a lot of developers. So if we go back to the fact that we're now on a Cambrian Explosion, a rise of distributed computing at scale, kind of old school concepts now completely recast, I won't say replatform, accelerated to new architecture. You got to run software on this stuff for the application. So you have, I don't want to say the three tier hardware and middleware apps, but you got to have software, so the hottest area is the open source models. Explain why this is important and how it ties into some of the things that Nvidia, Microsoft and Marvell are doing because they're also going open source too.

Olivia Wu

>> Yeah, I think Meta had always been a proponent of doing open source, and for us specifically, the PyTorch has been a very vibrant ecosystem, a great framework for us to develop our MTIA software on. And that has allowed us to very quickly port it a lot of the software onto the PyTorch, and we're able to basically run the software on both the GPU as well as the MTIA system. And having the open source environment, I think it really helps to encourage collaboration amongst all the engineers from different companies as well as the research community, so we can work together to really improve the software and also come up with new ideas on any features that we can put in to enhance the software.

>> What I love about Meta, formerly Facebook, is that in following that growth of that company here in Silicon Valley is they were their own customer zero at scale. In fact, people might not know this, but with Microsoft donated all their IP to form the Open Compute project. That was a huge change of, I would say 15, maybe 12 years ago. That concept shows that Facebook had to made their own. But now that they're scaling, you guys are now have a hardware mindset. So you're coming back to the Nvidia, Microsoft challenge and Marvell makes the chips, the new systems architecture has to tie the software and hardware together. So, when you want to have a really high-fast contributing computing, I mean all the hardware companies say, "We're not a hardware company, we're all more software company." Technically, they do have more software developers than hardware engineers, but it's a combination of the hardware and software that makes the scale work. So, I guess the question that popped into my head, I'd love to table it, is that what's the importance of the software-hardware relationship? If you had to boil it down for someone who's in high school or college or looking for a career change, why is the software-hardware relationship important?

Olivia Wu

>> Yeah, I think definitely what I see over at Meta is that by doing hardware and software co-design, we're able to really optimize every aspects of our silicon so that we can optimize both instruction sets, how the jobs is going to get dispatched, the whole memory subsystem, all the way out to what is the network requirement, all the scale in scale out and scale up that needs to be-

>> Don't forget scale across and scale in is the midterm, scale into the die and then scale out. I mean, this is where we're seeing the ecosystem. I know NVIDIA, you guys are heavy on the ecosystem as you guys have expanded out, Microsoft's always had an ecosystem. Marvell has an ecosystem. But the ecosystem's changing because now you're having these kinds of discussions because AI workload diversity, where to send the workload, if I know at end to end. But what if there's a model change or there's got to be intelligence, and the road to super intelligence says that the systems will be smarter. So managing that hardware relationship, you guys have been doing it. How important is that? Because CUDA is that today.

Sharon Clay

>> Well, there's always been this cycle of learning as applications create desperation and innovation in software to meet their demands. And then as those innovations mature, they standardize and that's what produces these opportunities for hardware optimization and innovation. And so an example now would be a CUDA 13.0, bringing support for advanced tile-based computing into a programmability for these platforms, and this optimizes computations for scientific programming and robotics as well as training. And then for inference, another very important growing area, again, CUDA is bringing higher level optimizations such as quantization for using multi-level precision and also something called layered fusion that improves the representation of the context needed for inference. So, the software is critical in delivering the value of these complex platforms and providing the opportunities for the hardware innovation.

>> Don't forget Omniverse. Omniverse is-

Sharon Clay

>> Absolutely....

>> one of my favorite things because I love the digital twin concept. I've been on this bandwagon, I've been totally on this because simulation has... Right now, the data is mostly synthetic, but you guys are open source the data, but it speaks to where the large scale is going. As more software comes in as the ecosystem partners come in, that synthetic data and that simulation will help write software. And so I think that's going to be a big deal. And I guess my question would be relative to the hyperscalers, at what point is this completely autonomous infrastructure? And that's the question we're asking ourselves. So Microsoft, you guys have been on the transformation on the cloud, past I'd say seven, eight years, complete run of cloud growth, build out success, and now you're looking at an AI wave coming. I mean, robotics is one of the hottest areas too. I mean that's going to be powered by many resources.

Danielle Shown

>> Well, the reality is that the model and the capabilities that we need today to meet our customer's demand and to really create value, we can't do it at any one layer of the stack alone. Obviously, when you look at it as an aggregate, almost as a product, and when you talk about the product, you're literally talking about silicon all the way up through what shows up in a data center. But the focus right now has to really be on strengthening partnerships using open AI models. There's a saying, if you want to go fast, go alone. If you want to go fast and far, go together.

>> Open AI models, not open AI models or both. I mean they're in the ecosystem too and Meta.

Danielle Shown

>> Both. Yes, right, however you want to look at it. And I think that the entire industry is really focused on how do we manage this magnitude of change? How do we do it in a reliable and a trustworthy way for our customers and how do we ensure that we're meeting things like sustainability goals? And Microsoft has a sustainability goal for 2030 and we are sticking to those goals and we're finding creative solutions. And we work with partners to identify those creative solutions because at the end of the day, even though the GPUs and the AI infrastructure is hugely power hungry, we still have to meet those sustainability goals.

>> Manisha, I want to talk about ecosystem in a context to entrepreneurship. Innovation and entrepreneurship have been buzzwords. I won't say cliches, guys, they're true. But in some markets, they just throw innovation at a word. "Oh, it's innovative." But now where the game changing shifts that's happening, the results come out of innovation. So, the innovation on the entrepreneurial side, say, from startups to actually inside companies. Our last panel, they talked about how they're using AI inside to make their testing and packaging better. It brings on other things. Once you're in the AI kind of rabbit hole if you will, you get better and you discover new things once you're in there. It's not like you sat back and said, "Hey, I'm going to discover something. You've got to kind of be in the arena, so to speak." So the question is on the chip side, we were also talking about how the iPhone was just a packaging option of phone, Blackberry. All the elements were there, but the innovation was the iPhone. So, we're seeing chips and devices coming where Nvidia is going to have all the horsepower. They got the super cloud and they got the open source developers. What's going to happen is we're going to start doing training on our phone. I mean, that's coming or my wearable or some 5G combination, what is it? Thor is your processor. I forget what they make. I think it's the Thor. You can have a device on a lamppost training, that's an AI factory basically. So, the chip's got to run it, because ecosystem has to foster innovation. How do you guys view that because the chips matter?

Manisha Gambhir

>> Yeah, so an excellent question, John. I think with all our hyperscaler customers, they keep us honest. Their goals with regards to power efficiency, sustainability, and like you said, AI factories. We don't want to be putting a nuclear power plant everywhere. So, the innovation that comes in chip design comes from power efficiency gains that we are getting by looking at chips, not just as singular devices. We're looking at heterogeneous integration. We're looking at devices that we look at system integration challenges and look at total cost of ownership. With regards to our custom silicon business, with all our hyperscalers, we've pushed the envelopes with regards to custom HBM, SRAM integration where we've driven gains in order of 60, 70% on memory efficiency and giving them the golden area to put more compute in. That is true innovation that's driving us ensure that our customers are hyperscalers, are able to do more with our custom silicon.

>> And just from your standpoint, the official position of the company, share with us the speed timetable. Everyone wants, I'm a bandwidth junkie, just want more bandwidth-

Manisha Gambhir

>> Love that question....

>> I want more performance. I'm always asking me Nvidia, "Can I get a GP for home?" This is coming. Can you scope the timeline of how fast the chips are going to get it so that an AI factory could sit in my home? I know Nvidia got that. What's that device you have? That's basically it... I won't go there.

Manisha Gambhir

>> I love that question. Everybody wants to shatter the Moore's Law now. We're gone... In transistor scaling, we're already at billions of transistors. We want to now look at this as integration in trillions of transistors, not at a single chip level, but as a system. From bandwidth point of view, to your point, we showcased earlier this year at OFC, a four-hundred giga, 448 data rate for our optical interconnects, which would push the boundaries for not only our scale-out network, but front-end networking as well. 2x per two years, that's what we've been pushing for and I think with die-to-die, with heterogeneous integration, I think we are going to go beyond that as well.

>> This is a AI panel with hyperscalers in the title. We have two here. I know you guys are customers of them and they're your customer. I guess the question is that co-design integration was mentioned, AI software and hardware co-design, the software stack.

Manisha Gambhir

>> Talk about how this all comes together because I think we're in an era where a new distributed computing, not reset, a evolutionary step function change is happening with software/hardware, but as you guys scale and you're supplying faster hardware/software, what's the co-design piece? Because you're also partners and customers. There's big spend there. So, it's not a sales thing. I don't want to be salesy, but I'm a salesman. I'm like a customer spending billions on new stuff to create a generational infrastructure. What's the co-design software stack partner-customer relationship look like as you guys work together?

Sharon Clay

>> Well, cooperative innovation I think is the thing. And as you point out, these advancements to bring AI infrastructure to mass deployment requires innovation working together across all of us. And so to accelerate that, we provide reference architectures for system design, for rack design, for full data center networking designs to our partners. And those designs have been tested internally in our own production deployments. Then beyond that, we've also now developed NVLink Fusion that allows for third-party AI chip manufacturers to link in directly with NVIDIA components to enable the hyperscalers to deploy our rack designs or NVLink seamlessly into their data center in a diverse partner ecosystem.

>> Yeah, the software stack too, I mean, you guys are living this right now. I asked that earlier, but as it comes together, you're going to have customers that have on-premise activity.

Sharon Clay

>> Yes,

>> Their data's gravity's there. The edge is coming fast. That's new data coming in too, by the way.

Danielle Shown

>> From all sources at all speeds at all-

>> How fast is that going to go back? What's the latency on? Again, these are technical questions.

Danielle Shown

>> So even for the software piece, the software comes into play a little bit later outside of hardware-supported softwares like the firmware and those kinds of things, but this co-design concept and partnering with companies like Marvell, and you're going to get a better rate of innovation when you have the domain experts and you have them next to people that maybe are experts in a different domain. But they all know what the end product needs to look like and they all know what their piece of the contribution is to the overall stack. And when you put a lot of great minds together, you get very different things than if you just try to go about this alone. And that's why I think those partnerships, albeit strategic, are so important. Because the whole chiplet and disaggregation and the requirement for advanced packaging and three-dimensional die stacking, and it's all about being able to fit more into that rack, whether it's the compute or whether it's the GPU, it's all about more and customers want more. More is always better. But then you have to think of innovations at every layer of the stack that actually allow you to scale that out. So even when you have the right innovations, you might have other limitations. And I'll use an example, we designed our own liquid cooling rack. So, why is that significant? Well, we can take that liquid cooling rack and combine it with AI hardware and set it up in a data center that is not purpose-built to accommodate liquid cooling. Innovation can come in all sorts of ways, but it really is about those partnerships-

>> Necessity is the mother of invention, is that the expression?

Danielle Shown

>> Yeah, exactly.

Manisha Gambhir

>> I have? I think we should rename Moore's Law into More's Law after your-

Danielle Shown

>> Yeah.

>> That's going to be the tagline, more.

Manisha Gambhir

>> Yeah, more.

>> We'll make a t-shirt out of it. I mean, this is the co-design as a team sport. I mean I love, again, Jensen mentions it or Jayshree at Arista talks about this. All the executives that are in semis all talk about being like, let's go along for the ride. It's a team sport. Because the co-design, I think it's always been there, but not that this many moving parts, it seems there's a lot more moving parts and there's no yet management book written on this yet. That's why I asked the question. Your thoughts on co-design, because there's practices, there's best practices that are emerging, there's techniques, any thoughts? Olivia?

Olivia Wu

>> Definitely on the software side, a lot of times we wanted to actually understand where the roadmap is going to go and understand what are the requirement two to three years down the road. And that actually helps us to drive and work with all of our vendors to provide the IP so we have that available by the time the silicon is ready. The other thing is outside of software, there's actually a lot of system level co-design as well as in the data center side. What are the limitations, as what you mentioned, not all of our data centers are plumped for liquid cooling. So, we also have a lot of these innovations that are created so that we can retrofit and deploy

>> I was going to use that word you said first, but that's project too, that takes more time. So, this is a time game too, so this is cool.

Olivia Wu

>> But if we make that open source and collaborate with many other companies, we now have a much more richer technology and more mature by the time we will need these to be deployed.

>> Sharon, your point earlier about standards are super critical in these major shifts because as things get standardized, they're better. Co-design, you're designing chips. You're not like this, you have to think, your aperture goes bigger. What's changed, Manisha, just from say a decade or two ago to now on the co-design? Any observations, comments on that?

Manisha Gambhir

>> Absolutely. I think what's driving this co-design is all our partners, customers, have their unique workload needs. So, what has changed is when AI has put all these applications in front of it, someone is doing a software as services, another customer is doing infrastructure as services. Their workloads start looking very, very different. The same transformer model applied to them from perception on training on generation and agentic AI, it all looks very different. So what has changed is we have to listen to our customers more closely, see what their memory needs are, what their expansion needs are, what their compute needs are, and play with them on XPU as well as XPU attach space and give them those customization. And that's what custom silicon is about. And for us, that's the biggest part of co-design.

>> Well, it's a super popular, we could do a six-hour podcast here. You guys are awesome leaders in the industry. My final question is a little bit more of the future vision breakthroughs that you guys see. I'll ask each of you to share thoughts on the future trends, a breakthrough that you see coming that you could share, a gut feeling, key specifics, something that you're working on. We'll go down the line, we'll start you off.

Sharon Clay

>> Well, from our perspective, it's twofold. First is the obvious scale piece. So, we're scaling up to now millions of GPUs, and that's just completely transforming what is possible in computing and in AI, but all of the scale is to a goal. And so what's I think really transformational is what does this enable in AI and what changes are coming from AI itself? And the coming of agentic AI is really exciting for us. This is a step that really lets us tackle multi-step, super complex problems that go across workloads as opposed to just your targeted inference solutions. And internally, we're evolving our entire engineering workflows with the arrival of agentic AI, and using NVIDIA components in the entire ecosystem to do this, such as NIMS for producing domain-specific and constrained inferencing, the NEMO platform for doing the customized training of models, and then agent IQ for managing all of this collection of complex agents. And so this leap forward now I think is a lot more than just performance. It is really unlocking now new capabilities that we didn't have before from this AI, which will lead to new innovation.

>> And it's exciting. There's a lot of computer science and engineering under the covers to make that happen. It's a great time to be in this field. Danielle, what's your vision on the future breakthroughs, key things?

Danielle Shown

>> I think right now we're just at the very beginning of what I'm going to call mass revolution. If you evolve, you're getting a little bit better. Things are changing a little bit each time. This is full scale revolution, and she talked about the agentic AI future. And I really truly do believe that today's models are about how do we do what we do faster? How do we do it in a more productive way? How can we foster innovation faster? How can we improve cycle times? I think that is going to evolve through the next couple of years as the capabilities in the AI software combined with AI hardware. As that continues to grow, we're going to shift from, hey, let's be more productive, let's be faster, to let's actually create new value streams that simply did not exist before AI came into the picture. And I honestly imagine a world, and I can't tell you how far out it is, where we're literally using the last generation AI hardware and software coupled together to produce the next generation of AI hardware and software, and it's just going to become a virtuous cycle. So, we won't just be doing things better and faster. We will fundamentally change how we do things.

>> It's like renewable computing.

Danielle Shown

>> Exactly.

>> It renews.

Danielle Shown

>> Yep. That's kind of what I see.

>> I love it. That's great. And of course, and just thinking about what's going to change in our lives too that the impact to the world is amazing. Manisha, your future vision, they got the chips we want the pair with our XPUs and all the other systems. What's the-

Manisha Gambhir

>> I love what you said, by the way. It's a very noble vision, let's put it this way. To get there, I think shattering that Moore's Law and putting that virtuous cycle in place, getting to million clusters working together, homogeneously. The first thing that I see, which is not so distant, I think heterogeneous integration with regards to 3D, three and a half D, 4D technology really rack-sized integration. That's something to come, to start this, spread this. Further down the line, and this goes into maybe device and all, this is not something any of us are hopefully working on. I think if we can get memory and compute together, so that we don't have that memory wall, we bring... Whether it's in memory compute or optical compute, and there are advancements and research advancements into it, I hope I see that in my lifetime when we don't really have to have them talk and build this great network because they're homogeneously together.

>> So, integration.

Manisha Gambhir

>> Integration.

>> Olivia, bring us home.

Olivia Wu

>> So, I think they all covered many aspects. I look at AI as something that we can utilize today to help improve the efficiency, but I think that we can actually utilize it to invent new things, whatever these... Whether it's new ideas or new streams of product that we might not even think about, and I think in time we will be able to use AI to build more AI and maybe new algorithm and make life much better.

>> Yeah, it's the human manual approach to human and machines. Then scale introduces whole nother rarefied air of value to... We don't know what that looks like here. I guess we'll see it when we see it.

Olivia Wu

>> Yes.

>> Well, thank you so much. You guys are the leaders for leaders here in the industry. Thanks for coming on theCUBE and sharing commentary on a very important topic. Appreciate your time. Thanks.

Danielle Shown

>> Thank you.

Olivia Wu

>> Thanks.

Sharon Clay

>> Thanks.

Manisha Gambhir

>> Thank you, John.

>> I'm John Furrier here with theCUBE and the NYC Wired community. We're breaking down the future of how AI and the semiconductor and the infrastructure is going to be enabling the change that's going to enter in all kinds of new capabilities and change the world and change our lives. And I even heard of data centers in space, so we'll say in space too. So of course, we'll be covering it. We're doing our job here. Thanks for watching.