theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | Rodrigo Liang, SambaNova Systems

More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Rodrigo Liang

CEO

SambaNova Systems

John Furrier

Co-Founder & Co-CEO

SiliconANGLE Media, Inc.

Info

Rodrigo Liang, SambaNova Systems

Rodrigo Liang

CEO SambaNova Systems

John Furrier

Co-Founder & Co-CEO SiliconANGLE Media, Inc.

HOST

search

>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, co-hosting with Dave Vellante, my co-host. Hello, I'm John Furrier with The Cube. We are here at The Cube's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. This is part of The Cube's NYSE Wired program and open community. We have back on The Cube, Cube alumni back for another appearance, just like a regular contributor. Rodrigo Liang, co-founder and CEO of Seminova. Part of our AI Factory series, one of our most popular series we started 2 years ago and really has been the precursor to the AI infrastructure boom. Very good to see you. Thanks for coming back on TheCUBE. I think Gemma talked to you the last 2 times when I was in California. Thanks for coming back on.

>> Yeah, thanks for having me. What an important time for us to be in this, in the AI industry.

>> We've seen each other now for a couple of years as part of this new NYC Wire community at events and on TheCUBE. Some significant changes in the past year. I mean, inference obviously A lot of insiders saw that early. The mainstream saw agents now kicking in. Coding was great, brought that in. Agent, Agentic brings up a whole nother paradigm shift around the role of the resource to service agents, what inference means there. So you start to see a whole shift. What has been the biggest change for you guys besides the billion dollars in funding you guys just closed, which we covered on Silicon Angle? That's validation. But what's been the biggest change this year?

>> Well, look, inference is now the economic center of AI and people are trying to figure out how to make that investment sustainable. And so as we went from training to inference, now people are thinking about how to deploy, deploy with sustainability, deploy with good energy consumption and all of those things. But most importantly, how to deploy in a way that they can get financial payback, right? We can't continue to borrow money forever without returns. And so being able to drive good economic payback for their investors is going to be an important part of data centers and actually as they move into inference.

>> Yeah, and people know that tokens equals revenue. That's well understood. Now we're starting to hear conversations about modeling out revenue growth. I think we're starting to see benchmarks now. We're saying this gigawatt equals this in revenue, this megawatt equals this in revenue. It's starting to— people start to quantify. And even a couple of years ago, I think Jensen Huang of NVIDIA said no one's spreadsheeted it out was his word. But yeah, that's the financial modeling. What have you seen there as CEO as a CEO and running your business, you're involved in a lot of these economic conversations, not just from a deployment standpoint, but the payback, a lot of these financial conversations.

>> Yes.

>> Energy and money are the two factors.

>> Yeah, exactly. Exactly. Well, the energy, the energy part, the data center cost and the capital acquisition of infrastructure, those on the cost side. On the revenue side, you can also dial that up. And so you can think about why people are driving speed like Samodova. We can generate speed because speed gets you more throughput. But more important than that is also speed at concurrence. You know, can you get many, many users getting the same speed at the same time because you want ultimate total output, total throughput per rack divided by that cost structure.

>> You guys have been at this for almost a decade now. We were talking before we came on camera about Hot Chips, which is the Stanford event that's well known as total nerd fest. It's Nerd Nation at Stanford, as everyone knows. But this is like the, the alpha, the state-of-the-art engineers really working on the next generation of accelerated technology. The conversation that I got away from that was there's a lot of work going on around processor called just processor generically, whatever unit you wanna call it, and memory, obviously HBM, and now you got solid state. But this kind of reminds me of the '90s when you had to use memory management utilities to swap in and out, but at such bigger scale. What has been the big thing for you guys now as you look at your roadmap How are you guys optimizing? Because everyone wants to squeeze as much tokens out of that energy, out of that processing, the relationship with the data to memory. These are all now part of the hardcore engineering vernacular.

>> Yeah, exactly. Well, ultimately you have these chips and these chips are constrained by what the technology can give you, both in terms of compute and the memory. And so for someone over— we're really focused on driving the total output. At the lowest cost and lowest power. And so if you can do that and before the session, we start talking about how people are measuring their compute in terms of megawatts and, and their revenue in terms of megawatt. Well, if you take that megawatt and you can actually put more racks in and each rack produces more tokens, that's how you generate more revenue per megawatt. Right. And so if you, if you take that chip, the same chip, and you increase the total output per chip, you're going to do a lot better when it comes to payback.

>> What are some of the things you guys got going on that you could share on Momentum side? Because that really is where everyone's looking at. You talk about some of the economics of buildout, but when you're in operations, you're building, operating, investing. Everyone's doing those three things at the same time.

>> Yeah.

>> On the upside, what, what do you see coming out? What do you guys have now? What's, what's the momentum?

>> Now, look, there are three incredibly exciting use cases that we see with some of the technology. One, people who have already deployed a lot of NVIDIA gear, we're partnering with NVIDIA on this with something called disaggregated inference. And what you're able to do is take NVIDIA gear make that partner with a Samanova gear and in that construct disaggregate the front, front part called pre-fill and the decode part using Samanova and generate 3x the total throughput. So existing hardware of NVIDIA can jump 3x in total throughput just by putting someone on our rack next to it. And that's a really, really exciting development for people who already have their gear and trying to get better economics. The other one that I think we're starting to see a lot of really exciting use case for is this distributed data center, existing data centers, air-cooled, and being able to actually use brownfield data centers that don't require new investment, new, you know, breaking ground, new liquid cooling, and just put some amount of air-cooled technology into it and then get significant advantage just by actually delivering faster at a lower cost, lower power.

>> So two major things that you said there. One is if I bought gear, call it gear, that's the term we use a lot, but a lot of systems Normal, normal depreciation would have that losing value.

>> Yeah.

>> Now you got value creation. So the price, the value of that gear is actually more critical. So that's good leverage. Yeah, that's a good use case.

>> Exactly.

>> Great.

>> Exactly.

>> One signs up for that all day long, I'm sure.

>> Exactly.

>> The other one is this new paradigm around disaggregated serving.

>> Yeah.

>> Which is also a precursor into disaggregated infrastructure.

>> Yeah.

>> Because what you just said is essentially adding new, new nodes out there. Data center nodes to be AI factories, basically, without all the requirements for the megawatts, gigawatt data centers that takes years to build.

>> Yeah, that's right. I mean, one of the biggest challenges we're seeing in the world today is how quickly can we stand up gigawatt data centers. And, and that's going to be more and more challenging, you know, as, as we think about the needs that we have and how much it impacts the energy grid. And so if we can actually reuse existing infrastructure around the world, existing infrastructure in the United States, where you already have energy allocated, spaces allocated, it's air-cooled, and you can deploy infrastructure to have state-of-the-art AI running on it. I think that's going to be an incredibly valuable way to actually get AI available to everybody at a much, much lower cost.

>> So does the word edge of the network go away when you have distributed computing paradigm where you have large data center nodes like an AI factory in Mega Texas or whatever it is?

>> Yeah.

>> And If you just got a cell tower that's got a building and power and network connectivity, that's air-cooled in there, that's an edge. That's still a node on the network, but it's technically an edge. So smaller factory configurations.

>> We actually call— yeah, we think of it as these massive kind of buildouts where you have huge gigawatt data centers. That's going to continue because people have to train models and the necessity for large clusters continue to exist. Now you go into what I'm calling distributed data centers. So take the 50 top metropolitan cities and Vista, for example, is a partner that building out these distributed data centers in existing brownfield data centers, re-energizing existing infrastructure with existing power, existing cooling, and then running that using someone over here. Now, edge, edge is the next thing I'm super excited, super excited, because if you think about true edge, which is where our mobile towers are, is where our users are, robotics, all these different things, there is still another wave coming where very, very localized computing is going to be serving very localized use cases of AI. And that's to me the true edge. And they need high performance, incredible high performance, ultra low latency, because now you're starting to talk about robotics and you talk about kind of the things that people want to use every day. And frankly, as a population, our patience is not very high. And so I want to get your thoughts.

>> I want to get you— what about robotics? I want to bring up safety because the number one conversation in all of our physical AI robotics series that we're running, Safety is the number one conversation. Security is in there, but I'd say number 1B, basically. 1A would be safety because robotics, you got to have a safe car, you got to have a safe robot. Yeah, that's really obvious. You go, okay, all the safety on AI conversation seems to be a tempest in a teapot because it's like people are working on safety.

>> Yeah.

>> What's your thoughts on all this negativity around safety? You got half the world being like, okay, I don't really understand what, but I hate it. I'm afraid. They don't understand. Then you have the people who understand, go, well, this is one of the best revolutions of all time in the computer industry. So you got— it's 50/50.

>> Yeah, yeah. No, it's understandable. Look, I think you look at a technology as transformative as this and you saw it over the weekend with the Frontier models and now, you know, you know, and more broadly around the model side that safety is important and it's a great, great wake-up call for all the leaders to start thinking about the fact that we've invested in the R&D of the models, but how do we invest in the safety and the guardrails around kind of how we use it. And so that's incredibly important. I think you're going to continue to see attention to that. And we saw this in early internet days where safety and security kind of became, well, something that was in the forefront of people's minds and entire industries got created from it.

>> And so, you know, I have a lot of respect for Dario, but I do think that he's not the poster child for safety when it comes to AI. I think he's— he's no noble effort to lay out his concerns when the air company's got a great track record in terms of the ethics. So, you know, give them the props for that. But there are many people in the industry that actually have been through this before and have actually done it.

>> Yeah.

>> I've seen where you had let chaos reign, rein in the chaos situations.

>> Yeah.

>> What do you think that he could learn from what Dario and others that are now aware of this could learn from the history of how Innovation can be chaotic and then reined in. Yeah, that's Andy Grove's favorite expression. Yeah, let chaos reign and rein in the chaos.

>> Yeah.

>> Look what happened with Intel under his regime and under Gordon Moore.

>> Yeah. Well, I mean, look, it starts with emphasis and attention. I think, you know, we're, we're at a time where the industry is starting to pay a lot of attention. And look, the, the AI genie is out of the bottle. It's not— we're not going to go back in. Right. But that said, it's not too much to me as much about not slowing things down, but shifting our attention to investing in the safety piece, right? Because there are portions of it that requires the attention. There's a lot to be done from the past, but requires the attention. And I think being able to actually take that energy and devote to it, I think is going to make those models significantly better.

>> It's funny, I watch some of the mainstream programs on TV and I read a lot, and the perspective on AI, and it kind of bothers me, is like, oh, that group of people tech people are controlling AI. I saw an actor on one of the shows kind of really kind of laying into the tech industry. I'm like, well, it's not just them. There's a lot of other people involved. But the question that I ask is rhetorical. If you inject intelligence into something, a network edge that you just pointed out.

>> Yeah.

>> What happens if you inject intel? So I think there's a right to be concerned. Guardrails and keep watching it. You don't want to let chaos take over, certainly. But yeah, I think we're going to have an experimentation period. We have to identify that. So that's noble. But there's a real question to ask. What does it mean to inject AI intelligence into a process, into a device? You're at the center of it. How would you look at that? How do you frame that?

>> Well, we look at it, you know, two pieces. There's the, you know, kind of frontier model and the race towards AGI. And then we have infrastructure on the infrastructure side, and that's getting smarter, too. And in Summit, we really focus on all the challenges around kind of creating safe, secure and cost-effective infrastructure. And so when I look at that, I think about one of the biggest challenges of infrastructure is making sure that it's sustainable and making sure that businesses aren't in a bubble. Right. It's all about producing infrastructure that has good long-term economic, economic value and allows you to actually consistently build. And that's kind of what we're thinking about. Drive that payback down, you know, time to payback down, drive the ROI up and making sure that people can actually invest on this side of the infrastructure in a sustainable and long-term way.

>> Yeah, I love that, that, that infrastructure side. Let's go there for a second because you brought this up earlier. I want to go back to it because I think it's one of the most nuanced, under-talked-about topics. Yeah, certainly in the tech circles it's talked about a lot. Disaggregated serving. Yeah, that's one of your killer use cases for Seminova where, you know, everyone loves NVIDIA. Hey, give me the GPUs. Everyone knows there's scarcity there. Now there's still other— now compute's booming with agents. So there's still infrastructure. But disaggregated serving really solves one of the major constraints.

>> Yeah.

>> Which is the volume of data and the lack of network coherency around managing it because prefill is the prompt and then the decode is what happens after on the math side. So you get math and math talking to math.

>> Yeah.

>> What does that tell us? Because that is a constraint that we see in other areas. Yeah. You mentioned disaggregated infrastructure. Everyone's going to have nodes. That's disaggregated. Yeah, this serving there too, maybe on the edge. But what is that disaggregated serving point to? It's not just bolting on accelerators. It really is a paradigm architectural shift.

>> Well, it's a precursor to kind of what I think is going to be a broad, long-term view of data centers, which is heterogeneous computing. And so you're going to be able to mix and match different technologies to run what you need for AI. And so you can actually have different chips running Prefill, different chips running decode and frankly different chips running applications that the agent is going to call. And so with Summit Nova, you know, we're squarely in the decode side of it. And so we're able to actually take older infrastructure. So you have an H100, now Bs will soon be older as well. You know, if you think about kind of the infrastructure that people have invested already, how do we actually extend the life? And you can actually disaggregate it, put it in Summit Nova rack and suddenly you've extended the value of that infrastructure for another 2, 3 years. And so that's an important thing for people to think about because that amortization of the infrastructure is something that a lot of people are really concerned about.

>> It really speaks to your economic opportunity for Seminova because we were just talking on our QPOD last, last episode about how in the history of the computer hardware industry, people would always want forward pricing. Now they're locking in pricing because they think it's going to go up. They think that the price of the H100s actually are not going to decay as fast. If anything, might even go up. Yeah, because of the innovation happening around it.

>> Yeah. Yeah. What interesting times where, you know, you got a convergence of many things, a convergence of these models and the performance, this unlimited demand, you know, this buildout that's incredible. And then you've got the supply chain constraints coming in. And so those three things actually coming together is actually driving a very, very interesting economic model. But here's, here's what we can— here's something we can all agree on, something we can all agree on when it comes to inference. Everybody wants to see the cost of inference come down. Right. And so whether, whether it's better cost in the supply chain, whether that's, you know, being able to actually lower the energy or data center costs or that's actually finding more supply of different types of architectures, we can all agree that we need to drive the cost and the energy is the key function. All right.

>> What's up for you guys in the second half of the year? We got TheCUBE will be at Open Compute, Supercomputing, AWS re:Invent, a variety of other events, infrastructure events happening in Silicon Valley as well. Big announcement. We're not going to be there. We're here in New York. What's on your focus area for the second half of the year? Obviously, inference is super hot.

>> Yeah.

>> Is that still doubling down on inference? Is that still the name of the game for you guys?

>> Yeah.

>> What's the focus?

>> Yeah, no, it's all about economics, is all about payback for, for, for data centers being able to actually drive these services. We think that with the launch of SM50, which is coming very soon here in terms of first shipments out, I think people are going to see an incredible opportunity for them to actually take and drive profitability into their services. After many, many years of actually investing, investing, investing, they're going to be able to start matching up their infrastructure that they have today with some of the technologies and drive a much, much higher profitability into their business.

>> Businesses. Great to see you. Final question, give a taste of, for the folks watching, what the business performance has been for your company and what's your outlook.

>> know, look, I mean, we're enjoying it. We've done 6 quarters of quarter-over-quarter doubling, and I think we'll continue to see incredible demand. The business is growing really, really fast and really, you know, showing that inference is something that people have a lot of interest around, and it's the right time for people to invest.

>> It's going to get bigger when the edge and everything gets connected. Thanks for coming on The Cube.

>> Yeah, thanks for having—

>> John Furrier, this is the AI Factory series. This is one of the most popular series. The AI infrastructure continues to accelerate and expand. This is just AI factories, these big centers of data centers. They'll go to traditional data centers in the enterprise. You'll start to see the edge develop and wearables. We all have our Oura Ring or a Whoop. They're all going to be connected too. We'll bring that coverage to you on The Cube. Thanks for watching.

Rodrigo Liang, SambaNova Systems

search

>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, co-hosting with Dave Vellante, my co-host. Hello, I'm John Furrier with The Cube. We are here at The Cube's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. This is part of The Cube's NYSE Wired program and open community. We have back on The Cube, Cube alumni back for another appearance, just like a regular contributor. Rodrigo Liang, co-founder and CEO of Seminova. Part of our AI Factory series, one of our most popular series we started 2 years ago and really has been the precursor to the AI infrastructure boom. Very good to see you. Thanks for coming back on TheCUBE. I think Gemma talked to you the last 2 times when I was in California. Thanks for coming back on.

>> Yeah, thanks for having me. What an important time for us to be in this, in the AI industry.

>> We've seen each other now for a couple of years as part of this new NYC Wire community at events and on TheCUBE. Some significant changes in the past year. I mean, inference obviously A lot of insiders saw that early. The mainstream saw agents now kicking in. Coding was great, brought that in. Agent, Agentic brings up a whole nother paradigm shift around the role of the resource to service agents, what inference means there. So you start to see a whole shift. What has been the biggest change for you guys besides the billion dollars in funding you guys just closed, which we covered on Silicon Angle? That's validation. But what's been the biggest change this year?

>> Well, look, inference is now the economic center of AI and people are trying to figure out how to make that investment sustainable. And so as we went from training to inference, now people are thinking about how to deploy, deploy with sustainability, deploy with good energy consumption and all of those things. But most importantly, how to deploy in a way that they can get financial payback, right? We can't continue to borrow money forever without returns. And so being able to drive good economic payback for their investors is going to be an important part of data centers and actually as they move into inference.

>> Yeah, and people know that tokens equals revenue. That's well understood. Now we're starting to hear conversations about modeling out revenue growth. I think we're starting to see benchmarks now. We're saying this gigawatt equals this in revenue, this megawatt equals this in revenue. It's starting to— people start to quantify. And even a couple of years ago, I think Jensen Huang of NVIDIA said no one's spreadsheeted it out was his word. But yeah, that's the financial modeling. What have you seen there as CEO as a CEO and running your business, you're involved in a lot of these economic conversations, not just from a deployment standpoint, but the payback, a lot of these financial conversations.

>> Yes.

>> Energy and money are the two factors.

>> Yeah, exactly. Exactly. Well, the energy, the energy part, the data center cost and the capital acquisition of infrastructure, those on the cost side. On the revenue side, you can also dial that up. And so you can think about why people are driving speed like Samodova. We can generate speed because speed gets you more throughput. But more important than that is also speed at concurrence. You know, can you get many, many users getting the same speed at the same time because you want ultimate total output, total throughput per rack divided by that cost structure.

>> You guys have been at this for almost a decade now. We were talking before we came on camera about Hot Chips, which is the Stanford event that's well known as total nerd fest. It's Nerd Nation at Stanford, as everyone knows. But this is like the, the alpha, the state-of-the-art engineers really working on the next generation of accelerated technology. The conversation that I got away from that was there's a lot of work going on around processor called just processor generically, whatever unit you wanna call it, and memory, obviously HBM, and now you got solid state. But this kind of reminds me of the '90s when you had to use memory management utilities to swap in and out, but at such bigger scale. What has been the big thing for you guys now as you look at your roadmap How are you guys optimizing? Because everyone wants to squeeze as much tokens out of that energy, out of that processing, the relationship with the data to memory. These are all now part of the hardcore engineering vernacular.

>> Yeah, exactly. Well, ultimately you have these chips and these chips are constrained by what the technology can give you, both in terms of compute and the memory. And so for someone over— we're really focused on driving the total output. At the lowest cost and lowest power. And so if you can do that and before the session, we start talking about how people are measuring their compute in terms of megawatts and, and their revenue in terms of megawatt. Well, if you take that megawatt and you can actually put more racks in and each rack produces more tokens, that's how you generate more revenue per megawatt. Right. And so if you, if you take that chip, the same chip, and you increase the total output per chip, you're going to do a lot better when it comes to payback.

>> What are some of the things you guys got going on that you could share on Momentum side? Because that really is where everyone's looking at. You talk about some of the economics of buildout, but when you're in operations, you're building, operating, investing. Everyone's doing those three things at the same time.

>> Yeah.

>> On the upside, what, what do you see coming out? What do you guys have now? What's, what's the momentum?

>> Now, look, there are three incredibly exciting use cases that we see with some of the technology. One, people who have already deployed a lot of NVIDIA gear, we're partnering with NVIDIA on this with something called disaggregated inference. And what you're able to do is take NVIDIA gear make that partner with a Samanova gear and in that construct disaggregate the front, front part called pre-fill and the decode part using Samanova and generate 3x the total throughput. So existing hardware of NVIDIA can jump 3x in total throughput just by putting someone on our rack next to it. And that's a really, really exciting development for people who already have their gear and trying to get better economics. The other one that I think we're starting to see a lot of really exciting use case for is this distributed data center, existing data centers, air-cooled, and being able to actually use brownfield data centers that don't require new investment, new, you know, breaking ground, new liquid cooling, and just put some amount of air-cooled technology into it and then get significant advantage just by actually delivering faster at a lower cost, lower power.

>> So two major things that you said there. One is if I bought gear, call it gear, that's the term we use a lot, but a lot of systems Normal, normal depreciation would have that losing value.

>> Yeah.

>> Now you got value creation. So the price, the value of that gear is actually more critical. So that's good leverage. Yeah, that's a good use case.

>> Exactly.

>> Great.

>> Exactly.

>> One signs up for that all day long, I'm sure.

>> Exactly.

>> The other one is this new paradigm around disaggregated serving.

>> Yeah.

>> Which is also a precursor into disaggregated infrastructure.

>> Yeah.

>> Because what you just said is essentially adding new, new nodes out there. Data center nodes to be AI factories, basically, without all the requirements for the megawatts, gigawatt data centers that takes years to build.

>> Yeah, that's right. I mean, one of the biggest challenges we're seeing in the world today is how quickly can we stand up gigawatt data centers. And, and that's going to be more and more challenging, you know, as, as we think about the needs that we have and how much it impacts the energy grid. And so if we can actually reuse existing infrastructure around the world, existing infrastructure in the United States, where you already have energy allocated, spaces allocated, it's air-cooled, and you can deploy infrastructure to have state-of-the-art AI running on it. I think that's going to be an incredibly valuable way to actually get AI available to everybody at a much, much lower cost.

>> So does the word edge of the network go away when you have distributed computing paradigm where you have large data center nodes like an AI factory in Mega Texas or whatever it is?

>> Yeah.

>> And If you just got a cell tower that's got a building and power and network connectivity, that's air-cooled in there, that's an edge. That's still a node on the network, but it's technically an edge. So smaller factory configurations.

>> We actually call— yeah, we think of it as these massive kind of buildouts where you have huge gigawatt data centers. That's going to continue because people have to train models and the necessity for large clusters continue to exist. Now you go into what I'm calling distributed data centers. So take the 50 top metropolitan cities and Vista, for example, is a partner that building out these distributed data centers in existing brownfield data centers, re-energizing existing infrastructure with existing power, existing cooling, and then running that using someone over here. Now, edge, edge is the next thing I'm super excited, super excited, because if you think about true edge, which is where our mobile towers are, is where our users are, robotics, all these different things, there is still another wave coming where very, very localized computing is going to be serving very localized use cases of AI. And that's to me the true edge. And they need high performance, incredible high performance, ultra low latency, because now you're starting to talk about robotics and you talk about kind of the things that people want to use every day. And frankly, as a population, our patience is not very high. And so I want to get your thoughts.

>> I want to get you— what about robotics? I want to bring up safety because the number one conversation in all of our physical AI robotics series that we're running, Safety is the number one conversation. Security is in there, but I'd say number 1B, basically. 1A would be safety because robotics, you got to have a safe car, you got to have a safe robot. Yeah, that's really obvious. You go, okay, all the safety on AI conversation seems to be a tempest in a teapot because it's like people are working on safety.

>> Yeah.

>> What's your thoughts on all this negativity around safety? You got half the world being like, okay, I don't really understand what, but I hate it. I'm afraid. They don't understand. Then you have the people who understand, go, well, this is one of the best revolutions of all time in the computer industry. So you got— it's 50/50.

>> Yeah, yeah. No, it's understandable. Look, I think you look at a technology as transformative as this and you saw it over the weekend with the Frontier models and now, you know, you know, and more broadly around the model side that safety is important and it's a great, great wake-up call for all the leaders to start thinking about the fact that we've invested in the R&D of the models, but how do we invest in the safety and the guardrails around kind of how we use it. And so that's incredibly important. I think you're going to continue to see attention to that. And we saw this in early internet days where safety and security kind of became, well, something that was in the forefront of people's minds and entire industries got created from it.

>> And so, you know, I have a lot of respect for Dario, but I do think that he's not the poster child for safety when it comes to AI. I think he's— he's no noble effort to lay out his concerns when the air company's got a great track record in terms of the ethics. So, you know, give them the props for that. But there are many people in the industry that actually have been through this before and have actually done it.

>> Yeah.

>> I've seen where you had let chaos reign, rein in the chaos situations.

>> Yeah.

>> What do you think that he could learn from what Dario and others that are now aware of this could learn from the history of how Innovation can be chaotic and then reined in. Yeah, that's Andy Grove's favorite expression. Yeah, let chaos reign and rein in the chaos.

>> Yeah.

>> Look what happened with Intel under his regime and under Gordon Moore.

>> Yeah. Well, I mean, look, it starts with emphasis and attention. I think, you know, we're, we're at a time where the industry is starting to pay a lot of attention. And look, the, the AI genie is out of the bottle. It's not— we're not going to go back in. Right. But that said, it's not too much to me as much about not slowing things down, but shifting our attention to investing in the safety piece, right? Because there are portions of it that requires the attention. There's a lot to be done from the past, but requires the attention. And I think being able to actually take that energy and devote to it, I think is going to make those models significantly better.

>> It's funny, I watch some of the mainstream programs on TV and I read a lot, and the perspective on AI, and it kind of bothers me, is like, oh, that group of people tech people are controlling AI. I saw an actor on one of the shows kind of really kind of laying into the tech industry. I'm like, well, it's not just them. There's a lot of other people involved. But the question that I ask is rhetorical. If you inject intelligence into something, a network edge that you just pointed out.

>> Yeah.

>> What happens if you inject intel? So I think there's a right to be concerned. Guardrails and keep watching it. You don't want to let chaos take over, certainly. But yeah, I think we're going to have an experimentation period. We have to identify that. So that's noble. But there's a real question to ask. What does it mean to inject AI intelligence into a process, into a device? You're at the center of it. How would you look at that? How do you frame that?

>> Well, we look at it, you know, two pieces. There's the, you know, kind of frontier model and the race towards AGI. And then we have infrastructure on the infrastructure side, and that's getting smarter, too. And in Summit, we really focus on all the challenges around kind of creating safe, secure and cost-effective infrastructure. And so when I look at that, I think about one of the biggest challenges of infrastructure is making sure that it's sustainable and making sure that businesses aren't in a bubble. Right. It's all about producing infrastructure that has good long-term economic, economic value and allows you to actually consistently build. And that's kind of what we're thinking about. Drive that payback down, you know, time to payback down, drive the ROI up and making sure that people can actually invest on this side of the infrastructure in a sustainable and long-term way.

>> Yeah, I love that, that, that infrastructure side. Let's go there for a second because you brought this up earlier. I want to go back to it because I think it's one of the most nuanced, under-talked-about topics. Yeah, certainly in the tech circles it's talked about a lot. Disaggregated serving. Yeah, that's one of your killer use cases for Seminova where, you know, everyone loves NVIDIA. Hey, give me the GPUs. Everyone knows there's scarcity there. Now there's still other— now compute's booming with agents. So there's still infrastructure. But disaggregated serving really solves one of the major constraints.

>> Yeah.

>> Which is the volume of data and the lack of network coherency around managing it because prefill is the prompt and then the decode is what happens after on the math side. So you get math and math talking to math.

>> Yeah.

>> What does that tell us? Because that is a constraint that we see in other areas. Yeah. You mentioned disaggregated infrastructure. Everyone's going to have nodes. That's disaggregated. Yeah, this serving there too, maybe on the edge. But what is that disaggregated serving point to? It's not just bolting on accelerators. It really is a paradigm architectural shift.

>> Well, it's a precursor to kind of what I think is going to be a broad, long-term view of data centers, which is heterogeneous computing. And so you're going to be able to mix and match different technologies to run what you need for AI. And so you can actually have different chips running Prefill, different chips running decode and frankly different chips running applications that the agent is going to call. And so with Summit Nova, you know, we're squarely in the decode side of it. And so we're able to actually take older infrastructure. So you have an H100, now Bs will soon be older as well. You know, if you think about kind of the infrastructure that people have invested already, how do we actually extend the life? And you can actually disaggregate it, put it in Summit Nova rack and suddenly you've extended the value of that infrastructure for another 2, 3 years. And so that's an important thing for people to think about because that amortization of the infrastructure is something that a lot of people are really concerned about.

>> It really speaks to your economic opportunity for Seminova because we were just talking on our QPOD last, last episode about how in the history of the computer hardware industry, people would always want forward pricing. Now they're locking in pricing because they think it's going to go up. They think that the price of the H100s actually are not going to decay as fast. If anything, might even go up. Yeah, because of the innovation happening around it.

>> Yeah. Yeah. What interesting times where, you know, you got a convergence of many things, a convergence of these models and the performance, this unlimited demand, you know, this buildout that's incredible. And then you've got the supply chain constraints coming in. And so those three things actually coming together is actually driving a very, very interesting economic model. But here's, here's what we can— here's something we can all agree on, something we can all agree on when it comes to inference. Everybody wants to see the cost of inference come down. Right. And so whether, whether it's better cost in the supply chain, whether that's, you know, being able to actually lower the energy or data center costs or that's actually finding more supply of different types of architectures, we can all agree that we need to drive the cost and the energy is the key function. All right.

>> What's up for you guys in the second half of the year? We got TheCUBE will be at Open Compute, Supercomputing, AWS re:Invent, a variety of other events, infrastructure events happening in Silicon Valley as well. Big announcement. We're not going to be there. We're here in New York. What's on your focus area for the second half of the year? Obviously, inference is super hot.

>> Yeah.

>> Is that still doubling down on inference? Is that still the name of the game for you guys?

>> Yeah.

>> What's the focus?

>> Yeah, no, it's all about economics, is all about payback for, for, for data centers being able to actually drive these services. We think that with the launch of SM50, which is coming very soon here in terms of first shipments out, I think people are going to see an incredible opportunity for them to actually take and drive profitability into their services. After many, many years of actually investing, investing, investing, they're going to be able to start matching up their infrastructure that they have today with some of the technologies and drive a much, much higher profitability into their business.

>> Businesses. Great to see you. Final question, give a taste of, for the folks watching, what the business performance has been for your company and what's your outlook.

>> know, look, I mean, we're enjoying it. We've done 6 quarters of quarter-over-quarter doubling, and I think we'll continue to see incredible demand. The business is growing really, really fast and really, you know, showing that inference is something that people have a lot of interest around, and it's the right time for people to invest.

>> It's going to get bigger when the edge and everything gets connected. Thanks for coming on The Cube.

>> Yeah, thanks for having—

>> John Furrier, this is the AI Factory series. This is one of the most popular series. The AI infrastructure continues to accelerate and expand. This is just AI factories, these big centers of data centers. They'll go to traditional data centers in the enterprise. You'll start to see the edge develop and wearables. We all have our Oura Ring or a Whoop. They're all going to be connected too. We'll bring that coverage to you on The Cube. Thanks for watching.