In this interview from theCUBE + NYSE Wired: AI Factories, Rodrigo Liang, co-founder and chief executive officer of SambaNova, joins theCUBE's John Furrier to discuss why inference has become the economic center of AI. Liang explains how the industry is shifting from pure infrastructure buildout to demonstrating sustainable financial payback, balancing energy consumption with revenue generation. He details SambaNova's focus on maximizing token output per rack at the lowest cost and power, and outlines how a partnership with NVIDIA using disaggregated inference can triple throughput on existing hardware deployments.
The conversation also explores how SambaNova is extending the life of brownfield data centers, deploying air-cooled infrastructure to avoid the years-long buildout required for new gigawatt facilities. Liang addresses growing public anxiety around AI safety, arguing the industry's attention should shift toward investing in guardrails rather than slowing innovation. He breaks down disaggregated serving as a precursor to heterogeneous computing, where different chips handle prefill and decode to extend the economic life of aging GPU infrastructure. Liang also previews the upcoming SN50 launch, which he expects will help data centers convert years of investment into profitability. With the company posting six consecutive quarters of quarter-over-quarter doubling, he offers a clear signal of where AI infrastructure is headed next: from massive centralized factories toward distributed data centers and, eventually, true edge computing serving robotics and everyday connected devices.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. If you don’t think you received an email check your
spam folder.
Sign in to AI Factories - Data Centers of the Future.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Register for AI Factories - Data Centers of the Future
Please fill out the information below. You will receive an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for AI Factories - Data Centers of the Future.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. If you don’t think you received an email check your
spam folder.
Sign in to AI Factories - Data Centers of the Future.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open the link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: AI Factories - Data Centers of the Future
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: AI Factories - Data Centers of the Future. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Rodrigo Liang, SambaNova Systems
In this interview from theCUBE + NYSE Wired: AI Factories, Rodrigo Liang, co-founder and chief executive officer of SambaNova, joins theCUBE's John Furrier to discuss why inference has become the economic center of AI. Liang explains how the industry is shifting from pure infrastructure buildout to demonstrating sustainable financial payback, balancing energy consumption with revenue generation. He details SambaNova's focus on maximizing token output per rack at the lowest cost and power, and outlines how a partnership with NVIDIA using disaggregated inference can triple throughput on existing hardware deployments.
The conversation also explores how SambaNova is extending the life of brownfield data centers, deploying air-cooled infrastructure to avoid the years-long buildout required for new gigawatt facilities. Liang addresses growing public anxiety around AI safety, arguing the industry's attention should shift toward investing in guardrails rather than slowing innovation. He breaks down disaggregated serving as a precursor to heterogeneous computing, where different chips handle prefill and decode to extend the economic life of aging GPU infrastructure. Liang also previews the upcoming SN50 launch, which he expects will help data centers convert years of investment into profitability. With the company posting six consecutive quarters of quarter-over-quarter doubling, he offers a clear signal of where AI infrastructure is headed next: from massive centralized factories toward distributed data centers and, eventually, true edge computing serving robotics and everyday connected devices.
play_circle_outlineInference as Economic Core: Energy Efficiency, Data Center Savings, and Token-Based MW-to-Revenue Models
replyShare Clip
play_circle_outlineDisaggregated Rack Pairing for Triple Inference Throughput Without Megawatt Buildouts in Air‑Cooled Brownfield Data Centers
replyShare Clip
play_circle_outlineExtending Accelerator Life via Disaggregated Heterogeneous Computing: Mixing Chips for Prefill, Decode, Application with SambaNova Racks
replyShare Clip
play_circle_outlineSambaNova SN50 Launch Boosts Inference Demand and Profitability After Six Quarters of Doubling
In this interview from theCUBE + NYSE Wired: AI Factories, Rodrigo Liang, co-founder and chief executive officer of SambaNova, joins theCUBE's John Furrier to discuss why inference has become the economic center of AI. Liang explains how the industry is shifting from pure infrastructure buildout to demonstrating sustainable financial payback, balancing energy consumption with revenue generation. He details SambaNova's focus on maximizing token output per rack at the lowest cost and power, and outlines how a partnership with NVIDIA using disaggregated inferenc...Read more
exploreKeep Exploring
How are organizations approaching the economics of AI inference deployment — including energy efficiency, data‑center costs, throughput/concurrency, and revenue modeling?add
What are three exciting use cases for this technology?add
How will heterogeneous computing change data centers, and how can SambaNova's approach help extend the value of existing AI infrastructure (e.g., H100s, B200s)?add
What is your focus area for the second half of the year — are you continuing to double down on inference?add
>> Palo Alto Studio Connection, Silicon Valley and Wall Street. I'm John Furrier, co-hosting with Dave Vellante, my co-host. Hello, I'm John Furrier with theCUBE. We are here at theCUBE's NYSE studio. Of course, we have our Palo Alto studio connecting Silicon Valley to Wall Street. This is part of theCUBE's NYSE Wired program and open community. We have back on theCUBE, Cube alumni back for another appearance, just like a regular contributor. Rodrigo Liang, co-founder and CEO of SambaNova. Part of our AI Factory series, one of our most popular series we started 2 years ago and really has been the precursor to the AI infrastructure boom. Very good to see you. Thanks for coming back on theCUBE. I think Gemma talked to you the last 2 times when I was in California. Thanks for coming back on.
Rodrigo Liang
>> Yeah, thanks for having me. What an important time for us to be in the AI
John Furrier
>> industry.We've seen each other now for a couple of years as part of this new NYSE Wired community at events and on theCUBE. Some significant changes in the past year. inference obviously A lot of insiders saw that early. The mainstream saw agents now kicking in. Code gen was great, brought that in. Agentic brings up a whole nother paradigm shift around the role of the resource to service agents, what inference means there. So you start to see a whole shift. What has been the biggest change for you guys besides the billion dollars in funding you guys just closed, which we covered on SiliconANGLE? That's validation. But what's been the biggest change this year?
Rodrigo Liang
>> Well, look, inference is now the economic center of AI and people are trying to figure out how to make that investment sustainable. And so as we went from training to inference, now people are thinking about how to deploy, deploy with sustainability, deploy with good energy consumption and all of those things. But most importantly, how to deploy in a way that they can get financial payback, right? We can't continue to borrow money forever without returns. And so being able to drive good economic payback for their investors is going to be an important part of data centers and actually as they move into inference.
John Furrier
>> Yeah, and people know that tokens equals revenue. That's well understood. Now we're starting to hear conversations about modeling out revenue growth. I think we're starting to see benchmarks now. We're saying this gigawatt equals this in revenue, this megawatt equals this in revenue. It's starting to— people start to quantify. And even a couple of years ago, I think Jensen Huang of NVIDIA said no one's spreadsheeted it out was his word. But yeah, that's the financial modeling. What have you seen there as a CEO and running your business, you're involved in a lot of these economic conversations, not just from a deployment standpoint, but the payback, a lot of these financial conversations.
Rodrigo Liang
>> Yes.
John Furrier
>> Energy and money are the two factors.
Rodrigo Liang
>> Yeah, exactly. Exactly. Well, the energy part, the data center cost and the capital acquisition of infrastructure, those are on the cost side. On the revenue side, you can also dial that up. And so you can think about why people are driving speed like SambaNova. We can generate speed because speed gets you more throughput. But more important than that is also speed at concurrency. can you get many, many users getting the same speed at the same time because you want ultimate total output, total throughput per rack divided by that cost structure.
John Furrier
>> You guys have been at this for almost a decade now. We were talking before we came on camera about Hot Chips, which is the Stanford event that's well known as total nerd fest. It's Nerd Nation at Stanford, as everyone knows. But this is like the alpha, the state-of-the-art engineers really working on the next generation of accelerated technology. The conversation that I got away from that was there's a lot of work going on around processor call it just processor generically, whatever unit you wanna call it, and memory, obviously HBM, and now you got solid state. But this kind of reminds me of the '90s when you had to use memory management utilities to swap in and out, but at such bigger scale. What has been the big thing for you guys now as you look at your roadmap How are you guys optimizing? Because everyone wants to squeeze as many tokens out of that energy, out of that processing, the relationship with the data to memory. These are all now part of the hardcore engineering vernacular.
Rodrigo Liang
>> Yeah, exactly. Well, ultimately you have these chips and these chips are constrained by what the technology can give you, both in terms of compute and the memory. And so for SambaNova— we're really focused on driving the token output. At the lowest cost and lowest power. And so if you can do that and before the session, we start talking about how people are measuring their compute in terms of megawatts and their revenue in terms of megawatt. Well, if you take that megawatt and you can actually put more racks in and each rack produces more tokens, that's how you generate more revenue per megawatt. Right. And so if you take that chip, the same chip, and you increase the token output per chip, you're going to do a lot better when it comes to payback.
John Furrier
>> What are some of the things you guys got going on that you could share on Momentum side? Because that really is where everyone's looking at. You talk about some of the economics of buildout, but when you're in operations, you're building, operating, investing. Everyone's doing those three things at the same
Rodrigo Liang
>> time.
John Furrier
>> Yeah.On the upside, what, what do you see coming out? What do you guys have now? What's the momentum?
Rodrigo Liang
>> Now, look, there are three incredibly exciting use cases that we see with some of the technology. One, people who have already deployed a lot of NVIDIA gear, we're partnering with NVIDIA on this with something called disaggregated inference. And what you're able to do is take NVIDIA gear and partner that with SambaNova gear and in that construct disaggregate the front part called pre-fill and the decode part using SambaNova and generate 3x the total throughput. So existing hardware of NVIDIA can jump 3x in total throughput just by putting a SambaNova rack next to it. And that's a really, really exciting development for people who already have their gear and are trying to get better economics. The other one that I think we're starting to see a lot of really exciting use cases for is this distributed data center, existing data centers, air-cooled, and being able to actually use brownfield data centers that don't require new investment, new, breaking ground, new liquid cooling, and just put SambaNova air-cooled technology into it and then get significant advantage just by actually delivering faster at a lower cost, lower power.
John Furrier
>> So two major things that you said there. One is if I bought gear, call it gear, that's the term we use a lot, but a lot of systems Normal depreciation would have that losing value.
Rodrigo Liang
>> Yeah.
John Furrier
>> Now you got value creation. So the price, the value of that gear is actually more accretive. So that's good leverage. Yeah, that's a good use case.
Rodrigo Liang
>> Exactly.
John Furrier
>> Great.
Rodrigo Liang
>> Exactly.
John Furrier
>> One signs up for that all day long, I'm sure.
Rodrigo Liang
>> Exactly.
John Furrier
>> The other one is this new paradigm around disaggregated serving.
Rodrigo Liang
>> Yeah.
John Furrier
>> Which is also a precursor into disaggregated infrastructure.
Rodrigo Liang
>> Yeah.
John Furrier
>> Because what you just said is essentially adding new nodes out there. Data center nodes to be AI factories, basically, without all the requirements for the megawatts, gigawatt data centers that takes years to build.
Rodrigo Liang
>> Yeah, that's right. one of the biggest challenges we're seeing in the world today is how quickly can we stand up gigawatt data centers. And that's going to be more and more challenging, as we think about the needs that we have and how much it impacts the energy grid. And so if we can actually reuse existing infrastructure around the world, existing infrastructure in the United States, where you already have energy allocated, spaces allocated, it's air-cooled, and you can deploy infrastructure to have state-of-the-art AI running on it. I think that's going to be an incredibly valuable way to actually get AI available to everybody at a much, much lower cost.
John Furrier
>> So does the word edge of the network go away when you have distributed computing paradigm where you have large data center nodes like an AI factory in maybe Texas or whatever it is?
Rodrigo Liang
>> Yeah.
John Furrier
>> And if you just got a cell tower that's got a building and power and network connectivity, that's air-cooled in there, that's an edge. That's still a node on the network, but it's technically an edge. So small form factor configurations.
Rodrigo Liang
>> We actually call— we think of it as these massive buildouts where you have huge gigawatt data centers. That's going to continue because people have to train models and the necessity for large clusters continues to exist. Now you go into what I'm calling distributed data centers. So take the 50 top metropolitan cities and VC2, for example, is a partner that's building out these distributed data centers in existing brownfield data centers, re-energizing existing infrastructure with existing power, existing cooling, and then running that using SambaNova here. Now, edge is the next thing I'm super excited about, because if you think about true edge, which is where our mobile towers are, is where our users are, robotics, all these different things, there is still another wave coming where very, very localized computing is going to be serving very localized use cases of AI. And that's to me the true edge. And they need high performance, incredible high performance, ultra low latency, because now you're starting to talk about robotics and you talk about the things that people want to use every day. And frankly, as a population, our patience is not very high. And so I want to get your thoughts.
John Furrier
>> I want to get your— what about robotics? I want to bring up safety because the number one conversation in all of our physical AI robotics series that we're running, Safety is the number one conversation. Security is in there, but I'd say number 1B, basically. 1A would be safety because robotics, you got to have a safe car, you got to have a safe robot. Yeah, that's really obvious. You go, okay, all the safety on AI conversation seems to be a tempest in a teapot because it's like people are working on safety.
Rodrigo Liang
>> Yeah.
John Furrier
>> What's your thoughts on all this negativity around safety? You got half the world being like, okay, I don't really understand it, but I hate it. I'm afraid. They don't understand. Then you have the people who understand, go, well, this is one of the best revolutions of all time in the computer industry. So you got— it's 50/50.
Rodrigo Liang
>> Yeah, yeah. No, it's understandable. Look, I think you look at a technology as transformative as this and you saw it over the weekend with the Frontier models and how, you know, and more broadly around the model side that safety is important and it's a great, great wake-up call for all the leaders to start thinking about the fact that we've invested in the R&D of the models, but how do we invest in the safety and the guardrails around kind of how we use it. And so that's incredibly important. I think you're going to continue to see attention to that. And we saw this in early internet days where safety and security kind of became, well, something that was in the forefront of people's minds and entire industries got created from it.
John Furrier
>> And so I have a lot of respect for Dario, but I do think that he's not the poster child for safety when it comes to AI. I think he's made a noble effort to lay out his concerns and the AI company's got a great track record in terms of the ethics. So give them the props for that. But there are many people in the industry that actually have been through this before and have actually done it.
Rodrigo Liang
>> Yeah.
John Furrier
>> I've seen where you had let chaos reign, rein in the chaos situations.
Rodrigo Liang
>> Yeah.
John Furrier
>> What do you think that he could learn from what Dario and others that are now aware of this could learn from the history of how Innovation can be chaotic and then reined in. Yeah, that's Andy Grove's favorite expression. Yeah, let chaos reign and rein in the chaos.
Rodrigo Liang
>> Yeah.
John Furrier
>> Look what happened with Intel under his regime and under Gordon Moore.
Rodrigo Liang
>> Yeah. Well, look, it starts with emphasis and attention. I think we're at a time where the industry is starting to pay a lot of attention. And look, the AI genie is out of the bottle. It's not— we're not going to go back in. Right. But that said, it's not so much to me about slowing things down, but shifting our attention to investing in the safety piece, right? Because there are portions of it that require attention. There's a lot to be done from the past, but requires the attention. And I think being able to actually take that energy and devote to it, I think is going to make those models significantly better.
John Furrier
>> It's funny, I watch some of the mainstream programs on TV and I read a lot, and the perspective on AI, and it kind of bothers me, is like, oh, that group of tech people are controlling AI. I saw an actor on one of the shows really kind of laying into the tech industry. I'm like, well, it's not just them. There's a lot of other people involved. But the question that I ask is rhetorical. If you inject intelligence into something, at the network edge, as you just pointed
Rodrigo Liang
>> out.
John Furrier
>> Yeah.What happens if you inject intel? So I think they're right to be concerned. Guardrails and keep watching it. You don't want to let chaos take over, certainly. But yeah, I think we're going to have an experimentation period. We have to identify that. So that's normal. But there's a real question to ask. What does it mean to inject AI intelligence into a process, into a device? You're at the center of it. How would you look at that? How do you frame that?
Rodrigo Liang
>> Well, we look at it in two pieces. There's the frontier model and the race towards AGI. And then we have infrastructure on the infrastructure side, and that's getting smarter, too. And at SambaNova, we really focus on all the challenges around creating safe, secure and cost-effective infrastructure. And so when I look at that, I think about one of the biggest challenges of infrastructure is making sure that it's sustainable and making sure that businesses aren't in a bubble. Right. It's all about producing infrastructure that has good long-term economic value and allows you to actually consistently build. And that's kind of what we're thinking about. Drive that payback down, time to payback down, drive the ROI up and making sure that people can actually invest on this side of the infrastructure in a sustainable and long-term way.
John Furrier
>> Yeah, I love that infrastructure side. Let's go there for a second because you brought this up earlier. I want to go back to it because I think it's one of the most nuanced, under-talked-about topics. Yeah, certainly in the tech circles it's talked about a lot. Disaggregated serving. Yeah, that's one of your killer use cases for SambaNova where, everyone loves NVIDIA. Hey, give me the GPUs. Everyone knows there's scarcity there. Now there's still other— now compute's booming with agents. So there's still infrastructure. But disaggregated serving really solves one of the major constraints.
Rodrigo Liang
>> Yeah.
John Furrier
>> Which is the volume of data and the lack of network coherency around managing it because prefill is the prompt and then the decode is what happens after on the memory side. So you get math and memory talking to each other.
Rodrigo Liang
>> Yeah.
John Furrier
>> What does that tell us? Because that is a constraint that we see in other areas. Yeah. You mentioned disaggregated infrastructure. Everyone's going to have nodes. That's disaggregated. Yeah, there's serving there too, maybe on the edge. But what does that disaggregated serving point to? It's not just bolting on accelerators. It really is a paradigm architectural shift.
Rodrigo Liang
>> Well, it's a precursor to what I think is going to be a broad, long-term view of data centers, which is heterogeneous computing. And so you're going to be able to mix and match different technologies to run what you need for AI. And so you can actually have different chips running Prefill, different chips running decode and frankly different chips running applications that the agent is going to call. And so with SambaNova, we're squarely in the decode side of it. And so we're able to actually take older infrastructure. So you have an H100, now B200s will soon be older as well. if you think about the infrastructure that people have invested already, how do we actually extend the life? And you can actually disaggregate it, put it in a SambaNova rack and suddenly you've extended the value of that infrastructure for another 2, 3 years. And so that's an important thing for people to think about because that amortization of the infrastructure is something that a lot of people are really concerned about.
John Furrier
>> It really speaks to your economic opportunity for SambaNova because we were just talking on our theCUBE Pod last episode about how in the history of the computer hardware industry, people would always want forward pricing. Now they're locking in pricing because they think it's going to go up. They think that the price of the H100s actually is not going to decay as fast. If anything, it might even go up. Yeah, because of the innovation happening around it.
Rodrigo Liang
>> Yeah. Yeah. We're in interesting times where you got a convergence of many things, a convergence of these models and the performance, this unlimited demand, this buildout that's incredible. And then you've got the supply chain constraints coming in. And so those three things actually coming together is actually driving a very, very interesting economic model. But here's what we can— here's something we can all agree on, something we can all agree on when it comes to inference. Everybody wants to see the cost of inference come down. Right. And so whether it's better cost in the supply chain, whether that's being able to actually lower their energy or data center costs or that's actually finding more supply of different types of architectures, we can all agree that we need to drive the cost down, and the energy is the key factor. All right.
John Furrier
>> What's up for you guys in the second half of the year? We got theCUBE will be at Open Compute, Supercomputing, AWS re:Invent, a variety of other events, infrastructure events happening in Silicon Valley as well. Big announcement. We're not going to be there. We're here in New York. What's your focus area for the second half of the year? Obviously, inference is super hot.
Rodrigo Liang
>> Yeah.
John Furrier
>> Is that still doubling down on inference? Is that still the name of the game for you guys?
Rodrigo Liang
>> Yeah.
John Furrier
>> What's the focus?
Rodrigo Liang
>> Yeah, no, it's all about economics, it's all about payback for data centers being able to actually drive these services. We think that with the launch of SN50, which is coming very soon here in terms of first shipments out, I think people are going to see an incredible opportunity for them to actually take and drive profitability into their services. After many, many years of actually investing, investing, investing, they're going to be able to start matching up their infrastructure that they have today with some of the technologies and drive a much, much higher profitability into their business.
John Furrier
>> Businesses. Great to see you. Final question, give us a taste of, for the folks watching, what the business performance has been for your company and what's your
Rodrigo Liang
>> outlook.we're enjoying it. We've done 6 quarters of quarter-over-quarter doubling, and I think we'll continue to see incredible demand. The business is growing really, really fast and really showing that inference is something that people have a lot of interest around, and it's the right time for people to
John Furrier
>> invest.It's going to get bigger when the edge and everything gets connected. Thanks for coming on theCUBE.
Rodrigo Liang
>> Yeah, thanks for having—
John Furrier
>> John Furrier, this is the AI Factories series. This is one of the most popular series. The AI infrastructure continues to accelerate and expand. This isn't just AI factories, these big centralized data centers. They'll go to traditional data centers in the enterprise. You'll start to see the edge develop and wearables. We all have our Oura Ring or a WHOOP. They're all going to be connected too. We'll bring that coverage to you on theCUBE. Thanks for watching.
John Furrier