theCUBE + NYSE Wired: AI Factories - Data Centers of the Future | Seshu Madhavapeddy, Frore Systems

Seshu Madhavapeddy, Frore Systems

In this interview from theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event, Glean co-founder and CEO Arvind Jain joins theCUBE’s John Furrier to unpack what’s really working in enterprise AI today and what comes next. Jain explains why knowledge access remains the first successful AI use case at scale and how Glean’s enterprise search brings AI into everyday work. He details the past year’s lessons with AI agents – from the need for guardrails, security, evaluation and monitoring to democratizing agent building so business owners (not just data scientists) can create production-grade agents. The conversation dives into Glean’s vision of the enterprise brain powered by an enterprise graph, highlighting the importance of deep context, human workflows and behavior to reduce “noise” and drive outcomes. Jain outlines core building blocks – hundreds of enterprise integrations and a growing actions library – that let agents securely read company knowledge and take actions across systems (e.g., CRM updates, HR tasks, calendar checks). He discusses how organizations are standing up AI Centers of Excellence, prioritizing “top 10–20” agents across functions like engineering, support and sales, and why a horizontal AI data platform that unifies structured and unstructured data – accessed conversationally and stitched together via standards like MCP – sets the foundation for AI factory-scale operations. Looking ahead, Jain says Glean’s upgraded assistant is evolving from reactive tool to proactive companion that anticipates tasks and accelerates productivity.

Share this session

Clips
More from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future

Seshu Madhavapeddy

CEO

Frore Systems

play_circle_outline Frore Systems' Seshu on Thermal Innovations: Building the Hidden Thermal Stack Powering AI Factories

play_circle_outline Liquid cooling replacing traditional air cooling for high-density GPUs

play_circle_outline Frore LiquidJet Cold Plate Improves Liquid Cooling 75%, Cuts Weight; GPUs 8°C Cooler Deliver Higher Token Throughput and Longer Lifespan

play_circle_outline Air Jet MEMS Active Cooling Tackles Edge AI Thermal Challenges in Tablets, Phones, Cameras and Gateways

Info
Transcript

Seshu Madhavapeddy, Frore Systems

Seshu Madhavapeddy

CEO Frore Systems

In this interview from theCUBE + NYSE Wired: AI Factories - Data Centers of the Future, Seshu Madhavapeddy, founder and chief executive officer of Frore Systems, joins theCUBE's John Furrier to discuss why thermal management is the hidden foundational layer powering the next generation of AI factories. Madhavapeddy introduces the concept of a "thermal stack" — the often-overlooked infrastructure responsible for removing the enormous heat generated by modern GPU clusters. He explains why delivering power to a data center is only half the equation, likening AI ... Read more

explore Keep Exploring

What is the "thermal stack" in AI infrastructure, and how does Frore Systems improve it to enhance compute and software performance? add

How are AI data centers addressing GPU heat removal, and why can innovations in thermal solutions (for example NVIDIA’s Kyber rack versus a standard NVL72 Vera Rubin rack) allow much higher GPU density and performance? add

How does liquid cooling with cold plates work on an NVIDIA Vera Rubin compute tray (what components generate heat and how is it removed), and what advantages does Frore Systems' cold plate provide? add

What demand are you seeing for your data‑center liquid‑cooling products, and how are you addressing thermal‑management challenges in edge AI and ultracompact devices? add

bolt Powered by CUBE AI

Seshu Madhavapeddy, Frore Systems

search

John Furrier

>> Welcome back here. I'm John Furrier, host of theCUBE. We are here at our NYSC Cube Studios, part of the NYSC Wired program and community. Of course, we have our Palo Alto studio connecting Silicon Valley and Wall Street. Technology is the market. This is part of our popular AI Factory series. We're talking to the leaders who are making it happen, bringing all the AI infrastructure to the table, enabling massive change and accelerated value in AI, AI applications, AI data. Seshu is here. He's the founder and CEO of Frore Systems. Seshu, thanks for coming on theCUBE remote. What a busy day here at the NYSE. It was very loud now. It's kind of the market's closed here in the East Coast. Thanks for coming in remote. Appreciate it.

Seshu Madhavapeddy

>> Thank you, John. Thank you for having me.

John Furrier

>> You know, the AI side is super hot on the infrastructure. There's so much going on and it's no longer servers. We've been documenting this for years now on theCUBE. They're larger-scale supercomputers and a lot's going on in them. You guys are playing a big role in that, leading that. Take a minute to explain what you guys are offering and how you fit in to the accelerated computing narrative.

Seshu Madhavapeddy

>> Definitely, John. Lots happening in AI factories. And most of the focus so far has been on software companies that do AI foundational models, agents, open claw on what have you. And there's also a lot of focus on the computing infrastructure, what we could call the thermal stack, where you've got GPUs, CPUs, networking, memory. And that basically, the compute stack, enables the software stack that runs on it. But one of the best catch secrets in the AI factories is that there's a third layer, a foundational layer that sits below the compute stack that we call the thermal stack. And that's basically all the infrastructure that you need to remove the heat that's generated by the compute stack. And so Frore Systems, and I'm the CEO and founder of Frore Systems, we innovate and we offer disruptive new products that enhance the capabilities of the thermal stack. And in doing so, we make the compute stack run better and the compute stack, in turn, allows the software stack to run better.

John Furrier

>> You know, this is one of the big revelations. And if you go back maybe five years ago, not even five, maybe 10, it's even worse. No one talked about energy and heat outside of the servers that needed fans and cooling. And if you look at what Jensen talks about at GTC, we're going to see a lot of it this year in a week is the five-layer cake, energy, chips, infrastructure models, and applications. Energy is at the bottom of the stack. It's almost like a Maslow's hierarchy of needs. So you got energy and chips. This is where the performance is being squeezed out of the market because if you don't get that right, the energy, which is needed to power everything, is the envelope. It's the bounding function for performance. Share your vision on this and why that's important and what people should know in this modern era, why this is so important.

Seshu Madhavapeddy

>> So that's a great segue, John. When people talk about energy, they always focus on power, delivering power to the GPUs in the data center. But there's actually two sides to the coin. It's not just enough to deliver the power. You also need to have very efficient way of removing heat. It's almost the other side of the coin. If you look at an AI data center, it's actually a thermal power station in reverse. In thermal power stations, you use energy or you use heat energy to generate power. Whereas, in a data center, you've got power going in and it all converts into heat. Now, if you don't have a very efficient way of removing that heat, then you're in a boatload of trouble. So when you talk about energy, you need to address both the issues. How do we deliver power to the data center? And how do we most efficiently remove the heat? And Frore Systems is a company that's innovating in thermal technologies that are going to do a great job of removing the heat in the AI factories.

John Furrier

>> Can you talk about the scope and the order of magnitude of the problems that emerge? Just paint a picture. The heat's in there because of the energy and the GPUs are throwing off heat. There's all kinds of different techniques, but if you don't get it right, it can be catastrophic. So can you share or paint a picture of some of the things that goes wrong if they don't get this right and what you guys are doing to solve that?

Seshu Madhavapeddy

>> Great. Sure. At the fundamental level, when you have heat that is generated by the GPU, if you don't remove it quickly, then basically the GPU gets hotter and hotter. And there's a point where if it gets too hot, you have a runaway condition, it might actually melt the GPU. So you need to efficiently remove the heat because as you pull power into the GPU and you run the AI models, all the energy that is sucked in is converted into heat. So if you don't remove the heat efficiently, the GPU will get hotter and hotter, and then ultimately it's going to become useless. So removing the heat efficiently is extremely important.

John Furrier

>> Yeah. No, it's just-

Seshu Madhavapeddy

>> And the way you do that nowadays in AI factories is by using liquid cooling. For a long time, as you mentioned, they were using air-based cooling using fans, and that ran out of steam very, very quickly. So people have now moved to liquid cooling. But even with liquid cooling, you have limits about how much heat you can remove if you are not very smart in terms of innovating the liquid cooling infrastructure. You got to make sure that it continues to improve and keep pace with the amount of heat that's being generated by every new generation of GPU. And let me tell you how important heat removal is. If you look at the latest proposal from NVIDIA, they've got the Rubin GPU, but right along Rubin GPU at CES, NVIDIA talked about a new rack configuration called Kyber. And what's different from Kyber compared to your standard NVL72 Vera Rubin rack? The only difference is actually thermal solution because if you have a very efficient thermal solution, you can have twice the number of GPUs in the same rack. And so the same GPU infrastructure, compute infrastructure, you can actually double the performance of the rack by just innovating on thermal solutions. So that's the kind of order of magnitude of improvement a great thermal solution can deliver.

John Furrier

>> You kind of beat me to the punch there, Seshu. I was going to ask about that because, first of all, the catastrophic failure of melting GPUs means a lot of money gets wasted away, but also things don't work. But I want to talk about that piece of the rack you mentioned about why that's important. The big trend is the density of these clusters is actually a benefit for NVIDIA. The way they're doing their designs, where these large scale systems are built, they're basically supercomputers. That's what I call them. People call them that. But the architecture is based upon density. It's almost reverse of what people think. That's going to require more than liquid cooling. Talk about why you guys are doing well here. What specifically in your product, where does it fit in? Because when I hear density, I think a lot of chips everywhere. Solid state memory, you got HBM close to the processors, you've got all these interconnects, and there's a lot of stuff going on. And they're all very densely packed by design. Talk about-

Seshu Madhavapeddy

>> That's right....

John Furrier

>> what you guys are doing, why it exists, and some of the traction and benefits.

Seshu Madhavapeddy

>> Yeah. So if you look at a compute tray, let's take an NVIDIA compute tray as an example in their Vera Rubin architecture. Each compute tray has four Rubin GPUs. It has two Vera CPUs. It has a BlueField DPU, and it also has several NICs, and it has a bunch of memory and two DC converters and voltage regulators that are actually drawing all this power and delivering it to the GPUs and the CPUs. So all that power that's pulled in and is delivered to the GPUs is converting into heat. So what you need to do is you need to look at each of these components on the compute tray and make sure that you have a very good infrastructure in place to be able to remove that heat. And a key element in a liquid cooling setup is what is called a cold plate. And the cold plate is basically something like this. It sits on top of each of the GPUs and the CPUs, and it has an inlet and an exit. And as water goes in and comes out of this cold plate, it's a mechanism inside which heat transfer happens, heat exchange happens. So as the water is going through the cold plate, it pulls the heat from the GPUs and cold water goes in and hot water comes out. And that's how you remove the heat. Now, there's a lot of science behind how you make these cold plates. The one I'm holding is from our company, Frore Systems, and this just happens to be 75% more efficient than the standard cold plates that are currently being used in the industry. So you have a cold plate that's 75% more efficient. That means that you can run the GPUs 75% faster, or alternatively, you can run them cooler. And if you're running a cooler GPU, you actually get more tokens per second for the same amount of power draw. And you also actually increase the lifespan of the GPU.

John Furrier

>> That's to my bounded energy point. That's the major advantage. Talk about the products that you guys sell. I really want to get into this because I think this is a real innovation. This makes the factories, one, more efficient and economical and higher performance. What are the products? Take us through the portfolio of what's working, what are the key, what are the hottest products? I don't mean that hot, but in terms of momentum, take us through the product line.

Seshu Madhavapeddy

>> So at Frore Systems, we are entirely focused on thermal solutions, and we are delivering products that solve the thermal problem from cloud to edge. So we have a whole bunch of products for the AI factory, starting with the LiquidJet Cold Plate that I was talking about. And this is, as I said, 75% better in heat efficiency compared to the standard cold plates that are currently used in AI factories. And also it's less than half the weight. So this is basically a standard cold plate that's used in the AI factory. And compared to that, the one we are offering is 75% more efficient and also less than half the weight. So we actually reduce the weight of the rack by offering our cold plate. Next, we are offering something even more ambitious. At this GTC, NVIDIA GTC, we are going to introduce our LiquidJet Nexus, which is an integrated solution that-

John Furrier

>> Oh, that's big. That's nice....

Seshu Madhavapeddy

>> yeah. It's an integrated solution that has multiple coal plates in one integrated solution that you can just slap on top of the compute tray and it removes all the heat in the computer tray with a hundred percent liquid cooling and it's super light and it completely eliminates all the hoses and connectors and manifolds. So it's a one integrated solution that simplifies and improves the performance of the compute tray. And more importantly, if you use this LiquidJet Nexus, your compute tray will be half the thickness. So you can double the number of compute trays that you have in one rack. So your compute density has improved by a hundred percent. That's what I meant earlier. Thermal solutions and innovation in thermal solutions can improve the compute density just as well as a new generation of NVIDIA GPUs can.

John Furrier

>> I love the Liquid-

Seshu Madhavapeddy

>> You can get=...

John Furrier

>> I love the Liquid-...

Seshu Madhavapeddy

>> twice the performance by going to the next generation GPUs, but within the same generation of GPU, you can get twice the performance by just using Frore Systems, LiquidJet Nexus instead of your standard cold plates that are currently used.

John Furrier

>> So you're delivering higher cooling capacity in the rack format, easy to deploy, less moving parts, connections. Is that right?

Seshu Madhavapeddy

>> And more efficient-

John Furrier

>> More efficient....

Seshu Madhavapeddy

>> because we are able to make sure that the GPUs run about eight degrees cooler. That's what it means when I say we are 75% more power efficient. And the GPU that runs eight degrees cooler is more efficient. It can generate more tokens per second for the same amount of power consumption.

John Furrier

>> Yeah. I think the new tagline for your company is it's cool and relevant. Very interesting. So take us through the demand side and the momentum of the business. There's a huge build-out going on. I'm seeing the numbers, just what we're reporting. Just on the data centers alone, just the monster data, big AI factories is pushing a trillion dollars. I was at MWC last week and just one carrier is looking at close to 90 billion refresh retrofit. So AI factories are coming. Today, I just wrote a big research report on it, dropped it last week. Hyperconverged Edge is happening. That's intelligence at the edge. You'll see towers, central offices essentially being retrofitted. I mean, they have power and they got energy and they got a cabinet or in some cases a building, but it's not your classic data centers. So talk about this idea of scale and the momentum you guys... Talk about the demand side that you guys are seeing. And take us through what you see the edge looking like.

Seshu Madhavapeddy

>> Well, the demand center on the data centers is phenomenal. It's staggering. So you can't produce this stuff fast enough. And especially when you've got stuff that's the best in the industry, the demand multiplies. So that's the kind of traction we are beginning to see in data centers for our LiquidJet and LiquidJet Nexus products. Across the industry, all the hyperscalers and data center service providers are demanding us to deliver them our LiquidJet and LiquidJet Nexus products for liquid cooling of data centers. But at the same time, that's not the only thing we do. We also offer thermal solutions for edge devices. Edge AI is also a reality. And when you have edge AI, basically what you need is more powerful associates and more powerful processors at the edge. And this is happening in ultracompact devices, whether you consider tablets or notebooks or smartphones or even smart cameras, these are extremely compact devices and they're going to have more and more AI in them. But as you have more AI in them, you need more computing. And as you have more computing, they're going to generate more heat. So if you don't have a very efficient way of removing the heat in the edge device, you have the same problem that I talked about earlier for data centers. So how do you increase the heat removal capability at the edge devices? Because these are all devices that are ultrasmall, ultracompact, and many cases, dustproof and water resistant. So they don't lend themselves for including a fan because fans are too bulky. And also these devices have to be quiet. Nobody wants a loud, personal device. So we came up with a product called Air Jet. This is a MEMS-based solid state active cooling chip. And you insert one of these Air Jet chips from Frore Systems into your ultracompact device, it doubles the performance of your device. Just like that because the world's best kept secret is that the processors and the SoCs in these devices are incredibly powerful already. In fact, the performance very often rivals desktops, but you're not able to realize that full performance in these devices because they're ultracompact and, therefore, their ability to remove heat is limited. And ultimately, you are limited in terms of how much performance you can derive from these very powerful processors based on how efficiently you're able to remove the heat. So by doubling the amount of heat you remove, you instantaneously double the performance of the device without changing anything else. And that's what we are offering for edge devices.

John Furrier

>> Oh, Seshu, you're sitting on a great opportunity. Congratulations. And we'll want to hear more at GTC next week. My final question for you is, what's your focus now? Obviously, you've got demand. I really appreciate you taking the time out of your busy day to share with us, but what is your focus? What's on your agenda? What's the North Star? What are you driving towards?

Seshu Madhavapeddy

>> Well, we've got the products that the market needs. So what we are currently entirely focused in is scaling our operations on growth. Manufacturing, supply chain, supply chain resiliency, geographic redundancy in terms of manufacturing facilities is where our focusing because we need to scale up in terms of delivering product at scale and meeting the demand that we're currently beginning to see in the industry, both for our liquid cooling solutions for the data center and edge-based solid state active cooling solutions for edge devices.

John Furrier

>> Well, the world appreciate what you do. You're making things smaller, faster, more performant, price points get lower in terms of the performance levels, more tokens per second. These AI factories are reality. They're starting in the big centers, going to go to the edge, probably Metro. A lot of headroom for you guys. Congratulations, and thank you for taking the time. I know you're busy. You got a lot of demand, people knocking on your door every day. Appreciate you sharing on theCUBE.

Seshu Madhavapeddy

>> Thank you, John. My pleasure. Thanks for having me on.

John Furrier

>> We'll see your team in San Jose at GTC next week, and thank you for your time.

Seshu Madhavapeddy

>> See you next week.

John Furrier

>> I'm John Fury, host of theCUBE. AI Factories is the beginning of an AI infrastructure movement that's continuing to accelerate the value of the applications that are coming on super fast. Again, it's accelerated market. AI is bringing effecting change across every vertical enabled by the large AI factories, big centers, now the edge all going to fill out. We'll hear more next week at NVIDIA. I'm John Furrier, host of theCUBE, here at the NYSC Wired program, a CUBE Original. Thanks for watching.