SC25 | Jonathan Ballon, Iceotope & Ace Stryker, Solidigm

Clips
More from SC25

Jonathan Ballon

CEO

Iceotope

Ace Stryker

Director of AI & Ecosystem Marketing

Solidigm

play_circle_outline Revolutionizing Data Center Cooling: Iceotope Tackles Heat Challenges with Precision Solutions and Water-Free Technology

play_circle_outline Rethinking Data Center Cooling: Addressing Heat Challenges with High-Performance Chips Beyond Traditional Air Cooling Methods

play_circle_outline Iceotope aims for a future without forced air and minimized water dependence.

play_circle_outline Revolutionizing Data Centers: The Shift to 100% Liquid-Cooled Storage Solutions with Solidigm Partnerships

Info
Transcript

Jonathan Ballon, Iceotope & Ace Stryker, Solidigm

Jonathan Ballon

CEO Iceotope

Ace Stryker

Director of AI & Ecosystem Marketing Solidigm

In this interview from SC25, Jonathan Ballon, chief executive officer of Iceotope, and Ace Stryker, director of AI ecosystems and marketing at Solidigm, join theCUBE’s John Furrier and Jackie McGuire to discuss the critical thermal challenges facing modern data centers. As AI workloads drive density to unprecedented levels, the industry faces a "burning platform" where traditional air cooling can no longer keep pace. Ballon details how Iceotope’s precision liquid cooling technology addresses this by eliminating fans and reducing power consumption by over 80%.... Read more

explore Keep Exploring

What are the main challenges currently facing the AI industry related to data centers? add

What is the role of density in the cooling systems of data centers, particularly in relation to the heat generated by high-performance chips like GPUs? add

What innovations and fundamental principles guided the development of the technology portfolio described? add

What are the implications of hardware design on the performance and cooling requirements of SSDs in future data centers? add

bolt Powered by CUBE AI

Jonathan Ballon, Iceotope & Ace Stryker, Solidigm

search

John Furrier

>> Welcome back everyone to theCube's live coverage here at Supercomputing 2025. I'm John Furrier, host of theCube with Jackie McGuire, our lead analyst to break down all the data center trends, cooling, power. We've got two great guests, Jonathan Ballon, CEO of Iceotope, Ace Stryker, director of AI Ecosystems and Marketing at Solidigm. Jonathan, welcome back to theCube. You're a Cube alumni. Ace, you're a pro now. Thanks for coming on.

Ace Stryker

>> Thank you. - Thanks for having me back, John.

Jackie McGuire

>> We didn't even give Ace a break.

John Furrier

>> You guys are doing great work. Describe what you guys are doing. You got the precision cooling solution. Talk about what you guys do.

Jonathan Ballon

>> Yeah, so we have been really anticipating this moment that we're in right now with so much heat being generated in the data center. We're really focused on three problems that the AI industry is facing. One is how do you get the compute as close to the data as possible? The second is how do you cool that compute without using any water? And the third is how do you do so using just an extraordinarily less amount of power? Those three issues seem to be just dominating the headlines and we solve all three of them.

John Furrier

>> Power and cool has been around for a while, so the energy discussion is great. Cooling is a dense problem. Density is becoming a big reason why Nvidia has been so successful. And while all these systems are working, how does the density piece fit into it and how does it all work for you guys?

Jonathan Ballon

>> Yeah, so density is a key issue. So, the industry has evolved from this place where everything was cooled with air, and of course that was a burning platform, particularly as Nvidia and others were increasing the thermal demands on the data center with increasingly higher performance chips. And so, the industry over the last couple of years has been adopting these cold plates to focus on the hottest part of the server, which of course is the GPU, which works extremely well. The challenge is that the rest of the IT in the data center, the power supply, the network, the storage, all of these other things are also increasing in heat proportionately with that compute and the air that cools the rest of that IT is struggling to keep up and we're going to reach a point in a couple of years where it will no longer be sufficient. And that's where Iceotope comes in.

Jackie McGuire

>> We're really close to that right now, right? So.

Jonathan Ballon

>> We're very close to that.

Jackie McGuire

>> So, historic, and you can correct me if I'm wrong. So, originally data centers, we had lots of mechanical cooling with chillers, and then we realized, "Okay, but if we build them in places that are cold and dry, we can do just evaporative cooling. " So, we got away from chillers, everybody started doing evaporative, but then it was like, "Oh, no, now we have to build chillers in places that are humid, so we can't use evaporative cooling. " So, we've kind of been on this seesaw and now, I mean, I've heard you've got three, four chillers linked together and you still can't get the water cold enough to create the mechanical cooling that you need. So, we're pretty much at that tipping point right now. And then the more we pack into these AI racks, the more heat they produce. To your point, I mean, so how do you see liquid cooling kind of fitting into this as we continue to evolve in density?

Jonathan Ballon

>> Well, we've been at this for a decade and we've been working with hyperscalers and silicon companies during that period of time to develop this portfolio of technology in anticipation of this moment. And we had some fundamental principles that we held in place, and one of them was imagine a future where we would have no forced air. And in that environment, of course, you don't need cold air because there is no air. And also in that environment, how do we imagine one where there's no need for water? So, when you design around those core principles, you eliminate a lot of the things that you're talking about. And so, in the solution that we have, we don't need any incremental water. The water that we use is circulated through the facility water loop and it just recirculates. And the water that we use is actually high temperature water. So, it can be as high as 55 degrees Celsius when it comes into the IT stack and the power reduction, because we're not cooling the air, we reduce the amount of power necessary by over 80%. It's very significant.

John Furrier

>> That's a huge point because most people think about cooling as keep everything coo so it doesn't fail. You guys actually are talking about performance increases with that because what you just said, if you lower the power-

Jonathan Ballon

>> Yeah. That's right. So-

John Furrier

>> How about that dynamic? Because most people think, "Hey, just keep it running."

Jonathan Ballon

>> Yeah. And you talked about density, and I want to get back to that question as well, but when we talk about performance, so throttling is of course going to take place if the entire IT is at risk of frying the server, so you're going to throttle your compute if those other components that we talked about start getting too hot. But when everything is operating at a consistent temperature and you have no risk of overheating the server. And what that means is if right now GPUs are operating on average 60%, 70%, maybe in best case scenarios, they're operating at 80% utilization. If I can come in and tell a data center operator that you can operate at 100% utilization. In fact, we have customers that are overclocking at all times with zero risk of overheating. That difference is a... That's a CapEx opportunity of incremental compute that's just sitting there waiting to be operationalized.

Jackie McGuire

>> It's also addressing some of the regulatory issues we're seeing, right? So you're now seeing communities push back on power use, on water use, and so as hyperscalers are looking to expand and build more data centers, I'd imagine that not needing billions of gallons of water every day, not needing additional watts and watts of power is probably going to really open up the number of places that these can be built, right?

Jonathan Ballon

>> Well, that's true, and especially when you start seeing communities rejecting some of these proposals to put large data centers in that environment. But also we're seeing it's not just with the hyperscalers, but you're seeing this spread of compute moving out to the edge. So, of course, the conversation largely focuses on the hyperscalers and these massive builds because that's where the action is. But whether it's a telco environment or it's putting compute on the edge of a Formula One race or in a genomics research lab, we're seeing people want to take compute, whether it's for sovereignty reasons, for latency reasons, whatever, and bring it as close to the source of that data as possible because it's edge inference capability. And in those environments where we don't have a fan, there's no noise. So, imagine running a high- performance compute node at the edge in your lab silently.

John Furrier

>> Yeah. You can read a book. - It's a paradigm shift.

John Furrier

>> Talk about the relationship with Solidigm because I've seen demos of their product when came out. They're kind of purpose-built to help you guys out. Talk about the relationship between how their hardware design on the SSDs works for you.

Jonathan Ballon

>> Absolutely. So, I mean, again, if you think with these design principles in mind and you fast-forward to a world not too distant future where we're building 600 kilowatt megawatt-plus racks that are 100% liquid-cooled, data centers that are going to be built in that 2027, 2028 timeframe will not have air built into the architecture. Everything in the IT stack will need to be liquid-cooled, including the storage. And so, we've been working with Solidigm to build high- density, 100% liquid-cooled storage solutions. And so the partnership that we've built is perfect for this, and Solidigm is ahead of the pack in terms of anticipating this moment so that their portfolio will be ready for that future. And we actually have products in development today that customers can deploy in some of those edge use cases that we talked about and be ready for and anticipate the high-scale, high-density-

John Furrier

>> Edge use cases as in terms of design of data centers or edge, edge like the network edge or both.

Jonathan Ballon

>> Yeah, both. - It sounds Solidigm think about this, so how do you think about, and how do you partner when you're partnering with other companies to try to think ahead three years? If you had asked me three years ago, you said you've been predicting this for 10 years, but if you had asked me three years ago what this partnership should look like, I probably would've told you the wrong thing. So, how do you think about that as you're building partnerships into the future about where AI is going and how liquid cooling plays into that?

Ace Stryker

>> Yeah, it's very important to find partners like Iceotope that have the same mentality in terms of skating to where the puck is going to be, right? The reality is if you're shopping today for an AI compute server, even if it's a "liquid- cooled" server, there's going to be fans in the box still for one reason, and that's to cool The storage. Storage is sort of the last holdout in terms of the key components that are not attached to the same sort of cooling infrastructure as everything else. And that's because it's been good enough up to this moment in time for PCIe Gen 4 and Gen 5 devices where you're pulling maybe 20 or 25 watts per SSD, air can keep up for the most part, right? But as you look ahead to Gen 6, which is going to be adopted in big numbers going into 2026 and Gen 7, beyond, well, you might be talking about a 40, 50, 60 watt power budget per SSD. It's a very different proposition and all of a sudden air is no longer sufficient. And so, it is sort of a delayed transition, but it's a similar transition to what we saw with the compute resources. The CPUs and GPUs are liquid cooled now by necessity, storage is moving in that direction next because the last thing you want is throttling on your SSDs. We've taken some measurements in our lab, but we've seen is you can run an SSD as hot as about 77 degrees Celsius before it starts to throttle, but at that moment you lose half your data throughput. And if you're not able to bring the temps down, you go up to 79, all of a sudden you've lost almost all of the ability to get data out of those drives. And so those are GPUs that are essentially starving for data at that point if the SSD thermals aren't managed properly,

John Furrier

>> What's the throttle? Is it data or heat or both? What's that throttle?

Ace Stryker

>> It's based on heat. It's based on the temperature of the heat generating components, the ASICs and the controller on the board. And so, there's throttle steps built into the firmware of most of these devices that will say, "All right, we got to pull the E-brake now before this thing melts. " Which folks wonder, "Okay, well, how big of a deal is it really if my storage devices get hot? " But when you start to explore the relationship between the storage thermals and the GPU utilization, which is a metric that a lot of folks measure and pay very close attention to, it's a lot easier to understand the impact there. I actually just did this survey. I looked across, there's an industry consortium called ML Commons that publishes AI benchmarking data, and they've got a storage test that gives you two metrics when you run it. One is the bandwidth, the throughput of the data coming out of the storage subsystem. The other is how many GPUs can you keep busy? And when you chart all the results, 150 different results from across the industry, bunch of different companies, it's an extremely linear relationship. In other words, you can't really squeeze more GPU utilization out of the system without increasing your storage bandwidth at the same rate. And if you're throttling, it's going to hurt both of those axes. You're going to end up taking a hit to your bottom line of business metrics that you care about in AI with things like tokens per second and this kind of stuff. And so, once you kind of connect those dots, it's a little easier to explain where we're headed with storage and why it's so important to be, as you say, thinking three years ahead on this stuff.

John Furrier

>> Jonathan, on your deep dive, talk about the business side of it because you mentioned some of the technology you're doing, what are some of the real- world deployments right now? What are you seeing today in terms of real-world examples? What are some of the ROIs? You mentioned some of that CapEx numbers. That's a huge value obviously, to get that of GPU utilization.

Jonathan Ballon

>> Yeah. - How do deployments look like?

Jonathan Ballon

>> Yeah, so deployments I would say really break down into a couple of categories. So, the hyperscaler market is something that we will intercept with a technology portfolio that we're in development with OEMs and partners with Solidigm today that will intercept in the 2027 timeframe. The portfolio that we have today are, for example, we launched something yesterday called the KulBox. This is a fully integrated, it's a KulBox. It's a fully integrated single SKU. It's a rack that has a switch, storage and compute server capability all in one that can go into a genomics research lab for example. We have one in Europe that's using it for DNA sequencing. They've got a research team that has a row of these things on-prem that their researchers are using for genomics research. You can imagine this going onto the track at a Formula One race. We have some of our fluid vendors are doing oil exploration and they're putting these on oil rigs out in the middle of the ocean to do analysis of seismic conditions and other things around oceanography. We have satellite companies that are using satellite imagery and doing local assessment of that. So anything where you've got high volumes of, in some cases, real-time data that you don't want to send back to the cloud, or you've got sovereignty reasons where you want to keep it on-prem for security or other reasons, this is the perfect use case.

Jackie McGuire

>> Yeah. Well, I think we've been talking all year basically about this kind of return to hybrid computing and the fact that it wasn't really meant for everybody to pivot all the way into the cloud. It's just the pricing model of the cloud made it a lot easier to buy things there than to go through the procurement process internally. So, I agree and I think to see if I'm an enterprise and I want to train a couple of petabytes into a model, it probably makes a lot more sense to have a KulBox that I can do that in than to go all the way to the cloud if that's all I actually need. And I think one of the things that is appreciated a lot in high-performance computing, but not necessarily outside yet, is the storage piece. And we talked about this in our opening. It's like we're talking about how these agents are going to do all of these things, but every single thing they do has to be stored somewhere and logged and compliance and regulatory requirements are going to make sure that that happens. And that's only going to exponentially increase as the memory continues to build on itself. So, I'm really glad that you guys are doing this work because I'm not necessarily sure people really understand that all of this stuff has to be stored somewhere and the storage needs to be just as performative as the compute, otherwise you're like an athlete that hasn't trained their reactant. I was just thinking about it when you're talking how they say that a batter has to decide whether to swing the bat or not before the pitcher actually throws. It's the same kind of with compute and storage. If you can't store it at the same velocity that you're computing at, it doesn't really work, right?

Ace Stryker

>> That's right. - The question on the energy piece, they're going to put it out there. Does the sustainable equations come into place? Does everyone has sustainable benchmarks? I mean, or is that kind of like, "We'll get to that later, we're just going to knock down the winds now?"

Jonathan Ballon

>> I mean, to be really direct, I think the goal of the company years ago was to sell into that sustainability ESG message. And the reality is that for the last couple of years, that's kind of a secondary or tertiary message. People don't really care about that right now. They care about time to compute. How fast can I get compute online? And if availability of water and power is in the way of that, then they care about it. But the cost of those things hasn't really been a concern yet. And certainly the regulatory issue has not been a concern.

John Furrier

>> So, you'd say that the massive trend that's driving your demand is more compute now or?

Jonathan Ballon

>> Time to compute. So let's talk about that for a second. Tier two CSPs and neoclouds are a customer of ours as a category because the supply chain has been largely consumed by the big spenders of the hypercloud. So, just getting access to raised floor supply, cabling, like some of these basic commodity items like HVAC. So, the ability for them to roll in a KulBox onto cement where they don't need a raised floor where they can plug it into a 240 outlet, they don't need water or a huge amount of additional power, time to compute is time to revenue for them. And if they can get close to 100 or more of utilization from that compute at a much lower total cost of ownership, that's competitive.

John Furrier

>> That's actually a huge point. Jackie and I were talking about this in our opening around the skills needed for the build-outs, like what was the stat? How lines, how much cables run?

Jonathan Ballon

>> 200 miles of cable- - 200 miles of cable, so all these things got to get built.

Jonathan Ballon

>> That's right. - Yeah.

John Furrier

>> Okay. You guys, what's the biggest thing that's happening right now in your business? Is it the demand, trying to deal with the demand? Is it skilled labor on your side? Is it product? What are some of the challenges you guys have right now that you are working through?

Ace Stryker

>> Well, we have what's been referred to as a high quality problem now at Solidigm which is that we are trying to make it as fast as we can. We're selling every bit that comes off the assembly line. And so, there continues to be a need to drive for innovation, both in performance and capacity. And those are kind of... We have separate product lines that optimize for one or the other of those. But ultimately, particularly with some of the stuff that you mentioned, agentic AI is such a tremendous driver of data intensity. You take one human prompt, please resolve this JIRA ticket for me or whatever, and the agent may enter this iterative loop and generate dozens or hundreds of incremental prompts. And if we're talking about rich media forms, audio and video, I mean, all that stuff multiplies. We talked a little in the last segment about KV cache, right? And that's all got to go somewhere. So, there's all these trends that just point to more and more demand for storage, more and more data that's got to live somewhere. And I think the trick going into next year is really going to be around efficiency, continue to be around efficiency, but it's going to be about not just maximum density in a storage device, but it's going to be about how can we intelligently manage placement of data between the non-volatile memory, the system memory, the high bandwidth memory on the GPU, right? I think there's some fertile grounds for innovation there that haven't been fully explored yet that can lead to significant improvements in the kind of efficiency that customers are clamoring for, to Jonathan's point.

John Furrier

>> Guys, really appreciate you coming on. We got a wrap, but I already asked you this question in the last segment. There's puns involved in this one, Jonathan, so be ready. What's the coolest thing you're working on right now besides the KulBox because that's-

Jonathan Ballon

>> Oh, the KulBox is cool, obviously, but-

Jackie McGuire

>> It's like R-22.

Jonathan Ballon

>> I would say the coolest thing we're working on is the power supply to support these one megawatt racks and doing 100% liquid-cooled power supply to enable what Nvidia is driving for the future, I think is just really extremely interesting from a technology point of view, but also what it unlocks for the industry is probably the funnest thing that we're working on.

Jackie McGuire

>> Awesome. - AI scale requires radical thinking and execution. Thanks for coming on. Congratulations on success. Thanks for coming on. Appreciate it. I'm John Furrier with Jackie McGuire here, live here at Supercomputing. We'll be right back after this short break.