Media Week NYC theCUBE + NYSE Wired | Bernie Malouin, Jetcool

Clips
More from Media Week NYC theCUBE + NYSE Wired

Bernie Malouin

Founder & CEO

JETCOOL Technologies Inc.

play_circle_outline Innovative Liquid Cooling Designs for High-Powered Data Centers: Direct-to-Chip Solutions and Jet Targeting Arrays

play_circle_outline History of company, spun out of MIT, focus on high-powered devices and AI.

play_circle_outline Revolutionizing Global Tech: Venture-Backed Company Reducing Power Consumption with Liquid Cooling Solutions

Info
Transcript

Bernie Malouin, Jetcool

Bernie Malouin

Founder & CEO JETCOOL Technologies Inc.

John Furrier, co-founder of theCUBE, interviews Bernie Malouin, Founder and CEO of JetCool, discussing their liquid cooling solutions for data centers. JetCool's technology targets hotspots on CPUs and GPUs, saving electricity and enabling denser compute for AI applications. The company was founded six years ago, spun out of MIT, and is backed by investors like Bosch and Raptor Capital. They focus on hiring engineering talent and working with customers ranging from end users to chip makers. JetCool is expanding globally and investing in partnerships and new p... Read more

explore Keep Exploring

What is JetCool's value proposition and solution in relation to the increasing power demands in data centers? add

What is the background and focus of JetCool as a company? add

What engineering disciplines does the company typically look for in team members? add

bolt Powered by CUBE AI

Bernie Malouin, Jetcool

search

>> Hello, welcome back to theCUBE's coverage here at New York City, the New York Stock Exchange. My name is John Furrier, host of theCUBE, and co-founder with Dave Vellante. We're here in New York City, opening up our East Coast location on the floor of the New York Stock Exchange, where all the action in business technology come together. Bernie Malouin is here, Founder and CEO of JetCool, helping us all get better at horsepower in our servers and data centers, new AI systems. Bernie, thanks for coming on theCUBE. Appreciate you.

Bernie Malouin

>> Absolutely, John, thanks for having me.

>> So obviously, we cover a lot of liquid coolant, as a big part of the innovation right now, in making sure we have servers. Power and coolant, the two hottest areas. You're in one of them. So what does JetCool do? Let's set the table, set the context, explain kind of what you guys do.

Bernie Malouin

>> Yeah, absolutely, John. So we do liquid cooling for our data centers, direct-to-chip liquid cooling. And we are really involved in data centers that are looking to densify compute, especially those bringing on the latest generation of AI, where we power the latest generation of GPUs. And also, save about 20% of the data center's electricity.

>> So we all know that we're constrained. Anyone in the industry has been following the CapEx build out. It's been a tsunami of just spend. It's CapEx war, all the big cloud guys are doing it. But also, the data center workloads are now being retrofitted. You're seeing the data center vibe come back, but it's kind of cloud operations, which means the work Nvidia's done, and all the semiconductor guys have built better chips. Custom silicon now is going to be the rage. So you're seeing a lot more big iron systems, if you will, get built. And yeah, I use that metaphor to the mainframe days, big iron, but that's what's happening. The idea of stacking servers up on a rack on a data center, with a top of rack switch, is over. You're starting to see the new servers be systems with a zillion servers with a lot of interconnects. Again, this is the new architecture for GenAI. It's not just GPUs, it's TPUs, it's all PUs, it's XPUs. So a lot of components now are coming into this new system design, because the demand for AI is so hot, and the applications need this horsepower. Essentially, super computing has been democratized and available to the masses in a data center footprint. Okay, great. Hey, rah, rah. More cat videos and more hard problems to solve like climate change. But there's one couple problems, power and cooling. You don't want these things getting really hot. This is what you guys do. So take us through what your value proposition, what's your solution?

Bernie Malouin

>> Yeah, so absolutely, it's a great point. Because what we're seeing is, just a few years ago, the average rack power was less than 20 kilowatts. And today, within Nvidia's NVL 72, that's now 120 kilowatts. At JetCool, we're doing things that are 300 kilowatts in a single rack. So the power is going through the roof, and for all the reasons that you just mentioned. What we do at JetCool is we bring a specialized liquid cooling, where we're actually, our solutions attach on to the CPU or the GPU device, and we bring an array of fluid jets. And those jets are targeting the hotspots on the individual silicon. So thinking about all the XPUs that you mentioned, John, we have solutions that target the hotspots on each of those individually. Whether you're working with Intel, or AMD, or Nvidia, or some other solution. And by really focusing on the hotspots of that silicon, we can do a better job of cooling down.

>> And actually, a little prop here.

Bernie Malouin

>> Yeah, absolutely.

>> It's a demo?

Bernie Malouin

>> Yeah, this is a demo. This is actually a solution for a-

>> up the camera right there....

Bernie Malouin

>> for the H 100 product. This is one of our cooling modules for Nvidia H 100. Inside of this is really something that we call the jet plate. That's this guy here. So this whiz bang is creating those arrays of little tiny fluid jets. There's 700 little fluid jets that are targeting the hotspots on the processor.

>> So us old school guys and gals who build their PCs know what a heat sink is. Remember the old days, you'd keep that?

Bernie Malouin

>> That's right. Yeah.

>> It feels like it's a similar kind of configuration, but not really. Or is it the same footprint, but you're using your tech with the jet packs?

Bernie Malouin

>> Exactly right. So from the outside, think of it as exactly that. So it's the same kinds of things that people are familiar with. But inside, it's really different. Because inside, instead of channels, we have these arrays of tiny jets. And you can see very miniature jets that are perforated into that device. So you can see arrays of non-uniform nozzles that are being generated to target the hotspots on that silicon.

>> This is awesome engineering. Explain the secret sauce. How does it work? Take us through some of the intricacies.

Bernie Malouin

>> So at a core level, we're using jets that have flow that's perpendicular to those surface of the processor that you want to cool. A lot of other liquid cooling approaches just flow liquid over a surface, parallel to the surface, and you get these boundary layers that develop with slow-moving fluids. But for us, we take that fluid and we turn it 90 degrees. So we can surgically remove the heat from those hotspots with flow that's crushing the boundary layer reference.

>> So you're monitoring surface area and you identify it. Do you pulse it with some cooling? Is that how it works? I'm oversimplifying.

Bernie Malouin

>> Yeah, no. We do target those hot spots. There's not a lot that goes on in real time, because workloads can change very quickly. But we're mapping out those hotspots and we're targeting them with these clusters of jets.

>> All right, explain the company history, origination, mission, how it's evolved, how you guys got here.

Bernie Malouin

>> Yeah, so the company's about six years old at JetCool, and we actually spun out of MIT, where I've been working the technology for 10 to 12 years. It started, actually, in aerospace. So we were doing things for defense. And back then we were cooling thousand watt plus devices, when nobody cared about thousand watt devices.

>> You had one customer, the government.

Bernie Malouin

>> Yeah. So we've been doing that for a long time. And then we spun out to focus on what we thought was a real opportunity, not just to focus on high-powered devices. AI wasn't really a think back then. But also to do a better for the planet, because in the end, we help reduce the electricity consumption.

>> And so when you guys spun out of MIT, obviously, still was need for the providers with hardware to have some cooling. What's been the catalyst? What's been some of the key moments for you in the company?

Bernie Malouin

>> Yeah, well we've seen a lot of adoption in liquid cooling, really in the past two and a half years. It's really gone crazy. A lot of that's been driven by AI. It's been driven by densification of compute. It's also been driven by power budgets. So even as we think about, a lot of our customers in financial services, they have co-location space where they might only have 10 or 12 KW per rack. And by using our liquid cooling, they can get the higher powered chip sets, but they also reduce that power consumption of the server 15 or 20%. And what that means is that in a fixed power budget, every seven servers you can get an eighth one in for free.

>> And they have the math on that too. They know what that translates their business value.

Bernie Malouin

>> Exactly.

>> On your guys' journey, funding status, were you guys obviously funded? What's the plan there? How well are you guys doing? Are you guys looking for more capital? What's the business plan?

Bernie Malouin

>> Yeah, so we're a venture backed company. We have great investors including folks like Bosch, Raptor Capital. Incutel is an investor with the company. So we have a great backing that allows us to do some really great things in this space and advanced electronics.

>> What's some of the engineering guys you guys have? Are you looking for data people, physical engineering? What's some of the engineering disciplines that you guys tend to look for in your team?

Bernie Malouin

>> We have very specialized technology, so we're always looking for great engineering talent, especially mechanical engineering, aerospace engineering, some electrical engineering. Those are the key areas for us as we build out some very advanced fluid dynamics to cool these -

>> And who's your customers now, the manufacturers? Who's your target customers?

Bernie Malouin

>> Yeah, so we span all the way from end users in the financial services industry, for example, to OEMs. We have a great partnership with Dell. All the way up to the chip makers. And of course, we work with chip makers collaboratively, because we have to for mapping out hotspots too.

>> Talk about the Dell relationship. How's that going?

Bernie Malouin

>> We have a great partnership with Dell. So with Dell, we're able to combine our liquid cooling solutions with some custom software that they develop, to really optimize the efficiency of their servers. And we've had great success with that partnership and their customers reducing power consumption.

>> Dell's done a good job on the AI. They see the future. Michael Dell has pivoted successfully in every inflection point. When the web came along, they were doing mail order, then he goes direct online. I'm expecting the AI PC and these new servers to be very configurable, having that power envelope. To me, that's a big advantage right now. Where's the puck going? To use the analogy, "Skate to where the puck is going," by Wayne Gretzky. For you, as you look at your business and you're navigating the landscape, what's your 20-mile stare out in the landscape? Where do you want to be?

Bernie Malouin

>> Well, we love to follow the model of the great one, Wayne Gretzky. I think, as we look at the densification of compute, we see just the need to conserve power and get that same densification of compute out. You look at, Google's talking about gigawatt scale data centers. That's the power consumption of the City of San Francisco. So all of a sudden, if you can cool those and save 20% of the electricity, you could power a nearby city with the savings that you can have for some of these data centers. So we see an increased efforts on reducing the power consumption because of the stress on power budgets.

>> One of the things I want to get your thoughts on, and something I've been riffing on, and I want to riff with you, if you don't mind, obviously, Nvidia's the world, clearly the data centers target customer for you guys, and that's where the action is right now. But if you look at what's happening, we're almost going back to the client server revolution, because the data processing aspect of AI is that you got these edge devices. And you can move data around, and post data, co-locate data. You get more storage everywhere. There was like IBM, Dell, they all have good storage solutions. So, okay, but I can have a massive super computer like capability, which is what's happening now, whether it's cloud or on-prem. But I can also push content and have replica data, or have highly available data, and high availability at the edge. So edges could be more powerful devices, drawing power. So I imagine a world where you have all these connected devices getting more capabilities, hence more power requirements, and also would be smaller footprint for you guys. So as you see that future, that's more of a systematic power envelope.

Bernie Malouin

>> Yeah.

>> What's your thoughts on that? Because I think this is where also cooling might come into handy. That would change the form factor.

Bernie Malouin

>> Yeah, it's a really good point, John, because there's a couple of dynamics there. We're seeing increased interest in that, as you start to think about, not just training of core LLM sets, but you start to think about inference at the edge and moving that further out. And that poses different problems, because you're not going to have a giant liquid cooling infrastructure if you're working at the edge. And so, one of the things that we think is an enabling technology there, that JetCool offers, is our self-contained liquid cooling. So we build entire liquid cooling solutions systems that fit inside of a computer server box. So you can deploy them in any air-cooled environment. You don't need to make the investment. And that's a way that people are now pushing those high-power computing, those inference at the edge further out, without having to make significant investments.

>> I think distributed computing is going to come down to physical footprint challenges, that I've said it before, it was overlooked because you just put it in a data center.

Bernie Malouin

>> That's right.

>> Or in an office.

Bernie Malouin

>> That's right.

>> All right, what's next for you guys? What's on the plan for you? What are some of your goals?

Bernie Malouin

>> Oh gosh. Well, it's a busy time. I checked. Right? So we're undertaking a globalization effort now. So a lot of our business for the past six years has been domestic in the US. Obviously, a lot of resurgence internationally, including in Europe and in parts of Asia. So we are expanding our operations now globally.

>> As you sit in your executive and board meetings, what's your conversations like? What are you investing in?

Bernie Malouin

>> Well, we're investing in making liquid cooling easy to access. So we're investing in partnerships, we're investing in distribution, we're investing in new product offerings that allow that complete solution to be deployed anywhere, whether it's a large facility or whether it's an edge application.

>> Well, Bernie, I really appreciate you, and congratulations on great technology. We need it. I'll give you the last word. Talk to the camera, the folks watching, could be students to hire, engineers to hire, customers to work with. Give the pitch, give the plug for the company.

Bernie Malouin

>> Absolutely. No, thanks for having me, John. And it's a great time in this industry. We're always interested in hiring talented individuals in engineering that can help change the world and have a better impact by reducing power consumption.

>> All right, thanks so much.

Bernie Malouin

>> Thanks.

>> Thank you for joining theCUBE. Okay, I'm John Furrier here, the New York Stock Exchange. This is our Cube East location, bringing all the innovation as technology and Wall Street come together. This is where the action is here on the East Coast. Thanks for watching.