SC24 | Lauren Witter, Omni Services & Mitchell Knight, CooliT

Clips
News
More from SC24

Mitchell Knight

Director of Product Management

CoolIT Systems

Transforming high-performance computing with advanced cooling and supply chain innovations

High-performance computing, or HPC, is evolving to meet the growing demands of artificial intelligence, large-scale data processing and increased computing power.Central to this transformation is the development of advanced cooling technologies to manage the heat generated by high-density systems. Complementing these breakthroughs, robust supply chains and innovative fluid-conveyance solutions ensure seamless implementation on a global scale.TheCUBE talks to Omni Services’ Lauren Witter and CoolIT Systems’ Mitchell Knight about the critical topics shaping HPC’s future

play_circle_outline Enhancing HPC Systems with Omni and CoolIT Collaboration for Advanced Cooling Solutions

play_circle_outline Focus on controlling IP and supply chain to ensure quality and safety.

play_circle_outline Maximizing Efficiency: Value-Added Omni Integrator Offers Plug and Play Cold Plate Solutions for Leak-Free Growth

Info
Transcript

Lauren Witter, Omni Services & Mitchell Knight, CooliT

Mitchell Knight

Director of Product Management CoolIT Systems

At SuperComputing 2024, Lauren and Mitchell from Omni Services discuss their division Cool Flow and its support for companies like CoolIT. CoolIT specializes in direct liquid cooling for chips and data centers. Mitchell showcases a copper cold plate for Intel Sapphire Rapid processors. They emphasize the importance of tubing meeting specific standards for leak-free connections and the need for robust solutions in cooling systems. They discuss challenges in cooling high-power AI systems and the importance of industry standardization. The conversation concludes... Read more

explore Keep Exploring

What is Omni all about and what is CoolIT's role in the industry? add

What is Coolflow, CoolIT, and the role of IP cleanliness spec in the liquid cooling division? add

What is the overarching goal of our business strategy in relation to supporting growth and value-added integration within the industry? add

bolt Powered by CUBE AI

Lauren Witter, Omni Services & Mitchell Knight, CooliT

search

Savannah Peterson

>> Good morning, high-performance computing fans, and welcome to Atlanta, Georgia. We are here on day one of three days of coverage on theCUBE at SuperComputing 2024. My name is Savannah Peterson, delighted to be joined with Dave Vellante. This is our first time doing the show together, Dave.

Dave Vellante

>> My first time at this show and it is a nerd fest and we love it.

Savannah Peterson

>> I know it's super talk nerdy to me, baby. It is-

Dave Vellante

>> It very much reminds me of GTC, they're a lot of similarities.

Savannah Peterson

>> I think that's a great parallel actually. So, our next segment, very excited. We've got two fabulous guests. Welcome, Lauren and Mitchell. Thank you so much for taking the time during a very busy week. Dave, you kind of teed this one up. What got you so excited about this?

Dave Vellante

>> Well, so as I was walking around GTC this year, not to pick on Cray and HPE, but I saw this Cray supercomputer, I saw these hoses and I know the folks at Omni. And I'm like, "What do you know about this business? Because that looks to me like amateur hour and I'm really worried that we're spending hundreds of billions of dollars on AI and we're going to have this weakest link of the hoses." So, maybe Lauren, you could start us off with what's Omni all about, Omni Services. Tell us about the company and then we'll get into CoolIT.

>> Yeah, absolutely. So, I'm Lauren Witter, vice president of sales at Omni. Omni is a fluid-conveyance specialist within basically the hose and fitting space. So, if you go back to when Omni was founded a little bit before my time, we were helping OEMs understand fluid-conveyance products, integrity of connection, largely in hydraulics. And in that world, Dave, there was a lot of standards, a lot of specifications. It was a lot easier to scale. Fast-forward, here we are in the great world of HPC and integrity of connection, as you mentioned, and leaks has never been more important. So, we've fast forwarded, we've created a tremendous division called Cool Flow, which is a product-focused group to support folks like CoolIT.

Dave Vellante

>> And CoolIT, I was saying that when I started in this industry, IBM mainframes had these cold plates, they were called thermal conduction modules. I was walking by an HPE booth today. I saw a Cray supercomputer, they definitely cleaned up that mess of hoses. Maybe it was an ancient one that we're showing, like the museum. And Mitchell, you guys are at the heart of that. Not only the direct liquid cooling on the chip, but also very large systems to cool data centers, right? We saw some of those in your booth. Amazing.

Mitchell Knight

>> Yeah.

Dave Vellante

>> Tell us about the company.

Mitchell Knight

>> No, for sure, Dave. So, I'm Mitchell Knight, I'm the director of product management at CoolIT. And just some very quick background on CoolIT. You mentioned the cold plates, that's always been the heart of our technology. I have here one of our Omni cold plates, which is a friction stir welding new technology that we've been using for cold plate production. As discussed, we cover from the cold plate all the way to the CDU level. We've deployed gigawatts of CDUs in the past. And I think when we're talking about the importance of the fluid conveyance to the cold plate, that's critical. We can go in, we can design the cold plate to be 100% robust, no failure modes. And if there's any failure in delivery of fluid to that cold plate, you're not going to have a functional system.

Dave Vellante

>> Can I see that?

Mitchell Knight

>> Yeah, I was-

Dave Vellante

>> show and tell here.

Mitchell Knight

>> Yeah.

Dave Vellante

>> It's fairly heavy. Check it out. Can you guys get that if we hold it up? Which camera?

Mitchell Knight

>> It's fully copper. So, you see the cover there, that is actually just the convection barrier, which is again, it prevents heat from essentially dissipating into the server. And really, promotes heat capture to liquid. Again, our proprietary Omni technology actually does use a no filler and no brazing material to actually create the cold plate. And so, that essentially means your cold plate is one solid piece of metal.

Dave Vellante

>> So, this would go on top of a Blackwell?

Mitchell Knight

>> Yeah, that goes on top actually specifically of an Intel Sapphire Rapid processor.

Dave Vellante

>> Okay. Fine. Intel, AMD. Okay.

Mitchell Knight

>> Likely wouldn't fit on a-

Dave Vellante

>> And the hoses... How does the fluid get conveyed? You could-

Mitchell Knight

>> It goes through tubing. So, we've got-

Savannah Peterson

>> Let's talk about our next crop over there.

Mitchell Knight

>> Yeah, A piece of tubing here, but essentially within your server, you're going to be routing meters and meters a lot of the time, depending on how complex, how many things you have to fit around, of tubing to each individual cold plate. So, if you think a lot of the NVIDIA systems, there's multiple GPUs. On some of the B200 systems, it's eight GPUs that you actually have to go and manage. There's switches. There's a whole bunch of liquid-cooled components you actually have within your server. The way you connect all of those cold plates and different cooling modules together is usually all through tubing. So, you need to have the right tubing and 100% the right tubing interface to the cold plate. So, that's-

Savannah Peterson

>> So, what does that mean? Unpack that for those maybe not as tubing literate as y'all. What does it mean to have the right tubing, Lauren? What makes that tubing the right tubing?

>> Yeah. So, interestingly, it's more complex than one would think. A lot of times tubing gets kind of thrown in that fasteners group in with the nuts and bolts. If you think of a large plant that you might see in your town. So, in this space, there's a lot of complexities. These particular tubes and hoses are peroxide-cured EPDM, that's built the way that they're compounded. There's different braiding, there's different HBV zero ratings. So, the industry's moved very fast. And what Omni's tried to do, and I'd say quite successfully is get ahead of that. And then, to Mitchell's point, how it connects to the cold plate and that's the integrity of connection part, right? That's where all the magic happens. And to your point, that's where that amateur hour can be very, very dangerous.

Dave Vellante

>> That's the weak link, right? It's not a lawn mower hose. You can see it's a high-quality hose, but it's the connection integrity you're saying. Explain that a little bit more.

>> Yeah, absolutely. So, they build all these different cold plates, different customers, they have all sorts of different attachment methods. And what we've come to do is sit with folks like CoolIT and prove an attachment method that we've tested specifically for this industry. We share that IP back to them, they integrate it into their plate, and now we have a great leak-free integrity of connection. So, it allows folks like Mitchell to focus on their IP. It allows us to focus on our IP. And probably most importantly, Dave, the volumes in this industry are out of control. Great supply chains win wars, we can scale very, very quickly.

Savannah Peterson

>> Speaking of supply chains, just because I noticed it, and Dave, I see you're eager to chat too, but I noticed here on the hose, you're making these in the USA.

>> We are.

Savannah Peterson

>> De-risking your supply chain there.

>> Yeah, so Mitchell and I've exchanged quite a bit, right? The industry, there's a lot of different things going on right now and dual-sourcing and things. So, yeah, we try wherever we can at this time to do manufacturing here in the US.

Dave Vellante

>> So, we talk in the security world all the time about security can't be an afterthought, it can't be bolted on after the whole system gets built. I presume the same is true here. You've got to design this in. So, who designs this in? Is it a thermal engineer? Is it a fluid engineer? Is there a-

Savannah Peterson

>> Great question ....

Dave Vellante

>> that architects this? Yeah.

Mitchell Knight

>> A lot of the time it is our design engineers. So, we work with tubing all the time, going into what we would call our passive cold plate loop products. And essentially, really, our focus is distributing coolant to our individual cold plates. And that's where relying on partners that really do have robust solutions makes it easier for our design team. So, they don't want to focus on designing the best hose barb interface, making sure that's robust. We ship a lot of our products filled from Canada, so our cool play loops are fully filled with coolant. Given that we're building a lot of these in Canada, it's fairly cold when we ship these. So, we do use a glycol water coolant solution that they're pre-filled with. It still does freeze at minus 18 degrees Celsius. So, you can think about robustness within the data center operating conditions, but the getting it to the data center is also critical. It has to handle freezing, expansion, everything else.

Dave Vellante

>> So, Lauren threw a quote out there, "Supply chains win wars." I think the quote is, "Generals win battles, supply chains win wars."

>> Absolutely.

Dave Vellante

>> So, let's double-click on that. Talk about the supply chain. How do you scale to accommodate what ostensibly is this massive pending business?

>> Yeah, absolutely. And I think that's what our Coolflow product portfolio was built for. We start with the integrity of connection, which that's what takes the most time. Getting early with folks like Mitchell when they're doing NPD types of products, deciding on how we're going to bring things together. Then, that allows us to work with various supply partners within the industry. A lot of times because of the scale, you do have to have multiple sources, multiple partnerships. And then, we have a specific division of the company that's focused on nothing but liquid cooling. So, really, Dave, the plan is to try to the industry together to adopt a lot of the same principles of this value-added integration, as Omni calls itself, then we can all scale together. Right now, everyone's doing their own little thing and then all of a sudden we talk about hockey stick growth, this is pogo stick. So, we have very, very limited forecast. So, the more we can agree on principles and materials, the more Mitchell and other competitors and other folks can leverage the same channel.

Dave Vellante

>> And you can try to eliminate choke points. You think about the semiconductor industry, you've got thin-film in Japan, you've got ASML in Netherlands-

Savannah Peterson

>> . Yeah, yeah....

Dave Vellante

>> which is one of them. You've got TSM in Taiwan, you've got only one of them.

Savannah Peterson

>> Exactly.

Dave Vellante

>> You've got the EDA manufacturers in United States.

Savannah Peterson

>> There's not a lot of people making chips.

Dave Vellante

>> There's like 10 choke points. So, you guys are anticipating that. What are the choke points today that you're aggressively trying to stamp out of the system in the value chain?

Mitchell Knight

>> There's a couple. I think we talk about peroxide , VBM hose. There's definitely a choke point right now on larger diameters of that tubing. So, specifically thinking two inch. When you're looking at next-generation AI racks, like those really high-density racks, you have to increase the diameter of essentially your flow path to that rack. And finding the right tubing made of the right material to accommodate that higher flow rate and still meet all the wetted materials requirements is very complex.

Savannah Peterson

>> So, building on that, you've obviously, you two worked together. How long have you been working together? How did you identify-

>> A couple years now....

Savannah Peterson

>> you were such compatible partners? This is a very specific industry.

Mitchell Knight

>> Yeah, it's quite niche. I think a lot of the time historically we've worked with component suppliers for our passive coldplay loops. And it's been focused on, "Hey, we just want you to go and do everything to our spec." And I think where we've found benefits is leveraging some of the expertise that these companies that have been working in fluid distribution in other industries for a huge amount of time, they can actually bring that expertise into our industry. And that's been a huge value add for our engineering team.

Dave Vellante

>> So NVIDIA's in the news this week. Obviously, they got earnings on Wednesday-

Savannah Peterson

>> Just this week?

Dave Vellante

>> Every week, but really in the news this week, right? Michael Dell put on LinkedIn a picture of their Blackwell, their first versions. I see them in the HPE booth. You guys have them in your booth. So, Blackwell's now hitting the market, they're melting racks. That's where you guys come in. So, you must be watching that pretty closely. What gives you confidence that you can cool these exceedingly-hot systems? Maybe talk a little bit about the roadmap. And you've addressed the supply chain, but the technology behind here, you've got some choke points, but are you confident that you can cool these systems going forward? What's your roadmap look like?

Mitchell Knight

>> Yes. I think when you're looking at roadmaps and technology to support, when everything came out on the NVL72 for example, and it was kind of a shock to the industry that, hey, we're going to have 120 kilowatts within a single rack. We have through the Frontier system, El Capitan, some of the true HPC systems. We've been doing more than 300 kilowatts in a single rack without concerns or issues. And that's 5.7 kilowatts within each individual server node. So, the numbers don't scare us. The next-generation numbers don't scare us. And I think there's a lot of overhead right now in single-phase direct-to-chip liquid cooling.

Savannah Peterson

>> I love the confidence. I'm glad you're not scared, so the rest of us aren't scared.

Mitchell Knight

>> It's more the power distribution. We can cool-

Savannah Peterson

>> There it is....

Mitchell Knight

>> 300 kilowatts in a rack without issue, 500 kilowatts within a single rack without issue. It's how you're going to run that amount of power to your single individual rack I think might actually be more of the gating item.

Savannah Peterson

>> You see different customers across the industries, what are some of the common mistakes or myths that people may not be considering when it comes to-

Dave Vellante

>> Great questions.

Mitchell Knight

>> There's quite a few. I think there's a lot of new players within the space. So, we see a lot of things that don't necessarily make sense or are things that we learned a long time ago.

Dave Vellante

>> That's got to be a concern for you because it could damage the reputation and then it spills over to everybody. So, you got to fight that.

Mitchell Knight

>> Yeah, for sure. And I think as liquid cooling becomes more mainstream, there's always this push that, "Hey, we want to get multiple vendors products on the same wetted materials loop. Coolant is going through one vendor's product and then going into another's." Where we see concerns is people are doing improper brazing processes within their cold plates, that creates particulates, contaminants that go into the coolant. That then makes their way into our systems, let's say it's our CDUs, and it can cause problems within our products. And if you're looking at a multivendor solution, it's tough. You don't have one throat to choke. You say, well, is-

Savannah Peterson

>> I know I was going to say who gets the blame in that scenario? Yeah.

Mitchell Knight

>> Yeah, it's different than air cooling, right? The air system, one heat sink isn't going to impact another heat sink too significantly.

Dave Vellante

>> And second sourcing is also an issue. Can you address that?

>> Yeah, absolutely. Really what Mitchell said is exactly what Coolflow, our liquid cooling division, was built on. We control the IP of the cleanliness spec. We understand the different nuances between the OEMs and the infrastructure and the integrators, but we control the supply channel of material. So, they know regardless of the brand of hose, the brand of coupler. We were chatting earlier, there's a lot of agnosticness because the end customer drives spec, sometimes CoolIT may drive spec. So, they know it's getting delivered to them the way that the industry expects it. So, they don't have to worry, "Well, I bought this from vendor A and I bought this from vendor B. Do they know our special nuances?" And that's really I think how we can scale this together and do it safely.

Savannah Peterson

>> What is... Go for it.

Dave Vellante

>> No, please. I...

Savannah Peterson

>> I got two questions for you. One, I just have to ask because such a nerd, what does testing look like for-

Dave Vellante

>> Great question....

Savannah Peterson

>> hoses?

>> .

Savannah Peterson

>> Yeah, because we're not exactly failing on a piece of equipment here.

>> Yeah, I could say a little bit to it, right? There are some proprietary nature because each customer is doing something different from our perspective-

Savannah Peterson

>> Oh, that's where you can't tell me the secrets?

>> Yeah, great. We can offline, offline. But I think what's unique about this space, if you look at hose in general, there's SAE standards and different standards that govern us, but this industry is very different. So, we're often finding ourselves validating at much colder temperatures -20, -30, -40, higher temperatures. For us, we spend a lot of time understanding where in the application it's going. Something like you have in your hand is going in a blade. It's kind of a perfect life for a hose if you will. Something that's going to be a source line that's hanging, that's a whole different integrity of connection because you've got a lot of weight and a lot of density to it. So, testing is not kind of one-size-fits-all, but we are working very heavily to get some standards for the industry as a whole and share those around.

Mitchell Knight

>> So, an interesting thing, and let's say you're talking about the distribution hoses into the racks, A lot of time within the data center, those hoses actually have to run through the same kind of cable trays and everything else that... Or in certain cases they have to run through HVAC and plenum zones. So, a lot of what the industry is doing right now is they're looking at cable specifications, so electrical cable specifications, and trying to apply them to tubing. And so you're looking at fire prevention specifications-

Savannah Peterson

>> Interesting....

Mitchell Knight

>> for something that's filled full of fluid and that's the concern, oh, hey your fluid-filled table-

Savannah Peterson

>> Very interesting....

Mitchell Knight

>> is going to cause a fire. Absolutely. So, yeah, I think there definitely needs to be more industry standardization that is going to add a lot of clarity and...

>> I think people are taking so many standards, like Mitchell says, and just trying to adopt them. And when you say talk nerdy, it's like we want to understand that because some of the things, the HV ratings and things aren't as relevant to hose as they would be to some of the electrical distribution components.

Dave Vellante

>> Yeah. So, I wanted to tease the panel tonight.

Savannah Peterson

>> Go for it.

Dave Vellante

>> So, we have a direct liquid cooling panel at 5:00 PM tonight East Coast time.

Savannah Peterson

>> I'm looking forward to that.

Dave Vellante

>> Dr. Luca Amalfi, he's one of the foremost experts on two-phase and it's a big debate whether or not two-phase, it can scale and if it's the right technology, there's cost associated with it, there's engineering factors. We got Tim Shedd coming on who's an engineering PhD from Dell. And Dion Harris from NVIDIA. So, we're going to get three different perspectives and tee that debate up.

Savannah Peterson

>> That's going to be super fantastic. I can't wait to watch that one. All right, I got one last question for you that kind of ties into... Well there's two questions that tie into one. So, one, I'm curious where the innovation goes next here, what you're trying to next. And on that note, since you've both been fabulous guests and maybe these are in tandem, what do you hope to be able to say at next SuperComputing that you can't yet say today?

>> I'll let you go first if you'd like.

Mitchell Knight

>> Yeah, so I think the main thing that we're looking to say at next SuperComputing and the big innovation that we've had, especially on the cold plate and tubing side, is going to be reducing our time to market. And right now, with how fast we've been getting a whole range of different components, processors, we really need to actually shorten our development cycles while still maintaining the level of robustness and quality. So, I think what I'd like to say is, "Hey, we've shortened our development like cycle to three months, two months." Right now, we're at four months I would say for a new passive cold plate loop product. And that has not been fast enough to keep up with how quickly the industry's moving.

>> I think ours is in the same vein, right? I think it's to support that and to support that growth, it's to help the industry as a whole understand this value-added integrator of Omni. You don't have to buy bulk hose and fittings and become hose assemblers and do all this testing. You can focus on cold plates and scale. Everything can come in ready to be plug and play and we can grow together, leak-free.

Savannah Peterson

>> Grow together leak-free, perfect line to close on. Look forward to discussing how the story's evolved at SuperComputing 2025 in St. Louis next year. Lauren and Mitchell, thank you so much.

>> Thank you both.

Dave Vellante

>> Thanks, guys.

Savannah Peterson

>> , it was a pleasure. Glad you're here with us at the show this time around.

Dave Vellante

>> I know. Good to be here.

Savannah Peterson

>> I know I can tell you're going to enjoy the nerdy ride. And I hope all of you enjoy the nerdy ride we're on here for our three days of coverage in Atlanta, Georgia at SuperComputing 2024. My name's Savannah Peterson, you're watching theCUBE, the leading source for enterprise tech news.