SC24 | Tim Shedd, Dell Technologies

Clips
News
More from SC24

Tim Shedd

Engineering Technologist, Office of the CTIO

Dell Technologies

Dell Technologies innovates AI infrastructure with scalable solutions

In the world of high-performance computing solutions and artificial intelligence, enterprises face mounting challenges to scale efficiently while managing the heat, power demands and complexity of cutting-edge infrastructure.Dell Technologies Inc. is addressing these challenges with innovations that prioritize flexibility, sustainability and rapid deployment, according to Tim Shedd (pictured), engineering technologist, office of the chief technology innovation officer, at Dell.Dell’s Tim Shedd talks with theCUBE about Dell’s innovations in high-performance computing.“Our strategy is to make it easy for our customers to deploy the compute they need to run their businesses,” he said

play_circle_outline Engineering innovations in liquid and air cooling for powerful systems

play_circle_outline Maximizing Hardware Platform Design: A Focus on Flexibility, Interoperability, and Scalability for Cost-Effective Cooling Solutions

play_circle_outline Maximizing Customer Value Through Transparency in Business Operations

play_circle_outline Emphasis on customer service and satisfaction

play_circle_outline Innovation in hardware and software offerings

Info
Transcript

Tim Shedd, Dell Technologies

Tim Shedd

Engineering Technologist, Office of the CTIO Dell Technologies

Three years of coverage at Atlanta, Georgia coming to a close. John Furrier discusses the challenges of AI. Tim from Dell talks about thermal strategy and the advancements in cooling technologies. The focus is on providing value and agility to customers. Dell is working on innovative solutions to cut down power consumption and increase efficiency. The team at Dell is proud of their patents and is continuously working on new solutions. Tim hints at upcoming announcements and developments in the field of AI and cooling technologies. Service is a key aspect that... Read more

explore Keep Exploring

What innovative approaches is Dell taking in the cooling system of their servers for customers from universities to supercomputers? add

What are some key focuses for Dell in terms of providing value to their customers in terms of reducing power consumption in data centers? add

What is the importance of being honest and transparent with customers in regards to power and cooling in the context of delivering value and maintaining uptime in the tech industry? add

What is one unique aspect of Dell that sets them apart from other companies and impresses customers? add

What is Dell's approach towards expanding their software package and ecosystem for customers? add

bolt Powered by CUBE AI

Tim Shedd, Dell Technologies

search

Savannah Peterson

>> Good afternoon, AI fans, and welcome back to Atlanta, Georgia. We are coming to the conclusion of our three days of coverage here on theCUBE. My name's Savannah Peterson, joined with my fellow favorite co-nerd, John Furrier. John, this show is so fun.

>> Three years we've been here, and the number one thing has always been more performance, but the big issues with AI is space on the floor in the data center and power and cooling. It used to be power and cooling, but you can power, and the cooling ... it's kind of like they're breaking out a little bit. So this segment's going to be great. Dell's done some amazing things, and the new rack with OCP compliance has been phenomenal. The booth's been buzzing, so we can't wait to get things going.

Savannah Peterson

>> Yeah, we've got some cool things going on.

>> Can't wait to get into this conversation.

Savannah Peterson

>> And who better to tell us all about them than Tim? Tim, thank you so much for coming to hang out with us today. It's been a busy show for you. You're a popular man.

>> Yes, indeed. These days, it's our time to shine.

Savannah Peterson

>> It is your time to shine. Lots of things shining about the Dell AI factory. You lead the thermal strategy over there, and I'm so excited to have this conversation with you, because the amount of things that go into thermal strategy is only getting more and more complex. So how do you go about that? How is Dell thinking about that?

>> Yeah, it's a great question. So up until recently, even a few years ago, we were looking at scaling out, and we were definitely looking at HPC and AI. We were doing that. But we're talking about 50, 60 kilowatts a rack, 19-inch racks, doing the whole power/PDU/PSU juggling. But cooling was just making sure we could keep things calm, right? About two to three years ago, we were looking to the future and saying, "Hey, this is literally going to get hot. This is going to be something where we're-"

Savannah Peterson

>> Pun intended.

>> Yeah, indeed. "We're going to need to plan on putting a whole bunch of accelerators together so that we can actually do the high speed networking to make that happen." But when I put a whole bunch of accelerators together, that is putting a whole bunch of heat in one place, and it's really that density that is the challenge. But that simultaneously addressed a couple of things that you brought up. It also means we can do more in less space, which is good for the data center, and it means that we need to think about how to deliver that as a unit so that it's not complex for the customer to deploy and be able to ideally just drop that on the data center floor, connect power, connect cooling, and the customer's making money. That's our goal, is get them working.

Savannah Peterson

>> That sounds wonderfully simplified. I love that.

>> That's really what we're trying to do.

>> Sounds like an AI factory to me.

>> That's right.

>> The AI factory is a great branding because it simplifies the concept. It's not just a server anymore. It's a bunch of things together as a system. And I think that's what we're seeing this show has turned into. It's a system show now, and the game is clear; productivity, AI apps and agents are coming, agentic systems, agentic infrastructure. So as the infrastructure creates these new layers, new software's emerging, and that's really where this advancement is going to really move the needle again. So even next year, I'm sure we'll see, and we've heard on theCUBE's event and many guests saying it's the next software that's yet to be written. Today's been leveled up with AI. Now we have the next level coming.

>> Indeed, and that brings up a really important point. So today, as you saw, we, to our knowledge, have the first enterprise ORV-III deployment. That's going to allow us to give a lot of density, but another thing that it does is we created a multi-generational hardware platform, because what you're mentioning is what's coming next, and we wanted to be ready for that today. So that rack infrastructure is designed today for 120 kilowatts. We've already got customers for 220 kilowatt racks, but we knew we needed to get to a half megawatt. So how do we do that? How do we design the power and cooling to allow that one platform to be flexible enough for our customers to be able to update to the next and next and next GPUs and software and networking and everything that needs to go in there?

>> Yeah. One point that wasn't really amplified because it was pretty obvious, it's on the rack itself when we did the booth analysis, was it's OCP compliant, Open Compute Project, which theCUBE was there when it kicked off. That was the beginning of the opensource of the infrastructure, and open standards was one of the top things. You're seeing it with ethernet, you're seeing it with the modularity that customers want in a heterogeneous environment. They want to have interoperability and integration now that gen.ai is not just connecting systems, but data is transferring. So all that's together. So open is a huge part. Take us through the mindset at Dell. Dell's always been open. I mean, Intel has been industry compatible machines, but this is where it gets really scalable. What's the thoughts there on the engineering side, the strategy with Dell? Take us through why open is such a big deal and what it means for the customer.

>> Sure. Lots of questions there. So this ties back to the first question-

Savannah Peterson

>> Well, you're sitting next to John Furrier.

>> Yeah. About thermal strategy, our strategy is to make it easy for our customers to deploy the compute they need to run their businesses. So when we're talking about looking at ORV-III and OCP standards, they're great, but we need to ... And so we have, we've adopted what we call the payload area. You can put any OCP compliant server in there that'll fit, it'll get powered. But we needed to go a step beyond that. We needed to make sure that there was liquid cooling that was also standard. And that's something where we were the first, to our knowledge, to put the OCP standard liquid connectors on there. So that's a standard interface for cooling as well as the OCP standard power. But the standard was thinking about 17 kilowatt racks, 30 kilowatt racks, not 200 kilowatt racks.

Savannah Peterson

>> Big difference there.

>> Yeah. So we needed to make the rack a bit deeper. We went from 1000 millimeters, or a meter deep, to 1.2 meters deep, and a little wider, from 600 millimeters to 750 millimeters. But what that gives us is the ability to really put the network cabling in for all these dense AI racks, bring the power in the rear to be able to get to those high power densities. All of this translates to a platform that is flexible, that will enable our customers to grow with that platform, and it also creates a package that we can pre-integrate for customers; deliver on-site, as I said, connect power and cooling, and you're up and running.

Savannah Peterson

>> Just out of the box?

>> Yes. It's a little harder than that, but-

Savannah Peterson

>> You're making it sound so simple, though. I think that's inspiring and enlightening. It's essentially providing a platform for agile innovation depending on what the customer's needs might be. You've got a lot of different partners, a lot of different players in this game. I'm just thinking about it and it must be pretty complex to figure out how to create a solution that goes like this. What's the development cycle like? How long is it taking you and the team to come up with this next generation?

>> So I'm not sure I can put an easy number on that. We've been working with OCP, as you noted, for a long time. And one of the core concepts here was DCMHS, the Data Center Modular Hardware System. What that does is that was partnering with silicon providers, NVIDIA, AMD, Intel, to say we want a common platform that we can drop in your processors, not this year, but five years from now, into this platform. That was key. So we've been working on that and been a leader in that. Also, jumping to cooling, those connectors, super important. One of my colleagues is a co-chair of that committee in OCP, and then that means working with all the fittings manufacturers, working with the cooling suppliers, working with power supply suppliers. So you're exactly right. It's a whole lot of folks that are involved in this, and this really comes together in a couple of years. It's not six months, but now-

Savannah Peterson

>> Well, it would have to. There's a lot going on there. Yeah.

>> But now that we've designed a platform and thought about things, for instance like this, we want this platform to be flexible so that we don't have to spend a couple of years every time. So until now, if you look at a super computer, you look at Frontier, you look at El Capitan, they're great, amazing pieces of engineering, but they're custom-made. They're basically one-offs. And that's not a negative statement, it's a statement of fact, because they needed to do something to fit that in. But you know what? That doesn't meet our needs to have a platform that can go from supercomputing to the neighborhood bank branch. And so what we said is we want a platform that I can put a two-processor server in, I can put an eight-GPU server in. It gets the right cooling automatically. Don't have to think about it. So we've engineered, among other things, but this is an example, we've engineered what we call impedance-matched servers. So no matter what you put in there, whether you put it in today or you put it in six years from now, it's going to get the cooling it needs automatically. You don't have to think about it, don't have to tweak the cooling system. It just works. It's that sort of forethought that's gone in that is going to allow us to be much more agile moving forward. It's not going to be a two-year cycle, because as you've seen, as we all see, we're not on two-year silicon cycles anymore. We're on maybe, what is it? Nine months? I'm not sure.

Savannah Peterson

>> No, exactly, six to nine months. The speed of hardware innovation is totally entering a new arena and era. Yeah, I think you're absolutely right. So we'll be able to grow together. What are the biggest challenges in the cooling of these larger systems and more powerful systems?

>> Yeah, so the challenge is to balance the high power, which requires really innovative cooling, with the configurability and flexibility that we know our customers need. Again, we're trying to address customers from universities all the way to the largest supercomputers in the world. And so-

Savannah Peterson

>> Yeah. Governments, universities, enterprise; everybody needs stuff.

>> So we've taken an approach that's a little different than some. We're growing the amount of the computer that is liquid-cooled, but we have to leave some parts open to air so that we can quickly service it, so we can allow the customer to change up and say, "Oh, we need to change networks for some reason. We're going to change all of our NICs." That's easy to do in our systems. But that also means that we're not just innovating in ... the big piece of this show is liquid cooling, and it's very awesome. I love it. But we have to innovate on air cooling too, and we're doing that, and that means rethinking. We don't want more fans in there. We don't want more noise. How can we use the air that's flowing through there now to cool better? And so you're going to see some innovations coming out from Dell early next year that'll show how we can use the same amount of air, or even less, to build higher-powered devices to get us there, to allow us that-

Savannah Peterson

>> How do you do that?

>> It comes down to physics. It comes down to-

Savannah Peterson

>> Yeah, well, I want to know the physics. I'm trying to wrap my mind around this a bit.

>> speed of light.

Savannah Peterson

>> Right, I know. Is gravitational force going to go a different direction? I mean, we've got the doctor in the house. Can you explain that? I'm curious.

>> So to some extent. To some extent we've got to wait until we can show it off all together-

Savannah Peterson

>> Oh, come on, Tim. I thought we were going down the right path there.

>> Yeah, yeah, yeah. No, but it's a matter of really getting in there and understanding the physics of e-transfer. How does heat get into the area? How do we move it out efficiently? And then saying, "Well, how can we do that without complicating the system, adding complexity and more service calls?" That's a thing. We don't want to add cost. And so we have some really creative people and it's been really fun to work with-

>> There's a lot of engineering in these areas to-

Savannah Peterson

>> I can't wait to get this physics lesson when you have to talk about it, because now I'm super curious. I'm going to have to go do some research.

>> Well, this is a show where you've got to have the proof, the sizzle and the steak, as they say. And the three areas we mentioned, there's real engineering going on. Open up the Dell playbook there and what you guys have done. We've been covering Dell over these past three years. Dell and your partners, like Broadcom and others, have been really engineering for those three areas; space, power and cooling, and price performance. I mean, that's the three areas where engineering's got to get done to solve those problems. What's some of the things that you can share inside Dell that you're excited about or you hope to share?

>> Yeah, absolutely. So that is absolutely our focus. We know that that's what wins business, as well as Dell's ability to provide longterm service and support. We hear that all the time. They like our computers, but they love our relationship, so I'm going to just put that out there. The other thing is listening to the customer and understanding their needs. They're telling us we want this compute. It's no mystery that power's hard to come by. And right now, in many data centers, cooling is consuming 40 to 50% of the power coming in. So we're working on technologies-

Savannah Peterson

>> That's really significant.

>> Yeah, that's huge....

>> to cut that down. It comes back to value. How can we provide value to the customers, make sure that our gear has the lowest total cost of ownership? And that includes things like something that we've hinted at here. We're going to have an announcement next year, where we're going to provide 100% to very warm water, say 90 to 95F water.

Savannah Peterson

>> Whoa.

>> To collect all of the heat in a 500 kilowatt rack. That allows most of our customers to remove chillers altogether from their systems, wherever you are in the world. And that makes a huge difference. That cuts that 50% number down to about 10%.

>> Wow, that's a huge gate. Huge gate.

Savannah Peterson

>> Yeah. And that is directly related to our customers' bottom line, and that's what we're all about.

>> Yeah, and they're going to get also a top line benefit with the Gen AI wave, because there's enough low-hanging fruit workloads that are going to get the Gen AI infusion .

>> Exactly. But real important, that doesn't come cheap, right?

>> Yeah.

>> So these racks, I'm not going to talk about exact numbers, but we've all heard they're expensive.

Savannah Peterson

>> You're such a tease, Tim.

>> They buy them in bunches too. You have the big boxes, the servers, you've got the air-cooled one, you got the liquid-cooled ones, but because they've got the , you need a couple of them.

>> Right, yeah, and you need time together. And the key thing is, if you're spending, let's just throw out a number, if you're spending $3 million for rack, you want that to start generating revenue as soon as you can. You don't have time to wait. You don't want to wait for the cooling to get hooked up. You don't want to wait for the power to get hooked up. So again, part of our focus within Dell is enabling that fastest time to deploy. It's not just getting to market, it's making sure the customer can actually use it as soon as possible after they've paid for it, because they've got to start ticking. Because when's the next processor coming out? Again, it's not two years from now, it's nine to 12 months from now. They need to be keeping that value.

>> Hey, the race is happening, Savannah and I were just talking about it. We briefly talked about it before we came on camera, about the CFO, they're being pulled in, because it's not just a capital budgeting exercise, "Hey, I need another server." This is a big system and it's a big purchase, and now you have other criteria. "I might use a GPU cloud with this rack." And so now the CFO's got to get into this because it's a technical changing of the business model, to your point about getting that value fast with Gen AI. So a whole nother TCO calculation gets made out, you've got to figure out risk management. If I'm going to use a compute as a service, I've got a CapEx. This is like first generation, new CFO problems that are opportunities.

>> That's right.

Savannah Peterson

>> Yeah, it's a whole shebang. Tim, I've got a question for you. You've got 14 patents already. How many new patents are you going to get next year when you release all of this stuff you're teasing us with?

Savannah Peterson

>> It'll double or triple.

Savannah Peterson

>> Ooh, wow.

>> Yeah, we're all working hard. That's not just me. That's the team. And we're really proud of that, by the way, as Dell, as an engineering team. Our management has been bragging about the fact that we're climbing up those patent ranks. You're going to see us right in the very top list of patent-generating organizations. And that's just a testament to the innovation to meet the customer needs. We're not doing this for fun, right? I mean, it is fun. Absolutely.

Savannah Peterson

>> Not for the patent vanity.

>> But it's not for the patent vanity at all. It's really that, "Hey, we've got to solve some problems and we've got to think about things in order to meet all of these requirements." We want to be agile, we want to be inexpensive, we want to be reliable, all these things. Okay, that's not something that's on the market today. How can we do that a little differently? How can we look at the physics again, take another angle at that? So yeah.

>> Well, as you said earlier, the engineering wins business because it's solving these areas of what customers care about. And the competition is all saying they have best power cooling, so how would you talk about the competition? I mean, they're saying, "Well, we've got this. We've got the best." Everyone seems to have the best.

Savannah Peterson

>> Or they are certainly marketing it as that. Whether that's the case-

>> How do you squint through the signal from the noise relative to all the hand-waving around power and cooling?

>> So a couple of statements on that. First, I've got colleagues at all of these places. They're all smart. And so in the end, we just are focused on being real honest with our customers, real transparent, telling them what they're getting. And the key thing really isn't necessarily to be the best, but to be able to deliver the value. And that isn't just the engineering, it's our sales staff that help get the systems there, and very importantly, it's our service organization that makes it happen and keeps it up so that they're making money all the time instead of down. And that's something that Dell brings really uniquely, is we have this worldwide service organization that can be there on the spot and deliver. I'm so impressed. One of the things I've been most impressed about with Dell is it can take some time, like I said, a little bit of a runway to get going, but once they're going, we've got the systems to take care of the customers worldwide, and that's really the closer. That's really what the customers say, "Hey, okay, price, features, yeah, it's all in the ballpark, but we like the service. We like that you are taking care of us."

Savannah Peterson

>> It's all about community building and making sure that your customers are happy and that that user experience is optimized. I've got one final question for you, Tim, and I realize you can't spill the beans on some of the secrets coming, but I suspect you can give me an answer to this question. When we're hanging out at Dell Tech World or at SuperComputing 2025 in St. Louis, what do you hope to be able to say then that you can't yet say today? Obviously within reason.

>> All right, so I think you're asking the same question a little different way. That's all right. So-

Savannah Peterson

>> It's like you're onto me or something. It's more I want your prediction for the sizzle reel. So it doesn't have to be super technical.

Savannah Peterson

>> I don't think it's a secret, I think you can expect to see a whole bunch more platforms coming out in our ORV-III platform. You're going to see a stronger continuous strength in our 19-inch platforms. And you're going to see these energy-saving technologies that I'm hinting at here. You're going to see those on the floor. You'll see us delivering. And I think it's going to become more evident to you and to the world just how complete a package Dell is delivering. It's not just a limited number of platforms, it's for the spectrum of customers, and again, the services and the software that enables it. That's continually growing. And I think what customers are going to see is that's the package. It's not just the hardware and the service, but also we're continually building and aggressively building our software package to make it easy to deploy. And there will be some expanded offerings as well, so you'll see more pieces of the ecosystem with Dell badges. Love our partners, they're great, but all those patents that I'm talking about, they're going to start appearing in Dell badge products that may surprise you.

Savannah Peterson

>> Well, I look forward to being surprised, Tim. Thank you so much for making time on such a busy week for you. Shout out to your whole thermal team, and you better give us a call right away when you are allowed to talk about it, because I know I speak for John and I when I say our curiosity is quite piqued.

>> Yes, we want the data.

Savannah Peterson

>> Yes, we definitely want the data. And John, thank you so much for hanging out. Thank you all for tuning into our fabulous three days of coverage here at SuperComputing 2024. We're in Atlanta, Georgia. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.