SC24 | Keynote Analysis, Day 1

Clips
News
More from SC24

Dave Vellante

Co-Founder & Co-CEO

SiliconANGLE Media, Inc.

Savannah Peterson

Principal Analyst & Host

SiliconANGLE Media, Inc.

Scalable AI infrastructure reshaping data centers: theCUBE insights from SC24

The boundaries of high-performance computing are being redefined by artificial intelligence, driving innovations that prioritize scalable AI infrastructure to meet growing demands for efficiency and power.As enterprises adopt advanced AI-ready systems, the tech industry is moving beyond theoretical applications to deliver tangible solutions. This transformation is reshaping data center capabilities, promoting sustainability and fostering collaboration across academia, government and private sectors to address global challenges, according to John Furrier (pictured, left), executive analyst at theCUBE Research

play_circle_outline Sneak Peek into Supercomputing 2024 Event: Collaboration with Government and Enterprise Transforms AI Infrastructure

play_circle_outline Importance of cooling technology in high-density chip systems

Info
Transcript

Keynote Analysis, Day 1

Dave Vellante

Co-Founder & Co-CEO SiliconANGLE Media, Inc.

HOST

Savannah Peterson

Principal Analyst & Host SiliconANGLE Media, Inc.

HOST

Supercomputing 2024 in Atlanta, Georgia has begun, with a focus on AI transformation in various industries. Major players like NVIDIA, AWS, and Cerebras are present, showcasing new technologies. The show highlights the collaboration between government, academia, and enterprise, with a keen interest in open science and data sharing. There is a shift towards AI hardware infrastructure and the emergence of new system builders in the industry. Discussions also revolve around cooling technologies, sustainability, and the future of quantum computing. The show empha... Read more

explore Keep Exploring

What are some observations about the changes happening at the show this year compared to previous years? add

What are some key considerations and challenges related to energy, power, and cooling in the context of modern computing technology? add

bolt Powered by CUBE AI

Keynote Analysis, Day 1

search

Savannah Peterson

>> Good morning and welcome to Atlanta, Georgia. We're here at Supercomputing 2024. My name's Savannah Peterson. Very excited to kick off this show with my favorite comrades, John Furrier and Dave Vellante. Gentlemen, how nice to be here today. How are you doing?

Dave Vellante

>> .

>> Third year covering the show, watching it transform to AI is awesome and I think it's turning into industry shows we hit predicted last year and the guest list is rocking this year. We're going to have tons of awesome people and it's again, continues... NVIDIA continues to do its thing. AWS is here. We have Cerebras founder coming on.

Savannah Peterson

>> Dell's got a huge booth.

>> Tons of people. Dell's got an AI factory here. They have an actual trailer. They brought the factory, Dave, so that should be fun.

Savannah Peterson

>> Let's go check that out.

>> Kristen Nicole's here. She's going to do some behind the scenes. So yeah, it should be great.

Dave Vellante

>> I was watching your preview. This show started in 1980.

Kristen Martin

>> 1988.

Dave Vellante

>> 1988, sorry. But in the 80s. Amazing.

Kristen Martin

>> Yeah, yeah, yeah. Exactly. 36 years.

Dave Vellante

>> As old as you are, but that's not old.

Kristen Martin

>> Thanks, Dave.

Dave Vellante

>> But wow, what a show. I mean, it reminds me it's almost maybe bigger than GTC.

Kristen Martin

>> Yeah.

Dave Vellante

>> Which was insane.

Kristen Martin

>> It's over 30,000 people here this week, which is pretty powerful. One of the things that I really like about this show, and you can see it, we'll have to get some B roll so everyone can see it is the intersection between government, academia, and enterprise. And I don't think you get the overlap of that at too many shows. It's just not that common. So it shows you about the collaboration. We saw in the keynote this morning. It was NASA up there. They were talking about open science and the value of opening up their tools and their datasets for space exploration across the board. It's actually, it's pretty interesting. John, we were here last year together in Denver, I believe it was. Yeah, it was Denver. What do you think is going to be the biggest difference between last year and this year?

>> I think the big difference last year was kind of like the stake in the ground that the shift is happening from an industry standpoint. This show has turned into an AI show. I think this year as you're starting to see real build out around the infrastructure hardware and where hardware is turning into systems. I was talking to folks at Broadcom last night, Cerebras and a bunch of the ship folks, and there's a new level of builder emerging. It's the classic hyperscalers on the high end. We all know who they are, Meta, AWS, Azure, they get in the big orders in on the GPUs and the components. Then you've got the enterprise like the Dells, the HPEs. They're going to embed their devices and systems, obviously servers. They control the IT, but there's a new class of merging I call the integrators that used to be around doing either working with Supermicros of the world, people who built their own white boxes, we covered that wave. This new integration of these builders are doing between a hundred, 300 million in revenue. They're the ones who actually building these new systems. So the Broadcoms and the chip guys are targeting these new builders because they're going to come into the enterprise and build it. Because if you want to own the software game, you got to build your own PC. You got to build your own servers and then connect it to the cloud. So I think this year the conversation is Supercomputing is democratized thanks to NVIDIA's messaging and all their work, but now the computers on premise, the data center technologies have to be rebuilt and then connected to the cloud for customization. So I think this year will be the year... the era of clustered systems on-premise edge where it's just the system that's going to matter. So I think this year we'll be marked as, okay, game is on, real solutions are coming out, AI PCs with H100s is going to be there. We have one here in the booth. You're going to start to see the game change and then the era's here, the chapters close, the old IT is over and the new systems are coming in. So nothing gets done until the hardware gets built and the software can run on it, and then that'll just explode the whole agentic issue.

Dave Vellante

>> I think the timing is interesting too because Blackwell's just starting to get out there. I've walked around the floor, I've seen several Blackwells and NVIDIA announces earnings tomorrow. The new Blackwells are melting racks right? You've seen that. So there's a whole challenge of how do you cool these things. I mean, liquid cooling is now back in a big way. And so we've got a panel this afternoon on direct liquid cooling, and we have one of the foremost experts in the field coming on to talk about phase change, and there's a big debate about whether or not that can scale. So we're going to tee that up. So it's super exciting. Back in my IDC days, which was years ago, the high-performance computing group reported to me, I didn't run it. It was run by somebody named Deborah Goldfarb. She was amazing, she was always telling me how eventually this world is going to go mainstream. We'd be like, "Yeah. Uh-huh," and it's taken 20 plus years, but now it's really here. It is going.

Savannah Peterson

>> I was going to say looking around.

Dave Vellante

>> Yeah, 100%.

Savannah Peterson

>> I always say this definitely qualifies.

Dave Vellante

>> No question. Your point about just enterprise and governments and commercial -

Savannah Peterson

>> And all the research that comes together too from the academic side is fantastic. I mean, we're looking at the Department of Energy here right across from the booth.

>> And speaking of energy, I tweeted last night with a little... the hot technology this year. I put a picture of the cooling because you brought up the cooling because there's air cooling, there's liquid cooling, and then there's on-chip cooling like JetCool. We've interviewed on the NYSE and the cooling brings up the density problem. You're seeing a lot more dense chips on the stuff and these clustered systems. So the sustainability and how do I figure out the compute resource and costs? So you're going to see a lot of kind of in the weeds conversation around the sustainability equation because things are melting because the old way of doing racks is broken, so everything old has to get retrofitted. So energy, power, cooling, these things are literally hot.

Dave Vellante

>> Well, we're seeing they're pushing the envelope. We're pushing the technology to new limits. And of course, NVIDIA publishes its roadmap. It's very transparent what they're doing and things are breaking. And that's kind of exciting actually because this industry has a great track record of fixing things that are broken and then extending that roadmap to keep the whole train moving.

Savannah Peterson

>> Well, we're at that inflection point. I think we've been talking about when are we going to make AI real? When are we going to start realizing some of these high performance computing gains in terms of the world, not just in terms of prototypes. And I think if we're at the point where we're melting racks and we've got this type of a conversation going on, I mean that means innovation is moving at light speed or at a super hot speed, which is, it's exciting. I mean, as someone who's been a hardware nerd their whole life, I feel like hardware's having its moment. I feel like we... I can't wait to go look around and check out all the booths and all the different displays. One of the things that's cool just for folks at home that you might not know that's really unique about this show is there is huge pieces of multimillion dollar hardware on display everywhere. So it's not just talking about software or showing some demos. There's full on racks, there's liquid cooling demos, there's steam going up. I've saw a lot of fun stuff walking in. So it really is an opportunity not just to talk about the tech, but to understand how it works, to see examples of it. I'm curious because two years ago when I joined y'all here, we were talking... it was about 50/50 between cooling and supercomputing and quantum as was a big conversation. Then last year, we barely talked about quantum at all. And so I'm curious to see what the dialogue becomes this year and whether or not that's going to be a conversation, if our focus on AI has just made everybody loving NVIDIA chips and that's what-

>> I think quantum's definitely going to happen. That's not going to dominate this year because it's still out there. But IBM and others see it coming fast because the first order of businesses get the clustered systems up and running because you got to power the software. Jensen Huang just gave a speech in Asia. He talked about how, okay, the supercomputing is coming into the enterprise and then obviously these large scale racks, and he made a point and said, if the software can't run first generation hardware, then that's job one. And he also said that operating at scale, and that's why they're building their dedicated racks because they can see stuff at scale that no one else can see. And AWS has the same approach with their cloud. They're running at scale. They see things that no one else sees. So that's going to give the software better reliability and that'll also trickle down into the enterprise and these other apps. But the quantum is definitely coming because there's use cases emerging once they get the NIST standard figured out around encryption because everyone's scared of encryption hacking. So I mean, I think the innovations in the data center is going to be this year's theme, quantum's right around the corner. The app frameworks is going to be key too. So that's going to be like how does the data work and how do I run the agentic systems?

Dave Vellante

>> I mean, you're right about the app framework, but we are still sort of geeking out in the hardware phase. I just want to go back. You were at the IBM analyst meeting with me and a bunch of us. Arvind Krishna said, "Quantum is going to be big, but not for 24 to 36 months." Now, I've used that line with a lot of industry experts and they're like, "Nah, that's too aggressive." But I like the fact that IBM's-

>> Aggressive as in early.

Dave Vellante

>> Yes. Yeah, aggressive as in early. But what I think IBM has responded is, look, it's going to be a hybrid initially. You're going to have a hybrid of quantum and von Neumann architectures to run existing applications, and some of the new apps are going to be running on quantum. That's another leapfrog. He basically said... he said, "The growth is potentially infinite." It's just unbelievable what's going to happen with quantum.

Savannah Peterson

>> Well, I think once we hit that tipping point, I actually loved that ambitious 24 to 36 months. I think that could be early, but we're seeing this adoption curve. We were at KubeCon last week, Cloud DataCom, and looking at the adoption of Kubernetes and the acceleration of AI, and I mean, everything's going like this. So I think, I don't know, everyone focusing on higher performance computing may lead to that acceleration of quantum popping out.

>> I mean, I think quantum probably won't be dominated. It's storage, networking, and compute is the dominant. Inference obviously comes after training.

Savannah Peterson

>> I think there's going to be a lot of conversations about inference this week if I were to put my money on it.

>> Inference . Yeah, exactly. Yeah.

Dave Vellante

>> Well, so there's a lot of conversation too about the scaling laws hitting a wall. There are people saying they're not hitting a wall. There are people that saying they are hitting the wall. Training becomes secondary in terms of the dominant workload to inference. I think they're both going to be huge. I do, and I think that-

Savannah Peterson

>> Look at you, an egalitarian.

Dave Vellante

>> I believe it. I think when you look at the scaling laws, basically what we're talking about is compute, data, and parameters have to scale together. And I just think there are conversations that the industry's running out of data, which I find hilarious. But they'll build synthetic data and they'll apply private data. And now is that going to be used to train these giant LLMs? No, but they will use synthetic data. So I think it continues, and the relevance is it just keeps getting hotter and hotter and hotter and we get to cool it.

Savannah Peterson

>> To your point, I don't know that I think we're going to run out of data. Looking at the keynote today, NASA up there, they're anticipating by 2029 they're going to be processing 500 petabytes of data, and that's open science data about our solar system. It's a lot of data.

Dave Vellante

>> Yeah. This is interesting because I'm glad you brought that up because the conversation is we're running out of public data to train on, but now you just introduced space.

Savannah Peterson

>> Yeah. Talk about infinite data.

Dave Vellante

>> Oceanic exploration.

Savannah Peterson

>> Oh, exactly.

Dave Vellante

>> And that's going to be public data and then again, bring synthetic data into it. Now, will that synthetic data be able to replicate JP Morgan's data? No, but it might make up some interesting scenarios that can then go into training. So Jensen makes the point. He goes, "Look, you need both. You need training and inference. It's not like training just goes away," and so it's this virtuous cycle.

Savannah Peterson

>> Well, and not everyone's going to need to train every single new model. We don't need to reinvent the wheel every single time we're building. This was a huge conversation last week in Utah, and it doesn't also have to be my personal private data. I would much rather it was trained on open science data or that we're learning from things like that. I think there's a lot of different avenues. One of the things I love about high-performance computing is it touches every aspect of our lives, from healthcare to finance, to the climate, to the planet, to science and research. I mean, there's really nothing that this doesn't impact from a human perspective, which is why I'm so excited to share the week with you both and to have all the conversations with our fabulous guest this week.

Dave Vellante

>> Yeah, looking forward to it.

>> Yeah, it'll be great.

Savannah Peterson

>> It's going to be great. You are not going to want to miss our three days of coverage here in Atlanta, Georgia. We're at Supercomputing 2024. My name's Savannah Peterson. You're watching theCUBE, the leading source for enterprise tech news.