SC24 | Chad Dunn, Dell Technologies

Clips
News
More from SC24

Chad Dunn

Vice President of Product Management for Artificial Intelligence and Data Management

Dell Technologies

Agentic AI, abstracting complexity and gen AI partnerships: Key highlights from Supercomputing 2024

Supercomputing 2024 blended innovation with thought leadership, exploring realistic themes around enterprise AI challenges and inherent complexities.The event showcased cutting-edge technologies and discussions, signaling a shift from experimentation to operational deployment in enterprise AI strategies.“We’re seeing that the enterprise spending is growing on generative AI, and I think we’re moving beyond the pilot and POC phase that the customers have been in. Now they’re looking for that ROI and they’re looking to justify that spend,” said Chad Dunn (pictured), vice president of product management, artificial intelligence and data management at Dell Technologies Inc

play_circle_outline SuperCompute '24: AI Trends and Enterprise Needs for GenAI with Focus on ROI and Security

play_circle_outline Dell AI Factory advancements include new server platforms, GPUs, software updates, and data lake house.

play_circle_outline Partnerships key for Dell's growth strategy in GenAI, leveraging expertise and reach.

play_circle_outline Handling data governance challenges in RAG processes with technology, partners, and professional services.

play_circle_outline Outlook for 2025: focus on AI, agentic workflows, use case optimization, inferencing in enterprise

Info
Transcript

Chad Dunn, Dell Technologies

Chad Dunn

Vice President of Product Management for Artificial Intelligence and Data Management Dell Technologies

Chad Dunn from Dell discussed key themes and trends at the post SuperCompute '24 event, focusing on generative AI and enterprise spending. Shift from pilot projects to ROI is noted. Data preparation, governance, and software complexity challenges are highlighted. Dell's AI factory is growing with new platforms and partnerships to support generative AI environments. The importance of partnerships for growth and use cases providing ROI is emphasized. RAG and agentic AI are emerging trends. Data governance and prep, supported by technology, are essential for suc... Read more

explore Keep Exploring

What are some current trends in enterprise spending and focus related to generative AI technology at SuperCompute? add

What are some of the highlights and advancements in AI technology at Dell currently? add

What is the significance of partnering with solution providers who have expertise in deploying and operating Generative AI in the fast-moving market of GenAI? add

What steps are involved in engaging with a customer in the AI process, specifically in terms of data discovery, governance, prep, and technology application? add

What is Chad's outlook for 2025 regarding AI and agentic workflows? add

bolt Powered by CUBE AI

Chad Dunn, Dell Technologies

search

Dave Vellante

>> Hello and welcome to our post SuperCompute '24 coverage from theCUBE. My name is Dave Vellante. And I'm pleased to welcome Chad Dunn from Dell to the program. Chad, good to see you again. Thanks for coming on.

Chad Dunn

>> Hey, you're welcome Dave. And it's really great to see you.

Dave Vellante

>> Yeah, so SuperCompute 2024 was pretty interesting. Chad, it felt like GTC, but more open systems. So coming out of SC '24, what were the key themes and trends that you felt are going to define the future of AI? I'm specifically interested in what you see as the customer needs for enterprises in GenAI.

Chad Dunn

>> Yeah, well look, I think that SuperCompute is really becoming an AI show, which is fantastic. So we saw a lot of interesting technology out there. And we also met with a lot of customers and talked to them about what they're seeing. We're certainly seeing a few trends. We're seeing that the enterprise spending is growing on generative AI and so I think we're sort of moving beyond the pilot and POC phase that the customers have been in and now they're looking for that ROI and they're looking to justify that spend. We see customers trying to get to a consistent infrastructure stack to run generative AI. They need something that not only fulfills their AI needs, but also is something that IT can support on a long-term basis. We're now seeing people think much more about security than they had in the past. We had this mad dash to the results of generative AI. Now we've got to go back and make sure that we're securing it, that we're operating it responsibly. And sustainability is also there. No surprise that this sort of technology takes a lot of power and there's a lot of work between us and our other partners to help our customers stay green as they adopt generative AI.

Dave Vellante

>> Our research definitely showing those are the key points were aligned on that. And I think just the comment on the spending, we agree, we're starting to see GenAI actually throw off a little bit of cash, a little bit of positive return. And the GenAI spend has been stealing from other areas. So there's been a little bit of a cap on overall IT spending. On our latest survey with our partner ETR shows that things are loosening up a little bit so hopefully 2025 is going to be a big year, but there are other roadblocks. I mean, particularly around data. There's surrounding, of course there's a lot of experimentation going on in the cloud, but not everybody wants to go to the cloud. In fact, there's a lot of data of course on-prem. We talk about data gravity all the time. What are you seeing as the biggest roadblocks with customers and how are you helping?

Chad Dunn

>> I think you're right. I think data is a huge challenge and every generative AI conversation I have with the customer turns into a data conversation. We probably spend 60% of our time on that. Customers are challenged with discovering the data that they want to use, the hygiene of the data, the format of the data, the governance of the data. When we talk about data fueling generative AI, it's not just the old saying, garbage in, garbage out. If it's garbage going in, it's going to be really expensive garbage coming out after what you spend on generative AI. So data prep, data hygiene and making sure that you have the highest quality data going into the process is really, really important. So there's a lot of focus on that. And data governance is really becoming everyone's job in the company now, not just a group of people who are tasked with looking after the data. I think that we're starting to see software abstract some of the complexity of generative AI apps. If I go back a year and look at what a stack would look like, there are a lot of components there that a customer needs to know how to operate. Now I think we do a good job at the AI factory of doing a lot of that hard work for them, but software is also starting to catch up. So you see things like Llama Stack and Semantic Kernel and the evolution of Lang Chains and these are all making it more approachable and more operatable, if that's the right word, for our customers. And then of course, complexity is still a big challenge, and not just the complexity of the technology. We all understand that. I think the complexity of the ecosystem and how quickly it's moving and how frothy it is right now. There are so many companies with so much innovation and sometimes in very small, but very meaningful areas and so even us as one of the leading vendors in the space, keeping up with that ecosystem with the right kind of partnering and go to market is a challenge and I know it's a challenge for our customers to review and potentially consume all that technology.

Dave Vellante

>> Yeah, I'd like to follow up on that. I'd love what you're doing with Meta and Llama Stack. I don't know if you're the only, but you're certainly a leading, if not the leading partner with Meta for on-prem, but these and Llama Stack brings some framework and standardization to simplify things for customers. At the same time, I don't know about you, I'm sure you're seeing the same thing, these models just keep getting better and better. I was playing around with Grok the other day. I tweeted . They do some amazing things under Elon. What's happening with Llama, just OpenAI, it's really amazing the pace. So how do you see customers? Are they able to pop models in, pop models out, replace models? How are they thinking about that? What's your advice in that regard?

Chad Dunn

>> Well, I'm going to cop out and say it depends. It's going to depend on your use case. So if you think about categories of these models, you look at something like Llama 3.2, which is absolutely massive and now multimodal, and we see so many enterprises picking this up. One, because it's open source and second because it's a pretty amazing model. And they're able to do incredible things with it. But then we also see customers where the use case is a little different. So it may be very sensitive internal data and very focused enterprise data that they want to operate on. So they may choose a smaller model, something that's more efficient, that doesn't have all the bells and whistles and do all the things that they don't really need to manipulate their internal enterprise data. So I think you're going to see the model ecosystem continue to evolve. Yes, we're going to continue to see the larger and larger LLMs, but at the same time look for companies to innovate on small, more efficient models that are very purpose built for the use case at hand.

Dave Vellante

>> I think that's right on. I mean, it does depend and that means that puts pressure on you to be open, to be flexible and support your customers with whatever dependency they have. You mentioned the AI factory before. I mean AI factory, you guys exploded that out of GTC last year. It was amazing to see what happened. That was really a key milestone and pivot point for Dell and its momentum. What's happening with the AI factory? What's new?

Chad Dunn

>> Well, there's a lot. I mean given how fast the technology is moving, how fast we have to move from a hardware perspective, the software perspective, there's always something new happening in AI factory. If I look at some of the big highlights right now it's addition of new server platforms in the XE line like the 7740 or even on the higher end, the IR 7000 and new GPUs coming from the video H200 MVL, the GB200 series. These are the really highest of the high end for those most demanding workloads and these are all liquid cooled systems as well. So we're going to start to see that liquid linked up from service providers down into the enterprise in the coming year. I'm certain of it. Silicon diversity among our accelerator partners with AMD soon Intel and of course, and NVIDIA being partner number one. Even on the software and the model side, what we're doing with the Dell Enterprise Hub with Hugging Face, we're adding new models there. We've got support for AMD accelerators, we've got Llama 3.2, we've got Mistral and there's more to come in that space. Lots of things happening in the coming year with the data lake house. And this is really evolving into a data platform that's going to service all the data prep needs that I talked about for your unstructured data as well as structured. And then AI PCs over on the client side. We've got a broad line of AI PCs and we're seeing amazing adoption of that technology now. So it's a pretty amazing time. It's pretty amazing technology to be involved with generative AI. It's amazing to be at Dell because we touch so many aspects of it from client and laptop right up to the huge liquid cooled infrastructures that are driving some of the most incredible innovation in the world.

Dave Vellante

>> It was amazing how many liquid cooling companies we saw at SC '24. I mean it's just exploding. We certainly see them at shows like Dell Tech World, but I mean they were out in force down in Atlanta. You mentioned liquid cooling and I was at your lab, one of your labs, your AI lab recently in Austin and you guys made a big deal out of the hybrid nature that you're still using air cooling and you're using software to optimize that air cooling. You're using warm water, which was really interesting. I think that was a recommendation from the OCP earlier this year. So you're living into that. The piece about Hugging Face, the optionality that we were talking about before with Enterprise Hub, that's really important. I also think it's relevant that your swim lane includes data management, so the lake houses is really important. I know you're not responsible for AI PCs, but you like I, probably want one.

Chad Dunn

>> I want one very badly.

Dave Vellante

>> Indeed.

Chad Dunn

>> Sam Burd if you're listening.

Dave Vellante

>> Yeah, definitely, send them our way. But I want to double down on partnerships. Dell positions itself as a partner first company. I don't think that was always, frankly, the perception of Dell and previously with that EMC, but you guys have leaned in to the partnerships. I think it's fundamental to your growth strategy. Maybe you could talk a little bit about that specifically in the context of GenAI.

Chad Dunn

>> Yeah, absolutely. Look, we recognize that it's a very fast-moving market and challenge partners and solution providers, they give us amazing leverage to reach those customers, but they're also bringing with them a lot of expertise. And I think this is the real value add. As we look across our partner landscape you see some of the really advanced ones who invested in the skill sets for deployment of generative AI or even operating generative AI environments for their customers. And so we see a lot of investment from those partners in that area. And those are the ones that we want to partner with because not only are they giving us the leverage to reach those customers, but they're also there to deploy these systems to work with these customers as a trusted advisor going forward. So it really is good for us because we simply couldn't get the kind of scale that our partner community can provide to us. So partners are going to continue to be extremely important to us.

Dave Vellante

>> I mean, we've seen you guys at the analyst events this year, last year. I mean Meta was up there, I mean, IBM is a partner, ServicNow as a partner. You mentioned Hugging Face, a number of customers in that the GPU cloud space, what you're doing with Llama Stack. And this continues to grow. I'm kind of hopeful that we'll see more at DTW was which is in May. I just booked my analyst acceptance so I'm excited for that. I mean it's like we haven't even ended 2024 chat and we're already prepping for next year's Dell Tech World.

Chad Dunn

>> I know. We seem to at Dell live our lives from one big Dell event to the next, and obviously the next one is the big premiere event. And for sure we're going to be talking about some amazing new partnerships that we're doing in the generative AI space. We'll have updates on the ones that you know about and I think we'll have some new names as well. And you're certainly going to see some big roadmap advancements from us in the AI factory as well as the Dell Data Lakehouse.

Dave Vellante

>> Yeah, thank you. We were talking earlier about some of the budgets, and I know you guys have gone through many enterprises. Initially you just got hundreds, at least certainly, maybe even thousands of the company the size of Dell of potential use cases. So you had to spend some time paring those down, prioritizing running the business cases. So what are you seeing emerge? What are some of the use cases that enterprises are prioritizing that are going to throw off cash that'll make GenAI self-funding?

Chad Dunn

>> Well, look, yeah, like any other company, we acted really quickly to start to leverage generative AI technology internally. And so the idea of let's let 1,000 flowers bloom and then see which ones we want to cultivate, which is what we did. And when we did a survey around the company from the very smallest little pocket of AI experimentation to some of the bigger projects, we came up with something 900 and somewhat projects. Not quite sustainable. I'm sure that the ROI is not there for every one of those 900, but the ROI and the payoff is going to be there for a lot of them. So what we've had to do, and we're having this conversation with many of our customers is start to be very mindful about what you put in pilot and assume that it's going to come out of pilot and it's going to have to go into production. And what would you do differently if that's the case? And some of the things that you would do differently is you wouldn't start a pilot unless you could document an ROI, something that finance could sign off on, someone with budget that could sign off on. You would want it to be very time-bound. We look at pilots internally right now and given how fast things move, if we can't do it in 90 days, then we'll put it off and we'll come back to it. Things that are feasible today are infeasible today in six months it can be very, very feasible with the speed that the technology is moving. So we've had to put a lot of internal governance around it to say, "Great, we've gone through this phase of experimentation, but we're going to have to come out the other end with something that IT can put into production and support." And we've seen examples of this. Externally we work with customers where RAG is one of those big horizontal patterns that we see for a lot of different use cases inside of these companies. And if you think about what RAG does, it's taking your internal data sources, documents, enterprise data, et cetera, out of their source systems, whether it's SharePoint or SAP or Oracle or Office 365, it's bringing all that data and chopping it up and putting in a vector database, but you effectively lost all the permissions and governments that those source systems had and now you've flattened that permission space into a vector database. How do you then protect who can see that data? Who can do things with that data? These are things that people are not usually thinking about when they go into a pilot. And so when you go back and revisit something to bring it into production often it could be a bridge too far to get it into production. So you got to think about these things early. Now, in terms of the use cases, look, it's the ones that we all know and love. It's going to be content generation, but it's going to be more specialized. Content generation for marketing or code generation for code optimization, for synthetic data. It's going to be variations on RAG for customer service or something that we do next best action to give our customer support agents guidance on how to help our customers resolve issues more quickly based on all of our internal enterprise data. Been great success with that. But you're going to see RAG for simple enterprise search. You're going to see financial companies use it for some of their common transactions, like third-party risk management, et cetera. So look for more and more of these horizontal patterns to emerge and then they're going to blossom into the use cases that really pay off for customers. And I think one of the horizontal use cases that's sort of on everyone's lips right now that we should be paying attention to is the rise of agentic AI where this becomes more and more automated and starts to judge your intent versus following your instructions. And I think that's going to be the technology of the year in terms of use cases in '25.

Dave Vellante

>> Actually, I want to ask you what your forecast is for 2025, but I want to come back to this problem that you mentioned. When you RAG all this data and you lose your governance, who can have access to what? How does Dell handle that? I'm presuming it's a combination of technology, partners, professional services. Maybe you could add a little color and double click on that for our audience.

Chad Dunn

>> It really is all of the above. I mean when we engage with a customer at whatever phase, often it's consulting led and it's a data discovery exercise and data governance exercise and data prep before we even get to the generative AI phase. And it's being able to think about those things and apply the right technology. Some cases that's going to be Dell Data Lakehouse where we are preserving the permissions of the source systems as we bring data into it. We're going to continue that as we add support for unstructured data, which I think is going to be incredibly useful. But really it's just the awareness and governance of what you're going to be able to do with that data, how you're going to treat that data. If I look at the professional services with AI factory we have lots and lots of them that can be part of AI factory. Data prep is required. That is not optional. We always want to have a data conversation with you before you go down that generative AI path and get something in place because we want to make sure that you're not putting garbage in and getting very expensive garbage back out.

Dave Vellante

>> What about code generation as a use case? I mean, I talked to developers, they're clearly leveraging the code generation capabilities of these LLMs. I talked to business analysts and they're loving it because they're not really professional coders. At the same time, sometimes in the media you see some negative sentiment, but I think on balance my takeaway is it's a pretty significant productivity enhancer. What have you guys seen internally and what are you seeing with customers in that context?

Chad Dunn

>> Yeah, I don't want to say that the acceptance of code generation is based on the age of the programmer. It might correlate a little bit, but I think it's the willingness to accept that change.

Dave Vellante

>> Even it's inversely proportional potentially.

Chad Dunn

>> It may correlate that way, that's all I think. But look, as a programmer this is your stock in trade and then somebody moves your cheese and you decide how you react to it. And we see people run toward it and leverage really, really quickly and then we see others that are more laggers. But you know what I think, most people get through it really, really quickly because they do start to see that ROI. And whether that's in things like generation or assistance on generating code or code optimization or an area where we sell some of our first big benefits, we're in test case generation for our software. So that was really the first big one for us. And we've expanded code generation to lots of different areas inside of Dell now, and we're certainly seeing our customers do the same.

Dave Vellante

>> Yeah. So I want to end with what your outlook is for 2025. You mentioned agents. I'm going to predict there'd be a lot of agent washing, but I think there's also going to be a separation of the wheat from the chaff. And I think there's some real interesting activity going on in substantive agentic, but what's your outlook for 2025, Chad?

Chad Dunn

>> Well, I mean obviously I work on AI Dell, so I got to be pretty bullish on AI, but I actually am. I think you're right. I think the ecosystem will continue to self optimize itself around the use cases that make sense. I think agentic workflows will be absolutely key to unlocking some of these new use cases to get us to that next level of productivity. I think you're going to see smaller and more focused use case based models to make it more efficient. And I think you're going to see more and more use cases that are just inferencing in the enterprise and look for us to adapt to those changes in the AI factory.

Dave Vellante

>> Chad, I appreciate you making some time for us post SC '24. Looking forward to SC '25. Like you said, it's become an AI show. Dell is right in the thick of it. You are within Dell, right in the thick of it. You've chosen well, my friend, with AI and your title and really doing great work with customers. So thanks so much. Have a great holiday and really appreciate your time.

Chad Dunn

>> All right. Thank you Dave. Take care now.

Dave Vellante

>> Yeah, you bet. Okay, and thanks for watching everybody. This is Dave Vellante for theCUBE. Remember all our activity is on siliconangle.com with the news and all these videos from SC '24 and our other events are on thecube.net. So check that out and we'll see you next time. Thanks for watching.