Generative AI has driven tech and business changes. Enterprises are shifting from exploring AI to implementing it for real-world applications to boost ROI and utilize data sets. Key AI trends for 2025: AI inference, secure language models, and preparing for an exascale future. Mainstream enterprises focus on AI inference over training large language models. Traditional large enterprises adopt AI projects by adjusting pre-trained models with data sources. RAG deployment can be complex. Shimon Ben-David discusses scaling infrastructure challenges in exascale environments. WEKA addresses issues through software optimization. Jonathan Martin stresses AI sustainability to reduce power consumption. WEKA’s software approach cuts hardware, optimizes power usage, and enables scaling in cloud environments. WEKA and GPU servers create a zero-footprint, highly scalable environment. Supercomputing event in Atlanta showcases WEKA’s initiatives for inferencing, exascale solutions, and power optimization.

Chad Dunn from Dell discussed key themes and trends at the post SuperCompute '24 event, focusing on generative AI and enterprise spending. Shift from pilot projects to ROI is noted. Data preparation, governance, and software complexity challenges are highlighted. Dell's AI factory is growing with new platforms and partnerships to support generative AI environments. The importance of partnerships for growth and use cases providing ROI is emphasized. RAG and agentic AI are emerging trends. Data governance and prep, supported by technology, are essential for successful implementation. Code generation is a significant use case improving efficiency. The outlook for AI in 2025 includes agentic workflows, smaller models, and enterprise inferencing. Dell is positioned to drive innovation in the AI space.

In a conversation as part of post-event #SC24 coverage from theCUBE, Chief Analyst Dave Vellante talks with Chad Dunn, VP of product management at Dell Technologies. They discuss the evolution of modern enterprise AI with faster, more scalable and performant infrastructural underpinnings. They also touch on the persisting data dilemma for AI enterprises.

Find out more about SiliconANGLE and theCUBE’s coverage of the latest tech events and news https://siliconangle.com/events/

They also talk about how frameworks, such as Llama Stack, Semantic Kernel, and LangChain, are abstracting the intricacies of AI applications, enabling companies to adopt and operate these technologies more easily.

Check out the full article https://siliconangle.com/2024/12/23/ai-strategies-abstracting-complexity-sc24/

Follow theCUBE's wall-to-wall coverage as the roving news desk for SiliconANGLE reports live from tech's top events https://siliconangle.com/category/cube-event-coverage/

Catch up on all episodes of SC24 https://www.youtube.com/playlist?list=PLenh213llmcaXIodD2IlNFz6spbu4As4w

#SC24 #theCUBE #theCUBEResearch #AI #AIhardware #GenerativeAI

Supercomputing 2024 in Atlanta, Georgia has begun, with a focus on AI transformation in various industries. Major players like NVIDIA, AWS, and Cerebras are present, showcasing new technologies. The show highlights the collaboration between government, academia, and enterprise, with a keen interest in open science and data sharing. There is a shift towards AI hardware infrastructure and the emergence of new system builders in the industry. Discussions also revolve around cooling technologies, sustainability, and the future of quantum computing. The show emphasizes the importance of high-performance computing in driving innovation across multiple sectors, from healthcare to finance to climate research. Stay tuned for more insights from the event.

At SuperComputing 2024, Lauren and Mitchell from Omni Services discuss their division Cool Flow and its support for companies like CoolIT. CoolIT specializes in direct liquid cooling for chips and data centers. Mitchell showcases a copper cold plate for Intel Sapphire Rapid processors. They emphasize the importance of tubing meeting specific standards for leak-free connections and the need for robust solutions in cooling systems. They discuss challenges in cooling high-power AI systems and the importance of industry standardization. The conversation concludes with a mention of a panel discussion on direct liquid cooling. Lauren and Mitchell express hopes for future innovations and collaboration in the industry. They reflect on the show's highlights and anticipate future developments in cooling technology.

The panelists discuss the WARRP reference architecture created by WEKA for AI inferencing using Run:ai and NVIDIA's software frameworks. The goal is to help customers understand and implement AI in production environments. They also touch on the importance of deploying AI models in real-time for maximum impact. The discussion highlights the evolution of AI adoption in enterprises and the role of open-source models in controlling data and reducing costs. The panelists stress the need for efficient GPU utilization as AI models become more prominent. They also talk about the challenges in scaling and orchestrating AI applications, emphasizing the importance of token economics in measuring ROI on AI investments. Overall, the panelists hope to see more energy and excitement around AI applications in the enterprise space at future SuperComputing events.

SuperComputing 2024 in Atlanta, Georgia featured a panel from NVIDIA, Supermicro, and WEKA announcing the industry's first Grace-based storage solutions to address power consumption and cost efficiency challenges in data centers, particularly in AI infrastructure. The collaboration has led to a balanced platform for high-performance computing and AI workloads, offering high memory bandwidth and power efficiency. With 72 cores on a single die, the system architecture ensures better performance for unpredictable usage patterns in HPC environments. Sustainability has been a focus, with a significant reduction in power consumption and carbon footprint. Supermicro's hardware development has leveraged the latest technologies for impressive results. The partnership between the companies is driving the development of integrated products pushing the boundaries of AI and HPC technologies. Anticipated innovations for SuperComputing 2025 aim to deliver greater value propositions and showcase industry-leading solutions.

Live coverage in Atlanta, Georgia for Supercomputing '24. Hosts John Furrier and Dave Vellante. Sam Werner, VP at IBM talks about the importance of storage in AI and HPC. Storage vendors focus on efficiency and speed due to power hungry GPUs. Exabytes of data requires efficient storage solutions. IBM's Scale system supports high-performance computing and AI, offering fast throughput and scalability. The system's efficiency and caching capabilities enable high performance and cost-effective storage solutions. IBM's focus is on making storage easier and more efficient, supporting AI and offering automatic recovery and diagnostics. They aim to bring AI capabilities into storage systems to optimize performance and data management. The evolution of storage architecture involves pushing intelligence to the storage layer to optimize data management and processing. Developers are getting closer to the hardware to improve performance and extract value. Storage remains a crucial aspect of modern computing, offering efficiency, scalability, and performance for AI and HPC applications.

SuperComputing 2024 is underway in Atlanta, Georgia. John Furrier and Savannah Peterson are discussing AI workloads and hardware advancements. Cine joins the conversation to talk about Dell's new solutions and the importance of knowing the fidelity of AI before deployment. They also discuss the PowerEdge XE9680 offering with Intel Gaudi 3 Accelerator and the benefits of optimizing this combination for cost reduction and efficient cluster building.

The conversation then delves into the importance of diversity in AI choice and the need to prioritize workloads based on performance and cost metrics. They discuss industry trends and vertical-specific use cases, such as legislative bill insights and autonomous AI agents for customer service support. The focus is on providing high-fidelity AI solutions through continuous monitoring and testing. Dell's XE9680 system is praised for offering choice across major providers and enabling innovation in the marketplace.

Looking ahead to the future, Cine hopes to see advancements in systematic software and robotics to drive top-line revenue and make life easier for users. Mania looks forward to more offerings from Dell, including the Intel Plus Intel server with Gaudi 3 in a PCIe four-factor. The conversation ends on a hopeful note, with the anticipation of new innovations and the potential for full AI takeover in the coming year.

Cerebras Systems offers the largest chip in the computer industry, 46,000 square millimeters in size, delivering exceptional compute performance for training and inference. Their innovative chip design incorporates redundant tiles to manage flaws, resulting in exceptional results in less time and with lower power consumption. Cerebras has made significant advancements in training and inference, outperforming competitors in speed and efficiency. Partnering with organizations globally, they have established a strong presence in the AI industry, focusing on high-performance inference. Their vision combines edge computing and centralized data processing to create a more efficient AI ecosystem. With the fastest inference performance in the industry, Cerebras is well-positioned to support various industries such as healthcare, energy, and finance. Their focus on hardware-software integration optimizes AI performance and drives advancements in the field.

Alex Bouzari, co-founder and CEO of DDN, discussed the transformation in the industry towards AI and data intelligence. DDN's technology enables the growth of AI across industries. The company's evolution from high-performance computing to AI has positioned them as a key player in the industry. They have focused on customer delight and stayed private to better serve their customers and focus on growth facilitated by AI. DDN's investments in technology aim to optimize performance, reduce data center space and power requirements, and accelerate access to data and content. The value add lies in acceleration, efficiency, and connecting the above the stack to the inside the stack. Discussions with customers focus on improving analytics performance and optimizing performance up and down the technology stack. Massive projects involving hundreds of thousands of GPUs are indicative of the scale and impact of AI on the global economy. DDN is poised to play a key role in the future of AI and data intelligence.

Coverage of SuperComputing 2024 in Atlanta is in full swing. Avi from Solidigm discusses their new product, a high-density SSD with 122.22 terabytes on a single U.2 form factor. This SSD is a game-changer in terms of performance and efficiency for AI applications. Solidigm's technology provides a 4X density improvement and a 10X bandwidth improvement compared to standard QLC drives. They work closely with partners to meet specific needs and are at the forefront of innovation in the evolving SSD market.

The panel at SC24 in the Atlanta Convention Center includes Dr. Luca Amalfi, CEO of Seguente, Dion Harris, director at NVIDIA, and Tim Shedd, an engineering technologist at Dell Technologies. Seguente focuses on direct liquid cooling for AI and HPC, Dell specializes in thermal strategy for data center products, and NVIDIA champions accelerated computing. The discussion revolves around the shift from air-cooled to liquid-cooled data centers, with direct liquid cooling enhancing heat transfer efficiency. Challenges in transitioning to liquid cooling include leakage concerns and supply chain scalability. Different cooling options like warm water-cooled, immersion, and phase change are being explored to improve data center efficiency. While Tim believes single-phase direct to chip cooling is optimal for Dell now, Luca argues that two-phase cooling using dielectric fluids offers advantages like reduced risk of leakage. The panel stresses the importance of adapting to innovative cooling solutions as the industry evolves, with a spectrum of air to liquid-cooled systems expected in the future. They also discuss powering and cooling high-performance data centers sustainably, emphasizing collaboration across the industry to meet increasing power demands.