SC25 | Isaiah Weiner, WEKA & Avi Shetty, Solidigm

Clips
More from SC25

Isaiah Weiner

Head of Product Management - Core Software

WEKA

Avi Shetty

Sr. Director—AI Enablement & Partnerships

Solidigm

play_circle_outline Solidigm's launch of a liquid-cooled SSD and its innovative features.

play_circle_outline The evolution of computing storage from large drives to compact SSDs.

play_circle_outline The impact of low latency and high bandwidth on AI workloads and inference.

play_circle_outline Predictions for future developments in storage solutions for AI applications.

play_circle_outline Elevating Storage to First-Class Citizen: Harnessing Customer Insights for Optimized AI Workload Solutions

Info
Transcript

Isaiah Weiner, WEKA & Avi Shetty, Solidigm

Isaiah Weiner

Head of Product Management - Core Software WEKA

Avi Shetty

Sr. Director—AI Enablement & Partnerships Solidigm

In this SC25 conversation from St. Louis, Missouri, theCUBE’s Savannah Peterson and Dave Vellante sit down with Solidigm’s Avi Shetty and WEKA’s Isaiah Weiner to unpack why storage is becoming a first-class citizen in AI infrastructure. Shetty shares how Solidigm is pushing density and efficiency with innovations such as liquid-cooled SSDs that enable fanless AI servers and ultra-high-capacity drives moving from 122TB today toward 256TB next year, while partnering with WEKA to deliver scalable NVMe-powered data platforms. Together they trace how GPUs, network... Read more

explore Keep Exploring

What notable advancements in storage solutions were highlighted by Solidigm at GTC? add

What developments and advancements have occurred in SSD technology over the past two decades? add

What are the key requirements for memory and storage from a storage-centric standpoint? add

What advancements or developments can be expected from Solidigm at Supercompute '26? add

What advancements in data center technology and customer deployment strategies are expected for megawatt racks by 2026? add

bolt Powered by CUBE AI

Isaiah Weiner, WEKA & Avi Shetty, Solidigm

search

Savannah Peterson

>> Good morning nerd fam and welcome to St. Louis, Missouri. We're here kicking off day one of Supercomputing. Very excited to be bringing you three days of live coverage on theCUBE. My name is Savannah Peterson joined by my copilot in this hardware rocket ship this week, Dave Vellante.

>> Hey, let's go. Let's go to the moon.

Dave Vellante

>> I'm ready. Yeah, I'm down.

Savannah Peterson

>> That would be fun. That would be quite the field trip.

>> On Mars.

Dave Vellante

>> One of those were a little more likely

Savannah Peterson

>> to come back from at this particular juncture. Broadcast from the moon would be pretty sweet though. I'm pumped about this, are you excited for today?

Dave Vellante

>> I'm excited for this week. - Yes. It's going to be an

Savannah Peterson

>> absolutely fantastic week. I'm excited to see a familiar face on the desk. Avi, welcome back to the show.

Avi Shetty

>> Thank you, Savannah. Thank you Dave for that too.

Savannah Peterson

>> And you brought a new friend, Isaiah.

Avi Shetty

>> Yes. - Isaiah, thank you so much for taking the

Savannah Peterson

>> time today.

>> Thank you very much.

Isaiah Weiner

>> Let's talk a little bit about what's happened in the last

Savannah Peterson

>> year since you and I last spoke just to catch up a little bit. Tell me everything that's been going on at Solidigm. Lots of action. You were smiling telling me demand is high. Give us the lay of the land.

Avi Shetty

>> Oh yeah, storage, I think last year when I spoke to you guys, GPU and HBM were the favorite child. And over the last year, we've seen storage put itself in its place where you've seen usages and certain solutions which have exposed the need for having high performing, reliant, scalable, high dense storage solutions, and that's exactly what we've been doing at Solidigm.

Savannah Peterson

>> That's your sweet spot, it's almost like you do that.

>> Yeah, that's correct. Two trends have emerged.

Avi Shetty

>> Obviously, when it comes to storage, you need low latency, fast super performing storage, and then on the other end you need high dense cost- effective storage solutions. So earlier this year at GTC, Solidigm launched their first, world's first liquid cooled SSD for storage.

Savannah Peterson

>> And it's even in the purple branding, I love that.

Avi Shetty

>> Oh yeah, it's liquid cool where the cold plate comes and touches our SSD and we have internal mechanism where the front and the backside cool itself. And as a result, now achieving fanless AI servers possible. And last year, I talked about our world's highest density SSD. Guess what?

>> How many little magic tricks

Savannah Peterson

>> and fun things are going to fly out of your pocket? Because I'm curious.

>> We are in HVM on this product.

Avi Shetty

>> We launched, I think just about Supercompute last year, and we are scaling, it's one of our hottest products this year. And guess what? We are not stopping. We are doubling this going, making 256 terabyte SSDs going in next year. A lot happening in the storage world for us at Solidigm, and we are excited to work with WEKA to bring all of this storage goodness to our customers in an easy, scalable solution way.

Dave Vellante

>> It's remarkable when you think about the progress that this industry has made. I mean, that used to be-

>> Can we see them? Will you pass the-

Savannah Peterson

>> Oh, yeah.

Dave Vellante

>> Be just racks. - Yeah, see that.

Avi Shetty

>> Racks and racks and racks and refrigerator size.

Dave Vellante

>> I mean, this is just unbelievable.

Isaiah Weiner

>> Dave, how big was the first hard drive?

Savannah Peterson

>> It's tiny. That's what I was trying to show. It's tiny.

Avi Shetty

>> This is tiny.

>> Five meg. - Exactly.

Savannah Peterson

>> This is smaller than a cell phone. I mean just to.

Avi Shetty

>> Oh, go ahead. - Please.

Avi Shetty

>> Oh no, I started working in SSDs maybe 20 years ago,

Dave Vellante

>> and my first SSD was Intel X25,

Dave Vellante

>> a 16 gigabyte SLC, SATA 2 SSD. This right there is a 122 terabyte, PCIe gen 5.

>> I honestly, I cannot believe what's happened.

Avi Shetty

>> Full saturating solution. - It's astounding.

Savannah Peterson

>> It's so light. - For all the heat this industry gets,

Dave Vellante

>> every year we produce stuff that just-

Savannah Peterson

>> Is that a pun?

Dave Vellante

>> Yeah, no pun intended.

Dave Vellante

>> We lower the cost per bit, we lower the cost of computing

Savannah Peterson

>> and it's just remarkable and it's technically quite a challenge. You got to feed the GPUs, you've got to have data. It's got to be balanced. I heard Satya Nadella talking the other day about, look, the magic of the software is sometimes you need low latency, sometimes you need fast performance from the LLMs and you got to feed these agents so there's a balanced approach as well. But you have to feed these GPUs, don't you, Isaiah?

Isaiah Weiner

>> Well, you do. And so if you just take a step back and look at the catch-up that, okay, so processors have gotten faster, but they've also gotten more cores, which means that the clock rate per core is smaller so you could run out a clock on one core so you need to be able to use all of them. PCI buses have caught up, but the networking world has caught up in the last five years. They've done a 15-year jump forward. And now with high performance and VME drives, now things are balanced. And when they're balanced, now you can actually feed them. But what do you need to do? You have to do some special things. Like the indirection unit on those drives is or what?

Avi Shetty

>> 16K. - 16K, right?

Isaiah Weiner

>> So essentially what's the block size, the native block size of the page on the drive. And that means that you need to be, particularly if you want to keep durability high, you want to be nice to the drive that does require smarts in the software and you need to intelligently place the data in the right spot. We started working together on QLC November 2023 so we've been working a long time together. They were the first QLC drives that we actually had in our hands. And we could start to work on, okay, well how do we make this work? And this morning we announced improvements for WEKA customers that they can get 90% with QLC can get 90% of the performance, but at a massive reduction.

>> That's the point about QLC.

Dave Vellante

>> I mean, explain that to people. I mean, you're talking about what's the relative cost advantage in QLC.

Isaiah Weiner

>> I mean, in many cases it could be 50% cost production, but this is on balance when you factor in everything from floor tile power up. But also you need to be able to keep the GPUs busy. What do you need in order to do that? You need very, very low latency. Do you need it for everything? No, but you do need it for some things because in the process for feeding the GPU, if you have something special, and this sort of, I guess a teaser for a future conversation with WEKA that you're having later this week, you need to be able to trick the GPU into taking information during that pre-fill phase, particularly for inference at scale. And if you have a model serving service, you need to be able to evict the old model and load the new model, and that all has to be transparent to the customer, and it has to happen really fast so that the time to first bite is within reasonable expectations.

Dave Vellante

>> Interesting, so the tricks we used to do was we'd wait for the platter to spin, we'd do all the stuff.

>> That's right, do all the tricks.

Dave Vellante

>> That's completely changed thanks to Flash, hasn't it?

Isaiah Weiner

>> It has. But some of the same challenges exist in the sense

Isaiah Weiner

>> that what's your bottleneck? So in the GPU, the memory is-

>> I was just going to ask you about the bottlenecks.

Savannah Peterson

>> We're in a really interesting, yeah.

>> And that's where the NVMe stuff comes in, right?

Savannah Peterson

>> Yeah. - As I think you were going to say.

Isaiah Weiner

>> Yeah, we're on the same page, Isaiah. Don't worry.

Savannah Peterson

>> We're definitely reading from the same book.

Isaiah Weiner

>> But I think we were talking about the evolution of the show over the last few years, and it used to be such a heavy GPU conversation. Then there's been this cooling going in, and now we're in this moment where it's what are people doing at scale? How are they achieving their AI goals? What is their challenges with inference? And so I'm curious, when you're navigating this with your customers, which are some of the largest companies in the world, how are you navigating the GPU bottleneck? How are you prioritizing your solution building, I guess is what I'm asking? Because you've got a lot of complexity.

Isaiah Weiner

>> Do you want to do it first? - Yeah, sure.

Avi Shetty

>> I'll give you a view from a pure storage- centric standpoint, and maybe you can talk about the solution aspect. So for us, Savannah, it comes down to four things, whether it's the inference or training, every SSD out there, every memory designer, you want the lowest latency, the highest bandwidth, the highest density. And at a lower cost, if I give-

Savannah Peterson

>> They want it all. Everybody wants it all.

>> If I give Isaiah an option, he wants a terabyte

Avi Shetty

>> density at DRAM-like performance and hard drive-like cost, right?

>> He's so needy. - Demanding.

Isaiah Weiner

>> Right? So what do you want to trade off?

Isaiah Weiner

>> It's a question of trade off.

Avi Shetty

>> Trade offs, that's right.

Avi Shetty

>> And that's where when we see our customers adopt our

Dave Vellante

>> storage solutions at the core data center

Savannah Peterson

>> where training's happening, you have enough power, you have enough room to scale up there. They want the best-in-class performance, the Gen- Fi lowest latency SSDs. But when it comes to inference, and a customer adoption is all varied. The inference data center could be in your back office, could be in your garage.

>> Could be on your phone. - Could be anywhere, right?

Avi Shetty

>> Yeah. - And depending on our customers,

Avi Shetty

>> whether they want more users to adopt, the need

Savannah Peterson

>> for high density storage comes in.

Isaiah Weiner

>> They want scalable solutions. Some we have certain customers where having deployed AI data centers in remote locations, which is not even accessible, they go once in six months, swap the drives, come back. So for them, you need high density. So from that standpoint, there is no from, we have a full portfolio of storage solutions. So depending on our customer needs, they pick the flavor and then the right software to go along with it.

Isaiah Weiner

>> You have this stair step up thing that happens where just for the sake of argument, let's say that you can have a certain number of drives in a system. If you have low density drives, then you need more computers, which amplifies the number of ports that you need, the amount of power draw and all these sorts of things. So high density, people think, oh, particularly for huts and other power constrained environments, people think, oh, that's where I need them. Yeah, it's true. But you also need them just on the data center floor because you want the balance of the network and the drives and stuff. And so if you have too much IO on the inside, but you can't get it out and that doesn't help. But now, with PCIe 5 and PCI 6 systems, you can get it out provided that you actually are sizing everything correctly. The service provider mentality though, and the enterprise mentality have radically different deployment models. So if I'm a service provider, I care about pods and blast radiuses and stuff like that. If I'm an enterprise, I'm just going to potentially cram it all into one giant cluster. And so the inference and training workloads look very different. What we see on the training side, if I think about our largest training customer and what they have, they have single, they have many clusters, but their largest cluster is just enormous. It's too big to fail, and there's 100,000 GPUs in front of it, and they need to get, some of their clients are more important than others. The software clients. They get single and double digit microsecond latencies on some of these calls because that's the way that they can actually update their models. They're looking to reduce their epoch times because they want to get 11,000 updates a year out in order to have iterative product development that is totally different from inference. Customers who are interested in how do I make sure that I don't lose people's attention? There was this cool retro computing show called Halt and Catch Fire that talked about this thing called the Doherty Threshold.

Dave Vellante

>> Of course, amazing show. - Remember the Doherty Threshold

Isaiah Weiner

>> was like 400 milliseconds on the echo return. Well, everything, it's old is new again. And it's the same thing with your inference cases where if the amount of time it takes to load that model, load the context window and give you time to first bite takes too long, people get bored and they move on. And it might be a batch- oriented inference workload, in which case who cares? But if it's on your phone or like, "Hey, I want to look at what animal is this or what bird sound is that I have five seconds," or something like this, maybe less.

>> I would say less. - Yeah.

Isaiah Weiner

>> I mean, inference is what makes AI real.

Savannah Peterson

>> None of the stuff that we're talking about in terms

Savannah Peterson

>> of its real-world application and experience exists if inference isn't optimized and people don't have that. We were talking about Grok earlier, Avi and I, and when you talk to Jonathan, inference is everything. I mean, that's the currency, right.

>> And it requires more data too.

Avi Shetty

>> Correct. - Yes.

Isaiah Weiner

>> Yes. - Which means back to density.

Avi Shetty

>> If you don't have the ability to actually handle that

Savannah Peterson

>> and give it the IO that it needs, you're going to feel it.

Dave Vellante

>> Back to those trade-offs that you were talking about,

Isaiah Weiner

>> there's always the pyramid, right? Processors are going to go faster than memory, are going to go faster than deep storage. And so you've got this sort of memory wall that you hit that you have to continuously balance, so how are you attacking that problem together?

Isaiah Weiner

>> The memory wall concept was defined in a 2024 paper.

Dave Vellante

>> Right. - And it essentially said

Isaiah Weiner

>> that memory has become the bottleneck for GPU feeding. And they were talking about it in terms of general GPU use didn't. I don't think they specified training versus inference. What we do is we extend the area of memory that the GPUs can use. So from a GPU perspective, nothing changes, it's still the same address ranges. But what we do is we are feeding at essentially memory bandwidth speeds into the pre-fill phase of that inference workload so that when the decode starts to happen, you're actually able to think, "Oh, I have this infinite amount of memory. " In practice, this means you need two things. You need the infrastructure to support it, which means the hardware and the software, but you also need to have hooks in your inference runtime so VLLM or TensorRT in order to take advantage of those things. And if you were paying attention to GTC, there was the Dynamo project with KV Cache. Now that is using GPU memory exclusively. What we did with augmented memory grid, which is the teaser for the future CUBE session for Watchers, is we extended that into storage. And this was at the first GTC of the year. We announced that we had done this and we were the reference design for it with NVIDIA.

Dave Vellante

>> So you're creating a virtual pool from the GPU memory to the storage layer?

>> I can think about it as a pool.

Isaiah Weiner

>> I mean, if you want to get a little bit more alien, it's a wormhole from the GPU into storage because you're looking through a keyhole of GPU memory into a vast universe of high performance, high density NVMe.

Dave Vellante

>> Sure. - So what we're really talking about here is

Savannah Peterson

>> AI economics at scale, AI economics to a degree.

Avi Shetty

>> Yeah. - This is why partnerships like

Savannah Peterson

>> yours are so important. You need each other to optimize that for your customers. Outside of the inference layer, the super technical bit that we were just talking about, when your customers, because you guys co-collab on a lot, when your customers come to you, what's the number one thing they're looking for in terms of a solution? Is it that I want to do it faster, better, stronger? Is it that I want to have the best stack and tools and storage for this? What are the words coming out of their mouths?

Avi Shetty

>> I mean, efficiency is one of the most important things across the spectrum. We talk about AI, you absolutely need to feed the GPU. That's the most expensive commodity in your whole data center. And anytime a GPU is idle, that's revenue lost.

>> Wasted money. - Wasted power consumed.

Avi Shetty

>> And as a result, efficiency becomes key.

Savannah Peterson

>> We talk about performance, performance, performance, but guess what? When things are performing, it also gets harder. So cooling-

Savannah Peterson

>> Quite literally, we were talking about how it's warm in here because of all the-

Avi Shetty

>> This is my first day at the conference and I have not had a chance to visit all the booths, but I literally saw 20 companies talking about cooling, immersion.

Savannah Peterson

>> Oh, at least. - Liquid.

Dave Vellante

>> Yeah. - That's just five minutes

Avi Shetty

>> of me walking from the entrance here.

Avi Shetty

>> And that's efficiency. Efficiency is not just about high performing, low latency. You need the entire solution to work in synergy with the networking, with the compute, with the software abstraction about it to give what our customers really want.

Isaiah Weiner

>> That means you need to measure stuff.

>> That's good. - And actually, I think it was last month

Isaiah Weiner

>> or the month before, the spec storage committee

Savannah Peterson

>> finalized the plans for integration with spec power, where you can take a power analyzer, which includes you're all in power, all in watts, which includes cooling. And say, all right, what's my performance per watt? And then once you know what your performance per watt is, you can go look at what's your IO per watt cost. And now you can balance that with what's my IO per watt per dollar, which is the real mitigating factor for a lot of people.

>> Per token, right? Because ultimately I'm producing

Dave Vellante

>> intelligence, which I guess I come up with interchangeable with tokens, right?

>> Absolutely. The whole deal is to bring down millions

Avi Shetty

>> of tokens, the dollar to generate millions of dollars tokens every iteration down, right?

>> And in the training side, I mean, look at GPT-5.

Dave Vellante

>> A lot of people complained about it when it was released, but it was a significant economic improvement over the previous generations. And then when you get to inference, it's a whole different set of economics.

Isaiah Weiner

>> And so if you're hosting your own model, this matters even more. There's lots of folks out there who do use services like that, and for them it's about the cost of the API. Yes, cost per token matters, but they really have no control over am I reusing a token? Am I recalculating? So if you want to peel that layer back and take control over the economic parameters, then you need to host your own models because that's where things like AMG and Dynamo and KV Cache and the efficiencies that high density, high performance NVMe come into play for.

Dave Vellante

>> I think too often storage is left out the equation. I always call the new Jensen's laws as buy more, make more or save more. That's the new economics.

>> Jeez, Dave. - But it's

Dave Vellante

>> actually true if you're power constrained, if

Savannah Peterson

>> that's your constraint and you can double or triple or quadruple your performance.

>> Or you're just yield constrained, right?

Dave Vellante

>> Exactly. But Jensen always emphasizes it's an entire

Isaiah Weiner

>> system, but he didn't talk a lot about the storage piece of it. That's where you guys come in and that balance that we've been talking about is critical. Otherwise, you're not going to be able to feed those GPUs and you're going to blow apart the economics.

Isaiah Weiner

>> You know that old adage about special effects that if you noticed them, they didn't do a good job?

>> They weren't. Yeah, exactly. - I feel like storage gets

Isaiah Weiner

>> relegated to that a lot.

Avi Shetty

>> Yeah, it does. But all the trends we are hearing-

Savannah Peterson

>> It's a good analogy, Isaiah.

Avi Shetty

>> All the trends we are hearing in the AI world,

Savannah Peterson

>> synthetic data, outpacing real data, the need for more inference data, all of that means there's more data. The AI data pipeline, you need to store that data effectively, reasonably well reliably, because that matters.

>> In the right place. - At the right place.

Avi Shetty

>> And as a result, I feel we at Solidigm feel we are in the

Savannah Peterson

>> right trend where we are meeting our customer demands. Though demands are right now outpacing supply across the industry, it's a pretty good year going into '26.

>> Do you see any signs of that demand waning?

Dave Vellante

>> I mean, I'm sure you sleep with one eye open in this business moving so fast. Is there any indication that demand is not going to continue to outstrip supply for the foreseeable future? .

Avi Shetty

>> I think we direct attach to NVIDIA GPUs, right? The more GPUs, every GPU, every B200 system out there has eight E1. s slots direct attach. And then to feed the GPU, you need an object store box, either from WEKA or from other suppliers.

Isaiah Weiner

>> File system ideally. - File systems.

Avi Shetty

>> And that continues to grow, which is a good thing.

>> Yeah. - Yeah.

Savannah Peterson

>> That is a good thing. Well, Avi,

Isaiah Weiner

>> your positive energy is contagious.

Dave Vellante

>> I appreciate it. I feel like you just uplifted us. I can tell from just your energy alone how good things are going from both of you. So let's take a look into the future. When we get to hang out at Supercomputing in 2026, what do you hope to be able to say then that you can't yet say today? Avi, I'm going to start with you, but then I'm coming to you.

Isaiah Weiner

>> That's more than a prediction, that's what do you got cooking?

>> Not a prediction.

Avi Shetty

>> I actually, next Supercompute '26, Solidigm will be there with a 256 terabyte. First time when we talk about 122 being a hot product, we are not stopping. We're going to go push the envelope to push the density bar higher, bring the cost down for our customers so that we can give them more density at scale. We are definitely going to do that. That's one prediction. And the second one, I feel you'll see more usages, RAG offload, KV cache, pushing a storage-centric design architecture where you are looking at storage when you're designing inference data centers or core data centers with storage in mind, with flash in mind. It has to be the case going forward.

Isaiah Weiner

>> Everybody thinks they're compute-bound, and then you give them a good storage and then they realize they weren't compute-bound, right?

>> I bet. Oh yeah. That is actually-

Isaiah Weiner

>> The wall clock comes down on its odds.

Savannah Peterson

>> Yeah. - I think we're going to see megawatt racks in

Isaiah Weiner

>> 2026 today.

Avi Shetty

>> Oh, that too. - We have customers who are deploying

Isaiah Weiner

>> almost 400 kilowatts per rack.

Dave Vellante

>> Yeah. - And that's with GPUs hot.

Isaiah Weiner

>> When they're not hot, when they're idle,

Savannah Peterson

>> it's still a 40 kilowatt rack, it's still a lot. And I would say that the storage as a first- class citizen in AI, I think is definitely going to be a realization that people have. But it takes a couple of things. It takes them realizing, "Oh, I wasn't compute bound," and that means they need to know more about their workload. And a lot of people don't. It also, it takes people realizing that tokenomics only get better when you host it yourself.

Savannah Peterson

>> I think that's an outstanding point. It's got to make you feel good hearing him say that, Avi, you're like, "I know we've been waiting. " And they're ready.

>> And more drives. - Bigger drives, bigger drives.

Isaiah Weiner

>> I love it. - Buy more drives, bigger drives.

Savannah Peterson

>> And that 256 terabyte drive PCIe 5 or PCI 6?

Isaiah Weiner

>> Five. - Okay.

Savannah Peterson

>> Well, we look forward to confirming both of those

Isaiah Weiner

>> predictions and seeing your new solutions.

Avi Shetty

>> When you have bigger drives, you need fewer of those.

Savannah Peterson

>> Yeah, it's more efficient.

Isaiah Weiner

>> The customer doesn't think so, they're just like,

Avi Shetty

>> "Oh, I can have more."

Savannah Peterson

>> Give me all the things.

Dave Vellante

>> The hardware hoarding is very-

>> Jevons paradox.

Dave Vellante

>> Lower the price, they'd buy more.

Isaiah Weiner

>> That's right. Yeah. Good bet. They're going to take on bigger problems is really what it boils down to. Nobody ever says, "Oh, I can save margin on my operating expenses," they just say, "Oh, I can do more," right.

Savannah Peterson

>> That's a really good point. This has been fantastic. What a beautiful way to start the show. Isaiah and Avi, thank you both so much.

>> Thanks guys. - Thanks.

Savannah Peterson

>> And thank you, Dave. Always fun.

Avi Shetty

>> I can see you getting excited taking notes over there

Dave Vellante

>> so that'll break it away.

Dave Vellante

>> Save this book for 2026, bring it back.

Savannah Peterson

>> Yes. - You've done that before.

Dave Vellante

>> I do it every- - Where do you think I got the idea?

Savannah Peterson

>> It's cut.

Dave Vellante

>> I hope you are just as inspired as we've been

Savannah Peterson

>> by this starting segment here at SC25 in St. Louis, Missouri. My name's Savannah Peterson. You are watching theCUBE, the leading source for enterprise tech news.