theCUBEd Awards | Hammerspace Global Data Platform by Hammerspace (Top Data Storage Innovation)

Clips
News
More from theCUBEd Awards

David Flynn

CEO

Hammerspace

How Hammerspace makes data accessible across any and all storage

Hammerspace Inc. is revolutionizing data storage by doing the formerly impossible: unifying data through a parallel NFS, or network file system, architecture. The technology has a lot of potential in the enterprise space, especially when it comes to generative artificial intelligence, according to David Flynn (pictured), chief executive officer of Hammerspace.David Flynn of Hammerspace talks about the advantages of data orchestration.“You should be able to have your data at the highest velocities from whichever data center and have it be the same data, even as it moves across systems, across sites and through time, it’s got to be always in the same logical place,” Flynn said

How Hammerspace makes data accessible across any and all storage

Hammerspace Inc. is revolutionizing data storage by doing the formerly impossible: unifying data through a parallel NFS, or network file system, architecture. The technology has a lot of potential in the enterprise space, especially when it comes to generative artificial intelligence, according to David Flynn (pictured), chief executive officer of Hammerspace.David Flynn of Hammerspace talks about the advantages of data orchestration.“You should be able to have your data at the highest velocities from whichever data center and have it be the same data, even as it moves across systems, across sites and through time, it’s got to be always in the same logical place,” Flynn said

play_circle_outline Tech Innovation CUBEd Awards celebrating winners and innovation in top data storage category

play_circle_outline Importance of having data constantly in motion and accessible

play_circle_outline Customer examples in large language model training, financial services, bio sciences, media and entertainment

play_circle_outline Integration of Hammerspace into AI clusters for shared and orchestrated data environment

play_circle_outline Hammerspace's secret sauce in delivering higher performance and agile data access

Info
Transcript

Hammerspace Global Data Platform by Hammerspace (Top Data Storage Innovation)

David Flynn

CEO Hammerspace

David Flynn, CEO of Hammerspace, joins theCUBE to discuss theCUBEd Awards and Hammerspace's groundbreaking approach in the data storage innovation category. With over a decade of experience, Flynn and Hammerspace are pioneering fluid data orchestration, leveraging an innovative architecture to combat the enduring challenges of data management. This discussion, led by John Furrier and theCUBE researchers, delves into Flynn's journey from Fusion-io to establishing Hammerspace as a leader in data platforms.

Key insights include how Hammerspace transcends ... Read more

explore Keep Exploring

What is the topic of discussion for today's interview with David Flynn, CEO of Hammerspace? add

What are some challenges and implications of having data locked in storage? add

What industries are currently utilizing Hammerspace for their workflows? add

What is the unique angle of Hammerspace in terms of storage systems and software integration for enterprise systems? add

What factor has made it difficult for the company to secure funding, despite their ability to deliver higher performance and more agile universal data access? add

bolt Powered by CUBE AI

Hammerspace Global Data Platform by Hammerspace (Top Data Storage Innovation)

search

>> Hello, everyone. Welcome to theCUBE Studio here in Palo Alto. I'm John Furrier, host of theCUBE. We are here to celebrate the Tech Innovation CUBEd Awards where all the winners are coming on the program. Our inaugural program of theCUBE Awards. I'm going to do another one this year. It was such a big success with 43 judges. The competition was fierce and we received a lot of huge, exceptional number of high-caliber entries showcasing the incredible innovation across the tech landscape. Today we're joined by David Flynn, CEO of Hammerspace to discuss the award-winning entry they had behind the innovation in one of the most contested categories. In fact, it's the hottest category, top data storage innovation. David, congratulations. Thanks for coming on. Great to see you. You've been a Cube alumni since 2010, 2015.

>> Back in the Fusion-io days. Glad to be here, John. Thanks.

>> Well, well done. Again, Hammerspace. Remember when you started the company, we chatted. You told me about the vision. Unifying was a big thing. I want to get into it, but before I get into this category was so competitive because storage right now with AI is huge because a lot of the big players, the top six customers, the CapEx spend's high, enterprise is booming. You saw the NVIDIA earnings recently yesterday. Enterprise, Dave Vellante and team on theCUBE research predict a trillion dollar market in the enterprise when you throw in everything, power and cooling, at a 15% CAGR over the next 10 years. Massive growth. That's huge numbers. Back in the enterprise. It never left. It kind of went to the cloud, but now it's connected and distributed computing, so you got data storage beginning to surge. Amazon earnings. You're looking at their earnings. Storage is booming for them too.

>> But I have to correct you on something. Despite the award, we're not a storage company any more than VMware was a server company. We're the abstraction layer with storage interfaces in, storage interfaces out. So in a sense, you could call it a storage company, but-

>> The top data platform.

>> There you go. Data platform would be a-

>> Yeah. I mean, the competitors out there, you know who they are, they have a storage component, and again, this is when you started Hammerspace. Share the vision of Hammerspace initially and where you started and where you are today because I think it's important for people to understand there's huge growth in storage as a category, but that's not the story. The story, it's the data layer, it's the software behind it and it's the macro and also the macro tech environment is more diverse and needs that scale. Explain the vision.

>> So the vision is you should be able to have your data at the highest velocities from whichever data center and have it be the same data, even as it moves across systems, across sites, and through time, it's got to be always in the same logical place. The same way when you switch your iPhone, you can find everything where you last left it because it lives in a logical or virtual place, that same thing has to happen now with the enterprise exabyte scale data to where it's simply omnipresent and preserved through time, not through putting it to rest into storage, but by having it live orchestrated in motion.

>> Scope the challenge on this, because... And that's a great example of the iPhone because it simplifies the thinking. Scope the degree of difficulty because most people might not understand how hard it is to actually do this, because storage, you move something, you store it. It's like putting it in the file cabinet, then you got to go get it, and it's the latency involved. We're talking about-

>> Having it sit there stored is trivial compared to having it be constantly in motion and continually accessible.

>> And talk about the scope of the challenge because the impact of the demand now with generative AI, because the word generative is the word intentionally there.

>> You're getting a lot of data.

>> It's basically querying at runtime an answer that wasn't pre-programmed. So a lot's going on at the new... The expectations of the applications are different. Scope the challenge, and then why.

>> To make this happen, we had to solve some of the industry's longest standing, most intractable problems, and the first is actually the linchpin for the others. The first one is bringing supercomputer parallel file system architecture into the world of enterprise standard NAS. That was attempted 15 years ago with parallel NFS, failed miserably because performance was a niche. The existing NAS vendors already had their file systems, they weren't parallel file systems, it would've been throw it out and rewrite, and they're very wedded to them. Takes a decade to harden a file system. So they had a conflicted interest and not a good commercial reason to deliver on it. And then it was a standards body effort. So this failed and that is about the time I left Fusion-io, and I said, "It's not the wrong idea, it's just a little bit ahead of its time and it's conflicted in the motivation." So I went and recruited the Linux kernel maintainer of the network file system stack, the part of Linux that does this and said, "Let's go fix it. Let's introduce it into the standard, let's build it into Linux and then let's build the premier data platform to use this new protocol." So that was intractable problem number one, what the industry as a whole failed at. They failed so miserably that even to this day, people don't believe us when we said that we make it work. Anybody who knows what they're saying, say, "Oh, that'll never work."

>> Yeah, they have scar tissue.

>> They have scar tissue, yeah. We have to fight the ghosts of dead companies past because of this, right? But that is only the linchpin. What it unlocks is the ability to build a truly unprecedented level of scalability in a very robust feature-rich NAS system. That's what we call Hyperscale NAS. And Hyperscale NAS, while it's built on Parallel NFS actually represents a fully feature-rich, robust NAS system. But even that is really just the starting gate because the next thing you can do is due to this architecture, you can now have data live anywhere and have it be the same logical place because it separates metadata from data, it allows it to stay the same thing. It's like separating the what from where, and now even though the where changes what stays the same.

>> Why is it important? Because we see stories all the time. We actually write a lot of them on theCUBE and SiliconANGLE moving data costs money. Also, there's also-

>> Not moving data costs you more money. Data gravity, that's the biggest thing because it's what keeps you vendor-locked. It's what keeps you stuck. It compromises the productivity of your teams, it keeps your GPUs from having the data that they need. Having data be stuck in storage is the cardinal sin.

>> So that's where people have to take it from storage and move it to somewhere else-

>> To store it somewhere else. And it's a copy. It's a different piece of data.

>> So talk about why the problem is this. So that's one problem. And then data gravity is the other one. What's the implication?

>> Well, and that's really this next long-standing intractable problem is the whole concept of having a single namespace which stays the same even though the data moves physically across different systems, having data live distributed on a physical basis but logically be unified. And so these other innovations are really just to get there to where data can now live physically distributed. And then another one is being able to render that data in multiple facilities simultaneously and have it be the same data locally accessible from everywhere. The paradox I like to use is how can you have your data in every place that you need it without ever having copied it? Because in an orchestration platform, those aren't copies. They're instantiations of the one same logical thing. And that is incredibly important for productivity of people and systems.

>> So if I put my skeptical hat on, I would say, "Man, that sounds like it's going to cost a lot because I'm storing data. I'm not copying it." So what's the cost side of this? What's the economic value?

>> Well, it's threefold, right? Obviously there's the utilization angle. You'll get much better utilization when data can flow freely and live exactly where it's best suited at that point in its life cycle. Put the cold and cheap stuff on the stuff that's cold and cheap, the hot stuff on the stuff that's high performance. So just getting the data where it's supposed to be. But I'll tell you the cost savings from improved utilizations, that is the least of the motivators for using a data orchestration platform. The main motivator is increased productivity of people and systems, and nowadays GPUs, so hard to come by, so expensive. I think Jensen said that's his cardinal sin is a GPU sitting idle. Well, that comes back to my sin of having data stuck in storage because it's not delivered to the GPU where it's not idle.

>> And the applications are demanding it, hence the GenAI is only as good as the data that it can get and use at low latency.

>> And that's the thing that's so watershed about all of this is that with AI, machines can now make sense out of what was otherwise garbage data, unstructured data, the bulk of data, things with rich media, things that it used to take a human to even perceive what it was, now that machines can do that. Suddenly, it's a goldmine with all of this data being-

>> Talk about the market opportunity now because we're seeing storage platforms and data platforms that have kind of come together because become one, because it truly is a data platform because they have to be available all the time. I mean, we've talked about this-

>> Well, I would make the argument it's hard to be a data platform as a storage company, because-

>> Why?

>> Well, you're conflicted because inherently storage companies want to be the thing to hold the data. So they benefit from data being trapped in that silo, they benefit from data gravity. As an orchestration layer, we move data across those systems. We make it so that data is no longer having a tendency to stay stuck where it was.

>> So that might have been the wrong thing. It's funny you mentioned that. I think of a search example, Google. Google had a thesis back 20-plus years ago. Their goal was to get the user off the site as fast as possible because their thesis was help them find what they're looking for. And so if they can get their answer faster, that was the value of search, which everyone else was going the other way, which is stay on site and search and stay on the platform. Google ended up winning everything because everyone was going this way.

>> It's all about productivity, and you could say the same thing here. Our goal is to be so transparent that people don't even know it's there, and I like how the team at Meta put it, black voodoo magic, because it let them go from one data center to another with only three minutes of downtime. Even though it took many days for all the data to move, they were accessing it immediately.

>> So you got the award, that's the Innovation Award. We've been following your company, success is happening. You're getting some great traction. Congratulations. Who's buying, who's using it? Share some examples of where you're in play right now, where's the value proposition. What's the environments look like? You don't have to name names. You can say large social network that has multiple platforms, messaging apps and social network, or explain who your customers are and why they're buying.

>> Well, what we've done is we've built the biggest and the best and that makes it easy to sell to the rest, because if you can do those large-scale challenges where it's tens of thousands, hundreds of thousands of GPUs, then everything else becomes easy. So the most prominent customer now, or the customers now are those who are building LLMs and a lot of the major names you would hear have used Hammerspace in those workflows and continue to use Hammerspace in the world of AI and so forth. The large language model training is of course exemplar number one, and Meta has blogged about it, so I can talk about it, their use of Hammerspace inside of their infrastructure. And it's deployed by the guys that actually build their own warm storage and all of that. So this is really quite a compliment that a team is competent as Meta-

>> They're strong, they're alpha, totally alpha on this large scale.

>> And there are others we can't speak about there.

>> How about the enterprise? What's the enterprise-

>> In the enterprise space, that's exciting. You have folks in the financial services space, in the bio and life sciences space, obviously in the media and entertainment world. My son is a big fan of the Dog Man comic books. His new Dog Man movie out was just recently in movie theaters and that was done almost entirely on Hammerspace on one of these studios.

>> So in the enterprise, the hot trend right now, I want to, since you're here, get your reaction to it, is that Dave and I are tracking these AI clusters. Dell calls them the AI factories, NVIDIA has the same name, but they're basically servers put together and a lot of things you did at Fusion-io are kind of in play in all these architectures.

>> Well, the NVMe. My baby from Fusion-io is now a half trillion dollar a year industry, the NVMe, and every single GPU server comes with boatloads of NVMe that sits there almost useless because it's a local island of storage. With Hammerspace, all of that can become shared-

>> So I wanted to ask you that because you have that unique history and being a pioneer in that part and clusters. As these enterprise start putting these systems together, they're going to start running software on it which is coming from the market, but there's a big storage fabric and all that data fabric. What's your angle on this? Because when I say storage, I mean, it's basically connection points.

>> So here's the beauty: Because we fix that parallel NFS and make the standard stuff work and what's built into Linux, what that means is that you don't have to have any custom software, not just on the client, but not even on the storage server. It can all just be virgin Linux and Hammerspace sits as a control plane separate on the side, and that's what makes it so foundationally different is that we can aggregate storage systems en masse as primitive as just white box Linux, and even the NVMe that's inside of the Linux servers that are your GPU servers. We call that tier zero. So you can incorporate tier zero into the greater data environment.

>> So you're native. Essentially because of the Linux innovation you're in, you can just plug in Hammerspace and it kind of takes over.

>> You spin up Hammerspace on the side, you register all of those Linux storage servers serving standard NFS to Hammerspace, and we turn all that into one big shared orchestrated data environment where data can flow into S3, can flow between data centers.

>> So you're unifying all that into Hammerspace?

>> That's right. Yeah.

>> Is that discovered automatically, or you have to manually do that?

>> So today you have to take your different storage assets and tell the control plane about them and then it will manage the data distributed on it. In the cloud, there are some interesting things you can do with basically having Hammerspace scale up and down and spin up cloud instances to provide higher performance or more high performance storage capacity, or spin down and have the data get tucked away into object storage automatically.

>> Every company has a secret sauce DNA in their culture.

>> Okay, yeah.

>> Faster, smaller, cheaper. If you're Intel in the old days, now you got NVIDIA has a big vision. What's Hammerspace's secret sauce?

>> Well, the secret sauce and what led to it being very difficult to get funding for the company is that we're using these architectural innovations that the industry failed to do to be able to deliver higher performance, much more agile universal data access, and people used to think data management moving data around was the antithesis of getting higher performance. Data management systems slow things down. So our unique thing is that by using this architectural innovation, we're able to deliver on much more agile access to data from anywhere at incredibly high performance rates.

>> So you had probably a lot of skeptics during the early days?

>> It's really only been in the last year or so that we've had enough proof points with the likes of Meta, Los Alamos National Labs that we could go and say, "No, no. It actually works."

>> So Andy Jassy used to say to me all the time on theCUBE and off camera, you have to be misunderstood for a long time if you're doing something really big because people won't get it. Was there a point in time in your journey... Obviously winning the awards is an accomplishment here. Was there a moment in time, a distinct moment, a turning point where it turned? Was it a deal? Was it a breakthrough for Hammerspace? Take us through any point in time you can remember and say, "It was that time I was drinking the coffee and I had the idea," or customer bought this or investor got it.

>> Yep. Well, a successful startup has a sequence of those on different dimensions. Having a lighthouse customer, like a Meta, is clearly one of those watershed events. Having investors jump in and lend a hand, especially when they had been resistant because it was too bold and big of a vision to start with, or just landing the key team members. I mean, if you really want to go back to it, it goes back to my discussions with Trond Myklebust, who's Linus Torvalds's appointed gatekeeper of this part of the Linux kernel and convincing him that even though the rest of the industry had abandoned the core tech that was going to be needed, that we could go and do it. And so recruiting the team, bringing in the financial assets, getting the customers. But the one that sort of still remains is getting the word out there that you can have your cake and eat it too, that such a thing as this that was thought of as impossible can actually do what it purports to do.

>> As a successful entrepreneur, obviously you have great pedigree. We've been following your career, you've been with theCUBE for 15 years and our journey too. There's always moments where things spin in the right direction and timing's everything. And what's going on right now in the market? Where are we now? Obviously a lot of entrepreneurs out there, whether they're startups, the journey's hard and there's moments times where you say, "Okay, you're either too early or you're too late." It's hard to time the market in anything, everyone knows that, but I mean, just for in general.

>> Well, the key thing is that when I started the project... Because it takes a decade to build a commercially-hardened enterprise file system. That's just for a traditional file system in storage. Now go build a data orchestration platform, multi-site access, all of that. It takes a long time to build this. So we had to start a long time ago, and it required a level of confidence that at some point the industry is going to need the kind of high performance and agility of access across sites, and AI, it only arrived, what, two years ago, 18, 24 months in earnest, and it really is the forcing function to cause the industry to grapple with these demons that have been there, these undelivered promises of having universal access to data from anywhere at high performance levels.

>> And a lot of competing interests, baggage. I mean, inertia-

>> There's definitely innovator's dilemma conflicts of interest with a lot of people.

>> I mean, market shifts. When it flips, you got to be on the right side of the history here. Clearly the AI has opened up everyone's eye. They see now application benefits with GenAI. Where's your growth market now? As you guys continue to succeed, what's the target? What's your growth strategy?

>> Yeah. Well, sort of in keeping with this do everything mantra. Obviously AI represents the ability to have some very customers that can help the company prove itself and finance itself, but the real business is going to be sort of further down the pyramid, enterprise mid-range, because these are problems that everybody wrestles. Even the smallest of companies has issue with data being localized and how do they own and control their data.

>> David, great to have you on and congratulations on winning the award, Hammerspace. Again, it was a tough category, a lot of players out there.

>> We are extremely honored from the recognition and it's very appreciated.

>> And I thought it was going to be... It was our first awards, theCUBE Awards. You've been CUBE'd, which is fine, a nice acronym. I want to thank our team at for that one. It was me and Dave and Rob Hof and Kristen and team might judge. 43 judges.

>> I was going to say, it wasn't just you guys judging because I know you guys too well, and I wouldn't be as flattered, but if you're saying to me there was like 47 other people who were having a say in this.

>> Even if I wanted to rig it for you, it wasn't. We had AI people running AI labs, we had Anand who runs AI at New York Stock Exchange, Center of Excellence. We had CTOs, we had other analysts. The outreach-

>> It's a deep honor.

>> I think we actually might have more, and ultimately, I think in the future, it could be even on the blockchain, but we're going to open this up again in the second half. You always worry about inaugural awards because at least you have one and it might not happen again, but we're definitely doing it again.

>> Well, I hope to be a part of that down the road.

>> Yeah, we'll see. And here's some financing news for you coming, any kind of news coming, you want... You're smiling.

>> There's some very interesting things going on right now on that front, and super excited. We'll definitely have to get together and talk about it.

>> We should do a session, talk about the Fusion-io and how that relates to all this great success in these AI factories, because a lot of that systems work is back. I mean, it's really kind of an amazing time for systems thinkers.

>> Yeah, there's a lot of dramatic things going on with flash, for example, now actually becoming the capacity tier, not just the performance tier because of the power shortage thanks to GPUs. So we're seeing some major industry shifts back in my area of solid state and Flash.

>> Well, let's do a deep-dive sometime. Again, congratulations to you and your team. Of course, theCUBE Awards has really highlights the tech innovation in the industry and celebrating the winners in this inaugural program. The competition was fierce, and of course, we're doing it again second half of the year. I'm John Furrier, host of theCUBE. Thanks for watching.