theCUBE + NYSE Wired: Physical AI & Robotics Leaders | Yoni Michael, Typedef AI

Clips
More from theCUBE + NYSE Wired: Physical AI & Robotics Leaders

Yoni Michael

Co-Founder & CEO

Typedef AI

play_circle_outline Shift from structured to unstructured data handling and AI capabilities in enterprise settings.

play_circle_outline Revolutionizing AI Workloads: A New Architecture for Efficient Management of Unstructured Data and Diverse Analysis Needs

play_circle_outline Recognition of the agent wave and the need for infrastructure to support AI agents.

play_circle_outline Role of observability and monitoring in operationalizing AI models effectively.

Info
Transcript

Yoni Michael, Typedef AI

Yoni Michael

Co-Founder & CEO Typedef AI

Yoni Michael, co-founder and chief executive officer at typedef Inc., joins theCUBE’s John Furrier during theCUBE + NYSE Wired: Robotics & AI Infrastructure Leaders 2025 event to discuss Typedef’s emergence from stealth and its mission to modernize AI data infrastructure. The conversation examines how AI inference and the handling of unstructured data are transforming the enterprise landscape.

Michael outlines how Typedef bridges gaps between AI models and production deployment, making real-time inference more efficient and adaptable. With startups a... Read more

explore Keep Exploring

What is the impact of AI on data processing and analysis in organizations? add

What are the challenges teams face when working with unstructured data and how should platforms be designed to address those challenges? add

What are the current trends and considerations in developing infrastructure for AI agents? add

What challenges do companies face when trying to deploy enterprise-level workloads involving large amounts of data and advanced models? add

bolt Powered by CUBE AI

Yoni Michael, Typedef AI

search

John Furrier

>> Hello, everyone. Welcome to theCUBE here in Palo Alto. I'm John Furrier, host of theCUBE. We are here for three days of coverage of Robotics & AI Leaders. We have all the top leaders coming and sharing their stories. We've got startups, we've got investors, and we've got players making it happen. You got Yoni Michael's here, Co-Founder and CEO of Typedef AI, featured on siliconangle.com today with news on some funding, announcing some funding. Welcome to the program. Thanks for coming on.

Yoni Michael

>> Thank you so much for having me. Been a big fan through the years, so it's an honor to be here.

John Furrier

>> Yeah, great to have you on. One of the things we love about the innovation cycle we're in now, it's not just leaders who have been around the block or Series C or public companies. It's really the startups too. And one of the things that's key in the AI is the data wave, which we've been saying on theCUBE, as you know, it's going to be unleashed once the AI infrastructure gets their act together. And they are, they're moving fast, and we're starting to see the supercomputing-like capability. We're starting to see the re-architecture of the formation of how data centers are being built. I just interviewed Amazon's top guy who does all their $20 billion deployment. It's clear it's here. CapEx is being spent. Now the operating model's starting to kick in and that's where the data plays. You guys just announced coming out of stealth. Talk about the company. What do you guys do? Explain, give us all the new hard news.

Yoni Michael

>> Yeah, yeah. So just came out of stealth today, so the timing was amazing for having us on, and I agree with you. I couldn't agree more. It's all about the data, and when you're trying to really power a lot of your analytics and products through AI, data quality in has to be top-notch. And so I've been in data infrastructure for almost a decade now and have worked with a lot of the traditional data platforms there. And so when we're going through and discussing with my co-founder Kostas what to build, we knew that it's time for some new infrastructure to help power the new requirements of data for AI specifically. And so what we've already come to know and be super familiar with are the Snowflakes, the Databricks of the world, these large-data platforms that are really great at powering structured Tabular data. But with AI coming in, that's been the biggest shift since, let's say, the brilliant guys at Spark and Berkeley Lab came out and built an amazingly massive distributed compute platform that is Spark, and the biggest shift since then has been AI. And so that was the premise when thinking about Typedef is how do we really enable teams and companies to leverage and harness the new capabilities that AI has and offers while considering the new requirements for these AI workloads? So it's no longer about, let's say, columns and rows and structured tables. Now, more than ever, teams are trying to leverage LLMs and language models and vision models and all the new types of models for inference there to work on unstructured data. And companies are sitting on vast and vast amounts of transcripts, of documents, of customer support tickets, of a lot of unstructured data, video, audio, all the different mediums that are there, but they really struggle to build insight and analysis on top of it. And frankly, our take is the kind of traditional platforms that are there weren't built from the ground up. So naturally, the architecture doesn't suit it, and what teams are trying to do is bolt on these capabilities, the AI-based capabilities there. But our take is we need to start from first principles and build a platform from the ground up that treats AI inference as a first-class citizen.

John Furrier

>> Yeah, and it's interesting too because you mentioned some of the bolt-ons and some of the refactoring of existing legacy. In the enterprise, that's a natural reaction, right? So, okay, I got these systems and most of the time they're brittle. And you're seeing also at the storage layer and the memory layer, things are getting closer to the processing power. So the compute tier becomes a huge part of how fast tokens can come out, and with reasoning, the token demand's going to require a large scale. So that's going to increase massive software opportunities or data opportunities where it goes beyond retrieval, augmentation, generation. So the enterprises right now we've seen, I'd love to get your thoughts on this because they're leaning in hard, but the adoption is slow because they're in POC.

Yoni Michael

>> Correct.

John Furrier

>> They're stuck in the mud a little bit with POC. One, there's too many POCs. And two, their evaluation capabilities were old-school, modern legacy approaches. Do they meet the resilience bar for security?

Yoni Michael

>> Yes.

John Furrier

>> It's like going to the airport and no one's invented TSA Pre and CLEAR yet. So the POCs are kind of stuck, so we're seeing a massive demand for production workloads. And so I want to get your thoughts on this because what's happening now is enterprises, they play with search, RAG is great. They get their documents, they take their customer databases, they create a data mesh. Whatever they do, they harmonize it, and now they go, "Wow, let's do agents." This is where the action is, and this is where the complexity's not as trivial as just vectorizing things. What's your reaction to that?

Yoni Michael

>> Yeah, that's 100% true. I think it goes back to the new nature of these workloads that we're trying to harness. And to go back to your point on the infrastructure and compute side, when Spark was built, you needed distributed compute. Everything was compute and memory-limiting. Now when you're working with AI, it's about new architecture, it's about GPUs, and thinking about how to manage inference at scale, how to batch inference, how to do distributed GPUs. So a lot of what you're limited by now is more of I/O. You're more I/O-bound with these workloads because you're waiting a lot of times for these models that take a lot longer to process, to return responses to you, and so that's-

John Furrier

>> And by the way, that burns energy too, power.

Yoni Michael

>> 100%. Yeah, 100%. And so that's where you see all the amazing capabilities and all the white papers that are being released from companies like Google and Amazon around how to actually run GPU clusters in an efficient manner, which is going to be huge as we go forward because the needs are there, and these companies are building out these massive clusters to be able to support the demand that they're seeing. But that's where our take is too is we think that we need to start thinking about inference as a first-class citizen, and if you start building platforms that are aware of the limitations and complexities and nuances that these models have from the ground up, so it's no longer about structured Tabular data, it's now more about context windows and partitioning and chunking and dealing with lots of unstructured texts that these platforms weren't able to do, while also managing some of the limitations of the models. Inference is a new transform, and so how do we really harness that power and allow teams to manage it in an efficient way while giving the same guarantees and rigidity that they're used to with structured pipelines and Tabular pipelines that everyone's come to love?

John Furrier

>> Yoni, I want to get your thoughts on this because if you look at the Snowflake and Databricks, which you mentioned, we cover them, been covering them since the beginning, they're two kind of microcosms of data that you can abstract that the enterprises are doing the same thing. Snowflake has the classic enterprise analytics business. They've been great in the cloud. We've been covering them. Databricks, more of a developer. You mentioned the Berkeley vibe. That's more a developer, more engineering-oriented. Snowflake is too, I don't want to throw cold water on something. They have engineering too. But Databricks made two moves. They bought Tabular and they bought Mosaic AI. Both bring in two things, like open tables formats, and then with Mosaic, you have programmable agents, more model-specific, and Jonathan Frankle, their chief scientist, talks about this all the time, is that that's a software layer approach that's data-specific, so it's almost data programming. And so agents will be there. So you have this Snow-Bricks dynamic, I call it, or other people call it too, we adopted that word.

Yoni Michael

>> They're converging, yeah.

John Furrier

>> Yeah. Clients have both in some cases and they run on Amazon-

Yoni Michael

>> 100%....

John Furrier

>> but now you've got the on-prem feature. So the enterprise is saying, "Well, I can run my compute in my data center where my IP is, my data, and run the cloud." You've got a hybrid environment. So with that in mind, the enterprises are going through this architectural reset. What are you recommending to them? What do you see in the enterprise? Because they have to do both. They have to handle the analytics workloads that have been around for a while, all those dashboards, while bringing in the programmable platform engineering mindset that we've seen come out of, say, KubeCon and the platform engineering world, which is basically large-scale data engineering. What's your thoughts on that?

Yoni Michael

>> Yeah, and you touched on it really well. Tabular, for example, the acquisition, the owners of Iceberg and that open table format, that's been the huge evolution in data infrastructure and data platforms in the last few years is interoperability, and that's what Tabular and iceberg table format really harnesses. You can now build your own data lake and then wire up whatever query engine or data platform is most efficient for the specific workload that you're trying to execute. So in some cases, in a lot of cases when we're talking to customers now, they're both Snowflake and Databricks customers and are using the data lake and Tab and Iceberg table format as a way to work with both platforms at the same time. And so that's the key here is that my take is there are, because of this interoperability that's being offered, there is room and opportunity for specialized query engines to come in that are a lot more efficient at, let's say, Tabular structured data, let's say what the business analysts want to do when they're using something like Snowflake, but then also the massive distributed compute needs that Spark is able to provide and Databricks is able to provide.

John Furrier

>> Well, you guys have a huge market opportunity, about $200 billion plus. You got the team. You guys were also in stealth mode for about a year, which means, I'm inferring this so you can verify if it's true, you're talking to customers, you're probably getting the product market fit nailed down. Talk about the customer angle on this because I think that's where, I think in the enterprise, it's not easy. It's complicated because, one, it's enterprise by default. You've got distributed computing, you've got cloud and on-prem, exploding on the on-prem side. What's their problem that you're solving? Is it the POC problem?

Yoni Michael

>> Yeah.

John Furrier

>> Is it getting stuff into production? Because modern gen AI is not really being adopted aggressively inside the enterprise, outside of some really risk-free use cases, like RAG and other search stuff.

Yoni Michael

>> Yeah. Yeah, I think you touched on it exactly is that 80% of AI projects that you see never see the daylight in production, right? It's very easy to build a great notebook that's leveraging these LLM models and build great prototypes of it. Where we're actually seeing the bottleneck and the problem for these teams and companies is actually taking it, operationalizing it into production, and running AI workloads at scale. And that's the exact problem that we're trying to solve.

John Furrier

>> And what did you guys do in stealth? What were some of the actions you guys took on your go-to-market plan?

Yoni Michael

>> Yeah, I mean, a lot of it was talking to as many teams, AI and data teams, as possible and really understanding and narrowing down where that bottleneck is coming. Why are all these projects not making it into production? Because it's very powerful if they can. There's tremendous business value to be had, like we said, with all the unstructured data that's being there that LLMs are really great at helping to do semantic analysis over. It's uncovering a lot more business outcomes that are there and the capabilities of building products on top of this unstructured data. So a lot of it was doing that kind of customer discovery and really narrowing in on what exactly can we provide that is going to be a bridge for them? And that's what we came up with, this AI-first data engine.

John Furrier

>> So I have to ask, while you were in stealth working with customers, I love that entrepreneurial formula, it's proven, but the agent wave hit right when you guys were in the middle of this, what did that yield in terms of new data from the field? What did you learn?

Yoni Michael

>> Yeah, I think it's definitely, everyone is very eager to try and arm their workforce with AI agents, and it's the year of that, and everyone's trying to use these frameworks so there's a lot of things like LangChain and Llamaindex that are really helping build a lot of this and harness some of it. And so we do need to build infrastructure that is built for AI agents and is able to support agentic workflows. And that's part of what we're thinking also with Typedef, and by design what we're doing is how do you provide these AI agents a way to also get access to the unstructured data? And we think if you're decoupling, let's say, batch inference from realtime AI inference, which is what our platform is built for, then you can have AI agents easily access all the structured data and do data analysis that way too. So it's a lot about building all of these tooling and infrastructure through common APIs that agents are able to access to really empower them to make a business impact.

John Furrier

>> We love the data business because, one, it's been covered 15 years since the old Hadoop days, then we saw Spark come in, now it's the data lakes and that, we're in the agent wave. The diversity of data types are key. You got unstructured, structured data, you've got now autonomous, semi-autonomous capabilities coming fast. So you've got diversity of data types, and then you've got diversity of models, and then to make it more complex or more interesting, you've got token demand. So now you have the price of tokens dropping in terms of the benchmarks and just overall value with the GPUs and some of the infrastructure stuff that we cover here this week and every day. So you got diverse models that need to talk to each other, and you got the diverse types, so the diverse types, unstructured, that's solving itself out. We'd love to see that work there. Now the focus on startups is a lot of these companies have ARRs of like $10 million. You're starting to see these numbers, but there's no R in there. There's no recurring because it's not in production. There's a one-year contract basically. So we're seeing the startups really eager to bust through the line, so to speak, and get into production. What are you seeing there? Because I think one of the things that's interesting is that the enterprise wants to go faster. It's not like they're putting the brakes on. They just don't have the mechanisms and the methodology to evaluate. So now we're seeing startups trying to figure out, "How do I rally around?" Or any company that's building in the AI stack, what in the AI stack can they rally around to what Kubernetes did for platform engineering? It was a nice orchestration layer. That helped a lot of that cloud-native, engineering terms. But now in the data world, modern gen AI, what is the rallying point? Is it the data layer? What's your thoughts on this? Because this seems to be the core problem.

Yoni Michael

>> Yeah, I mean, I think you're right. Everything's changing on a daily basis. Gemini, all the big foundation companies are releasing these new models with new capabilities. Latencies are changing, prices are changing. So it's really around being very inference-flexible, right? As we're talking to more teams and enterprises, you're not seeing it very settled in the sense that every company has a different inference strategy. Some companies you'll talk to, they'll go and they'll have these large enterprise contracts with the OpenAIs and Anthropics of the world and are fine leveraging those big foundation models. Other companies are using some of the middle-ground players, like the Fireworks, the Togethers, and trying to help host their own things. Other companies are hosting their own open-source models. And so our goal as a data platform is to be as flexible as possible for whatever inference strategy you have, while also providing the capabilities for teams to really be able to think about the complexities of each individual model. You need to have a lot of that domain knowledge built in, so like rate limits, token costs, latency, all of these things are very important. And so the capabilities of building model routing, model switching based on these properties of the models and allowing people to experiment very quickly and see results very quickly. The end-to-end time from essentially building a notebook into putting into production, that's the time to value that we're really focused on building.

John Furrier

>> And the enterprise, they want the user experience, they want to see the business value.

Yoni Michael

>> Developer experience too.

John Furrier

>> Unlocking is a term we've been hearing a ton. I mean, it's the most overused word. If you did a tag cloud on theCUBE, unlock is the key word. For the folks watching out there in the enterprise, what is your story pitch or value proposition? When do they call you in? Is there certain criteria? Is there certain smoke signals that happen? Are things burning down? Are things accelerating? What are some of the signs that they need you and what is your pitch?

Yoni Michael

>> Yeah, I think a lot of it is when you're thinking about trying to process a lot of these new modern workloads, that could be mixed. It's not only unstructured, it's also a mix of semi-structured and structured data, and you want the central platform to be able to do that and manage these workloads very efficiently, but also operationalizing it, right? And what that means is how do I actually think about traceability, observability, monitoring, all these inference pipelines and workflows that we have? And those are what are required to be able to put these into production. So any company that has lots of this type of data where they're really trying to harness the power of these LLMs ,vision models, ACR, OCR models, that's the idea there is when you would work with Typedef to be able to help that.

John Furrier

>> Well, we need you to get out to the enterprise and get these production workloads deployed. That's a big demand. Thanks for coming on theCUBE. Appreciate it, and congratulations on the coming out of stealth, breaking news here on theCUBE. Thanks for coming on.

Yoni Michael

>> Thanks so much for having me.

John Furrier

>> All right.

Yoni Michael

>> It's been a pleasure.

John Furrier

>> I'm John Furrier here with theCube. Breaking news, coming out of stealth. Again, the startups are emerging. There are more startups, but it's hard to crack the code on the enterprise. The enterprises need help getting stuff into production. It's a data game, it's an infrastructure game. Robotics as an edge device, we'll see more of that too. It's IoT. Again, great lineup here. More guests coming. Stay with us after this short break.