theCUBE + NYSE Wired: AI Agent Conference | Robert Nishihara, Anyscale

Clips
More from theCUBE + NYSE Wired: AI Agent Conference

Robert Nishihara

Co-Founder

Anyscale

play_circle_outline Ray project origins at UC Berkeley

play_circle_outline The Rising Dominance of AI in Distributed Computing in Today's Market

play_circle_outline Maximizing Efficiency with Kubernetes and Ray: The Future of Simplified, Flexible AI Technology

play_circle_outline Utilizing Ray for a variety of AI models

play_circle_outline Transition from AI experimentation to production

Info
Transcript

Robert Nishihara, Anyscale

Robert Nishihara

Co-Founder Anyscale

Understanding AI Infrastructure: AI Agent Conference 2025 Insights

In this exclusive interview recorded at theCUBE studios, Robert Nishihara of Anyscale joins John Furrier of SiliconANGLE Media Inc. to explore the rapidly evolving landscape of artificial intelligence infrastructure and the pivotal role of scalable computing. This conversation is part of theCUBE's coverage of the AI Agent Conference 2025.

Nishihara, known for pioneering work in AI infrastructure, discusses the challenges and opportunities presented by the need for scalable AI... Read more

explore Keep Exploring

What motivated the start of Ray? add

What was the inflection point that made distributed computing for AI (and tools like Ray/Anyscale) become a mainstream priority for businesses? add

How do Kubernetes, Ray, PyTorch, and VLLM differ in their target users and technical focus, and how do they complement each other in machine-learning workflows? add

Why do companies standardize their compute on Ray, and how does Ray enable platform teams to consolidate different model workloads (e.g., generative models, content moderation, recommendation, ranking)? add

What is the company's current status and strategic focus — how is the business doing and what is it concentrating on now given the surge of enterprise interest in AI? add

bolt Powered by CUBE AI

Robert Nishihara, Anyscale

search

John Furrier

>> Welcome back, everyone, to theCUBE studios. I'm John Furrier, your host here in Palo Alto for the AI Agent Conference preview. And also, the conversations around what's going on under the covers from the agents. What's the infrastructure enabling it? What are the best practice? How are people standing up the infrastructure software methodologies to bring in this next wave of value that's going to transform business? Robert Nishihara is here. He is the co-founder of Anyscale, a company we've been following since its founding. Robert, great to have you back on theCUBE here, talking about what we love to talk about, which is how do you stand up large-scale compute, large-scale networks, enabling the disruption. So, thanks for coming in.

Robert Nishihara

>> Yeah, thank you. Thank you so much.

John Furrier

>> So, Ray Summit we covered last time. Anyscale, watching you guys come out of Cal. Machine learning was hot, things were happening, but you had a vision with what AI could look like. Take us through the story because where you are now is you're on the doorstep of this massive surge of new infrastructure. You got agents coming in, there's demand for agents, so that's going to explode as the compute and as all the infrastructure gets leveled up. Take us through the story. How'd you get here?

Robert Nishihara

>> Yeah, absolutely. So, AI was hot at the time, but nothing like what you're seeing today. We got our start... I was doing AI research at UC Berkeley, and we found ourselves constantly running into bottlenecks with the tooling. AI's very computationally-intensive. And so, in order to do our research, to implement algorithms, to experiment, we constantly had to build our own tooling for scaling compute, for managing clusters, for moving data around and shuffling data, for getting stuff to run on spot instances. There's a tremendous amount of distributed systems infrastructure we needed to build out for ourselves and that led us to start Ray. So, Ray's an open-source project we started before we started Anyscale. We started Anyscale later on to commercialize Ray. And the whole bet with Ray and Anyscale was just that there's a growing need for scaling compute for AI, right? Often, if you are building models, if you're deploying those models, you're often not doing that on your laptop. You're doing it across a bunch of GPUs, across clusters of machines, and they're just a lot of hard systems challenges with making that happen. Now, you mentioned AI being hot. So, when we started Anyscale in 2019, the whole bet was on distributed computing for AI. That market didn't really exist, outside of early adopters. Of course, they were early adopters who were betting hard on scale, but the real inflection point for Ray was ChatGPT, when AI became a priority for every business and the need for scale just became far more obvious.

John Furrier

>> Yeah, and the tooling. I love the tooling problem because it reminds me of the expression, "Back in my day, we used to walk in the snow barefoot to school," and that was hard work. I mean, take us through that mindset at that time, because you know wanted to achieve the objective, but you had to build the tooling. I mean it was probably fun in the sense you had to build tools, but it wasn't on the top of your mind. It was like there was no other product. So, it was like, I won't say a grind, but yeah, a grind.

Robert Nishihara

>> At the start, we were doing it out of necessity because it was something we had to do in order to get our research done, but over time, it became the main thing, it became our primary focus. I didn't have a background in distributed systems, so there's a lot of learning, a lot of new stuff. But UC Berkeley has a long history of creating great open-source software, Apache Spark and other systems like that.

John Furrier

>> The DNA and the history of Cal Berkeley is just always revolutionary in that sense of game-changers. Was there a moment where you guys said, "I wish someone would just build some tooling for this"? Is that where Ray came from?

Robert Nishihara

>> I mean, if this had already existed, we absolutely would've just used the thing out there. We weren't looking to build stuff for the sake of building stuff.

John Furrier

>> It's interesting, we've been following you guys and we've also been following the hyperscalers, look at what Amazon Web Services was doing, and now Google Cloud, we just came back from Next. The market has now shifted. The category is there. You're seeing what Nvidia's doing with their GPUs and how their AI factories, they're calling them, are huge. I mean they're systems, so they're distributed systems, but they're supercompute is in a system. So, the system management changes. So, what now is the core problem that people have as you guys look at supercomputing being democratized, what is the core problem that you're facing now and what's going on?

Robert Nishihara

>> There are a lot of key challenges. Scale is a big one. Running AI workloads. And when I say AI workloads, I'm talking about training, inference, data pipelines, running those reliably at scale, making that cost-efficient, making it performant, giving flexibility to developers, because AI is changing very rapidly. You've got different deep learning frameworks, inference engines, models, new techniques becoming available all the time. And from the perspective of the platform teams that are enabling these researchers and developers, they don't want to make choices that lock their teams into specific technologies. You want to have the flexibility to adopt whatever the latest thing is that comes out or to try it out. And so, building systems that are performance and scalable, while at the same time maintaining flexibility and velocity for your teams is just huge.

John Furrier

>> I know you got a new CEO, so you've got a great leadership team there now and you've got some free rein to talk to customers and work on the product, which I know you love. I have to ask you, and I want to get your reaction to some of the market dynamics. Obviously, the hyperscale market's huge. I want to talk about Google in a second, but the AI on the enterprise side's massive, but that's downstream from the current demand. But I was talking to a financial services executive leader, technical guy. He runs all the platforms. He says, "Yeah, these AI factories are awesome. We're betting the business on. We have all this data and we're going to manage it. We're going to apply all the machine learning we've done from fraud detection and then start using generative AI." He goes, "The problem that I have is all this gear gets shipped to the loading dock," and I use that metaphorically because it's not really a loading dock. It gets shipped and it sits there. I asked him, "What operating system you running on? Is it Linux?" I mean, Linux was the old-school on a server. And I asked that intentionally because I want to know how do you run these things? Because they're hardware and software. So, what is the state-of-the-art operating system there? Nvidia says it's going to be MV cache, they're talking about other ways to do it. You're now talking on distributed computers. What is the operating system?

Robert Nishihara

>> That's a great question, and everybody, including ourselves, calls themselves an operating system.

John Furrier

>> Exactly.

Robert Nishihara

>> And in terms of literal operating systems, Linux is super common, but there's an important tech stack. To enable all the progress in AI, there is this massive infrastructure build out that's happening, that's both hardware infrastructure, which you see with Nvidia and other chip companies, TPUs and so forth. And there's software infrastructure to really connect the applications to the hardware. And that software infrastructure has been in flux, right? There's a lot of progress-

John Furrier

>> Moving fast. It's accelerating....

Robert Nishihara

>> and evolution happening there. But one common tech stack that we're seeing over and over is a combination of Kubernetes plus Ray, plus PyTorch, plus VLLM. And of course, different permutations of this there. You can swap in different pieces. There are a lot of great options here, but that's a very popular combination that-

John Furrier

>> Take me through the tech stack. Let's lay that out. So, what's each role of the stack? Kubernetes orchestrates... Take us through the stack.

Robert Nishihara

>> Let me start with Kubernetes versus... If I think about PyTorch, Ray, VLLM, those are targeting machine learning people, people who write Python code, who think about models and Hugging Face and batch sizes and learning rates and that type of stuff. Kubernetes is targeting platform engineers. This fundamentally is often a different audience. Sometimes they coexist in the same person, but it's often a different team, a different audience. The platform engineers are people who live and breathe YAML, like who configure clusters, who standardize policies across clusters and manage fleets and enforce permissions and things like that. So, individually, Kubernetes and Ray don't serve both audiences, but together they can really serve both audiences and meet both sets of requirements. So, that's Kubernetes. When it comes to Ray, PyTorch and VLLM, and I'm oversimplifying here, but one of the things that PyTorch and VLLM do extremely well is they really optimize the model on the chip, like on the GPU. They run the model in a really performant way on the hardware accelerators. And Ray doesn't do that. Ray is focused on the multi-machine scaling, on the distributed computing aspects. So, scheduling, handling machine failures, moving data, all of those kinds of things. So, they're very complementary, they're targeting the same user and it's very common to do batch inference by running Ray plus VLLM or train your models with Ray plus PyTorch. So, all together, this is a really powerful combination.

John Furrier

>> And I asked the operations system question mainly it's intentional because it's not the classic OS. It's not like, "Here's the NAS on my mini computer. Here's my Linux on my server." Because of the distributed nature, you guys are using words like scheduler. I mean these are words... The role of compiler has come up on theCUBE the past six months, at least a dozen times the word compiler. You go back six years, maybe twice. Compiler, schedulers, linking, loading, moving things around, IO, networking.

Robert Nishihara

>> I actually think it's a very good analogy. And if you think of the role of the operating system as sitting in between the hardware and the applications and connecting them together, on your laptop, that's the role the operating system plays. Now, in the distributed setting with AI, the hardware is much more complex because it's distributed, because there's all these accelerators you have to use, because the accelerators will fail and you have to handle failures and the applications are also way more complex, because again, they're distributed and need to run at a really large scale. So, you have increased complexity on both the hardware and the application side. So, I think the need for this software layer that sits in between is even greater, and that's a role that Kubernetes plus Ray can fill.

John Furrier

>> I think this is super important. I've been waiting on theCUBE to wait to have this conversation and substance. Because distributed computing has been around for a while, there's known paradigms, there's known science behind it, but the components are different. They look different. So, to make everything work, now you're going across things. So, look at cloud for instance. Let's take the simple SaaS market. Some are saying agents are going to be the next SaaS by order of magnitude bigger, but SaaS was easy. I would write an app, put it in the cloud and put it in the app store, and that's Dropbox, that's Airbnb. But to run AI at scale, you're basically enterprise by default. You can still use the cloud, but it's not as easy. But agents now run across multiple things. So, if I'm the PyTorch developer, the VLLM guy, I'm like, "Okay, I need some information around what the infrastructure looks like because I don't want cost overload." And you think FinOps. I mean look at the FinOps market, it's cloud-centric because it's known, but I don't know what's going to go on behind here, which GPU is being idle... So, again, it's complicated end-to-end. I think that's where I think you're onto something here... Because I mean, what's your thoughts and reaction to that? Because if you don't know what's going on from request to hardware, you're screwed basically.

Robert Nishihara

>> So, we look at this or I look at this through an infrastructure perspective. And when it comes to agents, no doubt agents will be huge. What happens with agents is that there's growing infrastructure complexity because you all of a sudden have much more complexity at inference time. You have models that are reasoning for long periods of time, there are many models being called that are invoking each other, that are communicating with each other. The models need to be able to connect to all of your data to be able to retrieve relevant data, ingest all of the data, reason about it. So, there are many different components to this. Some of the components to building agent applications that I think of, one is, how do I ingest all the data, right? As a business, I have tons of data in many different forms. That could be Google Drive, that could be Slack, that could be logs from my application. I need to ingest all of that data, build search indices, compute embeddings. There's a big data processing workload there. Separately from that, of course there's the search and retrieval part. My agents need to be able to query that, search it and retrieve information to feed into the context. And then, I also have an advanced reasoning model that is orchestrating the entire process of doing retrieval, querying the data, reasoning about it, taking actions and so forth. And of course, that may be one model, that may be many models that specialize, but those are some of the different components.

John Furrier

>> It's a complex system, so it's basically what it is. So, okay, so let's talk about data processing, because when you look at all the slides over the past year, we were at Nvidia GTC, Google Next, AWS re:Invent, Supercomputing last year. All the same things are out there. The world of storage is changing and the world of networking is changing. So, if you look at all the storage folks, because their relationship in the system... Because they have the data. Storage is storage, they store the data. They're now called data platforms. So, storage is changing and the role of networking is more prominent, that speaks to some of the things you're saying. So, the question is what is the data processing equation? How has it changed? And why is storage and networking changing from your perspective? Is it because the system's different and their role is different?

Robert Nishihara

>> So, I think one of the interesting insights here, when people talk about AI workloads, you tend to break it down into training and inference, those are the two things. You build the model and you use the model. There's a third important AI workload here, which is data processing and data pipelines. And so, every organization has tons of data. You collect this data because you want to use it, right? You want to get insights from it, you want to make decisions, draw conclusions. And the dominant way of doing that today is to run SQL queries on your data. If you have your data nicely structured in a database, in some kind of tabular format, you can run SQL queries and answer questions. Now, the vast majority of data isn't nicely structured in your database. And you have things like recordings of videos, like recordings of this conversation. You have audio, you have PDFs, contracts, you have recordings of meetings, you have images, tons of things like this, Slack conversations. And all of that is just inaccessible to SQL queries today. But there's a ton of value there. And what we're seeing is that advancing AI capabilities, like better and better models and multimodal models are enabling people to start to get value out of this unstructured data, to read a contract and deliver insights. And so, the process of going from data to value is increasingly an AI workload. It's shifting from being a SQL workload primarily to being an AI workload. And from an infrastructure perspective, that's a massive shift. You have systems built for running SQL on CPUs, that is changing to a future where you have systems for running AI models and inference on GPUs. And so, this is one of the biggest trends we see in infrastructure.

John Furrier

>> And this brings up the fact that the market didn't exist for you, scaling compute for AI, and that's now a category. I mean, you guys essentially created the category because you do this.

Robert Nishihara

>> We were betting on that from the start.

John Furrier

>> All right. So, talk about your relationship with Google because at Google Next we saw some activity. They've been all over Kubernetes. So, Kubernetes plus Ray and the stack is formed, that looks like visibility to you as a viable scalable approach?

Robert Nishihara

>> Yeah, so we announced the partnership with Google Cloud at Google Cloud Next, and this is related to the belief that Kubernetes plus Ray can be this operating system for AI and the importance of that that will play in the overall tech stack for AI compute. And we're going very deep with them to really make that an amazing experience, to really make it just the best way to run AI workloads.

John Furrier

>> So, if you're going to be an operating system, it's got to be tight, it's got to be simplified. One thing I liked about Google Next this year was when I talked to all the technical folks on the chip side, GPU side, they're simplifying things. How do you see this playing out in the market? Because okay, what does it mean for the customer? Give me some examples of some of the use cases that you see. What are you looking at for value and is it limited or is it unlimited scope for you in terms of opportunity?

Robert Nishihara

>> Yeah, great question. So, totally agree on the need for simplicity, and this has been part of our design philosophy with Ray from the start is have a small number of very flexible and general primitives that you can use to do anything on top. And as an operating system, you need to support all different use cases. It's not just one use case. And so, if you're going to support many different use cases, you could either build a ton of specialized functionality or you could have a small set of very general functionality. We've opted for a small set of very general functionality. And you asked about people using this, Uber trains all their models with Ray, right? Companies like Apple use Ray, Pinterest builds all their models with Ray, DeepSeek uses Ray-

John Furrier

>> They don't want to build their own tooling. Why build your own tooling? And now you have a platform.

Robert Nishihara

>> And that is exactly the perspective. A lot of these companies, they're sophisticated, they could build everything in-house, but it's going to slow them down. And so, you don't build everything. If there are great open-source tools, industry standards, you're going to take them off the shelf and use them.

John Furrier

>> You know, Robert, what I love about this market is conventional wisdom gets tossed out the window. So, let's just do some conventional wisdom on OS operating systems. You got to have standards across the entire organization. I got to have an application suite for it. So, in AI, is that conventional wisdom hold or is it irrelevant because it's an end-to-end, it's use case specific, it's environmental specific? I mean is there some standard... I mean obviously distributed computing is distributed computing, but that's an infrastructure layer. Are you sitting where you can say, "Hey, I can have Ray here. I don't need to have the entire company run Ray," or do they? What's your view on this?

Robert Nishihara

>> Both models work. A lot of companies that adopt Ray have a vision of standardizing a lot of their compute on Ray, and they might start using Ray for generative models. They want to do image generation, video generation, LLMs. And then, these companies... Canva's a good example. They started with building generative image models with Ray, but they have 100 models they build internally. Content-moderation models, recommendation models, ranking models, all across the board. And so, from the perspective of the platform team that's supporting all of this, you don't necessarily want a different tech stack for every different model. You want some consolidation. And the fact that Ray is so general enables that consolidation, enables you to have a common paradigm that you can use for a huge variety of workloads, but you start small, you start with one use case and you grow from there.

John Furrier

>> There's no requirement to have global distribution to get Ray going?

Robert Nishihara

>> For sure. No, no, no. You can just download Ray on your laptop today, install it, give it a try.

John Furrier

>> I mean, love it. So, first of all, thanks for coming in and sharing the first ever operating system conversation on what runs on the system, clusters, and congratulations. I want to ask about the open-source angle because I think this is huge. Another new dynamic that's... I mean it's not new to the software industry, but in the scale side of AI, the work that's being done in open source is a driver. Now, it's influx, as you mentioned, because it's just evolving and a lot of the work is getting done at the low chip level. Talk about the role of open source and is it in a waiting period zone? It's in the waiting room? Because you're seeing a lot more hardware changes. The software, hardware integration is super critical. We see that everywhere. Memory management's coming back and it feels like the '90s when you had to deal with memory. Now, you got HBM. SSD is a capacity tier for storage. Are we in a waiting room for this agentic layer as visibility around the tech stack for operating the hardware? Hardware? I mean systems. I mean it's hardware, middleware, app. I mean that's the old stack generically, but are we in a waiting period?

Robert Nishihara

>> So, this will take a little while to fully play out, but open-source has played a huge role in the development of AI and all the progress we've seen so far in AI. And not just in AI, but really across technology and I think that's going to continue to be the case. And I mentioned this tech stack for AI compute with Kubernetes, Ray, VLLM, PyTorch, all of these are open-source projects. And if one of these projects, Kubernetes, PyTorch, Ray, if it were not open source, people would not be using, it would not emerge as an industry standard. I think when it comes to infrastructure products, open-source has a massive adoption advantage. And also, I think open-source really accelerates progress in AI, just because you can have so many people contributing to it, looking at it, making improvements.

John Furrier

>> When I graduated from college, proprietary software was the norm. I know friends that started companies that didn't make it and they couldn't get at the software. We used to deal software, "Hey, here's the free source code," and then open-source happened. Okay, great. The question I want to ask you is, okay, now that we have open-source booming, what is going to be the impact of this next level? Because back in the '90s, the OSI model really changed the game, open systems interconnect. It was a stack that was seven-layer stack, and really it was TCP/IP and below that got fully standardized and became defacto. It evolved. It wasn't like someone declared that this is a standard spec. Obviously, physical layer was easy because that was ethernet and internet. What is the defacto version today that's emerging? Because I'm starting to see the same thing around Kubernetes, CNCF, some of these communities where the open-source is like normalized. Some people are saluting the flag of, "Let's just go with this and we'll all win." And that's what happened with TCP/IP because once it started happening, it became the de facto standard. No one declared it. It evolved out of the open.

Robert Nishihara

>> Yeah, and these things can play out in different ways.

John Furrier

>> What's de facto?

Robert Nishihara

>> With the infrastructure stack, it's all open-source. Now, which technologies people use? There's still new ones being developed.

John Furrier

>> Which ones do you think are facto right now, that you would say, "That's pretty much a good bet"?

Robert Nishihara

>> I mean, the ones I mentioned. Kubernetes is very widely used, right? PyTorch, very widely used. I would say Ray and VLLM are on a good track.

John Furrier

>> I mean, you've got your customer bases, that's the early-adopters. That's clearly out of the barn right now.

Robert Nishihara

>> Yeah. So, those are doing very well and I think there's a ton of momentum around these types of projects.

John Furrier

>> Well, I think you guys are de facto standard. I would agree with you. And I know you're kind of humble, but I'll say it, Ray has emerged. And open-source also is the arbiter of the value. I mean, no one just gets a free pass. I mean, talk about that dynamic for the people that don't know how open-source truly works, you don't just declare yourself the winner. Talk about how the arbiter of the de facto in the open source is not a free pass.

Robert Nishihara

>> I mean, building an open-source project... When we started Ray, the stuff we did to try to get people to use Ray is very similar to what you would do if you were building a company and building a product. You are sitting down with individual users, watching over their shoulder as they try it out, fixing the issues they run into, handholding them, getting feedback. You're also evangelizing it, writing blog posts, giving talks, running boot camps and tutorials, creating educational material. There's so much you're doing here to try to build that.

John Furrier

>> Enlighten.

Robert Nishihara

>> Yeah.

John Furrier

>> Enlighten the crowd. Okay, so let's talk about now. In the last couple of minutes we have left, I want to talk about where you guys are at now. Obviously, great commercial success. Congratulations with Anyscale on the Ray open-source initiative, which is well-earned. Congratulations. Where are you at in the business now, because we are seeing the AI and the enterprise absolutely smoking hot right now because they're in formulation mode. Yeah, they're getting gear, there's POCs everywhere, but you start to see them lock into generative views of how do they use generative AI. They're looking at distributed compute, everything you're talking about. What's on their mind? How is business? What's your focus right now?

Robert Nishihara

>> Yeah, so I mentioned for the first few years of the company, our focus was on growing Ray, getting everyone to use Ray. And our strategy from the start has been very simple. It was the obvious strategy of get everyone to use Ray and then build a platform that adds tremendous value for Ray users. And that inflection point with Ray adoption was around the time of ChatGPT, right? Where now all of a sudden you have Uber, Pinterest, Spotify, everybody building their ML platforms on top of Ray. And so, where we've been going very deep. Now, in addition, we're investing very heavily in continuing to grow Ray, and we're investing very heavily in building our commercial platform that adds value for the Ray users. And where we add value is a few different dimensions. A lot of it is tooling that you would otherwise have to build for managing Ray. So, this is things like observability tooling for being able to debug more easily, being able to profile performance and get good performance more easily. Tooling for governance, tracking costs, monitoring everything. So, there's the tooling aspect. There's also a huge amount that we do in terms of cost and performance optimizations, like reliability at scale, reliable usage of spot instances, general optimizing instance types to save money. So, there's a huge amount we do on that dimension as well. Now, where we've done really well in our sweet spot today, a lot of it is digital natives, like tech companies, people who are in production with AI and using AI very seriously. So, we work with customers like Canva or Coinbase, Instacart. We work with AI startups like Runway. We work with biotech companies that use AI very seriously, like Recursion Pharmaceuticals. So, these are companies that-

John Furrier

>> They're hardcore.

Robert Nishihara

>> Yeah, exactly.

John Furrier

>> They're totally hardcore.

Robert Nishihara

>> Our sweet spot today is companies that are using AI very heavily, care about scale, care about cost, care about all of these types of things. And a lot of where we are, of course, expanding now is in the traditional enterprise as well.

John Furrier

>> What's the learnings now? If you look back, just where you're at today looking at... By the way, it's a great future and those hardcore users will educate the fast followers. So, I think that's going to be a good play for you. But what did you learn and what do you look back into where you are now? What's the big takeaway for you?

Robert Nishihara

>> The big takeaway, and maybe this is obvious, is that a lot of the trajectory of our business has come down to the overall maturity of the market with AI. So, when AI was not a priority for most businesses, we'd go out and pitch distributed computing for AI, and it wasn't really landing. Now, that every company cares about AI, and as I would say around the time of ChatGPT, companies were starting to experiment with AI. They were asking themselves questions like, "Should I build a chatbot? Should we use AI for customer support? Should we use AI for internal productivity tools? How do we use AI?" It was very experimental, but what we've seen over the past couple of years is that these AI applications, which were in the prototype experimentation stage, are now making their way to production. And as they make their way to production, the nature of the problems you face changes, right? You start to care less about just experimentation. You start to care about scale, cost, developer velocity, all of these types of things. Time to market.

John Furrier

>> Security, hardened ability. I mean, one of the things that's coming out of the AI agent conference, preview mixture of expert series we're doing here and other interviews we've done in theCUBE is it's easy to throw a demo out there, hard to scale. That's a theme. So, speed is part of the game on both fronts. Hey, I do a demo, I got 1,000 users, but that's not in production.

Robert Nishihara

>> It's a very real demo to production .

John Furrier

>> Yeah, that is the problem I'm seeing in production. What do you say to those folks out there now that are technical, that are watching, that are going to be a fast follower to this tech stack you guys are seeing success in? What's the story and narrative to that group that's watching? What do they do? How do they engage?

Robert Nishihara

>> Look, in order to bridge this demo to production gap, we see companies need to invest very heavily in evals. Because if you are very good at evals, you can be more confident when you deploy. And also, as you upgrade your model, as you switch from one approach to another, you can have some confidence that that's going to work out. So, that's an area we see people over-invest. Also, it really depends on the domain. There are certain areas where the demo to production gap, like with robotics is just massive. Robotics is an area that's incredibly promising and also incredibly hard, and there are other areas where the consequences of lack of reliability are a little lower and you can ship stuff more easily.

John Furrier

>> Yeah. All right. Robert, thanks for coming in today. I really appreciate it and great to see you and congratulations on your success. My final question for you is, what's the coolest thing you're working on right now that you think is super cool?

Robert Nishihara

>> Look, I'm very excited about the multimodal data-processing stuff we mentioned. I think the nature of data processing is just going to completely change over the coming years and really shift from being SQL-centric on CPUs to being AI-centric on GPUs. And of course, SQL's not going anywhere. SQL's an incredibly powerful technology-

John Furrier

>> ....

Robert Nishihara

>> but if you think about the growth areas, it's AI, and that's how companies are going to get value out of data.

John Furrier

>> Yeah, SQL just goes into the fold.

Robert Nishihara

>> Yeah. I would say on top of that, I've been incredibly excited by what companies are doing with post-training using Ray. So, post-training is another... I'm sure you've heard people talk about pre-training saturating and post-training being the future of how we make models more intelligent. ByteDance is one example. They released an open-source framework for post-training on top of Ray. Nvidia has been post-training models with Ray. We see a number of other companies building post-training technology on top of Ray, and we think that's going to be an exciting growth area.

John Furrier

>> Well, Robert and the entire team at Anyscale, tell them we are pumped for them. We've been following you guys from the beginning. I mean the market changed, but you guys kept on doing great work. So, thanks for that and appreciate it. And thanks for being part of theCUBE preview of the AI Conference.

Robert Nishihara

>> Yeah, thank you so much.

John Furrier

>> All right. cool. I'm John Furrier, here inside theCUBE. The market's changing, and again, this wave is not stopping and it's all about the reasoning, it's all about that post-training, it's about data processing. The operating system of the future doesn't look like it did before. And again, it's large-scale systems problems, system opportunities, and theCUBE is bringing all the data here from our system. Thanks for watching.