theCUBE + NYSE Wired: AI Agent Conference | Jason Lopatecki, Arize AI

Clips
More from theCUBE + NYSE Wired: AI Agent Conference

Jason Lopatecki

CEO & Founder

Arize AI

play_circle_outline From AI Observability Startup to Coworker Agents: Six Years Building Automation That Transforms Jobs

play_circle_outline Arize Phoenix: Open-Source Observability and Evals for Agent Behavior, Context Graphs, and Autonomous Improvement

play_circle_outline Enterprise Automation Use Cases — Customer Support, Delivery, Healthcare, Development Productivity — Scaffolding, Change Management, Continuous Evaluation

Info

Jason Lopatecki, Arize AI

Jason Lopatecki

CEO & Founder Arize AI

search

John Furrier

>> Welcome back everyone to theCUBE Studios here at the New York Stock Exchange. I'm John Furrier, host of theCUBE. Of course, we have our Palo Alto studio connecting Silicon Valley and Wall Street as AI continues to be the game changer up and down the stack, the full stack and AI native developers are coming in. First programming, now agents coming in fast. We got Jason here. Part of the AI conference preview happening next week here in New York City, CEO and founder of Arize AI. Jason, thanks for coming on the program as a pre-game riff on what's coming up at the Agent Conference. You mentioned the AI Engineer Fair in San Francisco.

>> Yeah.

John Furrier

>> I mean, the market for agentic infrastructure is super hot right now and you're starting to see the formation of this. I love the AI engineering. It's kind of a DevOps vibe, but for data.

>> Yeah, yeah.

John Furrier

>> All the same cloud-native kind of things happening at this agent level. Thanks for coming on and previewing the conference.

>> Yeah, thanks for having me. I mean, it's one of the biggest revolutions I've seen in my lifetime. And every industry, every job, every position, will have agents that help you and assist you in almost like your coworkers and your job every day. And that's happening industry by industry. So one of the biggest shifts I've ever seen.

John Furrier

>> Yeah. And there's a lot of entrepreneurs too making it happen. Again, it's happening faster. I want to get into the speed side of the game and you've had multi-exit opportunities in the past, infrastructure and application market. But first, talk about what you guys are working on right now. You guys have a key part of the stack. Explain what you guys do and then we'll get into some of the things going on in the market.

>> Yeah. So we do agent evaluation and agent observability. So our vision is to help make the world's AI work and make it do what you want it to do. And people are building intelligence, human intelligence into these systems and controlling them and making sure they're doing what you want in your business, they're making the actions you want in your business. The scaffolding's there to make sure the things are scaling up and correctly doing the actions that you want. That is what we do. And it's one of the toughest problems, I think, in the space.

John Furrier

>> How long have you guys been around? When have you guys formed? When did you start tackling this opportunity?

>> Yeah, so we started in the early 2020, so I think five to six years, six years-ish. And we started as the original AI observability company. So we were tackling what was, I think, one of the hardest problems, which is understanding these really complex systems and making information for humans to make decisions on it. Explosion in the last two years, absolute explosion. And I just think in the last year, it's just agents. Agents are taking over so much of the work in businesses, are taking over a lot of work in development at this point. And just we're in the early innings of what is a revolution, I think, or evolution.

John Furrier

>> These agents are not only good for business, developers love agents. I mean, come on, look at the uptake on OpenClaw. I call it the new drug. Everyone's smoking OpenClaw these days. And it's like if you're under the age of 30, everyone's doing OpenClaw because it's such a cool thing because you can see what it does. And it might be a little fast and loose, a little chaotic, but that can be reigned in, but it's a steady state picture.

>> Oh yeah. I think what's fascinating is you have Claude Code, which has taken over really the engineer persona, the people building these agents and assistants. But you look at OpenClaw and its form factor is for the common person. It's for average people. I had a story from someone who's running a bar who was using OpenClaw to order inventory and just take care of these jobs that he had people do before. So I view OpenClaw as actually a bigger market than Claude Code even, and a much more accessible one for average people running businesses.

John Furrier

>> Talk about the category that's emerging because we mentioned speed of the game. Again, you've done other ventures in the past. There's usually an incubation period before idea. You get a beachhead, you grow it, you get some product market fit. Now with agents, there's so much work that could be done. I mean, forget the coding side. We're seeing that play out in real time, but the enterprise and corporate America, our lives are now impacted. What's the speed side of this? How would you scope that? How would you frame the velocity of the pace of play in this market right now?

>> I've never seen something this fast. And I, as a second-time entrepreneur and one who's run fast all my life, I've never seen ... The speed is 10X what I think a lot of people are used to. There's those of us who love it, but it just means that the things are changing and you need to be up to speed on the latest ... Literally every week, every month, you need to be using these technologies or you just are going to be ... You're going to be, I call it a dinosaur. You just are not going to be able to keep up with the rate of change that these systems are changing, but also the rate of change of the way of doing business. These things are changing the way you do business, and that is changing really quickly too.

John Furrier

>> I was having dinner last night here in New York City. I was talking to a entrepreneur. I was advising just as friend, and he asked me the team question, how do I put a team together in the age of agents? It was also kind of a culture question around who's a good partner, things of that nature in the ecosystem. And I said, "Look, it's like whitewater rafting level five. Everyone in the raft has to be paddling together." It's really fast. And that's a key piece of the team. What's your reaction to that? Because the speed is happening. It's verified. Everyone's talking about it. But what is also happening is these teams are forming, whether they're us humans or agents. So people are trying to find partners, build companies faster. Talk about this teamwork aspect and the kind of makeup of who have the metal to handle it.

>> It's funny. There's this term now in San Francisco called pilled. And it comes from, I think, the Matrix where you take the pill and you're in the matrix. And what it means is really that it's a form of saying, are people really using this technology a lot? And you want a team where everyone on the team is using AI every day to try to automate as much as possible. And the more your team is doing that, the better they are at these tools. They're just tools. They're powerful tools, but they're tools. And the more you learn them, the better the whole group is at using them because you're sharing, you're building tools that other people use. So really the important thing is the people in the raft are really using these things and using them a lot.

John Furrier

>> Yeah. And they're all working together. I got to ask you about the observability piece. I'm fascinated by that because we saw the role of observability to be very strategic and instrumental in cloud native, very important mechanism. Now, how does that translate into AI? Because you mentioned evaluation. That was hot last year at Databricks' Data and AI Summit. It also brings in the role of the HR department of agents. Are they doing their job? So explain the observability role in AI, how that's different from cloud native, and then evaluation and why that's so important.

>> Yeah. So on observability side, drastically different than the cloud space. And the reason is because the traces, these things that the system throws off are designed for self-improvement. You want these agents to get better on their own. You want these agents to be able to tackle a problem that they fail at and actually get better at it. And the observability and the systems that we've built are designed to help with these auto improvement workflows. It is the foundation. Observability is the foundation of agents auto-improving. And then evals are really just this fancy word of saying, "I'm going to leverage AI to assess and understand if my AI system's working correctly." And if you look at it, humans grade humans, teachers grade students. So having intelligence AI grade intelligence is not a crazy idea, but to scale out these systems, they're not going to work correctly without AI overseeing AI. And that's essentially what evals are is using AI to understand if AI is working correctly.

John Furrier

>> How should enterprises think about this agent adoption? Any advice, best practices, because they saw ... The wave one of gen AI was very easy. RAG was great, search and discovery, navigation, marketing materials, copy, check. But when coding hit last year and this year, people saw the productivity and they could actually quantify that value in terms of time of an engineer and then ship product, which drives revenue. That to me is a sign that the agents are going to come in super fast. What's your take on that? How would you explain ... Do you agree or what's your reaction to that? Because I think the enterprise is about to see an unleashing of agents coming in and either it's going to work or fail. What's your reaction to that?

>> Yeah, I mean, I think you have to ... I think that the enterprise problem is one largely of change management. You've got these systems, you got these ways of doing things that are frankly old at this point. And they were old. They're old a couple months ago. They're really old now. And how do you scale these up, scale agents up to take over processes? How do you encourage the employees who are driving this and encourage the systems that you have in place not to push back on it too, because there is going to be friction in this process? So I think you need all that around change management. And then you need scaffolding, which is what we provide, evals to make sure as you scale this up, the stuff's doing what you want. You're not saying the wrong things to customers, you're not making the wrong decisions in your business. So as you build out automation, you need to evaluate it to check it, to track it. You need to really think about change management within your org and the automation and stresses that automation's going to bring.

John Furrier

>> We got the agent conference happening. Simon Chan and his community together. Start as a meetup. Now it's full blown AI engineering focus. What is the sentiment in that group, that micro community? Because it feels a lot like DevOps-like mindset, but for agentic data infrastructure. So more databases, more kinds of different observability. What are some of the threshold issues being discussed? And can you share your thoughts?

>> I think it's a lot about how to ... I mean, I think all of us can see the power of these systems and the power and potential. And I think a lot of it right now is just how to unleash it in ways in different organizations and different verticals and different things in ways that you just kind of give you that 10X leap on what you're doing today. So a little bit is like how to work it through each use case, how to scale it, how to actually make sure it's doing what you want it to do. So those are some of the biggest problems. But from verticals like ... Every vertical you think of, from customer support to delivery to healthcare, every single one is looking at these automations and how to make sure they're doing what you want, and then how to make sure it's working in your business.

John Furrier

>> Yeah, a lot of mechanics going on. Put a plug in for your company and talk about some of the use cases you see people moving on today and use cases that you imagine will come super fast after that.

>> Yeah. So as you're thinking about your enterprise, as you're thinking about automating, say, customer service in some way, I want to have some automated support, you have to have our solution, Arize or our open source solution, Arize Phoenix, as something that's watching this system and making sure it's doing what you want it to do, to make sure those responses are correct to your customers, you want to capture this data, really these context graphs in a way that you can use for other agents, you can use in the rest of your business, you can use these decisions made in your business to improve it in ways you couldn't before. So you need evals, you need observability to really deliver this stuff and make it work well.

John Furrier

>> The agent conference is one of many conversations. You mentioned before we came on camera about an event in San Francisco. I mentioned the top AI engineer festival, fair. What other events should people be looking at right now this year? Because there's a lot of build, operate, and investors kind of looking at this one sliver of the stack that's going to unleash more AI native development action.

>> Absolutely. So those the AI engineer world fair put on by Swix, top billing, big plug. I love that. I think it's the best conference out there. So check that out. And then just a plug for us, we do have our conference June 4th called Arize Observe. If you're in San Francisco, feel free to grab that one. Obviously, lots of momentum in the AI space here.

John Furrier

>> Jason, it's great to have you on. Congratulations. You're in a hot area. A lot of headroom and a lot of growth. And the Linux Foundation just announced the Agentic AI Foundation, which we think might look like a 10X scale to CNCF, which was the KubeCon Kubernetes cloud native world, which did a lot of great work, but AI is going to build on top of it. So congratulations.

>> I mean, this is the 10X platform. I've never seen my life the growth and potential. So awesome.

John Furrier

>> Yeah. Great to see you. Thanks for coming on. Appreciate it.

>> Yeah. Thank you.

John Furrier

>> I'm John Furrier at theCUBE. The agents are coming and this is a business model transformation. Everyone's involved from the C-suite, the deep tech players, of course, the developers, all making it happen at the same time. It's a melting pot of innovation. Of course, we're doing our part to bring it to you. Thanks for watching.

Jason Lopatecki, Arize AI

search

John Furrier

>> Welcome back everyone to theCUBE Studios here at the New York Stock Exchange. I'm John Furrier, host of theCUBE. Of course, we have our Palo Alto studio connecting Silicon Valley and Wall Street as AI continues to be the game changer up and down the stack, the full stack and AI native developers are coming in. First programming, now agents coming in fast. We got Jason here. Part of the AI conference preview happening next week here in New York City, CEO and founder of Arize AI. Jason, thanks for coming on the program as a pre-game riff on what's coming up at the Agent Conference. You mentioned the AI Engineer Fair in San Francisco.

>> Yeah.

John Furrier

>> I mean, the market for agentic infrastructure is super hot right now and you're starting to see the formation of this. I love the AI engineering. It's kind of a DevOps vibe, but for data.

>> Yeah, yeah.

John Furrier

>> All the same cloud-native kind of things happening at this agent level. Thanks for coming on and previewing the conference.

>> Yeah, thanks for having me. I mean, it's one of the biggest revolutions I've seen in my lifetime. And every industry, every job, every position, will have agents that help you and assist you in almost like your coworkers and your job every day. And that's happening industry by industry. So one of the biggest shifts I've ever seen.

John Furrier

>> Yeah. And there's a lot of entrepreneurs too making it happen. Again, it's happening faster. I want to get into the speed side of the game and you've had multi-exit opportunities in the past, infrastructure and application market. But first, talk about what you guys are working on right now. You guys have a key part of the stack. Explain what you guys do and then we'll get into some of the things going on in the market.

>> Yeah. So we do agent evaluation and agent observability. So our vision is to help make the world's AI work and make it do what you want it to do. And people are building intelligence, human intelligence into these systems and controlling them and making sure they're doing what you want in your business, they're making the actions you want in your business. The scaffolding's there to make sure the things are scaling up and correctly doing the actions that you want. That is what we do. And it's one of the toughest problems, I think, in the space.

John Furrier

>> How long have you guys been around? When have you guys formed? When did you start tackling this opportunity?

>> Yeah, so we started in the early 2020, so I think five to six years, six years-ish. And we started as the original AI observability company. So we were tackling what was, I think, one of the hardest problems, which is understanding these really complex systems and making information for humans to make decisions on it. Explosion in the last two years, absolute explosion. And I just think in the last year, it's just agents. Agents are taking over so much of the work in businesses, are taking over a lot of work in development at this point. And just we're in the early innings of what is a revolution, I think, or evolution.

John Furrier

>> These agents are not only good for business, developers love agents. I mean, come on, look at the uptake on OpenClaw. I call it the new drug. Everyone's smoking OpenClaw these days. And it's like if you're under the age of 30, everyone's doing OpenClaw because it's such a cool thing because you can see what it does. And it might be a little fast and loose, a little chaotic, but that can be reigned in, but it's a steady state picture.

>> Oh yeah. I think what's fascinating is you have Claude Code, which has taken over really the engineer persona, the people building these agents and assistants. But you look at OpenClaw and its form factor is for the common person. It's for average people. I had a story from someone who's running a bar who was using OpenClaw to order inventory and just take care of these jobs that he had people do before. So I view OpenClaw as actually a bigger market than Claude Code even, and a much more accessible one for average people running businesses.

John Furrier

>> Talk about the category that's emerging because we mentioned speed of the game. Again, you've done other ventures in the past. There's usually an incubation period before idea. You get a beachhead, you grow it, you get some product market fit. Now with agents, there's so much work that could be done. I mean, forget the coding side. We're seeing that play out in real time, but the enterprise and corporate America, our lives are now impacted. What's the speed side of this? How would you scope that? How would you frame the velocity of the pace of play in this market right now?

>> I've never seen something this fast. And I, as a second-time entrepreneur and one who's run fast all my life, I've never seen ... The speed is 10X what I think a lot of people are used to. There's those of us who love it, but it just means that the things are changing and you need to be up to speed on the latest ... Literally every week, every month, you need to be using these technologies or you just are going to be ... You're going to be, I call it a dinosaur. You just are not going to be able to keep up with the rate of change that these systems are changing, but also the rate of change of the way of doing business. These things are changing the way you do business, and that is changing really quickly too.

John Furrier

>> I was having dinner last night here in New York City. I was talking to a entrepreneur. I was advising just as friend, and he asked me the team question, how do I put a team together in the age of agents? It was also kind of a culture question around who's a good partner, things of that nature in the ecosystem. And I said, "Look, it's like whitewater rafting level five. Everyone in the raft has to be paddling together." It's really fast. And that's a key piece of the team. What's your reaction to that? Because the speed is happening. It's verified. Everyone's talking about it. But what is also happening is these teams are forming, whether they're us humans or agents. So people are trying to find partners, build companies faster. Talk about this teamwork aspect and the kind of makeup of who have the metal to handle it.

>> It's funny. There's this term now in San Francisco called pilled. And it comes from, I think, the Matrix where you take the pill and you're in the matrix. And what it means is really that it's a form of saying, are people really using this technology a lot? And you want a team where everyone on the team is using AI every day to try to automate as much as possible. And the more your team is doing that, the better they are at these tools. They're just tools. They're powerful tools, but they're tools. And the more you learn them, the better the whole group is at using them because you're sharing, you're building tools that other people use. So really the important thing is the people in the raft are really using these things and using them a lot.

John Furrier

>> Yeah. And they're all working together. I got to ask you about the observability piece. I'm fascinated by that because we saw the role of observability to be very strategic and instrumental in cloud native, very important mechanism. Now, how does that translate into AI? Because you mentioned evaluation. That was hot last year at Databricks' Data and AI Summit. It also brings in the role of the HR department of agents. Are they doing their job? So explain the observability role in AI, how that's different from cloud native, and then evaluation and why that's so important.

>> Yeah. So on observability side, drastically different than the cloud space. And the reason is because the traces, these things that the system throws off are designed for self-improvement. You want these agents to get better on their own. You want these agents to be able to tackle a problem that they fail at and actually get better at it. And the observability and the systems that we've built are designed to help with these auto improvement workflows. It is the foundation. Observability is the foundation of agents auto-improving. And then evals are really just this fancy word of saying, "I'm going to leverage AI to assess and understand if my AI system's working correctly." And if you look at it, humans grade humans, teachers grade students. So having intelligence AI grade intelligence is not a crazy idea, but to scale out these systems, they're not going to work correctly without AI overseeing AI. And that's essentially what evals are is using AI to understand if AI is working correctly.

John Furrier

>> How should enterprises think about this agent adoption? Any advice, best practices, because they saw ... The wave one of gen AI was very easy. RAG was great, search and discovery, navigation, marketing materials, copy, check. But when coding hit last year and this year, people saw the productivity and they could actually quantify that value in terms of time of an engineer and then ship product, which drives revenue. That to me is a sign that the agents are going to come in super fast. What's your take on that? How would you explain ... Do you agree or what's your reaction to that? Because I think the enterprise is about to see an unleashing of agents coming in and either it's going to work or fail. What's your reaction to that?

>> Yeah, I mean, I think you have to ... I think that the enterprise problem is one largely of change management. You've got these systems, you got these ways of doing things that are frankly old at this point. And they were old. They're old a couple months ago. They're really old now. And how do you scale these up, scale agents up to take over processes? How do you encourage the employees who are driving this and encourage the systems that you have in place not to push back on it too, because there is going to be friction in this process? So I think you need all that around change management. And then you need scaffolding, which is what we provide, evals to make sure as you scale this up, the stuff's doing what you want. You're not saying the wrong things to customers, you're not making the wrong decisions in your business. So as you build out automation, you need to evaluate it to check it, to track it. You need to really think about change management within your org and the automation and stresses that automation's going to bring.

John Furrier

>> We got the agent conference happening. Simon Chan and his community together. Start as a meetup. Now it's full blown AI engineering focus. What is the sentiment in that group, that micro community? Because it feels a lot like DevOps-like mindset, but for agentic data infrastructure. So more databases, more kinds of different observability. What are some of the threshold issues being discussed? And can you share your thoughts?

>> I think it's a lot about how to ... I mean, I think all of us can see the power of these systems and the power and potential. And I think a lot of it right now is just how to unleash it in ways in different organizations and different verticals and different things in ways that you just kind of give you that 10X leap on what you're doing today. So a little bit is like how to work it through each use case, how to scale it, how to actually make sure it's doing what you want it to do. So those are some of the biggest problems. But from verticals like ... Every vertical you think of, from customer support to delivery to healthcare, every single one is looking at these automations and how to make sure they're doing what you want, and then how to make sure it's working in your business.

John Furrier

>> Yeah, a lot of mechanics going on. Put a plug in for your company and talk about some of the use cases you see people moving on today and use cases that you imagine will come super fast after that.

>> Yeah. So as you're thinking about your enterprise, as you're thinking about automating, say, customer service in some way, I want to have some automated support, you have to have our solution, Arize or our open source solution, Arize Phoenix, as something that's watching this system and making sure it's doing what you want it to do, to make sure those responses are correct to your customers, you want to capture this data, really these context graphs in a way that you can use for other agents, you can use in the rest of your business, you can use these decisions made in your business to improve it in ways you couldn't before. So you need evals, you need observability to really deliver this stuff and make it work well.

John Furrier

>> The agent conference is one of many conversations. You mentioned before we came on camera about an event in San Francisco. I mentioned the top AI engineer festival, fair. What other events should people be looking at right now this year? Because there's a lot of build, operate, and investors kind of looking at this one sliver of the stack that's going to unleash more AI native development action.

>> Absolutely. So those the AI engineer world fair put on by Swix, top billing, big plug. I love that. I think it's the best conference out there. So check that out. And then just a plug for us, we do have our conference June 4th called Arize Observe. If you're in San Francisco, feel free to grab that one. Obviously, lots of momentum in the AI space here.

John Furrier

>> Jason, it's great to have you on. Congratulations. You're in a hot area. A lot of headroom and a lot of growth. And the Linux Foundation just announced the Agentic AI Foundation, which we think might look like a 10X scale to CNCF, which was the KubeCon Kubernetes cloud native world, which did a lot of great work, but AI is going to build on top of it. So congratulations.

>> I mean, this is the 10X platform. I've never seen my life the growth and potential. So awesome.

John Furrier

>> Yeah. Great to see you. Thanks for coming on. Appreciate it.

>> Yeah. Thank you.

John Furrier

>> I'm John Furrier at theCUBE. The agents are coming and this is a business model transformation. Everyone's involved from the C-suite, the deep tech players, of course, the developers, all making it happen at the same time. It's a melting pot of innovation. Of course, we're doing our part to bring it to you. Thanks for watching.