Brian Benedict of Eliza, chief executive officer and co-founder, and Matt Bishop of Eliza, chief technology officer, appear on theCUBE and NYSE Wired to discuss agentic artificial intelligence adoption and developer velocity in enterprise environments. They explain how Eliza codifies engineering standards accelerates developer velocity and implements practical rollout strategies. theCUBE Research appears throughout and hosts steer the conversation toward real-world use cases model selection and token economics.
Key takeaways include a focus on targeted high-impact automation and the importance of data readiness. Benedict emphasizes that codifying engineering principles can increase output tenfold while reducing token spend. They underscore the need for data readiness and governance to enable scalable deployments. Bishop highlights developer velocity and provides case studies, including a request for proposal automation that reduces timelines from weeks to days. They address model diversification, private equity portfolio plays, build versus buy tradeoffs and self-hosting for sensitive data, offering practical guidance for engineering leaders product teams and investors evaluating agentic AI adoption in the enterprise.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For theCUBE + NYSE Wired: Mixture of Experts Series
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for theCUBE + NYSE Wired: Mixture of Experts Series.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: Mixture of Experts Series
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: Mixture of Experts Series. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Brian Benedict & Matt Bishop, Eliza
Brian Benedict of Eliza, chief executive officer and co-founder, and Matt Bishop of Eliza, chief technology officer, appear on theCUBE and NYSE Wired to discuss agentic artificial intelligence adoption and developer velocity in enterprise environments. They explain how Eliza codifies engineering standards accelerates developer velocity and implements practical rollout strategies. theCUBE Research appears throughout and hosts steer the conversation toward real-world use cases model selection and token economics.
Key takeaways include a focus on targeted high-impact automation and the importance of data readiness. Benedict emphasizes that codifying engineering principles can increase output tenfold while reducing token spend. They underscore the need for data readiness and governance to enable scalable deployments. Bishop highlights developer velocity and provides case studies, including a request for proposal automation that reduces timelines from weeks to days. They address model diversification, private equity portfolio plays, build versus buy tradeoffs and self-hosting for sensitive data, offering practical guidance for engineering leaders product teams and investors evaluating agentic AI adoption in the enterprise.
>> Palo Alto Studio Connection, Silicon Valley and Wall Street, I'm John Furrier, host, and here with Dave Vellante, my co-host.
Gemma Allen
>> Welcome back to theCUBE Studio here at the New York Stock Exchange. I'm Gemma Allen with NYSE Wired's Mixture of Experts. Joining me now are two founders who are bringing AI to life in enterprise environments. Welcome, Brian Benedict, CEO and co-founder of Eliza, and Matt Bishop, CTO of Eliza. Welcome, guys.
Brian Benedict
>> Thank you.
Matt Bishop
>> Thanks, Gemma.
Gemma Allen
>> Brian, you were here a couple of months ago.
Brian Benedict
>> Yeah.
Gemma Allen
>> That now feels like a couple of years ago in this world, in this life. First of all, bring us up to speed. What's been happening in your space since you were last on the show?
Brian Benedict
>> Yeah, wow. Lot to talk about there. Eliza has grown. We kind of started ourselves as an agentic AI native solution provider in the space. Since we got our inception and start, we partnered with OpenAI. We've been working across a myriad of different companies, whether it'd be private equity or large organizations, to really deliver workflows and agents across the board. I couldn't be more happy with the growth and the team and how we continually are transforming companies in this day and age. Yeah, very exciting stuff.
Gemma Allen
>> Exciting. Matt, we hear a lot about AI at enterprise level. AI is a game changer. The world as we know it is no more. We also hear skepticism around what value is being realized inside of enterprise workflows and environments. What are you seeing? Give us a reality check here, market narrative meets actual office cube.
Matt Bishop
>> We're seeing a lot of value gains with the engineering team. Traditional engineering is just slow, cumbersome. When organizations adopt coding tools like Codex, they two, three, 10X their engineers' output, and they get consistency across the organization as well on how to actually develop in a consistent and high quality way. The skepticism is, I think, unfounded here, for sure, especially in the engineering org.
Gemma Allen
>> One theme that has emerged since, Brian, you were last here is this whole idea of token leaderboards, tokenomics. You talk about engineers. We hear a lot about folks trying to max out token spend. The best boy or girl spends the most, which seems kind of counterintuitive, but perhaps we ... Talk a little bit about what's happening from the perspective of how folks inside enterprise are actually using, playing around, trying to understand the opportunity of these tools. What is that waste versus true outcome like?
Brian Benedict
>> That's a fantastic question. We see some organizations that are trying to token max and just throwing AI at absolutely everything as fast as humanly possible. What happens, though, is you end up really getting your CFO pretty upset pretty quickly. The other thing is the quality of the code base starts to degrade. What we see and what we advise our customers on is, when you really understand the engineering principles and you codify those in your AI development environment, you get consistency and quality across your entire engineering team. All your junior developers start to look like senior, seniors start to look like staff. You just raise everyone to the exact same standards and 10X their output. What happens when you get that consistency, you actually reduce token spend. Whereas other organizations that don't have that structure and discipline, they token max, and then the CFO gets a little upset with the entire engineering org.
Gemma Allen
>> Another thing we hear sometimes is this is a race, right? This is a race. You can't get left behind. Speed, speed, speed. Act now, think later tends to be the overall thesis of this moment. When you try and do that, like in these massive enterprise environments, you can also try and boil the ocean a little bit, right?
Brian Benedict
>> Yeah.
Gemma Allen
>> Where is the low hanging fruit? I'll ask this to either of you. Day one, you're showing up at some large enterprises, take some large bank for example, and you're like, "Okay, we need to figure out where to begin here," what does it look like?
Brian Benedict
>> From what we're seeing, there's definitely a two-sides-of-the coin perspective that companies are coming to us for. One is really on, "How do I build AI products and features faster so I can actually get through my backlog much quicker to actually compete in this market that is today?" The other side really is, "Looking at my back office, how can I have margin expansion? How do we create just an easier workflow and time for our own employees so that they're not burdened with so many manual processes?" I'd probably say going into this year, a lot of it was much more focused around feature optimization and new product optimization. I would probably say going in ... Like now, we're seeing a lot more 50/50 of companies coming to us really saying, "I need to look at the back office more," CFO office being one of the main areas like FP&A and some of the execution tasks that they're trying to alleviate from the burdensome work of manualness. I'm not sure how you feel about that.
Matt Bishop
>> We believe that task or the atomic unit of work, before you can ever really transform an organization, you need to understand how people actually work, who they work with, what tools they leverage. We actually created a platform called Task that allows us to agentically go out and interview the entire organization and understand exactly what you do from a day-to-day. That data then comes back to us, and we're able to work with Codex to create skills, workflow automations. We had a client that we did this for the RFP process, request for proposal process. It spans six different role types. We did, I think, 40 different agentic interviews in about 45 minutes. We took that data back, and we were able to build a bespoke RFP automation tool that looped in humans at just the right moment and then automated everything else. We took their RFP process from four to six weeks down to just a few days.
Gemma Allen
>> That's so interesting. Because in some respects, if AI can help you understand your full environment and discover what's there so you can really make intelligible decisions on what's possible, that alone would be a massive step forward for enterprise. We hear so much that folks wouldn't even know what they have deployed half the time. Going back to the back office scenario for a second and the reverse engineering opportunity for this moment we're in, since you were on the show too, Brian, IBM had an interesting week whereby Claude Code released this version that could basically take out COBOL, and people panicked. People were like, "Oh my God, IBM is going to be so screwed by this." Again, like everything, that kind of hype was quickly ended. But it is interesting, though, because there is so many legacy line of business applications across enterprise that have been so stifled for so long by the fact that they can't be upgraded. What are you guys seeing in that space? Is this typically folks are looking for new opportunities to grow proprietary tech fast, or is there also a huge opportunity here to actually solve some of the challenge and, I guess again, like I said, hamstrung opportunities ?
Brian Benedict
>> I'll take that one, and then you can jump in on this.
Matt Bishop
>> Okay.
Brian Benedict
>> A great example of this is ERP systems. There is a reason why there are so many new startups coming out with new agentic AI systems for ERP. We're seeing a lot of that coming through from our client roster asking, "How do we maneuver in this new world? Do we modernize our ERP? Do we rip and replace our ERP with these new systems?" So it all is case by case in terms of what really is the best recommendation, but you're seeing a lot of that stuff. Whether it's old legacy code, whether it's legacy ERP systems, people are on the fence right now, and even CRMs, and wondering, "Do I need a CRM?" Klarna went hard at removing Salesforce. Then they backed off and said, "Okay, we need to actually bring people back and do other things here." So it's just a really interesting time to see. My personal opinion, I still think it's a little bit of a wait-and-see approach. You got to still really figure out where you are in that life cycle and what really the solution sets because so many things are net new in these different places. I think a lot of people are dipping their toe in the water and so trying to figure it out in terms of what they're going to do next there. Matt, what do you think?
Gemma Allen
>> What are you seeing from the perspective of what folks are looking for from a metrics perspective around these engagements? What are the top five things people want to see at the end or halfway through progress in any engagement with Eliza?
Matt Bishop
>> I'd say developer velocity. We have one client that is a vertical SaaS roll up in legal tech. They have eight different brands, and they're trying to consolidate them all into one brand. When we came in and we taught their engineering org, we first codified their engineering standards and then taught their engineering org how to do agentic-enabled software development. We were able to cut their development time down in about a fifth. They were trying to consolidate one of these subsidiary brands into their new flagship product. It was supposed to take three months. It took two weeks. So from a metrics perspective, just sheer moving the product roadmap forward so quickly, it's a very obvious ROI and an investment value-add.
Gemma Allen
>> Staying with you for a second, Matt. A lot of AI projects fail, not necessarily because of the model or the technical capability but because of the data, because of the mess that came before. What are you seeing there? I'm also interested to understand, what are other companies you've seen in that space that are really cutting their teeth making a difference?
Matt Bishop
>> Yeah, definitely. The data is definitely a gating requirement to getting the full value of AI. That being said, you can take very targeted views to the data that you need to enable an experience. You don't need to create an entire data lake across all of your different brands to realize ROI. You can take a very strategic and surgical approach to unlock value quickly. That's what a lot of our PE partners are looking for, too. We don't want these large multi-year projects. We want targeted, immediate value creation.
Gemma Allen
>> Can we get into the models? I know you guys have a great partnership with OpenAI. It seems as though, again, there is a shifting narrative, somewhat of a ping-pong ball of enthusiasm as to which model is in favor in the press or in the narrative of any given week or month. Anthropic has had a pretty strong hold on enterprise, it feels like. You guys, I think, have some thoughts perhaps on that. Talk me through what you're seeing from the perspective of due diligence, folks fully understanding even, "What is the right model for me?" What does that process alone look like?
Brian Benedict
>> I'll just start by saying, I think eventually we're going to come to a point, and then you and I spoke about this in the past, of just the diversification of models. If you have simple models and simple tasks, you should be using small language models that are easy and cost effective. You have more diverse sets of use cases where you're going to be needing more reasoning models or different levels of instruction following, you should be utilizing more sophisticated, bigger models. I think companies are starting to collapse a little bit on that and starting to figure out, especially given the token maxxing challenges of today, how do we maneuver within the course of what we should use for what use case versus one model fits all? I think that's what we're seeing a lot of right now, a lot of conversations like that.
Matt Bishop
>> I think a lot of the decision point is coming down to the back office. It's, how do we enable agentic knowledge work? In the foundational model companies, they're really starting to invest in the Codex-type applications to unlock workflow automations and just make that overall experience for back office workers more agentified. I think rather than choosing between one model or another, it's more of like, what is the distribution medium and how are people actually interacting with it to deliver value at their company?
Gemma Allen
>> Right now we are still in a world, though, where there is an economy of scale in the deal side of this. You do work with one frontier lab, like Anthropic or OpenAI, and you probably as an enterprise get a better deal. Unlike the days of lore, you try and then, I guess, roll that out across multiple parts of your enterprise just to make sure that it's economically viable. Is the same thing happening here? If not, if you have cases whereby certain functions or certain groups have preferences for particular models, how sticky are they? How easy is it to decide tomorrow, "Listen, we should really continue to evolve this and need to test what's coming," even from the perspective perhaps of some of the open-weight models that really haven't been allowed in the door yet? So what are you seeing there, and what are your thoughts?
Matt Bishop
>> I think the strategic partnerships that OpenAI has with AWS is particularly interesting because I have an annual commitment as a CTO to AWS that I need to hit, and now I can run my OpenAI spend through that exact same commitment. So we are getting some economies of scale and consolidation through these really strategic partnerships. I can buy my OpenAI components as individual SKUs now within the AWS marketplace, which is allowing me a lot of flexibility as a CTO on how I actually want to deploy AI within the organization.
Gemma Allen
>> Okay, interesting. On the inference layer, we hear a lot about software, whether or not it's fully off to the races, whether or not it's where it could possibly be to help enterprises, especially work directly to maximize and optimize value from hardware spend, from metal spend. It seems as though that's still an evolving conversation. What are you guys actually seeing in terms of the ownership of this at an enterprise level? Do you see a world where enterprises are increasingly trying to foresee a future where they fully own their own stock, they manage it? Or again, because of all of the way in which tech has evolved, you imagine that we're always going to have these kind of dependencies, these customer/hostage relationships with these massive players?
Matt Bishop
>> We see certain areas in sensitive environments like healthcare where there's definitely a more interest in self-hosting open source models and keeping the data completely within their environment and not sending it to the foundational model providers. Even though the providers, they're HIPAA compliant, they have all the certifications, the CTOs within these organizations prefer to keep their data within their four walls. I think there will definitely still be use cases where you don't want data going out, and you need to control that experience and expense within your own environment.
Brian Benedict
>> One more thing I want to add to that, too. I don't think the build versus buy debate has ever been more in full effect than right now because you can build custom for what you need so much faster.
Gemma Allen
>> Wow.
Brian Benedict
>> When you think about, you have a CRM and you've got like 70% of all the for the CRM you're not even using, why are you paying for it? When you think of all these software solutions that have all these features and functionalities and you're like, "I use this much. Why am I paying for this? I can like build something agentically that just does that." That is what every CTO is looking at, what every CFO is looking at across their balance sheet, like, "What can we just replace and reduce that cost friction and get what we need?" Because a lot of the software solutions are built for the majority and the masses, not for you. I think there's never been a bigger debate right now than that-
Gemma Allen
>> Wow....
Brian Benedict
>> in terms of what's happening at enterprises today.
Gemma Allen
>> Well, folks, it seems like both of you and the folks at Eliza are front and center of that debate. To close, what's ahead? We're going to be back here hopefully six months to a year. What should we expect to be hearing from you? What are the big goals, big dreams? Break it down.
Brian Benedict
>> Sure. Matt, you want to start?
Matt Bishop
>> Sure thing. We partner heavily within private equity. One of the big focus areas is doing these large-scale portfolio assessments and understanding how we can create repeatable plays across that portfolio where you build once, deploy many, and rapidly evolve an organization that's backed by private equity and bring them into an AI native world. I think you'll see lots of economies of scale next time we chat.
Brian Benedict
>> I would just add, our own growth is our goal is to get to a hundred forward-deployed engineers-
Gemma Allen
>> Wow....
Brian Benedict
>> and to grow our company, and we continually are doing a great job at that. So we are hiring massively, so we're going to continue to do that. So hopefully next time I see you, there's an army of a hundred of us behind us.
Gemma Allen
>> Well, that is certainly music to my ear is this mad moment you're in. Folks, thanks so much for joining us on theCUBE.
Brian Benedict
>> You bet, Gemma.
Matt Bishop
>> Thanks for having us.
Brian Benedict
>> Thanks for having us.
Gemma Allen
>> I'm Gemma Allen coming to you from theCUBE's NYSE studio. This is Mixture of Experts, one of our programs at NYSE Wired. Thanks for watching.
>> Palo Alto Studio Connection, Silicon Valley and Wall Street, I'm John Furrier, host, and here with Dave Vellante, my co-host.
Gemma Allen
>> Welcome back to theCUBE Studio here at the New York Stock Exchange. I'm Gemma Allen with NYSE Wired's Mixture of Experts. Joining me now are two founders who are bringing AI to life in enterprise environments. Welcome, Brian Benedict, CEO and co-founder of Eliza, and Matt Bishop, CTO of Eliza. Welcome, guys.
Brian Benedict
>> Thank you.
Matt Bishop
>> Thanks, Gemma.
Gemma Allen
>> Brian, you were here a couple of months ago.
Brian Benedict
>> Yeah.
Gemma Allen
>> That now feels like a couple of years ago in this world, in this life. First of all, bring us up to speed. What's been happening in your space since you were last on the show?
Brian Benedict
>> Yeah, wow. Lot to talk about there. Eliza has grown. We kind of started ourselves as an agentic AI native solution provider in the space. Since we got our inception and start, we partnered with OpenAI. We've been working across a myriad of different companies, whether it'd be private equity or large organizations, to really deliver workflows and agents across the board. I couldn't be more happy with the growth and the team and how we continually are transforming companies in this day and age. Yeah, very exciting stuff.
Gemma Allen
>> Exciting. Matt, we hear a lot about AI at enterprise level. AI is a game changer. The world as we know it is no more. We also hear skepticism around what value is being realized inside of enterprise workflows and environments. What are you seeing? Give us a reality check here, market narrative meets actual office cube.
Matt Bishop
>> We're seeing a lot of value gains with the engineering team. Traditional engineering is just slow, cumbersome. When organizations adopt coding tools like Codex, they two, three, 10X their engineers' output, and they get consistency across the organization as well on how to actually develop in a consistent and high quality way. The skepticism is, I think, unfounded here, for sure, especially in the engineering org.
Gemma Allen
>> One theme that has emerged since, Brian, you were last here is this whole idea of token leaderboards, tokenomics. You talk about engineers. We hear a lot about folks trying to max out token spend. The best boy or girl spends the most, which seems kind of counterintuitive, but perhaps we ... Talk a little bit about what's happening from the perspective of how folks inside enterprise are actually using, playing around, trying to understand the opportunity of these tools. What is that waste versus true outcome like?
Brian Benedict
>> That's a fantastic question. We see some organizations that are trying to token max and just throwing AI at absolutely everything as fast as humanly possible. What happens, though, is you end up really getting your CFO pretty upset pretty quickly. The other thing is the quality of the code base starts to degrade. What we see and what we advise our customers on is, when you really understand the engineering principles and you codify those in your AI development environment, you get consistency and quality across your entire engineering team. All your junior developers start to look like senior, seniors start to look like staff. You just raise everyone to the exact same standards and 10X their output. What happens when you get that consistency, you actually reduce token spend. Whereas other organizations that don't have that structure and discipline, they token max, and then the CFO gets a little upset with the entire engineering org.
Gemma Allen
>> Another thing we hear sometimes is this is a race, right? This is a race. You can't get left behind. Speed, speed, speed. Act now, think later tends to be the overall thesis of this moment. When you try and do that, like in these massive enterprise environments, you can also try and boil the ocean a little bit, right?
Brian Benedict
>> Yeah.
Gemma Allen
>> Where is the low hanging fruit? I'll ask this to either of you. Day one, you're showing up at some large enterprises, take some large bank for example, and you're like, "Okay, we need to figure out where to begin here," what does it look like?
Brian Benedict
>> From what we're seeing, there's definitely a two-sides-of-the coin perspective that companies are coming to us for. One is really on, "How do I build AI products and features faster so I can actually get through my backlog much quicker to actually compete in this market that is today?" The other side really is, "Looking at my back office, how can I have margin expansion? How do we create just an easier workflow and time for our own employees so that they're not burdened with so many manual processes?" I'd probably say going into this year, a lot of it was much more focused around feature optimization and new product optimization. I would probably say going in ... Like now, we're seeing a lot more 50/50 of companies coming to us really saying, "I need to look at the back office more," CFO office being one of the main areas like FP&A and some of the execution tasks that they're trying to alleviate from the burdensome work of manualness. I'm not sure how you feel about that.
Matt Bishop
>> We believe that task or the atomic unit of work, before you can ever really transform an organization, you need to understand how people actually work, who they work with, what tools they leverage. We actually created a platform called Task that allows us to agentically go out and interview the entire organization and understand exactly what you do from a day-to-day. That data then comes back to us, and we're able to work with Codex to create skills, workflow automations. We had a client that we did this for the RFP process, request for proposal process. It spans six different role types. We did, I think, 40 different agentic interviews in about 45 minutes. We took that data back, and we were able to build a bespoke RFP automation tool that looped in humans at just the right moment and then automated everything else. We took their RFP process from four to six weeks down to just a few days.
Gemma Allen
>> That's so interesting. Because in some respects, if AI can help you understand your full environment and discover what's there so you can really make intelligible decisions on what's possible, that alone would be a massive step forward for enterprise. We hear so much that folks wouldn't even know what they have deployed half the time. Going back to the back office scenario for a second and the reverse engineering opportunity for this moment we're in, since you were on the show too, Brian, IBM had an interesting week whereby Claude Code released this version that could basically take out COBOL, and people panicked. People were like, "Oh my God, IBM is going to be so screwed by this." Again, like everything, that kind of hype was quickly ended. But it is interesting, though, because there is so many legacy line of business applications across enterprise that have been so stifled for so long by the fact that they can't be upgraded. What are you guys seeing in that space? Is this typically folks are looking for new opportunities to grow proprietary tech fast, or is there also a huge opportunity here to actually solve some of the challenge and, I guess again, like I said, hamstrung opportunities ?
Brian Benedict
>> I'll take that one, and then you can jump in on this.
Matt Bishop
>> Okay.
Brian Benedict
>> A great example of this is ERP systems. There is a reason why there are so many new startups coming out with new agentic AI systems for ERP. We're seeing a lot of that coming through from our client roster asking, "How do we maneuver in this new world? Do we modernize our ERP? Do we rip and replace our ERP with these new systems?" So it all is case by case in terms of what really is the best recommendation, but you're seeing a lot of that stuff. Whether it's old legacy code, whether it's legacy ERP systems, people are on the fence right now, and even CRMs, and wondering, "Do I need a CRM?" Klarna went hard at removing Salesforce. Then they backed off and said, "Okay, we need to actually bring people back and do other things here." So it's just a really interesting time to see. My personal opinion, I still think it's a little bit of a wait-and-see approach. You got to still really figure out where you are in that life cycle and what really the solution sets because so many things are net new in these different places. I think a lot of people are dipping their toe in the water and so trying to figure it out in terms of what they're going to do next there. Matt, what do you think?
Gemma Allen
>> What are you seeing from the perspective of what folks are looking for from a metrics perspective around these engagements? What are the top five things people want to see at the end or halfway through progress in any engagement with Eliza?
Matt Bishop
>> I'd say developer velocity. We have one client that is a vertical SaaS roll up in legal tech. They have eight different brands, and they're trying to consolidate them all into one brand. When we came in and we taught their engineering org, we first codified their engineering standards and then taught their engineering org how to do agentic-enabled software development. We were able to cut their development time down in about a fifth. They were trying to consolidate one of these subsidiary brands into their new flagship product. It was supposed to take three months. It took two weeks. So from a metrics perspective, just sheer moving the product roadmap forward so quickly, it's a very obvious ROI and an investment value-add.
Gemma Allen
>> Staying with you for a second, Matt. A lot of AI projects fail, not necessarily because of the model or the technical capability but because of the data, because of the mess that came before. What are you seeing there? I'm also interested to understand, what are other companies you've seen in that space that are really cutting their teeth making a difference?
Matt Bishop
>> Yeah, definitely. The data is definitely a gating requirement to getting the full value of AI. That being said, you can take very targeted views to the data that you need to enable an experience. You don't need to create an entire data lake across all of your different brands to realize ROI. You can take a very strategic and surgical approach to unlock value quickly. That's what a lot of our PE partners are looking for, too. We don't want these large multi-year projects. We want targeted, immediate value creation.
Gemma Allen
>> Can we get into the models? I know you guys have a great partnership with OpenAI. It seems as though, again, there is a shifting narrative, somewhat of a ping-pong ball of enthusiasm as to which model is in favor in the press or in the narrative of any given week or month. Anthropic has had a pretty strong hold on enterprise, it feels like. You guys, I think, have some thoughts perhaps on that. Talk me through what you're seeing from the perspective of due diligence, folks fully understanding even, "What is the right model for me?" What does that process alone look like?
Brian Benedict
>> I'll just start by saying, I think eventually we're going to come to a point, and then you and I spoke about this in the past, of just the diversification of models. If you have simple models and simple tasks, you should be using small language models that are easy and cost effective. You have more diverse sets of use cases where you're going to be needing more reasoning models or different levels of instruction following, you should be utilizing more sophisticated, bigger models. I think companies are starting to collapse a little bit on that and starting to figure out, especially given the token maxxing challenges of today, how do we maneuver within the course of what we should use for what use case versus one model fits all? I think that's what we're seeing a lot of right now, a lot of conversations like that.
Matt Bishop
>> I think a lot of the decision point is coming down to the back office. It's, how do we enable agentic knowledge work? In the foundational model companies, they're really starting to invest in the Codex-type applications to unlock workflow automations and just make that overall experience for back office workers more agentified. I think rather than choosing between one model or another, it's more of like, what is the distribution medium and how are people actually interacting with it to deliver value at their company?
Gemma Allen
>> Right now we are still in a world, though, where there is an economy of scale in the deal side of this. You do work with one frontier lab, like Anthropic or OpenAI, and you probably as an enterprise get a better deal. Unlike the days of lore, you try and then, I guess, roll that out across multiple parts of your enterprise just to make sure that it's economically viable. Is the same thing happening here? If not, if you have cases whereby certain functions or certain groups have preferences for particular models, how sticky are they? How easy is it to decide tomorrow, "Listen, we should really continue to evolve this and need to test what's coming," even from the perspective perhaps of some of the open-weight models that really haven't been allowed in the door yet? So what are you seeing there, and what are your thoughts?
Matt Bishop
>> I think the strategic partnerships that OpenAI has with AWS is particularly interesting because I have an annual commitment as a CTO to AWS that I need to hit, and now I can run my OpenAI spend through that exact same commitment. So we are getting some economies of scale and consolidation through these really strategic partnerships. I can buy my OpenAI components as individual SKUs now within the AWS marketplace, which is allowing me a lot of flexibility as a CTO on how I actually want to deploy AI within the organization.
Gemma Allen
>> Okay, interesting. On the inference layer, we hear a lot about software, whether or not it's fully off to the races, whether or not it's where it could possibly be to help enterprises, especially work directly to maximize and optimize value from hardware spend, from metal spend. It seems as though that's still an evolving conversation. What are you guys actually seeing in terms of the ownership of this at an enterprise level? Do you see a world where enterprises are increasingly trying to foresee a future where they fully own their own stock, they manage it? Or again, because of all of the way in which tech has evolved, you imagine that we're always going to have these kind of dependencies, these customer/hostage relationships with these massive players?
Matt Bishop
>> We see certain areas in sensitive environments like healthcare where there's definitely a more interest in self-hosting open source models and keeping the data completely within their environment and not sending it to the foundational model providers. Even though the providers, they're HIPAA compliant, they have all the certifications, the CTOs within these organizations prefer to keep their data within their four walls. I think there will definitely still be use cases where you don't want data going out, and you need to control that experience and expense within your own environment.
Brian Benedict
>> One more thing I want to add to that, too. I don't think the build versus buy debate has ever been more in full effect than right now because you can build custom for what you need so much faster.
Gemma Allen
>> Wow.
Brian Benedict
>> When you think about, you have a CRM and you've got like 70% of all the for the CRM you're not even using, why are you paying for it? When you think of all these software solutions that have all these features and functionalities and you're like, "I use this much. Why am I paying for this? I can like build something agentically that just does that." That is what every CTO is looking at, what every CFO is looking at across their balance sheet, like, "What can we just replace and reduce that cost friction and get what we need?" Because a lot of the software solutions are built for the majority and the masses, not for you. I think there's never been a bigger debate right now than that-
Gemma Allen
>> Wow....
Brian Benedict
>> in terms of what's happening at enterprises today.
Gemma Allen
>> Well, folks, it seems like both of you and the folks at Eliza are front and center of that debate. To close, what's ahead? We're going to be back here hopefully six months to a year. What should we expect to be hearing from you? What are the big goals, big dreams? Break it down.
Brian Benedict
>> Sure. Matt, you want to start?
Matt Bishop
>> Sure thing. We partner heavily within private equity. One of the big focus areas is doing these large-scale portfolio assessments and understanding how we can create repeatable plays across that portfolio where you build once, deploy many, and rapidly evolve an organization that's backed by private equity and bring them into an AI native world. I think you'll see lots of economies of scale next time we chat.
Brian Benedict
>> I would just add, our own growth is our goal is to get to a hundred forward-deployed engineers-
Gemma Allen
>> Wow....
Brian Benedict
>> and to grow our company, and we continually are doing a great job at that. So we are hiring massively, so we're going to continue to do that. So hopefully next time I see you, there's an army of a hundred of us behind us.
Gemma Allen
>> Well, that is certainly music to my ear is this mad moment you're in. Folks, thanks so much for joining us on theCUBE.
Brian Benedict
>> You bet, Gemma.
Matt Bishop
>> Thanks for having us.
Brian Benedict
>> Thanks for having us.
Gemma Allen
>> I'm Gemma Allen coming to you from theCUBE's NYSE studio. This is Mixture of Experts, one of our programs at NYSE Wired. Thanks for watching.
John Furrier
Gemma Allen