theCUBE + NYSE Wired: AI + Cloud Leaders | Randall Hunt, Caylent | AI + Cloud Leaders

Clips
More from theCUBE + NYSE Wired: AI + Cloud Leaders

Randall Hunt

CTO

Caylent

play_circle_outline Randall Hunt discusses the rapid evolution of AI technologies and model capabilities.

play_circle_outline Caylent Accelerate platform accelerating database migrations and modern architectures.

play_circle_outline Challenges in migration due to organizational inertia and the need for skill development.

play_circle_outline Adapting Agents: The Evolving Role of Automation and Human Oversight in Task Execution

play_circle_outline Expanding Horizons: Caylent's Vision for Context Engineering and AI Optimization in a Transformative Industry Landscape

Info
Transcript

Randall Hunt, Caylent | AI + Cloud Leaders

Randall Hunt

CTO Caylent

In this in-depth conversation from theCUBE + NYSE Wired: AI + Cloud Leaders event, Randall Hunt, CTO at Caylent, joins John Furrier in studio to dissect the fast-evolving AI and cloud landscape at the critical midpoint between AWS Summit and re:Invent 2025. Known for being early to every major shift – from APIs to serverless to LLMs – Hunt offers firsthand insights into Caylent’s platform strategy and why betting on a single model is a dead end in today’s pace of innovation.

The discussion unpacks Caylent Accelerate, a platform originally built for h... Read more

explore Keep Exploring

What are some notable developments in transformer architecture and the performance of generative models since their introduction in 2017? add

What challenges do auto-aggressive models face, and how has Caylent Accelerate evolved to address database migration processes? add

What is Caylent Accelerate and what does it aim to achieve in terms of DevOps modernization and application migration? add

What challenges and considerations are involved in the development of autonomous agents that require human intervention? add

What plans or developments are anticipated in the second half of the year? add

bolt Powered by CUBE AI

Randall Hunt, Caylent | AI + Cloud Leaders

search

>> Hey, welcome to theCUBE. I'm John Furrier, your host here in our New York City NYSE CUBE Studios. Dave Vellante is here with me all week for a media week on AI and cloud leaders. Of course, AWS is having their big summit. Everybody's in town. Of course, theCUBE is presenting a special halftime to re:Invent report digital event with all the leaders weighing in on the halftime between now and re:Invent because it feels like a whole year has passed in just six months, and it's just only going to get fast and furious from here. As the second half kicks in, so many storylines to talk about. OpenAI, Meta, NVIDIA, the enterprise is smoking hot and the signs that the consumerization side is coming where big breakthrough ideas will come out. So tons of action. Randall Hunt is here, CTO of Caylent, friend of theCUBE, CUBE alumni, also a contributor to theCube Collective and just a participant in the industry and a leader. Randall, great to see you again. Thanks for coming into our studio. Not bad here, the stock exchange.

Randall Hunt

>> Yeah, all the options traders in the background.

>> Yeah, you heard the setup. I mean, it feels like it used to be like during the internet, it's dog years. Every seven years is like your birthday. Here, it's faster. I mean, I got to say the AI tsunami, just the tech change, the disruption, the transformation. I mean digital transformation, a little cliche from a decade ago, but now we're in full business model transformation, stack transformation, architecture transformation. Intel surpassed by NVIDIA, news hitting today. Just like what's your take as a participant? You've seen the waves, you've been in them all.

Randall Hunt

>> So I think it's an interesting development because transformers as an architecture, which is the thing that powers GPT, which is generative pre-trained transformer, came out in 2017. I did a video on it back then. I had no idea it was going to lead to this. And I think what's very interesting is since November of 2022, the price performance of the models has increased at least 2X every four months so that the performance of ChatGPT 4.0 is now available. The intelligence, the raw intelligence of that model is now available 100 times cheaper. We've also seen the rise of reasoning models this year. I think that's an interesting development where you're spending more tokens to get more intelligence out of things. I think there's a lot of stuff going on that Amazon came out with Nova Sonic, which is sort of this mixed modality model where it's a speech to speech model that also has a text modality for calling tools in the background. It's a really rapidly developing field, and I think customers are trying to take advantage of it, but they kind of need experts to help them figure out what's the best thing. And I see companies betting on one model, and I just think that's such a terrible approach. I think you need to build a platform.

>> Why?

Randall Hunt

>> Because the models, like I said, are moving so quickly. One model family could develop capabilities that immediately allow you to differentiate yourself from your competitors. So I think it's better to build a platform that can constantly and consistently take advantage of the state of the art.

>> We're going to get into similar you're doing at Caylent, but I love talking with you because for the folks that don't know, you always are an early adopter on everything. You got your fingers in the betas, you do videos on them, you're really in the tech. And if you go back to the cloud, APIs were great. APIs, the building blocks, that was legendary. What's the equivalent now? Because you mentioned some of these architectural decisions around betting on models. It feels like very API-ish in the sense of you don't want to lock into one thing, that's like locking in an API into a UI or something. It's not even a concept anyone would do. What is the architectural constraints or opportunities? DeepSeek did some cool things by default, trying to reverse engineer CUDA and then look what they did. So I mean there's these accidental greatness that happens, but what is the thinking state of the art from a system architecture perspective, thinking about I want an API-like environment with all these models and MCPs out there, A2A, we're seeing that traction Linux Foundation's got it, there's so much going on. My mind's blown up.

Randall Hunt

>> Yeah, I think MCP to me is the protocol play, and it came out in November of '24, and I sent a message to the whole team. I was like, "Guys, I think we should look into this a little more deeply." And we did, and we bet pretty big on it. I still think that's probably the right approach. I know people have said things about security around it and everything, but I think what people fail to understand is those are the exact same security concerns we've always had. It's the same vector of executing untrusted code. So from a protocol standpoint, MCP makes a lot of sense to me. There's sampling and prompts, which I think people aren't taking enough advantage of within MCP. And then there's the overall operating system play of generative AI. And Karpathy talks about this. Andreessen Horowitz did their generative AI operating system LLM OS. I do think we're getting there, and I think tool use is one of the ways that we've seen things explode in capability.

>> Tool use as in they're easy to build or easy to use or new tools are being generated? What do you mean by tool use?

Randall Hunt

>> The models using tools instead of using the latent space of the model to do complex tasks. Because let me tell you, the most expensive possible way you can do math right now is using an LLM to say, "Hey, go do math." Let the computer do what the computer is good at. It's excellent at math, it can run billions of instructions per second. Just do the plain old-fashioned math on the computer. Let the LLM take on the reasoning and the abstractions, the complex flows of information from one tool to the next tool. So the LLM really becomes the orchestrator, and then the tools are performing even downstream specialized inputs.

>> So it's like distributed computing when that had hit, that was a good thing. So distributed AI understanding is kind of what I'm getting at. Am I getting that right? Because you're basically saying don't waste tokens on basic things.

Randall Hunt

>> Exactly. It's the most expensive way to do math if you ask the LLM to do math for you. We took a bunch of rocks and we turned them into magical calculators.

>> It's easier for me, but ChatGPT's the one has to lose it or .

Randall Hunt

>> Exactly.

>> Okay. So what do I do if I want to build systems? There's a lot of people we and I were just talking off camera. A lot of people are rethinking their business model, their business architecture, their financial architecture, their technical architecture. The entire business landscape is looking at a complete kind of changeover of how they execute. And it's not just an IT thing, it's everything. So what is the best technical approach?

Randall Hunt

>> The real moat that people have is a deep understanding of their in-customers and their in-customer's not just current access patterns, but their desired access patterns. So let's imagine you're an airline. And as an airline, you're responsible for selling seats through this website. People come to the website and do all of this stuff. If Delta or United or someone like that were to go and expose an MCP server, then Claude or ChatGPT, that assistant could go and book the flights on your behalf. So you literally have an assistant acting on behalf of the consumer to go and do these things. So I really think businesses should think about how these massive armies of virtual agents and assistants will be doing things for their end consumers. And then in the B2B world, you have to think about what processes can we accelerate? What things did we look at in the past? Lots of legacy software, I always say legacy software, it is not legacy if it's still working. But if it's become parasitic instead of symbiotic, then it's legacy.

>> Parasitic to what, the database or what?

Randall Hunt

>> Parasitic to your operating model. You have to employ specialized expertise or it's slow. You have to pay egregious licensing fees to Microsoft or Oracle. If you need to escape those parasitic relationships, you might've looked at that in the past and said, "Oh, this is going to cost too much to do the migration, so let's not even consider it." But with generative AI today, it is much more possible to take on those migrations, things that you might not have even considered before.

>> I mean, I love the super intelligence conversation. I love the fact that we had super cloud early and turned it to super computing. And then you're starting to hear super clusters, and I was talking to the Lambda guys about what they're doing with their spare cycles on GPUs. They're actually anonymizing the data to create a neural brain so that software could be written on the fly generatively. So if someone needs a Salesforce application.

Randall Hunt

>> That's an interesting one.

>> There it is. So you're starting to see these new paradigms emerging where it's like everything's prompted.

Randall Hunt

>> If the marginal cost of incremental software development goes closer and asymptotically towards zero, what happens to SaaS companies at that point? How do they stay relevant? Klarna, I'm sure you saw they ditched Salesforce and they built their own CRM. So is that the trend? I actually think SaaS has a bit of a moat. And like I said before, the moat is really understanding your end customer's access patterns, and then you do checks, right? The next step after that is you say, "Let me do a vibe check. Can the LLM actually solve this? Can the LLM do the thing I'm asking the customer to do or I'm asking the agent to do?" And then from there you create evals. And you remember Steve Ballmer when he went on stage and he was like, "Developers, developers, developers, developers." If I could do the same thing, but shout, "Evals, evals, evals," that would be my mantra today is evals are what? Take your experiment with an LLM into robust production capabilities.

>> Give an example, a working example in your mind's eye or a real example of how an eval would play out. And let's just say we were riffing on a new product idea. What would we do? Take us through what an eval would be.

Randall Hunt

>> So we were working with a customer on video understanding they wanted to build video semantic search. So they had millions and millions of hours of footage, and you have a couple of different ways of understanding that footage. And we fed this into the Amazon Nova models and we're able to process these petabytes of image data using Bedrock Batch. And we got the results back. And before we went and did the full petabyte scale load, we did a bunch of analyses. And so the first check is we go to Nova and we pass in one video and we say, "Tell me what's happening in this video and give me timestamps." And it gives you a decent-ish example, but then you have to build robustness. So how do you build robustness in that process is you give it adversarial examples. Your eval set becomes every kind of different video. You try to introduce additional context and you give it examples of what you want the outcome to look like. And when we did that, when we enriched that prompt and used our eval set to really measure how accurate the description and the understanding of the video was, we got much, much better results. And it also allowed us to move faster. So we can now change the prompt, run the eval set, and we would see what was passing, what wasn't passing. So it's really kind of like a CI, CD system for LLMs.

>> And I saw Matt Garman announce the curtain raiser for this event this week, Kiro, which is their can write codes, agentic hooks into it. One of the things I've been playing with on the vibe coding, I've been playing with mainly Replit and Lovable because they're more the consumer apps. I did like three or four apps that I've been trying to build with my team. I haven't had the time to write a doc. I just prompt it out. All the hooks were broken, there's no search. I mean, I probably could have gone done, get in the weeds. But with Kiro and some of the things you're talking about, there's an abstraction around prototyping, evaluating, and then actually getting into production, the hooks, APIs, the calls. What are some of the challenges? I mean, first of all, what's your take on Kiro?

Randall Hunt

>> So Kiro, we were in the beta for it, so we've been using it for months.

>> Kiro or Kiro?

Randall Hunt

>> I think it's Kiro.

>> Kiro, Kiro.

Randall Hunt

>> I'm not sure,

>> Kiro. Whatever.

Randall Hunt

>> We were in the beta for it. We've been using it for several months. And I'm really impressed with the pace of development. And we have a ton of developers at Caylent that use it, and I strongly believe it will accelerate our ability to deliver. And one of the things that sets it apart from other IDEs like Cursor or Windsurf for Codium or whatever it's called these days, is a spec-driven development model. So you have a vibe coding mode where you're just kind of off in the weeds asking it to do whatever, but you also have a spec-driven mode. And the spec-driven mode goes through a set of requirements, collecting requirements, you edit them, you click through them, then it goes into design, and then you say, "Okay, I want you to design this part. I want to change this part. I want to create an architecture diagram here and it'll use mermaid diagrams and everything." And then it goes into the implementation and it can form off subtasks and subagents to go and do that implementation. And something that I've been saying just over the last few days is the people who are going to be successful in a world of LLMs are the ones who have agency. The people who think from first principles, think about systems design and have agency. And I believe that Kiro is the agent for people with agency. So people who think systems thinkers are going to be able to use Kiro better than any other IDE. Now, one of the things that we've been doing in our pre-sales process, so when we are pitching to customers, we'll use Kiro or we'll use v0 and we'll live on that call while we're chatting with the customer. We will build demos with them. It's a great pre-sales tool. And then we show the customer and they're like, "Oh, this is Figma." No, no, this is code. This is live here.

>> This is real code.

Randall Hunt

>> This is deployed. You can go and click it. And to your point about the integration, I think as MCP becomes more prevalent, the integration only gets easier. There's certain things with Auth and Stripe and Checkout and payments where there'll still be manual processes involved. And if you're launching an app, there's Apple app store review. But overall, I just see everything accelerating and getting easier and easier.

>> And when you look at these projects with customers, one of the things that young developers are coming in, by the way, these tools are bringing in better demographics. You're bringing in people who have never coded before, who are exceptional problem solvers, creative, structured. So you start to see what was once an entry-level product management position or product marketing position, full blown end-end prototype to production.

Randall Hunt

>> And that's what I mean is the people who have agency are the ones who are going to be successful. You can just do things.

>> So you mentioned spec. I want to get back to this. I think this is where the rubber meets the road or the meat and the bone I should say. Because if a product manager's job is going to shrink because there's tool for them, what's the role of a team of who does what? What's the agency's priorities? Is it more front-end work? Is it database? Take me through your view on how the tech's going to impact the role of the product manager who used to be the CEO of the product. Now they're the engineer, and it reminds me of DevOps, the DevOps engineer's running IT. So-

Randall Hunt

>> One engineer for everything, no dev team, you're by yourself.

>> Solo product manager.

Randall Hunt

>> I think there's some credence to that view. But one of the things that we still see, these are auto-aggressive models, and so they're just echoing tokens that exist in their space, and there's amazing emergent behavior and effects that happen with that. But it will get stuck in different local minima and local maxima. And to overcome that minima, you need to be a decent enough engineer to understand what has been produced. And we see this all the time. We have something that we've built called Caylent Accelerate, and this is a platform that started out primarily for heterogeneous database migrations back in 2022, even before ChatGPT came out, we were leveraging AI to translate from SQL Server into Postgres stored procedures. And we realized over time as ChatGPT and other models got better and better and better, that there was an opportunity to begin to look at the entire migration, every component of it from a systems level view. And so Caylent Accelerate now, I mean, we move three times faster on heterogeneous database migrations. It's 70% of the cost. So migrations that were previously-

>> What kind of speed faster? Give me a benchmark on that roughly.

Randall Hunt

>> So we had a project that we had scoped in '22, I think at 48 weeks, and we were able to deliver it in 10 weeks. So it was a massive time saving. And I think what people don't realize though is that there is a ceiling on how fast you can go. Because even if the computer can move at light speed and do everything and all the agents are working all the time to do everything, there's people process change, there's behavioral change, there's new things and humans only move so quickly. But we launched Caylent Accelerate, and you might see some ads this week around New York Summit for it. We got some billboards and stuff, it should be fun. But we launched that because it's not just about databases anymore. Now it's about, okay, it's a DevOps modernization, it's an app code. How are we going to build faster? How are we going to migrate from leveraging AWS Transform? How are we going to migrate from these vSphere data centers and to AWS? And how are we going to move legacy PHP applications or ancient Java applications into more modern-

>> While you're on Caylent Accelerate, put more of a plugins. I would like to get this out there. What's the modern version of it? You said it started pre ChatGPT. What are the upgrades? What are the features now, and why are people using it? Who's using it?

Randall Hunt

>> A huge portion of our customers at this point are using it.

>> The technical users, business users, what's the-

Randall Hunt

>> Well, it's a platform for us to deliver faster. So our customers come to us and they're like, "Oh, I could never afford to get off of Oracle because it's going to take more time and effort than the cost of just paying the license fee." But that's no longer the case because we can take the entire migration, the entire spec and go much faster. So it's a platform for us to be able to get our customers to modern architecture.

>> Do they have that moment where it's like BS, they call you out on that?

Randall Hunt

>> They do.

>> Take me through that. What is the... I mean, this has been a struggle. The switching costs and the buy versus build has been... The bar has been so high, the cost, the pain, the suffering.

Randall Hunt

>> That's why I say there's a ceiling. I try not to oversell it. So we can move as quickly as possible, but the people in the organization still have to be bought into it. And sometimes organizational change has to happen too. And those are things that require a little bit of time. And so we work with our customers to understand, "Hey, you're going to move off of Oracle Data Lake onto say Redshift for this and these ETL jobs that you've had running for years on one server in your data center, we're going to move those to Glue or we're going to move those to Airflow or something like that." And that requires a knowledge transfer and an enrichment. So you have to pick up new skills. And again, I am going to keep coming back to this, that's where people with agency excel, people who are curious by nature and people who can break things down into systems.

>> It's like the difference between someone who jumps out of a plane with a parachute.

Randall Hunt

>> Yes.

>> You don't just do it.

Randall Hunt

>> Yes.

>> You sit in the classroom.

Randall Hunt

>> You study school certification.

>> You make sure it opens. If not, I mean, but there's a lot of, I mean, I'm joking aside, that's like a good metaphor because people are afraid that the parachute won't open. That's their job or their business, and you have to manage that. So it's a process and that the platform does that.

Randall Hunt

>> Yes. So the proof is in the results. So we have a bunch of customers that we walk people through. So we've got a bunch of case studies on our website, but essentially those customers come to us and say-

>> We'll take out Caylent Accelerate. We have Valerie on the AWS summit in DC. A lot of public, private action. You guys are doing very well. So we will keep an eye on that. You mentioned databases. You go back when we started theCUBE 16 years ago, the holy trinity of technology was storage, compute, networking. Now you throw databases in there. Now you have storage, compute, networking and databases, data feeds AI, that is now impactful because the data is where it resides. It's the bottom turtle as my land calls it, in that metaphor, in that example. What is the impact of NVIDIA and some of these large super computing environments that are emerging? Because you mentioned tokens earlier, it's you want to use those abstractions in the LLM to multiple models. Clearly there's a new wave coming and there's no denying it here on the board. It's all green. Tesla and NVIDIA's options are up, market's on fire, the China news is out. NVIDIA's impact and networking's impact. What are some of the underlying infrastructure goodness happening? We saw this with the cloud when Amazon was misunderstood early days, people who were in the know are like, "This is game changer. It's no-brainer." It was so obvious.

Randall Hunt

>> I remember when Lambda came out in 2014, I was behind stage at the keynote and I was watching all of the tech press kind of be like, "Oh, what is this thing?" And then I could see the people who were really in the know were like, "This is a game changer." And then I remember in 2015, one year later, no one was asking, what is AWS Lambda? Everyone was like, "Oh, how can I use more of it?" And I think we're in a similar era where certain advances are sleeper hits. They happen slowly and then all at once because you see the impact that they have for the early adopters and NVIDIA's play from the infrastructure standpoint. I mean we've got the P6e in Blackwell instances available in AWS now. If any customers have one of those, please can I just want to see all the RAM run top. So I think the infrastructure play is very interesting. The weird part is the frontier models have become so large, they don't really fit on a single piece of hardware. It requires networked hardware or multiple servers or multiple GPUs. And you can quantize them down. You can do all kinds of tricks to make it fit. But CUDA has this, NVIDIA's hardware accelerated stack, has some advantages in that it's widely adopted. But I think there's a lot of interesting stuff happening on the AWS side with Inferentia and Trainium. So that's Amazon's hardware accelerated stack and Anthropic is building on top of that. And I think the Neuron SDK and the Neuron Kernel Interface, those have drastically improved since two years ago. So I would see Inferentia and Trainium are starting to become a viable platform for you to build real world inference on. And it's all about what is your scale and what is your margin. So everyone wants the highest quality tokens at the lowest possible price as quickly as possible. So which hardware platform gives you that and what's the pro and con, the price performance benefit?

>> There was a thread that went out on LinkedIn from Adrian last week around NVIDIA's NVLink and Amazon not having more NVIDIA. He was kind of critical of AWS on that. He's doing some experiments. He's saying things are changing so fast. Is Amazon enough embedded with NVIDIA? Is NVIDIA a threat? What is your take? What Adrian was basically saying, he said one of the comments, "Hey, they're just better at GPUs than NVIDIA." Obviously they make GPUs, but Amazon's got their own custom syllabus. What's your take on that? Is it tempest in a teapot? Is it-

Randall Hunt

>> CUDA is a strong moat. I don't think NVIDIA's going away anytime soon. But if you're thinking about who's going to win, it's the vertical integrators, the people who own the hardware and the software and the platform for selling the hardware to other people, that's going to be much more successful in the long run than just selling the hardware. That's why NVIDIA's building DGX Cloud and all these other things, they realize this, Jensen's a smart person, so they're not going away anytime soon. I do think it's interesting that Amazon has purchased fewer GPUs overall. So if you look at the market share of AWS and how big that cloud is compared to the other clouds, you would think they would be buying more of the GPUs. But actually more of the GPUs are going towards the other hyperscalers. And I think that is Amazon essentially through their purchasing saying, "Okay, well, maybe we do need to bet big on our own hardware and it's always going to be a balance." And I think at the end of the day, Jassy and Garman, I think they realize they need to be the platform for everyone. But you can't... If you're making all of those bets, something has to give. You can't just blindly go and buy GPUs. It's not frugal. It's not a wise decision. And then I think a lot of these GPUs that others are buying, the Oracle's and the Microsoft's of the world, I think a lot of them are sitting idle. Honestly, I don't think they're being utilized.

>> As you look at your business is growing, you guys made great bets. You mentioned MCP earlier. Integration and engineering are kind of hand in glove. What's the view of agents? That's the hype. And it's still early. I mean, we started to see agents take tasks. You mentioned evaluation Databricks, I talked with Jonathan Frankel at Databricks who kicked off the Mosaic project over there, and just interviewed in Paris last week. They're all agreeing with evaluate, make sure that they can finish their tasks, they're orchestrated properly. How should agents be thought through in terms of setting the table? Who are bringing that in?

Randall Hunt

>> Right now, there's an issue of compounding error that exists with an agent. So right now, the kind of frontier best models every 10 minutes require human intervention. So if it's executing for 10 minutes, it requires human intervention at least once. And if you don't, and it just keeps executing on its own, the error compounds over time so that after an hour you're 60% likely to have had an error or however that math maths. But that frontier, that 10 minute frontier keeps pushing out. So the amount of time that an agent can autonomously execute keeps getting larger and larger. So we're probably a year or less away from an agent that can autonomously execute by itself without error for 24 hours. And when you have that, what business model changes? What are the principles that change when you have that happening? And one of the things that we've done is we've built with that Caylent Accelerate platform that I was talking about, those are all agents, but they're executing with human intervention. So there's human oversight. I think right now you need to design the platform for your business to have that human oversight in, but make it flexible enough that as the agents get better and better, as the autonomous execution time extends, the human oversight is coming in at the key moments. So how do you identify those key moments and going all the way back to Kiro, I think they did a good job with the spec, the design, the implementation. That was a good way of having-

>> Engineers need specs. Product managers used to do that. That's their job.

Randall Hunt

>> But I think agents... I would expect the Amazon and New York Summit to go and announce a lot of stuff around agents this year. And I just want to say this, agents to me, it's a little frustrating because it's the same thing we've been doing the whole time. It's using tools in a loop with inference. And I get that 2025 is the year of agents and everything, but we've been doing this now for so long.

>> It's software in the cloud. Randall, great to have you on. Quick plug, second half of the year, what's going into re:Invent? What's on your agenda? What do you have looking forward to this second half of the year?

Randall Hunt

>> I think Caylent, we have a very exciting second half of the year. I think people will have to stay tuned to see what happens there, but we are really leaning into this and building as much as we can along the Caylent Accelerate platform. And then the other side of it with re:Invent and all of this stuff happening there, I think this is the year we see the evolution from just prompt engineering into context engineering and context optimization. And we start to see how agents operate with the best possible context available.

>> All right, Randall Hunt, CTO here in theCUBE, breaking down all the action in the industry. Of course, his company's growing really, really fast. Of course, it's the mid-year report. So much has happened. We're going to continue the second half of year highly accelerated pace. Pace of play is faster than ever. Of course, theCUBE's doing its best to bring it to you. Thanks for watching.