Cloud AWS re:Invent Coverage | Randall Hunt, Caylent

Clips
News
More from Cloud AWS re:Invent Coverage

Randall Hunt

CTO

Caylent

The AWS impact: How Caylent is empowering businesses to innovate and scale in the cloud

Given that Amazon Web Services Inc. is pioneering cloud infrastructure, the AWS impact lies not just in technology but in changing the way businesses ...

play_circle_outline Introduction to theCUBE's coverage at AWS re:Invent 2024 with Randall Hunt, CTO of Caylent

play_circle_outline Discussion on the evolution of AWS services and infrastructure

play_circle_outline Maximizing AWS Impact: How Caylent's Premier Consulting Partnership Drives Success

play_circle_outline Innovative Collaboration with AWS for Testing and Feedback, Introduction to Nova Models for Video Understanding

play_circle_outline Amazon's focus on practical AI solutions to silence critics

Info
Transcript

Randall Hunt, Caylent

Randall Hunt

CTO Caylent

The interview with Randall Hunt, CTO of Caylent, focused on the evolution of AWS and the advancements in technology. He discussed the history of AWS, the concept of blast radius, and the continuous innovation happening at AWS. The conversation touched on the benefits of using AWS services, the integration of new technologies like generative AI, and the importance of staying ahead of customer needs. Randall also shared insights into the role of CFOs in understanding the cost structures around generative AI. The conversation highlighted the excitement and enthu... Read more

explore Keep Exploring

What event is the text referring to in Las Vegas in 2024? add

What advancements in technology and investments are being made in the field of physics, particularly in relation to undersea cables and silicon investments? add

What types of services do customers typically come to Caylent for help with in relation to AWS? add

What was the experience like partnering with AWS before the release of new services? add

What were Andy's gestures and statements about AI during the conversation? add

bolt Powered by CUBE AI

Randall Hunt, Caylent

search

>> Welcome back, everyone, to theCUBE's coverage here in Las Vegas for AWS re:Invent 2024, 12th year with theCUBE covering wall-to-wall coverage. I'm John Furrier, host of theCUBE. And the past year, we started phasing in theCUBE Collective. It's a unique, open community designed to elevate content creators, innovators, people who contribute to our mission of open source trusted networks. And our next guest is Randall Hunt, who is the CTO of Caylent, also former Amazonian. in many things in his career, certainly knows Amazon well, Amazon Web Services. Your business works closely with them now. CTO growth has been phenomenal. You're going to be on the partner keynote tomorrow. Great to see you.

Randall Hunt

>> Good to see you.

>> You had a big party last night. Congratulations. Just everything's going great for you and your team. Congratulations.

Randall Hunt

>> Yeah, it's great.

>> Thanks for being part of our community.

Randall Hunt

>> Well, thanks for having me. It's easy to build a good company when you got a good infrastructure platform to build it on, so we love playing around with all the new services. This is like developer Christmas.

>> Really, it's a party up and down the stack. It's a party in tech time. I was talking to Dave on our podcast. It's party time in tech. It's like I've never seen it this cool in all areas. Every aspect of the stack, no matter where you peel the onion or slice the salami, there's action. Semi's, similar language being talked about on theCube for the first time in decade anyone's ever talked about assemblers, and registers, and core dumps.

Randall Hunt

>> MIPS.

>> Calling down to the kernel level, you're starting to see democratization actually happening at levels that we've never seen before, like non-technical users coding. You're starting to see a lot more action in the data layer. Just I feel like we're in this gen two cloud next level, legit next level stuff that was standing on the shoulders of the cloud one, which was building blocks and seeing the news around inference as a building block. For the folks who don't know the inside AWS, that means something when they say building block versus a service. You start to see things connecting like serverless, what would that be? That became a ten-year success story. Now, you see things like inference and database. We're like, "Databases, servers, inference, building blocks, what's there?" Maybe they'll abstract away and manage more database to work together. You don't even have to deal with database. It's like what serverless did, wink, wink. I can see the dots connecting. You're starting to see the new architecture. What's your view on this, Randall, you've been close to the action. You've seen the evolution. Am I getting it right? Is that happening and are we starting to see this progression?

Randall Hunt

>> Well, I think you can even go back to Peter's keynote last night. Yeah, that was last night.

>> That was last night.

Randall Hunt

>> He talked a lot about the history of AWS. He talked about Barge, which was this 7,200 RPM disk, 260 drives or something like that, that they shoved in one server. It was like two tons. They had vibration control and all this stuff, and they talked about moving away from those spinning disks. The reason they did that was because of blast radius, and I think it was James Hamilton all the way back in 2016. He's the one who said, "Listen, we don't build data centers bigger than a hundred megawatts because of the blast radius." AWS has this cumulative knowledge and experience of running these operations at scale, and that gives them the ability to rapidly innovate based on that experience, when you look at every layer of the stack.

>> Explain blast radius, what that means? Someone might not know what it means.

Randall Hunt

>> It's like everything at scale. Everything fails all the time. You want to limit the scope of that failure. You don't want it to go-

>> AKA the blast radius of the fail at scale.

Randall Hunt

>> Yeah. When they had that barge server, that's 260 disks that go down all at once. Now, they don't have that same level of impact to multiple customers really. And they just acquired all this cumulative knowledge and experience in running these data centers that then allowed them to build on virtually everything from fiber optic. There's an applied photonics lab in Sunnyvale, I think where they're people wearing glasses and shining lasers down hollow fiber optic cables and they're really pushing the boundaries of physics. They invest in undersea cables, Trans-Pacific, Trans-Atlantic cables, the Haikiki cable, all of this stuff at one layer of the stack, the core layer of the stack was fascinating to hear about, the silicon investment with Annapurna and all of the stuff that Gaudi and the rest of the team are doing over there, phenomenal work. The keynote last night, they removed the inner plane and they showed the HPM for Tranium 2 and the giant Ultra Cluster, all this good stuff. I think the successive layers of innovation are continuing to compound for AWS. It's compounding interest. They just keep improving and I think it's going to be very difficult or very capital intensive for someone else to catch up at this point.

>> Yeah, it's like a barrier to entry. They go to business school, they teach you things like competitive strategy and one of the most basic principles is create a barrier to entry so other entrants don't come in and take some of those economics that your margin is my opportunity as Bezos is quote famously saying, Jesse's like I don't even think Jeff ever said that, but it does sound good. Scale is a competitive advantage and scale, now you're starting to see scale render itself in this compound way you're mentioning to customers. I had JP Morgan Chase on the CIO, Lori Beer earlier. She said, "We are critical infrastructure." They're moving 10 trillion a day. They have a $17 billion budget and they're operating at such a scale and she wants to teach people and she's openly about it and great. CUBE's open for business. Let's get going, more segment. Sure, we'll do that. But that's a customer running a scale.

Randall Hunt

>> And it's critical infrastructure. One of the things we're going to talk about in the keynote tomorrow is a customer of ours Live 360, they dispatch ambulances or emergency services. I think they dispatched 42,000 ambulances last year and that can't go down. People's lives depend on it. And that's another part of that core infrastructure that we were just talking about is they've expanded Route 53 application recovery controller into EKS now. So you have cellular architectures that you can build on top of Kubernetes, on top of API gateway. This again is the blast radius argument, but also, it's enabling it not just at the core infrastructure layer, but also in successive software layers on top.

>> It's interesting. I had an exclusive opportunity to meet with some of the leaders of AWS prior to this event. Bat Garmin's one. I posted that as the preview. Matt just published this morning. I had to wait because he did talk about some renews in there. Dave Brown, he's an OG in the EC2 group. He now runs Compute, which took over from Garmin, and I asked him directly, I'm like, "We're in era now, build it or buy it, and you're at scale." So there's a big hubbub around, "Hey, I'm going to get a big server, the God Cluster, I'm going to have it on prem." And my premise has been, if you don't connect to the cloud, then it's just a standalone island. And then I asked him, "Well, why can't I just get a bunch of Nvidia chips and put it together assuming I have power and everything?" And he says, "It's really hard and things fail," to your point. And he's just saying, "We just get really good at supply chain and we're getting really good at and making silicon. We can do six months before tape out, all simulation is done with digital twins." I'm like, "What?" Again, to your point, at what level does the on prem build your own not compare to either connecting to the cloud or going all cloud? Because Garmin is very confident and he's got a spring in his step around, hey, people are coming to the cloud.

Randall Hunt

>> This is a common kind of discussion that I think is important to have. You see people like DHH on Twitter and talking about how you should never use cloud and things like that. The thing is, if you have competent operators and you have the capital necessary to buy and operate these servers and build the connectivity, if you can sustain a level of reliability that's sufficient for your customers, it will absolutely be cheaper to go on prem. That's just a reality. But the downside is you're now paying a DevOps tax, you're paying a management tax. And so at some point you're doing the power tuning graph of how much does it make me doing on prem and running it myself versus running it in the cloud. And then the second side of it is the evolution of things that can truly only be built inside of the cloud because of the way the availability zones are set up, the way the regional models set up. Things like Aurora, you can't build that. You have to have AWS's infrastructure to be able to build something like Aurora.

>> Randall, one of the things that jumps in my head when you're talking is I a conversation at one re-Invent replay party. I was in the VIP area, and I see Adrian Cockroft there and I go, "Adrian, Hey, what are you doing here?" I work for Amazon. Now, this is years ago, I was being myself. But Adrian was instrumental at Netflix and a big customer of AWS, and he's very curious about theCUBE, so of course he's got very studious. He's asking me probing questions while we're having a cocktail. But I'll never forget because it was a moment where we bonded. But he asked a question that's very Amazonian, very Netflix-like, I don't know which culture. He said, "What are you optimizing for?" Because I was giving the whole vision of what we're trying to do. This is years ago. And he asked that very pointed question, "What are you optimizing for?" To your point, if I'm on-prem, what's my optimization? At some point, if you get over your skis on ops, you're optimizing for maintenance, not growth. And the DHH argument I've always said on repatriation is what are you optimizing for, growth or cost savings? It's a mindset. So if you're an entrepreneur and you're in a growing market, you have to optimize for growth. If you're in a mature market, I could get the optimization for cost effectiveness because there may not be growth, but then you might miss new markets.

Randall Hunt

>> There's the elasticity argument as well. Scaling up but also scaling down, people forget about that. The ability to scale down is always very important. And then the other component is you're not limited to not using on-prem if you use AWS. AWS could be your burst capacity. But Nathan Taber who runs EKS, he just launched I think on Sunday, or maybe it was Friday, it's hard to remember. He launched this new EKS auto mode that also has support for local zones, for on-prem for all of this other stuff. And you could previously do that with EKS, but the management layer was very different when you went to your on-prem notes or your self-hosted notes. Now it's the same. So it's your same Kubernetes. You can go multi-region. You can go on-prem and essentially AWS becomes the cloud operating system.

>> Talk about your role. I know you had the successful career at AWS. Feel free to share some of the projects you want, but you're obviously building with AWS. You're helping people with your knowledge of the secret tunnels and all the nuances of AWS because there is this value there. And then if you look at Amazon now and what are some of the key things that are happening? Two questions, give a little discussion about what you do as contacts and then talk about what Amazon's web sources opportunity is with respect to what's emerging now that's an opportunity for them and their customers because as you hit this inflection point where there's a disruptive enablement happening, there's disruption and there's enablement, which is a great dynamic. You want to see that. You don't want to actually see disruption and then that bad things happen, but it's enabling something that means change is afoot. What opportunities are emerging on the stack on Amazon that weren't possible prior to a few years ago or are now possible with Gen.AI? When I was talking to Verner prior a couple weeks ago, we were talking about the 10th anniversary of Lambda, which was essentially serverless, which came out of a discussion within AWS. You probably were hanging around that barbershop while the people were talking about that.

Randall Hunt

>> Tim Wagner and AJ Naron. Yeah.

>> And so there was no blueprint. No one knew what it could be. You knew it was the right thing to do because why not do it? Again, the serverless dynamic was one of those things where you knew it was going to run and be great, but what level? And it just happened and it just grew, a thousand flowers bloomed, so to speak. I feel like we're in that moment now with all this data and Gen.AI, what are you seeing for projects? How's that translating to what Amazon should be doing or could be doing or will be doing?

Randall Hunt

>> I'll explain Caylent first and then I'll jump into the second part of the question. My role, I'm chief technology officer at Caylent. We're all in on AWS premier tier consulting partner. We help people turn ideas into impact faster. So customers come to us, they want to do heterogeneous database migrations from SQL server to Postgres. They want to move from on-prem into AWS. They want to do app dev. They want to do virtually anything you could think of with regards to AWS, maybe not work docs. We do it and we build on AWS primitives as much as we can because the-

>> The glass radius is good.

Randall Hunt

>> Yeah. It just makes building really resilient technology pretty easy. And the thing that I am trying to focus on right now is staying ahead of where our customers are. Our customers, we're very lucky. We have a lot of really fantastic customers who are leaning into technology. Customers like Brain Box, and I think-

>> They were featured. Yeah.

Randall Hunt

>> Yeah. You and I spoke with them.

>> We spoke with them and they were featured in my story and they would mention it the keynote.

Randall Hunt

>> So they lean in and they're really interested in new technology. But we also have customers who technology is not their core competency. They want to focus on their product. And so we have customers like Life 360 where they want to build the best app they can, but they don't want to worry or focus on running this underlying infrastructure and undifferentiated heavy lifting as Verner would say. And so we or Jassy we try to do that for them. Caylent, we've had some really phenomenal growth. We won three partner of the year awards this year. We were the global migration partner. And my job is really just staying ahead of where even our fastest moving customers are and figuring out how we can distill that into actionable things. So all the new releases, developer Christmas this week. We got 56 announcements just on Sunday, so how many are we going to get for the rest of the week?

>> You're going to open those presents up, play with them.

Randall Hunt

>> Exactly.

>> And then find the best toys that work for you.

Randall Hunt

>> And one of the things that has been great about partnering with AWS is that we've been able to lean in before the services are released so that we can provide actionable concrete feedback right away to our customers when the new service comes out. Like the Nova models, we collaborated with AWS very tightly on testing those.

>> So you knew that was coming?

Randall Hunt

>> Yes.

>> Okay. Hiding the ball from theCube team. Everybody was surprised on that one.

Randall Hunt

>> I think there were the Amazon snipers hanging out watching anything I would say on the media.

>> We're going to take you off our masthead right now. Come on, you're holding back on us. Even internally, they held though. They didn't even brief the analyst under NDA. That was a very tight announcement.

Randall Hunt

>> Well, the most important part from that model in my opinion, is the video understanding component. We've done a lot of early testing with it and we have a lot of customers for everything from healthcare to security cameras to wildlife cameras to generative search over video. Those are all things that with the cost-effectiveness of those models, to your point earlier about what can we do now that was just untenable before? That's one of those things to ingest and create semantic search across all of that video in the past would've been heinously expensive, and it would've been inefficient and it wouldn't have been good. Now we have that capability, we have that possibility. Another thing that's happening in the generative AI side, and something that we developed was I call it the SQL polyglot, but our marketing team has a different name for it. And I forgot what it is. I'm sorry. It's like AI Evolution Database something. It's basically, we are able to take stored procedures that are in legacy formats, Oracle, SQL server, and move those into not just stored procedures within Postgres, but also extract out that functionality and move it into code if necessary. And that gives us a lot of flexibility.

>> It's like previous prompts, essentially. It's like previous prompts.

Randall Hunt

>> Yeah. And we can move very, very quickly. So the problem was these migrations, they used to be two to five year undertakings and you would grab a Accenture or a Deloitte or something and they'd spend the first year admiring the problem and then they'd spend the second year maybe doing the work. What we're able to do now is go in and in six weeks accomplish the work that would've previously taken two years. So things that were financially unapproachable before, not even within the realm of possibility are suddenly on the table. These are conversation topics that you can have with your CFO and say, "Hey, I can get us off this in six weeks. Can we do it?" "Yes."

>> That's a great point. I want to come back to the CFO piece because we're doing an AI CFO summit, first ever, in Silicon Valley on December 17th. It's going to be in our studio film day. We'll stream it out the next week. But it's not about finance, it's about impact of technology, business model, transformation, risk management. The CFO is being pulled into more conversations. What areas are you seeing the CFO conversation where they have to level up and learn new things? Just simple things like I've heard from some of our analysts where you have cyber insurance, cybersecurity insurance. They don't even do differentiation around the value of the data. And so new things are happening where I got data, it's not worth anything. If it gets hacked, who cares? Why should I even file something? You're starting to get into now much more deeper levels of granularity on things that are important to the business. So the CFOs are now involved and they're like, "What? What's AI?" Again-

Randall Hunt

>> I think the biggest challenge CFOs face right now is understanding the cost structure around generative AI. And I think previously we've spoken about the idea of tokenomics. What is the unit economic cost of operating generative AI at scale? And we have customers in virtually every vertical, but there's one customer that comes to mind that had say a particularly challenging CFO who wanted to know the precise cost of operating this generative AI system that we had built for them and we gave them a number that was modeled on a blended token rate and things like that. And the reality is that every four months since November of 2022, the price performance of these models has doubled or more with the slight exception of 3, 5 Haiku, the new one that Anthropic just came out with, which is actually 4X more expensive. Everyone is always going to want the highest quality tokens as quickly as possible at the lowest possible cost. And so the rate of each generative inference that you're thinking of right now, it's only going to get lower. It's not going to go up. So I think Donald Knuth's quote is premature optimization is the root of all evil. Some people may be doing a little too much financial engineering and over optimization on the cost right now, not realizing that the models are only going to be cheaper.

>> Because the price performance has changed. Not just price, not commodity.

Randall Hunt

>> Yes.

>> Ride the growth wave on performance and the cost lower. That's the whole premise.

Randall Hunt

>> Exactly.

>> Well, what's your takeaway from re-Invent so far? I know you've got some more work to do here with the events. Looking down as an independent Cube Collective member, how would you sum up this re-Invent so far? Obviously, it is like all the presents are hitting the table, so got that. What's the walk away?

Randall Hunt

>> Peter's keynote was an hour and a half and there was only one launch. I was not bored the entire time. I was actually enthralled. I thought it was a fantastic keynote. I thought he did a really good job. I really appreciated them diving deep on the infrastructure. This was Matt Garmin's first keynote and I thought he did very well. It's the kind of energy that I felt back in 2014. I am energized, I'm inspired, I'm excited. I want to go and play with all the new stuff that's out. I'm excited to see Swami's keynote. I'm excited to go to the partner keynote. And then we have Verner on Thursday and Verner's more of a narrative storyteller, so I expect there to be some more stories, fewer announcements in that one. And the takeaway that I have is that generative AI isn't going anywhere. It's still very early in the hype cycle. We still have a lot more to build there, but Amazon is not sleeping. AWS is not sleeping on the core infrastructure. S-III is still releasing innovations. The metadata created the tables.

>> Back to the basics.

Randall Hunt

>> I love it. I love it. Give me more of it. Let's just build on this cool stuff.

>> I like to look at between the lines because I like to study all the movements, but Matt did a great job. I'd say first keynote, he's not at the A level. Jassy obviously is pro, he's been there, but he's pretty close. So he hit the fire hose of he didn't miss a mark. But what got my attention right out of the gate was first sequence, heroes community.

Randall Hunt

>> Yes.

>> That was huge. That's the tell right there. No one picked that up. I didn't see anyone tweeting much about that. Everyone went gaga over the Apple guy. "Apple's doing," okay, the real story is not Apple. But community led with community and the second sequence was developers.

Randall Hunt

>> Yes.

>> Close to my heart, he had that slide up. I love developers. So those two things set the tone out of the gate. And then he didn't talk about gen AI. came-

Randall Hunt

>> Towards the end.

>> Two hours in. Okay, he was on S-III. He hit the Nvidia thing on the P-six instance. But he's talking about S-III tabular format. He's talking about D-SQL. This is going to the strength.

Randall Hunt

>> I love it. And the best part really is I don't know how many people remember the XML APIs back in 2008 or 2009. It was always developer led. And the people who were building it back then, they always thought about what is this at scale? There was never a run instance API. It was always run instances, plural. So they always thought about spinning up multiple servers and they always thought about this massive scale and all of this stuff. And to see that sort of energy developer focused side of things come back in this keynote, I was a little worried because I wasn't sure how Matt Garmin was going to perform in the keynote and what sort of announcements there would be. I'm not worried anymore. I am going to go buy more Amazon stock.

>> I saw your tweet, it was awesome. I think he proven a lot of the skeptics, and I wrote about this when he got appointed. I'm not sure if people liked my little reference to him being a wartime conciliary, but he's a product guy and he's smart. He knows what he's doing. So no one's going to sling BS by him. He's definitely strong there. He's got Andy's trust. He understands the Amazonian culture, so I felt he would be good. And then when I interviewed him in August, it was very clear he got the scale piece and I was starting to see he had full command, had very good command. Then I met him before the event. It was clear he was rocking and rolling. Super busy. He did the work. He works hard. He's a hard worker, super smart. So I think he calmed a lot of the skeptics. And I liked Andy's set piece too. I thought that was very respectful for Matt.

Randall Hunt

>> Yes.

>> The way he introduced as an OG, the godfather of cloud. And then Andy came in and just delivered. I want to play. Of course, he took the big announcement that was under embargo. I'm the boss. I'll take the big announcement. And Matt certainly has probably no problem laying that down. But Andy's gestures to me were very Andy, Jassy-like in the sense of, "Hey, let me tell you about AI." And so after about a year and a half of auditing, I'm sure as he said, and then when he came out, we're going to redo this, you can see him flexing there saying, "Look it," we're proud and loud about the fact that we're not going to buy into the hype. And then he gave examples of, obviously from an Amazon because of course, that's why he's there. He sequenced quickly to AWS. But we've been doing AI and here's the practical. And I thought, I think that will silence a lot of the critics. And then just at the end of the day, I think they said, "Just watch the scoreboard." Revenue, customer satisfaction. So I think we're in a show me market right now. Show me the beef, give me the goods. That's what I want. And I like that strategy, I think. Okay, go back to your core. Dance with the ones who brung you, developers, partners, customers, and okay. Game on. Competitively, Garmin and Jassy, both competitive.

Randall Hunt

>> Oh, very much so.

>> Watch out.

Randall Hunt

>> I think it's going to be a great rest of Re-Invent. I still haven't been able to catch up on all the announcements just today. I think a counter is over 100 at this point, and I cannot wait for the rest of the show.

>> Well, thanks for coming on. I want to thank you for being a contributor. I know we haven't done much content. You've been super busy. We're going to change that. Congratulations on all the success you're having in your company and being part of theCube Collective. It's something new we're doing, but we're trying to make this open group just to share as much content open source as possible. Really appreciate your insights and inside knowledge of Amazon and what you're doing and sharing that with us. Appreciate you.

Randall Hunt

>> Absolutely. And maybe next time we can do some demos.

>> Okay, great. We'll do some more remotes too. Okay. I'm John Furrier. Here we are in Las Vegas. Wall-to-wall coverage, the entire SiliconANGLE media team is here. We got all the coverage from the SiliconANGLE.com writers are here, theCUBE is here. CUBE research team is in force. theCUBE Collective is in force bringing all coverage to you, all the data, our fire hose. Go to Siliconangle.com, check out all the coverage. Thanks for watching.