Cloud AWS re:Invent Coverage | Jason Warner, poolside

Clips
News
More from Cloud AWS re:Invent Coverage

Jason Warner

Co-Founder & CEO

poolside

AI’s domain-specific shift: Poolside CEO discusses AI-driven software development

The swift advancement of artificial intelligence is transforming industries, with a significant impact on AI-driven software development....

play_circle_outline Revolutionizing Software Development: Training Foundation Models for AI Advancements

play_circle_outline Partnership with AWS for native offering

play_circle_outline Security and privacy concerns for highly regulated environments

play_circle_outline Early stages and advancements in AI industry

play_circle_outline Distinguishing AI Players: Key Proof Points and Partnership Strategies for Growth and Value Creation

Info
Transcript

Jason Warner, poolside

Jason Warner

Co-Founder & CEO poolside

TheCUBE covers AWS re:Invent 2024 in Las Vegas, focusing on infrastructure and innovation in low levels. Featured are Matt Garman, Andy Jassy, and Jason Warner of poolside, a Frontier AI company focused on software development offering AI tools to large enterprises. They work closely with AWS and stress the importance of understanding the AI landscape. Poolside collaborates with partners like Salesforce to enhance their services. In a fast-changing world, embracing technological advancements like AI is crucial for growth and innovation in software development... Read more

explore Keep Exploring

What is the focus of Frontier AI and poolside in the domain of software development? add

What advantages does offering poolside as a first-party offering on AWS provide for customers, especially very large enterprises? add

What are the considerations and challenges in providing state-of-the-art AI to customers without requiring them to trust the provider with their data in highly regulated or air-gapped environments? add

What is the backstory behind the creation of GenAI and its relationship with GitHub and OpenAI? add

What are some of the proof points that separate the pretenders from the players in the field of AI? add

bolt Powered by CUBE AI

Jason Warner, poolside

search

>> Welcome back everyone to theCUBE's coverage here in Las Vegas for AWS re:Invent 2024. My name's John Furrier, host of theCUBE. It's our 12th re:Invent. 2013, I remember vividly when I first met James Hamilton, he came on, he explained the whole back end of AWS and then he never came on again because the PR analyst took him away. A moment in time that was very memorable. We're in a similar time now, but it's a little bit different. People want to know what's going on. The infrastructure's changing so fast and the innovation at the low levels of the infrastructure is so important. It's been the big theme of the show here at re:Invent, going back to the basics. Matt Garman, Andy Jassy even made an appearance and we got a great guest here, Jason Warner, CEO of poolside, mentioned in the keynote. Also, I wrote about it in my preview story. Jason, great to have you on and congratulations for all the great props here, but also the work you're doing.

Jason Warner

>> Oh, appreciate it. Thanks for having me.

>> So you guys got some very interesting things going on. I actually had a chance to sit down with Matt Garman, but also Dave Brown prior to the event. We went deep on Trainium2, some of the ultra servers and how the chips are just getting better and better and they're getting good at it. But a lot of the developers going down to the hardware level, the value is, it's like reminds me of the old days of when you do memory management in the '90s and squeeze as much out of the hardware. But the value now and the performance at the software level is huge. And with AI becoming so important for our company's assets, and we're hearing that here, CIO, Lori Beer was talking going to me yesterday about how much they have more exabytes, OpenAI training on a couple petabytes, not a couple of petabytes. JPMorgan Chase has more data than OpenAI did. So they have a challenge. Every company has to get AI primitives. This is something that you guys are close to doing, you're doing it now. Take us through what poolside's doing and then we'll get into some of the things you're doing here and why what you're doing is very relevant.

Jason Warner

>> Sure. poolside is Frontier AI, so think Anthropic, OpenAI, but we're focused on the domain of software. So building our own foundation models from scratch oriented towards software development and the umbrella term of software development, not just code creation or code assist, but the umbrella category of software development. And we have the full stack, so foundation models, middleware and applications as well. One of the key differences we install in customer's premise, so in VPC or air-gapped environments, and then we fine tune all on all the customer data and that never leaves their boundary. So whatever security boundary or controls they have in place, it never leaves. In essence, what we're doing is we're creating this new AI primitives inside their organizations that they can build around. And yes, our applications are powered by poolside, but other applications can be powered by poolside as well.

>> Yeah, I love that. Let's back up for a second. You're going a little fast for me. When you said foundation model for software, the category. Is that just knowing the languages? Knowing all the data structures? Take us through that. What are you actually developing around? What are you learning? What's that key asset? Take us through some of the specifics.

Jason Warner

>> Sure. So we have a couple different world views at poolside and they might be fun to talk about as well. But in essence what this boils down to, if the world knows about general purpose models essentially, data distributions, average data distributions over the training set, we have the same data, but we're orienting all of our training towards software development. What that means in practice is just that. As we continue our training, we're saying, "Hey, we want our model to be the best at software development." We happen to know about butterfly anatomy, but we're okay if it forgets things about butterfly anatomy, if you're preferential treating software assets.

>> So knowing the software development life cycle, the modern life cycle, knowing the process, something that you did at GitHub a lot, the warfare. It's not just code, it's everything around the mechanics. Am I getting that right?

Jason Warner

>> Yes. So this is one of the differences between something that might be a code assist versus software development. And if you net things out over a couple of years as well, this is what will happen. So right now we think of ourselves living in a code assist world from AI, but in reality we got to think about software development, life cycles, software development as an entire category, not just code completion or chat that's saying, "This is what this code does."

>> So you guys are very focused like laser on software?

Jason Warner

>> Laser focused on software. We believe also going back to worldviews. We have a proprietary technique which we call reinforcement learning via code execution feedback, which allows us to generate a mass amount of synthetic data to advance the domain, but we happen to also believe that we have a pathway to AGI, similar to OpenAI and Anthropic. We think we'll get there faster via this mechanism. Just so happens that software is the category that we want to serve.

>> It's like having the old simple days of had a compiler. Now, you've got all the synthetic codes, you've got this whole nother model of learning and getting accuracy and then kind of building. So take us through the impact of customer. Again, back to where we see this going. We wrote, I think two years ago, we published on our research team the power law of models, and we were the first ones that came out and said there's not going to be one model to rule the world. Because as obvious as the day that you're going to have specialty models, you're going to have the power law of models and that's going to be the reality.

Jason Warner

>> Yeah. We know this, the entire industry knows this. It's hard to wrap our head around for software for one particular reason, but we already know this. We know that there's going to be genomics or pharmacological models, robotics models, full self-driving models. They're not going to be run by general purpose models. And we know general purpose models serve a purpose as well. I think what has confused people is that the moment that we added source code to general purpose model data sets, the general purpose models got smarter. And so we imply that that was how we were going to advance the general purpose models, and they did, but we flipped it and we said, "Well, what we're actually saying is the source code that's important here and that's what we're focused on."

>> Tell about the relationship with AWS because what I find is I think this is an untapped, groundbreaking area because software is going to be the key asset and we don't yet know what's coming for software. I think AI's kind of leveled up from the '80s when I learned it and you couldn't really do anything with it back then. And then transformers came out, brought everything up to leveled up, and then the new algorithms coming out. We still don't know what's going to come. So I think we're going to have a tsunami of innovation around stuff we've never seen before.

Jason Warner

>> Yeah. On the AWS front, I'll get to that one in a minute, but I think this kind of worldview that we have and belief that we have is important to state here, which is if I'm looking over a 10-year time horizon, I think of three things that are effectively going to matter for the digital world. There's going to be access to electricity, access to chips that make and run these new emergent AI intelligence layers. And so you think about those three key assets that actually matter and the rest of them could be rounding errors. There's a lot of stuff that just doesn't matter anymore, but all of a sudden if you understand where people are laser focused, you understand kind of the decisions that are being made. And now we're at re:Invent and we see what AWS is announcing with Trainium2 and Trainium3 and what they're doing with energy and all the other hyperscalers, what they're doing. This is where we're focused. We're focused on understanding the partner landscape and then what we're doing through the intelligence layer. So specifically on AWS. So we've announced a broad strategic partnership with them. So poolside is available in a first-party offering on AWS. So native just as you could access or buy S3 or EC2, or anything else like that, you can buy poolside. That's great from a customer perspective because they're familiar with the terms and services and the MSAs and all of the different legal arrangements that they already have with AWS. So they don't have to get familiar with a startup. And they can also commit, spend, retire their AWS spend by buying poolside. As we already said, what poolside is, this is great for very large enterprises.

>> By the way, I love your worldviews and one thing I want to ask since we're on the worldview, you got a great perspective and unique is as you get these specialty models, I think you're going to see these models that are going to be so good and they are encapsulating all that knowledge. They got to work together like a brain. And so the fusion of data comes in. So now you have a whole different data flow model. I mean, so you have to have all kinds of new constructs like what is memory? What is this? What is that? So as models, let's just say we build the biggest cube model around narratives and technologies, for example, we could maybe do that, right? Okay, now great. Now you ping me for something, I got to give you some answer. I'm the part of the brain. Maybe it's going to be in this. How do you see that evolving?

Jason Warner

>> This is a topic even between my co-founder and I and just all of us in the industry that are talking about this, but this could get into dangerous territory because it could look a little bit like a nut job when you start talking about this, but I do think that what we're going to end up with is you're going to end up with these intelligence layers. I'm going to use that word for now, and they're going to start end up talking to each other. And so what you're going to see is you're going to see these protocol changes. You're going to see these asking each other questions, "Hey, my customer asked me to do this. Can you help me understand if this is the right thing to go do? Hey, you're a specialty pharmacological model. Help me understand what this request really is and process it." Of course, I think you're going to end up with that because no matter what we think about OpenAI and Anthropic, they're not going to know all of the different things, same as we're not going to know all of the different things. So it makes sense that it's going to happen. Now, that's how I view the world evolving. Then of course it's going to happen in the enterprise because you can see what's going to happen is in the enterprise is they're going to start choosing vendors and they're going to say, "Hey, I want you to be my software layer. I want you to be my general purpose layer. I want you to be my robotics layer, but I want you all to work together because I don't want this weird handoff that's going to happen." Now, we're not there yet, but of course that's a natural evolution because the customers are going to dictate that we do that.

>> Yeah. And to your point about the old model of like, "You're my vendors, now go play together and I'll call database and I'll connect through an API." The latencies are so long on that, you got to be very tightly coupled but cohesive in its own way and it's going to be super fast. It can't be-

Jason Warner

>> I mean, it's going to get interesting really quickly here. The fun part here is that think about the type of people who have raised enough money, and this is another issue in the space, but we're not having infinite number of people who are going to be building these foundation models. The tables are largely probably set already with the people who are going to be the players in this space. So it's going to be under 20 amount of people in the world who are going to start to have these conversations. So we're going to have a lot of frenemies talking to each other.

>> I think what you're doing with Amazon, I want to get the specific around the trainings because I think as you get down to the level of the hardware, you get those advances. If you can come in and be that nexus or that point in the enterprise where you can be a primitive to be that mission control if you will, or I don't know what word to use, but if you're going to be the primitive to say, "Hey, let's do our software together." And kind of orchestrate and integrate and make all that kind of run together for their reasons. Every enterprise has their own little way to do business.

Jason Warner

>> Yes, they do.

>> So they got to have rough software to run that.

Jason Warner

>> Yeah. This is why it's so important that you take state-of-the-art language models and state-of-the AI, but also marry it with their context. Because imagine you're a large enterprise and you spent the last 20 years figuring out your environment and you say, "We have all of these different policies." If you think about what the software life cycle really is, it does two things. It's Gates saying we've approved of these certain steps that need to happen for code to be released to production or it's policy management adhering to their protocols and standards. And if you can't marry those things with the AI, it's almost useless.

>> I was talking to a friend the other day and said, "I'm so old. We had one watch when we grew up and then Swatches came out and you had multiple versions of a watch and now people can have disposable watches. So the conversation was we might be in a world where applications can be built for a specific use case and then thrown away in 10 minutes and we're going to be living in a world where, "Oh, I'm going to be at the sportsbook here in Vegas. Write me an app for my picks." Swami gave an example of finding free food on stage today. So I think disposable apps maybe. I use that word for a lack of -

Jason Warner

>> Yeah, I call them ephemeral.

>> Yeah, ephemeral apps.

Jason Warner

>> So I think that's a world that we'll exist. So I think that just like we've seen already is that source code itself has served a purpose and it's around because of reasons, but it doesn't necessarily need to be for a certain set of things. And I think we already know this to be true because if you look at the way the networking works at UDP and TCP, we already are okay with packets being dropped along the way as long as the message gets exchanged and that the outcome is achieved. So software doesn't necessarily need to be that different.

>> So Jason, imagine like, "Hey Siri, find me directions to the local place to go watch the football game here, dining directions, write me an app to do... " Like done. I see that. This isn't happening now.

Jason Warner

>> These are inevitabilities at this point. This is the train that we're on. It's just a matter of time until all the pieces have to come into play.

>> Okay, so take us through your vision of enterprise with Amazon because coming back full circle to the practical, say I'm at JPMorgan Chase, they're an infrastructure. They're a huge scale. Then you've got the mid-range enterprises, I call them middle of America enterprise. They got rack and stack, they got data centers, some cloud, lift and shift. They want to transform their business and saying, "Hey, I never really invested in a lot of development. I've outsourced everything. I got an IT department. I'm in the cloud. Things are rocking and rolling. I want to just go next level now." What's the pass?

Jason Warner

>> This kind of goes again, back to worldviews, but also what I think is important. So we're building poolside from the beginning to be used in the narrowly environments, the most highly regulated or air-gapped environments. With one very particular point in that I don't want customers to have to trust me with their data, but I must give them state-of-the-art AI that can be married with their data because it's so vital to their own business. This is where I think that we as tool builders and vendors have to do. We have to understand their concerns. And so because of that, our vision is going to be that we have to work with the biggest and the best in the world to do that. But like you said, the JP Morgan's of the world, there's no world in which they should hand over their data to get access to AI. That's a devil's trade you don't want to do and they can't do it. So to serve them in a way that gives them access to the best tools, you have to meet them where they are. Again, I go to the air gap because the government, there's no world in which if I think I'm building the world's best AI for software development, I don't want the people to protect me while I'm sleeping to now have access to this and so I have to understand what those concerns are. And so far, people who are AI vendors have not built for them.

>> Okay. Talk about the Trainium stuff. You've been working for a while. I know you had the pre access. You and Anthropic have deep, deep relationships with Amazon to get the early look because that's where the innovation. What are you seeing there? What's the report? What would you share with folks? Any commentary, observations?

Jason Warner

>> Yeah. So we've been training on Nvidia for quite some time, H100 and H200s, as well as Trainium. What we saw with Trainium is particularly 1 to 2 was we saw an immediate 40% uptick in price performance and 50% increase in performance out of the box without any sort of optimizations on the Trainium2 chip. It was amazing. And then our team works very closely with the Annapurna team on the kernels.

>> Got it. And then as you guys look at these complex engineering assignments, I think this is the real secret to poolside is that when you start getting into reasoning and reinforced learning, the things you were mentioning earlier, then that shit gets real. I mean, that's when it gets real.

Jason Warner

>> Well and a couple of things that we don't talk about too widely, but we have two models at the moment. One we call malibu, which is our reasoning model, and one we call point, which is our code completion model. Well, our code completion model is a non-transformers based model. It's one of the first in the world that's moved on from that architecture and it gives us an amazing speed improvements. But what that really means is that we can take advantage of inference like nobody's business and our next generation model is moving to that architecture as well. So again, inference is a really big deal.

>> Why is you getting those advantages with the non-transformer model? What was the overhead involved?

Jason Warner

>> Well, because transformers has quadratic attention and this has linear attention. And so because of the way you're going to run it, it just speeds everything up. And that matters to us because the way that we're going to train, we have a different training architecture in the future which uses some inference heavy.

>> That's your IP you going?

Jason Warner

>> Yes.

>> Okay, got it. Tell about this code execution feedback loop. How does that work? What does that mean for folks watching in terms of is that going to help them get better, faster and the role and talk about how the synthetic data accelerates that?

Jason Warner

>> This is a have our cake and eat it too scenario, and I'll lay it out what that means. But effectively, this is the reason why we can go to a customer and say, "We don't need to get access to your data." Because we can generate so much synthetic code data and software data that we can work our way out of the data hole. The dirty little secret of the AI industry is that all frontier AI companies have access to the same data. We all have access to the corpus of the internet and the open source code data out there. And so we're all in the hunt. We're all data hungry. And so we're all compute hungry and data hungry.

>> Elbowing each other for the best data. It's fighting for data.

Jason Warner

>> We are fighting for data. But we can generate so much of this now with this reinforcement learning via code execution feedback as we call it, that we don't need to get access to customer's data because we can generate our way out of this. So it allows us a different go-to-market motion, it allows us to have a different strategy at play, but in reality, this is our way that we are going to advance the techniques and software. So in practice, what does this look like? Right now, we're running about 250,000 real world code bases in our runtime environment, and we're generating tens of millions of tasks and synthetic tasks at the moment based upon those things. They go right back into our feedback training loop.

>> That's where you guys get that kind of QA version going. That's where you get the quality.

Jason Warner

>> That's where we get, we see all the advancements, all the feature advancements and all the different advancements. Again, we've stand on the shoulders of giants. This is patterned after AlphaGo. We've known this for some time from DeepMind. This is how DeepMind took AlphaGo and made it a superhuman go player. So if you think about what software is, software is as close as you can get to a deterministic system that's not deterministic if you will, and we can run it on a computer so we can fully almost perfectly simulate it. And so it allows us a lot of liberties and we're taking advantage of all those liberties.

>> It definitely does. Good liberties take advantage and this is also a renaissance for computer science too. If you think about, I mean, how many people do you know that have been waiting for this moment in the industry because this is hardcore computer science.

Jason Warner

>> I met my co-founder at GitHub in 2017. When I joined GitHub in 2017, this was two years pre-acquisition, I tried to buy his company to do this at GitHub and it was too early. This was pre-transformers paper, this was free attention papers, free transformers, and here we are.

>> do that because being early sometimes can be death to the death sentence.

Jason Warner

>> Yeah, and I think we're hitting the right wave at the right time. Obviously, in the office of the CTO, we went on and built GitHub Copilot post-acquisition under a very different strategy in conjunction with OpenAI. But now is the right time. This is when you can advance the entire state of the art.

>> If you had to summarize for the folks watching out there, because this comes up a lot on theCUBE. People kind of coming into the loop on what GenAI is. There are certain proof points, call it social proof, call it real proof. What are some of the proof points that separate the pretenders from the players? There's a lot of people trying to squint through, "Oh, look at that RAG applications. It got vector embeds. Oh, okay, come on, that's good search. Okay, it's data." You're getting into real computer science, meal moving aspect. How does someone differentiate between players and pretenders? I shouldn't say it that way, maybe just make-

Jason Warner

>> But I think I understand what you're asking. I think let's start with simple distinctions. Are you an AI producer or an AI consumer? And it gets confusing right now. We see many, many, many companies who are being funded by VCs who are technically AI consumers being funded like their AI producers. It's a simple distinction. Are you building basically foundational language models, AI producer, or are you consuming someone else's foundational AI models, AI consumer?

>> Or what about if you have a lot of data?

Jason Warner

>> So if you have a lot data-

>> If you're one of those people that has the data farm that people want.

Jason Warner

>> So then you're an AI supplier in that case, partnering with folks. But it's interesting because there's a lot of organizations who are very large organizations, but they're not technical enough at the moment. I'm not trying to begrudge anybody, but there's many people in the industry who won't be capable of building very large foundational models. And so they've got to find unique partnerships here. This is something that we do. We work with people who have developer ecosystems and we want to partner by saying, you can take poolside and we can further train that as you're entering-

>> You're the provider. They're a consumer.

Jason Warner

>> Yeah. So a good example-

>> That's a bad thing. You're enabling value.

Jason Warner

>> No, and think about some of these ecosystems like look at Salesforce. Salesforce has a hundred million developers as they call it. And if we were to take poolside, combine it with all of their data, all of their code assets, all of their documentation and whatnot, and Salesforce could serve their ecosystem with state-of-the-art AI in a way that they couldn't before.

>> Jason, I could go an hour with you. Thanks so much for your time. Definitely let's follow up. Love what you guys are doing. Again, there's a lot of world views in this because the world is changing super fast. There's not a lot of talent actually out there either to make it happen, but people are jumping in. I mean, if I'm in college right now, I would be on this.

Jason Warner

>> What I wouldn't give to be 20 again.

>> I mean.

Jason Warner

>> I mean right now.

>> Like me on theCUBE. I see that all the time. I mean, think about the appetite. You could just go crazy. It's just a really amazing time. I've never seen anything like it.

Jason Warner

>> Me either.

>> Thanks so much. Appreciate it.

Jason Warner

>> Thanks for having me. Appreciate it.

>> Okay, getting all the action here. poolside on theCUBE here. Again, the world's changing how software's going to be written and the value, are you a consumer? Are you a producer? Again, AI is changing everything and we're going to continue to cover it up and down the stack. I'm John Furrier. Thanks for watching.